Multicast reduction assisted by network devices

Abstract

The present disclosure relates to multicast reduction assisted by network devices. Systems and techniques for performing multicast reduction operations. In at least one embodiment, a network device receives first network data associated with a multicast operation to be collectively performed by at least a plurality of endpoints. The network device reserves resources to process second network data to be received from the endpoints and sends the first network data to a plurality of additional network devices. The network device receives the second network data and processes the second network data using the reserved resources.

Description

Technical Field

[0001] At least one embodiment involves performing a multicast reduction operation assisted by a network device. Background Technology

[0002] Multicast reduction can be used to distribute computational load among several endpoint devices and combine results from the endpoint devices into a complete result. This can improve the efficiency of multicast reduction operations. Attached Figure Description

[0003] Figure 1 A system for performing multicast reduction operations according to at least one embodiment is shown;

[0004] Figure 2 A network device for performing various aspects of multicast reduction operation according to at least one embodiment is shown;

[0005] Figure 3 An endpoint and a graphics processing unit for performing various aspects of a multicast reduction operation according to at least one embodiment are shown;

[0006] Figure 4 The process for performing multicast reduction operations according to at least one embodiment is illustrated;

[0007] Figure 5 The downlink transmission of multicast reduced data conforming to a topology according to at least one embodiment is illustrated;

[0008] Figure 6 The uplink transmission of multicast reduced data conforming to a topology according to at least one embodiment is illustrated;

[0009] Figure 7 The process for performing a multicast reduction operation according to at least one embodiment is illustrated;

[0010] Figure 8 The process for performing multicast reduction operations for a network device according to at least one embodiment is illustrated;

[0011] Figure 9 A distributed system according to at least one embodiment is shown;

[0012] Figure 10 An exemplary data center according to at least one embodiment is shown;

[0013] Figure 11 A client-server network according to at least one embodiment is shown;

[0014] Figure 12An example of a computer network according to at least one embodiment is shown;

[0015] Figure 13A A networked computer system according to at least one embodiment is shown;

[0016] Figure 13B A networked computer system according to at least one embodiment is shown;

[0017] Figure 13C A networked computer system according to at least one embodiment is shown;

[0018] Figure 14 The illustration shows one or more components of a system environment according to at least one embodiment, in which the service can be provided as a third-party network service;

[0019] Figure 15 A cloud computing environment according to at least one embodiment is shown;

[0020] Figure 16 This illustrates a set of functional abstraction layers provided by a cloud computing environment according to at least one embodiment;

[0021] Figure 17 A supercomputer at the chip level is illustrated according to at least one embodiment;

[0022] Figure 18 A supercomputer at the rack module level is shown according to at least one embodiment;

[0023] Figure 19 A supercomputer at the rack level is shown according to at least one embodiment;

[0024] Figure 20 A supercomputer at the entire system level is illustrated according to at least one embodiment;

[0025] Figure 21A The inference and / or training logic according to at least one embodiment is illustrated;

[0026] Figure 21B The inference and / or training logic according to at least one embodiment is illustrated;

[0027] Figure 22 The training and deployment of a neural network according to at least one embodiment are illustrated;

[0028] Figure 23 The architecture of a network system according to at least one embodiment is shown;

[0029] Figure 24 The architecture of a network system according to at least one embodiment is shown;

[0030] Figure 25 A control plane protocol stack according to at least one embodiment is shown;

[0031] Figure 26 A user plane protocol stack according to at least one embodiment is shown;

[0032] Figure 27 The components of a core network according to at least one embodiment are shown;

[0033] Figure 28 Components of a system supporting Network Function Virtualization (NFV) according to at least one embodiment are shown;

[0034] Figure 29 A processing system according to at least one embodiment is shown;

[0035] Figure 30 A computer system according to at least one embodiment is shown;

[0036] Figure 31 A system according to at least one embodiment is shown;

[0037] Figure 32 An exemplary integrated circuit according to at least one embodiment is shown;

[0038] Figure 33 A computing system according to at least one embodiment is shown;

[0039] Figure 34 An APU according to at least one embodiment is shown;

[0040] Figure 35 A CPU according to at least one embodiment is shown;

[0041] Figure 36 An exemplary accelerator integration slice according to at least one embodiment is shown;

[0042] Figures 37A-37B An exemplary graphics processor according to at least one embodiment is shown;

[0043] Figure 38A A graphics core according to at least one embodiment is shown;

[0044] Figure 38B A GPGPU according to at least one embodiment is shown;

[0045] Figure 39A A parallel processor according to at least one embodiment is shown;

[0046] Figure 39B A processing cluster according to at least one embodiment is shown;

[0047] Figure 39C A graphics multiprocessor according to at least one embodiment is shown;

[0048] Figure 40 A software stack of a programming platform according to at least one embodiment is shown;

[0049] Figure 41 The illustration shows an embodiment according to at least one of the embodiments. Figure 40 The CUDA implementation of the software stack;

[0050] Figure 42 The illustration shows an embodiment according to at least one of the embodiments. Figure 40 The ROCm implementation of the software stack;

[0051] Figure 43 The illustration shows an embodiment according to at least one of the embodiments. Figure 40 The OpenCL implementation of the software stack;

[0052] Figure 44 Software supported by a programming platform according to at least one embodiment is shown; and

[0053] Figure 45 A method for using at least one embodiment is shown. Figures 40-43 Compiled code executed on the programming platform. Detailed Implementation

[0054] Numerous specific details have been set forth in the preceding and following description to provide a more thorough understanding of at least one embodiment. However, it will be apparent to those skilled in the art that the inventive concept can be practiced without one or more of these specific details.

[0055] Figure 1 A system for performing a multicast reduction operation is illustrated according to at least one embodiment. In at least one embodiment, the multicast reduction operation is a computational task performed collaboratively by a distributed computing system. For example, the distributed computing system may include an endpoint 102 that initiates the multicast reduction operation, which will be collaboratively performed by endpoints 104, each of which performs some part of the computational task. Network devices 106a-f participate in the execution of the multicast reduction operation by distributing data to endpoints 104 and coalescing data returned from endpoints 104.

[0056] In at least one embodiment, the multicast reduction operation is performed in stages. In the first stage, which may be referred to as the multicast stage, endpoint 102 initiates the multicast reduction operation by sending data through the structure of network devices 106a-f to endpoint 104. In the second stage, endpoint 104 then performs the corresponding part of the computation task. In the third stage, which may be referred to as the reduction stage, data from endpoint 104 flows back to the initiating endpoint 102 through the structure. In this stage, network devices may combine or reduce the data they receive. For example, network device 106b may receive data transmitted from each of network devices 106d-f and combine the data to transmit to network device 106a.

[0057] In at least one embodiment, the flow of network data through the structure of network devices 106a-f is performed according to a multicast reduction tree established before initiating a multicast reduction operation. Furthermore, the multicast reduction operation (sometimes referred to as a transaction) acquires resources as it flows through the structure. In at least one embodiment, hop-by-hop routing and reservation schemes are used, and include support for redirecting transaction responses back to resources reserved on behalf of the initiating transaction. In at least one embodiment, the hop-by-hop routing supports autonomous cleanup and recovery of orphaned resources and prevents deadlocks in looped topologies. Embodiments may also include support for minimizing segmented storage that may result from dynamically allocated resources, supporting repeatable reduction behavior, and minimizing unique routing data that may stress cross-routing channels. In at least one embodiment, these features are supported by placing hop-by-hop routing information in the network header to index responses back to the network device corresponding to explicitly reserved resources. This routing information can be remapped in the transaction request header at each hop as data flows to endpoint 104, and previous routing information can be restored at each hop as data returns through the structure. In at least one embodiment, reserved resources are tracked, and timeouts are used to initiate automatic cleanup if a transaction is lost in the system.

[0058] In at least one embodiment, the endpoint (e.g., any of the endpoints 102-104 described herein) is a computing device including one or more processors and memory for storing instructions executed by the processors. The endpoint may further include one or more network interfaces, and hardware and / or software components for initiating multicast reduction operations. In at least one embodiment, the endpoint includes a graphics processing unit (“GPU”) or a parallel processing unit (“PPU”) that includes aspects supporting the initiation or execution of the multicast reduction operations described herein. Note that although embodiments of the endpoint may be described herein as including a GPU, similar components (e.g., a PPU) may be substituted without impairing the associated functionality.

[0059] In at least one embodiment, endpoint 102 is the endpoint that initiates a multicast reduction operation. As described above, the data associated with initiating the multicast reduction operation originates from endpoint 102, flows through the network device structure, and is received by endpoint 104. In at least one embodiment, endpoint 104 is the endpoint that receives certain portions of the data, uses the data to perform the corresponding multicast reduction operation, and returns the data to the initiating endpoint 102 through the network structure.

[0060] In at least one embodiment, the network architecture includes multiple network devices, such as the depicted network devices 106a-f. In at least one embodiment, the network devices include any one of a switch, hub, router, or other device for receiving and transmitting data. These network devices may include one or more network interfaces, one or more processors, and memory for storing instructions executed by the processors.

[0061] In at least one embodiment, the multicast reduction tree includes information defining the topology of network devices and endpoints that will be used to process the multicast reduction operation. The topology may include information indicating paths that route data through the network structure to endpoint 104, and paths that route data from endpoint 104 to the initiating endpoint 102 of the multicast reduction operation.

[0062] Figure 2 A network device is illustrated according to at least one embodiment for performing various aspects of multicast reduction operations. In example 200, network device 206 corresponds to Figure 1 Any of the network devices 106a-f depicted herein. In at least one embodiment, network device 206 includes a processor 212 and a memory 214 for storing instructions that, when executed by processor 212, cause network device 206 to perform the operations described herein associated with performing multicast reduction operations.

[0063] Network device 206 further includes network interfaces 210 and 220 for receiving and transmitting data from the network. Incoming data may be stored in input buffer 216, while outgoing data is stored in output buffer 218. In at least one embodiment, input buffer 216 and / or output buffer 218 correspond to a portion of memory 214, but in other embodiments, separate memory may be included.

[0064] In at least one embodiment, network device 206 receives data associated with multicast data transmission. This data may include one or more headers with routing information, and data to be transmitted by one or more endpoints (such as...). Figure 1The data used by endpoint 104 (described in the diagram). During the multicast phase, as data flows from the originating endpoint to the destination endpoint, network device 206 can transmit output data to N destinations (endpoints or additional network devices) for a given set of input data, where N may depend on the specific topology of the multicast reduction. In some embodiments, data is replicated to each output port of network device 206, or to each output port of network device 206 configured according to a multicast reduction topology. During the reduction phase, network device 206 can receive return data from N sources corresponding to the previous N destinations. This data can then be combined, or reduced, and returned to the previous network device or host.

[0065] In at least one embodiment, network device 206 reserves resources to perform reduction as data flows outward during the multicast phase. For example, in at least one embodiment, network device 206 receives data for multicast reduction operations, reserves memory (such as space in input buffer 216 or memory 214) for storing and processing returned data, and forwards the received data to N destinations. Then, during the reduction phase, network device 206 receives data from multicast reduction endpoints (such as from...) Figure 1 The data returned by one of the endpoints 104 depicted is used to perform a reduction using reserved memory resources. The reserved memory resources are then released. Implementations can reserve, utilize, and release other resource types in a similar manner.

[0066] In at least one embodiment, network device 206 manages header information in incoming and outgoing data to facilitate the processing of multicast reduction operations. In at least one embodiment, this includes 1) reserving header information for incoming downlink network packets, 2) storing header information in outgoing downlink network packets, 3) using the header information in incoming uplink network packets to perform reduction using the reserved resources and release the reserved resources, and 4) using the reserved header information to send the reduced network data uplink. In at least one embodiment, because the network devices in the architecture follow the above operations, network device 206 ensures that uplink network packets in the multicast reduction operation will include header information for managing the resources reserved for the reduction operation.

[0067] In at least one embodiment, network device 206 stores information including one or more of the following in the outgoing downlink header: information describing the reduced topology, pointers to reserved resources, index information, etc.

[0068] In at least one embodiment, network device 206 maintains a reduction buffer. This reduction buffer may include memory reserved for processing multicast reduction data as data is returned uplink. The buffer may include space for storing data returned uplink during reduction, and in some embodiments, this space may be further used for combining data.

[0069] In at least one embodiment, network device 206 maintains a table or other data structure that includes information related to the uplink data returned during reduction. Header information stored by network device 206 and included in the uplink data returned to network device 206 may include information indexed to that table or other data structure.

[0070] Figure 3 Example 300 of an endpoint and a graphics processing unit for performing various aspects of a multicast reduction operation according to at least one embodiment is shown. This may correspond to... Figure 1 Endpoint 304 of any of the endpoints 102 and 104 depicted herein may include processor 314 and memory 312 for storing instructions that, when executed by processor 314, cause the endpoint to perform operations as described herein associated with performing a multicast reduction operation.

[0071] In at least one embodiment, endpoint 304 includes network interface 310 for transmitting or receiving network data. This network data may be associated with the initiation or processing of a multicast reduction operation. For example, in at least one embodiment, endpoint 304 may correspond to... Figure 1 Endpoint 102, as depicted, initiates a multicast reduction operation by transmitting a request to perform the operation, along with any associated data, to one or more other endpoints. In another example, endpoint 304 participates in processing the initiated multicast reduction operation by receiving and processing uplink and downlink data associated with performing the operation, similar to... Figure 1 Endpoint 104 is described in relation to this.

[0072] In at least one embodiment, endpoint 304 includes a graphics processing unit 308. In at least one embodiment, the graphics processing unit includes a host interface 320, memory 322, and processor 324. The host interface includes circuitry for transferring data between components of the graphics processing unit 308 and endpoint 304, such as processor 314 or memory 312. The graphics processing unit 308 may include a plurality of processors 324 and includes circuitry designed to optimize the parallel execution of tasks, which may potentially include, but are not limited to, graphics rendering, physics calculations, and machine learning. The graphics processing unit 308 may be further configured to optimize data parallelism, data transfer, and optimized execution of specialized instruction sets implemented by the processors 324.

[0073] In at least one embodiment, endpoint 304 includes software (such as a device driver) to interface with graphics processing unit 308. The device driver, or other software and / or circuitry of endpoint 304 and / or graphics processing unit 308, is adapted to facilitate processing of multicast reduction operations.

[0074] In at least one embodiment, a device driver or other software and / or circuitry is adapted to facilitate the initiation of a multicast reduction operation. For example, initiating a device driver in endpoint 102 may cause endpoint 102 to initiate a multicast reduction operation. In at least one embodiment, the multicast reduction operation is initiated by a driver when data is written to a mapped region of memory, such as the mapped region of memory 312 of endpoint 304, or the mapped region of memory 322 of graphics processing unit 308. Here, the mapped region of memory may refer to a physical or virtual memory region specified in relation to a potential or existing multicast reduction operation. For example, in at least one embodiment, a range of virtual memory addresses may be specified as the input to the multicast reduction operation, so that when data is written to this region, via a method similar to... Figure 1 The network device structure depicted is automatically transmitted to other endpoints by a driver. Similarly, device drivers or other software and / or circuitry can be adapted to facilitate the receiving endpoint's processing of the appropriate portion of a multicast reduction operation. For example, endpoint 304 can receive request data associated with a multicast reduction operation and store this data in a mapped area of ​​memory 322 of graphics processing unit 308. Graphics processing unit 308 can then process the request data related to the multicast reduction operation, just as it processes any other operation. The result of this operation can then be written to an area of ​​memory 322 mapped as output, and the resulting data is returned to the originating endpoint via a structure.

[0075] Figure 4 A process for performing various aspects of a multicast reduction operation according to at least one embodiment is illustrated. Although example process 400 is depicted as a series of steps or operations, it is understood that embodiments of process 400 may include steps or operations that are modified, reordered, or omitted unless explicitly stated or logically required, such as when the output of one step or operation is used as the input of another step or operation.

[0076] Embodiments of example process 400 can be executed by any of a variety of devices, including, but not limited to, network devices. For example, in at least one embodiment, example process 400 is performed by a device similar to... Figure 1 The network equipment 106a-f or described in the document Figure 2The following description refers to steps or operations performed by such a network device, but in some embodiments, they may also be implemented in other suitable devices.

[0077] At position 402, the network device receives network data associated with a multicast operation. This multicast operation is performed jointly by multiple endpoints associated with the multicast operation, such as... Figure 1 Endpoint 104 is depicted in the image. Similar to... Figure 1 The network device depicted, performing example process 400, can be part of a network fabric where devices receive and transmit data according to a topology. In some embodiments, this topology may be associated with a specific multicast operation or a category of operations. Therefore, network data received by the network device may include data used by one or more downlink endpoints in a multicast reduction operation. In the downlink phase of multicast reduction, data flows from the source endpoint through the fabric to several endpoints.

[0078] At position 404, the network device reserves resources to process data received from the endpoint associated with the multicast operation. This is done in the expectation of receiving return data in the subsequent uplink phase of the multicast reduction operation. In at least one embodiment, reserving resources while data flows downlink facilitates processing when data returns uplink.

[0079] At point 406, the network device transmits the received data to multiple network devices. In at least one embodiment, the network device transmits some or all of the received data to each of a plurality of downlink devices, which may include other network devices or endpoints. For example, the network device may forward its received network data to each of N other network devices or endpoints, depending on an applicable topology. In at least one embodiment, the network device accomplishes this step by using multicast network transmission to send copies of the data to each of N destinations.

[0080] In at least one embodiment, the network data continues to flow through the structure in a similar manner until it is received by the endpoints. These endpoints can then perform multicast reduction operations on their respective portions and send return data uplink through the structure.

[0081] At 408, the network device receives the return data. In at least one embodiment, if the network device sends requested data to N downstream destinations, it can expect (unless timeout or other error) to receive return data from each of the N destinations. In at least one embodiment, these responses are cached in the memory resources reserved at step or operation 404.

[0082] At 410, the network device uses reserved resources to process data received from the endpoint. For example, the network device may use reserved storage space, or other reserved resources, to temporarily store the received return data and reduce N responses to a single unitary result. In another aspect of this example, consider the case where N = 2, and the two responses correspond to separate data transmissions including {“ABC”} and {“DEF”}, respectively. In at least one embodiment, reducing these responses may then yield an array {“ABC” + “DEF”}.

[0083] At 412, the network device transmits processed data. In at least one embodiment, depending on the applicable topology, data is uplinked to one or more devices during the downlink phase of the multicast reduction operation, and data is received from those one or more devices.

[0084] Figure 5 Downlink transmission of multicast reduced data conforming to a topology is illustrated according to at least one embodiment. In at least one embodiment, the data transmitted downlink includes information for one or more of command, control, and routing.

[0085] In at least one embodiment, topology refers to the routes or paths that data will take in the downlink and uplink directions during a multicast reduction operation. This may include information indicating which network devices and endpoints are used in the multicast reduction operation, and how the data is routed through these network devices and endpoints.

[0086] For example, Figure 5 The topology in Example 500 may include information indicating the multicast reduction operations to be performed using network devices 506b, 506d-e and endpoints 502, 504a-c.

[0087] Figure 5 Example 500 further depicts the structure of how multicast reduction data might flow through network devices 506a-f and endpoints 502, 504. In Example 500, a multicast reduction operation is initiated by endpoint 502, which sends multicast reduction data downlink to network device 506b. In at least one embodiment, this is accomplished based on the topology associated with the multicast reduction operation. This association can be based on any of a variety of factors, potentially including but not limited to global network configuration, the configuration of initiating endpoint 502, or the topology independently assigned to each multicast reduction operation.

[0088] Then, network device 506b can receive the data and, according to... Figure 4 Example procedure 400 and the applicable topology forward data to network devices 506d-e. Note that for illustrative purposes, it is assumed that some network devices 506a,c,f are excluded from the applicable topology.

[0089] Then network device 506d can receive data from 506b according to the applicable topology and send the data to endpoint 504a. Similarly, network device 506e can receive data from network device 506b according to the topology and forward the data to endpoints 504b and 504c. Each of these network devices 506d-e is based on the topology... Figure 4 Examples of related process 400 are used to process the received data.

[0090] Before returning the result of the processing upstream, each of the endpoints 504a-c that receives request data related to the multicast reduction operation can then continue to perform any applicable processing in response to the received data. The nature of the resulting data will vary depending on the nature of the multicast reduction operation. As a simple example, each of the endpoints 504a-c can receive a request to sum an array of numbers and return data including the sum of those numbers. Thus, endpoint 504a can sum [1,2] and return upstream [3], endpoint 504b can sum [2,2] and return upstream [4], and endpoint 504c can sum [2,3] and return upstream [5].

[0091] Figure 6 The uplink transmission of multicast reduced data conforming to a topology according to at least one embodiment is illustrated. (Continued) Figure 5 Example 500 shows that multicast reduction operations can be associated with the topology of the structure that indicates how multicast reduced data flows through the structure in the downlink and uplink directions. Figure 6 In Example 600, it is assumed that data has flowed down from source endpoint 602 to endpoints 604a-c, and endpoints 604a-c have each processed the corresponding part of their multicast reduction operation and are returning data uplink.

[0092] In at least one embodiment, data returned from endpoints 604a-c follows the reverse of the path used to send data to the respective endpoints. For example, if data is downlinked from endpoint 602 to endpoint 604a via network devices 606b and 606d, then data can uplink from endpoint 604a to endpoint 602 via network devices 606d and 606b. Similarly, as Figure 6 As shown, data from endpoint 604b can be returned uplink on the path that includes network devices 606e and 606b, and the same applies to data from endpoint 604c.

[0093] Figure 7A process for performing various aspects of a multicast reduction operation according to at least one embodiment is illustrated. Although example process 700 is described as a series of steps or operations, it will be understood that embodiments of process 700 may include steps or operations that are modified, reordered, or omitted unless explicitly stated or logically required, such as when the output of one step or operation is used as the input of another.

[0094] Embodiments of example process 700 can be executed by any of a variety of devices, including, but not limited to, network devices. For example, in at least one embodiment, example process 700 is performed by a device similar to... Figure 1 The network equipment 106a-f or described in the document Figure 2 The following description refers to any implementation of the network device 206 depicted. The steps or operations described below are performed by such a network device, but in some embodiments, they may also be implemented in other suitable devices.

[0095] At point 702, the network device receives network data for multicast reduction operations. This data is sent from the uplink device as part of the downlink phase of the multicast reduction operation.

[0096] At 704, the network device stores routing information obtained from the header of the incoming network data. This routing information includes data indicating the source of the incoming network data and allowing data to be moved uplink to be returned to the uplink device.

[0097] At 706, the network device reserves resources for reduction. In at least one embodiment, the reserved resources include one or more reduction buffers for storing data as it returns from the downlink device. In at least one embodiment, a buffer is assigned on a per-port basis for each multicast reduction operation. For example, if the network device includes six ports, and each of the six ports is used to multicast data to the downlink device, six reduction buffers may be reserved, with one reduction buffer reserved for each of the six ports.

[0098] At point 708, the network device inserts routing information into the header of the outgoing network data. In at least one embodiment, the routing information includes the network address of the network device, which can then be used by the downlink device to route returning data uplink to the network device.

[0099] At 710, the network device sends network data to one or more downlink devices. In at least one embodiment, this includes routing information inserted at 708. In at least one embodiment, the downlink network device to which the data is sent is identified, at least in part, based on information obtained from the header of the incoming data. This information potentially includes, but is not limited to: information identifying the applicable topology, information identifying the next downlink hop in the structure, the addresses of one or more endpoints to which the data should be sent, etc.

[0100] At 712, the network device receives network data returned in association with the multicast reduction operation. In at least one embodiment, this data is returned from the downlink device to the network device using previously inserted routing information.

[0101] At point 714, the network device uses reserved resources to reduce the returned network data.

[0102] At point 716, the network device uses stored routing information to send reduced network data to one or more uplink devices.

[0103] Figure 8 A process is illustrated for performing various aspects of a multicast reduction operation by a network device according to at least one embodiment. Although example process 800 is depicted as a series of steps or operations, it is understood that embodiments of process 800 may include steps or operations that are modified, reordered, or omitted unless explicitly stated or logically required, such as when the output of one step or operation is used as the input of another step or operation.

[0104] Embodiments of example process 800 can be executed by any of a variety of devices, including, but not limited to, network devices. For example, in at least one embodiment, example process 800 is performed by a device similar to... Figure 1 The network equipment 106a-f or described in the document Figure 2 The following description refers to any implementation of the network device 206 depicted. The steps or operations performed by such a network device are described below, but in some embodiments, they may also be implemented in other suitable devices.

[0105] At 802, the network device receives network data for multicast reduction operations to be performed by multiple endpoints. This network data may be referred to as first network data to distinguish it from data returned upstream to the network device. The network device in example procedure 800 may refer to any of these network devices. In at least one embodiment, the first network data includes one or more of command, control, or routing information.

[0106] Endpoints can be similar to Figure 1Any endpoint 104 depicted. Multicast reduction operations can be performed collaboratively by the endpoints, with each endpoint participating in the multicast reduction operation using network data sent to them from the network device. For example... Figure 1 The data described is moved down from the initiating endpoint to the endpoint via various network devices.

[0107] In at least one embodiment, the endpoints include parallel processing units or other processors, and network data received by the endpoints is used by the parallel processing units or other processors to perform a portion of a multicast operation. Each endpoint performs a portion of the operation, and the endpoints collectively perform the operation (excluding operations performed by the network device, such as reduction). For example, in at least one embodiment, a multicast reduction operation may include evaluating the output of a layer of a neural network. Each endpoint may use the parallel processing unit included in the endpoint to perform a portion of the evaluation. The result of the evaluation may then be uplinked and reduced by the uplink network device.

[0108] In at least one embodiment, the transmission associated with the multicast reduction operation is initiated by the endpoint. In at least one embodiment, this may include something similar to... Figure 1 Any of the endpoints 102 and 104 depicted. For example, in at least one embodiment, endpoint 102 initiates a multicast reduction operation by writing data to a memory location designated as associated with the multicast reduction operation. In at least one embodiment, the memory location is located within the memory of a parallel processing unit or is otherwise accessible to that memory. When data is written, the initiating endpoint can respond by sending network data (including some or all of the data written to the memory location) to a downstream network device. Note that in some cases, the initiating endpoint may send data to some intervening network devices (such as network device 106a), which in turn may send some or all of the data to other network devices (such as network device 106b). Each of these network devices can process the received network data in a similar manner. In at least one embodiment, endpoint (such as...) Figure 1 Any of the endpoints 104 in the process can return multicast reduced data upstream in a similar manner, for example, by writing the data to the memory location of the parallel processing unit. This method allows the parallel processing unit of the endpoint to efficiently perform its part of the multicast reduction operation without having to manage the details of receiving and sending data through the structure.

[0109] At 804, the network device reserves resources to process second network data received from multiple endpoints. This may include reserving resources in association with a multicast reduction operation based on the expected data returned from the endpoints. In at least one embodiment, the second network packet includes part or all of the result of a multicast reduction operation performed from a downlink device and returned uplink. At each stage, partial results may be combined until a complete result is obtained. In at least one embodiment, information about the multicast reduction operation may be conveyed through headers in the network data received by the network device. For example, in at least one embodiment, one or more headers in the network data include routing information. This may include hop-by-hop routing information for routing data according to the topology. In at least one embodiment, one or more headers in the first network data include reduction information associated with the multicast reduction operation. These headers may also include information mapping between the network data and the virtual memory space of the endpoint devices.

[0110] In at least one embodiment, the network device may obtain routing information that may be included in the header of downlink network data. The network device may then store the routing information.

[0111] In at least one embodiment, cycles in the topology of the structure are identified, and resource allocation is based at least in part on the identification of these cycles. For example, in at least one embodiment, undetected loops in the topology may lead to unplanned resource reservation cycles, which can be avoided by detecting such loops and adjusting resource reservations accordingly.

[0112] At point 806, the network device sends first network data to multiple additional network devices. In at least one embodiment, before sending the first network data, the network device inserts routing information into the header of the outgoing data. In at least one embodiment, this new routing information allows downlink devices to route data back uplink according to the applicable topology. During the uplink phase, the network device retrieves stored information and uses the stored information to route network data to the next set of uplink devices.

[0113] At point 808, the network device receives network data that is being returned uplink. To distinguish it from the data being transmitted downlink, this data may be referred to as second network data. This second network data is received after the first network data has been processed by one or more endpoints and is being returned uplink.

[0114] At 810, the network device processes the second network data. This processing may include reduction operations, such as combining data. In at least one embodiment, reduction is performed at least in part based on reduction information obtained from one or more headers of the first network data. This reduction information may include any information used to facilitate the reduction of incoming data, potentially including, but not limited to, information indicating where the reduced data should be sent uplink, information indicating how much response is expected from the downlink device, and information indicating how the data returned uplink should be combined.

[0115] In some cases, errors or other conditions may interrupt or interfere with the processing of multicast reduction operations. This can include downlink devices (including network devices and endpoints) that fail to return data to uplink devices. In at least one embodiment, this situation can be handled by releasing reserved resources after a threshold amount of time has elapsed. In at least one embodiment, the network device releases reserved resources in response to determining that a threshold amount of time has elapsed since sending network data to the downlink device and that at least one of the downlink network devices has not yet responded to receiving the network data sent downlink.

[0116] In at least one embodiment, the network device may discard network traffic associated with a multicast reduction operation in response to a timeout. For example, the network device may determine that a threshold amount of time has elapsed since network data was sent to a downstream device, and determine that at least one of the downstream network devices has not responded. Then, in response to this determination, any subsequent data received by the network device, as well as data associated with the same multicast reduction operation, may be discarded by the network device.

[0117] Servers and data centers

[0118] The following figures illustrate, but are not limited to, systems based on exemplary network servers and data centers that can be used to implement at least one embodiment.

[0119] Figure 9 A distributed system 900 according to at least one embodiment is illustrated. In at least one embodiment, the distributed system 900 includes one or more client computing devices 902, 904, 906, and 908 configured to execute and operate client applications, such as web browsers, proprietary clients, and / or variations thereof, on one or more networks 910. In at least one embodiment, a server 912 may be communicatively coupled to remote client computing devices 902, 904, 906, and 908 via network 910.

[0120] In at least one embodiment, server 912 may be adapted to run one or more services or software applications, such as services and applications that manage session activity for single sign-on (SSO) access across multiple data centers. In at least one embodiment, server 912 may also provide other services or software applications, which may include non-virtual and virtual environments. In at least one embodiment, these services may be provided as web-based services or cloud services or under a Software as a Service (SaaS) model to users of client computing devices 902, 904, 906, and / or 908. In at least one embodiment, users operating client computing devices 902, 904, 906, and / or 908 may in turn use one or more client applications to interact with server 912 to utilize the services provided by these components.

[0121] In at least one embodiment, software components 918, 920, and 922 of system 900 are implemented on server 912. In at least one embodiment, one or more components of system 900 and / or the services provided by these components may also be implemented by one or more client computing devices 902, 904, 906, and / or 908. In at least one embodiment, a user operating a client computing device can then utilize one or more client applications to use the services provided by these components. In at least one embodiment, these components may be implemented using hardware, firmware, software, or a combination thereof. It should be understood that various different system configurations are possible and may differ from the distributed system 900. Therefore, Figure 9 The embodiments shown are examples of distributed systems for implementing the systems of the embodiments, and are not intended to be limiting.

[0122] In at least one embodiment, client computing devices 902, 904, 906, and / or 908 may include different types of computing systems. In at least one embodiment, the client computing device may include a portable handheld device (e.g., Cellular phone Computing tablets, personal digital assistants (PDAs), or wearable devices (e.g., Google) Head-mounted display), running software (such as Microsoft Windows) And / or various mobile operating systems (such as iOS, Windows Phone, Android, BlackBerry 10, Palm OS, and / or variants thereof). In at least one embodiment, the device may support different applications, such as various Internet-related applications, email, short message service (SMS) applications, and may use various other communication protocols. In at least one embodiment, the client computing device may also include a general-purpose personal computer, which, by way of example, includes various versions of Microsoft... Apple Personal computers and / or laptops running Linux operating systems. In at least one embodiment, the client computing device can be running various commercially available operating systems. The client computing device may be a workstation computer operating system similar to UNIX, including but not limited to various GNU / Linux operating systems such as Google Chrome OS. In at least one embodiment, the client computing device may further include electronic devices capable of communicating over one or more networks 910, such as thin client computers, internet-enabled gaming systems (e.g., with or without...). Gesture input devices include Microsoft Xbox game consoles and / or personal messaging devices. Although Figure 9 The distributed system 900 is shown as having four client computing devices, but can support any number of client computing devices. Other devices (such as devices with sensors) can interact with the server 912.

[0123] In at least one embodiment, network 910 in distributed system 900 can be any type of network capable of supporting data communication using any of the various available protocols, including but not limited to TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (System Network Architecture), IPX (Internet Packet Switching), AppleTalk, and / or variations thereof. In at least one embodiment, network 910 can be a local area network (LAN), an Ethernet-based network, Token Ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, or a wireless network (e.g., in the IEEE 802.11 protocol suite). Networks operating under any of the wireless protocols (and / or any other wireless protocols), and / or any combination of these and / or other networks.

[0124] In at least one embodiment, server 912 may consist of one or more general-purpose computers, dedicated server computers (including, by way of example, PC (personal computer) servers, etc. Servers (including mid-range servers, mainframe computers, rack servers, etc.), server farms, server clusters, or any other suitable arrangement and / or combination thereof. In at least one embodiment, server 912 may include one or more virtual machines running a virtual operating system or other computing architectures involving virtualization. In at least one embodiment, one or more flexible pools of logical storage devices may be virtualized to maintain virtual storage devices for the server. In at least one embodiment, the virtual network may be controlled by server 912 using software-defined networking. In at least one embodiment, server 912 may be adapted to run one or more services or software applications.

[0125] In at least one embodiment, server 912 can run any operating system, and any commercially available server operating system. In at least one embodiment, server 912 can also run any of a variety of additional server applications and / or mid-level applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, etc. Servers, database servers, and / or variations thereof. In at least one embodiment, exemplary database servers include, but are not limited to, those commercially available from Oracle, Microsoft, Sybase, IBM (International Business Machines), and / or variations thereof.

[0126] In at least one embodiment, server 912 may include one or more applications for analyzing and merging data feeds and / or event updates received from users of client computing devices 902, 904, 906, and 908. In at least one embodiment, data feeds and / or event updates may include, but are not limited to, data received from one or more third-party information sources and continuous data streams. feed, Updates or real-time updates may include real-time events related to sensor data applications, financial quotes, network performance measurement tools (e.g., network monitoring and business management applications), clickstream analysis tools, vehicle traffic monitoring, and / or their changes. In at least one embodiment, server 912 may also include one or more applications for displaying data feeds and / or real-time events via one or more display devices of client computing devices 902, 904, 906, and 908.

[0127] In at least one embodiment, the distributed system 900 may further include one or more databases 914 and 916. In at least one embodiment, the databases may provide mechanisms for storing information such as user interaction information, usage pattern information, adaptation rule information, and other information. In at least one embodiment, databases 914 and 916 may reside in various locations. In at least one embodiment, one or more databases 914 and 916 may reside on a non-transitory storage medium local to server 912 (and / or within server 912). In at least one embodiment, databases 914 and 916 may be located remotely from server 912 and communicate with server 912 via a network-based connection or a dedicated connection. In at least one embodiment, databases 914 and 916 may reside in a storage area network (SAN). In at least one embodiment, any necessary files for performing functions belonging to server 912 may be appropriately stored locally on server 912 and / or remotely. In at least one embodiment, databases 914 and 916 may include relational databases, such as databases adapted to store, update, and retrieve data in response to SQL-formatted commands.

[0128] Figure 10 An exemplary data center 1000 according to at least one embodiment is shown. In at least one embodiment, the data center 1000 includes, but is not limited to, a data center infrastructure layer 1010, a framework layer 1020, a software layer 1030, and an application layer 1040.

[0129] In at least one embodiment, such as Figure 10 As shown, the data center infrastructure layer 1010 may include a resource coordinator 1012, grouped computing resources 1014, and node computing resources (“nodes CR”) 1016(1)-1016(N), where “N” represents any complete positive integer. In at least one embodiment, nodes CR 1016(1)-1016(N) may include, but are not limited to, any number of central processing units (“CPUs”) or other processors (including accelerators, field-programmable gate arrays (“FPGAs”), graphics processors, etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state drives or disk drives), network input / output (“NW I / O”) devices, network switches, virtual machines (“VMs”), power modules, and cooling modules, etc. In at least one embodiment, one or more nodes CR 1016(1)-1016(N) may be servers having one or more of the aforementioned computing resources.

[0130] In at least one embodiment, the grouped computing resources 1014 may include individual groups (not shown) of node CRs housed in one or more racks, or a plurality of racks (also not shown) housed in data centers in various geographical locations. Individual groups of node CRs within the grouped computing resources 1014 may include computing, networking, memory, or storage resources that can be configured or allocated to support groups of one or more workloads. In at least one embodiment, several node CRs, including CPUs or processors, may be grouped within one or more racks to provide computing resources to support one or more workloads. In at least one embodiment, one or more racks may also include any number of power modules, cooling modules, and network switches, in any combination.

[0131] In at least one embodiment, resource coordinator 1012 may be configured or otherwise control one or more nodes CR1016(1)-1016(N) and / or grouped computing resources 1014. In at least one embodiment, resource coordinator 1012 may include a Software Design Infrastructure (“SDI”) management entity for data center 1000. In at least one embodiment, resource coordinator 1012 may include hardware, software, or some combination thereof.

[0132] In at least one embodiment, such as Figure 10 As shown, the framework layer 1020 includes, but is not limited to, a job scheduler 1032, a configuration manager 1034, a resource manager 1036, and a distributed file system 1038. In at least one embodiment, the framework layer 1020 may include a framework of software 1052 supporting the software layer 1030 and / or one or more applications 1042 of the application layer 1040. In at least one embodiment, the software 1052 or application 1042 may respectively include web-based service software or applications, such as services or applications provided by Amazon Web Services, Google Cloud, and Microsoft Azure. In at least one embodiment, the framework layer 1020 may be, but is not limited to, a free and open-source software web application framework, such as Apache Spark, which can utilize the distributed file system 1038 for large-scale data processing (e.g., "big data"). TM(Hereinafter referred to as "Spark"). In at least one embodiment, the job scheduler 1032 may include a Spark driver to facilitate the scheduling of workloads supported by various layers of data center 1000. In at least one embodiment, the configuration manager 1034 may be able to configure different layers, such as software layer 1030 and framework layer 1020 including Spark and a distributed file system 1038 for supporting large-scale data processing. In at least one embodiment, the resource manager 1036 is able to manage cluster or group computing resources mapped to or allocated to support distributed file system 1038 and job scheduler 1032. In at least one embodiment, cluster or group computing resources may include grouped computing resources 1014 on data center infrastructure layer 1010. In at least one embodiment, the resource manager 1036 may coordinate with resource coordinator 1012 to manage these mapped or allocated computing resources.

[0133] In at least one embodiment, the software 1052 included in the software layer 1030 may include software used by at least a portion of nodes CR1016(1)-1016(N), grouped computing resources 1014, and / or the distributed file system 1038 of the framework layer 1020. One or more types of software may include, but are not limited to, Internet web page search software, email virus scanning software, database software, and streaming video content software.

[0134] In at least one embodiment, the application layer 1040 may include one or more applications 1042 that can be used by at least a portion of nodes CR1016(1)-1016(N), grouped computing resources 1014, and / or the distributed file system 1038 of the framework layer 1020. The one or more types of applications may include, but are not limited to, CUDA applications, 5G network applications, artificial intelligence applications, data center applications, and / or variations thereof.

[0135] In at least one embodiment, any of the configuration manager 1034, resource manager 1036, and resource coordinator 712 can implement any number and type of self-modification actions based on any amount and type of data acquired in any technically feasible manner. In at least one embodiment, self-modification actions can mitigate potentially poor configuration decisions by data center operators of data center 1000 and can prevent underutilization and / or poor performance of the data center.

[0136] Figure 11A client-server network 1104, formed by a plurality of interconnected network server computers 1102, is illustrated according to at least one embodiment. In at least one embodiment, in system 1100, each network server computer 1102 stores data accessible to other network server computers 1102 and client computers 1106 and networks 1108 linked to the wide area network 1104. In at least one embodiment, the configuration of the client-server network 1104 may change over time when client computers 1106 and one or more networks 1108 connect and disconnect from network 1104, and when one or more backbone server computers 1102 are added to or removed from network 1104. In at least one embodiment, the client-server network includes client computers 1106 and networks 1108 when they are connected to network server computers 1102. In at least one embodiment, the term "computer" includes any device or machine capable of accepting data, applying prescribed processes to the data, and providing the results of those processes.

[0137] In at least one embodiment, the client-server network 1104 stores information accessible to the network server computer 1102, the remote network 1108, and the client computer 1106. In at least one embodiment, the network server computer 1102 is formed from a mainframe computer, a minicomputer, and / or a microcomputer, each having one or more processors. In at least one embodiment, the server computer 1102 is linked together via wired and / or wireless transmission media (such as wires, fiber optic cables) and / or microwave transmission media, satellite transmission media, or other conductive, optical, or electromagnetic wave transmission media. In at least one embodiment, the client computer 1106 accesses the network server computer 1102 via similar wired or wireless transmission media. In at least one embodiment, the client computer 1106 can be linked to the client-server network 1104 using a modem and standard telephone communication networks. In at least one embodiment, alternative carrier systems (such as cable and satellite communication systems) can also be used to link to the client-server network 1104. In at least one embodiment, other private or time-sharing carrier systems can be used. In at least one embodiment, the network 1104 is a global information network, such as the Internet. In at least one embodiment, the network is a private intranet using protocols similar to the Internet but with added security measures and restricted access controls. In at least one embodiment, network 1104 is a private or semi-private network using proprietary communication protocols.

[0138] In at least one embodiment, client computer 1106 is any end-user computer, and may also be a mainframe computer, minicomputer, or microcomputer with one or more microprocessors. In at least one embodiment, server computer 1102 may sometimes be used as a client computer to access another server computer 1102. In at least one embodiment, remote network 1108 may be a local area network (LAN), a network added to a wide area network via an independent service provider (ISP) for the Internet, or another group of computers interconnected via wired or wireless transmission media with fixed or time-varying configurations. In at least one embodiment, client computer 1106 may independently or via remote network 1108 be linked to and access network 1104.

[0139] Figure 12An example 1200 of a computer network 1208 connecting one or more computing machines according to at least one embodiment is illustrated. In at least one embodiment, network 1208 can be any type of electrically connected group of computers, including, for example, the Internet, intranet, local area network (LAN), wide area network (WAN), or an interconnection combination of these network types. In at least one embodiment, the connection within network 1208 can be a remote modem, Ethernet (IEEE 802.3), Token Ring (IEEE 802.5), Fiber Distributed Data Link Interface (FDDI), Asynchronous Transfer Mode (ATM), or any other communication protocol. In at least one embodiment, the computing devices linked to the network can be desktop computers, servers, portable, handheld, set-top boxes, personal digital assistants (PDAs), terminals, or any other desired type or configuration. In at least one embodiment, network-connected devices can vary widely in terms of processing power, internal memory, and other performance characteristics, depending on their functionality. In at least one embodiment, communication within the network, as well as communication to or from computing devices connected to the network, can be wired or wireless. In at least one embodiment, network 1208 may at least partially comprise the global public Internet, which typically connects multiple users according to the Transmission Control Protocol / Internet Protocol (TCP / IP) specification in a client-server model. In at least one embodiment, the client-server network is the dominant model for communication between two computers. In at least one embodiment, a client computer (“client”) issues one or more commands to a server computer (“server”). In at least one embodiment, the server fulfills client commands by accessing available network resources and returning information to the client in accordance with the client commands. In at least one embodiment, client computer systems and network resources residing on the network server are assigned network addresses for identification during communication between network elements. In at least one embodiment, communication from other network-connected systems to the server will include the network address of the relevant server / network resource as part of the communication, such that the appropriate destination of the data / request is identified as the recipient. In at least one embodiment, when network 1208 comprises the global Internet, the network address is an IP address in TCP / IP format, which can at least partially route data to email accounts, websites, or other Internet tools residing on the server. In at least one embodiment, information and services residing on a web server can be made available to a client computer's web browser via a domain name (e.g., www.site.com) (which maps to the IP address of the web server).

[0140] In at least one embodiment, multiple clients 1202, 1204, and 1206 are connected to network 1208 via respective communication links. In at least one embodiment, each of these clients can access network 1208 via any desired form of communication, such as via dial-up modem connection, cable link, digital subscriber line (DSL), wireless or satellite link, or any other form of communication. In at least one embodiment, each client can communicate using any machine compatible with network 1208 (e.g., personal computer (PC), workstation, dedicated terminal, personal data assistant (PDA), or other similar device). In at least one embodiment, clients 1202, 1204, and 1206 may or may not be located in the same geographical area.

[0141] In at least one embodiment, multiple servers 1210, 1212, and 1214 are connected to network 1218 to serve clients communicating with network 1218. In at least one embodiment, each server is typically a powerful computer or device that manages network resources and responds to client commands. In at least one embodiment, the server includes computer-readable data storage media, such as hard disk drives and RAM memory, that store program instructions and data. In at least one embodiment, servers 1210, 1212, and 1214 run applications that respond to client commands. In at least one embodiment, server 1210 may run a web server application for responding to client requests for HTML pages, and may also run a mail server application for receiving and routing emails. In at least one embodiment, other applications, such as an FTP server or media server for streaming audio / video data to clients, may also run on server 1210. In at least one embodiment, different servers may be dedicated to performing different tasks. In at least one embodiment, server 1210 may be a dedicated web server for managing website-related resources for different users, while server 1212 may be dedicated to providing email management. In at least one embodiment, the other servers may be dedicated to a combination of two or more services, such as media (audio, video, etc.), File Transfer Protocol (FTP), or services generally available or provided over the network. In at least one embodiment, each server may be located in the same or different location as the other servers. In at least one embodiment, multiple servers may exist to perform mirroring tasks for users, thereby mitigating congestion or minimizing traffic directed to and from a single server. In at least one embodiment, servers 1210, 1212, and 1214 are under the control of a web hosting provider that maintains and delivers third-party content over network 1218.

[0142] In at least one embodiment, the web hosting provider delivers services to two different types of clients. In at least one embodiment, one type, which may be referred to as a browser, requests content, such as web pages, email messages, video clips, etc., from servers 1210, 1212, and 1214. In at least one embodiment, a second type (which may be referred to as a user) hires the web hosting provider to maintain network resources (such as websites) and make them available to the browser. In at least one embodiment, the user contracts with the web hosting provider to make memory space, processor capacity, and communication bandwidth available to the network resources they desire, according to the amount of server resources the user expects to utilize.

[0143] In at least one embodiment, in order for a web hosting provider to serve both clients, the application managing network resources hosted on the server must be properly configured. In at least one embodiment, the program configuration process involves defining a set of parameters that at least partially control the application's response to browser requests and also at least partially define the server resources available to a particular user.

[0144] In one embodiment, intranet server 1216 communicates with network 1208 via a communication link. In at least one embodiment, intranet server 1216 communicates with server manager 1218. In at least one embodiment, server manager 1218 includes a database of application configuration parameters used by servers 1210, 1212, and 1214. In at least one embodiment, a user modifies database 1220 via intranet 1216, and server manager 1218 interacts with servers 1210, 1212, and 1214 to modify application parameters such that they match the contents of the database. In at least one embodiment, a user logs into intranet 1216 by connecting to intranet 1216 via computer 1202 and entering authentication information such as a username and password.

[0145] In at least one embodiment, when a user wishes to log in to a new service or modify an existing service, the intranet server 1216 authenticates the user and provides the user with an interactive screen display / control panel that allows the user to access configuration parameters for a specific application. In at least one embodiment, multiple modifiable text boxes describing aspects of the user's website or other network resources are presented to the user. In at least one embodiment, if the user desires to increase the storage space reserved for their website on the server, a field is provided where the user specifies the desired storage space. In at least one embodiment, in response to receiving this information, the intranet server 1216 updates the database 1220. In at least one embodiment, the server manager 1218 forwards the information to the appropriate server and uses the new parameters during application operation. In at least one embodiment, the intranet server 1216 is configured to provide the user with access to configuration parameters of network resources (e.g., web pages, email, FTP sites, media sites, etc.) that the user has contracted with a web hosting service provider.

[0146] Figure 13A A networked computer system 1300A according to at least one embodiment is illustrated. In at least one embodiment, the networked computer system 1300A includes a plurality of nodes or personal computers (“PCs”) 1302, 1318, 1320. In at least one embodiment, the personal computer or node 1302 includes a processor 1314, memory 1316, a camera 1304, a microphone 1306, a mouse 1308, a speaker 1310, and a monitor 1312. In at least one embodiment, PCs 1302, 1318, 1320 may each run one or more desktop servers, such as those on an internal network within a given company, or may be servers on a general network not limited to a specific environment. In at least one embodiment, each PC node in the network has one server, such that each PC node in the network represents a specific network server with a specific network URL address. In at least one embodiment, each server defaults to a default webpage for the user of that server, and the default webpage itself may contain embedded URLs pointing to further subpages for that user on that server, or to pages on other servers on the network or on other servers.

[0147] In at least one embodiment, nodes 1302, 1318, 1320 and other nodes of the network are interconnected via medium 1322. In at least one embodiment, medium 1322 may be a communication channel such as Integrated Services Digital Network (“ISDN”). In at least one embodiment, the individual nodes of the networked computer system may be connected via various communication media, including a local area network (“LAN”), a simple old-fashioned telephone line (“POTS”) (sometimes referred to as the Public Switched Telephone Network (“PSTN”)), and / or variations thereof. In at least one embodiment, the individual nodes of the network may also constitute users of computer systems interconnected via a network such as the Internet. In at least one embodiment, each server on the network (running from a specific node of the network at a given instance) has a unique address or identifier within the network, which may be specified according to a URL.

[0148] In at least one embodiment, multiple multipoint conferencing units (“MCUs”) can therefore be used to transmit data to and from various nodes or “endpoints” of the conferencing system. In at least one embodiment, in addition to various other communication media (such as nodes connected via the Internet), the nodes and / or MCUs may be interconnected via ISDN links or through a local area network (“LAN”). In at least one embodiment, the nodes of the conferencing system may typically be directly connected to a communication medium (such as a LAN) or connected via an MCU, and the conferencing system may include other nodes or components, such as routers, servers, and / or variations thereof.

[0149] In at least one embodiment, processor 1314 is a general-purpose programmable processor. In at least one embodiment, the processor of a node in the networked computer system 1300A may also be a dedicated video processor. In at least one embodiment, the different peripheral devices and components of a node (such as those of node 1302) may differ from those of other nodes. In at least one embodiment, nodes 1318 and 1320 may be configured to be the same as or different from node 1302. In at least one embodiment, the node may be implemented on any suitable computer system other than a PC system.

[0150] Figure 13BA networked computer system 1300B according to at least one embodiment is illustrated. In at least one embodiment, system 1300B illustrates a network (such as LAN 1324) that can be used to interconnect various nodes that can communicate with each other. In at least one embodiment, multiple nodes, such as PC nodes 1326, 1328, and 1330, are attached to LAN 1324. In at least one embodiment, nodes may also be connected to the LAN via a network server or other means. In at least one embodiment, system 1300B includes other types of nodes or elements, including, for example, routers, servers, and nodes.

[0151] Figure 13C A networked computer system 1300C is illustrated according to at least one embodiment. In at least one embodiment, system 1300C illustrates a WWW system with communication across a backbone communication network (such as the Internet 1332), the backbone communication network being usable for various nodes interconnecting the network. In at least one embodiment, the WWW is a set of protocols operating on top of the Internet and allows a graphical interface system to operate on it to access information via the Internet. In at least one embodiment, the Internet 1332 attached to the WWW consists of multiple nodes, such as PCs 1340, 1342, and 1344. In at least one embodiment, nodes interface with other nodes of the WWW via WWW HTTP servers (such as servers 1334 and 1336). In at least one embodiment, PC 1344 may be a PC forming a node of network 1332, and PC 1344 itself runs its server 1336, although for illustrative purposes... Figure 13C PC 1344 and server 1336 are shown separately.

[0152] In at least one embodiment, the WWW is a distributed type of application characterized by WWW HTTP, the WWW protocol, which runs on top of the Internet's Transmission Control Protocol / Internet Protocol (“TCP / IP”). In at least one embodiment, the WWW can therefore be characterized by a set of protocols running on the Internet (i.e., HTTP) as its “backbone”.

[0153] In at least one embodiment, the web browser is an application running on a node of a network in a WWW-compatible network system, allowing users of a particular server or node to view such information and thus allowing users to search for graphics and text-based documents linked together using hypertext links embedded in documents or files available from servers that understand HTTP. In at least one embodiment, when a user uses another server on a network such as the Internet to retrieve a given webpage from a first server associated with a first node, the retrieved document may have different hypertext links embedded therein, and a local copy of the page created locally by the user is also retrieved. In at least one embodiment, when a user clicks a hypertext link, locally stored information associated with the selected hypertext link is generally sufficient to allow the user's machine to open a connection over the Internet to the server indicated by the hypertext link.

[0154] In at least one embodiment, more than one user may be coupled to each HTTP server, for example, via a LAN (such as LAN 1338, as shown with respect to WWW HTTP server 1334). In at least one embodiment, system 1300C may also include other types of nodes or elements. In at least one embodiment, the WWW HTTP server is an application running on a machine such as a PC. In at least one embodiment, each user can be considered to have a unique "server," as shown with respect to PC 1344. In at least one embodiment, a server can be considered to be a server such as WWW HTTP server 1334 that provides access to the network for a LAN or multiple nodes or multiple LANs. In at least one embodiment, there are multiple users, each user having a desktop PC or a node on the network, each desktop PC potentially setting up a server for its users. In at least one embodiment, each server is associated with a specific network address or URL that, when accessed, provides a default webpage for that user at that specific network address or URL. In at least one embodiment, the webpage may contain further links (embedded URLs) pointing to further subpages for that user on that server, or to other servers on the network or to pages on other servers on the network.

[0155] Cloud computing and services

[0156] The following figures illustrate, but are not limited to, exemplary cloud-based systems that can be used to implement at least one embodiment.

[0157] In at least one embodiment, cloud computing is a computing style in which dynamically scalable and typically virtualized resources are provided as a service over the Internet. In at least one embodiment, users do not need knowledge of, expertise in, or control over the technical infrastructure supporting them, which may be referred to as "in the cloud." In at least one embodiment, cloud computing consolidates infrastructure into services, Platform as a Service (PaaS), Software as a Service (SaaS), and other variations with common Internet-dependent themes to meet users' computing needs. In at least one embodiment, a typical cloud deployment (such as in a private cloud (e.g., an enterprise network)) or a data center (DC) in a public cloud (e.g., the Internet) may consist of thousands of servers (or alternatively, VMs), hundreds of Ethernet, Fibre Channel, or Fibre Channel over Ethernet (FCoE) ports, switching and storage infrastructure, etc. In at least one embodiment, the cloud may also consist of network service infrastructure such as IPsec VPN hubs, firewalls, load balancers, wide area network (WAN) optimizers, etc. In at least one embodiment, remote subscribers can securely access cloud applications and services via a VPN tunnel (such as an IPsec VPN tunnel).

[0158] In at least one embodiment, cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage devices, applications, and services) that can be quickly configured and released with minimal management effort or service provider interaction.

[0159] In at least one embodiment, cloud computing is characterized by on-demand self-service, where consumers can automatically and unilaterally provision computing power, such as server time and network storage, as needed, without human interaction with each service provider. In at least one embodiment, cloud computing is characterized by broad network access, where capabilities are available on the network and accessed via standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs). In at least one embodiment, cloud computing is characterized by resource pooling, where a provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically signed and reallocated based on consumer demand. In at least one embodiment, there is a sense of location independence because consumers typically have no control or knowledge of the exact location of the resources provided, but may be able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0160] In at least one embodiment, examples of resources include storage, processing, memory, network bandwidth, and virtual machines. In at least one embodiment, cloud computing is characterized by rapid elasticity, where capacity can be rapidly and elastically provisioned (in some cases automatically) to scale down and up rapidly. In at least one embodiment, for consumers, the capacity available for provisioning generally appears unlimited and can be purchased at any time in any quantity. In at least one embodiment, cloud computing is characterized by measured services, where the cloud system automatically controls and optimizes resource usage by leveraging metering capabilities at some level of abstraction suitable for service types (e.g., storage, processing, bandwidth, and active user accounts). In at least one embodiment, resource usage can be monitored, controlled, and reported, thereby providing transparency to both the providers and consumers of the services utilized.

[0161] In at least one embodiment, cloud computing may be associated with various services. In at least one embodiment, cloud Software as a Service (SaaS) may refer to the ability provided to consumers as a service that uses applications from a provider running on cloud infrastructure. In at least one embodiment, applications may be accessed from different client devices via a thin client interface such as a web browser (e.g., web-based email). In at least one embodiment, consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage, or even the capabilities of individual applications, with possible exceptions of limited user-specific application configuration settings.

[0162] In at least one embodiment, Cloud Platform as a Service (PaaS) can refer to a service in which the ability to provide consumers with the capability to deploy consumer-created or acquired applications onto cloud infrastructure, these applications being created using programming languages ​​and tools supported by the provider. In at least one embodiment, the consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but has control over the deployed applications and, possibly, the configuration of the application hosting environment.

[0163] In at least one embodiment, cloud infrastructure as a service (IaaS) can refer to a service in which the capabilities provided to consumers are processing, storage, networking, and other basic computing resources that consumers can deploy and run, including operating systems and applications. In at least one embodiment, consumers do not manage or control the underlying cloud infrastructure, but instead have control over the operating system, storage, deployed applications, and possibly limited control over selected networking components (e.g., host firewalls).

[0164] In at least one embodiment, cloud computing can be deployed in different ways. In at least one embodiment, a private cloud can refer to cloud infrastructure that is operated solely by an organization. In at least one embodiment, a private cloud can be managed by an organization or a third party and can exist on-site or off-site. In at least one embodiment, a community cloud can refer to cloud infrastructure shared by several organizations and supporting a specific community with shared concerns (e.g., mission, security requirements, policies, and compliance considerations). In at least one embodiment, a community cloud can be managed by an organization or a third party and can exist on-site or off-site. In at least one embodiment, a public cloud can refer to cloud infrastructure that is available to the general public or a large industry group and is owned by an organization providing cloud services. In at least one embodiment, a hybrid cloud can refer to cloud infrastructure that is composed of two or more clouds (private, community, or public) that remain a single entity but are bound together by standardization or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds). In at least one embodiment, the cloud computing environment is service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability.

[0165] Figure 14 One or more components of a system environment 1400 according to at least one embodiment are illustrated, wherein services can be provided as third-party network services. In at least one embodiment, the third-party network may be referred to as a cloud, cloud network, cloud computing network, and / or variations thereof. In at least one embodiment, system environment 1400 includes one or more client computing devices 1404, 1406, and 1408, which can be used by users to interact with a third-party network infrastructure system 1402 that provides third-party network services (which may be referred to as cloud computing services). In at least one embodiment, third-party network infrastructure system 1402 may include one or more computers and / or servers.

[0166] It should be understood that Figure 14 The third-party network infrastructure system 1402 described herein may have components other than those described. Furthermore, Figure 14 An embodiment of a third-party network infrastructure system is described. In at least one embodiment, the third-party network infrastructure system 1402 may have a greater than Figure 14 The more or fewer components depicted may be combined into two or more components, or may have different component configurations or arrangements.

[0167] In at least one embodiment, client computing devices 1404, 1406, and 1408 may be configured to operate client applications, such as web browsers, which may be used by a user of the client computing devices to interact with a third-party network infrastructure system 1402 to use proprietary client applications or other applications that provide services provided by the third-party network infrastructure system 1402. Although the exemplary system environment 1400 is shown as having three client computing devices, any number of client computing devices can be supported. In at least one embodiment, other devices, such as devices with sensors, may interact with the third-party network infrastructure system 1402. In at least one embodiment, one or more networks 1410 may facilitate communication and data exchange between client computing devices 1404, 1406, and 1408 and the third-party network infrastructure system 1402.

[0168] In at least one embodiment, the services provided by the third-party network infrastructure system 1402 may include hosting services available on demand to users of the third-party network infrastructure system. In at least one embodiment, various services may also be provided, including but not limited to online data storage and backup solutions, web-based email services, hosted office suites and document collaboration services, database management and processing, managed technical support services, and / or variations thereof. In at least one embodiment, the services provided by the third-party network infrastructure system can be dynamically expanded to meet the needs of its users.

[0169] In at least one embodiment, a specific instantiation of a service provided by the third-party network infrastructure system 1402 may be referred to as a "service instance". In at least one embodiment, generally, any service available to a user from the third-party network service provider system via a communication network (such as the Internet) is referred to as a "third-party network service". In at least one embodiment, in a public third-party network environment, the servers and systems constituting the third-party network service provider system are different from the servers and systems on the customer's own premises. In at least one embodiment, the third-party network service provider system may host applications, and users may subscribe to and use applications on demand via a communication network (such as the Internet).

[0170] In at least one embodiment, services within a third-party network infrastructure may include protected computer network access to storage, hosted databases, hosted web servers, software applications, or other services provided to users by a third-party network provider. In at least one embodiment, services may include password-protected access to remote storage devices on a third-party network via the Internet. In at least one embodiment, services may include a web-based hosted relational database and a scripting language middleware engine for private use by network developers. In at least one embodiment, services may include access to email software applications hosted on a website hosted by a third-party network provider.

[0171] In at least one embodiment, the third-party network infrastructure system 1402 may include a suite of application, middleware, and database service providers delivered to clients in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. In at least one embodiment, the third-party network infrastructure system 1402 may also provide “big data” related computing and analytics services. In at least one embodiment, the term “big data” is generally used to refer to extremely large datasets that can be stored and manipulated by analysts and researchers to visualize large amounts of data, detect trends, and / or otherwise interact with the data. In at least one embodiment, big data and related applications may be hosted and / or manipulated by the infrastructure system at many levels and at different scales. In at least one embodiment, dozens, hundreds, or thousands of processors linked in parallel may act on such data to present the data or simulate external forces on the data or what it represents. In at least one embodiment, these datasets may involve structured data (such as structured data organized in a database or otherwise according to a structured model) and / or unstructured data (e.g., emails, images, data blobs (binary large objects), web pages, complex event processing). In at least one embodiment, by leveraging the ability of the embodiment to focus more (or less) computing resources relatively quickly on the target, third-party network infrastructure systems can be better used to perform tasks on large datasets based on the needs of enterprises, government agencies, research organizations, private individuals, groups of like-minded individuals or organizations, or other entities.

[0172] In at least one embodiment, the third-party network infrastructure system 1402 can be adapted to automatically provide, manage, and track customer subscriptions to services provided by the third-party network infrastructure system 1402. In at least one embodiment, the third-party network infrastructure system 1402 can provide third-party network services via different deployment models. In at least one embodiment, services can be provided under a public third-party network model, wherein the third-party network infrastructure system 1402 is owned by an organization selling third-party network services, and the services are available to the general public or various industry enterprises. In at least one embodiment, services can be provided under a private third-party network model, in which the third-party network infrastructure system 1402 operates only for a single organization and can provide services to one or more entities within that organization. In at least one embodiment, third-party network services can also be provided under a community third-party network model, wherein the third-party network infrastructure system 1402 and the services provided by the third-party network infrastructure system 1402 are shared by several organizations in the relevant community. In at least one embodiment, third-party network services can also be provided under a hybrid third-party network model, which is a combination of two or more different models.

[0173] In at least one embodiment, the services provided by the third-party network infrastructure system 1402 may include one or more services offered under the Software as a Service (SaaS) category, the Platform as a Service (PaaS) category, the Infrastructure as a Service (IaaS) category, or other service categories that include hybrid services. In at least one embodiment, a customer may subscribe to one or more services provided by the third-party network infrastructure system 1402 via a subscription order. In at least one embodiment, the third-party network infrastructure system 1402 then performs processing to provide the services in the customer's subscription order.

[0174] In at least one embodiment, the services provided by the third-party network infrastructure system 1402 may include, but are not limited to, application services, platform services, and infrastructure services. In at least one embodiment, application services may be provided by the third-party network infrastructure system via a SaaS platform. In at least one embodiment, the SaaS platform may be configured to provide third-party network services belonging to the SaaS category. In at least one embodiment, the SaaS platform may provide the ability to build and deliver a suite of on-demand applications on an integrated development and deployment platform. In at least one embodiment, the SaaS platform may manage and control the underlying software and infrastructure used to provide SaaS services. In at least one embodiment, by utilizing the services provided by the SaaS platform, customers can utilize applications running on the third-party network infrastructure system. In at least one embodiment, customers can obtain application services without needing to purchase separate licenses and support. In at least one embodiment, a variety of different SaaS services may be provided. In at least one embodiment, examples include, but are not limited to, services that provide solutions for sales performance management, enterprise integration, and business agility for large organizations.

[0175] In at least one embodiment, the platform service may be provided by a third-party network infrastructure system 1402 via a PaaS platform. In at least one embodiment, the PaaS platform may be configured to provide third-party network services belonging to the PaaS category. In at least one embodiment, examples of platform services may include, but are not limited to, services that enable organizations to merge existing applications on a shared public architecture, and the ability to build new applications utilizing shared services provided by the platform. In at least one embodiment, the PaaS platform may manage and control the underlying software and infrastructure used to provide the PaaS service. In at least one embodiment, customers can access the PaaS service provided by the third-party network infrastructure system 1402 without the need for customers to purchase separate licenses and support.

[0176] In at least one embodiment, by leveraging services provided by a PaaS platform, customers can employ programming languages ​​and tools supported by a third-party network infrastructure system and also control the deployed services. In at least one embodiment, the platform services provided by the third-party network infrastructure system may include database third-party network services, middleware third-party network services, and third-party network services. In at least one embodiment, the database third-party network service may support a shared service deployment model that enables organizations to aggregate database resources and provide database-as-a-service to customers in the form of a database third-party network. In at least one embodiment, within the third-party network infrastructure system, the middleware third-party network service can provide customers with a platform to develop and deploy different business applications, and the third-party network service can provide customers with a platform to deploy applications.

[0177] In at least one embodiment, various infrastructure services may be provided by an IaaS platform within a third-party network infrastructure system. In at least one embodiment, infrastructure services facilitate the management and control of underlying computing resources (such as storage, networking, and other basic computing resources) by customers utilizing services provided by SaaS and PaaS platforms.

[0178] In at least one embodiment, the third-party network infrastructure system 1402 may further include infrastructure resources 1430 for providing resources for offering various services to customers of the third-party network infrastructure system. In at least one embodiment, infrastructure resources 1430 may include a pre-integrated and optimized combination of hardware (such as servers, storage, and networking resources) for performing services provided by PaaS platforms and SaaS platforms, as well as other resources.

[0179] In at least one embodiment, resources in the third-party network infrastructure system 1402 can be shared by multiple users and dynamically reallocated as needed. In at least one embodiment, resources can be allocated to users in different time zones. In at least one embodiment, the third-party network infrastructure system 1402 can enable a first group of users in a first time zone to utilize the resources of the third-party network infrastructure system for a specified number of hours, and subsequently enable the reallocation of the same resources to another group of users located in a different time zone, thereby maximizing resource utilization.

[0180] In at least one embodiment, multiple internal shared services 1432, shared by different components or modules of the third-party network infrastructure system 1402, may be provided to enable services provided by the third-party network infrastructure system 1402. In at least one embodiment, these internal shared services may include, but are not limited to, security and identity services, integration services, enterprise library services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, services for enabling third-party network support, email services, notification services, file transfer services, and / or variations thereof.

[0181] In at least one embodiment, the third-party network infrastructure system 1402 can provide comprehensive management of third-party network services (e.g., SaaS, PaaS, and IaaS services) within the third-party network infrastructure system. In at least one embodiment, the third-party network management functionality may include the ability and / or variations thereof for provisioning, managing, and tracking customer subscriptions received by the third-party network infrastructure system 1402.

[0182] In at least one embodiment, such as Figure 14As shown, third-party network management functions can be provided by one or more modules, such as order management module 1420, order coordination module 1422, order supply module 1424, order management and monitoring module 1426, and identity management module 1428. In at least one embodiment, these modules may include or be provided using one or more computers and / or servers, which may be general-purpose computers, dedicated server computers, server farms, server clusters, or any other suitable arrangement and / or combination.

[0183] In at least one embodiment, in step 1434, a customer using a client device (such as client computing devices 1404, 1406, or 1408) interacts with the third-party network infrastructure system 1402 by requesting one or more services provided by the third-party network infrastructure system 1402 and placing an order for a subscription to one or more services provided by the third-party network infrastructure system 1402. In at least one embodiment, the customer may access a third-party network user interface (UI), such as third-party network UI 1412, third-party network UI 1414, and / or third-party network UI 1416, and place orders via these UIs. In at least one embodiment, order information received by the third-party network infrastructure system 1402 in response to a customer placing an order may include information identifying the customer and the one or more services provided by the third-party network infrastructure system 1402 that the customer wishes to subscribe to.

[0184] In at least one embodiment, in step 1436, the order information received from the customer may be stored in the order database 1418. In at least one embodiment, if this is a new order, a new record may be created for the order. In at least one embodiment, the order database 1418 may be one of several databases operated by a third-party network infrastructure system 1418 and in conjunction with other system components.

[0185] In at least one embodiment, in step 1438, the order information can be forwarded to the order management module 1420, which can be configured to perform billing and accounting functions related to the order, such as verifying the order and, after verification, booking an order.

[0186] In at least one embodiment, in step 1440, information about the order may be transmitted to an order coordination module 1422, which is configured to coordinate the provision of services and resources for orders placed by customers. In at least one embodiment, the order coordination module 1422 may use the services of the order provisioning module 1424 for provisioning. In at least one embodiment, the order coordination module 1422 enables the management of business processes associated with each order and applies business logic to determine whether an order should continue to be provisioned.

[0187] In at least one embodiment, in step 1442, upon receiving a new subscription order, the order coordination module 1422 sends a request to the order provisioning module 1424 to allocate resources and configure the resources required to satisfy the subscription order. In at least one embodiment, the order provisioning module 1424 implements resource allocation for services subscribed by customers. In at least one embodiment, the order provisioning module 1424 provides an abstraction level between third-party network services provided by the third-party network infrastructure system 1400 and the physical implementation layer used to supply resources for providing the requested services. In at least one embodiment, this allows the order coordination module 1422 to be isolated from implementation details, such as whether services and resources are actually provisioned in real-time or pre-provisioned and allocated / assigned only upon request.

[0188] In at least one embodiment, in step 1444, once the service and resources are provided, a notification instructing the subscribing customer that the requested service is now ready for use can be sent. In at least one embodiment, information (e.g., a link) can be sent to the customer, enabling the customer to begin using the requested service.

[0189] In at least one embodiment, in step 1446, the customer's subscription order can be managed and tracked by the order management and monitoring module 1426. In at least one embodiment, the order management and monitoring module 1426 can be configured to collect usage statistics regarding customer use of the subscription service. In at least one embodiment, statistics can be collected for storage usage, data transfer volume, number of users, and the amount and / or changes in system power-on and power-off times.

[0190] In at least one embodiment, the third-party network infrastructure system 1400 may include an identity management module 1428 configured to provide identity services, such as access management and authorization services within the third-party network infrastructure system 1400. In at least one embodiment, the identity management module 1428 may control information about customers who wish to utilize services provided by the third-party network infrastructure system 1402. In at least one embodiment, such information may include information authenticating the identity of such customers and information describing which actions those customers are authorized to perform relative to various system resources (e.g., files, directories, applications, communication ports, memory segments, etc.). In at least one embodiment, the identity management module 1428 may also include managing descriptive information about each customer, as well as information about how and by whom that descriptive information can be accessed and modified.

[0191] Figure 15 A cloud computing environment 1502 according to at least one embodiment is illustrated. In at least one embodiment, the cloud computing environment 1502 includes one or more computer systems / servers 1504, with computing devices such as personal digital assistants (PDAs) or cellular phones 1506A, desktop computers 1506B, laptop computers 1506C, and / or automotive computer systems 1506N communicating with the one or more computer systems / servers 1504. In at least one embodiment, this allows infrastructure, platforms, and / or software to be provided as services from the cloud computing environment 1502 so that each client does not need to maintain such resources individually. It should be understood that... Figure 15 The types of computing devices 1506A-N shown are intended to be illustrative only, and the cloud computing environment 1502 can communicate with any type of computerized device via any type of network and / or network / addressable connection (e.g., using a web browser).

[0192] In at least one embodiment, the computer system / server 1504, which may be represented as a cloud computing node, may operate with many other general-purpose or special-purpose computing system environments or configurations. In at least one embodiment, computing systems, environments, and / or configurations suitable for use with the computer system / server 1504 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computers, and distributed cloud computing environments that include any of the aforementioned systems or devices, and / or variations thereof.

[0193] In at least one embodiment, the computer system / server 1504 can be described in the general context of computer system executable instructions (such as program modules) executed by the computer system. In at least one embodiment, the program module includes routines, programs, objects, components, logic, data structures, etc., that perform a specific task or implement a specific abstract data type. In at least one embodiment, the exemplary computer system / server 1504 can be practiced in a distributed cloud computing environment, where the task is performed by a remote processing device linked via a communication network. In at least one embodiment, in a distributed cloud computing environment, the program module may reside in both local and remote computer system storage media, including memory storage devices.

[0194] Figure 16 The cloud computing environment 1502 is shown according to at least one embodiment. Figure 15 This provides a set of functional abstractions. It should be understood beforehand. Figure 16 The components, layers, and functions shown are intended to be illustrative only, and may vary.

[0195] In at least one embodiment, the hardware and software layer 1602 includes hardware and software components. Examples of hardware components in at least one embodiment include mainframes, servers based on various RISC (Reduced Instruction Set Computer) architectures, various computing systems, supercomputing systems, storage devices, networks, networking components, and / or variations thereof. Examples of software components in at least one embodiment include network application server software, various application server software, various database software, and / or variations thereof.

[0196] In at least one embodiment, the virtualization layer 1604 provides an abstraction layer from which exemplary virtual entities such as virtual servers, virtual storage, virtual networks (including virtual private networks), virtual applications, virtual clients, and / or variations thereof can be provided.

[0197] In at least one embodiment, the management layer 1606 provides various functionalities. In at least one embodiment, resource provisioning provides the dynamic acquisition of computing resources and other resources for performing tasks within the cloud computing environment. In at least one embodiment, metering provides usage tracking of resources within the cloud computing environment, and billing or invoicing for the consumption of these resources. In at least one embodiment, resources may include application software licenses. In at least one embodiment, security provides authentication for users and tasks, and protection for data and other resources. In at least one embodiment, the user interface provides access to the cloud computing environment for both users and system administrators. In at least one embodiment, service level management provides the allocation and management of cloud computing resources to meet required service levels. In at least one embodiment, service level agreement (SLA) management provides the pre-deployment and acquisition of cloud computing resources, anticipating future demand for those resources according to the SLA.

[0198] In at least one embodiment, workload layer 1608 provides functionality that leverages a cloud computing environment. Examples of workloads and functionalities that may be provided from this layer in at least one embodiment include: mapping and navigation, software development and management, educational services, data analysis and processing, transaction processing, and service delivery.

[0199] Supercomputing

[0200] The following figures illustrate, but are not limited to, exemplary supercomputer-based systems that can be used to implement at least one embodiment.

[0201] In at least one embodiment, a supercomputer can refer to a hardware system exhibiting significant parallelism and comprising at least one chip, wherein the chips in the system are interconnected via a network and housed in a hierarchically organized enclosure. In at least one embodiment, a large hardware system filling a server room with several racks is a specific example of a supercomputer, each rack containing several board / rack modules, each board / rack module containing several chips all interconnected by a scalable network. In at least one embodiment, a single rack of such a large hardware system is another example of a supercomputer. In at least one embodiment, a single chip exhibiting significant parallelism and comprising several hardware components can also be considered a supercomputer, because as feature sizes may decrease, the number of hardware components that can be incorporated into a single chip may also increase.

[0202] Figure 17A chip-level supercomputer according to at least one embodiment is illustrated. In at least one embodiment, the main computation is performed within a finite state machine (1704) referred to as a thread unit, inside an FPGA or ASIC chip. In at least one embodiment, a task and synchronization network (1702) connects to the finite state machine and is used to dispatch threads and perform operations in the correct order. In at least one embodiment, a memory network (1706, 1710) is used to access a multi-level partitioned on-chip cache hierarchy (1708, 1712). In at least one embodiment, a memory controller (1716) and an off-chip memory network (1714) are used to access off-chip memory. In at least one embodiment, an I / O controller (1718) is used for cross-chip communication when the design is not suitable for a single logic chip.

[0203] Figure 18 A supercomputer at the rack module level is illustrated according to at least one embodiment. In at least one embodiment, within the rack module, there are multiple FPGA or ASIC chips (1802) connected to one or more DRAM cells (1804) constituting the main accelerator memory. In at least one embodiment, each FPGA / ASIC chip is connected to its adjacent FPGA / ASIC chip using a wide on-board bus with differential high-speed signaling (1806). In at least one embodiment, each FPGA / ASIC chip is also connected to at least one high-speed serial communication cable.

[0204] Figure 19 A rack-level supercomputer according to at least one embodiment is shown. Figure 20 A supercomputer at the entire system level is illustrated according to at least one embodiment. In at least one embodiment, see [link to at least one embodiment]. Figure 19 and Figure 20High-speed serial optical fiber or copper cable (1902, 2002) is used to implement scalable, potentially incomplete, hypercube networks between rack modules within a rack and across the entire system rack. In at least one embodiment, one of the FPGA / ASIC chips in the accelerator is connected to the host system via a PCI-Express connection (2004). In at least one embodiment, the host system includes a host microprocessor (2008) on which the software portion of the application runs, and memory consisting of one or more host memory DRAM cells (2006) consistent with the memory on the accelerator. In at least one embodiment, the host system may be a separate module on one of the racks or may be integrated with one of the modules of the supercomputer. In at least one embodiment, a circular topology of cube connections provides communication links to create a hypercube network for a large supercomputer. In at least one embodiment, a group of FPGA / ASIC chips on a rack module can act as a single hypercube node, increasing the total number of external links per group compared to a single chip. In at least one embodiment, a group comprises chips A, B, C, and D on a rack module having an internal wide differential bus connecting A, B, C, and D in a circular organization. In at least one embodiment, there are 12 serial communication cables connecting the rack module to the outside world. In at least one embodiment, chip A on the rack module is connected to serial communication cables 0, 1, and 2. In at least one embodiment, chip B is connected to cables 3, 4, and 5. In at least one embodiment, chip C is connected to cables 6, 7, and 8. In at least one embodiment, chip D is connected to cables 9, 10, and 11. In at least one embodiment, the entire group {A, B, C, D} constituting the rack module can form a hypercube node within a supercomputer system, containing up to 2^12 = 4096 rack modules (16384 FPGA / ASIC chips). In at least one embodiment, in order for chip A to send a message outward on link 4 of group {A, B, C, D}, the message must first be routed to chip B using an on-board differential wide bus connection. In at least one embodiment, messages arriving on link 4 from group {A, B, C, D} (i.e., to chip A) must also first be routed to the correct destination chip (A) within group {A, B, C, D}. In at least one embodiment, parallel supercomputer systems of other sizes can also be implemented.

[0205] AI

[0206] The following figures illustrate, but are not limited to, exemplary artificial intelligence-based systems that can be used to implement at least one embodiment.

[0207] Figure 21AInference and / or training logic 2115 is shown for performing inference and / or training operations associated with one or more embodiments. The following is in conjunction with... Figure 21A and / or Figure 21B Provide details about reasoning and / or training logic 2115.

[0208] In at least one embodiment, the inference and / or training logic 2115 may include, but is not limited to, code and / or data storage 2101 for storing forward and / or output weights and / or input / output data, and / or other parameters for configuring neurons or layers of a neural network trained and / or used for inference in aspects of one or more embodiments. In at least one embodiment, the training logic 2115 may include or be coupled to the code and / or data storage 2101 for storing graph code or other software to control timing and / or sequence, wherein weight and / or other parameter information is loaded to configure the logic, including integer and / or floating-point units (collectively, arithmetic logic units (ALUs)). In at least one embodiment, code (such as graph code) loads weight or other parameter information into the processor ALU based on the architecture of the neural network to which such code corresponds. In at least one embodiment, the code and / or data storage 2101 stores weight parameters and / or input / output data for each layer of a neural network that is trained or used in conjunction with one or more embodiments during forward propagation of input / output data and / or weight parameters during training and / or inference using aspects of one or more embodiments. In at least one embodiment, any portion of the code and / or data storage 2101 may be included together with other on-chip or off-chip data storage devices, including the processor's L1, L2, or L3 cache memory or system memory.

[0209] In at least one embodiment, any portion of the code and / or data storage 2101 may be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, the code and / or data storage 2101 may be a cache memory, dynamic random-addressable memory (“DRAM”), static random-addressable memory (“SRAM”), non-volatile memory (e.g., flash memory), or other storage devices. In at least one embodiment, the choice of whether the code and / or data storage 2101 is internal or external to the processor, for example, or includes DRAM, SRAM, flash memory, or some other storage type, may depend on the available on-chip storage relative to off-chip storage, the latency requirements of the training and / or inference functions being performed, the batch size of the data used in the inference and / or training of the neural network, or some combination of these factors.

[0210] In at least one embodiment, the inference and / or training logic 2115 may include, but is not limited to, code and / or data storage 2105 for storing backpropagation and / or output weights and / or input / output data corresponding to neurons or layers of a neural network trained and / or used for inference in aspects of one or more embodiments. In at least one embodiment, the code and / or data storage 2105 stores weight parameters and / or input / output data for each layer of a neural network trained or used in conjunction with one or more embodiments during backpropagation of input / output data and / or weight parameters during training and / or inference in aspects of one or more embodiments. In at least one embodiment, the training logic 2115 may include or be coupled to the code and / or data storage 2105 to store graph code or other software to control timing and / or sequencing, wherein weight and / or other parameter information will be loaded to configure logic, including integer and / or floating-point units (collectively referred to as Arithmetic Logic Units (ALUs)).

[0211] In at least one embodiment, code (such as graph code) enables the loading of weights or other parameter information into the processor ALU based on the architecture of the neural network corresponding to such code. In at least one embodiment, any portion of the code and / or data storage 2105 may be included together with other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory. In at least one embodiment, any portion of the code and / or data storage 2105 may be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, the code and / or data storage 2105 may be cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other storage devices. In at least one embodiment, the choice of whether the code and / or data storage 2105 is internal or external to the processor, for example, or includes DRAM, SRAM, flash memory, or some other storage type, may depend on the available on-chip storage relative to off-chip storage, the latency requirements of the training and / or inference functions being performed, the batch size of the data used in the inference and / or training of the neural network, or some combination of these factors.

[0212] In at least one embodiment, code and / or data storage 2101 and code and / or data storage 2105 may be separate storage structures. In at least one embodiment, code and / or data storage 2101 and code and / or data storage 2105 may be combined storage structures. In at least one embodiment, code and / or data storage 2101 and code and / or data storage 2105 may be partially combined and partially separated. In at least one embodiment, any portion of code and / or data storage 2101 and code and / or data storage 2105 may be included together with other on-chip or off-chip data storage (including the processor's L1, L2, or L3 cache or system memory).

[0213] In at least one embodiment, the inference and / or training logic 2115 may include, but is not limited to, one or more arithmetic logic units (“ALUs”) 2110, including integer and / or floating-point units, for performing logical and / or mathematical operations at least in part based on or instructed by training and / or inference code (e.g., graph code), the results of which may produce activations (e.g., output values ​​from layers or neurons within a neural network) stored in activation storage 2120, which is a function of input / output and / or weight parameter data stored in code and / or data storage 2101 and / or code and / or data storage 2105. In at least one embodiment, activations stored in activation memory 2120 are generated based on linear algebra and / or matrix-based mathematics performed by ALU 2110 in response to execution instructions or other code, wherein weight values ​​stored in code and / or data storage 2105 and / or data storage 2101 are used as operands along with other values ​​(such as bias values, gradient information, momentum values, or other parameters or hyperparameters), any or all of which may be stored in code and / or data storage 2105 or code and / or data storage 2101 or in another memory on or off the chip.

[0214] In at least one embodiment, one or more ALUs 2110 are included within one or more processors or other hardware logic devices or circuits, while in another embodiment, one or more ALUs 2110 may be external to the processor or other hardware logic devices or circuits (e.g., coprocessors) that use them. In at least one embodiment, ALUs 2110 may be included within an execution unit of a processor or otherwise within an ALU library accessible by the execution unit of the processor, which may be within the same processor or distributed among different types of processors (e.g., central processing unit, graphics processing unit, fixed-function unit, etc.). In at least one embodiment, code and / or data storage 2101, code and / or data storage 2105, and activation storage 2120 may share a processor or other hardware logic device or circuit, while in another embodiment, they may be in different processors or other hardware logic devices or circuits, or in some combination of the same and different processors or other hardware logic devices or circuits. In at least one embodiment, any portion of the activation storage 2120 may be included together with other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory. Furthermore, inference and / or training code may be stored together with other code accessible to the processor or other hardware logic or circuitry and acquired and / or processed using the processor's fetch, decode, schedule, execute, retire, and / or other logic circuitry.

[0215] In at least one embodiment, the active memory 2120 may be a cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other storage device. In at least one embodiment, the active memory 2120 may be wholly or partially within or outside one or more processors or other logic circuits. In at least one embodiment, the choice of whether the active memory 2120 is inside or outside the processor, for example, or includes DRAM, SRAM, flash memory, or some other storage type, may depend on the available on-chip storage relative to off-chip storage, the latency requirements of the training and / or inference functions being performed, the batch size of the data used in the inference and / or training of the neural network, or some combination of these factors.

[0216] In at least one embodiment, Figure 21A The inference and / or training logic 2115 shown can be used in conjunction with an application-specific integrated circuit (“ASIC”), such as those from Google. Processing unit, from Graphcore TM Inference processing unit (IPU), or from Intel Corporation (For example, a "Lake Crest" processor. In at least one embodiment, Figure 21A The inference and / or training logic 2115 shown can be used in conjunction with central processing unit (“CPU”) hardware, graphics processing unit (“GPU”) hardware or other hardware such as field programmable gate array (“FPGA”)).

[0217] Figure 21B Inference and / or training logic 2115 according to at least one embodiment is illustrated. In at least one embodiment, the inference and / or training logic 2115 may include, but is not limited to, hardware logic in which computational resources are dedicated or otherwise exclusively used in conjunction with weight values ​​or other information corresponding to one or more neuron layers within a neural network. In at least one embodiment, Figure 21B The inference and / or training logic 2115 shown can be combined with an application-specific integrated circuit (ASIC) (such as those from Google). Processing unit, from Graphcore TM Inference processing unit (IPU), or from Intel Corporation (For example, "Lake Crest") processors are used. In at least one embodiment, Figure 21B The inference and / or training logic 2115 shown can be used in conjunction with central processing unit (CPU) hardware, graphics processing unit (GPU) hardware, or other hardware such as a field-programmable gate array (FPGA). In at least one embodiment, the inference and / or training logic 2115 includes, but is not limited to, code and / or data storage 2101 and code and / or data storage 2105, which can be used to store code (e.g., graph code), weight values, and / or other information, including bias values, gradient information, momentum values, and / or other parameter or hyperparameter information. Figure 21B In at least one embodiment described herein, each of code and / or data storage 2101 and code and / or data storage 2105 is associated with a dedicated computing resource (e.g., computing hardware 2102 and computing hardware 2106). In at least one embodiment, each of computing hardware 2102 and computing hardware 2106 includes one or more ALUs that perform mathematical functions (such as linear algebraic functions) on the information stored in code and / or data storage 2101 and code and / or data storage 2105, respectively, and the results are stored in active storage 2120.

[0218] In at least one embodiment, each code and / or data storage 2101 and 2105 and corresponding computing hardware 2102 and 2106 corresponds to a different layer of the neural network, such that the activation result from one storage / computation pair 2101 / 2102 of the code and / or data storage 2101 and computing hardware 2102 is provided as input to the next storage / computation pair 2105 / 2106 of the code and / or data storage 2105 and computing hardware 2106 to mirror the conceptual organization of the neural network. In at least one embodiment, each of the storage / computation pairs 2101 / 2102 and 2105 / 2106 may correspond to more than one neural network layer. In at least one embodiment, additional storage / computation pairs (not shown) following or parallel to the storage / computation pairs 2101 / 2102 and 2105 / 2106 may be included in the inference and / or training logic 2115.

[0219] Figure 22 Training and deployment of a deep neural network according to at least one embodiment are illustrated. In at least one embodiment, an untrained neural network 2206 is trained using a training dataset 2202. In at least one embodiment, the training framework 2204 is the PyTorch framework, while in other embodiments, the training framework 2204 is TensorFlow, Boost, Caffe, Microsoft Cognitive Toolkit / CNTK, MXNet, Chainer, Keras, Deeplearning4j, or other training frameworks. In at least one embodiment, the training framework 2204 trains the untrained neural network 2206 and enables it to be trained using the processing resources described herein to generate a trained neural network 2208. In at least one embodiment, the weights may be randomly selected or selected by pre-training using a deep belief network. In at least one embodiment, training may be performed in a supervised, partially supervised, or unsupervised manner.

[0220] In at least one embodiment, supervised learning is used to train an untrained neural network 2206, wherein the training dataset 2202 includes inputs paired with desired outputs for input, or wherein the training dataset 2202 includes inputs with known outputs, and the outputs of the neural network 2206 are manually graded. In at least one embodiment, the untrained neural network 2206 is trained in a supervised manner, and inputs from the training dataset 2202 are processed, and the resulting outputs are compared with a set of expected or desired outputs. In at least one embodiment, the error is then backpropagated through the untrained neural network 2206. In at least one embodiment, a training framework 2204 adjusts the weights controlling the untrained neural network 2206. In at least one embodiment, the training framework 2204 includes tools for monitoring how well the untrained neural network 2206 converges toward a model (such as a trained neural network 2208) adapted to generate correct answers (such as results 2214) based on input data (such as a new dataset 2212). In at least one embodiment, the training framework 2204 repeatedly trains the untrained neural network 2206 while using a loss function and tuning algorithms (such as stochastic gradient descent) to adjust the weights to refine the output of the untrained neural network 2206. In at least one embodiment, the training framework 2204 trains the untrained neural network 2206 until the untrained neural network 2206 achieves the desired accuracy. In at least one embodiment, the trained neural network 2208 can then be deployed to implement any number of machine learning operations.

[0221] In at least one embodiment, unsupervised learning is used to train an untrained neural network 2206, wherein the untrained neural network 2206 attempts to train itself using unlabeled data. In at least one embodiment, the unsupervised learning training dataset 2202 will include input data without any associated output data or "ground truth" data. In at least one embodiment, the untrained neural network 2206 can learn groupings within the training dataset 2202 and can determine how each input relates to the untrained dataset 2202. In at least one embodiment, unsupervised training can be used to generate self-organizing maps in the trained neural network 2208 capable of performing operations useful in reducing the dimensionality of the new dataset 2212. In at least one embodiment, unsupervised training can also be used to perform anomaly detection, which allows the identification of data points in the new dataset 2212 that deviate from the normal patterns of the new dataset 2212.

[0222] In at least one embodiment, semi-supervised learning can be used, which is a technique in which a mixture of labeled and unlabeled data is included in the training dataset 2202. In at least one embodiment, the training framework 2204 can be used to perform incremental learning, such as through transfer learning techniques. In at least one embodiment, incremental learning enables the trained neural network 2208 to adapt to a new dataset 2212 without forgetting the knowledge injected into the trained neural network 1408 during initial training.

[0223] 5G network

[0224] The following figures illustrate, but are not limited to, exemplary 5G-based systems that can be used to implement at least one embodiment.

[0225] Figure 23 An architecture of a system 2300 for a network according to at least one embodiment is illustrated. In at least one embodiment, system 2300 is shown as including user equipment (UE) 2302 and UE 2304. In at least one embodiment, UE 2302 and 2304 are shown as smartphones (e.g., handheld touchscreen mobile computing devices capable of connecting to one or more cellular networks), but may also include any mobile or non-mobile computing device, such as a personal digital assistant (PDA), pager, laptop computer, desktop computer, wireless handheld device, or any computing device including a wireless communication interface.

[0226] In at least one embodiment, either UE 2302 and UE 2304 may include an Internet of Things (IoT) UE, which may include a network access layer designed for low-power IoT applications utilizing short-lived UE connections. In at least one embodiment, the IoT UE may utilize technologies such as machine-to-machine (M2M) or machine-type communication (MTC) for exchanging data with an MTC server or device via a Public Land Mobile Network (PLMN), Proximity-Based Service (ProSe), or Device-to-Device (D2D) communication, sensor networks, or the IoT network. In at least one embodiment, the M2M or MTC data exchange may be machine-initiated data exchange. In at least one embodiment, the IoT network describes interconnected IoT UEs, which may include uniquely identifiable embedded computing devices (within the Internet infrastructure) with short-lived connections. In at least one embodiment, the IoT UE may execute background applications (e.g., keep-alive messages, state updates, etc.) to facilitate connectivity to the IoT network.

[0227] In at least one embodiment, UE 2302 and UE 2304 may be configured to connect (e.g., communicatively coupled) to a radio access network (RAN) 2316. In at least one embodiment, RAN 2316 may be, for example, an evolved Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access Network (E-UTRAN), a NextGen RAN (NG RAN), or some other type of RAN. In at least one embodiment, UE 2302 and UE 2304 utilize connection 2312 and connection 2314, respectively, each connection including a physical communication interface or layer. In at least one embodiment, connections 2312 and 2314 are shown as air interfaces for implementing communication coupling and may be consistent with cellular communication protocols such as the Global System for Mobile Communications (GSM) protocol, Code Division Multiple Access (CDMA) network protocol, Push-to-Talk (PTT) protocol, Cellular PTT (POC) protocol, Universal Mobile Telecommunications System (UMTS) protocol, 3GPP Long Term Evolution (LTE) protocol, 5G protocol, New Radio (NR) protocol, and variants thereof.

[0228] In at least one embodiment, UEs 2302 and 2304 may also directly exchange communication data via the ProSe interface 2306. In at least one embodiment, the ProSe interface 2306 may alternatively be referred to as a sidelink interface, which includes one or more logical channels, including but not limited to the Physical Sidelink Control Channel (PSCCH), Physical Sidelink Shared Channel (PSSCH), Physical Sidelink Discovery Channel (PSDCH), and Physical Sidelink Broadcast Channel (PSBCH).

[0229] In at least one embodiment, UE 2304 is shown configured to access access point (AP) 2310 via connection 2308. In at least one embodiment, connection 2308 may include a local wireless connection, such as a connection consistent with any IEEE 802.11 protocol, wherein AP 2310 will include Wireless Fidelity. Router. In at least one embodiment, AP 2310 is shown as connected to the Internet but not to the core network of a wireless system.

[0230] In at least one embodiment, RAN 2316 may include one or more access nodes enabling connectivity between 2312 and 2314. In at least one embodiment, these access nodes (ANs) may be referred to as base stations (BS), NodeBs, evolved NodeBs (eNBs), next-generation NodeBs (gNBs), RAN nodes, etc., and may include ground stations (e.g., ground access points) or satellite stations providing coverage within a geographic area (e.g., a cell). In at least one embodiment, RAN 2316 may include one or more RAN nodes (e.g., macro RAN node 2318) for providing macrocell coverage and one or more RAN nodes (e.g., low-power (LP) RAN node 2320) for providing femtocells or picocells (e.g., cells with smaller coverage areas, smaller user capacity, or higher bandwidth compared to macrocells).

[0231] In at least one embodiment, either RAN node 2318 or 2320 may terminate the air interface protocol and may be the first contact point for UEs 2302 and 2304. In at least one embodiment, either RAN node 2318 or 2320 may implement various logical functions of RAN 2316, including but not limited to Radio Network Controller (RNC) functions such as radio bearer management, uplink and downlink dynamic radio resource management, and data packet scheduling and mobility management.

[0232] In at least one embodiment, UE 2302 and UE 2304 may be configured to communicate with each other or with either RAN node 2318 and RAN node 2320 via a multi-carrier communication channel using orthogonal frequency division multiplexing (OFDM) communication signals, according to various communication technologies such as, but not limited to, orthogonal frequency division multiple access (OFDMA) communication technology (e.g., for downlink communication) or single-carrier frequency division multiple access (SC-FDMA) communication technology (e.g., for uplink and ProSe or sidelink communication), and / or variations thereof. In at least one embodiment, the OFDM signal may include multiple orthogonal subcarriers.

[0233] In at least one embodiment, the downlink resource grid can be used for downlink transmissions from either RAN nodes 2318 and 2320 to UEs 2302 and 2304, while uplink transmissions can utilize similar techniques. In at least one embodiment, the grid can be a time-frequency grid, referred to as a resource grid or time-frequency resource grid, which represents the physical resources in the downlink within each time slot. In at least one embodiment, this time-frequency plane representation is a common practice in OFDM systems, making it intuitive for radio resource allocation. In at least one embodiment, each column and each row of the resource grid corresponds to an OFDM symbol and an OFDM subcarrier, respectively. In at least one embodiment, the duration of the resource grid in the time domain corresponds to a time slot in a radio frame. In at least one embodiment, the minimum time-frequency unit in the resource grid is represented as a resource element. In at least one embodiment, each resource grid comprises multiple resource blocks that describe the mapping of certain physical channels to resource elements. In at least one embodiment, each resource block comprises a set of resource elements. In at least one embodiment, in the frequency domain, this can represent the minimum number of resources that can currently be allocated. In at least one embodiment, there are several different physical downlink channels that use such resource blocks for transmission.

[0234] In at least one embodiment, the Physical Downlink Shared Channel (PDSCH) can carry user data and higher-layer signaling to UEs 2302 and 2304. In at least one embodiment, the Physical Downlink Control Channel (PDCCH) can carry information such as transmission format and resource allocation related to the PDSCH channel. In at least one embodiment, it can also notify UEs 2302 and 2304 of transmission format, resource allocation, and HARQ (Hybrid Automatic Repeat Request) information related to the uplink shared channel. In at least one embodiment, typically, downlink scheduling (allocating control and shared channel resource blocks to UE 2302 within the cell) can be performed at either RAN nodes 2318 and 2320 based on channel quality information fed back from either UE 2302 or 2304. In at least one embodiment, downlink resource allocation information can be transmitted on the PDCCH used for (e.g., allocated to) each of UEs 2302 and 2304.

[0235] In at least one embodiment, the PDCCH can use Control Channel Elements (CCEs) to transmit control information. In at least one embodiment, PDCCH complex-valued symbols can first be organized into quadruplets before being mapped to resource elements, and then permuted using a sub-block interleaver for rate matching. In at least one embodiment, one or more of these CCEs can be used to transmit each PDCCH, where each CCE can correspond to nine sets of four physical resource elements referred to as resource element groups (REGs). In at least one embodiment, four Quadrature Phase Shift Keying (QPSK) symbols can be mapped to each REG. In at least one embodiment, depending on the size of the downlink control information (DCI) and channel conditions, one or more CCEs can be used to transmit the PDCCH. In at least one embodiment, there can be four or more different PDCCH formats (e.g., aggregation levels, L = 1, 2, 4, or 8) with different numbers of CCEs as defined in LTE.

[0236] In at least one embodiment, the Enhanced Physical Downlink Control Channel (EPDCCH) using PDSCH resources can be used for control information transmission. In at least one embodiment, one or more Enhanced Control Channel Elements (ECCEs) can be used to transmit the EPDCCH. In at least one embodiment, each ECCE can correspond to nine sets of four physical resource elements referred to as Enhanced Resource Element Groups (EREGs). In at least one embodiment, the ECCE can have a different number of EREGs in some cases.

[0237] In at least one embodiment, RAN 2316 is shown communicatively coupled to core network (CN) 2338 via S1 interface 2322. In at least one embodiment, CN 2338 may be an evolved packet core (EPC) network, a NextGen packet core (NPC) network, or some other type of CN. In at least one embodiment, S1 interface 2322 is divided into two parts: S1-U interface 2326, which carries service data between RAN nodes 2318 and 2320 and serving gateway (S-GW) 2330; and S1-Mobility Management Entity (MME) interface 2324, which is the signaling interface between RAN nodes 2318 and 2320 and MME 2328.

[0238] In at least one embodiment, CN 2338 includes MME 2328, S-GW 2330, Packet Data Network (PDN) Gateway (P-GW) 2334, and Home Subscriber Server (HSS) 2332. In at least one embodiment, MME 2328 may functionally resemble the control plane of a legacy General Packet Radio Service (GPRS) Support Node (SGSN). In at least one embodiment, MME 2328 may manage mobility aspects of access, such as gateway selection and tracking area list management. In at least one embodiment, HSS 2332 may include a database for network users, including subscription-related information to support network entities in handling communication sessions. In at least one embodiment, CN 2338 may include one or more HSS 2332s, depending on the number of mobile users, device capacity, network organization, etc. In at least one embodiment, HSS 2332 may provide support for routing / roaming, authentication, authorization, naming / addressing resolution, location dependencies, etc.

[0239] In at least one embodiment, the S-GW 2330 can terminate the S1 interface 2322 toward the RAN 2316 and route data packets between the RAN 2316 and the CN 2338. In at least one embodiment, the S-GW 2330 can be a local mobility anchor point for inter-RAN node handover and can also provide an anchor point for inter-3GPP mobility. In at least one embodiment, other responsibilities may include lawful interception, charging, and some policy enforcement.

[0240] In at least one embodiment, P-GW 2334 can terminate the SGi interface toward the PDN. In at least one embodiment, P-GW 2334 can route data packets between EPC network 2338 and external networks (such as networks including application server 2340 (or application function (AF))) via Internet Protocol (IP) interface 2342. In at least one embodiment, application server 2340 can be an element that provides applications using IP bearer resources using a core network (e.g., UMTS Packet Service (PS) domain, LTE PS data service, etc.). In at least one embodiment, P-GW 2334 is shown communicatively coupled to application server 2340 via IP communication interface 2342. In at least one embodiment, application server 2340 can also be configured to support one or more communication services (e.g., Voice over Internet Protocol (VoIP) sessions, PTT sessions, group communication sessions, social networking services, etc.) of UEs 2302 and 2304 via CN 2338.

[0241] In at least one embodiment, P-GW 2334 may also be a node for policy enforcement and charging data collection. In at least one embodiment, Policy and Charging Enforcement Function (PCRF) 2336 is the policy and charging control element of CN 2338. In at least one embodiment, in a non-roaming scenario, a single PCRF may exist in the Home Public Land Mobile Network (HPLMN) associated with the UE's Internet Protocol Connectivity Access Network (IP-CAN) session. In at least one embodiment, in a roaming scenario with local traffic breaches, two PCRFs may exist associated with the UE's IP-CAN session: the Home PCRF (H-PCRF) within the HPLMMN and the Visited PCRF (V-PCRF) within the Visited Public Land Mobile Network (VPLMN). In at least one embodiment, PCRF 2336 may be communicatively coupled to application server 2340 via P-GW 2334. In at least one embodiment, application server 2340 may signal PCRF 2336 to indicate new service flows and select appropriate Quality of Service (QoS) and charging parameters. In at least one embodiment, PCRF 2336 can supply this rule to a Policy and Charging Enforcement Function (PCEF) (not shown) with an appropriate Service Flow Template (TFT) and identifier for a QoS Class (QCI), which is initiated by the QoS and charging specified by the application server 2340.

[0242] Figure 24 The architecture of a system 2400 of a network according to some embodiments is shown. In at least one embodiment, the system 2400 is shown to include a UE 2402, a 5G access node or RAN node (shown as (R)AN node 2408), a user plane function (shown as UPF 2404), a data network (DN 2406), which may be, for example, an operator service, an Internet access or a third-party service, and a 5G core network (5GC) (shown as CN 2410).

[0243] In at least one embodiment, CN 2410 includes authentication server functionality (AUSF 2414); core access and mobility management functionality (AMF 2412); session management functionality (SMF 2418); network exposure functionality (NEF 2416); policy control functionality (PCF 2422); network function (NF) repository functionality (NRF 2420); unified data management (UDM 2424); and application functionality (AF 2426). In at least one embodiment, CN 2410 may also include other elements not shown, such as structured data storage network functionality (SDSF), unstructured data storage network functionality (UDSF), and variations thereof.

[0244] In at least one embodiment, the UPF 2404 may act as an anchor point for mobility within and between RATs, an external PDU session point interconnected to the DN2406, and a branch point supporting multi-homed PDU sessions. In at least one embodiment, the UPF 2404 may also perform packet routing and forwarding, packet inspection, user plane portion of policy rule enforcement, lawful packet interception (UP collection), service usage reporting, QoS processing for the user plane (e.g., packet filtering, gating, UL / DL rate enforcement), uplink service verification (e.g., SDF-to-QoS flow mapping), transport-level packet marking in uplink and downlink, and downlink packet buffering and downlink data notification triggering. In at least one embodiment, the UPF 2404 may include an uplink classifier to support routing service flows to the data network. In at least one embodiment, the DN 2406 may represent various network operator services, Internet access, or third-party services.

[0245] In at least one embodiment, AUSF 2414 can store data for authentication of UE 2402 and handle authentication-related functions. In at least one embodiment, AUSF 2414 can facilitate a common authentication framework for various access types.

[0246] In at least one embodiment, AMF 2412 can be responsible for registration management (e.g., for registering UE 2402, etc.), connection management, reachability management, mobility management, and lawful interception of AMF-related events, as well as access authentication and authorization. In at least one embodiment, AMF 2412 can provide SM message transmission for SMF 2418 and act as a transparent proxy for routing SM messages. In at least one embodiment, AMF 2412 can also provide UE 2402 with SMS functionality (SMSF) (…). Figure 24 Transmission of Short Message Service (SMS) messages between (not shown). In at least one embodiment, AMF 2412 may act as a Security Anchoring Function (SEA), which may include interaction with AUSF 2414 and UE 2402 and receiving an intermediate key established as a result of the UE 2402 authentication process. In at least one embodiment, in the case of using USIM-based authentication, AMF 2412 may retrieve security material from AUSF 2414. In at least one embodiment, AMF 2412 may also include a Security Context Management (SCM) function, which receives from the SEA a key it uses to derive the access network-specific key. Furthermore, in at least one embodiment, AMF 2412 may be the termination point (N2 reference point) of the RAN CP interface, the termination point of NAS (NI) signaling, and perform NAS encryption and integrity protection.

[0247] In at least one embodiment, AMF 2412 can also support NAS signaling with UE 2402 via the N3 Interoperability Function (IWF) interface. In at least one embodiment, the N3IWF can be used to provide access to untrusted entities. In at least one embodiment, the N3IWF can be the termination point of the N2 and N3 interfaces of the control plane and user plane, respectively, thus enabling the processing of N2 signaling from the SMF and AMF for PDU sessions and QoS, encapsulation / decapsulation of IPSec and N3 tunnel packets, marking N3 user plane packets in the uplink, and implementing QoS corresponding to the N3 packet marking, taking into account the QoS requirements associated with such marking received via N2. In at least one embodiment, the N3IWF can also relay uplink and downlink control plane NAS (NI) signaling between UE 2402 and AMF 2412, and relay uplink and downlink user plane packets between UE 2402 and UPF 2404. In at least one embodiment, the N3IWF also provides a mechanism for establishing an IPsec tunnel with the UE 2402.

[0248] In at least one embodiment, the SMF 2418 may be responsible for session management (e.g., session establishment, modification, and release, including tunnel maintenance between the UPF and AN nodes); UE IP address allocation and management (including optional authorization); selection and control of UP functions; configuring traffic redirection at the UPF to route traffic to the appropriate destination; interface termination towards policy control functions; policy enforcement and QoS control portions; lawful interception (for SM events and interfaces to the LI system); termination of the SM portion of NAS messages; downlink data notification; initiator of AN-specific SM information, which is sent to the AN via the AMF on N2; and determination of the SSC mode of the session. In at least one embodiment, the SMF 2418 may include the following roaming functions: handling local implementation to apply QoS SLAB (VPLMN); charge data collection and charge interface (VPLMN); lawful interception (for SM events in the VPLMN and interface to the LI system); and supporting interaction with external DNs to transmit signaling for PDU session authorization / authentication performed by the external DNs.

[0249] In at least one embodiment, NEF 2416 can provide means for securely exposing services and capabilities provided to third parties by 3GPP network functions, internal exposure / re-exposure, application functions (e.g., AF 2426), edge computing or fog computing systems, etc. In at least one embodiment, NEF 2416 can authenticate, authorize, and / or throttle AFs. In at least one embodiment, NEF 2416 can also translate information exchanged with AF 2426 and information exchanged with internal network functions. In at least one embodiment, NEF 2416 can translate between AF service identifiers and internal 5GC information. In at least one embodiment, NEF 2416 can also receive information from other network functions (NFs) based on the ability to expose other network functions. In at least one embodiment, this information can be stored as structured data at NEF 2416 or stored at a data storage NF using a standardized interface. In at least one embodiment, the stored information can then be re-exposed by NEF 2416 to other NFs and AFs, and / or used for other purposes, such as analysis.

[0250] In at least one embodiment, the NRF 2420 may support service discovery functionality, receiving NF discovery requests from NF instances and providing information about discovered NF instances to the NF instances. In at least one embodiment, the NRF 2420 also maintains information about available NF instances and the services they support.

[0251] In at least one embodiment, the PCF 2422 may provide policy rules to control plane functions for enforcement, and may also support a unified policy framework for managing network behavior. In at least one embodiment, the PCF 2422 may also implement a front-end (FE) for accessing subscription information related to policy decisions in the UDR of the UDM 2424.

[0252] In at least one embodiment, UDM 2424 can process subscription-related information to support network entities in handling communication sessions and can store subscription data of UE 2402. In at least one embodiment, UDM 2424 can include two parts: an application FE and a user data repository (UDR). In at least one embodiment, UDM can include a UDM FE responsible for handling credentials, location management, subscription management, etc. In at least one embodiment, several different front-ends can serve the same user in different transactions. In at least one embodiment, UDM-FE accesses sub-subscription information stored in UDR and performs authentication credential processing; user identification processing; access authorization; registration / mobility management; and subscription management. In at least one embodiment, UDR can interact with PCF 2422. In at least one embodiment, UDM 2424 can also support SMS management, where SMS-FE implements similar application logic as described above.

[0253] In at least one embodiment, AF 2426 can provide application effects on service routing, access to Network Capability Exposure (NCE), and interaction with a policy framework for policy control. In at least one embodiment, NCE can be a mechanism allowing 5GC and AF 2426 to provide information to each other via NEF 2416, which can be used for edge computing implementations. In at least one embodiment, network operators and third-party services can be hosted near the attached access point of UE 2402 to achieve efficient service delivery by reducing end-to-end latency and load on the transport network. In at least one embodiment, for edge computing implementations, 5GC can select UPF 2404 close to UE 2402 and perform service bootstrapping from UPF 2404 to DN 2406 via the N6 interface. In at least one embodiment, this can be based on UE subscription data, UE location, and information provided by AF 2426. In at least one embodiment, AF 2426 can influence UPF (re)selection and service routing. In at least one embodiment, based on operator deployment, when AF 2426 is considered a trusted entity, the network operator may allow AF 2426 to interact directly with the relevant NF.

[0254] In at least one embodiment, CN 2410 may include an SMSF, which may be responsible for SMS subscription checks and authentication, and relay SM messages to / from UE 2402 to / from other entities, such as SMS-GMSC / IWMSC / SMS routers. In at least one embodiment, SMS may also interact with AMF 2412 and UDM 2424 for a notification process that UE 2402 is available for SMS delivery (e.g., setting a UE unreachable flag and notifying UDM 2424 when UE 2402 is available for SMS).

[0255] In at least one embodiment, system 2400 may include the following service-based interfaces: Namf: a service-based interface presented by AMF; Nsmf: a service-based interface presented by SMF; Nnef: a service-based interface presented by NEF; Npcf: a service-based interface presented by PCF; Nudm: a service-based interface presented by UDM; Naf: a service-based interface presented by AF; Nnrf: a service-based interface presented by NRF; and Nausf: a service-based interface presented by AUSF.

[0256] In at least one embodiment, system 2400 may include the following reference points: N1: a reference point between the UE and the AMF; N2: a reference point between the (R)AN and the AMF; N3: a reference point between the (R)AN and the UPF; N4: a reference point between the SMF and the UPF; and N6: a reference point between the UPF and the data network. In at least one embodiment, there may be more reference points and / or service-based interfaces between NF services in the NF; however, for clarity, these interfaces and reference points have been omitted. In at least one embodiment, the NS reference point may be between the PCF and the AF; the N7 reference point may be between the PCF and the SMF; the N11 reference point may be between the AMF and the SMF, and so on. In at least one embodiment, CN 2410 may include an Nx interface, which is an inter-CN interface between the MME and the AMF 2412 to enable interoperability between CN 2410 and CN 7224.

[0257] In at least one embodiment, system 2400 may include a plurality of RAN nodes (such as (R)AN nodes 2408), wherein an Xn interface is defined between two or more (R)AN nodes 2408 (e.g., gNBs) connected to 5GC 410, between (R)AN nodes 2408 (e.g., gNBs) and eNBs (e.g., macro RAN nodes) connected to CN 2410, and / or between two eNBs connected to CN 2410.

[0258] In at least one embodiment, the Xn interface may include an Xn user plane (Xn-U) interface and an Xn control plane (Xn-C) interface. In at least one embodiment, Xn-U may provide unguaranteed delivery of user plane PDUs and support / provide data forwarding and flow control functions. In at least one embodiment, Xn-C may provide management and error handling functions, functions for managing the Xn-C interface, and mobility support for UE 2402 in connected mode (e.g., CM-CONNECTED), including functions for managing UE mobility for connected modes between one or more (R)AN nodes 2408. In at least one embodiment, mobility support may include context transfer from the old (source) serving (R)AN node 2408 to the new (target) serving (R)AN node 2408; and control of user plane tunneling between the old (source) serving (R)AN node 2408 and the new (target) serving (R)AN node 2408.

[0259] In at least one embodiment, the Xn-U protocol stack may include a transport network layer built on top of the Internet Protocol (IP) transport layer and a GTP-U layer on top of UDP and / or one or more IP layers for carrying user plane PDUs. In at least one embodiment, the Xn-C protocol stack may include an application layer signaling protocol (referred to as the Xn Application Protocol (Xn-AP)) and a transport network layer built on top of the SCTP layer. In at least one embodiment, the SCTP layer may be on top of the IP layers. In at least one embodiment, the SCTP layer provides guaranteed delivery of application layer messages. In at least one embodiment, point-to-point transmission is used to deliver signaling PDUs in the transport IP layer. In at least one embodiment, the Xn-U protocol stack and / or the Xn-C protocol stack may be the same as or similar to the user plane and / or control plane protocol stacks shown and described herein.

[0260] Figure 25 This is an illustration of a control plane protocol stack according to some embodiments. In at least one embodiment, control plane 2500 is shown as a communication protocol stack between UE 2302 (or alternatively, UE 2304), RAN 2316, and MME 2328.

[0261] In at least one embodiment, PHY layer 2502 can transmit or receive information used by MAC layer 2504 through one or more air interfaces. In at least one embodiment, PHY layer 2502 can also perform link adaptive or adaptive modulation and coding (AMC), power control, cell search (e.g., for initial synchronization and handover purposes), and other measurements used by higher layers (e.g., RRC layer 2510). In at least one embodiment, PHY layer 2502 can further perform error detection, forward error correction (FEC) encoding / decoding of the transport channel, modulation / demodulation of the physical channel, interleaving, rate matching, mapping to the physical channel, and multiple-input multiple-output (MIMO) antenna processing.

[0262] In at least one embodiment, MAC layer 2504 can perform mapping between logical channels and transport channels, multiplexing MAC service data units (SDUs) from one or more logical channels onto a transport block (TB) to be delivered to the PHY via the transport channel, demultiplexing MAC SDUs from the transport block (TB) delivered from the PHY via the transport channel onto one or more logical channels, multiplexing MAC SDUs onto the TB, scheduling information reporting, error correction via hybrid automatic repeat request (HARD), and logical channel prioritization.

[0263] In at least one embodiment, the RLC layer 2506 can operate in multiple operating modes, including: Transparent Mode (TM), Unacknowledged Mode (UM), and Acknowledged Mode (AM). In at least one embodiment, the RLC layer 2506 can perform the transmission of upper-layer protocol data units (PDUs), error correction via Automatic Repeat Request (ARQ) for AM data transmission, and the concatenation, segmentation, and reassembly of RLC SDUs for UM and AM data transmission. In at least one embodiment, the RLC layer 2506 can also perform resegmentation of RLC data PDUs for AM data transmission, reordering of RLC data PDUs for UM and AM data transmission, detection of duplicate data for UM and AM data transmission, discarding of RLC SDUs for UM and AM data transmission, detection of protocol errors in AM data transmission, and performance of RLC reconstruction.

[0264] In at least one embodiment, the PDCP layer 2508 can perform header compression and decompression of IP data, maintain PDCP sequence numbers (SNs), perform intra-sequence delivery of higher-layer PDUs when reconstructing lower layers, eliminate duplication of lower-layer SDUs when reconstructing lower layers for radio bearers mapped on RLC AM, encrypt and decrypt control plane data, perform integrity protection and integrity verification of control plane data, discard data based on control timers, and perform security operations (e.g., encryption, decryption, integrity protection, integrity verification, etc.).

[0265] In at least one embodiment, the main services and functions of the RRC layer 2510 may include broadcasting system information (e.g., included in a Master Information Block (MIB) or System Information Block (SIB) associated with the Non-Access Stratum (NAS), broadcasting system information associated with the Access Stratum (AS), paging, establishment, maintenance, and release of RRC connections between the UE and the E-UTRAN (e.g., RRC connection paging, RRC connection establishment, RRC connection modification, and RRC connection release), establishment, configuration, maintenance, and release of point-to-point radio bearers, security functions including key management, inter-Radio Access Technology (RAT) mobility, and measurement configuration for UE measurement reporting. In at least one embodiment, the MIB and SIB may include one or more Information Elements (IEs), each of which may include a separate data field or data structure.

[0266] In at least one embodiment, UE 2302 and RAN 2316 may use a Uu interface (e.g., LTE-Uu interface) to exchange control plane data via a protocol stack including PHY layer 2502, MAC layer 2504, RLC layer 2506, PDCP layer 2508 and RRC layer 2510.

[0267] In at least one embodiment, a Non-Access Stratum (NAS) protocol (NAS protocol 2512) forms the highest layer of the control plane between UE 2302 and MME 2328. In at least one embodiment, NAS protocol 2512 supports the mobility and session management procedures of UE 2302 to establish and maintain an IP connection between UE 2302 and P-GW 2334.

[0268] In at least one embodiment, the Si Application Protocol (Si-AP) layer (Si-AP layer 2522) may support the functionality of the Si interface and include basic procedures (EP). In at least one embodiment, the EP is the interaction unit between RAN 2316 and CN 2328. In at least one embodiment, Si-AP layer services may include two groups: UE-associated services and non-UE-associated services. In at least one embodiment, these services perform functions, including but not limited to: E-UTRAN Radio Access Bearer (E-RAB) management, UE capability indication, mobility, NAS signaling transmission, RAN Information Management (RIM), and configuration transfer.

[0269] In at least one embodiment, the Flow Control Transport Protocol (SCTP) layer (or alternatively, the Flow Control Transport Protocol / Internet Protocol (SCTP / IP) layer) (SCTP layer 2520) may be partially based on the IP protocol supported by IP layer 2518 to ensure reliable delivery of signaling messages between RAN 2316 and MME 2328. In at least one embodiment, L2 layer 2516 and L1 layer 2514 may refer to the communication links (e.g., wired or wireless) used by the RAN node and MME to exchange information.

[0270] In at least one embodiment, RAN 2316 and one or more MMEs 2328 can utilize the S1-MME interface to exchange control plane data via a protocol stack including L1 layer 2514, L2 layer 2516, IP layer 2518, SCTP layer 2520 and Si-AP layer 2522.

[0271] Figure 26 This is an illustration of a user plane protocol stack according to at least one embodiment. In at least one embodiment, user plane 2600 is shown as a communication protocol stack between UE 2302, RAN 2316, S-GW 2330, and P-GW 2334. In at least one embodiment, user plane 2600 may utilize the same protocol layer as control plane 2500. In at least one embodiment, UE 2302 and RAN 2316 may utilize a Uu interface (e.g., LTE-Uu interface) to exchange user plane data via a protocol stack including PHY layer 2502, MAC layer 2504, RLC layer 2506, and PDCP layer 2508.

[0272] In at least one embodiment, the General Packet Radio Service (GPRS) Tunneling Protocol (GTP-U) layer (GTP-U layer 2604) for the user plane can be used to carry user data within the GPRS core network and between the radio access network and the core network. In at least one embodiment, the transmitted user data can be packets of any format, such as IPv4, IPv6, or PPP. In at least one embodiment, the UDP and IP Security (UDP / IP) layer (UDP / IP layer 2602) can provide checksums for data integrity, port numbers for addressing different functions at the source and destination, and encryption and authentication of selected data streams. In at least one embodiment, the RAN 2316 and S-GW 2330 can utilize the S1-U interface to exchange user plane data via a protocol stack including L1 layer 2514, L2 layer 2516, UDP / IP layer 2602, and GTP-U layer 2604. In at least one embodiment, the S-GW 2330 and P-GW 2334 can utilize the S5 / S8a interface to exchange user plane data via a protocol stack including L1 layer 2514, L2 layer 2516, UDP / IP layer 2602, and GTP-U layer 2604. In at least one embodiment, as described above... Figure 25 The NAS protocol discussed here supports the mobility and session management process of UE 2302 to establish and maintain an IP connection between UE 2302 and P-GW 2334.

[0273] Figure 27 A component 2700 of a core network according to at least one embodiment is illustrated. In at least one embodiment, components of CN 2038 may be implemented in a physical node or a separate physical node, the separate physical node including components for reading and executing instructions from a machine-readable medium or a computer-readable medium (e.g., a non-transitory machine-readable storage medium). In at least one embodiment, network function virtualization (NFV) is used to virtualize any or all of the aforementioned network node functions via executable instructions stored in one or more computer-readable storage media (described further in detail below). In at least one embodiment, a logical instantiation of CN 2338 may be referred to as network slice 2702 (e.g., network slice 2702 is shown as including HSS 2332, MME 2328, and S-GW 2330). In at least one embodiment, a logical instantiation of a portion of CN 2338 may be referred to as network subslice 2704 (e.g., network subslice 2704 is shown as including P-GW 2334 and PCRF 2336).

[0274] In at least one embodiment, the NFV architecture and infrastructure can be used to virtualize one or more network functions onto physical resources comprising a combination of industry-standard server hardware, storage hardware, or switches, which may alternatively be performed by dedicated hardware. In at least one embodiment, the NFV system can be used to perform virtual or reconfigurable implementations of one or more EPC components / functions.

[0275] Figure 28 This is a block diagram illustrating the components of a system 2800 for supporting Network Functions Virtualization (NFV) according to at least one embodiment. In at least one embodiment, the system 2800 is shown to include a virtualization infrastructure manager (shown as VIM 2802), a network functions virtualization infrastructure (as shown as NFVI 2804), a VNF manager (as shown as VNFM 2806), virtualized network functions (shown as VNF 2808), a component manager (shown as EM 2810), an NFV coordinator (shown as NFVO 2812), and a network manager (shown as NM 2814).

[0276] In at least one embodiment, VIM 2802 manages the resources of NFVI 2804. In at least one embodiment, NFVI 2804 may include physical or virtual resources and applications (including hypervisors) for executing system 2800. In at least one embodiment, VIM 2802 may utilize NFVI 2804 to manage the lifecycle of virtual resources (e.g., the creation, maintenance, and teardown of virtual machines (VMs) associated with one or more physical resources), track VM instances, track performance, fault and security of VM instances and associated physical resources, and expose VM instances and associated physical resources to other management systems.

[0277] In at least one embodiment, VNFM 2806 can manage VNF 2808. In at least one embodiment, VNF 2808 can be used to perform EPC components / functions. In at least one embodiment, VNFM 2806 can manage the lifecycle of VNF 2808 and track the performance, faults, and security of the virtual aspects of VNF 2808. In at least one embodiment, EM 2810 can track the performance, faults, and security of the functional aspects of VNF 2808. In at least one embodiment, tracking data from VNFM 2806 and EM 2810 may include, in at least one embodiment, performance measurement (PM) data used by VIM 2802 or NFVI 2804. In at least one embodiment, both VNFM 2806 and EM 2810 can scale up / down the number of VNFs in system 2800.

[0278] In at least one embodiment, NFVO 2812 can coordinate, authorize, release, and occupy resources of NFVI 2804 to provide requested services (e.g., to perform EPC functions, components, or slices). In at least one embodiment, NM 2814 can provide an end-user function package responsible for managing a network that may include network elements having VNFs, non-virtualized network functions, or both (management of VNFs may occur via EM 2810).

[0279] Computer-based systems

[0280] The following figures present, but are not limited to, exemplary computer-based systems that can be used to implement at least one embodiment.

[0281] Figure 29 A processing system 2900 according to at least one embodiment is illustrated. In at least one embodiment, the system 2900 includes one or more processors 2902 and one or more graphics processors 2908, and may be a single-processor desktop system, a multi-processor workstation system, or a server system having a large number of processors 2902 or processor cores 2907. In at least one embodiment, the processing system 2900 is a processing platform incorporated within a system-on-a-chip (SoC) integrated circuit for use in mobile, handheld, or embedded devices.

[0282] In at least one embodiment, the processing system 2900 may include or be integrated into a server-based gaming platform, including a game console, mobile game console, handheld game console, or online game console, which are game and media consoles. In at least one embodiment, the processing system 2900 is a mobile phone, smartphone, tablet computing device, or mobile internet device. In at least one embodiment, the processing system 2900 may also include components coupled to or integrated into a wearable device, such as a smartwatch wearable device, smart glasses device, augmented reality device, or virtual reality device. In at least one embodiment, the processing system 2900 is a television or set-top box device having one or more processors 2902 and a graphical interface generated by one or more graphics processors 2908.

[0283] In at least one embodiment, each of the one or more processors 2902 includes one or more processor cores 2907 to process instructions that, when executed, perform operations against the system and user software. In at least one embodiment, each of the one or more processor cores 2907 is configured to process a particular instruction set 2909. In at least one embodiment, the instruction set 2909 may facilitate Complex Instruction Set Computing (CISC), Reduced Instruction Set Computing (RISC), or computation via Very Long Instruction Word (VLIW). In at least one embodiment, the plurality of processor cores 2907 may each process a different instruction set 2909, which may include instructions that facilitate emulation of other instruction sets. In at least one embodiment, the processor cores 2907 may also include other processing devices, such as digital signal processors (DSPs).

[0284] In at least one embodiment, processor 2902 includes cache memory 2904. In at least one embodiment, processor 2902 may have a single internal cache or multiple levels of internal caches. In at least one embodiment, the cache memory is shared among various components of processor 2902. In at least one embodiment, processor 2902 also uses an external cache (e.g., a Level 3 (L3) cache or a last-level cache (LLC)) (not shown), which can be shared among processor cores 2907 using known cache coherence techniques. In at least one embodiment, processor 2902 further includes a register file 2906, which may include different types of registers for storing different types of data (e.g., integer registers, floating-point registers, status registers, and instruction pointer registers). In at least one embodiment, register file 2906 may include general-purpose registers or other registers.

[0285] In at least one embodiment, one or more processors 2902 are coupled to one or more interface buses 2910 to transmit communication signals, such as address, data, or control signals, between the processors 2902 and other components in the system 2900. In at least one embodiment, the interface bus 2910 may be a processor bus, such as a version of the Direct Media Interface (DMI) bus. In at least one embodiment, the interface bus 2910 is not limited to the DMI bus and may include one or more peripheral component interconnect buses (e.g., PCI, PCI Express), memory buses, or other types of interface buses. In at least one embodiment, the processor 2902 includes an integrated memory controller 2916 and a platform controller hub 2930. In at least one embodiment, the memory controller 2916 facilitates communication between storage devices and other components of the processing system 2900, while the platform controller hub (PCH) 2930 provides connectivity to input / output (I / O) devices via a local I / O bus.

[0286] In at least one embodiment, memory device 2920 may be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, a flash memory device, a phase-change memory device, or a device with suitable performance for use as processor memory. In at least one embodiment, memory device 2920 may be used as system memory of processing system 2900 to store data 2922 and instructions 2921 for use when one or more processors 2902 execute an application or process. In at least one embodiment, memory controller 2916 is also coupled to an optional external graphics processor 2912, which may communicate with one or more graphics processors 2908 of processor 2902 to perform graph and media operations. In at least one embodiment, display device 2911 may be connected to processor 2902. In at least one embodiment, display device 2911 may include one or more internal display devices, such as those in mobile electronic devices or portable computer devices, or external display devices connected via a display interface (e.g., DisplayPort). In at least one embodiment, the display device 2911 may include a head-mounted display (HMD), such as a stereoscopic display device for virtual reality (VR) or augmented reality (AR) applications.

[0287] In at least one embodiment, the platform controller hub 2930 enables peripheral devices to connect to the storage device 2920 and the processor 2902 via a high-speed I / O bus. In at least one embodiment, the I / O peripheral devices include, but are not limited to, an audio controller 2946, a network controller 2934, a firmware interface 2928, a wireless transceiver 2926, a touch sensor 2925, and a data storage device 2924 (e.g., a hard disk drive, flash memory, etc.). In at least one embodiment, the data storage device 2924 may be connected via a memory interface (e.g., SATA) or via a peripheral bus, such as a peripheral component interconnect bus (e.g., PCI, PCIe). In at least one embodiment, the touch sensor 2925 may include a touchscreen sensor, a pressure sensor, or a fingerprint sensor. In at least one embodiment, the wireless transceiver 2926 may be a Wi-Fi transceiver, a Bluetooth transceiver, or a mobile network transceiver, such as a 3G, 4G, or LTE transceiver. In at least one embodiment, the firmware interface 2928 enables communication with the system firmware, and may be, for example, a Unified Extensible Firmware Interface (UEFI). In at least one embodiment, network controller 2934 may enable network connectivity to a wired network. In at least one embodiment, a high-performance network controller (not shown) is coupled to interface bus 2910. In at least one embodiment, audio controller 2946 is a multi-channel high-definition audio controller. In at least one embodiment, processing system 2900 includes an optional legacy I / O controller 2940 for coupling legacy (e.g., Personal System 2 (PS / 2)) devices to processing system 2900. In at least one embodiment, platform controller hub 2930 may also be connected to one or more Universal Serial Bus (USB) controllers 2942 that connect input devices, such as a keyboard and mouse combination 2943, a camera 2944, or other USB input devices.

[0288] In at least one embodiment, instances of the memory controller 2916 and platform controller hub 2930 may be integrated into a discrete external graphics processor, such as external graphics processor 2912. In at least one embodiment, the platform controller hub 2930 and / or the memory controller 2916 may be external to one or more processors 2902. For example, in at least one embodiment, the processing system 2900 may include an external memory controller 2916 and a platform controller hub 2930, which may be configured as a memory controller hub and a peripheral controller hub in a system chipset communicating with the processor 2902.

[0289] Figure 30A computer system 3000 according to at least one embodiment is illustrated. In at least one embodiment, the computer system 3000 may be a system having interconnected devices and components, a System-on-a-Chip (SoC), or some combination thereof. In at least one embodiment, the computer system 3000 is formed by a processor 3002, which may include execution units for executing instructions. In at least one embodiment, the computer system 3000 may include, but is not limited to, components such as the processor 3002, which employs execution units including logic to execute algorithms for process data. In at least one embodiment, the computer system 3000 may include a processor, such as one available from Intel Corporation of Santa Clara, California. Processor family, Xeon™ XScale™ and / or StrongARM™ Core TM or Nervana TM A microprocessor may be used, although other systems (including PCs, engineering workstations, set-top boxes, etc.) with other microprocessors may also be used. In at least one embodiment, the computer system 3000 may execute a version of the Windows operating system available from Microsoft Corporation of Redmond, Washington, although other operating systems (such as UNIX and Linux), embedded software, and / or graphical user interfaces may also be used.

[0290] In at least one embodiment, the computer system 3000 can be used in other devices, such as handheld devices and embedded applications. Some examples of handheld devices include cellular phones, Internet Protocol (IP) devices, digital cameras, personal digital assistants (“PDAs”), and handheld PCs. In at least one embodiment, the embedded application may include a microcontroller, a digital signal processor (“DSP”), a system-on-a-chip (SoC), a network computer (“NetPC”), a set-top box, a network hub, a wide area network (“WAN”) switch, or any other system capable of executing one or more instructions according to at least one embodiment.

[0291] In at least one embodiment, the computer system 3000 may include, but is not limited to, a processor 3002, which may include, but is not limited to, one or more execution units 3008 configured to execute a Computational Unified Device Architecture (“CUDA”). (Developed by NVIDIA Corporation, Santa Clara, California) In at least one embodiment, the CUDA program is at least a part of a software application written in the CUDA programming language. In at least one embodiment, the computer system 3000 is a single-processor desktop or server system. In at least one embodiment, the computer system 3000 may be a multiprocessor system. In at least one embodiment, the processor 3002 may include, but is not limited to, a CISC microprocessor, a RISC microprocessor, a VLIW microprocessor, a processor implementing instruction set combinations, or any other processor device, such as a digital signal processor. In at least one embodiment, the processor 3002 may be coupled to a processor bus 3010, which can transmit data signals between the processor 3002 and other components in the computer system 3000.

[0292] In at least one embodiment, processor 3002 may include, but is not limited to, a Level 1 (“L1”) internal cache memory (“cache”) 3004. In at least one embodiment, processor 3002 may have a single internal cache or multiple levels of internal cache. In at least one embodiment, the cache memory may reside external to processor 3002. In at least one embodiment, processor 3002 may include a combination of internal and external caches. In at least one embodiment, register file 3006 may store different types of data in various registers, including but not limited to integer registers, floating-point registers, status registers, and instruction pointer registers.

[0293] In at least one embodiment, an execution unit 3008, including but not limited to logic for performing integer and floating-point operations, is also located within the processor 3002. The processor 3002 may also include a microcode (“ucode”) read-only memory (“ROM”) for storing microcode of certain macro instructions. In at least one embodiment, the execution unit 3008 may include logic for processing a packaged instruction set 3009. In at least one embodiment, by including the packaged instruction set 3009 in the instruction set of the general-purpose processor 3002, along with the associated circuitry for executing the instructions, packaged data in the general-purpose processor 3002 can be used to perform operations used by many multimedia applications. In at least one embodiment, many multimedia applications can be executed more quickly and efficiently by using the full width of the processor's data bus to perform operations on the packaged data, which may eliminate the need to transfer smaller data units on the processor's data bus to perform one or more operations on a data element at a time.

[0294] In at least one embodiment, the execution unit 3008 may also be used in a microcontroller, embedded processor, graphics device, DSP, and other types of logic circuitry. In at least one embodiment, the computer system 3000 may include, but is not limited to, the memory 3020. In at least one embodiment, the memory 3020 may be implemented as a DRAM device, SRAM device, flash memory device, or other storage device. The memory 3020 may store instructions 3019 and / or data 3021 represented by data signals that can be executed by the processor 3002.

[0295] In at least one embodiment, the system logic chip may be coupled to processor bus 3010 and memory 3020. In at least one embodiment, the system logic chip may include, but is not limited to, a memory controller hub (“MCH”) 3016, and processor 3002 may communicate with MCH 3016 via processor bus 3010. In at least one embodiment, MCH 3016 may provide a high-bandwidth memory path 3018 to memory 3020 for instruction and data storage, as well as for storage of graphics commands, data, and textures. In at least one embodiment, MCH 3016 may initiate data signals between processor 3002, memory 3020, and other components in computer system 3000, and bridge data signals between processor bus 3010, memory 3020, and system I / O 3022. In at least one embodiment, the system logic chip may provide a graphics port for coupling to a graphics controller. In at least one embodiment, MCH 3016 can be coupled to memory 3020 via high-bandwidth memory path 3018, and graphics / video card 3012 can be coupled to MCH 3016 via Accelerated Graphics Port (“AGP”) interconnect 3014.

[0296] In at least one embodiment, the computer system 3000 may use system I / O 3022 as a proprietary hub interface bus to couple MCH 3016 to I / O controller hub (“ICH”) 3030. In at least one embodiment, ICH 3030 may provide direct connectivity to certain I / O devices via a local I / O bus. In at least one embodiment, the local I / O bus may include, but is not limited to, a high-speed I / O bus for connecting peripheral devices to memory 3020, chipset, and processor 3002. Examples may include, but are not limited to, an audio controller 3029, a firmware hub (“Flash BIOS”) 3028, a wireless transceiver 3026, a data storage 3024, a conventional I / O controller 3023 including user input 3025 and a keyboard interface, a serial expansion port 3077 (e.g., USB), and a network controller 3034. Data storage 3024 may include a hard disk drive, floppy disk drive, CD-ROM device, flash memory device, or other mass storage device.

[0297] In at least one embodiment, Figure 30 A system comprising interconnected hardware devices or "chips" is shown. In at least one embodiment, Figure 30 An exemplary SoC can be shown. In at least one embodiment, Figure 30 The devices shown can be interconnected with proprietary interconnects, standardized interconnects (e.g., PCIe), or some combination thereof. In at least one embodiment, one or more components of system 3000 are interconnected using a compute fast link (CXL) interconnect.

[0298] Figure 31 A system 3100 according to at least one embodiment is illustrated. In at least one embodiment, system 3100 is an electronic device utilizing processor 3110. In at least one embodiment, system 3100 may be, for example, but not limited to, a laptop computer, tower server, rack server, blade server, desktop computer, tablet computer, mobile device, telephone, embedded computer, or any other suitable electronic device.

[0299] In at least one embodiment, system 3100 may include, but is not limited to, processor 3110 communicatively coupled to any suitable number or type of components, peripherals, modules, or devices. In at least one embodiment, processor 3110 is coupled using a bus or interface, such as I... 2 C-bus, System Management Bus (“SMBus”), Low Pin Count (LPC) bus, Serial Peripheral Interface (“SPI”), High Definition Audio (“HDA”) bus, Serial Advanced Technology Accessory (“SATA”) bus, USB (versions 1, 2, and 3) or Universal Asynchronous Receiver / Transmitter (“UART”) bus. In at least one embodiment, Figure 31 A system is illustrated, comprising interconnected hardware devices or "chips". In at least one embodiment, Figure 31 An exemplary SoC can be shown. In at least one embodiment, Figure 31 The device shown can be interconnected with proprietary interconnects, standardized interconnects (e.g., PCIe), or some combination thereof. In at least one embodiment, Figure 31 One or more components are interconnected using Computational Fast Link (CXL) interconnects.

[0300] In at least one embodiment, Figure 31 This may include a display 3124, a touchscreen 3125, a touchpad 3130, a near-field communication unit (“NFC”) 3145, a sensor hub 3140, a thermal sensor 3146, a fast chipset (“EC”) 3135, a trusted platform module (“TPM”) 3138, a BIOS / firmware / flash memory (“BIOS, FW Flash”) 3122, a DSP 3160, a solid-state drive (“SSD”) or hard disk drive (“HDD”) 3120, a wireless local area network unit (“WLAN”) 3150, a Bluetooth unit 3152, a wireless wide area network unit (“WWAN”) 3156, a global positioning system (GPS) 3155, a camera (“USB 3.0 camera”) 3154 (e.g., a USB 3.0 camera), or a low-power double data rate (“LPDDR”) memory unit (“LPDDR3”) 3115 implemented in, for example, the LPDDR3 standard. These components may each be implemented in any suitable manner.

[0301] In at least one embodiment, other components may be communicatively coupled to processor 3110 via the components discussed above. In at least one embodiment, accelerometer 3141, ambient light sensor (“ALS”) 3142, compass 3143, and gyroscope 3144 may be communicatively coupled to sensor hub 3140. In at least one embodiment, thermal sensor 3139, fan 3137, keyboard 3146, and touchpad 3130 may be communicatively coupled to EC 3135. In at least one embodiment, speaker 3163, earphone 3164, and microphone (“mic”) 3165 may be communicatively coupled to audio unit (“audio codec and Class D amplifier”) 3164, which in turn may be communicatively coupled to DSP 3160. In at least one embodiment, audio unit 3164 may include, for example, but not limited to, audio encoder / decoder (“codec”) and Class D amplifier. In at least one embodiment, SIM card (“SIM”) 3157 may be communicatively coupled to WWAN unit 3156. In at least one embodiment, components such as WLAN unit 3150, Bluetooth unit 3152, and WWAN unit 3156 can be implemented as next-generation form factor (NGFF).

[0302] Figure 32 An exemplary integrated circuit 3200 according to at least one embodiment is illustrated. In at least one embodiment, the exemplary integrated circuit 3200 is a SoC (System-on-a-Chip) that can be fabricated using one or more IP cores. In at least one embodiment, the integrated circuit 3200 includes one or more application processors 3205 (e.g., CPUs), at least one graphics processor 3210, and may additionally include an image processor 3215 and / or a video processor 3220, any of which may be a modular IP core. In at least one embodiment, the integrated circuit 3200 includes peripheral or bus logic, which includes a USB controller 3225, a UART controller 3230, an SPI / SDIO controller 3235, and an I / O controller 3225. 2 S / I 2 C controller 3240. In at least one embodiment, integrated circuit 3200 may include display device 3245 coupled to one or more of high-definition multimedia interface (HDMI) controller 3250 and mobile industrial processor interface (MIPI) display interface 3255. In at least one embodiment, storage may be provided by flash memory subsystem 3260, including flash memory and flash memory controller. In at least one embodiment, a memory interface may be provided via memory controller 3265 for accessing SDRAM or SRAM memory devices. In at least one embodiment, some integrated circuits also include embedded security engine 3270.

[0303] Figure 33A computing system 3300 according to at least one embodiment is illustrated. In at least one embodiment, the computing system 3300 includes a processing subsystem 3301 having one or more processors 3302 and a system memory 3304 communicating via an interconnect path that may include a memory hub 3305. In at least one embodiment, the memory hub 3305 may be a separate component within a chipset assembly or may be integrated within one or more processors 3302. In at least one embodiment, the memory hub 3305 is coupled to an I / O subsystem 3311 via a communication link 3306. In at least one embodiment, the I / O subsystem 3311 includes an I / O hub 3307 that enables the computing system 3300 to receive input from one or more input devices 3308. In at least one embodiment, the I / O hub 3307 may enable a display controller, which is included in one or more processors 3302, for providing output to one or more display devices 3310A. In at least one embodiment, one or more display devices 3310A coupled to the I / O hub 3307 may include local, internal, or embedded display devices.

[0304] In at least one embodiment, the processing subsystem 3301 includes one or more parallel processors 3312 coupled to a memory hub 3305 via a bus or other communication link 3313. In at least one embodiment, the communication link 3313 may be one of many standards-based communication link technologies or protocols, such as, but not limited to, PCIe, or may be a vendor-specific communication interface or communication architecture. In at least one embodiment, the one or more parallel processors 3312 form a compute-intensive parallel or vector processing system that may include a large number of processing cores and / or processing clusters, such as a multi-core integrated (MIC) processor. In at least one embodiment, the one or more parallel processors 3312 form a graphics processing subsystem capable of outputting pixels to one of one or more display devices 3310A coupled via an I / O hub 3307. In at least one embodiment, the one or more parallel processors 3312 may also include a display controller and a display interface (not shown) to enable direct connection to one or more display devices 3310B.

[0305] In at least one embodiment, system storage unit 3314 may be connected to I / O hub 3307 to provide a storage mechanism for computing system 3300. In at least one embodiment, I / O switch 3316 may be used to provide an interface mechanism to enable connectivity between I / O hub 3307 and other components, such as network adapter 3318 and / or wireless network adapter 3319 which may be integrated into the platform, and various other devices that can be added via one or more additional devices 3320. In at least one embodiment, network adapter 3318 may be an Ethernet adapter or another wired network adapter. In at least one embodiment, wireless network adapter 3319 may include one or more Wi-Fi, Bluetooth, NFC, or other network devices comprising one or more radios.

[0306] In at least one embodiment, the computing system 3300 may include other components not explicitly shown, including USB or other port connections, optical storage drives, video capture devices and / or variations thereof, and may also be connected to the I / O hub 3307. In at least one embodiment, for Figure 33 The communication paths that interconnect the various components can be implemented using any suitable protocol, such as PCI (Peripheral Component Interconnect) based protocols (e.g., PCIe), or other bus or point-to-point communication interfaces and / or protocols (e.g., NVLink high-speed interconnect or interconnect protocols).

[0307] In at least one embodiment, one or more parallel processors 3312 include circuitry optimized for graphics and video processing (e.g., including video output circuitry) and constitute a graphics processing unit (GPU). In at least one embodiment, one or more parallel processors 3312 include circuitry optimized for general-purpose processing. In at least one embodiment, components of the computing system 3300 may be integrated with one or more other system elements on a single integrated circuit. In at least one embodiment, one or more parallel processors 3312, memory hub 3305, processor 3302, and I / O hub 3307 may be integrated into a system-on-a-chip (SoC) integrated circuit. In at least one embodiment, components of the computing system 3300 may be integrated into a single package to form a system-in-package (SIP) configuration. In at least one embodiment, at least a portion of the components of the computing system 3300 may be integrated into a multi-chip module (MCM) that can interconnect with other MCMs to a modular computing system. In at least one embodiment, the I / O subsystem 3311 and display device 3310B are omitted from the computing system 3300.

[0308] Processing system

[0309] The following figures illustrate, but are not limited to, exemplary processing systems that can be used to implement at least one embodiment.

[0310] Figure 34 An accelerated processing unit (“APU”) 3400 according to at least one embodiment is illustrated. In at least one embodiment, the APU 3400 was developed by AMD Inc. of Santa Clara, California. In at least one embodiment, the APU 3400 can be configured to execute applications, such as CUDA programs. In at least one embodiment, the APU 3400 includes, but is not limited to, a core complex 3410, a graphics complex 3440, an architecture 3460, an I / O interface 3470, a memory controller 3480, a display controller 3492, and a multimedia engine 3494. In at least one embodiment, the APU 3400 can be, but is not limited to, any combination of any number of core complexes 3410, any number of graphics complexes 3440, any number of display controllers 3492, and any number of multimedia engines 3494. For illustrative purposes, multiple instances of similar objects are indicated herein by reference numerals, wherein the reference numerals identify the object, and the numbers in parentheses identify the desired instances.

[0311] In at least one embodiment, the core complex 3410 is a CPU, the graphics complex 3440 is a GPU, and the APU 3400 is a processing unit that is not limited to 3410 and 3440 integrated onto a single chip. In at least one embodiment, some tasks may be assigned to the core complex 3410, while other tasks may be assigned to the graphics complex 3440. In at least one embodiment, the core complex 3410 is configured to execute main control software associated with the APU 3400, such as an operating system. In at least one embodiment, the core complex 3410 is the main processor of the APU 3400, which controls and coordinates the operation of other processors. In at least one embodiment, the core complex 3410 issues commands to control the operation of the graphics complex 3440. In at least one embodiment, the core complex 3410 may be configured to execute host executable code derived from CUDA source code, and the graphics complex 3440 may be configured to execute device executable code derived from CUDA source code.

[0312] In at least one embodiment, the core complex 3410 includes, but is not limited to, cores 3420(1)-3420(4) and L3 cache 3430. In at least one embodiment, the core complex 3410 may include, but is not limited to, any combination of any number of cores 3420 and any number and type of cache. In at least one embodiment, the cores 3420 are configured to execute instructions of a specific instruction set architecture (“ISA”). In at least one embodiment, each core 3420 is a CPU core.

[0313] In at least one embodiment, each core 3420 includes, but is not limited to, a fetch / decode unit 3422, an integer execution engine 3424, a floating-point execution engine 3426, and an L2 cache 3428. In at least one embodiment, the fetch / decode unit 3422 fetches instructions, decodes these instructions, generates micro-operations, and dispatches individual micro-instructions to the integer execution engine 3424 and the floating-point execution engine 3426. In at least one embodiment, the fetch / decode unit 3422 may simultaneously dispatch one micro-instruction to the integer execution engine 3424 and another micro-instruction to the floating-point execution engine 3426. In at least one embodiment, the integer execution engine 3424 performs operations not limited to integer and memory operations. In at least one embodiment, the floating-point engine 3426 performs operations not limited to floating-point and vector operations. In at least one embodiment, the fetch-decode unit 3422 dispatches micro-instructions to a single execution engine, which replaces both the integer execution engine 3424 and the floating-point execution engine 3426.

[0314] In at least one embodiment, each core 3420(i) can access an L2 cache 3428(i) included in core 3420(i), where i is an integer representing a specific instance of core 3420. In at least one embodiment, each core 3420 included in core complex 3410(j) is connected to other cores 3420 included in core complex 3410(j) via an L3 cache 3430(j) included in core complex 3410(j), where j is an integer representing a specific instance of core complex 3410. In at least one embodiment, a core 3420 included in core complex 3410(j) can access all L3 caches 3430(j) included in core complex 3410(j), where j is an integer representing a specific instance of core complex 3410. In at least one embodiment, the L3 cache 3430 may include, but is not limited to, any number of slices.

[0315] In at least one embodiment, the graphics complex 3440 can be configured to perform computational operations in a highly parallel manner. In at least one embodiment, the graphics complex 3440 is configured to perform graphics pipeline operations, such as drawing commands, pixel operations, geometric calculations, and other operations associated with rendering an image to a display. In at least one embodiment, the graphics complex 3440 is configured to perform graphics-independent operations. In at least one embodiment, the graphics complex 3440 is configured to perform both graphics-related and graphics-independent operations.

[0316] In at least one embodiment, the graphics complex 3440 includes, but is not limited to, any number of computing units 3450 and an L2 cache 3442. In at least one embodiment, the computing units 3450 share the L2 cache 3442. In at least one embodiment, the L2 cache 3442 is partitioned. In at least one embodiment, the graphics complex 3440 includes, but is not limited to, any number of computing units 3450 and any number (including zero) and type of cache. In at least one embodiment, the graphics complex 3440 includes, but is not limited to, any number of dedicated graphics hardware.

[0317] In at least one embodiment, each computing unit 3450 includes, but is not limited to, any number of SIMD units 3452 and shared memory 3454. In at least one embodiment, each SIMD unit 3452 implements a SIMD architecture and is configured to execute operations in parallel. In at least one embodiment, each computing unit 3450 may execute any number of thread blocks, but each thread block executes on a single computing unit 3450. In at least one embodiment, a thread block includes, but is not limited to, any number of execution threads. In at least one embodiment, a workgroup is a thread block. In at least one embodiment, each SIMD unit 3452 executes a different warp. In at least one embodiment, a warp is a group of threads (e.g., 16 threads), where each thread in the warp belongs to a single thread block and is configured to process different datasets based on a single instruction set. In at least one embodiment, prediction can be used to disable one or more threads in a warp. In at least one embodiment, a channel is a thread. In at least one embodiment, a work item is a thread. In at least one embodiment, a wavefront is a warp. In at least one embodiment, different wavefronts in a thread block can be synchronized together and communicate via shared memory 3454.

[0318] In at least one embodiment, architecture 3460 is a system interconnect that facilitates data and control transfers across core complex 3410, graphics complex 3440, I / O interface 3470, memory controller 3480, display controller 3492, and multimedia engine 3494. In at least one embodiment, in addition to or instead of architecture 3460, APU 3400 may also include, but is not limited to, any number and type of system interconnects that facilitate data and control transfers across any number and type of components that may be directly or indirectly linked, either internally or externally to APU 3400. In at least one embodiment, I / O interface 3470 represents any number and type of I / O interface (e.g., PCI, PCI-Extended (“PCI-X”), PCIe, Gigabit Ethernet (“GBE”), USB, etc.). In at least one embodiment, various types of peripheral devices are coupled to I / O interface 3470. In at least one embodiment, the peripheral device coupled to the I / O interface 3470 may include, but is not limited to, a keyboard, mouse, printer, scanner, joystick or other types of game controllers, media recording devices, external storage devices, network interface cards, etc.

[0319] In at least one embodiment, the display controller AMD92 displays images on one or more display devices (e.g., liquid crystal display (LCD) devices). In at least one embodiment, the multimedia engine 240 includes, but is not limited to, any number and type of multimedia-related circuitry, such as video decoders, video encoders, image signal processors, etc. In at least one embodiment, the memory controller 3480 facilitates data transfer between the APU 3400 and the unified system memory 3490. In at least one embodiment, the core complex 3410 and the graphics complex 3440 share the unified system memory 3490.

[0320] In at least one embodiment, the APU 3400 implements a memory subsystem, including but not limited to any number and type of memory controllers 3480 and memory devices (e.g., shared memory 3454) that may be dedicated to a single component or shared among multiple components. In at least one embodiment, the APU 3400 implements a cache subsystem, including but not limited to one or more cache memories (e.g., L2 cache 2728, L3 cache 3430, and L2 cache 3442), each cache memory being component-private or shared among any number of components (e.g., core 3420, core complex 3410, SIMD unit 3452, compute unit 3450, and graphics complex 3440).

[0321] Figure 35A CPU 3500 according to at least one embodiment is illustrated. In at least one embodiment, the CPU 3500 was developed by AMD Inc. of Santa Clara, California. In at least one embodiment, the CPU 3500 can be configured to execute an application. In at least one embodiment, the CPU 3500 is configured to execute host control software, such as an operating system. In at least one embodiment, the CPU 3500 issues commands to control the operation of an external GPU (not shown). In at least one embodiment, the CPU 3500 can be configured to execute host executable code derived from CUDA source code, and the external GPU can be configured to execute device executable code derived from such CUDA source code. In at least one embodiment, the CPU 3500 includes, but is not limited to, any number of core complexes 3510, architectures 3560, I / O interfaces 3570, and memory controllers 3580.

[0322] In at least one embodiment, the core complex 3510 includes, but is not limited to, cores 3520(1)-3520(4) and L3 cache 3530. In at least one embodiment, the core complex 3510 may include, but is not limited to, any combination of any number of cores 3520 and any number and type of cache. In at least one embodiment, the cores 3520 are configured to execute instructions of a specific ISA. In at least one embodiment, each core 3520 is a CPU core.

[0323] In at least one embodiment, each core 3520 includes, but is not limited to, a fetch / decode unit 3522, an integer execution engine 3524, a floating-point execution engine 3526, and an L2 cache 3528. In at least one embodiment, the fetch / decode unit 3522 fetches instructions, decodes these instructions, generates micro-operations, and dispatches individual micro-instructions to the integer execution engine 3524 and the floating-point execution engine 3526. In at least one embodiment, the fetch / decode unit 3522 may simultaneously dispatch one micro-instruction to the integer execution engine 3524 and another micro-instruction to the floating-point execution engine 3526. In at least one embodiment, the integer execution engine 3524 performs operations not limited to integer and memory operations. In at least one embodiment, the floating-point engine 3526 performs operations not limited to floating-point and vector operations. In at least one embodiment, the fetch-decode unit 3522 dispatches micro-instructions to a single execution engine, which replaces both the integer execution engine 3524 and the floating-point execution engine 3526.

[0324] In at least one embodiment, each core 3520(i) can access an L2 cache 3528(i) included in core 3520(i), where i is an integer representing a specific instance of core 3520. In at least one embodiment, each core 3520 included in core complex 3510(j) is connected to other cores 3520 in core complex 3510(j) via an L3 cache 3530(j) included in core complex 3510(j), where j is an integer representing a specific instance of core complex 3510. In at least one embodiment, a core 3520 included in core complex 3510(j) can access all L3 caches 3530(j) included in core complex 3510(j), where j is an integer representing a specific instance of core complex 3510. In at least one embodiment, the L3 cache 3530 may include, but is not limited to, any number of slices.

[0325] In at least one embodiment, structure 3560 is a system interconnect that facilitates data and control transfers across core complexes 3510(1)-3510(N) (where N is a positive integer), I / O interface 3570, and memory controller 3580. In at least one embodiment, in addition to or instead of structure 3560, CPU 3500 may also include, but is not limited to, any number and type of system interconnects that facilitate data and control transfers across any number and type of components that may be directly or indirectly linked, either inside or outside CPU 3500. In at least one embodiment, I / O interface 3570 represents any number and type of I / O interfaces (e.g., PCI, PCI-X, PCIe, GBE, USB, etc.). In at least one embodiment, various types of peripheral devices are coupled to I / O interface 3570. In at least one embodiment, peripheral devices coupled to I / O interface 3570 may include, but are not limited to, displays, keyboards, mice, printers, scanners, joysticks or other types of game controllers, media recording devices, external storage devices, network interface cards, etc.

[0326] In at least one embodiment, memory controller 3580 facilitates data transfer between CPU 3500 and system memory 3590. In at least one embodiment, core complex 3510 and graphics complex 3540 share system memory 3590. In at least one embodiment, CPU 3500 implements a memory subsystem, which includes, but is not limited to, any number and type of memory controllers 3580 and memory devices that may be dedicated to a component or shared among multiple components. In at least one embodiment, CPU 3500 implements a cache subsystem, which includes, but is not limited to, one or more cache memories (e.g., L2 cache 3528 and L3 cache 3530), each cache memory may be component-private or shared among any number of components (e.g., core 3520 and core complex 3510).

[0327] Figure 36 An exemplary accelerator integration slice 3690 according to at least one embodiment is illustrated. As used herein, a "slice" includes a designated portion of the processing resources of an accelerator integrated circuit. In at least one embodiment, the accelerator integrated circuit provides cache management, memory access, environment management, and interrupt management services for multiple graphics processing engines among multiple graphics acceleration modules. Each graphics processing engine may comprise a separate GPU. Optionally, the graphics processing engine may include different types of graphics processing engines within the GPU, such as graphics execution units, media processing engines (e.g., video encoders / decoders), samplers, and blit engines. In at least one embodiment, a graphics acceleration module may be a GPU having multiple graphics processing engines. In at least one embodiment, the graphics processing engines may be individual GPUs integrated on a general-purpose package, line card, or chip.

[0328] The application's effective address space 3682 within system memory 3614 stores process element 3683. In one embodiment, process element 3683 is stored in response to a GPU call 3681 from an application 3680 executing on processor 3607. Process element 3683 contains the processing state of the corresponding application 3680. A job descriptor (WD) 3684 contained in process element 3683 may be a single job requested by the application or may contain pointers to job queues. In at least one embodiment, WD 3684 is a pointer to a job request queue in the application's effective address space 3682.

[0329] The graphics acceleration module 3646 and / or the various graphics processing engines may be shared by all or some processes in the system. In at least one embodiment, infrastructure may be included for establishing a processing state and sending the WD 3684 to the graphics acceleration module 3646 to begin operation in a virtualized environment.

[0330] In at least one embodiment, a dedicated process programming model is used for implementation. In this model, a single process owns the graphics acceleration module 3646 or an individual graphics processing engine. Since the graphics acceleration module 3646 is owned by a single process, the hypervisor initializes the accelerator integrated circuit for the owned partition, and the operating system initializes the accelerator integrated circuit for the owned partition when the graphics acceleration module 3646 is allocated.

[0331] During operation, the WD fetch unit 3691 in the accelerator integrated slice 3690 fetches the next WD 3684, which includes instructions for the work to be performed by one or more graphics processing engines of the graphics acceleration module 3646. Data from the WD 3684 can be stored in register 3645 and used by the memory management unit (MMU) 3639, interrupt management circuitry 3647, and / or environment management circuitry 3648, as shown. For example, an embodiment of the MMU 3639 includes segment / page roaming circuitry for accessing segment / page tables 3686 within the OS virtual address space 3685. The interrupt management circuitry 3647 can handle interrupt events (INT) 3692 received from the graphics acceleration module 3646. When performing graph operations, the effective address 3693 generated by the graphics processing engine is translated into an actual address by the MMU 3639.

[0332] In one embodiment, the same register set 3645 is copied for each graphics processing engine and / or graphics acceleration module 3646 and can be initialized by the hypervisor or operating system. Each of these copied registers can be included in the accelerator integration slice 3690. Exemplary registers that can be initialized by the hypervisor are shown in Table 1.

[0333] Table 1 – Registers for Supervisor Initialization

[0334] 2 Real Address (RA) plan processing area pointer 3 Authorization mask overwrite register 4 Interrupt vector table input offset 5 Interrupt vector table entry restrictions 6 Status Register 7 Logical partition ID 8 Real Address (RA) Manager Accelerator Utilization Record Pointer 9 Storage description register

[0335] Table 2 shows exemplary registers that can be initialized by the operating system.

[0336] Table 2 – Operating System Initialization Registers

[0337] 2 Valid Address (EA) Environment Save / Restore Pointer 3 Virtual Address (VA) accelerator utilization record pointer 4 Virtual address (VA) stores segment table pointers 5 mask of authority 6 Job descriptor

[0338] In one embodiment, each WD 3684 is specific to a particular graphics acceleration module 3646 and / or a particular graphics processing engine. It contains all the information required for the graphics processing engine to perform its work or to do so, or it may be a pointer to a memory location where the application has established a command queue for the work to be done.

[0339] Figures 37A-37B An exemplary graphics processor according to at least one embodiment herein is illustrated. In at least one embodiment, any exemplary graphics processor may be manufactured using one or more IP cores. In addition to the illustrations, other logic and circuitry may be included in at least one embodiment, including additional graphics processor / cores, peripheral interface controllers, or general-purpose processor cores. In at least one embodiment, the exemplary graphics processor is used within a System-on-a-Chip (SoC).

[0340] Figure 37A An exemplary graphics processor 3710 of a SoC integrated circuit according to at least one embodiment is shown, which can be manufactured using one or more IP cores. Figure 37B An additional exemplary graphics processor 3740 of a SoC integrated circuit according to at least one embodiment is shown, which can be manufactured using one or more IP cores. In at least one embodiment, Figure 37A The graphics processor 3710 is a low-power graphics processor core. In at least one embodiment, Figure 37B The graphics processor 3740 is a higher-performance graphics processor core. In at least one embodiment, each graphics processor 3710, 3740 may be... Figure 5 A variant of the 510 graphics processor.

[0341] In at least one embodiment, the graphics processor 3710 includes a vertex processor 3705 and one or more fragment processors 3715A-3715N (e.g., 3715A, 3715B, 3715C, 3715D to 3715N-1 and 3715N). In at least one embodiment, the graphics processor 3710 can execute different shader programs via separate logic, such that the vertex processor 3705 is optimized to perform operations for the vertex shader program, while one or more fragment processors 3715A-3715N perform fragment (e.g., pixel) shading operations for fragments or pixels or shader programs. In at least one embodiment, the vertex processor 3705 performs the vertex processing stage of the 3D graphics pipeline and generates primitive and vertex data. In at least one embodiment, the fragment processors 3715A-3715N use the primitive and vertex data generated by the vertex processor 3705 to generate framebuffers for display on a display device. In at least one embodiment, the fragment processors 3715A-3715N are optimized to execute fragment shader programs as provided in the OpenGL API, which can be used to perform operations similar to those of pixel shader programs provided in the Direct 3D API.

[0342] In at least one embodiment, the graphics processor 3710 additionally includes one or more MMUs 3720A-3720B, caches 3725A-3725B, and circuit interconnects 3730A-3730B. In at least one embodiment, one or more MMUs 3720A-3720B provide a virtual-to-physical address mapping for the graphics processor 3710, including for the vertex processor 3705 and / or fragment processors 3715A-3715N, which can reference vertex or image / texture data stored in memory, in addition to vertex or image / texture data stored in one or more caches 3725A-3725B. In at least one embodiment, one or more MMUs 3720A-3720B can be synchronized with other MMUs within the system, including with... Figure 32 One or more application processors 3205, image processors 3215, and / or video processors 3220 are associated with one or more MMUs, enabling each processor 3205-3220 to participate in a shared or unified virtual memory system. In at least one embodiment, one or more circuit interconnects 3730A-3730B enable the graphics processor 3710 to connect to other IP cores within the SoC via the SoC's internal bus or via a direct connection.

[0343] In at least one embodiment, the graphics processor 3740 includes Figure 37A The graphics processor 3710 includes one or more MMUs 3720A-3720B, caches 3725A-3725B, and circuit interconnects 3730A-3730B. In at least one embodiment, the graphics processor 3740 includes one or more shader cores 3755A-3755N (e.g., 3755A, 3755B, 3755C, 3755D, 3755E, 3755F, to 3755N-1 and 3755N) that provide a unified shader core architecture, wherein a single core or type of core can execute all types of programmable shader code, including shader program code for implementing vertex shaders, fragment shaders, and / or compute shaders. In at least one embodiment, the number of shader cores may vary. In at least one embodiment, the graphics processor 3740 includes an inter-core task manager 3745 that acts as a thread dispatcher to assign execution threads to one or more shader cores 3755A-3755N and a tile unit 3758 to accelerate tile-based rendering operations, wherein rendering operations of a scene are subdivided in image space, for example, to take advantage of local spatial consistency within the scene or to optimize the use of internal caches.

[0344] Figure 38AA graphics core 3800 according to at least one embodiment is shown. In at least one embodiment, the graphics core 3800 may include... Figure 32 The graphics processor 3210 is located within it. In at least one embodiment, the graphics core 3800 may be... Figure 37B The graphics core 3800 uses a unified shader core 3755A-3755N. In at least one embodiment, the graphics core 3800 includes a shared instruction cache 3802, texture units 3818, and cache / shared memory 3820, which are common to execution resources within the graphics core 3800. In at least one embodiment, the graphics core 3800 may include multiple slices 3801A-3801N or partitions of each core, and the graphics processor may include multiple instances of the graphics core 3800. Slices 3801A-3801N may include supporting logic, including local instruction caches 3804A-3804N, thread schedulers 3806A-3806N, thread dispatchers 3808A-3808N, and a set of registers 3810A-3810N. In at least one embodiment, slices 3801A-3801N may include a set of additional functional units (AFU) 3812A-3812N, floating-point units (FPU) 3814A-3814N, integer arithmetic logic units (ALU) 3816A-3816N, address calculation units (ACU) 3813A-3813N, double-precision floating-point units (DPFPU) 3815A-3815N, and matrix processing units (MPU) 3817A-3817N.

[0345] In one embodiment, the FPU 3814A-3814N can perform single-precision (32-bit) and half-precision (16-bit) floating-point operations, while the DPFPU 3815A-3815N can perform double-precision (64-bit) floating-point operations. In at least one embodiment, the ALU 3816A-3816N can perform variable-precision integer operations with 8-bit, 16-bit, and 32-bit precision, and can be configured for mixed-precision operations. In at least one embodiment, the MPU 3817A-3817N can also be configured for mixed-precision matrix operations, including half-precision floating-point operations and 8-bit integer operations. In at least one embodiment, the MPU 3817A-3817N can perform various matrix operations to accelerate CUDA programs, including enabling support for accelerated Generalized Matrix-to-Matrix Multiplication (GEMM). In at least one embodiment, the AFU 3812A-3812N can perform additional logical operations not supported by floating-point or integer units, including trigonometric operations (e.g., Sine, Cosine, etc.).

[0346] Figure 38BA general-purpose graphics processing unit (GPGPU) 3830 is illustrated in at least one embodiment. In at least one embodiment, the GPGPU 3830 is highly parallel and suitable for deployment on a multi-chip module. In at least one embodiment, the GPGPU 3830 can be configured such that highly parallel computational operations can be executed by a GPU array. In at least one embodiment, the GPGPU 3830 can be directly linked to other instances of the GPGPU 3830 to create a multi-GPU cluster to improve execution time for CUDA programs. In at least one embodiment, the GPGPU 3830 includes a host interface 3832 for connection to a host processor. In at least one embodiment, the host interface 3832 is a PCIe interface. In at least one embodiment, the host interface 3832 can be a vendor-specific communication interface or communication structure. In at least one embodiment, the GPGPU 3830 receives commands from the host processor and uses a global scheduler 3834 to assign execution threads associated with those commands to a set of compute clusters 3836A-3836H. In at least one embodiment, computing clusters 3836A-3836H share cache memory 3838. In at least one embodiment, cache memory 3838 can be used as an advanced cache of cache memory within computing clusters 3836A-3836H.

[0347] In at least one embodiment, the GPGPU 3830 includes memory 3844A-3844B coupled to the computing cluster 3836A-3836H via a set of memory controllers 3842A-3842B. In at least one embodiment, memory 3844A-3844B may include various types of memory devices, including dynamic random access memory (DRAM) or graphics random access memory, such as synchronous graphics random access memory (SGRAM), including graphics double data rate (GDDR) memory.

[0348] In at least one embodiment, computing clusters 3836A-3836H each include a set of graphics cores, such as Figure 38A The graphics core 3800 may include various types of integer and floating-point logic units, capable of performing computational operations at various precisions, including computations suitable for CUDA programs. In at least one embodiment, at least a subset of the floating-point units in each computing cluster 3836A-3836H may be configured to perform 16-bit or 32-bit floating-point operations, while different subsets of the floating-point units may be configured to perform 64-bit floating-point operations.

[0349] In at least one embodiment, multiple instances of the GPGPU 3830 can be configured to operate as a computing cluster. In at least one embodiment, the computing clusters 3836A-3836H can implement any technically feasible communication technology for synchronization and data exchange. In at least one embodiment, the multiple instances of the GPGPU 3830 communicate via a host interface 3832. In at least one embodiment, the GPGPU 3830 includes an I / O hub 3839 that couples the GPGPU 3830 to a GPU link 3840, enabling direct connection to other instances of the GPGPU 3830. In at least one embodiment, the GPU link 3840 is coupled to a dedicated GPU-to-GPU bridge, enabling communication and synchronization among the multiple instances of the GPGPU 3830. In at least one embodiment, the GPU link 3840 is coupled to a high-speed interconnect for sending and receiving data to and from other GPGPUs or parallel processors. In at least one embodiment, the multiple instances of the GPGPU 3830 reside in a separate data processing system and communicate via a network device accessible via the host interface 3832. In at least one embodiment, the GPU link 3840 may be configured to connect to a host processor, supplementing or replacing the host interface 3832. In at least one embodiment, the GPGPU 3830 may be configured to execute CUDA programs.

[0350] Figure 39A A parallel processor 3900 according to at least one embodiment is shown. In at least one embodiment, various components of the parallel processor 3900 may be implemented using one or more integrated circuit devices, such as programmable processors, application-specific integrated circuits (ASICs), or FPGAs.

[0351] In at least one embodiment, the parallel processor 3900 includes a parallel processing unit 3902. In at least one embodiment, the parallel processing unit 3902 includes an I / O unit 3904 that enables communication with other devices, including other instances of the parallel processing unit 3902. In at least one embodiment, the I / O unit 3904 can be directly connected to other devices. In at least one embodiment, the I / O unit 3904 is connected to other devices using a hub or switch interface (e.g., memory hub 1405). In at least one embodiment, the connection between the memory hub 1405 and the I / O unit 3904 forms a communication link. In at least one embodiment, the I / O unit 3904 is connected to a host interface 3906 and a memory crossbar switch 3916, wherein the host interface 3906 receives commands for performing processing operations, and the memory crossbar switch 3916 receives commands for performing memory operations.

[0352] In at least one embodiment, when host interface 3906 receives a command buffer via I / O unit 3904, host interface 3906 can direct work operations to execute those commands to front end 3908. In at least one embodiment, front end 3908 is coupled to scheduler 3910, which is configured to assign commands or other work items to processing array 3912. In at least one embodiment, scheduler 3910 ensures that processing array 3912 is correctly configured and in an active state before assigning tasks to processing array 3912. In at least one embodiment, scheduler 3910 is implemented via firmware logic executed on a microcontroller. In at least one embodiment, the microcontroller-implemented scheduler 3910 can be configured to perform complex scheduling and work assignment operations at both coarse and fine granular levels, enabling fast preemption and environment switching of threads executing on processing array 3912. In at least one embodiment, host software can demonstrate workloads scheduled on processing array 3912 via one of multiple graphics processing doorbells. In at least one embodiment, the workload can then be automatically distributed on the processing array 3912 by the scheduler 3910 logic within the microcontroller, which includes the scheduler 3910.

[0353] In at least one embodiment, the processing array 3912 may include up to "N" processing clusters (e.g., clusters 3914A, 3914B to 3914N). In at least one embodiment, each cluster 3914A-3914N of the processing array 3912 may execute a large number of concurrent threads. In at least one embodiment, the scheduler 3910 may use various scheduling and / or work allocation algorithms to allocate work to the clusters 3914A-3914N of the processing array 3912, which may vary depending on the workload generated by each type of program or computation. In at least one embodiment, scheduling may be handled dynamically by the scheduler 3910, or may be partially assisted by compiler logic during the compilation of program logic configured to be executed by the processing array 3912. In at least one embodiment, different clusters 3914A-3914N of the processing array 3912 may be assigned to process different types of programs or to perform different types of computations.

[0354] In at least one embodiment, the processing array 3912 can be configured to perform various types of parallel processing operations. In at least one embodiment, the processing array 3912 is configured to perform general-purpose parallel computing operations. In at least one embodiment, the processing array 3912 may include logic for performing processing tasks, including filtering video and / or audio data, performing modeling operations, including physical operations, and performing data transformations.

[0355] In at least one embodiment, the processing array 3912 is configured to perform parallel graphics processing operations. In at least one embodiment, the processing array 3912 may include additional logic to support the execution of such graphics processing operations, including but not limited to texture sampling logic for performing texture operations, as well as tessellation logic and other vertex processing logic. In at least one embodiment, the processing array 3912 may be configured to execute shader programs related to graphics processing, such as, but not limited to, vertex shaders, tessellation shaders, geometry shaders, and pixel shaders. In at least one embodiment, the parallel processing unit 3902 may transfer data from system memory via I / O unit 3904 for processing. In at least one embodiment, during processing, the transferred data may be stored in on-chip memory (e.g., parallel processor memory 3922) and then written back to system memory.

[0356] In at least one embodiment, when the parallel processing unit 3902 is used to perform graph processing, the scheduler 3910 may be configured to divide the processing workload into tasks of approximately equal size to better distribute graphics processing operations among the multiple clusters 3914A-3914N of the processing array 3912. In at least one embodiment, portions of the processing array 3912 may be configured to perform different types of processing. For example, in at least one embodiment, a first portion may be configured to perform vertex shading and topology generation, a second portion may be configured to perform tessellation and geometry shading, and a third portion may be configured to perform pixel shading or other screen-space operations to generate a rendered image for display. In at least one embodiment, intermediate data generated by one or more of the clusters 3914A-3914N may be stored in a buffer to allow intermediate data to be transferred between the clusters 3914A-3914N for further processing.

[0357] In at least one embodiment, the processing array 3912 may receive processing tasks to be executed via a scheduler 3910, which receives commands defining the processing tasks from a front end 3908. In at least one embodiment, the processing task may include an index of data to be processed, such as surface (patch) data, raw data, vertex data, and / or pixel data, as well as state parameters and commands defining how the data is processed (e.g., what program to execute). In at least one embodiment, the scheduler 3910 may be configured to acquire an index corresponding to a task, or may receive an index from the front end 3908. In at least one embodiment, the front end 3908 may be configured to ensure that the processing array 3912 is configured to be active before initiating a workload specified by an incoming command buffer (e.g., a batch buffer, push buffer, etc.).

[0358] In at least one embodiment, each of one or more instances of the parallel processing unit 3902 may be coupled to the parallel processor memory 3922. In at least one embodiment, the parallel processor memory 3922 may be accessed via a memory crossbar switch 3916, which may receive memory requests from the processing array 3912 and the I / O unit 3904. In at least one embodiment, the memory crossbar switch 3916 may be accessed via a memory interface 3918. In at least one embodiment, the memory interface 3918 may include a plurality of partition units (e.g., partition units 3920A, 3920B to 3920N), each of which may be coupled to a portion (e.g., a memory cell) of the parallel processor memory 3922. In at least one embodiment, the plurality of partition units 3920A-3920N are configured to be equal to the number of memory units, such that the first partition unit 3920A has a corresponding first memory unit 3924A, the second partition unit 3920B has a corresponding memory unit 3924B, and the Nth partition unit 3920N has a corresponding Nth memory unit 3924N. In at least one embodiment, the number of partition units 3920A-3920N may not be equal to the number of memory devices.

[0359] In at least one embodiment, memory cells 3924A-3924N may include various types of memory devices, including dynamic random access memory (DRAM) or graphics random access memory, such as synchronous graphics random access memory (SGRAM), including graphics double data rate (GDDR) memory. In at least one embodiment, memory cells 3924A-3924N may also include 3D stacked memory, including but not limited to high bandwidth memory (HBM). In at least one embodiment, rendering targets such as frame buffers or texture maps may be stored across memory cells 3924A-3924N, allowing partitioning cells 3920A-3920N to write portions of each rendering target in parallel to efficiently utilize the available bandwidth of the parallel processor memory 3922. In at least one embodiment, local instances of the parallel processor memory 3922 may be excluded to facilitate a unified memory design that combines system memory with local cache memory.

[0360] In at least one embodiment, any of the clusters 3914A-3914N of the processing array 3912 can process data to be written to any memory cell 3924A-3924N within the parallel processor memory 3922. In at least one embodiment, the memory crossbar switch 3916 can be configured to transfer the output of each cluster 3914A-3914N to any partition cell 3920A-3920N or another cluster 3914A-3914N, and the clusters 3914A-3914N can perform further processing operations on the output. In at least one embodiment, each cluster 3914A-3914N can communicate with the memory interface 3918 via the memory crossbar switch 3916 to read from or write to various external storage devices. In at least one embodiment, the memory crossbar switch 3916 has a connection to a memory interface 3918 for communication with I / O unit 3904, and a connection to a local instance of parallel processor memory 3922, thereby enabling processing units within different processing clusters 3914A-3914N to communicate with system memory or other memory not local to parallel processing unit 3902. In at least one embodiment, the memory crossbar switch 3916 may use virtual channels to separate traffic flows between clusters 3914A-3914N and partition units 3920A-3920N.

[0361] In at least one embodiment, multiple instances of the parallel processing unit 3902 may be provided on a single insert card, or multiple insert cards may be interconnected. In at least one embodiment, different instances of the parallel processing unit 3902 may be configured to interoperate, even if the different instances have different numbers of processing cores, different numbers of local parallel processor memories, and / or other configuration differences. For example, in at least one embodiment, some instances of the parallel processing unit 3902 may include higher-precision floating-point units relative to other instances. In at least one embodiment, a system combining one or more instances of the parallel processing unit 3902 or the parallel processor 3900 may be implemented in various configurations and form factors, including but not limited to desktop, laptop, or handheld personal computers, servers, workstations, game consoles, and / or embedded systems.

[0362] Figure 39B A processing cluster 3994 according to at least one embodiment is shown. In at least one embodiment, the processing cluster 3994 is included within a parallel processing unit. In at least one embodiment, the processing cluster 3994 is... Figure 39AAn instance of one of the processing clusters 3914A-3914N. In at least one embodiment, the processing cluster 3994 can be configured to execute a number of threads in parallel, wherein the term "thread" refers to an instance of a specific program executing on a particular set of input data. In at least one embodiment, a Single Instruction Multiple Data (SIMD) instruction issuing technique is used to support the parallel execution of a large number of threads without providing multiple independent instruction units. In at least one embodiment, a Single Instruction Multiple Threading (SIMT) technique is used to support the parallel execution of a large number of generally synchronous threads, which uses a common instruction unit configured to issue instructions to a set of processing engines within each processing cluster 3994.

[0363] In at least one embodiment, the operation of the processing cluster 3994 can be controlled by a pipeline manager 3932 that assigns processing tasks to the SIMT parallel processors. In at least one embodiment, the pipeline manager 3932... Figure 39A The scheduler 3910 receives instructions and manages the execution of these instructions via the graphics multiprocessor 3934 and / or texture unit 3936. In at least one embodiment, the graphics multiprocessor 3934 is an exemplary instance of a SIMT parallel processor. However, in at least one embodiment, the processing cluster 3994 may include various types of SIMT parallel processors with different architectures. In at least one embodiment, the processing cluster 3994 may include one or more instances of the graphics multiprocessor 3934. In at least one embodiment, the graphics multiprocessor 3934 can process data, and the data cross switch 3940 can be used to distribute the processed data to one of a number of possible destinations (including other shader units). In at least one embodiment, the pipeline manager 3932 can facilitate the distribution of processed data by specifying the destination of the processed data to be distributed via the data cross switch 3940.

[0364] In at least one embodiment, each graphics multiprocessor 3934 within the processing cluster 3994 may include the same set of functional execution logic (e.g., arithmetic logic units, load-memory units (LSUs), etc.). In at least one embodiment, the functional execution logic may be configured in a pipelined manner, wherein new instructions may be issued before previous instructions complete. In at least one embodiment, the functional execution logic supports a variety of operations, including integer and floating-point arithmetic, comparison operations, Boolean operations, shift operations, and computation of various algebraic functions. In at least one embodiment, the same functional unit hardware may be used to perform different operations, and any combination of functional units may exist.

[0365] In at least one embodiment, instructions transmitted to the processing cluster 3994 constitute threads. In at least one embodiment, a group of threads executed across a set of parallel processing engines is a thread group. In at least one embodiment, the thread group executes programs on different input data. In at least one embodiment, each thread within the thread group may be assigned to a different processing engine within the graphics multiprocessor 3934. In at least one embodiment, the thread group may include fewer threads than the number of processing engines within the graphics multiprocessor 3934. In at least one embodiment, when the number of threads included in the thread group is less than the number of processing engines, one or more processing engines may be idle during a loop that is processing the thread group. In at least one embodiment, the thread group may also include more threads than the number of processing engines within the graphics multiprocessor 3934. In at least one embodiment, when the thread group includes more threads than the number of processing engines within the graphics multiprocessor 3934, processing can be performed in consecutive clock cycles. In at least one embodiment, multiple thread groups can be executed simultaneously on the graphics multiprocessor 3934.

[0366] In at least one embodiment, the graphics multiprocessor 3934 includes an internal cache memory for performing load and store operations. In at least one embodiment, the graphics multiprocessor 3934 may forgo the internal cache and use a cache memory within the processing cluster 3994 (e.g., L1 cache 3948). In at least one embodiment, each graphics multiprocessor 3934 may also access partition units (e.g., ...). Figure 39A The L2 cache is located within partition units 3920A-3920N, which are shared among all processing clusters 3994 and can be used to transfer data between threads. In at least one embodiment, the graphics multiprocessor 3934 can also access off-chip global memory, which may include one or more of local parallel processor memory and / or system memory. In at least one embodiment, any memory outside of the parallel processing unit 3902 can be used as global memory. In at least one embodiment, the processing cluster 3994 includes multiple instances of the graphics multiprocessor 3934, which can share common instructions and data that can be stored in the L1 cache 3948.

[0367] In at least one embodiment, each processing cluster 3994 may include an MMU 3945 configured to map virtual addresses to physical addresses. In at least one embodiment, one or more instances of the MMU 3945 may reside in Figure 39AThe memory interface 3918 is located within the MMU 3945. In at least one embodiment, the MMU 3945 includes a set of page table entries (PTEs) for mapping virtual addresses to physical addresses of tiles (more information about tiles is discussed) and optionally to cache line indices. In at least one embodiment, the MMU 3945 may include an address translation back buffer (TLB) or a cache that may reside within the graphics multiprocessor 3934, the L1 cache 3948, or the processing cluster 3994. In at least one embodiment, physical addresses are processed to allocate surface data access locality for efficient request interleaving between partition units. In at least one embodiment, cache line indices may be used to determine whether a request for a cache line is a hit or a miss.

[0368] In at least one embodiment, the processing cluster 3994 can be configured such that each graphics multiprocessor 3934 is coupled to a texture unit 3936 to perform texture mapping operations, such as determining texture sample locations, reading texture data, and filtering texture data. In at least one embodiment, texture data is read as needed from an internal texture L1 cache (not shown) or from an L1 cache within the graphics multiprocessor 3934, and texture data is also retrieved from an L2 cache, local parallel processor memory, or system memory. In at least one embodiment, each graphics multiprocessor 3934 outputs a processed task to a data crossbar switch 3940 to provide the processed task to another processing cluster 3994 for further processing or to store the processed task in an L2 cache, local parallel processor memory, or system memory via a memory crossbar switch 3916. In at least one embodiment, a pre-raster operation unit (preROP) 3942 is configured to receive data from the graphics multiprocessor 3934 and direct the data to a ROP unit, which may be associated with a partitioning unit (e.g., [missing information]). Figure 39A The PreROP 3942 unit is located together with the partition units 3920A-3920N. In at least one embodiment, the PreROP 3942 unit can perform optimizations for color blending, organize pixel color data, and perform address translation.

[0369] Figure 39C A graphics multiprocessor 3996 according to at least one embodiment is illustrated. In at least one embodiment, the graphics multiprocessor 3996 is... Figure 39BThe graphics multiprocessor 3934 is described. In at least one embodiment, the graphics multiprocessor 3996 is coupled to the pipeline manager 3932 of the processing cluster 3994. In at least one embodiment, the graphics multiprocessor 3996 has an execution pipeline including, but not limited to, an instruction cache 3952, an instruction unit 3954, an address mapping unit 3956, a register file 3958, one or more GPGPU cores 3962, and one or more LSUs 3966. The GPGPU cores 3962 and LSUs 3966 are coupled to cache memory 3972 and shared memory 3970 via memory and cache interconnect 3968.

[0370] In at least one embodiment, instruction cache 3952 receives a stream of instructions to be executed from pipeline manager 3932. In at least one embodiment, instructions are cached in instruction cache 3952 and dispatched to instruction unit 3954 for execution. In one embodiment, instruction unit 3954 may dispatch instructions as thread groups (e.g., thread bundles), assigning each thread of the thread group to a different execution unit within GPGPU core 3962. In at least one embodiment, instructions can access any local, shared, or global address space by specifying an address within a unified address space. In at least one embodiment, address mapping unit 3956 may be used to translate addresses in the unified address space into different memory addresses that can be accessed by LSU 3966.

[0371] In at least one embodiment, register file 3958 provides a set of registers for the functional units of graphics multiprocessor 3996. In at least one embodiment, register file 3958 provides temporary storage for operands of data paths connected to functional units of graphics multiprocessor 3996 (e.g., GPGPU core 3962, LSU 3966). In at least one embodiment, register file 3958 is partitioned among each functional unit, such that a dedicated portion of register file 3958 is allocated to each functional unit. In at least one embodiment, register file 3958 is partitioned among different thread groups being executed by graphics multiprocessor 3996.

[0372] In at least one embodiment, each of the GPGPU cores 3962 may include an FPU and / or an ALU for executing instructions of the graph multiprocessor 3996. The GPGPU cores 3962 may be architecturally similar or may differ in architecture. In at least one embodiment, a first portion of the GPGPU core 3962 includes a single-precision FPU and an integer ALU, while a second portion of the GPGPU core includes a double-precision FPU. In at least one embodiment, the FPU may implement the IEEE 754-3908 standard for floating-point arithmetic or enable variable-precision floating-point arithmetic. In at least one embodiment, the graphics multiprocessor 3996 may additionally include one or more fixed-function or special-function units to perform specific functions, such as copying rectangles or pixel blending operations. In at least one embodiment, one or more of the GPGPU cores 3962 may also include fixed-function or special-function logic.

[0373] In at least one embodiment, the GPGPU core 3962 includes SIMD logic capable of executing a single instruction on multiple sets of data. In at least one embodiment, the GPGPU core 3962 can physically execute SIMD4, SIMD8, and SIMD16 instructions, and logically execute SIMD1, SIMD2, and SIMD32 instructions. In at least one embodiment, the SIMD instructions for the GPGPU core can be generated by a shader compiler at compile time, or automatically generated when executing a program written and compiled for a Single Program Multiple Data (SPMD) or SIMT architecture. In at least one embodiment, multiple threads of a program configured for a SIMT execution model can be executed using a single SIMD instruction. For example, in at least one embodiment, eight SIMD threads performing the same or similar operations can be executed in parallel using a single SIMD8 logic unit.

[0374] In at least one embodiment, the memory and cache interconnect 3968 is an interconnect network connecting each functional unit of the graphics multiprocessor 3996 to the register file 3958 and the shared memory 3970. In at least one embodiment, the memory and cache interconnect 3968 is a cross-switch interconnect that allows the LSU 3966 to perform load and store operations between the shared memory 3970 and the register file 3958. In at least one embodiment, the register file 3958 can operate at the same frequency as the GPGPU core 3962, resulting in very low latency for data transfer between the GPGPU core 3962 and the register file 3958. In at least one embodiment, the shared memory 3970 can be used to enable communication between threads executing on functional units within the graphics multiprocessor 3996. In at least one embodiment, for example, the cache memory 3972 can be used as a data cache to cache texture data communicated between functional units and texture units 3936. In at least one embodiment, the shared memory 3970 can also be used as a program-managed cache. In at least one embodiment, in addition to the automatically cached data stored in cache memory 3972, the thread executing on GPGPU core 3962 can also programmatically store data in shared memory.

[0375] In at least one embodiment, a parallel processor or GPGPU, as described herein, is communicatively coupled to the host / processor core to accelerate graphics operations, machine learning operations, pattern analysis operations, and various general-purpose GPU (GPGPU) functions. In at least one embodiment, the GPU may be communicatively coupled to the host processor / core via a bus or other interconnect (e.g., high-speed interconnects such as PCIe or NVLink). In at least one embodiment, the GPU may be integrated with the core on the same package or chip and communicatively coupled to the core via an internal processor bus / interconnect (i.e., within the package or chip). In at least one embodiment, regardless of how the GPU is connected, the processor core may assign work to the GPU in the form of a sequence of commands / instructions contained in the WD. In at least one embodiment, the GPU then uses dedicated circuitry / logic to efficiently process these commands / instructions.

[0376] General computing

[0377] The following figures illustrate, but are not limited to, exemplary software configurations used in general computing to implement at least one embodiment.

[0378] Figure 40A software stack of a programming platform according to at least one embodiment is illustrated. In at least one embodiment, the programming platform is a platform for accelerating computational tasks by utilizing hardware on a computing system. In at least one embodiment, software developers can access the programming platform through libraries, compiler instructions, and / or extensions to programming languages. In at least one embodiment, the programming platform may be, but is not limited to, CUDA, Radeon Open Computing Platform (“ROCm”), OpenCL (OpenCL developed by Khronosgroup). TM ), SYCL or Intel One API.

[0379] In at least one embodiment, the software stack 4000 of the programming platform provides an execution environment for the application 4001. In at least one embodiment, the application 4001 may include any computer software capable of being launched on the software stack 4000. In at least one embodiment, the application 4001 may include, but is not limited to, artificial intelligence (“AI”) / machine learning (“ML”) applications, high-performance computing (“HPC”) applications, virtual desktop infrastructure (“VDI”) or data center workloads.

[0380] In at least one embodiment, application 4001 and software stack 4000 run on hardware 4007. In at least one embodiment, hardware 4007 may include one or more GPUs, CPUs, FPGAs, AI engines, and / or other types of computing devices supporting a programming platform. In at least one embodiment, such as using CUDA, software stack 4000 may be vendor-specific and compatible only with devices from a specific vendor. In at least one embodiment, such as using OpenCL, software stack 4000 may be used with devices from different vendors. In at least one embodiment, hardware 4007 includes a host connected to one or more devices that can be accessed via application programming interface (API) calls to perform computational tasks. In at least one embodiment, compared to the host within hardware 4007, which may include, but is not limited to, a CPU (but may also include computing devices) and its memory, devices within hardware 4007 may include, but are not limited to, GPUs, FPGAs, AI engines, or other computing devices (but may also include CPUs) and their memory.

[0381] In at least one embodiment, the software stack 4000 of the programming platform includes, but is not limited to, multiple libraries 4003, a runtime 4005, and a device kernel driver 4006. In at least one embodiment, each library 4003 may include data and programming code that can be used by a computer program and utilized during software development. In at least one embodiment, library 4003 may include, but is not limited to, pre-written code and subroutines, classes, values, type specifications, configuration data, documentation, help data, and / or message templates. In at least one embodiment, library 4003 includes functions optimized for execution on one or more types of devices. In at least one embodiment, library 4003 may include, but is not limited to, functions for performing mathematical, deep learning, and / or other types of operations on the device. In at least one embodiment, library 3803 is associated with a corresponding API 3802, which may include one or more APIs that expose functions implemented in library 3803.

[0382] In at least one embodiment, application 4001 is written as source code, which is compiled into executable code, as follows: Figure 45 This will be discussed in more detail. In at least one embodiment, the executable code of application 4001 may run at least partially on an execution environment provided by software stack 4000. In at least one embodiment, during the execution of application 4001, code that needs to run on the device (compared to the host) may be obtained. In this case, in at least one embodiment, runtime 4005 may be invoked to load and start the necessary code on the device. In at least one embodiment, runtime 4005 may include any technically feasible runtime system capable of supporting the execution of application 4001.

[0383] In at least one embodiment, runtime 4005 is implemented as one or more runtime libraries associated with a corresponding API (which is shown as API 4004). In at least one embodiment, one or more such runtime libraries may include, but are not limited to, functions for memory management, execution control, device management, error handling and / or synchronization, etc. In at least one embodiment, memory management functions may include, but are not limited to, functions for allocating, dealing with, and copying device memory, and for transferring data between host memory and device memory. In at least one embodiment, execution control functions may include, but are not limited to, functions for launching functions on the device (sometimes referred to as "kernels" when the function is a global function that can be called from the host), and functions for setting attribute values ​​in buffers maintained by the runtime libraries for a given function to be executed on the device.

[0384] In at least one embodiment, the runtime library and the corresponding API 4004 can be implemented in any technically feasible manner. In at least one embodiment, one (or any number of) APIs may expose a low-level set of functions for fine-grained control of the device, while another (or any number of) APIs may expose such a higher-level set of functions. In at least one embodiment, a high-level runtime API can be built on top of the low-level APIs. In at least one embodiment, one or more runtime APIs may be language-specific APIs layered on top of language-independent runtime APIs.

[0385] In at least one embodiment, device kernel driver 4006 is configured to facilitate communication with the underlying device. In at least one embodiment, device kernel driver 4006 may provide APIs such as API 4004 and / or low-level functions upon which other software depends. In at least one embodiment, device kernel driver 4006 may be configured to compile intermediate representation (“IR”) code into binary code at runtime. In at least one embodiment, for CUDA, device kernel driver 4006 may compile non-hardware-specific parallel thread execution (“PTX”) IR code into binary code (cached compiled binary code) for a specific target device (sometimes referred to as “final” code) at runtime. In at least one embodiment, doing so allows the final code to run on the target device, which may not exist when the source code was initially compiled into PTX code. Alternatively, in at least one embodiment, the device source code may be compiled into binary code offline, without requiring device kernel driver 4006 to compile the IR code at runtime.

[0386] Figure 41 The illustration shows an embodiment according to at least one of the embodiments. Figure 40 The software stack 4000 is a CUDA implementation. In at least one embodiment, the CUDA software stack 4100 on which an application 4101 can be launched includes a CUDA library 4103, a CUDA runtime 4105, a CUDA driver 4107, and a device kernel driver 4108. In at least one embodiment, the CUDA software stack 4100 executes on hardware 4109, which may include a CUDA-enabled GPU developed by NVIDIA Corporation of Santa Clara, California.

[0387] In at least one embodiment, application 4101, CUDA runtime 4105, and device kernel driver 4108 can respectively perform functions similar to those of application 4001, runtime 4005, and device kernel driver 4006, in combination with the above. Figure 40The CUDA driver 4107 is described in at least one embodiment. In at least one embodiment, the CUDA driver API 4107 includes a library (libcuda.so) that implements the CUDA driver API 4106. In at least one embodiment, similar to the CUDA runtime API 4104 implemented by the CUDA runtime library (cudart), the CUDA driver API 4106 may expose, but is not limited to, functions for memory management, execution control, device management, error handling, synchronization, and / or graphics interoperability. In at least one embodiment, the CUDA driver API 4106 differs from the CUDA runtime API 4104 in that the CUDA runtime API 4104 simplifies device code management by providing implicit initialization, context (similar to processes) management, and module (similar to dynamically loaded libraries) management. In contrast to the high-level CUDA runtime API 4104, in at least one embodiment, the CUDA driver API 4106 is a low-level API that provides finer-grained control over the device, particularly regarding context and module loading. In at least one embodiment, the CUDA driver API 4106 may expose functions for context management that are not exposed by the CUDA runtime API 4104. In at least one embodiment, the CUDA driver API 4106 is also language-independent and supports, in addition to the CUDA runtime API 4104, OpenCL, for example. Furthermore, in at least one embodiment, development libraries, including the CUDA runtime 4105, can be considered separate from the driver components, including the user-mode CUDA driver 4107 and the kernel-mode device driver 4108 (sometimes also referred to as the "display" driver).

[0388] In at least one embodiment, CUDA library 4103 may include, but is not limited to, mathematical libraries, deep learning libraries, parallel algorithm libraries, and / or signal / image / video processing libraries, which parallel computing applications (e.g., application 4101) may utilize. In at least one embodiment, CUDA library 4103 may include mathematical libraries such as the cuBLAS library, which is an implementation of basic linear algebra subroutines (“BLAS”) for performing linear algebra operations; the cuFFT library for computing the Fast Fourier Transform (“FFT”); and the cuRAND library for generating random numbers, etc. In at least one embodiment, CUDA library 4103 may include deep learning libraries, such as the cuDNN library for primitives of deep neural networks and the TensorRT platform for high-performance deep learning inference, etc.

[0389] Figure 42 The illustration shows an embodiment according to at least one of the embodiments. Figure 40The software stack 4000 is a ROCm implementation. In at least one embodiment, the ROCm software stack 4200 on which the application 4201 can be launched includes a language runtime 4203, a system runtime 4205, a thunk 4207, a ROCm kernel driver 4208, and a device kernel driver 4209. In at least one embodiment, the ROCm software stack 4200 executes on hardware 4209, which may include a ROCm-enabled GPU developed by AMD Inc. of Santa Clara, California.

[0390] In at least one embodiment, application 4201 can perform actions in conjunction with the above. Figure 40 The discussed application 4001 has similar functionality. Additionally, in at least one embodiment, the language runtime 4203 and system runtime 4205 can perform functions combined with the above. Figure 40 The runtime 4005 discussed has similar functionality. In at least one embodiment, the language runtime 4203 and the system runtime 4205 differ in that the system runtime 4205 is a language-independent runtime that implements the ROCr System Runtime API 4204 and utilizes the Heterogeneous System Architecture (“HAS”) runtime API. In at least one embodiment, the HAS runtime API is a thin-user mode API that exposes interfaces for accessing and interacting with AMD GPUs, including functions for memory management, kernel execution control dispatched via architecture, error handling, system and agent information, and runtime initialization and shutdown, etc. In at least one embodiment, compared to the system runtime 4205, the language runtime 4203 is an implementation of a language-specific runtime API 4202 layered on top of the ROCr System Runtime API 4204. In at least one embodiment, the language runtime API may include, but is not limited to, the Portable Heterogeneous Computing Interface (“HIP”) language runtime API, the Heterogeneous Computing Compiler (“HCC”) language runtime API, or the OpenCL API, etc. In particular, the HIP language is an extension of the C++ programming language, a functionally similar version with CUDA mechanisms, and in at least one embodiment, the HIP language runtime API includes elements combined with the above. Figure 41 The functions discussed are similar to those in CUDA runtime API 4104, such as those used for memory management, execution control, device management, error handling, and synchronization.

[0391] In at least one embodiment, the thunk (ROCt) 4207 is an interface that can be used to interact with the underlying ROCm driver 4208. In at least one embodiment, the ROCm driver 4208 is a ROCk driver, which is a combination of an AMD GPU driver and a HAS kernel driver (amdkfd). In at least one embodiment, the AMD GPU driver is a device kernel driver for GPUs developed by AMD, which performs the above-described combination. Figure 41 The device kernel driver 4106 discussed has similar functionality. In at least one embodiment, the HAS kernel driver is a driver that allows different types of processors to share system resources more efficiently via hardware features.

[0392] In at least one embodiment, various libraries (not shown) may be included in the ROCm software stack 4200 above the language runtime 4203, and provide integration with the above. Figure 41 The discussed CUDA library 4103 has similar functionality. In at least one embodiment, various libraries may include, but are not limited to, mathematical, deep learning, and / or other libraries, such as the hipBLAS library which implements functions similar to CUDA cuBLAS, the rocFFT library which is similar to CUDA cuFFT for computing FFT, etc.

[0393] Figure 43 The illustration shows an embodiment according to at least one of the embodiments. Figure 40 The software stack 4000 is an OpenCL implementation. In at least one embodiment, the OpenCL software stack 4300 on which the application 4301 can be launched includes an OpenCL framework 4305, an OpenCL runtime 4306, and a driver 4307. In at least one embodiment, the OpenCL software stack 4300 executes on non-vendor-specific hardware 4109. In at least one embodiment, because devices developed by different vendors support OpenCL, specific OpenCL drivers may be required for interoperability with hardware from such vendors.

[0394] In at least one embodiment, the application 4301, the OpenCL runtime 4306, the device kernel driver 4307, and the hardware 4308 can respectively execute the above-described combination. Figure 40 The application 4001, runtime 4005, device kernel driver 4006, and hardware 4007 discussed have similar functionality. In at least one embodiment, application 4301 also includes an OpenCL kernel 4302 with code that will be executed on the device.

[0395] In at least one embodiment, OpenCL defines a "platform" that allows a host to control devices connected to that host. In at least one embodiment, the OpenCL framework provides a platform layer API and a runtime API, shown as Platform API 4303 and Runtime API 4305. In at least one embodiment, Runtime API 4305 uses a context to manage the execution of the kernel on the device. In at least one embodiment, each identified device can be associated with a respective context, which Runtime API 4305 can use to manage the device's command queue, program objects and kernel objects, shared memory objects, etc. In at least one embodiment, Platform API 4303 discloses functions that allow the device context to select and initialize devices, submit work to devices via command queues, and enable data transfers to and from devices, etc. Additionally, in at least one embodiment, the OpenCL framework provides various built-in functions (not shown), including mathematical functions, relational functions, and image processing functions, etc.

[0396] In at least one embodiment, compiler 4304 is also included in the OpenCL framework 4305. In at least one embodiment, the source code can be compiled offline before executing the application or online during application execution. Unlike CUDA and ROCm, the OpenCL application in at least one embodiment can be compiled online by compiler 4304, which is included to represent any number of compilers that can be used to compile source code and / or IR code (e.g., Standard Portable Intermediate Representation (“SPIR-V”) code) into binary code. Alternatively, in at least one embodiment, the OpenCL application can be compiled offline before executing such an application.

[0397] Figure 44 Software supported by a programming platform according to at least one embodiment is illustrated. In at least one embodiment, the programming platform 4404 is configured to support various programming models 4403, middleware and / or libraries 4402, and frameworks 4401 that an application 4400 may depend on. In at least one embodiment, the application 4400 may be an AI / ML application implemented using, for example, a deep learning framework (e.g., MXNet, PyTorch, or TensorFlow), which may depend on libraries such as cuDNN, the NVIDIA Collective Communications Library (“NCCL”), and / or the NVIDIA Developer Data Loading Library (“DALI”) CUDA library to provide accelerated computation on the underlying hardware.

[0398] In at least one embodiment, the programming platform 4404 can be a combination of the above-described components. Figure 41 , Figure 42 and Figure 43 One of the described CUDA, ROCm, or OpenCL platforms. In at least one embodiment, the programming platform 4404 supports multiple programming models 4403, which are abstractions of the underlying computing system that allow for the expression of algorithms and data structures. In at least one embodiment, the programming model 4403 may expose features of the underlying hardware to improve performance. In at least one embodiment, the programming model 4403 may include, but is not limited to, CUDA, HIP, OpenCL, C++ Accelerated Massive Parallelism (“C++AMP”), Open Multiprocessing (“OpenMP”), Open Accelerator (“OpenACC”), and / or Vulcan Compute.

[0399] In at least one embodiment, the library and / or middleware 4402 provides an abstract implementation of the programming model 4404. In at least one embodiment, such a library includes data and programming code that can be used by a computer program and utilized during software development. In at least one embodiment, in addition to those available from the programming platform 4404, such middleware also includes software that provides services to applications. In at least one embodiment, the library and / or middleware 4402 may include, but is not limited to, cuBLAS, cuFFT, cuRAND, and other CUDA libraries, or rocBLAS, rocFFT, rocRAND, and other ROCm libraries. Additionally, in at least one embodiment, the library and / or middleware 4402 may include NCCL and ROCm communication collection library (“RCCL”) libraries, which provide communication routines for GPUs, MIOpen libraries for deep learning acceleration, and / or intrinsic libraries for linear algebra, matrix and vector operations, geometric transformations, numerical solvers, and related algorithms.

[0400] In at least one embodiment, the application framework 4401 depends on libraries and / or middleware 4402. In at least one embodiment, each application framework 4401 is a software framework for implementing a standard structure of application software. In at least one embodiment, AI / ML applications can be implemented using frameworks such as Caffe, Caffe2, TensorFlow, Keras, PyTorch, or MxNet deep learning frameworks.

[0401] Figure 45 Compilation code according to at least one embodiment is shown to be used in Figures 40-43The application is executed on one of the programming platforms. In at least one embodiment, compiler 4501 receives source code 4500, which includes both host code and device code. In at least one embodiment, compiler 4501 is configured to convert source code 4500 into host executable code 4502 for execution on a host and device executable code 4503 for execution on a device. In at least one embodiment, source code 4500 may be compiled offline before executing the application or compiled online during application execution.

[0402] In at least one embodiment, source code 4500 may include code in any programming language supported by compiler 4501, such as C++, C, Fortran, etc. In at least one embodiment, source code 4500 may be included in a single-source file, which has a mixture of host code and device code, and indicates the location of the device code therein. In at least one embodiment, the single-source file may be a .cu file including CUDA code or a .hip.cpp file including HIP code. Alternatively, in at least one embodiment, source code 4500 may include multiple source code files instead of a single-source file, in which the host code and device code are separate.

[0403] In at least one embodiment, compiler 4501 is configured to compile source code 4500 into host executable code 4502 for execution on a host and device executable code 4503 for execution on a device. In at least one embodiment, compiler 4501 performs operations including resolving source code 4500 into an abstract system tree (AST), performing optimizations, and generating executable code. In at least one embodiment where source code 4500 comprises a single source file, compiler 4501 may separate device code and host code within such a single source file, compile the device code and host code into device executable code 4503 and host executable code 4502 respectively, and link device executable code 4503 and host executable code 4502 together in a single file, as described below. Figure 34 To be discussed in more detail.

[0404] In at least one embodiment, the host executable code 4502 and the device executable code 4503 can be in any suitable format, such as binary code and / or IR code. In the case of CUDA, in at least one embodiment, the host executable code 4502 may include native object code, while the device executable code 4503 may include code in a PTX intermediate representation. In at least one embodiment, in the case of ROCm, both the host executable code 4502 and the device executable code 4503 can include target binary code.

[0405] At least one embodiment of this disclosure may be described under the following terms:

[0406] 1. A network device, comprising:

[0407] At least one processor; and

[0408] At least one memory, comprising instructions that, in response to execution by the at least one processor, cause the network device to:

[0409] Receive first network data associated with a multicast operation to be performed jointly by at least multiple endpoints;

[0410] Resources of the network device are reserved to process second network data received from the plurality of endpoints, wherein the amount of resources to be reserved is determined at least in part based on information obtained from the header of the first network data.

[0411] The first network data is sent to a plurality of additional network devices, which are identified at least in part based on the information obtained from the header.

[0412] Receive the second network data; and

[0413] The reserved resources are used to process the second network data.

[0414] 2. The network device according to Clause 1, wherein the header includes information indicating a mapping between the first network data and the virtual memory space of the endpoint device.

[0415] 3. The network device according to clause 1 or 2, wherein one of the plurality of endpoints includes a parallel processing unit, and wherein at least a portion of the first network data is written into the memory of the parallel processing unit.

[0416] 4. A network device according to any one of clauses 1 to 3, wherein the first network data is sent to the network device in response to a write operation to the memory of a parallel processing unit, wherein the first network data includes data written to the memory by the write operation.

[0417] 5. The network device according to any one of clauses 1 to 4, wherein processing the second network data includes: reducing the second network data based at least in part on reduction information obtained from the header.

[0418] 6. The network device according to any one of clauses 1 to 5, wherein the network device reduces and transmits the second network data to the sender of the first network data.

[0419] 7. The network device according to any one of clauses 1 to 6, wherein the at least one memory includes further instructions that, in response to execution by the at least one processor, cause the network device to:

[0420] Store the information obtained from the header of the first network data; and

[0421] The information is retrieved in response to receiving the second network data.

[0422] 8. The network device according to any one of clauses 1 to 7, wherein the header includes reduction information and routing information for the multicast operation.

[0423] 9. A network device according to any one of clauses 1 to 8, wherein the at least one memory includes further instructions that, in response to execution by the at least one processor, cause the network device to:

[0424] Before sending the first network data to the plurality of additional network devices, the reduction information and routing information in the header are updated.

[0425] 10. The network device according to any one of clauses 1 to 9, wherein the plurality of additional network devices includes at least one of a switch, a router, or an endpoint.

[0426] 11. A network device according to any one of clauses 1 to 10, wherein the at least one memory includes further instructions that, in response to execution by the at least one processor, cause the network device to:

[0427] In response to determining that a threshold amount of time has elapsed since the first network data was sent, and that at least one of the additional network devices has not yet responded to receiving the first network data, the reserved resources are released.

[0428] 12. A non-transitory machine-readable medium having instructions stored thereon, the instructions, when executed by one or more processors, causing the one or more processors to at least:

[0429] Receive, at the network device, first network data associated with a multicast operation to be performed jointly by at least multiple endpoints;

[0430] Resources of the network device are reserved to process second network data received from the plurality of endpoints, wherein the resources to be reserved are determined at least in part based on information obtained from the first network data;

[0431] The first network data is sent to a plurality of additional network devices, which are identified at least in part based on the information obtained from the first network data;

[0432] Receive the second network data; and

[0433] The reserved resources are used to process the second network data.

[0434] 13. The non-transitory machine-readable medium as described in Clause 12, wherein the first network data includes one or more headers, the one or more headers including information indicating a mapping between the first network data and the virtual memory space of an endpoint device.

[0435] 14. The non-transitory machine-readable medium according to clause 12 or 13, wherein one of the plurality of endpoints includes a parallel processing unit, and wherein at least a portion of the first network data is used by the parallel processing unit to perform at least a portion of the multicast operation.

[0436] 15. A non-transitory machine-readable medium according to any one of clauses 12 to 14, wherein the network device receives first network data in response to at least one read or write operation to the memory of a parallel processing unit at an endpoint.

[0437] 16. A non-transitory machine-readable medium according to any one of clauses 12 to 15, wherein processing the second network data includes reducing the second network data based at least in part on reduction information obtained from one or more headers of the first network data.

[0438] 17. A non-transitory machine-readable medium according to any one of clauses 12 to 16, having stored thereon further instructions that, if executed by one or more processors, cause said one or more processors to at least:

[0439] Store the information obtained from the header of the first network data;

[0440] In response to receiving the second network data, the information is retrieved; and

[0441] The information is used to process the second network data.

[0442] 18. A non-transitory machine-readable medium according to any one of clauses 12 to 17, wherein one or more headers of the first network data include reduction information and routing information for the multicast operation.

[0443] 19. A non-transitory machine-readable medium according to any one of clauses 12 to 18, having stored thereon further instructions that, if executed by one or more processors, cause said one or more processors to at least:

[0444] Before sending the first network data to the plurality of additional network devices, the reduction information and routing information in the header are updated.

[0445] 20. A non-transitory machine-readable medium according to any one of clauses 12 to 19, having stored thereon further instructions that, if executed by one or more processors, cause said one or more processors to at least:

[0446] In response to determining that a threshold amount of time has elapsed since the first network data was sent, and that at least one of the additional network devices has not yet responded to the receipt of the first network data, the reserved resources are released.

[0447] 21. A method comprising:

[0448] Receive, at the network device, first network data associated with a multicast operation to be performed jointly by at least multiple endpoints;

[0449] Resources of the network device are reserved to process second network data received from the plurality of endpoints, wherein the resources to be reserved are determined at least in part based on information obtained from the first network data;

[0450] The first network data is transmitted from the network device to a plurality of additional network devices, which are identified at least in part based on information obtained from the first network data;

[0451] At the network device, the second network data is received; and

[0452] The network device uses the reserved resources to process the second network data.

[0453] 22. The method according to Clause 21, wherein the first network data includes one or more headers, the one or more headers including information indicating a mapping between the first network data and the virtual memory space of the endpoint device.

[0454] 23. The method according to clause 21 or 22, wherein one of the plurality of endpoints includes a parallel processing unit, and wherein at least a portion of the first network data is used by the parallel processing unit to perform at least a portion of the multicast operation.

[0455] 24. The method according to any one of clauses 2 to 23, wherein the network device receives first network data transmitted in response to at least one read or write operation on the memory of a parallel processing unit at an endpoint.

[0456] 25. The method according to any one of clauses 21 to 24, further comprising:

[0457] The second network data is processed at least in part based on reduction information obtained from one or more headers of the first network data.

[0458] 26. The method according to any one of clauses 21 to 25, further comprising:

[0459] Store the information obtained from the header of the first network data;

[0460] In response to receiving the second network data, the information is retrieved; and

[0461] The retrieved information is used to process the second network data.

[0462] 27. The method according to any one of clauses 21 to 26, wherein one or more headers of the first network data include reduction information and routing information for the multicast operation.

[0463] 28. The method according to any one of clauses 21 to 27, further comprising:

[0464] Before sending the first network data to the plurality of additional network devices, the reduction information and routing information in the header are updated.

[0465] 29. The method according to any one of clauses 21 to 28, further comprising:

[0466] In response to determining that a threshold amount of time has elapsed since the first network data was sent, and that at least one of the additional network devices has not yet responded to receiving the first network data, the reserved resources are released.

[0467] 30. The method according to any one of clauses 21 to 29, further comprising:

[0468] It is determined that the network device's resources are insufficient to process the second network data to be received from the plurality of endpoints; and

[0469] Hold the first network data until sufficient resources are available.

[0470] Other variations are within the spirit of this disclosure. Therefore, while the disclosed technology is readily adaptable to various modifications and alternative configurations, certain embodiments thereof are illustrated in the accompanying drawings and have been described in detail above. However, it should be understood that the disclosure is not intended to be limited to one or more specific forms disclosed, but rather, it is intended to cover all modifications, alternative configurations, and equivalents falling within the spirit and scope of this disclosure as defined in the appended claims.

[0471] Unless otherwise stated or obviously contradicted by the context, the terms “a,” “an,” and “the,” and similar references, used in the context of describing the disclosed embodiments (particularly in the context of the appended claims), should be interpreted as encompassing both singular and plural forms, rather than as definitions of the terms. Unless otherwise stated, the terms “comprising,” “having,” “including,” and “containing” should be interpreted as open-ended terms (meaning “including, but not limited to”). The term “connection” (referring to a physical connection where not modified) should be interpreted as partially or wholly contained, attached to, or joined together, even with some intervention. Unless otherwise indicated herein, references to numerical ranges herein are intended only as a way of abbreviating each individual value falling within that range, and each individual value is incorporated into the specification as if it were separately described herein. In at least one embodiment, unless otherwise indicated or contradicted by the context, the use of the terms “set” (e.g., “item set”) or “subset” should be interpreted as a non-empty set comprising one or more members. Furthermore, unless otherwise indicated or contradicted by the context, the term “subset” of the corresponding set does not necessarily mean an appropriate subset of the corresponding set, but rather that the subset and the corresponding set can be equal.

[0472] Unless otherwise explicitly stated or clearly contradicted by the context, connective phrases such as “at least one of A, B, and C” or “at least one of A, B, and C” are understood in the context to generally refer to items, terms, etc., which can be A or B or C, or any non-empty subset of the set A, B, and C. For example, in an illustrative example of a set with three members, the connective phrases “at least one of A, B, and C” and “at least one of A, B, and C” refer to any of the following sets: {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Therefore, such connective language is generally not intended to imply that some embodiments require the presence of at least one of A, at least one of B, and at least one of C. Additionally, unless otherwise stated or contradicted by the context, the term “multiple” indicates a plural state (e.g., “multiple items” means multiple items). In at least one embodiment, the number of items in the multiple items is at least two, but may be more if explicitly indicated or indicated by the context. Furthermore, unless otherwise stated or clearly understood from the context, the phrase “based on” means “at least partially based on” rather than “based on only”.

[0473] Unless otherwise indicated herein or clearly contradicted by the context, the operations of the processes described herein may be performed in any suitable order. In at least one embodiment, processes such as those described herein (or variations thereof and / or combinations thereof) are executed under the control of one or more computer systems configured with executable instructions and are implemented as code (e.g., executable instructions, one or more computer programs, or one or more application programs) that are executed jointly on one or more processors via hardware or a combination thereof. In at least one embodiment, the code is stored in the form of a computer program on a computer-readable storage medium, which in at least one embodiment includes a plurality of instructions executable by one or more processors. In at least one embodiment, the computer-readable storage medium is a non-transitory computer-readable storage medium that excludes transient signals (e.g., propagating transient electrical or electromagnetic transmissions) but includes non-transitory data storage circuitry (e.g., buffers, caches, and queues). In at least one embodiment, code (e.g., executable code or source code) is stored on one or more non-transitory computer-readable storage media (or other memory for storing executable instructions) on which executable instructions are stored, which, when executed by one or more processors of a computer system (i.e., as a result of execution), cause the computer system to perform the operations described herein. In at least one embodiment, the set of non-transitory computer-readable storage media comprises multiple non-transitory computer-readable storage media, and one or more of the individual non-transitory storage media lack all the code, but the multiple non-transitory computer-readable storage media collectively store all the code. In at least one embodiment, the executable instructions are executed such that different instructions are executed by different processors; in at least one embodiment, the non-transitory computer-readable storage media store the instructions, and the main central processing unit (“CPU”) executes some instructions while the graphics processing unit (“GPU”) executes other instructions. In at least one embodiment, different components of the computer system have separate processors, and the different processors execute different subsets of the instructions.

[0474] Therefore, in at least one embodiment, the computer system is configured to implement one or more services that perform the operations of the processes described herein, either individually or collectively, and such a computer system is configured with suitable hardware and / or software to enable the implementation of the operations. Furthermore, the computer system implementing at least one embodiment of this disclosure is a single device, and in another embodiment it is a distributed computer system comprising multiple devices operating in different ways, such that the distributed computer system performs the operations described herein, and that a single device does not perform all the operations.

[0475] The use of any and all examples or exemplary language (e.g., “such as”) provided herein is intended only to better illustrate embodiments of this disclosure and does not constitute a limitation on the scope of the disclosure unless otherwise required. No language in the specification should be construed as indicating that any unclaimed element is essential to the practice of the disclosure.

[0476] All references cited in this article, including publications, patent applications and patents, are incorporated herein by reference as if each reference were individually and specifically indicated to be incorporated herein by reference and the entire contents of which are described herein.

[0477] The terms “coupled” and “connected”, and their derivatives, may be used in the specification and claims. It should be understood that these terms may not be intended to be synonyms with each other. Rather, in certain examples, “connected” or “coupled” may be used to indicate that two or more elements are in direct or indirect physical or electrical contact with each other. “Coupled” may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other.

[0478] Unless otherwise expressly stated, it will be understood that throughout this specification, terms such as “processing,” “computing,” “determining,” etc., refer to the actions and / or processes of a computer or computing system or similar electronic computing device that process and / or convert data represented as physical quantities (e.g., electrons) in the registers and / or memory of the computing system into other data represented as physical quantities in the memory, registers, or other such information storage, transmission, or display devices of the computing system.

[0479] Similarly, the term "processor" can refer to any device or part of memory that processes electronic data from registers and / or memory and converts that electronic data into other electronic data that can be stored in registers and / or memory. As a non-limiting example, a "processor" can be a CPU or a GPU. A "computing platform" can include one or more processors. As used herein, in at least one embodiment, a "software" process can include software and / or hardware entities that perform work over time, such as tasks, threads, and intelligent agents. Likewise, each process can refer to multiple processes that execute instructions sequentially or intermittently, or in parallel. The terms "system" and "method" are used interchangeably herein, provided that a system can embody one or more methods, and a method can be considered a system.

[0480] In at least one embodiment, an arithmetic logic unit is a set of combinational logic circuits that takes one or more inputs to produce a result. In at least one embodiment, a processor uses an arithmetic logic unit to implement mathematical operations, such as addition, subtraction, or multiplication. In at least one embodiment, an arithmetic logic unit is used to implement logical operations, such as logical AND / OR or XOR. In at least one embodiment, an arithmetic logic unit is stateless and is made of physical switching components, such as semiconductor transistors, arranged to form logic gates. In at least one embodiment, an arithmetic logic unit may operate internally as a stateful logic circuit with an associated clock. In at least one embodiment, an arithmetic logic unit may be configured as an asynchronous logic circuit whose internal state is not maintained in an associated set of registers. In at least one embodiment, a processor uses an arithmetic logic unit to combine operands stored in one or more registers of the processor and produce an output that can be stored by the processor in another register or memory location.

[0481] In at least one embodiment, as a result of processing instructions retrieved by the processor, the processor presents one or more inputs or operands to the arithmetic logic unit (ALU), causing the ALU to produce a result at least partially based on instruction code provided to the ALU of the inputs. In at least one embodiment, the instruction code provided by the processor to the ALU is at least partially based on instructions executed by the processor. In at least one embodiment, combinational logic in the ALU processes the inputs and produces an output placed on a bus within the processor. In at least one embodiment, the processor selects a destination register, memory location, output device, or output storage location on the output bus to time the processor so that the result produced by the ALU is sent to the desired location.

[0482] In this document, reference may be made to obtaining, acquiring, receiving, or inputting analog or digital data into a subsystem, computer system, or computer-implemented machine. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog and digital data can be accomplished in various ways, such as by receiving data as a paramet...

Claims

1. A network device, comprising: At least one processor; as well as At least one memory, comprising instructions that, in response to execution by the at least one processor, cause the network device to: Receive first network data associated with a multicast operation to be performed jointly by at least multiple endpoints; Resources of the network device are reserved to process second network data received from the plurality of endpoints, wherein the amount of resources to be reserved is determined at least in part based on information obtained from the header of the first network data. The first network data is sent to a plurality of additional network devices, which are identified at least in part based on the information obtained from the header; Receive the second network data; as well as The reserved resources are used to process the second network data.

2. The network device of claim 1, wherein the header includes information indicating a mapping between the first network data and the virtual memory space of the endpoint device.

3. The network device of claim 1, wherein one of the plurality of endpoints includes a parallel processing unit, and wherein at least a portion of the first network data is written to the memory of the parallel processing unit.

4. The network device of claim 1, wherein the first network data is sent to the network device in response to a write operation to the memory of the parallel processing unit, wherein the first network data includes data written to the memory by the write operation.

5. The network device according to claim 1, wherein processing the second network data includes: The second network data is reduced at least in part based on the reduction information obtained from the header.

6. The network device according to claim 1, wherein the network device reduces and transmits the second network data to the sender of the first network data.

7. The network device of claim 1, wherein the at least one memory includes further instructions, the instructions being responsive to execution by the at least one processor, causing the network device to: Store the information obtained from the header of the first network data; and The information is retrieved in response to receiving the second network data.

8. The network device according to claim 1, wherein the header includes reduction information and routing information for the multicast operation.

9. The network device of claim 1, wherein the at least one memory includes further instructions, the instructions being responsive to execution by the at least one processor, causing the network device to: Before sending the first network data to the plurality of additional network devices, the reduction information and routing information in the header are updated.

10. The network device of claim 1, wherein the plurality of additional network devices includes at least one of a switch, a router, or an endpoint.

11. The network device of claim 1, wherein the at least one memory includes further instructions, the instructions being responsive to execution by the at least one processor, causing the network device to: In response to determining that a threshold amount of time has elapsed since the first network data was sent, and that at least one of the additional network devices has not yet responded to receiving the first network data, the reserved resources are released.

12. A non-transitory machine-readable medium having instructions stored thereon, the instructions being responsive to execution by one or more processors to cause the one or more processors to at least: Receive, at the network device, first network data associated with a multicast operation to be performed jointly by at least multiple endpoints; Resources of the network device are reserved to process second network data received from the plurality of endpoints, wherein the resources to be reserved are determined at least in part based on information obtained from the first network data; The first network data is sent to a plurality of additional network devices, which are identified at least in part based on the information obtained from the first network data; Receive the second network data; as well as The reserved resources are used to process the second network data.

13. The non-transitory machine-readable medium of claim 12, wherein the first network data includes one or more headers, the one or more headers including information indicating a mapping between the first network data and the virtual memory space of an endpoint device.

14. The non-transitory machine-readable medium of claim 12, wherein one of the plurality of endpoints includes a parallel processing unit, and wherein at least a portion of the first network data is used by the parallel processing unit to perform at least a portion of the multicast operation.

15. The non-transitory machine-readable medium of claim 12, wherein the network device receives first network data transmitted in response to at least one read or write operation on the memory of a parallel processing unit at an endpoint.

16. The non-transitory machine-readable medium of claim 12, wherein processing the second network data comprises: The second network data is reduced at least in part based on reduction information obtained from one or more headers of the first network data.

17. The non-transitory machine-readable medium of claim 12, having stored thereon further instructions, which, if executed by one or more processors, cause the one or more processors to at least: Store the information obtained from the header of the first network data; In response to receiving the second network data, the information is retrieved; and The information is used to process the second network data.

18. The non-transitory machine-readable medium of claim 12, wherein one or more headers of the first network data include reduction information and routing information for the multicast operation.

19. The non-transitory machine-readable medium of claim 12, having stored thereon further instructions, which, if executed by one or more processors, cause the one or more processors to at least: Before sending the first network data to the plurality of additional network devices, update the reduction information and routing information in the header.

20. The non-transitory machine-readable medium of claim 12, having stored thereon further instructions, which, if executed by one or more processors, cause the one or more processors to at least: The network data associated with the multicast operation is discarded, at least in part, based on the determination that a threshold amount of time has elapsed since the first network data was sent and that at least one of the additional network devices has not yet responded to the receipt of the first network data.

21. A network data processing method, comprising: Receive, at the network device, first network data associated with a multicast operation to be performed jointly by at least multiple endpoints; Resources of the network device are reserved to process second network data received from the plurality of endpoints, wherein the resources to be reserved are determined at least in part based on information obtained from the first network data; The first network data is transmitted from the network device to a plurality of additional network devices, which are identified at least in part based on information obtained from the first network data; The second network data is received at the network device. as well as The network device uses the reserved resources to process the second network data.

22. The method of claim 21, wherein the first network data includes one or more headers, the one or more headers including information indicating a mapping between the first network data and the virtual memory space of the endpoint device.

23. The method of claim 21, wherein one of the plurality of endpoints includes a parallel processing unit, and wherein at least a portion of the first network data is used by the parallel processing unit to perform at least a portion of the multicast operation.

24. The method of claim 21, wherein the network device receives first network data transmitted in response to at least one read or write operation on the memory of a parallel processing unit at an endpoint.

25. The method of claim 21, further comprising: The second network data is processed at least in part based on reduction information obtained from one or more headers of the first network data.

26. The method of claim 21, further comprising: Store the information obtained from the header of the first network data; Retrieve the information in response to receiving the second network data; as well as The retrieved information is used to process the second network data.

27. The method of claim 21, further comprising: Identify periods in the topology; as well as The resources are reserved at least in part based on the identification of the cycle.

28. The method of claim 21, further comprising: Before sending the first network data to the plurality of additional network devices, update the reduction information and routing information in the header.

29. The method of claim 21, further comprising: In response to determining that a threshold amount of time has elapsed since the first network data was sent, and that at least one of the additional network devices has not yet responded to receiving the first network data, the reserved resources are released.

30. The method of claim 21, further comprising: It is determined that the network device has insufficient resources to process the second network data to be received from the plurality of endpoints; as well as Hold the first network data until sufficient resources are available.

Images