Communication apparatus for generating and eliminating redundant data packets

Abstract

Communications apparatus that generates and eliminates redundant data packets is disclosed, and specifically apparatus, systems, and techniques that eliminate redundant data packets are disclosed. In at least one embodiment, a communications apparatus generates redundant data packets and transmits them in multiple packet streams. In at least one embodiment, a communications apparatus eliminates redundant data packets from received packet streams.

Description

TECHNICAL FIELD

[0001] At least one embodiment relates to generating and eliminating redundant data packets. For example, at least one embodiment relates to a communication device that generates redundant data packets, transmits the redundant data packets in multiple packet streams, and eliminates redundant data packets from a received packet stream. As another example, at least one embodiment relates to a communication device that implements various novel techniques described herein. BACKGROUND

[0002] Devices (e.g., computers, autonomous vehicles, etc.) often stream data packets over a network using a high-speed method. This high-speed method often uses an unreliable protocol to transmit packets, where the receiving device does not send acknowledgements to the sending device and / or the sending device does not retransmit lost or corrupted packets to the receiving device. To help increase the probability that the receiving device receives all of the packets, the sending device can transmit the packets and copies of the packets in multiple parallel data streams to the receiving device. Because the data streams are transmitted over the network on multiple different paths, the likelihood that the receiving device receives at least one copy of each packet (or equivalent packet) is increased.

[0003] Even when multiple parallel data streams are sent to the receiving device, it is not guaranteed that all of the packets in a particular stream will arrive at the receiving device, or that the packets will arrive at the receiving device in the same order that the packets were transmitted. Therefore, the receiving device must reconstruct the transmitted data by storing, reordering, processing, and interleaving the packets from two (or more) received data streams. This approach provides reliable data transmission, but uses a large amount of memory, memory bandwidth, and connection (e.g., Peripheral Component Interconnect Express (“PCIe”) bus) bandwidth, and places a heavy processing burden on the processor of the sending device, which can increase data latency and can limit data throughput and / or performance of the overall system. BRIEF DESCRIPTION OF DRAWINGS

[0004] Figure 1 An example of a system including a sending device connected to a receiving device over a network is shown, in accordance with at least one embodiment;

[0005] Figure 2 An example set of components of a sending communication device is shown, in accordance with at least one embodiment;

[0006] Figure 3 An example packet format is shown, in accordance with at least one embodiment;

[0007] Figure 4 An example set of components of a receiving communication device is shown, in accordance with at least one embodiment;

[0008] Figure 5 A block diagram illustrating a data communication system is shown schematically, in accordance with at least one embodiment;

[0009] Figure 6 A block diagram illustrating a data reconstruction process that can be performed by a recipient communication device is shown schematically, in accordance with at least one embodiment;

[0010] Figure 7 A block diagram illustrating details of a method for directly placing packet data performed by a recipient communication device is shown schematically, in accordance with at least one embodiment;

[0011] Figure 8 A block diagram illustrating a data communication system for transmitting radio data is shown schematically, in accordance with at least one embodiment;

[0012] Figure 9 A block diagram illustrating a method for directly placing packet data in a buffer in a recipient memory is shown schematically, in accordance with at least one embodiment;

[0013] Figure 10 A flowchart of a method that can be performed by a sender device, in accordance with at least one embodiment, is shown;

[0014] Figure 11 A flowchart of a method that can be performed by a recipient device, in accordance with at least one embodiment, is shown;

[0015] Figure 12 A distributed system is shown, in accordance with at least one embodiment;

[0016] Figure 13 An exemplary data center is shown, in accordance with at least one embodiment;

[0017] Figure 14 A client-server network is shown, in accordance with at least one embodiment;

[0018] Figure 15 A computer network is shown, in accordance with at least one embodiment;

[0019] Figure 16A A networked computer system is shown, in accordance with at least one embodiment;

[0020] Figure 16B A networked computer system is shown, in accordance with at least one embodiment;

[0021] Figure 16C A networked computer system is shown, in accordance with at least one embodiment;

[0022] Figure 17One or more components of a system environment in which services can be offered as third party network services are illustrated in accordance with at least one embodiment;

[0023] Figure 18 A cloud computing environment is illustrated in accordance with at least one embodiment;

[0024] Figure 19 A set of functional abstraction layers offered by a cloud computing environment is illustrated in accordance with at least one embodiment;

[0025] Figure 20 A supercomputer at the chip level is illustrated in accordance with at least one embodiment;

[0026] Figure 21 A supercomputer at the rack module level is illustrated in accordance with at least one embodiment;

[0027] Figure 22 A supercomputer at the rack level is illustrated in accordance with at least one embodiment;

[0028] Figure 23 A supercomputer at the entire system level is illustrated in accordance with at least one embodiment;

[0029] Figure 24A Inference and / or training logic is illustrated in accordance with at least one embodiment;

[0030] Figure 24B Inference and / or training logic is illustrated in accordance with at least one embodiment;

[0031] Figure 25 Training and deployment of a neural network is illustrated in accordance with at least one embodiment;

[0032] Figure 26 An architecture of a network system is illustrated in accordance with at least one embodiment;

[0033] Figure 27 An architecture of a network system is illustrated in accordance with at least one embodiment;

[0034] Figure 28 A control plane protocol stack is illustrated in accordance with at least one embodiment;

[0035] Figure 29 A user plane protocol stack is illustrated in accordance with at least one embodiment;

[0036] Figure 30 Components of a core network are illustrated in accordance with at least one embodiment;

[0037] Figure 31 Components of a system that supports network function virtualization (NFV) are illustrated in accordance with at least one embodiment;

[0038] Figure 32 A processing system is shown in accordance with at least one embodiment;

[0039] Figure 33 A computer system is shown in accordance with at least one embodiment;

[0040] Figure 34 A system is shown in accordance with at least one embodiment;

[0041] Figure 35 An exemplary integrated circuit is shown in accordance with at least one embodiment;

[0042] Figure 36 A computing system is shown in accordance with at least one embodiment;

[0043] Figure 37 An APU is shown in accordance with at least one embodiment;

[0044] Figure 38 A CPU is shown in accordance with at least one embodiment;

[0045] Figure 39 An exemplary accelerator integration slice is shown in accordance with at least one embodiment;

[0046] Figures 40A-40B An exemplary graphics processor is shown in accordance with at least one embodiment;

[0047] Figure 41A A graphics core is shown in accordance with at least one embodiment;

[0048] Figure 41B A GPGPU is shown in accordance with at least one embodiment;

[0049] Figure 42A A parallel processor is shown in accordance with at least one embodiment;

[0050] Figure 42B A processing cluster is shown in accordance with at least one embodiment;

[0051] Figure 42C A graphics multiprocessor is shown in accordance with at least one embodiment;

[0052] Figure 43 A software stack of a programming platform is shown in accordance with at least one embodiment;

[0053] Figure 44 A CUDA implementation of the software stack of Figure 43 is shown in accordance with at least one embodiment;

[0054] Figure 45 A software stack of a programming platform is shown in accordance with at least one embodiment;Figure 43 ROCm implementation of the software stack of

[0055] Figure 46 illustrates a software stack supported by a programming platform in accordance with at least one embodiment Figure 43 OpenCL implementation of the software stack of

[0056] Figure 47 illustrates software supported by a programming platform in accordance with at least one embodiment; and

[0057] Figure 48 illustrates compiled code for execution on a programming platform of Figures 43-46 in accordance with at least one embodiment. DETAILED DESCRIPTION

[0058] Figure 1 illustrates an example of a system 100 that includes a sender device 102 connected to a recipient device 104 through a network 106, in accordance with at least one embodiment. One or both of the sender device 102 and the recipient device 104 can be implemented by one or more computing devices, virtual machines, autonomous vehicles, video cameras, broadcast television devices, one or more sensor devices, and the like. The sender device 102 includes a sender communication apparatus 112, a sender processor 114 (e.g., a central processing unit (“CPU”)), and a sender memory 116. The recipient device 104 includes a recipient communication apparatus 124, a recipient processor 126 (e.g., a CPU), and a recipient memory 128. The sender communication apparatus 112 and the recipient communication apparatus 124 can each be implemented as a network interface controller (“NIC”), a network interface card, a network adapter, a local area network (“LAN”) adapter, a physical network interface, a host channel adapter (“HCA”), an Ethernet NIC, and the like. Optionally, the sender communication apparatus 112 and / or the recipient communication apparatus 124 can be connected to multiple host devices and can simultaneously send and receive packet traffic or streams for each host device.

[0059] The sender processor 114 sends a message 130 or a link to the message 130 to the sender communication apparatus 112. The message 130 can include multiple packets 132, or it can be packetized into multiple packets 132 by the sender communication apparatus 112. The multiple packets 132 have an original order. In Figure 1In the illustrated embodiment, the sender communication device 112 is shown duplicating the plurality of modified packets 134 into a single plurality of duplicate packets 136, but in alternative embodiments, the sender communication device 112 can duplicate the plurality of modified packets 134 two or more times to create multiple series of duplicate packets. In the illustrated embodiment, the packets P1-P3 are modified to create packets P1'-P3', respectively, and the modified packets P1'-P3' are duplicated to create duplicate packets D1-D3, respectively. The sender communication device 112 then communicates a sent packet 138S to the recipient device 104 that includes the plurality of modified packets 134S and the plurality of duplicate packets 136S sent over the network 106 on different communication paths, respectively. In other words, the sender communication device 112 communicates the sent packet 138S in two or more separate streams to the recipient communication device 124.

[0060] The recipient communication device 124 receives a packet 138R that includes a received plurality of modified packets 134R and a received plurality of duplicate packets 136R. In other words, the recipient communication device 124 receives from the sender communication device 112 two or more separate streams that include received packets 138R. As will be described below, each packet 138S includes a packet identifier, and the received packets 138R can include two or more packets with the same packet identifier. The recipient communication device 124 retains at most a single packet with each unique packet identifier (e.g., the first received packet), and can discard the remaining received packets 138R. In at least one embodiment, the payload can include and / or serve as the packet identifier. In such an embodiment, the recipient communication device 124 retains at most a single packet with each payload (e.g., the first received packet), and can discard the remaining received packets 138R.

[0061] The received plurality of modified packets 134R can differ from the transmitted plurality of modified packets 134S because one or more packets can have been lost or corrupted during transmission. Further, the transmitted plurality of modified packets 134S has a first transmission order, and the received plurality of modified packets 134R has a first reception order. The order of the transmitted plurality of modified packets 134S can change during transmission such that the first reception order differs from the first transmission order. In the illustrated example, the received plurality of modified packets 134R includes packets PI' and P3' received in that order, with packet P2' having been lost during transmission. Similarly, the received plurality of duplicate packets 136R can differ from the transmitted plurality of duplicate packets 136S because one or more packets can have been lost or corrupted during transmission. Further, the transmitted plurality of duplicate packets 136S has a second transmission order, and the received plurality of duplicate packets 136R has a second reception order. The order of the transmitted plurality of duplicate packets 136S can change during transmission such that the second reception order differs from the second transmission order. In the illustrated example, the received plurality of duplicate packets 136R includes packets D2, DI, and D3 received in that order, with packet D3 having been corrupted during transmission.

[0062] As described above, the recipient communication device 124 retains at most a single instance of each received packet 138R to obtain a plurality of modified received packets 140 and discards the remaining received packets 138R. For example, the recipient communication device 124 can retain a first uncorrupted instance of each received packet 138R. Optionally, the recipient communication device 124 reorders the single instance of the received packets 138R to match the original order of the plurality of packets 132 as closely as possible. For example, the recipient communication device 124 can process each packet individually and build an ordered data set (e.g., implemented as a sliding window). The ordered data set expects a predetermined number of packets with a set of particular packet identifiers. The recipient communication device 124 can pass a first packet received with each particular packet identifier and discard or ignore subsequent packets received with a duplicate packet identifier or with a packet identifier that is not one of the particular packet identifiers. Each first packet received is placed in a packet position in the ordered data set in the original order. The original order can be indicated by the packet identifiers. For example, the packet identifiers can be sequence values, each value specifying a position within the original order and a position in or corresponding to the ordered data set. A particular position in the ordered data set can be empty if no packet with a packet identifier corresponding to the particular position is received. When a packet is received, a first packet with one of the particular packet identifiers is placed in the ordered data set in a position corresponding to the packet identifier of the packet. In the illustrated example, the recipient communication device 124 saves packets P1', D2, P3' as the plurality of modified received packets 140.

[0063] The ordered data set (e.g., implemented as a sliding window) can not have enough positions for all first packets received by the recipient communication device 124. Thus, the ordered data set can be at least partially emptied (e.g., after receiving a predetermined number of packets) and reset to expect a subsequent set of particular packet identifiers. In this way, the recipient communication device 124 orders the first packets as they are received and creates the modified received packets 140. For example, the ordered data set can expect a set of particular packet identifiers including values 7-12. When the ordered data set has received a first packet with a minimum of the particular packet identifier values (e.g., packet identifier value 7), the packet can be removed from the ordered data set, placed in a modified received packet 140, and the ordered data set reset to expect particular packet identifiers including values 8-13.

[0064] The receiving communication device 124 then removes the modifications from the plurality of modified received packets 140 to obtain a plurality of reconstructed packets 142. In the illustrated example, the plurality of reconstructed packets 142 includes packets R1-R3. The receiving communication device 124 reconstructs a reconstructed message 144 stored in the receiving memory 128 in a buffer 150. The reconstructed message 144 includes information from the message 130. If at least one uncorrupted copy of each of the sent packets 138S is received, the reconstructed message 144 can be the same copy of the message 130. Thus, in at least one embodiment, the plurality of reconstructed packets 142 is the same as the plurality of packets 132. In at least one embodiment, the receiving communication device 124 can depacketize the plurality of reconstructed packets 142 to create the reconstructed message 144.

[0065] Figure 2 Example components of the sending communication device 112 are shown in accordance with at least one embodiment. The sending communication device 112 can be implemented by packet processing circuitry or other hardware performing functions attributed to the sending communication device 112. Referring to Figure 2 , the sending communication device 112 includes a sending host interface 202, a packet identifier engine 204 (e.g., a packet sequence number (“PSN”) engine), a sending steering engine 206, a packet replication engine 208, and a sending network interface 210 connected to the network 106. The sending network interface 210 includes a plurality of sending connections 212 and 214 (e.g., ports), each connected to the network 106.

[0066] The sending processor 114 executes instructions, including in this example a data processing application 224 (e.g., a video processing application) and a driver 266 for controlling the operation of the sending communication device 112. The instructions can be stored by the sending memory 116. The sending host interface 202 is connected to the sending processor 114 and the sending memory 116 by a sending bus 228 (e.g., a PCIe bus). The sending host interface 202 receives the message 130 from the sending processor 114 over the sending bus 228 and packetizes the message 130 into the plurality of packets 132 according to one or more desired transport protocols (e.g., IEEE 802.1CB, SMPTE 2022-7, etc.). The plurality of packets 132 can include duplicate packets, depending on implementation details.

[0067] Figure 3 Example packet formats 300 and 320 are shown in accordance with at least one embodiment. The packet formats 300 and 320 are non-limiting examples of packet formats that are compliant with the IEEE 802.1CB standard. The format 300 can be used to create each of the plurality of packets 132. Referring to Figure 3Format 300 can include a destination address field 302, a source address field 304, a virtual local area network ("VLAN") field 306, and a data or payload field 308. The destination address field 302, the source address field 304, and the VLAN field 306 together are a packet header 310 (e.g., a transport protocol-specific header). The packet header 310 identifies a unique flow or packet stream. The destination address field 302 and the source address field 304 can each store a media access control ("MAC") address.

[0068] The packet processing circuit 220 is coupled between the sender host interface 202 and the sender network interface 210. For simplicity, Figure 2 Only certain specific elements of the packet processing circuit 220 (e.g., the packet identifier engine 204, the sender steering engine 206, and the packet replication engine 208) are shown for creating the plurality of modified packets 134 and the plurality of replicated packets 136. However, these elements can be combined with other components for sending and receiving packets for other purposes and according to other types of protocols. The packet processing circuit 220 is typically implemented in hardware logic circuitry within one or more integrated circuit chips. The logic can be hardwired or programmable, or a combination of hardwired and programmable elements. Additionally or alternatively, certain functions of the packet processing circuit 220 can be implemented in software or firmware executed by an embedded microprocessor or microcontroller.

[0069] The sender host interface 202 forwards the plurality of packets 132 to the packet identifier engine 204 (e.g., an Internet Protocol Security ("IPsec") engine), which provides a plurality of packet identifiers 222 to the sender steering engine 206 along with the plurality of packets 132. The plurality of packet identifiers 222 includes a unique packet identifier for each of the plurality of packets 132. The packet identifier engine 204 can generate the packet identifiers by incrementing a previous packet number (starting from a base or initial packet identifier) by an increment (e.g., 1) for each packet.

[0070] The sender steering engine 206 classifies the relevant packets and performs actions on the packets according to configurable rules and tables. The sender steering engine 206 determines the path of the packets through the packet processing circuit 220. The sender steering engine 206 can add one or more transport protocol-specific headers and a redundancy tag ("RTAG") or redundancy header to each of the plurality of packets 132 according to the rules, thereby modifying the plurality of packets 132 and creating the plurality of modified packets 134. The redundancy header of each of the plurality of modified packets 134 includes the packet identifier of the modified packet.

[0071] Returning to Figure 2In many real-time streaming protocols, such as the Real-time Transport Protocol (“RTP”) and IEEE 802.1CB, each packet in each transport stream contains a respective sequence number, such as a PSN, which can be used at the recipient device 104 to detect lost packets and restore packet data to transmission order. At the recipient device 104, these reordering and reconstruction functions can be performed by the recipient communication means 124 rather than the recipient processor 126 (see Figure 1 , Figure 4 and Figure 8 ). Although these figures illustrate a single data communication sent in multiple streams, the recipient device 104 can concurrently receive and process many such data communications from one or more different sources. As non-limiting examples, the message 130 can include one or more video frames packaged by the sender communication means 112 into video packets. In such embodiments, the multiple packets 132 are video packets, and the multiple packet identifiers 222 can be implemented as PSNs (e.g., generated according to IEEE 802.1CB) assigned to the video packets and stored in the packet identifier fields 330 (see Figure 3 ) of the multiple modified packets 134. Further, the one or more video frames can be divided into several slices, each stored in the payload field 308 (see Figure 3 ) of one of the video packets. The slices can each have a predefined fixed size. By other non-limiting examples, the multiple packet identifiers 222 can be implemented as other types of sequence numbers (e.g., generated according to other standards) that can be stored in the packet identifier fields 330 (see Figure 3 ) of the packet headers 324, and each payload field 308 can contain a slice of data pieces having a predefined per-slice fixed size.

[0072] The sender steering engine 206 forwards the multiple modified packets 134 to the packet replication engine 208. The packet replication engine 208 replicates each of the multiple modified packets 134 a predetermined number of times, and transmits each of the multiple modified packets 134S and its copies (e.g., the multiple replicated packets 136S) to the recipient communication means 124 via the multiple sender connections 212 and 214. Thus, the sender communication means 112 transmits multiple packet streams, respectively on the multiple sender connections 212 and 214, each of the multiple packet streams conveying the message 130. The multiple sender connections 212 and 214 (e.g., ports) can have different addresses, which are used as respective source addresses for the packet streams. Alternatively, the streams can be transmitted over the same port. Optionally, the packet processing circuitry 220 can include an encryption component that encrypts the multiple modified packets 134 before they are replicated by the packet replication engine 208.

[0073] Referring Figure 3 The modified format 320 can be used to create each of the plurality of modified packets 134, and the plurality of replicated packets 136 can each also have the format 320. The modified format 320 can include a destination address field 302, a source address field 304, a VLAN field 306, a payload field 308, and a redundancy header 322 (e.g., an RTAG header). The destination address field 302, the source address field 304, the VLAN field 306, and the redundancy header 322 together are a modified packet header 324. As described above, the values stored in the destination address field 302, the source address field 304, and the VLAN field 306 identify a unique flow or packet stream. Thus, at least one of these fields will be different in the modified packet headers 324 of the plurality of modified packets 134R and the plurality of replicated packets 136R. The packet replication engine 208 modifies the source address stored in the source address field 304, the destination address stored in the destination address field 302, and / or the VLAN identifier stored in the VLAN field 306 in the plurality of replicated packets 136 so that the plurality of modified packets 134 and the plurality of replicated packets 136 are sent along different paths through the network 106, and the modified packet headers 324 in the plurality of replicated packets 136R are different from the plurality of modified packets 134R. In the illustrated embodiment, the redundancy header 322 includes an EtherType field 326, a reserved portion 328, and a packet identifier field 330 (e.g., to store a packet sequence number (“PSN”) value). The EtherType field 326 can identify the protocol encapsulated in the payload field 308 and can be used by the recipient device 104 (see FIG. 1) to determine how to process the payload. The reserved portion 328 can be used to store other information, such as a timestamp, a sequence number, or other information. The packet identifier field 330 can store a value that identifies the packet stream or flow. In the illustrated embodiment, the packet identifier field 330 stores a PSN value. The PSN value can be used by the recipient device 104 to determine how to process the payload field 308. For example, the PSN value can be used to determine the order in which the payload field 308 should be processed. In the illustrated embodiment, the PSN value is a 32-bit value. In other embodiments, the PSN value can be a different size. In other embodiments, the packet identifier field 330 can store a different value that identifies the packet stream or flow. Figure 1 and Figure 4 ) to determine how to process the payload.

[0074] The packet processing circuit 220 can reduce or eliminate jitter (or delay) between two or more packets having the same packet identifier because the plurality of replicated packets 136R are not created until just before they are sent to the recipient device 104. Thus, the sent packets 138S can be sent over the plurality of sender connections 212 and 214 (e.g., ports), and the delay between those packets having the same packet identifier will be low or minimal (e.g., at most 100 nanoseconds between packets having the same packet identifier).

[0075] Figure 4Example components of a recipient communication device 124 are shown, in accordance with at least one embodiment. The recipient communication device 124 can be implemented by packet processing circuitry or other hardware that performs the functions attributed to the recipient communication device 124. The recipient processor 126 executes instructions, which in this example include a data processing application 430 (e.g., a video processing application) and a driver 432 for controlling the operation of the recipient communication device 124. The instructions can be stored by the recipient memory 128.

[0076] With reference to Figure 4 The recipient communication device 124 includes a recipient host interface 402, a dispersion engine 404, an anti-replay engine 406, a recipient boot engine 408, a packet parsing engine 410, and a recipient network interface 412 connected to the network 106. The recipient host interface 402 is connected to the recipient processor 126 and the recipient memory 128 by a recipient bus 418 (e.g., a PCIe bus).

[0077] The recipient network interface 412 includes a plurality of recipient connections 414 and 416 (e.g., ports) each connected to the network 106. Each of the packet streams (the plurality of modified packets 134R and the plurality of replicated packets 136R) is received by the recipient network interface 412. For example, the packet streams can be received by the plurality of recipient connections 414 and 416, respectively. The recipient host interface 402 is connected to the recipient network interface 412, which is connected to the network 106. The plurality of recipient connections 414 and 416 (e.g., ports) can have different addresses that are used as respective destination addresses for the packet streams. For example, the recipient connections 414 and 416 can receive the plurality of modified packets 134R and the plurality of replicated packets 136R, respectively. Alternatively, the streams can be received through the same port.

[0078] The packet processing circuitry 420 is coupled between the recipient host interface 402 and the recipient network interface 412. For simplicity, Figure 4 Only certain specific elements of the packet processing circuitry 420 (e.g., the dispersion engine 404, the anti-replay engine 406, the recipient boot engine 408, and the packet parsing engine 410) are shown for creating the reconstructed message 144. However, these elements can be combined with other components for sending and receiving packets for other purposes and according to other types of protocols. The packet processing circuitry 420 is typically implemented in hardware logic circuitry within one or more integrated circuit chips. The logic can be hard-wired or programmable, or a combination of hard-wired and programmable elements. Additionally or alternatively, certain functions of the packet processing circuitry 420 can be implemented in software or firmware executed by an embedded microprocessor or microcontroller. The packet processing circuitry 420 has a separate instance for each stream that the packet processing circuitry 420 receives and processes.

[0079] The receiver network interface 412 forwards the received packet 138R to the packet parsing engine 410. The packet parsing engine 410 (e.g., FlexParser) is a hardware engine that works with a conventional or native parser (not shown). The packet parsing engine 410 parses the flexible packet format (e.g., only partially unknown at tapeout) so that steering and other stateless offloading can be performed (e.g., by the receiver steering engine 408) on those packets. For each packet stream, the packet parsing engine 410 (e.g., FlexParser) parses the packets, identifies those packets that include the redundant header 322 (see Figure 3 ) and forwards them to the receiver steering engine 408. Thus, the packet parsing engine 410 identifies those packets with the redundant header 322 that includes the packet identifier field 330 storing the packet identifier (e.g., PSN).

[0080] To process the received packets 138R, the packet processing circuitry 420 reads and uses information posted by the driver 432 executing on the receiver processor 126 in the appropriate work queue 422 in the receiver memory 128. This information enables the packet parsing logic of the packet parsing engine 410 to locate and extract from each received packet 138R the packet identifier (e.g., PSN) stored in the packet identifier field 330 (see Figure 3 ) and the payload 424 (e.g., slice) stored in the payload field 308 (see Figure 3 ).

[0081] The driver 432 creates a corresponding work queue 422 (commonly referred to as a queue pair or QP) for the received packets 138R. For each fragment (e.g., video frame) in the incoming stream, the driver 432 posts a corresponding work item (referred to as a work queue element or WQE) into the appropriate work queue 422 that points to a buffer 150 that will receive the data in the fragment. In some embodiments, this functionality is implemented as follows:

[0082] • Upon receiving a control message from the sender device 102 indicating that a stream (e.g., the plurality of modified packets 134S sent and the plurality of duplicate packets 136S sent) is to be started, the driver 432 creates a QP for the flow or stream and specifies the following parameters in the QP context in the receiver memory 128 to be accessed by the receiver communication apparatus 124 via the receiver bus 418, for example:

[0083] ° packet size;

[0084] ° header size;

[0085] ° the expected range of packet sequence numbers in the segment;

[0086] ° the PSN offset within the packet header; and

[0087] ° the size and offset of the payload data (not including the frame check sum following the data). In some embodiments, for each segment (which can communicate in tens, hundreds, or even thousands of IP packets), the driver 432 issues a WQE in the queue 422 containing the following information for access by the recipient communication device 124:

[0088] ° the initial packet identifier (e.g., PSN) of the segment (denoted by variable “X”); and

[0089] ° the initial or base address of the buffer 150 used to reconstruct the segment (denoted by variable “A”). The WQE thus specifies the location and contents of the buffer 150, which is divided into slices or payloads 424 of the payload size indicated by the QP context. In some implementations, the WQE contains a memory key (“MKEY”) for use by the recipient communication device 124 in accessing the buffer 150. Alternatively, the above information can be included in the QP context, which points to a circular buffer to which the recipient communication device 124 will write the payloads 424 consecutively (thereby relieving the software of the burden of continuously allocating buffers and issuing WQEs).

[0090] ° the driver 432 identifies the first and second streams and associates them with the same QP.

[0091] The recipient steering engine 408 classifies and performs actions on the packets according to configurable rules and tables. The recipient steering engine 408 determines the path of the packets through the packet processing circuit 420. The recipient steering engine 408 identifies the packet identifiers within the redundant headers 322 and routes the packet identifiers and associated payloads to the anti-replay engine 406.

[0092] The anti-replay engine 406 provides anti-replay protection (e.g., as part of a secure encryption pipeline) that is a protocol for establishing a secure connection between two nodes (e.g., the sender device 102 and the recipient device 104) in the network 106. The anti-replay engine 406 can help prevent a bad actor from injecting packets into a packet stream or making changes to existing packets within a packet stream. The anti-replay engine 406 can help prevent a bad actor from impacting a denial of service attack in which the bad actor attempts to overwhelm or flood the recipient device 104 with a large number of identical packets. The protocol implemented by the anti-replay engine 406 specifies that a base or initial packet identifier (e.g., PSN) is established when a connection is established between the sender device 102 and the recipient device 104. The protocol also specifies that consecutive packets are assigned and transmitted with packet identifiers (e.g., PSNs) that increment from the initial packet identifier (represented by the variable “X”), and the anti-replay engine 406 evaluates packets received within a sliding window to determine which packets should be passed and which packets should be dropped. For example, if the size of the sliding window is set to 10 packets, the sliding window expects packet identifier values 1-10, the anti-replay engine 406 receives ten packets in the order [1, 2, 3, 4, 5, 6, 7, 9, 10, 11] within the sliding window, and the anti-replay engine 406 receives an eighth packet outside of the sliding window, the anti-replay engine 406 will drop the eighth packet. Alternatively, the eighth packet can be stored in an error queue and processed by instructions executed by the recipient processor 126. As described above, the packet processing circuit 420 has a separate instance for each stream that the packet processing circuit 420 receives and processes. Thus, the anti-replay engine 406 has a separate instance and sliding window for each stream. The anti-replay engine 406 can combine these separate sliding windows into a combined sliding window such that only one packet associated with each unique packet identifier is passed by the anti-replay engine 406. The size of the separate and / or combined sliding window can be a user-set parameter value. For example, a user can input the size of the separate and / or combined sliding window into a graphical user interface (not shown) generated by the recipient device 104 and displayed by the recipient device 104 and / or a display device connected to the recipient device 104. Thus, the anti-replay engine 406 determines whether to pass or drop each packet based at least in part on the packet identifier such that at most a single packet including each unique value of the packet identifier is passed to the scatter engine 404. The modified format 320 used by the received packets 138R causes those packets to appear as normal packets to the anti-replay engine 406 and the anti-replay engine 406 will track them and eliminate duplicates in the same way that the anti-replay engine 406 would eliminate duplicate packets received from a bad actor or a malicious packet source.Optionally, the packet processing circuit 420 can include a decryption component that decrypts the received packet 138R before it is processed by the anti-replay engine 406.

[0093] The scatter engine 404 uses the packet identifier (e.g., PSN) along with the base address of the buffer 150 (denoted by the variable "X") to map each payload 424 (e.g., slice) to a respective address in the buffer 150, and thus writes the data payload 424 to the appropriate address via the receiver network interface 412. In other words, the scatter engine 404 determines the correct memory location or address for the payload of a packet stored in the reconstructed message 144 in the receiver memory 128. The scatter engine 404 forwards the memory address and the packet payload to the receiver host interface 402, which passes them to the receiver processor 126 and / or the receiver memory 128 for storage in the reconstructed message 144 stored in the receiver memory 128. The system 100 can help reduce the load on the receiver bus 418 by eliminating duplicate packets within the receiver communication device 124 and sending only one copy of each packet on the receiver bus 418. Thus, the system 100 (see Figure 1 ) helps ensure that only data reconstructed from a single set of packets is sent on the receiver bus 418. The receiver communication device 124 notifies the receiver processor 126 when the reconstructed message 144 is complete.

[0094] In some embodiments, the receiver processor 126 allocates a single buffer 150 based on an expected segment size and data rate. These stream parameters can be pre-set or sent between the sender device 102 and the receiver device 104 over the network 106 in a control message. In such embodiments, the packet size can be fixed and known to the receiver processor 126. Each transmitted packet is labeled with a consecutive packet identifier (e.g., PSN) provided by the RTP or other suitable protocol.

[0095] The hardware logic of the receiver communication device 124 uses the known packet data size and the multiple packet identifiers 222 (see Figure 2) checks data integrity, optionally reorders packets, and eliminates redundant data. Thus, the receiving communication device 124 writes the data payload 424 into the appropriate location in the allocated buffer 150 according to the transmission order (and independent of the reception order), while discarding data from redundant packets. The process of packet reception and reordering can thus be completely transparent to the receiving processor 126, which can only process complete data segments. Assuming that the receiving communication apparatus 124 is capable of receiving data from the network 106 at wire speed, the latency and throughput of the video data transmission at the receiving device 104 is limited only by the processing and bus access rate of the receiving communication apparatus 124. Thus, the system 100 (see Figure 1 ) can maximize data bandwidth, and / or significantly reduce the processing load, memory footprint, and power consumption of the receiving processor 126.

[0096] While the above embodiments specifically relate to the scenario where the sending device 102 transmits multiple packet streams (with redundancy) to the receiving device 104, the system 100 can transmit a single packet stream (without redundancy), as well as packet streams with higher redundancy. The receiving communication apparatus 124 uses multiple packet identifiers 222, as well as a fixed size per slice, to map the data payload stored in the payload field 308 (see Figure 3 ) to the corresponding address in the allocated memory buffer 150 using a linear mapping of the packet identifiers 222. For example, in alternative embodiments (as shown in Figure 5 and Figure 6 ), the packet stream is transmitted according to the enhanced Common Public Radio Interface (“eCPRI”), and the data payload that the receiving communication apparatus 124 is to write into the buffer 150 includes radio data. In one of these embodiments, the header 310 includes an Open Radio Access Network (“O-RAN”) application layer header, and the start physical resource block (“startPrbu”) value contained in the O-RAN header is used as the packet identifier.

[0097] As a non-limiting example, the system 100 can be used to transmit packets within, to, and from an industrial application, where, for example, the sender device 102 is an automated machine tool and the receiver device 104 is a controller or server. As another non-limiting example, the system 100 can be used to transmit packets within, to, and from an autonomous vehicle. A typical autonomous vehicle (e.g., the sender device 102) has multiple sensors that collect real-time data that is transmitted to one or more processors (e.g., the receiver processor 126 of the receiver device 104) using an unreliable transport protocol. However, for safety reasons, the information transmitted within, to, and from an autonomous vehicle must be reliably received. To help ensure that information (e.g., sensor data) is reliably received, standards such as Institute of Electrical and Electronics Engineers (“IEEE”) 802.1CB have been developed that require duplicate packets to be sent over different communication channels. IEEE 802.1CB and similar protocols specify frame replication and elimination protocols. The system 100 can be used to implement such standards.

[0098] As yet another non-limiting example, the system 100 can be used to transmit and receive broadcast (e.g., live) television using an unreliable transport protocol (e.g., over the Internet using the User Datagram Protocol (“UDP”)). For example, a camera can capture images or video frames of an event and send them to the sender device 102, which sends the sender packets 138S to the receiver device 104. If something happens to a portion of the sender packets 138S, such as one or more packets being lost or corrupted, the camera cannot simply re-capture the images and the sender device 102 cannot simply re-send the lost or corrupted packets, because the event captured in the video frames can no longer be available for broadcast. To help reduce the impact of lost or corrupted packets on the broadcast message, standards such as Society of Motion Picture and Television Engineers (“SMPTE”) 2022-7 have been developed that require duplicate packets to be sent over different communication channels. When using SMPTE 2022-7, IEEE 802.1CB or other protocols can be used to specify frame replication and elimination protocols. As will be described below, the system 100 can be used to implement such standards.

[0099] Figure 5 A block diagram schematically illustrating an example data communication system 500 including a plurality of transmitting (“Tx”) host computers 524, 526, and 528, a packet data network 522, and a receiving (“Rx”) host computer 540 is shown in accordance with at least one embodiment. The Tx host computers 524, 526, and 528 are each implementations of the sender device 102 (see Figure 1 and Figure 2 ) the packet data network 522 is the network 106 (seeFigure 1 、 Figure 2 and Figure 4 ) are implementations of the receiver device 104 (see Figure 1 and Figure 4 ). Thus, the Tx host computers 524, 526, and 528 and the Rx host computer 540 are each connected to a packet data network 522. By way of non-limiting example, the packet data network 522 can be implemented as an Internet Protocol (“IP”) network.

[0100] In the illustrated embodiment, each of the Tx host computers 524, 526, and 528 receives streaming input data, e.g., from a respective video camera 530. This input data includes a series of data segments, e.g., video frames or fields (e.g., of a predefined size). Each of the Tx host computers 524, 526, and 528 encapsulates the data in a stream of data packets, each having a header 310 (see Figure 3 ) and a data payload stored in the payload field 308 (see Figure 3 ). The data payloads can each have a predefined, fixed size containing a respective slice of a given data segment. As noted above, the header 310 includes a respective packet identifier field 330 (e.g., storing a startPrbu value and / or a PSN). Although the Tx host computers 524, 526, and 528 and the respective video cameras 530 are shown as separate physical entities in Figure 5 , in some implementations, the host computer is incorporated into the video camera (typically also together with the transmitter communication apparatus 112).

[0101] The transmitter communication apparatus 112 of each of the Tx host computers 524, 526, and 528 transmits the data packets over the packet data network 522 in two packet streams 534 and 536, indicated by the dashed arrows in Figure 5 . The stream 534 includes a plurality of modified packets 134R (see Figure 1 and Figure 4 ), and the stream 536 includes a plurality of duplicate packets 136R (see Figure 1 and Figure 4). Thus, each slice's respective copies are transmitted simultaneously in two packets, one in each of two packet streams 534 and 536. Streams 534 and 536 are typically transmitted over different respective paths through network 522, e.g., by sending each of streams 534 and 536 through a different one of multiple sender connections 212 and 214 (e.g., ports) of sender communication device 112 that are respectively connected to different switches 538 in network 522. The packets are typically transmitted using an unreliable transport protocol, e.g., RTP transport over UDP, which does not guarantee that any particular packet will reach its destination. By transmitting two streams 534 and 536, Tx host computer 524 ensures that at least one copy of each packet will reach its destination without exception. For even higher delivery confidence, two or even more redundant streams (each like stream 536) can be transmitted and processed at Rx host computer 540 in the manner described herein. Alternatively, as noted above, the techniques described herein are useful in efficiently processing even a single packet stream by receiver communication device 124 of Rx host computer 540 when redundancy is not needed.

[0102] Streams 534 and 536, as well as packet streams sent by other Tx host computers 526 and 528, are addressed to Rx host computer 540, which is connected to network 522 by receiver communication device 124. Upon receiving data packets from network 522, receiver communication device 124 uses multiple packet identifiers 222 (e.g., startPrbu values and / or PSNs) to map packets in both packet streams 534 and 536 to respective addresses in buffers 150 in receiver memory 128 of Rx host computer 540. Receiver communication device 124 writes data payloads to the respective addresses while eliminating redundant data, so that buffers 150 accurately contain one copy of each slice of data segments ordered according to multiple packet identifiers 222 (e.g., startPrbu values and / or PSNs). Video frames are thus immediately available in receiver memory 128 for retransmission (e.g., over a television network) or other access.

[0103] Figure 6 A block diagram of a process 600 illustratively showing data reconstruction that can be performed by receiver communication device 124 is shown, according to one embodiment. Each camera 530 (see Figure 5) to capture an image frame 650 that is partitioned into slices for transmission by the sender communication device 112 in a sequence of data packets 652-658. Each of the data packets 652-658 includes a header 660 containing a packet identifier 662 (e.g., a PSN, a startPrbu value, etc.) and a payload 664 containing a respective one of the slices of the image frame 650. The sender communication device 112 transmits two copies of each of the packets 652, 654, 656, 658 (e.g., one copy in each of the streams 534 and 536).

[0104] In the illustrated embodiment, one or more of the packets 652-658 are lost or corrupted in transmission over the network 522, and the other packets arrive out of order at the recipient communication device 124. The recipient communication device 124 detects and discards the corrupted packets, e.g., by computing and checking packet checksums or other error detection codes, as provided by the applicable protocol. In the illustrated example, the recipient communication device 124 receives only the packets 654, 658, and 656 (in that order) from the stream 534 and the packets 652, 656, and 658 from the stream 536. However, using the packet identifiers 662, the recipient communication device 124 is able to place the data from the appropriate payloads 664 directly into the designated buffers 150 in the recipient memory 128 (see Figure 1 、 Figure 4 、 Figure 5 and Figure 8 ), and thus reconstruct the complete image frame 668 in the recipient memory 128. As previously described, the recipient communication device 124 eliminates redundant data so that the buffers 150 accurately contain one copy of each slice of the frame 650, ordered by the packet identifiers 662. The reconstruction process is transparent to software running on the Rx host computer 540 (see Figure 5 ) and does not require software or the recipient processor 126 (see Figure 1 、 Figure 4 and Figure 8 ) to participate in data reordering or redundant elimination.

[0105] Figure 7 A block diagram illustrating details of a method for direct placement of packet data performed by the recipient communication device 124 is shown, in accordance with at least one embodiment. Figure 7 A block diagram illustrating details of a method for direct placement of packet data performed by the recipient communication device 124 is shown, in accordance with at least one embodiment. Figure 4 ) how the dispersal engine 404 (see

[0106] • SMPTE 2022-5 / 6 / 7

[0107] • VSF TR-03 / IETF RFC 4175

[0108] • intoPIX TICO

[0109] • Sony IP Live RDD 34 and NMI

[0110] • NewTek

[0111] • GigE vision GVSP

[0112] Figure 7 The illustrated packets belong to a stream of all packets with a fixed and pre-known payload size. As mentioned above, the driver 432 (see Figure 1 , Figure 4 and Figure 8 ) running on the receiver processor 126 (see Figure 4 ) allocates the base address of the buffer 150 and informs the receiver communication device 124 of the base address of the buffer 150 to which each data segment in the stream will be written. Within the packet header 660, the RTP header 700 includes a timestamp 702 and other protocol information 704 in addition to the packet identifier 662 (e.g., PSN). Based on the context information provided by the driver 432, the packet parsing logic of the packet parsing engine 410 proceeds to the known location of the packet identifier 662 in each packet and extracts the packet identifier 662, as well as the data slice 80 from the payload 664. On this basis, the scatter engine 404 is able to write the data slices from the payload 664 in the correct order to the buffer 150 through a simple linear translation of the packet identifier 662 (e.g., PSN).

[0113] In other words, for a stream of packets starting with an initial packet identifier represented by the variable "X", the data payload in each packet has a size represented by the variable "B", and the base address in the buffer 150 is represented by the variable "A", the scatter engine 404 (see Figure 4 ) writes the payload of each packet with a packet identifier represented by the variable "PSN" to an address determined by the following Equation 1:

[0114] Payload Address = A + (PSN - X) x B Equation 1

[0115] When the payloads of two valid packets from different, respective flows in the redundancy transport scheme map to the same address (whether they have the same or different PSNs), ultimately only one of them will be written to the buffer 150. The scatter engine 404 can write the payload of only the first-arriving packet to the buffer 150 and discard the payload of the other packet.

[0116] For some protocols, the segment definition is also indicated in the packet header 660, enabling an alternative implementation that further reduces the involvement of the driver 432. For example, the recipient communication device 124 can operate as follows:

[0117] • The segments in most video applications correspond to a single frame (or a single field for interlaced video).

[0118] • In some video protocols, an end-of-frame marker is entered in the packet transport header: the first and last packets in a segment are identified by special fields in the transport header that indicate that they are the first or last packet.

[0119] • When the frame size is fixed and the frame start and end are marked in this way in the header, the recipient communication device 124 can automatically start writing data to the allocated host buffer upon receiving the start marker. Thus, the need to specify segment boundaries is also offloaded from software to the hardware logic of the recipient communication device 124.

[0120] When the recipient communication device 124 is implemented as a NIC that operates at very high speeds (e.g., receives incoming video data at 400 Gbps), accessing the recipient memory 128 from the NIC over the recipient bus 418 can become a bottleneck. To reduce the pressure on the recipient bus 418, the packet processing circuit 420 can monitor the packet identifiers (e.g., PSNs) and, when the payload from a given packet having a particular packet identifier in one stream has already been written to the buffer 150, simply discard the corresponding packet from another stream, rather than overwriting data already in the buffer 150. To this end, the packet processing circuit 420 can maintain a record for each stream, such as a vector of packet identifiers (e.g., PSNs) and a rolling packet identifier (e.g., PSN) window (e.g., a sliding window). Each packet that arrives causes the processing circuit 420 to flip a bit in the vector of packet identifiers (e.g., PSNs). When the bit for a given packet identifier is set and a corresponding packet from another stream arrives, the packet parsing logic of the packet parsing engine 410 discards the latter packet without further processing.

[0121] Figure 8A block diagram illustrating a data communication system 810 for wireless radio data transmission is shown in accordance with at least one embodiment. System 810 operates in accordance with the enhanced Common Public Radio Interface (“eCPRI”) specification and Open Radio Access Network (O-RAN) fronthaul requirements as defined in the O-RAN Fronthaul Working Group Control, User and Synchronization Plane Specification (published by the O-RAN Alliance as Technical Specification ORAN-WG4.CUS.0-v01.00 (2019)).

[0122] System 810 includes an O-RAN Distributed Unit (“O-DU”) 812, an O-RAN Radio Unit (O-RU) 814, and a network 816, which are implementations of receiver device 104, transmitter device 102, and network 106. In system 810, O-DU 812 communicates with O-RU 814 over network 816 (such as an IP network), for example, in accordance with the eCPRI specification and O-RAN specification described above. O-DU 812 includes a receiver processor 126 (e.g., one or more general-purpose computer processors) with receiver memory 128 and is connected to network 816 through receiver communication 124. O-RU 814 includes a transmitter processor 114 (e.g., one or more general-purpose computer processors) with transmitter memory 116 and is connected to network 816 through transmitter communication 112. O-DU 812 and O-RU 814 typically include other specialized interfaces. For example, O-RU 814 can include a radio transceiver 818 connected to an antenna 820.

[0123] O-RU 814 transmits a sequence of data packets over network 816, including respective headers and data payloads containing data segments of radio signals received by O-RU 814. Each data segment includes one or more physical resource blocks (“PRBs”), which are divided into a number of slices in the form of data samples, typically including alternating iSamples and qSamples as defined by the O-RAN specification. The samples have a fixed size, between 1 bit and 16 bits per sample, which is defined in the O-RAN header of each packet, as referenced below in Figure 9As explained, the sending communication device 112 (e.g., NIC) transmits data packets over network 816, typically using an unreliable transport protocol, such as eCPRI based on UDP. Upon receiving a data packet from network 816, the receiving communication device 124 (e.g., NIC) maps the samples contained in the packet to corresponding addresses in buffer 150 of the receiving memory 128 of O-DU 812, whereby the samples are ordered in buffer 150 according to the corresponding packet identifier (e.g., sequence number) included in the O-RAN header of the packet. This makes the radio data immediately available in the receiving memory 128 for processing by O-DU 812 in the correct order.

[0124] Figure 9 It shows that according to Figure 8 The embodiment schematically illustrates a block diagram of a method for placing packet data directly into a buffer 150 in the receiver's memory 128. This method is applied to O-RAN data packets received by the O-DU 812. For ease of illustration, Figure 9 Packet 900 is shown, one of the packets received by O-DU 812. Packet 900 includes a network and transport header 932, which includes an IP header 934, a UDP header 936, and an eCPRI header 938, as defined in the O-RAN specification above. Following the network and transport header 932 is an O-RAN application header 940 (as defined in the O-RAN specification) and a payload 942 containing a sequence of data samples. The IP header 934 may include a destination address field (e.g., ...). Figure 3 The destination address field 302 and the source address field 304 (as shown) are shown in the figure. Figure 3 The source address field 304 is shown. The UDP header 936 may include a UDP source port field (not shown) identifying the port from which it sent the packet and / or a UDP destination port field (not shown) identifying the port to which it sent the packet.

[0125] The packet replication engine 208 modifies the IP header 934 (e.g., the source address stored in the source address field or the destination address stored in the destination address field) and / or UDP header 936 (e.g., the value stored in the UDP source port field and / or the value stored in the UDP destination port field) in multiple replicated packets 136, such that the multiple modified packets 134 and the multiple replicated packets 136 are sent along different paths through network 106, and the modified packet header 324 in the multiple replicated packets 136R is different from that in the multiple modified packets 134R.

[0126] The O-RAN header 940 includes a set of frame ID fields 944, followed by section ID fields 946, which define the data layout of the data segments in the payload 942. These section ID fields 946 include, among others, a startPrbu field 948, which identifies the starting physical resource block (“PRB”) in the packet 900, and thus can be used as a sequence number for mapping samples in the payload 942. A numPrbu field 950 indicates the number of consecutive PRBs in the section; a udCompHdr field 951 indicates the size of each sample in bits, i.e., the size of each slice of PRBs in the payload 942. When the data packet contains multiple data sections, the section ID fields 946 will be repeated as a header for each section.

[0127] The receiving communication device 124 uses a linear mapping of the startPrbu value in the startPrbu field 948 to extract the data samples from the payload and write them to the respective addresses 954 in the buffer 150 in the receiving memory 128. The startPrbu value indicates the address 154 at which the receiving communication device 124 will write the first sample in each section, followed by the subsequent samples. Specifically, in the present embodiment, the receiving communication device 124 maps the samples to the addresses 154 for the first sample in each section using the following Equation 2:

[0128] Payload Address = A + (SN - X) x B Equation 2

[0129] In this formula, the variable “X” represents the initial packet identifier (e.g., sequence number), the variable “SN” represents the current packet identifier (e.g., sequence number) obtained from the startPrbu value in the startPrbu field 148, the variable “B” represents the fixed size of the samples, which is obtained from the udCompHdr value in the udCompHdr field 151, and the variable “A” represents the base address of the buffer 150.

[0130] Although the above embodiment specifically relates to data transmission from the O-RU 814 to the O-DU 812, the O-RU 814 can similarly apply the principles of the present embodiment when buffering data transmitted by the O-DU 812.

[0131] Figure 10A flowchart of a method 1000 that can be performed by the sender device 102 is shown, in accordance with at least one embodiment. In a first block 1002, the sender processor 114 sends the message 130 or a link to the message 130 stored in the memory 118 to the sender host interface 202 of the sender communication apparatus 112. The sender processor 114 can execute a data processing application 224 (e.g., a video processing application) that receives image frames from a camera (e.g., the camera 530). The sender processor 114 can communicate the message 130 or a link to it to the sender host interface via the sender bus 228 and using the driver 226.

[0132] In a next block 1004, the sender host interface 202 creates a plurality of packets 132 by packetizing the message 130 and forwards the plurality of packets 132 to the packet identifier engine 204. In block 1006, the packet identifier engine 204 creates a packet identifier for the plurality of packets 132 and forwards both the packet identifier and the plurality of packets 132 to the sender steering engine 206. In block 1008, the sender steering engine 206 creates a plurality of modified packets 134 by adding the redundant header 322 (see Figure 3 ) to each of the plurality of packets 132 and forwards the plurality of modified packets 134 to the packet replication engine 208. In block 1010, the packet replication engine 208 creates a plurality of replicated packets 136 by replicating the plurality of modified packets 134 and sends both the plurality of modified packets 134 and the plurality of replicated packets 136 to the sender network interface 210. In block 1012, the sender network interface 210 sends the plurality of modified packets 134 and the plurality of replicated packets 136 over the network 106 to the recipient communication apparatus 124. Thus, in block 1012, the sender network interface 210 sends a send packet 138S comprising a send plurality of modified packets 134S and a send plurality of replicated packets 136S over the network 106 to the recipient communication apparatus 124. The method 1000 then terminates.

[0133] Figure 11 A flowchart of a method 1100 that can be performed by the recipient device 104 is shown, in accordance with at least one embodiment. In a first block 1102, the recipient network interface 412 receives the receive packets 138R and forwards them to the packet resolution engine 410. In block 1104, the packet resolution engine 410 resolves each receive packet 138R, identifies the redundant header 322 (see Figure 3) and forwards the identified packets to the recipient steering engine 408. In block 1106, the recipient steering engine 408 identifies the packet identifier in each identified packet, obtains buffer information, and forwards the buffer information, packet identifier, and payload to the anti-replay engine 406. The recipient steering engine 408 can obtain the buffer information from the driver 432. As a non-limiting example, the buffer information can include the MKEY, the base address of the buffer 150, and the like. In block 1108, the anti-replay engine 406 decides for each received packet 138R whether the packet is forwarded to the scatter engine 404 or discarded. In block 1110, the scatter engine 404 determines an address in the buffer 150 for each payload of each forwarded packet and forwards the address and payload to the recipient host interface 402. In block 1112, the recipient host interface 402 stores the payloads in their respective addresses. In block 1114, when the reconstituted message 144 is complete, the scatter engine 404 notifies the recipient processor 126 via the recipient host interface 402. The method 1100 then terminates.

[0134] In the following description, numerous specific details are set forth to provide a more thorough understanding of at least one embodiment. However, it will be apparent to one skilled in the art that the inventive concept can be practiced without one or more of these specific details.

[0135] Servers and Data Centers

[0136] The following figures set forth, but are not limited to, example network server and data center based systems that can be used to implement at least one embodiment.

[0137] Figure 12 A distributed system 1200, in accordance with at least one embodiment, is shown. In at least one embodiment, the distributed system 1200 includes one or more client computing devices 1202, 1204, 1206, and 1208, which are configured to execute and operate a client application such as a web browser, a proprietary client, and / or variants thereof. In at least one embodiment, server 1212 can be communicatively coupled with remote client computing devices 1202, 1204, 1206, and 1208 via network 1210.

[0138] In at least one embodiment, server 1212 can be adapted to run one or more services or software applications, such as services and applications that can manage session activity for single sign-on (SSO) access across multiple data centers. In at least one embodiment, server 1212 can also provide other services, or software applications, which can include non-virtual and virtual environments. In at least one embodiment, these services can be provided as web-based services or cloud services or under a software as a service (SaaS) model to users of client computing devices 1202, 1204, 1206, and / or 1208. In at least one embodiment, users operating client computing devices 1202, 1204, 1206, and / or 1208 can in turn utilize one or more client applications to interact with server 1212 to utilize services provided by these components.

[0139] In at least one embodiment, software components 1218, 1220, and 1222 of system 1200 are implemented on server 1212. In at least one embodiment, one or more components of system 1200 and / or services provided by these components can also be implemented by one or more of client computing devices 1202, 1204, 1206, and / or 1208. In at least one embodiment, users operating these client computing devices can then utilize one or more client applications to use services provided by these components. In at least one embodiment, these components can be implemented in hardware, firmware, software, or combinations thereof. It should be appreciated that various Figure 12 The illustrated embodiment is one example of a distributed system for implementing an embodiment system and is not intended to be limiting.

[0140] In at least one embodiment, client computing devices 1202, 1204, 1206, and / or 1208 can include different types of computing systems. In at least one embodiment, client computing devices can include portable handheld devices (e.g., an iPhone® by Apple Inc. of Cupertino, California, a cellular phone, computing tablet, a personal digital assistant (PDA), or a wearable device (e.g., a Google head-mounted display) running software such as the Android® operating system (OS) by Google, Inc. of Mountain View, California, or the iOS® operating system (OS) by Apple Inc. of Cupertino, California. And / or various mobile operating systems (such as iOS, Windows Phone, Android, BlackBerry 10, Palm OS, and / or variants thereof). In at least one embodiment, the device may support different applications, such as various Internet-related applications, email, short message service (SMS) applications, and may use various other communication protocols. In at least one embodiment, the client computing device may also include a general-purpose personal computer, for example, a general-purpose personal computer running various versions of Microsoft... Apple Personal computers and / or laptops running Linux operating systems. In at least one embodiment, the client computing device can be running various commercially available operating systems. The client computing device may be a workstation computer operating system similar to UNIX, including but not limited to various GNU / Linux operating systems such as Google Chrome OS. In at least one embodiment, the client computing device may further include electronic devices capable of communicating over one or more networks 1210, such as thin client computers, internet-enabled gaming systems (e.g., with or without...). Gesture input devices include Microsoft Xbox game consoles and / or personal messaging devices. Despite Figure 12 The distributed system 1200 is shown as having four client computing devices, but can support any number of client computing devices. Other devices (such as devices with sensors) can interact with the server 1212.

[0141] In at least one embodiment, network 1210 in distributed system 1200 can be any type of network capable of supporting data communication using any of the various available protocols, including but not limited to TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (System Network Architecture), IPX (Internet Packet Switching), AppleTalk, and / or variations thereof. In at least one embodiment, network 1210 can be a local area network (LAN), an Ethernet-based network, Token Ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, or a wireless network (e.g., in the IEEE 802.11 protocol suite). Networks operating under any of the wireless protocols (and / or any other wireless protocols), and / or any combination of these and / or other networks.

[0142] In at least one embodiment, server 1212 may consist of one or more general-purpose computers, dedicated server computers (for example, including PC (personal computer) servers), servers, server farms, server clusters, or any other appropriate arrangement and / or combination of servers. In at least one embodiment, server 1212 can include one or more virtual machines running a virtual operating system, or other computing architecture involving virtualization. In at least one embodiment, one or more pools of flexible logical storage devices can be virtualized in order to maintain virtual storage for servers. In at least one embodiment, virtual networks can be controlled by server 1212 using software-defined networking. In at least one embodiment, server 1212 can be adapted to run one or more services or software applications.

[0143] In at least one embodiment, server 1212 can run any operating system, as well as any commercially available server operating system. In at least one embodiment, server 1212 can also run any of a variety of additional server applications and / or mid-tier applications, including HTTP (HyperText Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, JAVA® servers, database servers, and / or server applications, and / or variations thereof. In at least one embodiment, example database servers include, but are not limited to, those available from Oracle, Microsoft, Sybase, IBM (International Business Machines), and / or variations thereof.

[0144] In at least one embodiment, server 1212 can include one or more applications for analyzing and correlating data feeds and / or event updates received from users of client computing devices 1202, 1204, 1206, and 1208. In at least one embodiment, data feeds and / or event updates can include, but are not limited to, data received from one or more third-party information sources and continuous data streams, which can include real-time events or feeds, updates, or real-time updates, which can include real-time events related to sensor data applications, financial quotes, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, and / or variations thereof. In at least one embodiment, server 1212 can also include one or more applications for displaying data feeds and / or real-time events via one or more display devices of client computing devices 1202, 1204, 1206, and 1208.

[0145] In at least one embodiment, distributed system 1200 can also include one or more databases 1214 and 1216. In at least one embodiment, databases can provide a mechanism for storing information such as user interaction information, usage pattern information, adaptation rule information, and other information. In at least one embodiment, databases 1214 and 1216 can reside in various locations. In at least one embodiment, one or more of databases 1214 and 1216 can reside on a non-transitory storage medium local to (and / or in) server 1212. In at least one embodiment, databases 1214 and 1216 can be remote from server 1212 and in communication with server 1212 via a network-based connection or a dedicated connection. In at least one embodiment, databases 1214 and 1216 can reside in a storage area network (SAN). In at least one embodiment, any necessary files for performing functions attributed to server 1212 can be stored locally on server 1212 and / or remotely as appropriate. In at least one embodiment, databases 1214 and 1216 can include a relational database such as one adapted to store, update, and retrieve data in response to SQL-formatted commands.

[0146] In at least one embodiment, distributed system 1200 can be used to implement system 100 (see Figure 1 ). In such an embodiment, sending device 102 and / or receiving device 104 can each be implemented as server 1212, client computing device 1202, client computing device 1204, client computing device 1206, or client computing device 1208. Network 106 can be implemented as network 1210.

[0147] In at least one embodiment, distributed system 1200 can be used to implement data communication system 500 (see Figure 5 ). In such an embodiment, one or more of Tx host computers 524, 526, and 528 and / or Rx host computer 540 can each be implemented as server 1212, client computing device 1202, client computing device 1204, client computing device 1206, or client computing device 1208. Packet data network 522 can be implemented as network 1210.

[0148] In at least one embodiment, distributed system 1200 can be used to implement data communication system 810 (see Figure 8 ). In such an embodiment, O-DU 812 and / or O-RU 814 can each be implemented as server 1212, client computing device 1202, client computing device 1204, client computing device 1206, or client computing device 1208. Network 816 can be implemented as network 1210.

[0149] Figure 13 An example data center 1300 is shown, in accordance with at least one embodiment. In at least one embodiment, the data center 1300 includes, without limitation, a data center infrastructure layer 1310, a framework layer 1320, a software layer 1330, and an application layer 1340.

[0150] In at least one embodiment, as shown in Figure 13 the data center infrastructure layer 1310 can include a resource orchestrator 1312, grouped computing resources 1314, and node computing resources (“node C.R.s”) 1316(1)-1316(N), where “N” represents any whole, positive integer. In at least one embodiment, the node C.R.s 1316(1)-1316(N) can include, without limitation, any number of central processing units (“CPUs”) or other processors (including accelerators, field programmable gate arrays (“FPGAs”), graphics processors, etc.), memory devices (e.g., dynamic random access memory), storage devices (e.g., solid state or disk drives), network input / output (“NW I / O”) devices, network switches, virtual machines (“VMs”), power modules, and cooling modules, etc. In at least one embodiment, one or more of the node C.R.s 1316(1)-1316(N) can be a server having one or more of the above-described computing resources.

[0151] In at least one embodiment, the grouped computing resources 1314 can include individual groups of node C.R.s housed within one or more racks (not shown), or housed within many racks (also not shown) within data centers at various geographic locations. An individual group of node C.R.s within the grouped computing resources 1314 can include a grouped computing, network, memory, or storage resource that can be configured or allocated to support one or more workloads. In at least one embodiment, several node C.R.s including CPUs or processors can be grouped within one or more racks to provide computing resources to support one or more workloads. In at least one embodiment, one or more racks can also include any number and combination of power modules, cooling modules, and network switches.

[0152] In at least one embodiment, the resource orchestrator 1312 can configure or otherwise control one or more of the node C.R.s 1316(1)-1316(N) and / or the grouped computing resources 1314. In at least one embodiment, the resource orchestrator 1312 can include a software design infrastructure (“SDI”) management entity for the data center 1300. In at least one embodiment, the resource orchestrator 1312 can comprise hardware, software, or some combination thereof.

[0153] In at least one embodiment, such as Figure 13 As shown, framework layer 1320 includes, but is not limited to, job scheduler 1332, configuration manager 1334, resource manager 1336, and distributed file system 1338. In at least one embodiment, framework layer 1320 may include a framework of software 1352 supporting software layer 1330 and / or one or more applications 1342 supporting application layer 1340. In at least one embodiment, software 1352 or application 1342 may respectively include web-based service software or applications, such as services or applications provided by Amazon Web Services, Google Cloud, and Microsoft Azure. In at least one embodiment, framework layer 1320 may be, but is not limited to, a free and open-source software web application framework, such as Apache Spark™ (hereinafter referred to as "Spark") which can utilize distributed file system 1338 for large-scale data processing (e.g., "big data"). In at least one embodiment, job scheduler 1332 may include Spark drivers to facilitate the scheduling of workloads supported by the various layers of data center 1300. In at least one embodiment, configuration manager 1334 may be able to configure different layers, such as software layer 1330 and framework layer 1320 including Spark and distributed file system 1338 for supporting large-scale data processing. In at least one embodiment, resource manager 1336 is able to manage cluster or group computing resources mapped to or allocated to support distributed file system 1338 and job scheduler 1332. In at least one embodiment, cluster or group computing resources may include grouped computing resources 1314 on data center infrastructure layer 1310. In at least one embodiment, resource manager 1336 may coordinate with resource coordinator 1312 to manage these mapped or allocated computing resources.

[0154] In at least one embodiment, the software 1352 included in software layer 1330 may include software used by at least a portion of nodes CR1316(1)-1316(N), grouped computing resources 1314, and / or the distributed file system 1338 of framework layer 1320. One or more types of software may include, but are not limited to, Internet web page search software, email virus scanning software, database software, and streaming video content software.

[0155] In at least one embodiment, one or more application programs 1342 included in application layer 1340 can include one or more types of application programs used by at least portions of node C.R.s 1316(1)-1316(N), grouped computing resources 1314, and / or distributed file system 1338 of framework layer 1320. One or more types of application programs can include, but are not limited to CUDA application programs, 5G network application programs, artificial intelligence application programs, data center application programs, and / or variations thereof.

[0156] CUDA application programs, 5G network application programs, artificial intelligence application programs, data center application programs, and / or variations thereof.

[0157] In at least one embodiment, any of configuration manager 1334, resource manager 1336, and resource orchestrator 1312 can implement any number and type of self-modifying actions based on any number and type of data acquired in any technically feasible manner. In at least one embodiment, self-modifying actions can relieve data center operators of data center 1300 from making possibly poor configuration decisions and can avoid underutilized and / or poorly performing portions of a data center.

[0158] Figure 14 FIG. illustrates a system 1400 including a client-server network 1404 formed of a plurality of network server computers 1402 interconnected in accordance with at least one embodiment. In at least one embodiment, each network server computer 1402 stores data accessible by other network server computers 1402 and by client computers 1406 and networks 1408 linked to a wide area network 1404. In at least one embodiment, configuration of client-server network 1404 can change over time as client computers 1406 and one or more networks 1408 connect and disconnect with network 1404, and as one or more server computers 1402 are added to or removed from network 1404. In at least one embodiment, client-server network includes client computers 1406 and networks 1408 when they are connected with network server computers 1402. In at least one embodiment, term computer includes any device or machine that is capable of accepting data, applying prescribed processes to data, and providing results of processes.

[0159] In at least one embodiment, client-server network 1404 stores information accessible to web server computers 1402, remote network 1408, and client computers 1406. In at least one embodiment, web server computers 1402 are formed from mainframe computers, minicomputers, and / or microcomputers each having one or more processors. In at least one embodiment, server computers 1402 are linked together through wired and / or wireless transmission media, such as wire, fiber optic cable, and / or microwave transmission media, satellite transmission media, or other conductive, optical, or electromagnetic wave transmission media. In at least one embodiment, client computers 1406 access web server computers 1402 through similar wired or wireless transmission media. In at least one embodiment, client computers 1406 can link into client-server network 1404 using modems and standard telephone communication networks. In at least one embodiment, alternative carrier systems, such as cable and satellite communication systems, can also be used to link into client-server network 1404. In at least one embodiment, other private or time-shared carrier systems can be used. In at least one embodiment, network 1404 is a global information network, such as the Internet. In at least one embodiment, network is a private intranet using similar protocols as the Internet but with added security measures and restricted access controls. In at least one embodiment, network 1404 is a private or semi-private network using proprietary communication protocols.

[0160] In at least one embodiment, client computers 1406 are any end-user computers and can also be mainframe computers, minicomputers, or microcomputers having one or more microprocessors. In at least one embodiment, server computers 1402 can sometimes act as client computers accessing another server computer 1402. In at least one embodiment, remote network 1408 can be a local area network, a network added to a wide area network through an independent service provider (ISP) for the Internet, or another group of computers interconnected by wired or wireless transmission media having a fixed or changing configuration over time. In at least one embodiment, client computers 1406 can link into and access network 1404 independently or through remote network 1408.

[0161] In at least one embodiment, system 1400 can be used to implement system 100 (see Figure 1 ). In such an embodiment, sender device 102 and / or recipient device 104 can each be implemented as one or more of web server computers 1402 or one or more of client computers 1406. Network 106 can be implemented as remote network 1408.

[0162] In at least one embodiment, system 1400 can be used to implement data communication system 500 (see Figure 5 ). In such an embodiment, Tx host computers 524, 526, and 528 and / or one or more of Rx host computers 540 can each be implemented as one or more of network server computers 1402 or one or more of client computers 1406. Packet data network 522 can be implemented as remote network 1408.

[0163] In at least one embodiment, system 1400 can be used to implement data communication system 810 (see Figure 8 ). In such an embodiment, O-DU 812 and / or O-RU 814 can each be implemented as one or more of network server computers 1402 or one or more of client computers 1406. Network 816 can be implemented as remote network 1408.

[0164] Figure 15A system 1500 including one or more computer networks connecting one or more computer machines is shown in accordance with at least one embodiment. In at least one embodiment, network 908 can be any type of electrically connected computer group, including, for example, the following networks: the Internet, an intranet, a local area network (LAN), a wide area network (WAN), or an interconnected combination of these network types. In at least one embodiment, connections within network 1508 can be remote modems, Ethernet (IEEE 802.3), Token Ring (IEEE 802.5), Fiber Distributed Datalink Interface (FDDI), Asynchronous Transfer Mode (ATM) or any other communications protocol. In at least one embodiment, computing devices linked to a network can be desktops, servers, portable, handheld, set-top, personal digital assistants (PDAs), terminals, or any other desired type or configuration. In at least one embodiment, network-connected devices can vary widely in processing speed, internal memory, and other performance aspects, depending on their functionality. In at least one embodiment, communications within a network, as well as to or from computing devices connected to a network, can be wired or wireless. In at least one embodiment, network 1508 can comprise, at least in part, the worldwide public Internet, which typically connects multiple users according to a client-server model according to Transmission Control Protocol / Internet Protocol (TCP / IP) specifications. In at least one embodiment, a client-server network is a dominant model for communication between two computers. In at least one embodiment, a client computer (“client”) issues one or more commands to a server computer (“server”). In at least one embodiment, a server fulfills client commands by accessing available network resources and returning information to a client according to client commands. In at least one embodiment, client computer systems and network resources residing on network servers are assigned network addresses for identification during communications between elements of a network. In at least one embodiment, communications from other network-connected systems to a server will include the network address of the relevant server / network resource as part of the communication, so that the proper destination of data / requests is identified as a recipient. In at least one embodiment, when network 1508 comprises the global Internet, network addresses are IP addresses in TCP / IP format, which can route data at least in part to an email account, website, or other Internet tool residing on a server. In at least one embodiment, information and services residing on network servers can be available to web browsers of client computers through domain names (e.g., www.site.com), which map to IP addresses of network servers.

[0165] In at least one embodiment, multiple clients 1502, 1504, and 1506 connect to the network 1508 via respective communication links. In at least one embodiment, each of these clients can access the network 1508 via any desired form of communication, such as via a dial-up modem connection, cable link, digital subscriber line (DSL), wireless or satellite link, or any other form of communication. In at least one embodiment, each client can communicate using any machine compatible with the network 1508 (e.g., a personal computer (PC), workstation, dedicated terminal, personal data assistant (PDA), or other similar device). In at least one embodiment, the clients 1502, 1504, and 1506 can or can not be located in the same geographic region.

[0166] In at least one embodiment, multiple servers 1510, 1512, and 1514 are connected to the network 1518 to serve clients in communication with the network 1518. In at least one embodiment, each server is typically a powerful computer or device that manages network resources and responds to client commands. In at least one embodiment, servers include computer-readable data storage media, such as hard drive and RAM memory, that store program instructions and data. In at least one embodiment, the servers 1510, 1512, 1514 run application programs in response to client commands. In at least one embodiment, the server 1510 can run a web server application to respond to client requests for HTML pages, and can also run a mail server application to receive and route electronic mail. In at least one embodiment, other application programs can also run on the server 1510, such as an FTP server or media server to stream audio / video data to clients. In at least one embodiment, different servers can be dedicated to performing different tasks. In at least one embodiment, the server 1510 can be a dedicated web server that manages resources related to a website for different users, while the server 1512 can be dedicated to providing electronic mail (email) management. In at least one embodiment, other servers can be dedicated to media (audio, video, etc.), file transfer protocol (FTP), or a combination of any two or more services typically available or provided over a network. In at least one embodiment, each server can be in the same or different location as other servers. In at least one embodiment, there can be multiple servers performing mirror tasks for users, thereby relieving congestion or minimizing traffic directed to and from a single server. In at least one embodiment, the servers 1510, 1512, 1514 are under the control of a web hosting provider that maintains and delivers third-party content over the network 1518.

[0167] In at least one embodiment, a web hosting provider delivers services to two different types of clients. In at least one embodiment, one type, which can be referred to as a browser, requests content from servers 1510, 1512, 1514, such as web pages, email messages, video clips, etc. In at least one embodiment, a second type, which can be referred to as a user, hires the web hosting provider to maintain a network resource, such as a website, and make it available to browsers. In at least one embodiment, users contract with the web hosting provider to make memory space, processor capacity, and communication bandwidth available to their desired network resource according to the amount of server resources that the user desires to utilize.

[0168] In at least one embodiment, in order for the web hosting provider to provide services to both clients, the application programs that manage the network resources hosted by the servers must be properly configured. In at least one embodiment, the program configuration process involves defining a set of parameters that at least partially control the application program's response to browser requests, and also at least partially define the server resources available to a particular user.

[0169] In one embodiment, intranet server 1516 communicates with network 1508 via a communication link. In at least one embodiment, intranet server 1516 communicates with server manager 1518. In at least one embodiment, server manager 1518 includes a database of application program configuration parameters used in servers 1510, 1512, 1514. In at least one embodiment, a user modifies database 1520 via intranet 1516, and server manager 1518 interacts with servers 1510, 1512, 1514 to modify the application program parameters so that they match the contents of the database. In at least one embodiment, a user logs onto intranet 1516 by connecting to intranet 1516 via computer 1502 and entering authentication information such as a user name and password.

[0170] In at least one embodiment, when a user wishes to log into a new service or modify an existing service, the intranet server 1516 authenticates the user and provides the user with an interactive screen display / control panel that allows the user to access configuration parameters for a particular application. In at least one embodiment, the user is presented with a number of modifiable text boxes that describe aspects of the user's website or other web resources' configuration. In at least one embodiment, if the user desires to increase the amount of memory space reserved on the server for their website, the user is provided with a field in which the user specifies the desired amount of memory space. In at least one embodiment, in response to receiving this information, the intranet server 1516 updates the database 1520. In at least one embodiment, the server manager 1518 forwards this information to the appropriate server and the new parameters are used during operation of the application. In at least one embodiment, the intranet server 1516 is configured to provide the user with access to configuration parameters for web-hosted resources (e.g., web pages, email, FTP sites, media sites, etc.) that the user has contracted with a web-hosting service provider.

[0171] In at least one embodiment, the system 1500 can be used to implement the system 100 (see Figure 1 ). In such an embodiment, the sender device 102 and / or the receiver device 104 can each be implemented as the intranet server 1516, one or more of the clients 1502, 1504, and 1506, or one or more of the servers 1510, 1512, and 1514. The network 106 can be implemented as the remote network 1508.

[0172] In at least one embodiment, the system 1500 can be used to implement the data communication system 500 (see Figure 5 ). In such an embodiment, one or more of the Tx host computers 524, 526, and 528 and / or the Rx host computer 540 can each be implemented as the intranet server 1516, one or more of the clients 1502, 1504, and 1506, or one or more of the servers 1510, 1512, and 1514. The packet data network 522 can be implemented as the remote network 1508.

[0173] In at least one embodiment, the system 1500 can be used to implement the data communication system 810 (see Figure 8 ). In such an embodiment, the O-DU 812 and / or the O-RU 814 can each be implemented as the intranet server 1516, one or more of the clients 1502, 1504, and 1506, or one or more of the servers 1510, 1512, and 1514. The network 816 can be implemented as the remote network 1508.

[0174] Figure 16A A networked computer system 1600A is shown, in accordance with at least one embodiment. In at least one embodiment, the networked computer system 1600A includes a plurality of nodes or personal computers ("PCs") 1602, 1618, 1620. In at least one embodiment, the personal computers or nodes 1602 include a processor 1614, a memory 1616, a video camera 1604, a microphone 1606, a mouse 1608, a speaker 1610, and a monitor 1612. In at least one embodiment, the PCs 1602, 1618, 1620 can each run one or more desktop servers for an internal network within a given company, or can be servers for a general purpose network not limited to a particular environment. In at least one embodiment, there is one server per PC node of the network, such that each PC node of the network represents a particular network server having a particular network URL address. In at least one embodiment, each server has a default web page for a user of that server by default, which default web page can itself contain embedded URLs pointing to further sub-pages of that user on that server, or to other servers on the network or pages on other servers.

[0175] In at least one embodiment, the nodes 1602, 1618, 1620 and other nodes of the network are interconnected via a medium 1622. In at least one embodiment, the medium 1622 can be a communication channel such as an Integrated Services Digital Network ("ISDN"). In at least one embodiment, the various nodes of the networked computer system can be connected by various communication media, including a Local Area Network ("LAN"), a plain old telephone line ("POTS") (sometimes referred to as the Public Switched Telephone Network ("PSTN")), and / or variants thereof. In at least one embodiment, the various nodes of the network can also constitute computer system users interconnected via a network such as the Internet. In at least one embodiment, each server on the network (running from a particular node of the network at a given instance) has a unique address or identification within the network, which can be specified according to a URL.

[0176] In at least one embodiment, a plurality of multipoint conference units ("MCUs") can thus be used to transmit data to and from the various nodes or "endpoints" of the conferencing system. In at least one embodiment, the nodes and / or MCUs can be interconnected via ISDN links or through a Local Area Network ("LAN"), in addition to various other communication media such as nodes connected through the Internet. In at least one embodiment, the nodes of the conferencing system can typically be connected either directly to a communication medium such as a LAN or through an MCU, and the conferencing system can include other nodes or elements such as routers, servers, and / or variants thereof.

[0177] In at least one embodiment, processor 1614 is a general-purpose programmable processor. In at least one embodiment, processors of networking computer system 1600A’s nodes can also be specialized video processors. In at least one embodiment, different peripherals and components of nodes, such as those of node 1602, can differ from those of other nodes. In at least one embodiment, nodes 1618 and 1620 can be configured the same as or different from node 1602. In at least one embodiment, nodes can be implemented on any suitable computer system in addition to a PC system.

[0178] In at least one embodiment, networking computer system 1600A can be used to implement system 100 (see Figure 1 ). In such an embodiment, sender device 102 and / or recipient device 104 can each be implemented as one or more of PCs 1602, 1618, 1620. Network 106 can be implemented as medium 1622.

[0179] In at least one embodiment, networking computer system 1600A can be used to implement data communication system 500 (see Figure 5 ). In such an embodiment, one or more of Tx host computers 524, 526, and 528 and / or Rx host computer 540 can each be implemented as one or more of PCs 1602, 1618, 1620. Packet data network 522 can be implemented as medium 1622.

[0180] In at least one embodiment, networking computer system 1600A can be used to implement data communication system 810 (see Figure 8 ). In such an embodiment, O-DU 812 and / or O-RU 814 can each be implemented as one or more of PCs 1602, 1618, 1620. Network 816 can be implemented as medium 1622.

[0181] Figure 16B Networking computer system 1600B is shown, in accordance with at least one embodiment. In at least one embodiment, system 1600B shows a network, such as LAN 1624, which can be used to interconnect various nodes that can communicate with each other. In at least one embodiment, attached to LAN 1624 are a number of nodes, such as PC nodes 1626, 1628, 1630. In at least one embodiment, nodes can also connect to a LAN via a web server or other device. In at least one embodiment, system 1600B includes other types of nodes or elements, for example, including routers, servers, and nodes.

[0182] In at least one embodiment, networked computer system 1600B can be used to implement system 100 (see Figure 1 ) In such an embodiment, sender device 102 and / or receiver device 104 can each be implemented as one or more of PC nodes 1626, 1628, 1630. Network 106 can be implemented as LAN 1624.

[0183] In at least one embodiment, networked computer system 1600B can be used to implement data communication system 500 (see Figure 5 ) In such an embodiment, one or more of Tx host computers 524, 526, and 528 and / or Rx host computer 540 can each be implemented as one or more of PC nodes 1626, 1628, 1630. Packet data network 522 can be implemented as LAN 1624.

[0184] In at least one embodiment, networked computer system 1600B can be used to implement data communication system 810 (see Figure 8 ) In such an embodiment, O-DU 812 and / or O-RU 814 can each be implemented as one or more of PC nodes 1626, 1628, 1630. Network 816 can be implemented as LAN 1624.

[0185] Figure 16C Networked computer system 1600C is shown in accordance with at least one embodiment. In at least one embodiment, system 1600C shows a WWW system with communication across a backbone communication network, such as Internet 1632, which can be used to interconnect various nodes of a network. In at least one embodiment, the WWW is a set of protocols that operate on top of the Internet, and allows graphical interface systems to operate on it in order to access information over the Internet. In at least one embodiment, attached to Internet 1632 in the WWW are a number of nodes, such as PCs 1640, 1642, 1644. In at least one embodiment, nodes interface with other nodes of the WWW through a WWW HTTP server, such as servers 1634, 1636. In at least one embodiment, PC 1644 can be a PC that forms a node of network 1632, and PC 1644 itself runs its server 1636, although PC 1644 and server 1636 are shown separately in Figure 16C for purposes of illustration.

[0186] In at least one embodiment, the WWW is a distributed type of application characterized by WWW HTTP, the protocol of the WWW, which runs on top of the Transmission Control Protocol / Internet Protocol (“TCP / IP”) of the Internet. In at least one embodiment, the WWW can thus be characterized by a set of protocols (i.e., HTTP) that run on the Internet as its “backbone.”

[0187] In at least one embodiment, a web browser is an application that runs on a node of a network system compatible with the WWW type, which allows a user of a particular server or node to view such information, and thus allows the user to search through graphical and text-based files that are linked together using hypertext links embedded in documents or files available from servers on a network that understand HTTP. In at least one embodiment, when a user uses another server on a network such as the Internet to retrieve a given web page of a first server associated with a first node, the retrieved document can have different hypertext links embedded in it, and a local copy of the page is created locally at the retrieval user’s machine. In at least one embodiment, when a user clicks on a hypertext link, the locally stored information related to the selected hypertext link is typically sufficient to allow the user’s machine to open a connection over the Internet to the server indicated by the hypertext link.

[0188] In at least one embodiment, more than one user can be coupled to each HTTP server, e.g., through a LAN such as LAN 1638, such as shown with respect to WWW HTTP server 1634. In at least one embodiment, system 1600C can also include other types of nodes or elements. In at least one embodiment, a WWW HTTP server is an application that runs on a machine such as a PC. In at least one embodiment, each user can be considered to have a unique “server,” as shown with respect to PC 1644. In at least one embodiment, a server can be considered to be a server such as WWW HTTP server 1634 that provides access to a network for a LAN or more nodes or more LANs. In at least one embodiment, there are multiple users, each with a desktop PC or node of a network, each desktop PC potentially setting up a server for its user. In at least one embodiment, each server is associated with a particular network address or URL that, when accessed, provides a default web page for that user. In at least one embodiment, the web page can contain further links (embedded URLs) that point to further sub-pages of that user on that server, or to other servers on the network or to pages on other servers on the network.

[0189] In at least one embodiment, networked computer system 1600C can be used to implement system 100 (seeFigure 1 ) In such embodiments, the sender device 102 and / or the receiver device 104 can each be implemented as one or more of the PCs 1640, 1642, 1644. The network 106 can be implemented as the LAN 1638.

[0190] In at least one embodiment, the networked computer system 1600C can be used to implement the data communication system 500 (see Figure 5 ) In such embodiments, the Tx host computers 524, 526, and 528 and / or one or more of the Rx host computers 540 can each be implemented as one or more of the PCs 1640, 1642, 1644. The packet data network 522 can be implemented as the LAN 1638.

[0191] In at least one embodiment, the networked computer system 1600C can be used to implement the data communication system 810 (see Figure 8 ) In such embodiments, the O-DU 812 and / or the O-RU 814 can each be implemented as one or more of the PCs 1640, 1642, 1644. The network 816 can be implemented as the LAN 1638.

[0192] Cloud computing and services

[0193] The following figures set forth, but are not limited to, example cloud-based systems that can be used to implement at least one embodiment.

[0194] In at least one embodiment, cloud computing is a style of computing in which dynamically scalable and often virtualized resources are offered as services over the Internet. In at least one embodiment, users do not need to have knowledge, expertise, or control over the technical infrastructure in which the services operate, which can be referred to as being “in the cloud.” In at least one embodiment, cloud computing converges infrastructure as a service, platform as a service, software as a service, and other variants with a common theme of reliance on internet-dependent common themes to meet users’ computing needs. In at least one embodiment, a typical cloud deployment, such as in a private cloud (e.g., an enterprise network) or a data center (DC) in a public cloud (e.g., the Internet), can consist of thousands of servers (or alternatively, VMs), hundreds of Ethernet, Fibre Channel or Fibre Channel over Ethernet (FCoE) ports, switching and storage infrastructure, etc. In at least one embodiment, a cloud can also consist of network services infrastructure, such as IPsec VPN hubs, firewalls, load balancers, wide area network (WAN) optimizers, etc. In at least one embodiment, remote subscribers can securely access cloud applications and services by connecting via a VPN tunnel, such as an IPsec VPN tunnel.

[0195] In at least one embodiment, cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.

[0196] In at least one embodiment, cloud computing is characterized by on-demand self-service, wherein consumers can unilaterally provision computing capability, such as server time and network storage, as needed automatically (without human interaction with each service provider). In at least one embodiment, cloud computing is characterized by broad network access, wherein capabilities are available over a network and accessed through standard mechanisms that promote the use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs). In at least one embodiment, cloud computing is characterized by resource pooling, wherein provider’s computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand. In at least one embodiment, there is a sense of location independence, as consumers generally do not have control or knowledge of the exact location of the provided resources, but can be able to specify location at a higher level of abstraction (e.g., country, state, or data center). In at least one embodiment, examples of resources include storage, processing, memory, network bandwidth, and virtual machines. In at least one embodiment, cloud computing is characterized by rapid elasticity, wherein capabilities can be rapidly and elastically provisioned, in some cases automatically, to quickly scale out and rapidly release to quickly scale in. In at least one embodiment, for consumers, the capabilities available for provisioning commonly appear to be unlimited and can be purchased in any quantity at any time. In at least one embodiment, cloud computing is characterized by measured service, wherein cloud systems automatically control and optimize resource usage by leveraging a metering capability at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). In at least one embodiment, resource usage can be monitored, controlled, and reported providing transparency for both the provider and consumer of the utilized service.

[0197] In at least one embodiment, cloud computing can be associated with various services. In at least one embodiment, cloud software as a service (SaaS) can refer to the capability provided to the consumer is to use the provider’s applications running on a cloud infrastructure. In at least one embodiment, the applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based e-mail). In at least one embodiment, the consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with a possible exception of limited user-specific application configuration settings.

[0198] In at least one embodiment, cloud platform as a service (PaaS) can refer to a service in which consumers have the ability to deploy onto the cloud infrastructure created or acquired applications and interfaces created using programming languages and tools supported by the provider. In at least one embodiment, consumers do not manage or control underlying cloud infrastructure including network, servers, operating systems, or storage, but have control over the deployed applications and possibly application hosting environment configurations.

[0199] In at least one embodiment, cloud infrastructure as a service (IaaS) can refer to a service in which a consumer can deploy and run arbitrary software which can include an operating system and applications. In at least one embodiment, the consumer does not manage or control or does not have control over the underlying cloud infrastructure including network, servers, operating systems, or storage, but has control over operating systems, storage, deployed applications, and possibly limited control of select networking components (e.g., host firewalls).

[0200] In at least one embodiment, cloud computing can be deployed in different ways. In at least one embodiment, a private cloud can refer to the cloud infrastructure on behalf of a single organization. In at least one embodiment, a private cloud can be managed by the organization or a third party and can exist on-premises or off-premises. In at least one embodiment, a community cloud can refer to the cloud infrastructure shared by several organizations or community of interest. In at least one embodiment, a community cloud can be managed by the organizations or a third party and can exist on-premises or off-premises. In at least one embodiment, a public cloud can refer to the cloud infrastructure made available to general public or a large industry group. In at least one embodiment, a hybrid cloud can refer to the blending of two or more types of cloud infrastructures, for example, private, community or public, which remain unique entities but are bound together, using standard or proprietary technologies to enable data and application portability. In at least one embodiment, a cloud computing environment is service-oriented with a focus on statelessness, low coupling, modularity, and semantic interoperability.

[0201] Figure 17One or more components of a system environment 1700, in accordance with one or more embodiments, in which services can be provided as third party network services, are shown. In at least one embodiment, the third party network can be referred to as a cloud, a cloud network, a cloud computing network, and / or variations thereof. In at least one embodiment, system environment 1700 includes one or more client computing devices 1704, 1706, and 1708, which can be used by users to interact with a third party network infrastructure system 1702 that provides third party network services, which can be referred to as cloud computing services. In at least one embodiment, third party network infrastructure system 1702 can include one or more computers and / or servers.

[0202] It should be appreciated that Figure 17 Third party network infrastructure system 1702, depicted in Figure 17 Embodiments of a third party network infrastructure system are depicted. In at least one embodiment, third party network infrastructure system 1702 can have more or fewer components than those depicted in Figure 17 In at least one embodiment, depicted in

[0203] In at least one embodiment, client computing devices 1704, 1706, and 1708 can be configured to operate a client application such as a web browser, a proprietary client application, or some other application that can be used by a user of the client computing device to interact with third party network infrastructure system 1702 to use services provided by third party network infrastructure system 1702. Although example system environment 1700 is illustrated and described as having three client computing devices, any number of client computing devices can be supported. In at least one embodiment, other devices, such as devices having sensors, etc., can interact with third party network infrastructure system 1702. In at least one embodiment, one or more networks 1710 can facilitate communications and exchange of data between client computing devices 1704, 1706, and 1708 and third party network infrastructure system 1702.

[0204] In at least one embodiment, services provided by third party network infrastructure system 1702 can include hosting of services that are available on demand to users of the third party network infrastructure system. In at least one embodiment, various services can also be provided, including but not limited to online data storage and backup solutions, Web-based electronic mail services, managed office suites and document collaboration services, database management and processing, managed technical support services, and / or variations thereof. In at least one embodiment, services provided by the third party network infrastructure system can dynamically scale to meet the needs of its users.

[0205] In at least one embodiment, a particular instantiation of a service provided by a third party network infrastructure system 1702 can be referred to as a “service instance.” In at least one embodiment, generally, any service available to a user from a third party network service provider system over a communication network, such as the internet, is referred to as a “third party network service.” In at least one embodiment, in a public third party network environment, servers and systems that make up a third party network service provider system are distinct from a customer’s own on-premise servers and systems. In at least one embodiment, a third party network service provider system can host an application, and a user can order and utilize the application on-demand via a communication network, such as the internet.

[0206] In at least one embodiment, a service in a computer network third party network infrastructure can include protected computer network access to storage, a hosted database, a hosted web server, a software application, or other service provided by a third party network provider to a user. In at least one embodiment, a service can include password protected access to remote storage on a third party network over the internet. In at least one embodiment, a service can include a web services-based hosted relational database and a scripting language middleware engine for private use by a networked developer. In at least one embodiment, a service can include access to an email software application hosted on a third party network provider’s website.

[0207] In at least one embodiment, third party network infrastructure system 1702 can include a suite of applications, middleware, and database service offerings that are delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. In at least one embodiment, third party network infrastructure system 1702 can also provide “big data” related computing and analytics services. In at least one embodiment, the term “big data” is commonly used to describe the extremely large data sets that can be stored and manipulated by analysts and researchers in order to visualize large amounts of data, detect trends, and / or otherwise interact with the data. In at least one embodiment, big data and related applications can be hosted and / or manipulated by infrastructure systems at many levels and in different scales. In at least one embodiment, tens, hundreds, or thousands of processors linked in parallel can act on such data in order to present the data or simulate an outside force on the data or what it represents. In at least one embodiment, these data sets can involve structured data such as structured data organized in databases or otherwise according to structured models and / or unstructured data (e.g., email, images, data blobs (binary large objects), web pages, complex event processing). In at least one embodiment, by leveraging the ability of embodiments to relatively quickly focus more (or less) computing resources on a target, third party network infrastructure system can be better utilized to perform tasks on big data sets based on demand from businesses, government agencies, research organizations, private individuals, groups of like-minded individuals or organizations, or other entities.

[0208] In at least one embodiment, third party network infrastructure system 1702 can be adapted to automatically provide, manage and track customer subscriptions for services provided by third party network infrastructure system 1702. In at least one embodiment, third party network infrastructure system 1702 can provide third party network services via different deployment models. In at least one embodiment, services can be provided under a public third party network model in which third party network infrastructure system 1702 is owned by an organization that sells third party network services and makes the services available to the general public or different industry enterprises. In at least one embodiment, services can be provided under a private third party network model in which third party network infrastructure system 1702 operates for a single organization only and can provide services for one or more entities within the organization. In at least one embodiment, third party network services can also be provided under a community third party network model in which third party network infrastructure system 1702 and the services provided by third party network infrastructure system 1702 are shared by several organizations in a related community. In at least one embodiment, third party network services can also be provided under a hybrid third party network model, which is a combination of two or more different models.

[0209] In at least one embodiment, services provided by third party network infrastructure system 1702 can include one or more services provided under Software as a Service (SaaS) category, Platform as a Service (PaaS) category, Infrastructure as a Service (IaaS) category, or other categories of services including hybrid services. In at least one embodiment, a customer, via a subscription order, can order one or more services provided by third party network infrastructure system 1702. In at least one embodiment, third party network infrastructure system 1702 then performs processes to provide the services in customer’s subscription order.

[0210] In at least one embodiment, services provided by third party network infrastructure system 1702 can include, without limitation, application services, platform services, and infrastructure services. In at least one embodiment, application services can be provided by third party network infrastructure system via a SaaS platform. In at least one embodiment, SaaS platform can be configured to provide third party network services that fall under SaaS category. In at least one embodiment, SaaS platform can provide capabilities to build and deliver a suite of on-demand applications on an integrated development and deployment platform. In at least one embodiment, SaaS platform can manage and control underlying software and infrastructure for providing the SaaS services. In at least one embodiment, by utilizing the services provided by SaaS platform, customers can utilize applications that are executed on third party network infrastructure system. In at least one embodiment, customers can obtain application services without the customer’s need to purchase, provision, and manage the underlying hardware and software. In at least one embodiment, various different SaaS services can be provided. In at least one embodiment, examples include, without limitation, services that provide solutions for sales performance management, enterprise integration, and business flexibility for large organizations.

[0211] In at least one embodiment, platform services can be provided by third party network infrastructure system 1702 via a PaaS platform. In at least one embodiment, PaaS platform can be configured to provide third party network services that fall under PaaS category. In at least one embodiment, examples of platform services can include, without limitation, services that enable organizations to create, deliver, and optimize their own applications, websites, and services without managing the underlying hardware and software. In at least one embodiment, platform services can enable customers to focus on business logic, rather than systems management. In at least one embodiment, PaaS platform can manage and control the underlying software and infrastructure for providing the PaaS services. In at least one embodiment, customers can acquire PaaS services provided by third party network infrastructure system 1702 without the need for customers to purchase, provision, and manage the underlying hardware and software.

[0212] In at least one embodiment, by utilizing the services provided by a PaaS platform, customers can use programming languages and tools supported by third-party network infrastructure systems and also control deployed services. In at least one embodiment, platform services provided by a third-party network infrastructure system can include database third-party network services, middleware third-party network services, and third-party network services. In at least one embodiment, database third-party network services can support shared service deployment models that enable organizations to pool database resources and offer customers database as a service in the form of a database third-party network. In at least one embodiment, in a third-party network infrastructure system, middleware third-party network services can provide a platform for customers to develop and deploy various business applications, and third-party network services can provide a platform for customers to deploy applications.

[0213] In at least one embodiment, various different infrastructure services can be provided by an laaS platform in a third-party network infrastructure system. In at least one embodiment, infrastructure services facilitate management and control of the underlying computing resources (such as storage, networking, and other fundamental computing resources) by customers that utilize the services provided by SaaS platforms and PaaS platforms.

[0214] In at least one embodiment, a third-party network infrastructure system 1702 can also include infrastructure resources 1730 for providing resources used to offer various services to customers of the third-party network infrastructure system. In at least one embodiment, infrastructure resources 1730 can include pre-integrated and optimized combinations of hardware, such as servers, storage, and networking resources, for the execution of services offered by PaaS platforms and SaaS platforms, among other resources.

[0215] In at least one embodiment, resources in a third-party network infrastructure system 1702 can be shared by multiple users and dynamically re-allocated per demand. In at least one embodiment, resources can be allocated to users in different time zones. In at least one embodiment, a third-party network infrastructure system 1702 can enable a first group of users in a first time zone to utilize resources of the third-party network infrastructure system for a specified number of hours and subsequently enable reallocation of same resources to another group of users located in a different time zone, thereby maximizing resource utilization.

[0216] In at least one embodiment, multiple internal shared services 1732, shared by different components or modules of the third-party network infrastructure system 1702, may be provided to enable services provided by the third-party network infrastructure system 1702. In at least one embodiment, these internal shared services may include, but are not limited to, security and identity services, integration services, enterprise library services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, services for enabling third-party network support, email services, notification services, file transfer services, and / or variations thereof.

[0217] In at least one embodiment, the third-party network infrastructure system 1702 can provide comprehensive management of third-party network services (e.g., SaaS, PaaS, and IaaS services) within the third-party network infrastructure system. In at least one embodiment, the third-party network management functionality may include the ability and / or variations thereof for provisioning, managing, and tracking customer subscriptions received by the third-party network infrastructure system 1702.

[0218] In at least one embodiment, such as Figure 17 As shown, third-party network management functions can be provided by one or more modules, such as order management module 1720, order coordination module 1722, order supply module 1724, order management and monitoring module 1726, and identity management module 1728. In at least one embodiment, these modules may include or be provided using one or more computers and / or servers, which may be general-purpose computers, dedicated server computers, server farms, server clusters, or any other suitable arrangement and / or combination.

[0219] In at least one embodiment, in step 1734, a customer using a client device (such as client computing device 1704, 1706, or 1708) interacts with the third-party network infrastructure system 1702 by requesting one or more services provided by the third-party network infrastructure system 1702 and placing an order for a subscription to one or more services provided by the third-party network infrastructure system 1702. In at least one embodiment, the customer may access a third-party network user interface (UI), such as third-party network UI 1712, third-party network UI 1714, and / or third-party network UI 1716, and place orders via these UIs. In at least one embodiment, order information received by the third-party network infrastructure system 1702 in response to a customer placing an order may include information identifying the customer and the one or more services provided by the third-party network infrastructure system 1702 that the customer wishes to subscribe to.

[0220] In at least one embodiment, order information received from a customer can be stored in an order database 1718 at step 1736. In at least one embodiment, if this is a new order, a new record can be created for the order. In at least one embodiment, order database 1718 can be one of several databases operated by third party network infrastructure system 1718 and in conjunction with other system elements.

[0221] In at least one embodiment, order information can be forwarded to an order management module 1720 at step 1738, which can be configured to perform billing and accounting functions related to orders, such as validating an order, and upon validation, provisioning an order.

[0222] In at least one embodiment, information about an order can be communicated to an order coordination module 1722 at step 1740, which is configured to coordinate the provisioning of services and resources for orders placed by customers. In at least one embodiment, order coordination module 1722 can use the services of an order provisioning module 1724 for provisioning. In at least one embodiment, order coordination module 1722 enables management of business processes associated with each order and application of business logic to determine whether an order should continue with provisioning.

[0223] In at least one embodiment, when a new subscription order is received, order coordination module 1722 sends a request to order provisioning module 1724 to allocate resources and configure resources needed to fulfill the subscription order at step 1742. In at least one embodiment, order provisioning module 1724 implements resource allocation for services ordered by customers. In at least one embodiment, order provisioning module 1724 provides a level of abstraction between third party network services offered by third party network infrastructure system 1700 and physical implementation layers used for resources to be applied to provision the requested services. In at least one embodiment, this enables order coordination module 1722 to be isolated from implementation details, such as whether services and resources are provisioned in real-time or pre-provisioned and only allocated / assigned upon request.

[0224] In at least one embodiment, once services and resources are provisioned, a notification can be sent to a subscribing customer indicating that the requested services are now ready for use at step 1744. In at least one embodiment, information (e.g., a link) can be sent to the customer that enables the customer to begin using the requested services.

[0225] In at least one embodiment, at step 1746, orders for customer subscriptions can be managed and tracked by order management and monitoring module 1726. In at least one embodiment, order management and monitoring module 1726 can be configured to collect usage statistics regarding customer usage of a subscription service. In at least one embodiment, statistics can be collected for amount of storage used, amount of data transferred, number of users, and amount and / or changes in system up times and system down times.

[0226] In at least one embodiment, third party network infrastructure system 1700 can include an identity management module 1728 that is configured to provide identity services, such as access management and authorization services in third party network infrastructure system 1700. In at least one embodiment, identity management module 1728 can control information about customers who wish to utilize services provided by third party network infrastructure system 1702. In at least one embodiment, such information can include information that authenticates the identities of such customers and information that describes which actions are authorized for those customers with respect to various system resources (e.g., files, directories, applications, communication ports, memory segments, etc.). In at least one embodiment, identity management module 1728 can also include functionality to manage descriptive information about each customer and about how that descriptive information can be accessed and modified as well as by whom.

[0227] In at least one embodiment, system environment 1700 can be used to implement system 100 (see Figure 1 ). In such an embodiment, sender device 102 and / or receiver device 104 can each be implemented as one or more of client computing devices 1704, 1706, and 1708 or third party network infrastructure system 1702. Network 106 can be implemented as network 1710.

[0228] In at least one embodiment, system environment 1700 can be used to implement data communication system 500 (see Figure 5 ). In such an embodiment, one or more of Tx host computers 524, 526, and 528 and / or Rx host computer 540 can each be implemented as one or more of client computing devices 1704, 1706, and 1708 or third party network infrastructure system 1702. Packet data network 522 can be implemented as network 1710.

[0229] In at least one embodiment, system environment 1700 can be used to implement data communication system 810 (see Figure 8). In such embodiments, the O-DU 812 and / or O-RU 814 can each be implemented as one or more of the client computing devices 1704, 1706, and 1708 or the third-party network infrastructure system 1702. The network 816 can be implemented as the network 1710.

[0230] Figure 18 A cloud computing environment 1802 is shown in accordance with at least one embodiment. In at least one embodiment, the cloud computing environment 1802 includes one or more computer systems / servers 1804, such as computing devices of a personal digital assistant (PDA) or cellular telephone 1806A, a desktop computer 1806B, a laptop computer 1806C, and / or a computer system 1806N of an automobile that communicate with the one or more computer systems / servers 1804. In at least one embodiment, this allows infrastructure, platforms, and / or software to be offered as services available from the cloud computing environment 1802, with the client devices not needing to maintain such resources individually. It should be understood that Figure 18 The types of computing devices 1806A-N shown in FIG. 18 are intended to be illustrative only and that the cloud computing environment 1802 can communicate with any type of computerized device over any type of network and / or network / addressable connection (e.g., using a web browser).

[0231] In at least one embodiment, the computer system / server 1804 can be operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of computing systems, environments, and / or configurations that can be suitable for use with computer system / server 1804 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices, and / or variants thereof.

[0232] In at least one embodiment, the computer system / server 1804 can be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. In at least one embodiment, program modules include routines, programs, objects, components, logic, data structures, and the like that perform particular tasks or implement particular abstract data types. In at least one embodiment, an exemplary computer system / server 1804 can be practiced in distributed cloud computing environments with remote processing devices that are linked through a communications network. In at least one embodiment, in a distributed cloud computing environment, program modules can be located in both local and remote computer system storage media including memory storage devices.

[0233] In at least one embodiment, cloud computing environment 1802 can be used to implement system 100 (see Figure 1 ). In such an embodiment, sender device 102 and / or recipient device 104 can each be implemented as one or more of computer system / server 1804, PDA or cellular telephone 1806A, desktop computer 1806B, laptop computer 1806C, or automobile computer system 1806N.

[0234] In at least one embodiment, cloud computing environment 1802 can be used to implement data communication system 500 (see Figure 5 ). In such an embodiment, one or more of Tx host computers 524, 526, and 528 and / or Rx host computer 540 can each be implemented as one or more of computer system / server 1804, PDA or cellular telephone 1806A, desktop computer 1806B, laptop computer 1806C, or automobile computer system 1806N.

[0235] In at least one embodiment, cloud computing environment 1802 can be used to implement data communication system 810 (see Figure 8 ). In such an embodiment, O-DU 812 and / or O-RU 814 can each be implemented as one or more of computer system / server 1804, PDA or cellular telephone 1806A, desktop computer 1806B, laptop computer 1806C, or automobile computer system 1806N.

[0236] Figure 19 A set of functional abstraction layers provided by cloud computing environment 1802 (see Figure 18 ) are shown. It should be understood that Figure 19 the components, layers, and functions shown in FIG. 19 are intended to be illustrative only and that components, layers, and functions can be different than shown in FIG. 19.

[0237] In at least one embodiment, hardware and software layer 1902 includes hardware and software components. In at least one embodiment, examples of hardware components include mainframes, in one

[0238] In at least one embodiment, virtualization layer 1904 provides an abstraction layer from which the following exemplary virtual entities can be provided: virtual servers; virtual storage; virtual networks, including virtual private networks; virtual applications and virtual clients; and / or other virtual entities.

[0239] In at least one embodiment, management layer 1906 provides various functions. In at least one embodiment, resource provisioning provides dynamic procurement of computing resources and other resources that are utilized to perform tasks within the cloud computing environment. In at least one embodiment, metering provides usage tracking of resources in use, for example, the usage of application software licenses. In at least one embodiment, provisioning provides a single point of management functionality that enables ubiquitous underwriting and rapid deployment of resources by orchestrating different layers of the cloud computing environment. In at least one embodiment, security provides identity verification for users and tasks, as well as protection for data and other resources. In at least one embodiment, user interface provides access to the cloud computing environment for both users and system administrators. In at least one embodiment, service level management provides cloud computing resource allocation and management such that required service levels are met. In at least one embodiment, service level agreement (SLA) planning and fulfillment provides pre-arrangement for, and fulfillment of, cloud computing resources to provide desired level of service to a user.

[0240] In at least one embodiment, workload layer 1908 provides functionality for which the cloud computing environment can be utilized. In at least one embodiment, examples of workloads and functions which can be provided from this layer include: mapping and navigation; software development and lifecycle management; virtual classroom education delivery; data analysis and mining; transaction processing; and, mission critical management.

[0241] Supercomputing

[0242] The following figures set forth, but are not limited to, exemplary supercomputer-based systems that can be used to implement at least one embodiment.

[0243] In at least one embodiment, a supercomputer can refer to a hardware system that exhibits significant parallelism and includes at least one chip, where the chips in the system are interconnected by a network and are placed in a hierarchically organized enclosure. In at least one embodiment, a large hardware system that fills a room with several racks, each containing several board / rack modules, each containing several chips all interconnected by a scalable network, is one specific example of a supercomputer. In at least one embodiment, a single rack of such a large hardware system is another example of a supercomputer. In at least one embodiment, a single chip that exhibits significant parallelism and contains several hardware components can also be considered a supercomputer, as the amount of hardware that can be incorporated in a single chip can increase as feature sizes can decrease.

[0244] Figure 20A supercomputer at the chip level is shown, according to at least one embodiment. In at least one embodiment, within an FPGA or ASIC chip, primary computation is performed within a finite state machine (2004) called a thread unit. In at least one embodiment, a task and synchronization network (0202) connects finite state machines and is used to dispatch threads and perform operations in the correct order. In at least one embodiment, a memory network (2006, 2010) is used to access a multi-level partitioned on-chip cache hierarchy (2008, 2012). In at least one embodiment, an off-chip memory network (2014) is used to access off-chip memory using a memory controller (2016). In at least one embodiment, an I / O controller (2018) is used for cross-chip communication when a design does not fit on a single logic chip.

[0245] Figure 21 A supercomputer at the rack module level is shown, according to at least one embodiment. In at least one embodiment, within a rack module, there are multiple FPGA or ASIC chips (2102) connected to one or more DRAM units (2104) that make up a main accelerator memory. In at least one embodiment, each FPGA / ASIC chip is connected to its neighboring FPGA / ASIC chip using a wide bus on board with differential high-speed signaling (2106). In at least one embodiment, each FPGA / ASIC chip is also connected to at least one high-speed serial communication cable.

[0246] Figure 22 A supercomputer at the rack level is shown, according to at least one embodiment. Figure 23 A supercomputer at the entire system level is shown, according to at least one embodiment. In at least one embodiment, see Figure 22 and Figure 23Between and across racks of rack modules in a rack, using high-speed serial optical or copper cables (2202, 2302) to implement an extensible, possibly incomplete hypercube network. In at least one embodiment, one of the FPGA / ASIC chips of an accelerator is connected to a host system (2304) through a PCI-Express connection. In at least one embodiment, the host system includes a host microprocessor (2308) on which software portions of an application run, and a memory consisting of one or more host memory DRAM units (2306) that are kept coherent with the memory on the accelerator. In at least one embodiment, the host system can be a separate module on one of the racks, or can be integrated with one of the modules of a supercomputer. In at least one embodiment, a cyclic topology of cube connections provides communication links to create a hypercube network for a large supercomputer. In at least one embodiment, a small group of FPGA / ASIC chips on a rack module can act as a single hypercube node, such that the total number of external links per group is increased compared to a single chip. In at least one embodiment, a group contains chips A, B, C, and D on a rack module, which has an internal wide differential bus connecting A, B, C, and D in a ring organization. In at least one embodiment, there are 12 serial communication cables that connect the rack module to the outside world. In at least one embodiment, chip A on a rack module is connected to serial communication cables 0, 1, 2. In at least one embodiment, chip B is connected to cables 3, 4, 5. In at least one embodiment, chip C is connected to 6, 7, 8. In at least one embodiment, chip D is connected to 9, 10, 11. In at least one embodiment, the entire group {A, B, C, D} making up a rack module can form a hypercube node within a supercomputer system, with up to 212= 4096 rack modules (16,384 FPGA / ASIC chips). In at least one embodiment, in order for chip A to send a message out on link 4 of the group {A, B, C, D}, the message must first be routed to chip B with an on-board differential wide bus connection. In at least one embodiment, a message arriving on link 4 destined for the group {A, B, C, D} of chips (i.e., to B) must also first be routed to the correct destination chip (A) inside the group {A, B, C, D}. In at least one embodiment, other sizes of parallel supercomputer systems can also be implemented.

[0247] In at least one embodiment, sender device 102 and / or receiver device 104 can each be implemented using a supercomputer of at least one of Figures 20-23 In at least one embodiment, one or more of Tx host computers 524, 526, and 528 and / or Rx host computer 540 can each be implemented using a supercomputer of at least one of Figures 20-23at least one of the supercomputers 2102, 2104, and / or 2106. In at least one embodiment, each of O-DU 812 and / or O-RU 814 can be implemented using Figures 20-23 at least one of the supercomputers 2102, 2104, and / or 2106.

[0248] Artificial Intelligence

[0249] The following figures illustrate, but are not limited to, example artificial intelligence-based systems that can be used to implement at least one embodiment.

[0250] Figure 24A Inference and / or training logic 2415 are used to perform inferencing and / or training operations associated with one or more embodiments. Details regarding inference and / or training logic 2415 are described in more detail below in conjunction with FIGS. 2 and 5. Figure 24A and / or Figure 24B Details regarding inference and / or training logic 2415 are provided below in conjunction with FIGS. 2 and 5.

[0251] In at least one embodiment, inference and / or training logic 2415 can include, without limitation, code and / or data storage 2401 for storing forward and / or output weight and / or input / output data, and / or other parameters for configuring neurons or layers of a neural network being trained and / or used for inferencing in aspects of one or more embodiments. In at least one embodiment, training logic 2415 can include or be coupled to code and / or data storage 2401 for storing graph code or other software to control timing and / or order, in which weight and / or other parameter information will be loaded to configure logic, including integer and / or floating point units (collectively, arithmetic logic unit(s) (ALU(s))).

[0252] In at least one embodiment, any portions of code and / or data storage 2401 can be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, code and / or code and / or data storage 2401 can be cache memory, dynamic random addressable memory (“DRAM”), static random addressable memory (“SRAM”), non-volatile memory (e.g., Flash), or other storage. In at least one embodiment, whether code and / or code and / or data storage 2401 is internal or external to a processor, e.g., or includes DRAM, SRAM, Flash, or some other storage type, can depend on available storage on-chip versus off-chip, latency requirements of training and / or inferencing functions being performed, batch size of data used in inferencing and / or training of neural networks, or some combination of these factors.

[0253] In at least one embodiment, inference and / or training logic 2415 can include, without limitation, code and / or data storage 2405 for storing backward and / or output weights and / or input / output data corresponding to neurons or layers of a neural network being trained and / or used for inferencing in aspects of one or more embodiments. In at least one embodiment, code and / or data storage 2405 stores weight parameters and / or input / output data for each layer of a neural network that is trained in conjunction with one or more embodiments during backpropagation of input / output data and / or weight parameters during training and / or inferencing using aspects of one or more embodiments. In at least one embodiment, training logic 2415 can include or be coupled to code and / or data storage 2405 to store graph code or other software to control timing and / or sequence in which weight and / or other parameter information will be loaded to configure logic, including integer and / or floating point units (collectively, arithmetic logic units (ALUs)).

[0254] In at least one embodiment, code such as graph code causes a processor ALU to load weight or other parameter information into based on an architecture of a neural network that such code corresponds to. In at least one embodiment, any portion of code and / or data storage 2405 can be included with other on-chip or off-chip data storage, including a processor’s LI, L2, or L3 cache or system memory. In at least one embodiment, any portion of code and / or data storage 2405 can be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, code and / or data storage 2405 can be cache memory, DRAM, SRAM, non-volatile memory (e.g., Flash) or other storage. In at least one embodiment, whether code and / or data storage 2405 is internal or external to a processor, e.g., or comprises DRAM, SRAM, Flash, or some other storage type, can depend on available storage on-chip versus off-chip, latency requirements of training and / or inferencing functions being performed, batch size of data used in inferencing and / or training of a neural network, or some combination of these factors.

[0255] In at least one embodiment, code and / or data storage 2401 and code and / or data storage 2405 can be separate storage structures. In at least one embodiment, code and / or data storage 2401 and code and / or data storage 2405 can be combined storage structures. In at least one embodiment, code and / or data storage 2401 and code and / or data storage 2405 can be partially combined and partially separate. In at least one embodiment, any portion of code and / or data storage 2401 and code and / or data storage 2405 can be included with other on-chip or off-chip data storage, including a processor’s LI, L2, or L3 cache or system memory.

[0256] In at least one embodiment, inference and / or training logic 2415 can include, without limitation, one or more arithmetic logic units (“ALUs”) 2410, including integer and / or floating point units, for performing logical and / or mathematical operations based, at least in part, on training and / or inference code (e.g., graphics code) or instructions therefrom. Results of such operations can produce activations (e.g., output values from layers or neurons within a neural network) that are stored in activation storage 2420, which is a function of input / output and / or weight parameter data stored in code and / or data storage 2401 and / or code and / or data storage 2405. In at least one embodiment, activations stored in activation storage 2420 are generated in accordance with linear algebra and / or matrix-based mathematics performed by ALUs 2410 in response to executing instructions or other code, where weight values stored in code and / or data storage 2405 and / or data storage 2401 are used as operands along with other values such as bias values, gradient information, momentum values, or other parameters or hyperparameters, any or all of which can be stored in code and / or data storage 2405 or code and / or data storage 2401 or another storage on- or off-chip.

[0257] In at least one embodiment, one or more ALUs 2410 are included in one or more processors or other hardware logic devices or circuits, while in another embodiment, one or more ALUs 2410 can be external to a processor or other hardware logic device or circuit using them (e.g., a co-processor). In at least one embodiment, ALUs 2410 can be included within execution units of a processor or otherwise within a bank of ALUs accessible by execution units of a processor, either within same processor or distributed between different types of different processors (e.g., central processing units, graphics processing units, fixed function units, etc.). In at least one embodiment, code and / or data storage 2401, code and / or data storage 2405, and activation storage 2420 can share a processor or other hardware logic device or circuit, while in another embodiment, they can be in different processors or other hardware logic devices or circuits, or in some combination of same and different processors or other hardware logic devices or circuits. In at least one embodiment, any portion of activation storage 2420 can be included with other on-chip or off-chip data storage, including L1, L2, or L3 cache(s) of a processor or system memory. Moreover, inference and / or training code can be stored with other code that is accessible and useable by a processor or other hardware logic or circuitry along with fetch, decode, schedule, execute, retirement, and / or other logic of a processor to fetch and / or process.

[0258] In at least one embodiment, the active memory 2420 may be a cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other storage device. In at least one embodiment, the active memory 2420 may be wholly or partially within or outside one or more processors or other logic circuits. In at least one embodiment, the choice of whether the active memory 2420 is inside or outside the processor, for example, or includes DRAM, SRAM, flash memory, or some other storage type, may depend on the available on-chip storage relative to off-chip storage, the latency requirements of the training and / or inference functions being performed, the batch size of the data used in the inference and / or training of the neural network, or some combination of these factors.

[0259] In at least one embodiment, Figure 24A The inference and / or training logic 2415 shown can be used in conjunction with an application-specific integrated circuit (“ASIC”), such as those from Google. Processing unit, from Graphcore TM Inference processing unit (IPU), or from Intel Corporation (For example, a "Lake Crest" processor. In at least one embodiment, Figure 24A The inference and / or training logic 2415 shown can be used in conjunction with central processing unit (“CPU”) hardware, graphics processing unit (“GPU”) hardware or other hardware such as field programmable gate array (“FPGA”)).

[0260] Figure 24B Inference and / or training logic 2415 according to at least one embodiment is illustrated. In at least one embodiment, the inference and / or training logic 2415 may include, but is not limited to, hardware logic in which computational resources are dedicated or otherwise exclusively used in conjunction with weight values ​​or other information corresponding to one or more neuron layers within a neural network. In at least one embodiment, Figure 24B The inference and / or training logic 2415 shown can be combined with an application-specific integrated circuit (ASIC) (such as those from Google). Processing unit, from Graphcore TM Inference processing unit (IPU), or from Intel Corporation (For example, "Lake Crest") processors are used. In at least one embodiment, Figure 24BThe inference and / or training logic 2415 illustrated in FIG. 24 can be used in conjunction with central processing unit (CPU) hardware, graphics processing unit (GPU) hardware, or other hardware, such as field programmable gate arrays (FPGAs). In at least one embodiment, the inference and / or training logic 2415 includes, without limitation, code and / or data storage 2401 and code and / or data storage 2405, which can be used to store code (e.g., graph code), weight values, and / or other information, including bias values, gradient information, momentum values, and / or other parameter or hyperparameter information. In Figure 24B In at least one embodiment, each of code and / or data storage 2401 and code and / or data storage 2405 is associated with a dedicated computing resource (e.g., computing hardware 2402 and computing hardware 2406, respectively). In at least one embodiment, each of computing hardware 2402 and computing hardware 2406 includes one or more ALUs that perform only mathematical functions (such as linear algebra functions) on information stored in code and / or data storage 2401 and code and / or data storage 2405, respectively, with results being stored in activation storage 2420.

[0261] In at least one embodiment, each code and / or data storage 2401 and 2405 and corresponding computing hardware 2402 and 2406, respectively, correspond to different layers of a neural network, such that resulting activations from one storage / computing pair 2401 / 2402 are provided as input to the next storage / computing pair 2405 / 2406 in code and / or data storage 2405 and computing hardware 2406, in order to mirror the conceptual organization of a neural network. In at least one embodiment, each of storage / computing pairs 2401 / 2402 and 2405 / 2406 can correspond to more than one neural network layer. In at least one embodiment, additional storage / computing pairs (not shown) after or in parallel with storage / computing pairs 2401 / 2402 and 2405 / 2406 can be included in inference and / or training logic 2415.

[0262] In at least one embodiment, sender device 102 and / or receiver device 104 can use inference and / or training logic 2415 to perform inferencing and / or training operations. In at least one embodiment, one or more of Tx host computer 524, 526, and 528 and / or Rx host computer 540 can use inference and / or training logic 2415 to perform inferencing and / or training operations. In at least one embodiment, O-DU 812 and / or O-RU 814 can use inference and / or training logic 2415 to perform inferencing and / or training operations.

[0263] Figure 25 Training and deployment of a deep neural network is shown in accordance with at least one embodiment. In at least one embodiment, an untrained neural network 2506 is trained using a training dataset 2502. In at least one embodiment, training framework 2504 is a PyTorch framework, while in other embodiments, training framework 2504 is a TensorFlow, Boost, Caffe, Microsoft Cognitive Toolkit / CNTK, MXNet, Chainer, Keras, Deeplearning4j, or other training framework. In at least one embodiment, training framework 2504 trains untrained neural network 2506 and enables it to train using processing resources described herein to generate a trained neural network 2508. In at least one embodiment, weights can be chosen randomly or by pre-training using a deep belief network. In at least one embodiment, training can be performed in a supervised, partially supervised, or unsupervised manner.

[0264] In at least one embodiment, untrained neural network 2506 is trained using supervised learning, where training dataset 2502 includes inputs paired with desired outputs for the inputs, or where training dataset 2502 includes inputs with known outputs and the outputs of neural network 2506 are manually graded. In at least one embodiment, untrained neural network 2506 is trained in a supervised manner and inputs from training dataset 2502 are processed and the resulting outputs are compared to a set of expected or desired outputs. In at least one embodiment, errors are then backpropagated through untrained neural network 2506. In at least one embodiment, training framework 2504 adjusts weights that control untrained neural network 2506. In at least one embodiment, training framework 2504 includes tools for monitoring how well untrained neural network 2506 is converging towards a model, such as trained neural network 2508, that is suitable for generating correct answers, such as results 2514, based on input data, such as new dataset 2512. In at least one embodiment, training framework 2504 repeatedly trains untrained neural network 2506 while using a loss function and adjustment algorithm, such as stochastic gradient descent, to adjust weights to refine outputs of untrained neural network 2506. In at least one embodiment, training framework 2504 trains untrained neural network 2506 until untrained neural network 2506 achieves a desired accuracy. In at least one embodiment, trained neural network 2508 can then be deployed to implement any number of machine learning operations.

[0265] In at least one embodiment, unsupervised learning is used to train an untrained neural network 2506, wherein the untrained neural network 2506 attempts to train itself using unlabeled data. In at least one embodiment, the unsupervised learning training dataset 2502 will include input data without any associated output data or "ground truth" data. In at least one embodiment, the untrained neural network 2506 can learn groupings within the training dataset 2502 and can determine how each input relates to the untrained dataset 2502. In at least one embodiment, unsupervised training can be used to generate self-organizing maps in the trained neural network 2508 capable of performing operations useful in reducing the dimensionality of the new dataset 2512. In at least one embodiment, unsupervised training can also be used to perform anomaly detection, which allows the identification of data points in the new dataset 2512 that deviate from the normal patterns of the new dataset 2512.

[0266] In at least one embodiment, semi-supervised learning can be used, which is a technique in which a mixture of labeled and unlabeled data is included in the training dataset 2502. In at least one embodiment, the training framework 2504 can be used to perform incremental learning, such as through transfer learning techniques. In at least one embodiment, incremental learning enables the trained neural network 2508 to adapt to a new dataset 2512 without forgetting the knowledge injected into the trained neural network 1408 during initial training.

[0267] In at least one embodiment, the sending device 102 and / or the receiving device 104 may use Figure 25 The process shown is used to train and / or deploy deep neural networks. In at least one embodiment, one or more of the Tx host computers 524, 526, and 528 and / or the Rx host computer 540 can be used. Figure 25 The process shown is used to train and / or deploy deep neural networks. In at least one embodiment, the O-DU 812 and / or O-RU 814 can be used Figure 25 The process shown is used to train and / or deploy deep neural networks.

[0268] 5G network

[0269] The following figures illustrate, but are not limited to, exemplary 5G-based systems that can be used to implement at least one embodiment.

[0270] Figure 26The architecture of a system 2600 of a network is shown, in accordance with at least one embodiment. In at least one embodiment, system 2600 is shown to include a user equipment (UE) 2602 and a UE 2604. In at least one embodiment, UEs 2602 and 2604 are illustrated as smartphones (e.g., handheld touchscreen mobile computing devices connectable to one or more cellular networks) but can also include any mobile or non-mobile computing device, such as personal data assistants (PDAs), pagers, laptop computers, desktop computers, wireless handsets, or any computing device including a wireless communications interface.

[0271] In at least one embodiment, any of UEs 2602 and 2604 can comprise an Internet of Things (IoT) UE, which can comprise a network access layer designed for low-power IoT applications utilizing short-lived UE connections. In at least one embodiment, an IoT-UE can utilize technologies such as machine-to-machine (M2M) or machine-type communications (MTC), for exchanging data with an MTC server or device via a public land mobile network (PLMN), a proximity-based service (ProSe) or device-to-device (D2D) communication, sensor networks, or IoT networks. In at least one embodiment, M2M or MTC data exchange can be a machine-initiated data exchange. In at least one embodiment, an IoT network describes interconnecting IoT UEs, which can include uniquely identifiable embedded computing devices (within the Internet infrastructure), with short-lived connections with a backend

[0272] In at least one embodiment, UEs 2602 and 2604 can be configured to connect (e.g., communicatively couple) with a radio access network (RAN) 2616. In at least one embodiment, RAN 2616 can be, for example, an evolved- universal mobile telecommunications system (UMTS) terrestrial radio access network (E-UTRAN), a NextGen RAN (NG RAN), or some other type of RAN. In at least one embodiment, UEs 2602 and 2604 utilize connections 2612 and 2614, respectively, each of which includes a physical

[0273] In at least one embodiment, UEs 2602 and 2604 may also directly exchange communication data via the ProSe interface 2606. In at least one embodiment, the ProSe interface 2606 may alternatively be referred to as a sidelink interface, which includes one or more logical channels, including but not limited to the Physical Sidelink Control Channel (PSCCH), Physical Sidelink Shared Channel (PSSCH), Physical Sidelink Discovery Channel (PSDCH), and Physical Sidelink Broadcast Channel (PSBCH).

[0274] In at least one embodiment, UE 2604 is shown configured to access access point (AP) 2610 via connection 2608. In at least one embodiment, connection 2608 may include a local wireless connection, such as a connection consistent with any IEEE 802.11 protocol, wherein AP 2610 will include Wireless Fidelity. Router. In at least one embodiment, AP 2610 is shown as connected to the Internet but not to the core network of a wireless system.

[0275] In at least one embodiment, RAN 2616 may include one or more access nodes enabling connectivity between 2612 and 2614. In at least one embodiment, these access nodes (ANs) may be referred to as base stations (BS), NodeBs, evolved NodeBs (eNBs), next-generation NodeBs (gNBs), RAN nodes, etc., and may include ground stations (e.g., ground access points) or satellite stations providing coverage within a geographic area (e.g., a cell). In at least one embodiment, RAN 2616 may include one or more RAN nodes (e.g., macro RAN node 2618) for providing macrocell coverage and one or more RAN nodes (e.g., low-power (LP) RAN node 2620) for providing femtocells or picocells (e.g., cells with smaller coverage areas, smaller user capacity, or higher bandwidth compared to macrocells).

[0276] In at least one embodiment, either RAN node 2618 or 2620 may terminate the air interface protocol and may serve as the first contact point for UEs 2602 and 2604. In at least one embodiment, either RAN node 2618 or 2620 may implement various logical functions of RAN 2616, including but not limited to Radio Network Controller (RNC) functions such as radio bearer management, uplink and downlink dynamic radio resource management, and data packet scheduling and mobility management.

[0277] In at least one embodiment, UEs 2602 and 2604 can be configured to communicate using orthogonal frequency division multiplexing (OFDM) communication signals with each other or with any of RAN nodes 2618 and 2620 over a multicarrier communication channel in accordance with various communication techniques, such as, but not limited to, orthogonal frequency division multiple access (OFDMA) communication techniques (e.g., for downlink communications) or single carrier frequency division multiple access (SC-FDMA) communication techniques (e.g., for uplink and ProSe or sidelink communications), and / or variants thereof. In at least one embodiment, OFDM signals can include a plurality of orthogonal subcarriers.

[0278] In at least one embodiment, a downlink resource grid can be used for downlink transmissions from any of RAN nodes 2618 and 2620 to UEs 2602 and 2604, while uplink transmissions can utilize a similar approach. In at least one embodiment, a grid can be a time-frequency grid, called a resource grid or time-frequency resource grid, which is the physical resource in the downlink in each slot. In at least one embodiment, such a time-frequency plane representation is a common practice for OFDM systems, which makes it intuitive for radio resource allocation. In at least one embodiment, each column and each row of the resource grid corresponds to one OFDM symbol and one OFDM subcarrier, respectively. In at least one embodiment, the duration of the resource grid in the time domain corresponds to one slot, which is 0.5 milliseconds in LTE. In at least one embodiment, the smallest time-frequency unit in a resource grid is denoted as a resource element. In at least one embodiment, each resource element covers one subcarrier per one symbol period. In at least one embodiment, a resource grid may

[0279] In at least one embodiment, a physical downlink shared channel (PDSCH) can carry user data and higher layer signaling to UEs 2602 and 2604. In at least one embodiment, a physical downlink control channel (PDCCH) can carry information regarding the resources allocated for the PDSCH and other information. In at least one embodiment, it can also inform UEs 2602 and 2604 about the transport format, resource allocation, and HARQ information related to the uplink shared channel. In at least one embodiment, generally, downlink scheduling (allocation of control and shared channel resource blocks to UEs 2602 within a cell) can be performed at any of RAN nodes 2618 and 2620 based on channel quality information feedback from any of UEs 2602 and 2604. In at least one embodiment, downlink resource allocation information can be sent on the PDCCH used for (e.g., allocated to) each of UEs 2602 and 2604.

[0280] In at least one embodiment, PDCCH can use control channel elements (CCEs) to convey control information. In at least one embodiment, before being mapped to resource elements, PDCCH complex-valued symbols can first be organized into quadruplets, which can then be permuted using a sub-block interleaver for rate matching. In at least one embodiment, one or more CCEs can be used to convey a PDCCH, where each CCE can correspond to nine sets of four resource element groups (REGs), referred to as resource element groups (REGs). In at least one embodiment, four Quadrature Phase Shift Keying (QPSK) symbols can be mapped to each REG. In at least one embodiment, depending on the downlink control information (DCI) size and the channel condition, one or more CCEs can be used to send a PDCCH. In at least one embodiment, there can be four or more different PDCCH formats (e.g., aggregation level, L=1, 2, 4, or 8) defined in LTE with different numbers of CCEs.

[0281] In at least one embodiment, an enhanced physical downlink control channel (EPDCCH) can be used for control information transmission using PDSCH resources. In at least one embodiment, one or more enhanced control channel elements (ECCEs) can be used to convey an EPDCCH. In at least one embodiment, each ECCE can correspond to nine sets of four physical resource elements called enhanced resource element groups (EREGs). In at least one embodiment, an ECCE can have other numbers of EREGs in some situations.

[0282] In at least one embodiment, RAN 2616 is shown to be communicatively coupled to core network (CN) 2638 via S1 interface 2622. In at least one embodiment, CN 2638 can be an evolved packet core (EPC) network, a NextGen Packet Core (NPC) network, or some other type of CN. In at least one embodiment, S1 interface 2622 is split into two parts: the S1 -U interface 2626, which carries traffic data between RAN nodes 2618 and 2620 and serving gateway (S-GW) 2630, and the S1 -Mobility Management Entity (MME) interface 2624, which is a signaling interface between RAN nodes 2618 and 2620 and MME 2628.

[0283] In at least one embodiment, CN 2638 includes MME 2628, S-GW 2630, a packet data network (PDN) gateway (P-GW) 2634, and a home subscriber server (HSS) 2632. In at least one embodiment, MME 2628 can be similar in function to a control plane of legacy Serving General Packet Radio Service (GPRS) Support Nodes (SGSN). In at least one embodiment, MME 2628 can manage mobility aspects in access such as gateway selection and tracking area management. In at least one embodiment, HSS 2632 can comprise a database for network users, including subscription-related information to support the network entities’ handling of communication sessions. In at least one embodiment, CN 2638 can comprise one or more HSSs 2632 depending on the number of mobile subscribers, on the capacity of the equipment, on the organization of the network, etc. In at least one embodiment, HSS 2632 can provide support for routing / roaming, authentication, authorization, naming / addressing resolution, location dependencies, etc.

[0284] In at least one embodiment, S-GW 2630 can terminate the S1 interface 2622 towards RAN 2616, and routes data packets between RAN 2616 and CN 2638. In at least one embodiment, S-GW 2630 can be a local mobility anchor for inter-RAN node handovers and also can provide an anchor for inter-3 GPP mobility. In at least one embodiment, other responsibilities can include lawful intercept, charging, and some policy enforcement and charging.

[0285] In at least one embodiment, P-GW 2634 can terminate a SGi interface toward a PDN. In at least one embodiment, P-GW 2634 can route data packets between a EPC network 2638 and an external network such as the Internet 2642, including a network including an application server 2640, or a network-based game server. In at least one embodiment, application server 2640 can be an element offering applications using IP bearer resources. In at least one embodiment, P-GW 2634 is shown to be communicatively coupled to an application server 2640 via an IP communications interface 2642. In at least one embodiment, application server 2640 can also be configured to support one or more communication services for UEs 2602 and 2604 via CN 2638, such as Voice-over-Internet Protocol (VoIP) sessions, PTT sessions, group communication sessions, social networking services, and the like.

[0286] In at least one embodiment, P-GW 2634 can also be a node for policy enforcement and charging data collection. In at least one embodiment, a Policy and Charging Enforcement Function (PCRF) 2636 is a policy and charging control element of CN 2638. In at least one embodiment, in a non-roaming scenario, there can be a single PCRF 2636 in a Home Public Land Mobile Network (HPLMN) associated with a UE’s Internet Protocol Connectivity Access Network (IP-CAN) session. In at least one embodiment, in a roaming scenario with local breakout of traffic, there can be two PCRFs associated with a UE’s IP-CAN session: a Home PCRF (H-PCRF) within a HPLMN and a Visited PCRF (V-PCRF) within a Visited Public Land Mobile Network (VPLMN). In at least one embodiment, PCRF 2636 can be communicatively coupled to application server 2640 via P-GW 2634. In at least one embodiment, application server 2640 can signal a PCRF 2636 to indicate a new service flow and select appropriate Quality of Service (QoS) and charging

[0287] Figure 27The architecture of a system 2700 of a network is shown in accordance with some embodiments. In at least one embodiment, system 2700 is shown to include a UE 2702, a 5G access node or RAN node (shown as (R)AN node 2708), a user plane function (shown as UPF 2704), a data network (DN 2706), e.g., which can be operator services, Internet access or third party services, and a 5G core network (5GC) (shown as CN 2710).

[0288] In at least one embodiment, CN 2710 includes an authentication server function (AUSF 2714); a core access and mobility management function (AMF 2712); a session management function (SMF 2718); a network exposure function (NEF 2716); a policy control function (PCF 2722); a network function (NF) repository function (NRF 2720); a unified data management (UDM 2724); and an application function (AF 2726). In at least one embodiment, CN 2710 can also include other elements not shown, such as a structured data storage network function (SDSF), an unstructured data storage network function (UDSF), and variations thereof.

[0289] In at least one embodiment, UPF 2704 can act as an anchor point for intra-RAT and inter-RAT mobility, an external PDU session point of interconnect to DN 2706, and a branching point for multi-homed PDU session. In at least one embodiment, UPF 2704 can also perform packet routing and forwarding, packet inspection, enforce per-user packet filtering rules (e.g., deep packet inspection), law enforcement interception of communications (UP collection), traffic usage reporting, perform QoS handling for user planes (e.g., packet filtering, gating, UL / DL traffic flow control), inter-UP mobility handling, transport level packet marking in the uplink and downlink, and downlink packet buffering and downlink data notification triggering. In at least one embodiment, UPF 2704 can include an uplink classifier to support routing uplink traffic to a data network. In at least one embodiment, DN 2706 can represent various network operator services, Internet access, or third party services.

[0290] In at least one embodiment, AUSF 2714 can store data for authentication of UE 2702 and handle authentication-related functionality. In at least one embodiment, AUSF 2714 can facilitate a common authentication framework for various access types.

[0291] In at least one embodiment, AMF 2712 can be responsible for registration management (e.g., for registering UE 2702, etc.), connection management, reachability management, mobility management, and lawful interception of AMF-related events, as well as access authentication and authorization. In at least one embodiment, AMF 2712 can provide SM message transmission for SMF 2718 and act as a transparent proxy for routing SM messages. In at least one embodiment, AMF 2712 can also provide UE 2702 with SMS functionality (SMSF) (…). Figure 27 Transmission of Short Message Service (SMS) messages between (not shown). In at least one embodiment, AMF 2712 may act as a Security Anchoring Function (SEA), which may include interaction with AUSF 2714 and UE 2702 and receiving an intermediate key established as a result of the UE 2702 authentication process. In at least one embodiment, in the case of using USIM-based authentication, AMF 2712 may retrieve security material from AUSF 2714. In at least one embodiment, AMF 2712 may also include a Security Context Management (SCM) function, which receives from the SEA a key it uses to derive the access network-specific key. Furthermore, in at least one embodiment, AMF 2712 may be the termination point (N2 reference point) of the RAN CP interface, the termination point of NAS (NI) signaling, and perform NAS encryption and integrity protection.

[0292] In at least one embodiment, AMF 2712 can also support NAS signaling with UE 2702 via the N3 Interworking Function (IWF) interface. In at least one embodiment, the N3IWF can be used to provide access to untrusted entities. In at least one embodiment, the N3IWF can be the termination point of the N2 and N3 interfaces of the control plane and user plane, respectively, thus enabling the processing of N2 signaling from SMF and AMF for PDU sessions and QoS, encapsulation / decapsulation of IPSec and N3 tunnel packets, marking N3 user plane packets in the uplink, and implementing QoS corresponding to the N3 packet marking, taking into account the QoS requirements associated with such marking received via N2. In at least one embodiment, the N3IWF can also relay uplink and downlink control plane NAS (NI) signaling between UE 2702 and AMF 2712, and relay uplink and downlink user plane packets between UE 2702 and UPF 2704. In at least one embodiment, the N3IWF also provides a mechanism for establishing an IPsec tunnel with the UE 2702.

[0293] In at least one embodiment, the SMF 2718 can be responsible for session management (e.g., session establishment, modify and release, including tunnel maintain between UPF and AN node); UE IP address allocation and management (including optional authorization); selection and control of UP function; configuration of traffic steering at UPF to route traffic to appropriate destination; interface termination towards policy control function; control part of policy enforcement and QoS; lawful intercept (for SM events and interface to LI system); termination of SM part of NAS messages; downlink data notification; initiator of AN specific SM information, sent over N2 to AN via AMF; determining SSC mode of a session. In at least one embodiment, the SMF 2718 can include the following roaming functionality: handling local enforcement to apply QoS SLAs (VPLMN); charging data collection and charging interface (VPLMN); lawful intercept (in VPLMN for SM events and interface to LI system); support interaction with external DNs to transfer signaling for PDU session authorization / authentication by external DN.

[0294] In at least one embodiment, the NEF 2716 can provide means for securely exposing services and capabilities offered by 3 GPP network functions to third parties, internal exposure / re- exposure, application functions (e.g., AF 2726), edge computing or fog computing systems, or the like. In at least one embodiment, the NEF 2716 can authenticate, authorize, and / or throttle AFs. In at least one embodiment, the NEF 2716 can also translate information exchanged with AFs 2726 and information exchanged with internal network functions. In at least one embodiment, the NEF 2716 can translate between AF service identifiers and internal 5GC information. In at least one embodiment, the NEF 2716 can also receive information from other network functions (NFs) based on their exposed capabilities. In at least one embodiment, this information can be stored at the NEF 2716 as structured data, or at a data storage NF using standardized interfaces. In at least one embodiment, the stored information can then be re-exposed by the NEF 2716 to other NFs and AFs, and / or used for other purposes, such as analytics.

[0295] In at least one embodiment, the NRF 2720 can support service discovery functions, receive NF discovery requests from NF instances, and provide NF instances with information of discovered NF instances. In at least one embodiment, the NRF 2720 also maintains information of available NF instances and their supported services.

[0296] In at least one embodiment, the PCF 2722 can provide policy rules to control plane functions to enforce them, and can also support a unified policy framework to govern network behavior. In at least one embodiment, the PCF 2722 can also implement a front end (FE) to access subscription information relevant for policy decisions in a UDM 2724 UDR.

[0297] In at least one embodiment, the UDM 2724 can handle subscription-related information to support network entities in handling communication sessions, and can store subscription data for UEs 2702. In at least one embodiment, the UDM 2724 can include two parts, an Application FE and a User Data Repository (UDR). In at least one embodiment, the UDM can include a UDM FE that is responsible for processing credentials, location management, subscription management, and the like. In at least one embodiment, several different front ends can service the same user in different transactions. In at least one embodiment, the UDM-FE accesses subscription information stored in the UDR and performs authentication credential processing; user identification processing; access authorization; registration / mobility management; and subscription management. In at least one embodiment, the UDR can interact with the PCF 2722. In at least one embodiment, the UDM 2724 can also support SMS management, where an SMS-FE implements similar application logic as previously discussed.

[0298] In at least one embodiment, the AF 2726 can provide application influence over traffic routing, access to network exposure capabilities (NCE), and interaction with policy framework for policy control. In at least one embodiment, NCE can be a mechanism that allows 5GC and AFs 2726 to provide information to each other via the NEF 2716, which can be used for edge computing implementations. In at least one embodiment, network operators and third party services can be hosted close to the UE 2702’s point of attachment to the access network to achieve efficient service delivery with reduced end-to-end latency and load on transport networks. In at least one embodiment, for edge computing implementations, the 5GC can choose a UPF 2704 close to the UE 2702, and perform traffic steering from the UPF 2704 to DN 2706 via an N6 interface. In at least one embodiment, this can be based on UE subscription data, UE location, and information provided by AFs 2726. In at least one embodiment, AFs 2726 can influence UPF (re)selection and traffic routing. In at least one embodiment, based on operator deployment, when AFs 2726 are considered trusted entities, the network operator can allow AFs 2726 to directly interact with relevant NFs.

[0299] In at least one embodiment, CN 2710 can include an SMSF, which can be responsible for SMS subscription checking and verification, and relaying SM messages to / from UE 2702 to / from other entities, such as an SMS-GMSC / IWMSC / SMS-router. In at least one embodiment, the SMS can also interact with AMF 2712 and UDM 2724 for a notification procedure that the UE 2702 is available for SMS transfer (e.g., setting a UE not reachable flag, and notifying UDM 2724 when UE 2702 is available for SMS).

[0300] In at least one embodiment, system 2700 can include the following service-based interfaces exposed by NFs: Namf: service-based interface exposed by AMF; Nsmf: service-based interface exposed by SMF; Nnef: service-based interface exposed by NEF; Npcf: service-based interface exposed by PCF; Nudm: service-based interface exposed by UDM; Naf: service-based interface exposed by AF; Nnrf: service-based interface exposed by NRF; and Nausf: service-based interface exposed by AUSF.

[0301] In at least one embodiment, system 2700 can include the following reference points: N1 : reference point between UE and AMF; N2: reference point between (R)AN and AMF; N3: reference point between (R)AN and UPF; N4: reference point between SMF and UPF; and N6: reference point between UPF and data network. In at least one embodiment, there can be more reference points and / or service-based interfaces between NFs in NFs, however, these interfaces and reference points have been omitted for clarity. In at least one embodiment, a NS reference point can be between a PCF and an AF; a N7 reference point can be between a PCF and a SMF; a N11 reference point between an AMF and a SMF; and / or the like. In at least one embodiment, CN 2710 can include an Nx interface, which is an inter-CN interface between a MME and AMF 2712 in order to enable interworking between CN 2710 and CN 7227.

[0302] In at least one embodiment, system 2700 can include multiple RAN nodes, such as (R)AN nodes 2708, wherein an Xn interface is defined between two or more (R)AN nodes 2708 (e.g., gNBs) connected to 5GC 410, between a (R)AN node 2708 (e.g., gNB) and an eNB (e.g., macro RAN node) connected to CN 2710, and / or between two eNBs connected to CN 2710.

[0303] In at least one embodiment, Xn interface can include an Xn user plane (Xn-U) interface and an Xn control plane (Xn-C) interface. In at least one embodiment, Xn-U can provide unguaranteed delivery of user plane PDUs and support / provide data forwarding and flow control functionality. In at least one embodiment, Xn-C can provide management and error handling functionality, functionality to manage Xn-C interfaces; mobility support for UEs 2702 in a connected mode (e.g., CM-CONNECTED) including functionality to manage connected mode UE mobility between one or more (R)AN nodes 2708. In at least one embodiment, mobility support can include context transfer from an old (source) serving (R)AN node 2708 to new (target) serving (R)AN node 2708; and control user plane tunneling between old (source) serving (R)AN node 2708 to new (target) serving (R)AN node 2708.

[0304] In at least one embodiment, a protocol stack of Xn-U can include a transport network layer built on Internet Protocol (IP) transport layer and a GTP-U layer at the top of UDP and / or one or more IP layers for carrying user plane PDUs. In at least one embodiment, a protocol stack of Xn-C can include an application layer signaling protocol (referred to as Xn Application Protocol (Xn-AP)) and a transport network layer built on SCTP layer. In at least one embodiment, the SCTP layer can be on top of IP layer. In at least one embodiment, the SCTP layer provides a guaranteed delivery of application layer messages. In at least one embodiment, in transport IP layer, point-to-point transmission is used to deliver signaling PDUs. In at least one embodiment, Xn-U protocol stack and / or Xn-C protocol stack can be the same as or similar to user plane and / or control plane protocol stacks shown and described herein.

[0305] Figure 28 is an illustration of a control plane protocol stack in accordance with some embodiments. In at least one embodiment, control plane 2800 is shown as a communication protocol stack between UE 2602 (or alternatively, UE 2604), RAN 2616, and MME 2628.

[0306] In at least one embodiment, PHY layer 2802 can transmit or receive information used by MAC layer 2804 over one or more air interfaces. In at least one embodiment, PHY layer 2802 can also perform link adaptation or adaptive modulation and coding (AMC), power control, cell search (e.g., for initial synchronization and handover purposes), and other measurements used by higher layers, such as RRC layer 2810. In at least one embodiment, PHY layer 2802 can further perform error detection on the transport channels, forward error correction (FEC) coding / decoding of the transport channels, modulation / demodulation of physical channels, interleaving, rate matching, mapping to / from physical channels, and Multiple Input Multiple Output (MIMO) antenna processing.

[0307] In at least one embodiment, MAC layer 2804 can perform mapping between logical channels and transport channels, multiplexing of MAC service data units (SDUs) from one or more logical channels into transport blocks (TB) to be delivered to PHY on transport channels, demultiplexing of MAC SDUs to one or more logical channels from TBs delivered from PHY on transport channels, multiplexing of MAC SDUs onto TBs, scheduling information reporting, error correction through hybrid automatic repeat request (HARQ), and logical channel prioritization.

[0308] In at least one embodiment, RLC layer 2806 can operate in multiple modes of operation, including: transparent mode (TM), unacknowledged mode (UM), and acknowledged mode (AM). In at least one embodiment, RLC layer 2806 can perform transfer of upper layer protocol data units (PDUs), error correction through automatic repeat request (ARQ) for AM data transfers, and concatenation, segmentation, and reassembly of RLC SDUs for UM and AM data transfers. In at least one embodiment, RLC layer 2806 can also perform re-segmentation of RLC data PDUs for AM data transfers, reordering of RLC data PDUs for UM and AM data transfers, duplicate detection for UM and AM data transfers, discard of RLC SDUs for UM and AM data transfers, detection of protocol errors for AM data transfers, and perform RLC re-establishment.

[0309] In at least one embodiment, PDCP layer 2808 can perform header compression and decompression of IP data, maintain PDCP sequence numbers (SNs), perform in-sequence delivery of upper layer PDUs at re-establishment of lower layers, eliminate duplication of lower layer SDUs at re-establishment of lower layers for radio bearers mapped on RLC AM, cipher and decipher control plane data, integrity protect and integrity verify control plane data, perform data discard based on control timers, and perform security operations (e.g., ciphering, deciphering, integrity protection, integrity verification, etc.).

[0310] In at least one embodiment, the main services and functions of RRC layer 2810 can include broadcast of system information (e.g., included in master information block (MIB) or system information blocks (SIBs) related to non-access stratum (NAS)), broadcast of system information related to the access stratum (AS), paging, establishment, maintenance and release of an RRC connection between the UE and E-UTRAN (e.g., RRC connection paging, RRC connection establishment, RRC connection modification, and RRC connection release), establishment, configuration, maintenance and release of point-to-point Radio Bearers, security functions including key management, inter-RAT mobility, and measurement configuration for a UE to measure.

[0311] In at least one embodiment, UE 2602 and RAN 2616 can utilize a Uu interface (e.g., LTE-Uu interface) to exchange control plane data via a protocol stack including PHY layer 2802, MAC layer 2804, RLC layer 2806, PDCP layer 2808, and RRC layer 2810.

[0312] In at least one embodiment, a non-access stratum (NAS) protocol (NAS protocols 2812) forms the highest stratum of the control plane between UE 2602 and MME 2628. In at least one embodiment, NAS protocols 2812 support the mobility of UE 2602 and session management procedures to establish and maintain IP connectivity between UE 2602 and P-GW 2634.

[0313] In at least one embodiment, an Si application protocol (S1-AP) layer (S1-AP layer 2822) can support functions of the S1 interface and include elementary procedures (EPs). In at least one embodiment, an EP is a unit of interaction between RAN 2616 and CN 2628. In at least one embodiment, S1-AP layer services can include two groups: UE-associated services and non-UE-associated services. In at least one embodiment, these services perform functions including, but not limited to: E-UTRAN Radio Access Bearer (E-RAB) management, UE capability indication, mobility, NAS signaling transfer, RAN Information Management (RIM), and configuration transfer.

[0314] In at least one embodiment, a stream control transmission protocol (SCTP) layer (alternatively referred to as a stream control transmission protocol / internet protocol (SCTP / IP) layer) 2820 can ensure reliable delivery of signaling messages between RAN 2616 and MME 2628 based in part on IP protocols supported by IP layer 2818. In at least one embodiment, L2 layer 2816 and L1 layer 2814 can refer to communication links (e.g., wired or wireless) used by RAN nodes and MMEs to exchange information.

[0315] In at least one embodiment, RAN 2616 and one or more MMEs 2628 can utilize an S1-MME interface to exchange control plane data via a protocol stack including L1 layer 2814, L2 layer 2816, IP layer 2818, SCTP layer 2820, and Si-AP layer 2822.

[0316] Figure 29 is a diagram of a user plane protocol stack in accordance with at least one embodiment. In at least one embodiment, user plane 2900 is shown as a communication protocol stack between UE 2602, RAN 2616, S-GW 2630, and P-GW 2634. In at least one embodiment, user plane 2900 can utilize the same protocol layers as control plane 2800. In at least one embodiment, for example, UE 2602 and RAN 2616 can utilize a Uu interface (e.g., an LTE-Uu interface) to exchange user plane data via a protocol stack including PHY layer 2802, MAC layer 2804, RLC layer 2806, PDCP layer 2808.

[0317] In at least one embodiment, a general packet radio service (GPRS) tunneling protocol (GTP-U) layer (GTP-U layer 2904) for user plane can be used to carry user data within a GPRS core network and between a radio access network and a core network. In at least one embodiment, for example, user data transported can be packets in any of IPv4, IPv6, or PPP formats. In at least one embodiment, a UDP / IP layer (UDP / IP layer 2902) can provide checksums for data integrity, port numbers for addressing different functions at the source and destination, and encryption and authentication on selected data flows. In at least one embodiment, RAN 2616 and S-GW 2630 can utilize an S1-U interface to exchange user plane data via a protocol stack comprising L1 layer 2814, L2 layer 2816, UDP / IP layer 2902, and GTP-U layer 2904. In at least one embodiment, S-GW 2630 and P-GW 2634 can utilize a S5 / S8a interface to exchange user plane data via a protocol stack comprising L1 layer 2814, L2 layer 2816, UDP / IP layer 2902, and GTP-U layer 2904. In at least one embodiment, as discussed above with respect to FIG. 29, a NAS protocol supports mobility and session management procedures for UE 2602 to establish and maintain IP connectivity between UE 2602 and P-GW 2634. Figure 30

[0318] Figure 31 Components of a core network are illustrated according to at least one embodiment 3000. In at least one embodiment, components of CN 2638 can be implemented in one physical node or in separate physical nodes including components to read and execute instructions from a machine-readable or computer-readable medium (e.g., a non-transitory machine-readable storage medium). In at least one embodiment, network function virtualization (NFV) is used to virtualize any or all of the above-described network node functions via executable instructions stored in one or more computer-readable storage mediums (described in further detail below). In at least one embodiment, a logical instantiation of CN 2638 can be referred to as a network slice 3002 (e.g., network slice 3002 is illustrated as including HSS 2632, MME 2628, and S-GW 2630). In at least one embodiment, a logical instantiation of a portion of CN 2638 can be referred to as a network sub-slice 3004 (e.g., network sub-slice 3004 is illustrated as including P-GW 2634 and PCRF 2636).

[0319] ​In at least one embodiment, NFV architecture and infrastructure can be used to virtualize one or more network functions onto a common physical platform(s) comprising industry standard server hardware, storage hardware, or switches in combination with virtualization technologies. In at least one embodiment, NFV can be used to consolidate many network equipment types onto industry standard high volume server hardware, storage hardware, and switches.

[0320] Figure 26 is a block diagram illustrating components of a system 3100 to support network function virtualization (NFV), according to at least one embodiment. In at least one embodiment, system 3100 is shown to include a virtualization infrastructure manager (shown as VIM 3102), a network function virtualization infrastructure (shown as NFVI 3104), a VNF manager (shown as VNFM 3106), a virtualized network function (shown as VNF 3108), an element manager (shown as EM 3110), an NFV orchestrator (shown as NFVO 3112), and a network manager (shown as NM 3114).

[0321] In at least one embodiment, VIM 3102 manages the resources of NFVI 3104. In at least one embodiment, NFVI 3104 can include physical or virtual resources and applications (including a hypervisor) used to execute system 3100. In at least one embodiment, VIM 3102 can utilize NFVI 3104 to manage the life cycle of virtual resources (e.g., creation, maintenance, and teardown of virtual machines (VMs) associated with one or more physical resources), track VM instances, track performance, faults, and security of VM instances and associated physical resources, and expose VM instances and associated physical resources to other management systems.

[0322] In at least one embodiment, VNFM 3106 can manage VNF 3108. In at least one embodiment, VNF 3108 can be used to execute EPC components / functions. In at least one embodiment, VNFM 3106 can manage life cycle of VNF 3108 and track performance, faults, and security of virtual aspects of VNF 3108. In at least one embodiment, EM 3110 can track performance, faults, and security of functional aspects of VNF 3108. In at least one embodiment, tracking data from VNFM 3106 and EM 3110 can include, for example, performance measurement (PM) data used by VIM 3102 or NFVI 3104. In at least one embodiment, both VNFM 3106 and EM 3110 can scale up / down the number of VNFs of system 3100.

[0323] In at least one embodiment, NFVO 3112 can coordinate, authorize, release, and occupy resources of NFVI 3104 in order to provide the requested service (e.g., to execute an EPC function, component, or slice). In at least one embodiment, NM 3114 can provide a package of end-user functions that are responsible for managing a network that can include network elements that are VNFs, non-virtualized network functions, or both (management of VNFs can occur via EM 3110).

[0324] In at least one embodiment, network 106 can be implemented using system 2600 (see Figure 27 ), system 2700 (see Figure 31 ), and / or system 3100 (see Figure 26 ). In such embodiments, transmitting device 102 and / or receiving device 104 can each be implemented as UE 2602 or UE 2604.

[0325] In at least one embodiment, packet data network 522 can be implemented using system 2600 (see Figure 27 ), system 2700 (see Figure 31 ), and / or system 3100 (see Figure 26 ). In such embodiments, Tx host computers 524, 526, and 528 and / or one or more of Rx host computers 540 can each be implemented as UE 2602 or UE 2604.

[0326] In at least one embodiment, network 816 can be implemented using system 2600 (see Figure 27 ), system 2700 (see Figure 31 ), and / or system 3100 (see Figure 32 ). In such embodiments, O-DU 812 and / or O-RU 814 can each be implemented as UE 2602 or UE 2604.

[0327] Computer-Based System

[0328] The following figures set forth, without limitation, exemplary computer-based systems that can be used to implement at least one embodiment.

[0329] Figure 33A processing system 3200 according to at least one embodiment is shown. In at least one embodiment, system 3200 includes one or more processor(s) 3202 and one or more graphics processing unit(s) 3208, and can be a single processor desktop system, a multiprocessor workstation system, or a server system having many processor cores 3207 or processor cores 3207. In at least one embodiment, processing system 3200 is a processing platform incorporated within a system on a chip (SoC) integrated circuit for use in mobile, handheld, or embedded devices.

[0330] In at least one embodiment, processing system 3200 can include, or be incorporated within a server-based gaming platform, a game console, a mobile gaming console, a handheld game console, or an online game console including game and media consoles. In at least one embodiment, processing system 3200 is a mobile phone, a smart phone, a tablet device, or a mobile internet device. In at least one embodiment, processing system 3200 can also include a wearable device, coupled to or integrated within, such as a smart watch wearable device, a smart eyewear device, an augmented reality device, or a virtual reality device. In at least one embodiment, processing system 3200 is a television or set-top box device having one or more processor(s) 3202, as well as a graphical interface generated by one or more graphics processing unit(s) 3208.

[0331] In at least one embodiment, one or more processor(s) 3202 each include one or more processor cores 3207 to process instructions which, when executed, implement the operations for system and user software. In at least one embodiment, each of the one or more processor cores 3207 is configured to process a specific instruction set 3209. In at least one embodiment, instruction set 3209 can facilitate Complex Instruction Set Computing (CISC), Reduced Instruction Set Computing (RISC), or computing via a Very Long Instruction Word (VLIW). In at least one embodiment, multiple processor cores 3207 can each process a different instruction set 3209, which can include instructions to facilitate emulation of other instruction sets. In at least one embodiment, processor core 3207 can also include other processing devices, such a Digital Signal Processor (DSP).

[0332] In at least one embodiment, processor 3202 includes cache memory 3204. In at least one embodiment, processor 3202 can have a single internal cache or multiple levels of internal caches. In at least one embodiment, cache memory is shared among multiple components of processor 3202. In at least one embodiment, processor 3202 also uses an external cache (e.g., a three-level (L3) cache, or last level cache (LLC)) (not shown), which can be shared between processor cores 3207 using known cache coherency techniques. In at least one embodiment, additionally included in processor 3202 are register file 3206, which processor 3202 can include different types of registers to store different types of data (e.g., integer registers, floating point registers, status registers, and instruction pointer registers). In at least one embodiment, register file 3206 can include general registers or other registers.

[0333] In at least one embodiment, one or more processor(s) 3202 are coupled with one or more interface bus(es) 3210 for communicating data between processor 3202 and other components in system 3200, such as address, data, or control signals. In at least one embodiment, interface bus 3210 can be a version of a processor bus, such as a direct media interface (DMI) bus, in at least one embodiment. In at least one embodiment, interface bus 3210 is not limited to DMI bus, and can include one or more peripheral component interconnect buses (e.g., a PCI, PCI Express), memory buses, or other types of interface buses. In at least one embodiment, processor 3202 includes integrated memory controller 3216 and platform controller hub 3230. In at least one embodiment, memory controller 3216 facilitates communication between memory and other components of processing system 3200, while platform controller hub 3230 provides connections to input / output (I / O) devices to local I / O bus.

[0334] In at least one embodiment, memory device 3220 can be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, flash memory device, or a phase change memory device, among others. In at least one embodiment, memory device 3220 can be a system memory of processing system 3200, to store data 3222 and instructions 3221 for use when one or more processors 3202 executes an application or process. In at least one embodiment, memory controller 3216 also couples with an optional external graphics processor 3212, which can communicate with one or more graphics processors 3208 in processors 3202 to perform graphics and media operations.

[0335] In at least one embodiment, platform controller hub 3230 enables peripherals to connect to storage devices 3220 and processor 3202 via a high-speed I / O bus. In at least one embodiment, I / O peripherals include, but are not limited to, an audio controller 3246, a network controller 3234, a firmware interface 3228, a wireless transceiver 3226, a touch sensor 3225, a data storage device 3224 (e.g., hard disk drive, flash memory, etc.). In at least one embodiment, data storage device 3224 can connect to the storage bus via a storage interface (e.g., SATA) or via a peripheral bus, such as a peripheral component interconnect bus (e.g., PCI, PCIe). In at least one embodiment, touch sensor 3225 can include a touch screen sensor, a pressure sensor, or a fingerprint sensor. In at least one embodiment, wireless transceiver 3226 can be a Wi-Fi transceiver, a Bluetooth transceiver, or a mobile network transceiver such as a 3G, 4G, or Long Term Evolution (LTE) transceiver. In at least one embodiment, firmware interface 3228 enables communication with system firmware, for example, and can be a unified extensible firmware interface (UEFI). In at least one embodiment, network controller 3234 can enable network connectivity to one or more private or public networks. In at least one embodiment, a high-performance network controller (not shown) is coupled to interface bus 3210. In at least one embodiment, audio controller 3246 is a multi-channel high definition audio controller. In at least one embodiment, processing system 3200 includes an optional legacy I / O controller 3240 for coupling legacy (e.g., Personal System 2 (PS / 2)) devices to processing system 3200. In at least one embodiment, platform controller hub 3230 can also connect to one or more Universal Serial Bus (USB) controllers 3242 that connect into input devices such as keyboard and mouse 3243 combinations, camera 3244, or other USB input devices.

[0336] In at least one embodiment, memory controller 3216 and instances of platform controller hub 3230 can be integrated into a discrete external graphics processor, such as external graphics processor 3212. In at least one embodiment, platform controller hub 3230 and / or memory controller 3216 can be external to one or more processor(s) 3202. For example, in at least one embodiment, processing system 3200 can include an external memory controller 3216 and platform controller hub 3230, which can be configured as a memory controller hub and a peripheral controller hub in a system chipset that is in communication with processor(s) 3202.

[0337] In at least one embodiment, transmitting device 102 and / or receiving device 104 can each be implemented using processing system 3200. In such embodiments, transmitting processor 114 and / or receiving processor 126 can each be implemented as processor 3202. Each of transmitting memory 116 and / or receiving memory 128 can be implemented using any memory of processing system 3200, such as memory device 3220 and / or data storage device 3224. Each of transmitting bus 228 and / or receiving bus 418 can be implemented using any bus of processing system 3200, such as interface bus 3210. Transmitting communication apparatus 112 and / or receiving communication apparatus 124 can each be implemented as network controller 3234.

[0338] In at least one embodiment, Tx host computers 524, 526, and 528 and / or one or more of Rx host computers 540 can each be implemented using processing system 3200. In such embodiments, transmitting processor 114 and / or receiving processor 126 can each be implemented as processor 3202. Each of transmitting memory 116 and / or receiving memory 128 can be implemented using any memory of processing system 3200, such as memory device 3220 and / or data storage device 3224. Each of transmitting bus 228 and / or receiving bus 418 can be implemented using any bus of processing system 3200, such as interface bus 3210. Transmitting communication apparatus 112 and / or receiving communication apparatus 124 can each be implemented as network controller 3234.

[0339] In at least one embodiment, each of O-DU 812 and / or O-RU 814 can be implemented using processing system 3200. In such embodiments, transmitting processor 114 and / or receiving processor 126 can each be implemented as processor 3202. Each of transmitting memory 116 and / or receiving memory 128 can be implemented using any memory of processing system 3200, such as memory device 3220 and / or data storage device 3224. Each of transmitting bus 228 and / or receiving bus 418 can be implemented using any bus of processing system 3200, such as interface bus 3210. Transmitting communication apparatus 112 and / or receiving communication apparatus 124 can each be implemented as network controller 3234.

[0340] Figure 33A computer system 3300 according to at least one embodiment is illustrated. In at least one embodiment, the computer system 3300 may be a system having interconnected devices and components, a System-on-a-Chip (SoC), or some combination thereof. In at least one embodiment, the computer system 3300 is formed by a processor 3302, which may include execution units for executing instructions. In at least one embodiment, the computer system 3300 may include, but is not limited to, components such as the processor 3302, which employs execution units including logic to execute algorithms for process data. In at least one embodiment, the computer system 3300 may include a processor, such as one available from Intel Corporation of Santa Clara, California. Processor family, Xeon™ XScale™ and / or StrongARM™ Core TM or Nervana TM A microprocessor may be used, although other systems (including PCs, engineering workstations, set-top boxes, etc.) with other microprocessors may also be used. In at least one embodiment, computer system 3300 may execute a version of the Windows operating system available from Microsoft Corporation of Redmond, Washington, although other operating systems (such as UNIX and Linux), embedded software, and / or graphical user interfaces may also be used.

[0341] In at least one embodiment, the computer system 3300 can be used in other devices, such as handheld devices and embedded applications. Some examples of handheld devices include cellular phones, Internet Protocol (IP) devices, digital cameras, personal digital assistants (“PDAs”), and handheld PCs. In at least one embodiment, the embedded application may include a microcontroller, a digital signal processor (“DSP”), a system-on-a-chip (SoC), a network computer (“NetPC”), a set-top box, a network hub, a wide area network (“WAN”) switch, or any other system capable of executing one or more instructions according to at least one embodiment.

[0342] In at least one embodiment, the computer system 3300 may include, but is not limited to, a processor 3302, which may include, but is not limited to, one or more execution units 3308 configured to execute a Computational Unified Device Architecture (“CUDA”). a CUDA program. In at least one embodiment, a CUDA program is at least a portion of a software application written in the CUDA programming language. In at least one embodiment, computer system 3300 is a single processor desktop or server system. In at least one embodiment, computer system 3300 can be a multiprocessor system. In at least one embodiment, processor 3302 can include, without limitation, a CISC microprocessor, a RISC microprocessor, a VLIW microprocessor, a processor implementing a combination of instruction sets, or any other processor device, such as a digital signal processor, for example. In at least one embodiment, processor 3302 can be coupled to a processor bus 3310 that can transmit data signals between processor 3302 and other components in computer system 3300.

[0343] In at least one embodiment, processor 3302 can include, without limitation, a level 1 (“L1”) internal cache memory (“cache”) 3304. In at least one embodiment, processor 3302 can have a single internal cache or multiple levels of internal caches. In at least one embodiment, cache memory can reside in the processor 3302’s external. In at least one embodiment, processor 3302 can include a combination of internal and external caches. In at least one embodiment, register file 3306 can store different types of data within various registers including, without limitation, integer registers, floating point registers, status registers, and instruction pointer registers.

[0344] In at least one embodiment, execution unit 3308, including, without limitation, logic to perform integer and floating point operations, also resides in processor 3302. Processor 3302 can also include microcode (“ucode”) read-only memory (“ROM”) that stores microcode for certain macro instructions. In at least one embodiment, execution unit 3308 can include logic to handle a packed instruction set 3309. In at least one embodiment, by including the packed instruction set 3309 in the instruction set of a general-purpose processor 3302, along with associated circuitry to execute the instructions, the general-purpose processor 3302 can be used to perform the operations on packed data that many multimedia applications use. In at least one embodiment, by using the full width of the processor’s data bus when performing operations on packed data, many multimedia applications can be accelerated as compared to using load / store type architectures, which can require multiple Tens of load and store operations per application.

[0345] In at least one embodiment, execution unit 3308 can also be used in microcontrollers, embedded processors, graphics devices, DSPs, and other types of logic circuits. In at least one embodiment, computer system 3300 can include, but not limited to, memory 3320. In at least one embodiment, memory 3320 can be implemented as a DRAM device, SRAM device, flash memory device, or other memory device. Memory 3320 can store data and / or instructions, which can be executed by processor 3302, as represented by data signals.

[0346] In at least one embodiment, system logic chip can be coupled to processor bus 3310 and memory 3320. In at least one embodiment, system logic chip can include, without limitation, a memory controller hub (“MCH”) 3316, and processor 3302 can communicate with MCH 3316 via processor bus 3310. In at least one embodiment, MCH 3316 can provide a high bandwidth memory path 3318 to memory 3320 for instruction and data storage and for storage of graphics commands, data, and textures. In at least one embodiment, MCH 3316 can direct data signals between processor 3302, memory 3320, and other components in computer system 3300, and can

[0347] In at least one embodiment, computer system 3300 can use system I / O 3322 as a proprietary hub interface bus to couple MCH 3316 to I / O controller hub (“ICH”) 3330. In at least one embodiment, ICH 3330 can provide direct connection to some I / O devices and can be connected to other devices via a local I / O bus. In at least one embodiment, local I / O bus can include, without limitation, a high-speed I / O bus for connecting peripheral devices to memory 3320, a chipset, and processor 3302. Examples can include, without limitation, an audio controller 3329, a firmware hub (“Flash BIOS”) 3328, a wireless transceiver 3326, a data storage 3324, a legacy I / O controller 3323 containing user input 3325 and keyboard interface, a serial expansion port 3377 (e.g., USB), and a network controller 3334. Data storage 3324 can include a hard disk drive, a floppy disk drive, a CD-ROM device, a flash memory device, or other mass storage device.

[0348] In at least one embodiment, Figure 33 A system including interconnected hardware devices or “chips” is shown. In at least one embodiment, Figure 33 An exemplary SoC can be shown. In at least one embodiment, Figure 34 Devices shown in FIG. 33 can be interconnected with proprietary interconnects, standardized interconnects (e.g., PCIe), or some combination thereof. In at least one embodiment, one or more components of system 3300 are interconnected using a compute express link (CXL) interconnect.

[0349] In at least one embodiment, sender device 102 and / or receiver device 104 can each be implemented using computer system 3300. In such embodiments, sender processor 114 and / or receiver processor 126 can each be implemented as processor 3302. Sender memory 116 and / or receiver memory 128 can each be implemented using any of the memory of computer system 3300, such as memory 3320 and / or data storage 3324. Sender bus 228 and / or receiver bus 418 can each be implemented using any bus of computer system 3300, such as processor bus 3310. Sender communication 112 and / or receiver communication 124 can each be implemented as network controller 3334.

[0350] In at least one embodiment, one or more of Tx host computers 524, 526, and 528 and / or Rx host computer 540 can each be implemented using computer system 3300. In such embodiments, sender processor 114 and / or receiver processor 126 can each be implemented as processor 3302. Sender memory 116 and / or receiver memory 128 can each be implemented using any of memory of computer system 3300, such as memory 3320 and / or data storage 3324. Sender bus 228 and / or receiver bus 418 can each be implemented using any bus of computer system 3300, such as processor bus 3310. Sender communication device 112 and / or receiver communication device 124 can each be implemented as network controller 3334.

[0351] In at least one embodiment, O-DU 812 and / or O-RU 814 can each be implemented using computer system 3300. In such embodiments, sender processor 114 and / or receiver processor 126 can each be implemented as processor 3302. Sender memory 116 and / or receiver memory 128 can each be implemented using any of memory of computer system 3300, such as memory 3320 and / or data storage 3324. Sender bus 228 and / or receiver bus 418 can each be implemented using any bus of computer system 3300, such as processor bus 3310. Sender communication device 112 and / or receiver communication device 124 can each be implemented as network controller 3334.

[0352] Figure 34 System 3400 is shown in accordance with at least one embodiment. In at least one embodiment, system 3400 is an electronic device that utilizes processor 3410. In at least one embodiment, system 3400 can be, for example and without limitation, a laptop, a tower server, a rack server, a blade server, a desktop computer, a tablet, a mobile device, a phone, an embedded computer, or any other suitable electronic device.

[0353] In at least one embodiment, system 3400 can include, without limitation, processor 3410 communicatively coupled to any suitable number or kind of components, peripherals, modules, or devices. In at least one embodiment, processor 3410 is coupled using a bus or interface, such as I 2a C bus, a system management bus (“SMBus”), a low pin count (LPC) bus, a serial peripheral interface (“SPI”), a high-definition audio (“HDA”) bus, a serial advanced technology attachment (“SATA”) bus, a USB (versions 1, 2, 3), or a universal asynchronous receiver / transmitter (“UART”) bus. In at least one embodiment, Figure 34 A system is shown that includes interconnected hardware devices or “chips.” In at least one embodiment, Figure 34 An exemplary SoC can be shown. In at least one embodiment, Figure 34 Devices shown in FIG. 6 can be interconnected with proprietary interconnects, standardized interconnects (e.g., PCIe), or some combination thereof. In at least one embodiment, Figure 34 One or more components of FIG. 6 are interconnected using compute express link (CXL) interconnects.

[0354] In at least one embodiment, Figure 35 may include a display 3424, a touchscreen 3425, a touchpad 3430, a near field communication unit (“NFC”) 3445, a sensor hub 3440, a thermal sensor 3446, an express chipset (“EC”) 3435, a trusted platform module (“TPM”) 3438, a BIOS / firmware / flash memory (“BIOS, FW Flash”) 3422, a DSP 3460, a solid state disk (“SSD”) or hard drive (“HDD”) 3420, a wireless local area network unit (“WLAN”) 3450, a Bluetooth unit 3452, a wireless wide area network unit (“WWAN”) 3456, a global positioning system (GPS) 3455, a camera (“USB 3.0 camera”) 3454 (e.g., USB 3.0 camera), or a low power double data rate (“LPDDR”) memory unit (“LPDDR3”) 3415 implemented in, for example, LPDDR3 standard. These components can each be implemented in any suitable manner.

[0355] In at least one embodiment, other components can be communicatively coupled to processor 3410 by components discussed above. In at least one embodiment, an accelerometer 3441, an ambient light sensor (“ALS”) 3442, a compass 3443, and a gyroscope 3444 can be communicatively coupled to sensor hub 3440. In at least one embodiment, a thermal sensor 3439, a fan 3437, a keyboard 3446, and a touchpad 3430 can be communicatively coupled to EC 3435. In at least one embodiment, a speaker 3463, a headphone 3464, and a microphone (“mic”) 3465 can be communicatively coupled to an audio unit (“audio codec and class D amplifier”) 3464, which in turn can be communicatively coupled to DSP 3460. In at least one embodiment, audio unit 3464 can include, without limitation, an audio coder / decoder (“codec”) and a class D amplifier. In at least one embodiment, a SIM card (“SIM”) 3457 can be communicatively coupled to WWAN unit 3456. In at least one embodiment, components such as WLAN unit 3450 and Bluetooth unit 3452, and WWAN unit 3456 can be implemented into a next generation form factor (NGFF).

[0356] In at least one embodiment, sender device 102 and / or receiver device 104 can each be implemented using system 3400. In such an embodiment, sender processor 114 and / or receiver processor 126 can each be implemented as processor 3410. Sender memory 116 and / or receiver memory 128 can each be implemented using any memory of system 3400, such as BIOS, FW flash 3422, and / or LPDDR3 3415. Sender bus 228 and / or receiver bus 418 can each be implemented using any bus of system 3400. Sender communication 112 and / or receiver communication 124 can each be implemented as WLAN unit 3450 and / or WWAN unit 3456.

[0357] In at least one embodiment, one or more of Tx host computers 524, 526, and 528 and / or Rx host computer 540 can each be implemented using system 3400. In such embodiments, transmitting processor 114 and / or receiving processor 126 can each be implemented as processor 3410. Transmitting memory 116 and / or receiving memory 128 can each be implemented using any memory of system 3400, such as BIOS, FW flash 3422, and / or LPDDR3 3415. Transmitting bus 228 and / or receiving bus 418 can each be implemented using any bus of system 3400. Transmitting communication device 112 and / or receiving communication device 124 can each be implemented as WLAN unit 3450 and / or WW AN unit 3456.

[0358] In at least one embodiment, O-DU 812 and / or O-RU 814 can each be implemented using system 3400. In such embodiments, transmitting processor 114 and / or receiving processor 126 can each be implemented as processor 3410. Transmitting memory 116 and / or receiving memory 128 can each be implemented using any memory of system 3400, such as BIOS, FW flash 3422, and / or LPDDR3 3415. Transmitting bus 228 and / or receiving bus 418 can each be implemented using any bus of system 3400. Transmitting communication device 112 and / or receiving communication device 124 can each be implemented as WLAN unit 3450 and / or WW AN unit 3456.

[0359] Figure 36 An exemplary integrated circuit 3500 is shown, in accordance with at least one embodiment. In at least one embodiment, exemplary integrated circuit 3500 is a SoC, which can be fabricated using one or more IP cores. In at least one embodiment, integrated circuit 3500 includes one or more application processors 3505 (e.g., CPUs), at least one graphics processor 3510, and can additionally include an image processor 3515 and / or a video processor 3520, any of which can be a modular IP core. In at least one embodiment, integrated circuit 3500 includes peripheral or bus logic including a USB controller 3525, a UART controller 3530, an SPI / SDIO controller 3535, and an I2S / I2C port 3540, any of which can be a modular IP core. 2 S / I 2C Controller 3540. In at least one embodiment, integrated circuit 3500 can include a display device 3545 coupled to one or more of a High-Definition Multimedia Interface (HDMI) controller 3550 and a mobile industry processor interface (MIPI) display interface 3555. In at least one embodiment, storage can be provided by a flash memory subsystem 3560 including flash memory and a flash memory controller. In at least one embodiment, a memory interface can be provided via a memory controller 3565 for access to SDRAM or SRAM memory devices. In at least one embodiment, some integrated circuits also include an embedded Security Engine 3570.

[0360] In at least one embodiment, sender device 102 and / or receiver device 104 can each be implemented using integrated circuit 3500. In such embodiments, sender processor 114 and / or receiver processor 126 can each be implemented using any processor, such as application processor 3505, graphics processor 3510, image processor 3515, and / or video processor 3520 of integrated circuit 3500. Sender memory 116 and / or receiver memory 128 can each be implemented using any memory of integrated circuit 3500. Sender bus 228 and / or receiver bus 418 can each be implemented using any bus of integrated circuit 3500. Sender communication device 112 and / or receiver communication device 124 can each be implemented as any communication device of integrated circuit 3500.

[0361] In at least one embodiment, one or more of Tx host computers 524, 526, and 528 and / or Rx host computer 540 can each be implemented using integrated circuit 3500. In such embodiments, sender processor 114 and / or receiver processor 126 can each be implemented using any processor, such as application processor 3505, graphics processor 3510, image processor 3515, and / or video processor 3520 of integrated circuit 3500. Sender memory 116 and / or receiver memory 128 can each be implemented using any memory of integrated circuit 3500. Sender bus 228 and / or receiver bus 418 can each be implemented using any bus of integrated circuit 3500. Sender communication device 112 and / or receiver communication device 124 can each be implemented as any communication device of integrated circuit 3500.

[0362] In at least one embodiment, O-DU 812 and / or O-RU 814 can each be implemented using integrated circuit 3500. In such embodiments, sender processor 114 and / or receiver processor 126 can each be implemented using any processor, such as application processor 3505, graphics processor 3510, image processor 3515, and / or video processor 3520 of integrated circuit 3500. Sender memory 116 and / or receiver memory 128 can each be implemented using any memory of integrated circuit 3500. Sender bus 228 and / or receiver bus 418 can each be implemented using any bus of integrated circuit 3500. Sender communication device 112 and / or receiver communication device 124 can each be implemented as any communication device of integrated circuit 3500.

[0363] Figure 36 A computing system 3600 according to at least one embodiment is shown. In at least one embodiment, computing system 3600 includes a processing subsystem 3601 with one or more processor(s) 3602 and system memory 3604 communicating via an interconnection path 3605 that can include a memory hub 3605. In at least one embodiment, memory hub 3605 can be a separate component containing only a memory controller 3605, while in at least one embodiment, memory hub 3605 can be integrated into processor 3602. In at least one embodiment, memory hub 3605 couples with processor 3602 via a processor bus 3608. In at least one embodiment, memory hub 3605 is coupled with system memory 3604, input / output (I / O) subsystem 3611 and graphics subsystem 3612 via memory bus 3606. In at least one embodiment, memory bus 3606 can be one of many types of memory buses, including a point-to-point bus, a shared bus, and others. In at least one embodiment, processor 3602 can comprise a single processing unit or can comprise two, four, eight or greater numbers of processing units, depending on performance and cost constraints. In at least one embodiment, processor 3602 can include single integrated access graphics (SIAG) processor and shared L2 cache 3603.

[0364] In at least one embodiment, processing subsystem 3601 includes one or more parallel processors 3612 coupled to memory hub 3605 via a bus or other communication link 3613. In at least one embodiment, communication link 3613 can be one of many based on well-known bus or

[0365] In at least one embodiment, system storage 3614 can be connected to I / O hub 3607 to provide a storage mechanism for computing system 3600. In at least one embodiment, I / O

[0366] In at least one embodiment, computing system 3600 can include other components not explicitly shown, including USB or other port connections, optical storage drives, video capture devices, and / or the like, which can also be connected to I / O hub 3607. In at least one embodiment, communication paths interconnecting various components in computing system 3600 can implement one or more busses of the same type or several different types of busses, such as a memory bus or memory bus system, a peripheral bus, or local bus with a different bus architecture Figure 37 Communication paths interconnecting various components in computing system 3600 can use any suitable protocols including, for example, PCI (Peripheral Component Interconnect) based protocols (e.g., PCI Express), or other bus or point-to-point communication interfaces and / or protocols (e.g., NVLink high-speed interconnect, or interconnect protocols).

[0367] In at least one embodiment, parallel processor(s) 3612 include circuitry optimized for graphics and video processing, including video output circuitry in at least one embodiment, and can be configured for general purpose processing. In at least one embodiment, parallel processor(s) 3612 incorporate circuitry optimized for a particular design and can include, among other elements, one or more general purpose processing cores (“GPUs”). In at least one embodiment, a GPU may

[0368] In at least one embodiment, sending device 102 and / or receiving device 104 can each be implemented using computing system 3600. In such an embodiment, sending processor 114 and / or receiving processor 126 can each be implemented using any processor of computing system 3600, such as processor 3602. Sending memory 116 and / or receiving memory 128 can each be implemented using any memory of computing system 3600, such as system memory 3604. Sending bus 228 and / or receiving bus 418 can each be implemented using any bus of computing system 3600, such as communication link 3613. Sending communication device 112 and / or receiving communication device 124 can each be implemented as any communication device of computing system 3600, such as network adapter 3618 and / or wireless network adapter 3619.

[0369] In at least one embodiment, one or more of Tx host computers 524, 526, and 528 and / or Rx host computer 540 can each be implemented using computing system 3600. In such embodiments, sender processor 114 and / or receiver processor 126 can each be implemented using any processor of computing system 3600, such as processor 3602. Sender memory 116 and / or receiver memory 128 can each be implemented using any memory of computing system 3600, such as system memory 3604. Sender bus 228 and / or receiver bus 418 can each be implemented using any bus of computing system 3600, such as communication link 3613. Sender communication device 112 and / or receiver communication device 124 can each be implemented as any communication device of computing system 3600, such as network adapter 3618 and / or wireless network adapter 3619.

[0370] In at least one embodiment, O-DU 812 and / or O-RU 814 can each be implemented using computing system 3600. In such embodiments, sender processor 114 and / or receiver processor 126 can each be implemented using any processor of computing system 3600, such as processor 3602. Sender memory 116 and / or receiver memory 128 can each be implemented using any memory of computing system 3600, such as system memory 3604. Sender bus 228 and / or receiver bus 418 can each be implemented using any bus of computing system 3600, such as communication link 3613. Sender communication device 112 and / or receiver communication device 124 can each be implemented as any communication device of computing system 3600, such as network adapter 3618 and / or wireless network adapter 3619.

[0371] Processing system

[0372] The following figures illustrate, without limitation, exemplary processing systems that can be used to implement at least one embodiment.

[0373] Figure 38An accelerated processing unit (“APU”) 3700, in accordance with at least one embodiment, is shown. In at least one embodiment, APU 3700 is developed by AMD Corporation of Santa Clara, California. In at least one embodiment, APU 3700 can be configured to execute application programs such as CUDA programs. In at least one embodiment, APU 3700 includes, without limitation, core complex 3710, graphics complex 3740, fabric 3760, I / O interface 3770, memory controllers 3780, display controllers 3792, and multimedia engines 3794. In at least one embodiment, APU 3700 can include, without limitation, any combination of any number of core complexes 3710, any number of graphics complexes 3740, any number of display controllers 3792, and any number of multimedia engines 3794. For purposes of illustration, multiple instances of like objects are denoted with reference numerals including a leading asterisk and a number identifying the object, and a number in parentheses identifying the instance.

[0374] In at least one embodiment, core complex 3710 is a CPU, graphics complex 3740 is a GPU, and APU 3700 is a processing unit that integrates, without limitation, 3710 and 3740 onto a single chip. In at least one embodiment, some tasks can be assigned to core complex 3710, while other tasks can be assigned to graphics complex 3740. In at least one embodiment, core complex 3710 is configured to execute host software associated with APU 3700, such as an operating system. In at least one embodiment, core complex 3710 is a master processor of APU 3700 that controls and coordinates operations of other processors. In at least one embodiment, core complex 3710 issues commands that control operations of graphics complex 3740. In at least one embodiment, core complex 3710 can be configured to execute host executable code derived from CUDA source code, and graphics complex 3740 can be configured to execute device executable code derived from CUDA source code.

[0375] In at least one embodiment, core complex 3710 includes, without limitation, cores 3720(1)-3720(4) and L3 cache 3730. In at least one embodiment, core complex 3710 can include, without limitation, any number of cores 3720 and any number and type of caches in any combination. In at least one embodiment, cores 3720 are configured to execute instructions of a particular instruction set architecture (“ISA”). In at least one embodiment, each core 3720 is a CPU core.

[0376] In at least one embodiment, each core 3720 includes, without limitation, a fetch / decode unit 3722, an integer execution engine 3724, a floating point execution engine 3726, and an L2 cache 3728. In at least one embodiment, fetch / decode unit 3722 fetches instructions, decodes such instructions, generates micro-operations, and dispatches individual micro-instructions to integer execution engine 3724 and floating point execution engine 3726. In at least one embodiment, fetch / decode unit 3722 can simultaneously dispatch one micro-instruction to integer execution engine 3724 and another micro-instruction to floating point execution engine 3726. In at least one embodiment, integer execution engine 3724 executes not limited to integer and memory operations. In at least one embodiment, floating point engine 3726 executes not limited to floating point and vector operations. In at least one embodiment, fetch-decode unit 3722 dispatches micro-instructions to a single execution engine in place of both integer execution engine 3724 and floating point execution engine 3726.

[0377] In at least one embodiment, each core 3720(i) can access L2 cache 3728(i) included in core 3720(i), where i is an integer representing a particular instance of core 3720. In at least one embodiment, each core 3720 included in core complex 3710(j) is connected to other cores 3720 included in core complex 3710(j) via L3 cache 3730(j) included in core complex 3710(j), where j is an integer representing a particular instance of core complex 3710. In at least one embodiment, cores 3720 included in core complex 3710(j) can access all L3 caches 3730(j) included in core complex 3710(j), where j is an integer representing a particular instance of core complex 3710. In at least one embodiment, L3 cache 3730 can include, without limitation, any number of slices.

[0378] In at least one embodiment, graphics complex 3740 can be configured to perform compute operations in a highly parallel manner. In at least one embodiment, graphics complex 3740 is configured to perform graphics pipeline operations such as draw commands, pixel operations, geometric computations, and other operations associated with rendering images to a display. In at least one embodiment, graphics complex 3740 is configured to perform operations that are not graphics related. In at least one embodiment, graphics complex 3740 is configured to perform both graphics related operations and operations that are not graphics related.

[0379] In at least one embodiment, graphics processing engine 3710 includes, without limitation, any number of graphics processing clusters 3740 and shared function unit(s) 3750. In at least one embodiment, graphics processing clusters 3740 share shared function unit(s) 3750. In at least one embodiment, shared function unit(s) 3750 are partitioned. In at least one embodiment, graphics processing engine 3710 includes, without limitation, any number of graphics processing clusters 3740 and any number (including zero) and type of shared function unit(s). In at least one embodiment, graphics processing engine 3710 includes, without limitation, any number of specialized graphics hardware.

[0380] In at least one embodiment, each graphics processing cluster 3740 includes, without limitation, any number of SIMD units 3752 and shared memory 3754. In at least one embodiment, each SIMD unit 3752 implements a SIMD architecture and is configured to execute operations in parallel. In at least one embodiment, each graphics processing cluster 3740 can execute any number of thread blocks, but each thread block executes on a single graphics processing cluster 3740. In at least one embodiment, a thread block includes, without limitation, any number of threads. In at least one embodiment, a workgroup is a thread block. In at least one embodiment, each SIMD unit 3752 executes a different thread warp. In at least one embodiment, a thread warp is a group of threads (e.g., 16 threads), where each thread in a thread warp belongs to a single thread block and is configured to process a different set of data based on a single instruction set. In at least one embodiment, one or more threads in a thread warp can be disabled using predication. In at least one embodiment, a lane is a thread. In at least one embodiment, a work item is a thread. In at least one embodiment, a wavefront is a thread warp. In at least one embodiment, different wavefronts in a thread block can be synchronized together and communicate via shared memory 3754.

[0381] In at least one embodiment, fabric 3760 is a system interconnect that facilitates data and control transmissions across core complex 3710, graphics complex 3740, I / O interface 3770, memory controllers 3780, display controller 3792, and multimedia engine 3794. In at least one embodiment, APU 3700 can include, without limitation, any number and type of system interconnects in addition to or instead of fabric 3760 that facilitate data and control transmissions across any number and type of directly or indirectly linked components that can be internal or external to APU 3700. In at least one embodiment, I / O interface 3770 represents any number and type of I / O interface (e.g., PCI, PCI-Extended (“PCI-X”), PCIe, Gigabit Ethernet (“GBE”), USB, etc.). In at least one embodiment, various types of peripheral devices are coupled to I / O interface 3770. In at least one embodiment, peripheral devices coupled to I / O interface 3770 can include, without limitation, a keyboard, a mouse, a printer, a scanner, a joystick or other type of game controller, a media recording device, an external storage device, a network interface card, etc.

[0382] In at least one embodiment, display controller AMD92 displays images on one or more display devices, such as liquid crystal display (“LCD”) devices. In at least one embodiment, multimedia engine 3794 includes, without limitation, any number and type of multimedia-related circuitry, such as a video decoder, a video encoder, an image signal processor, etc. In at least one embodiment, memory controllers 3780 facilitate data transfers between APU 3700 and unified system memory 3790. In at least one embodiment, core complex 3710 and graphics complex 3740 share unified system memory 3790.

[0383] In at least one embodiment, APU 3700 implements a memory subsystem that includes, without limitation, any number and type of memory controllers 3780 and memory devices (e.g., shared memory 3754) that can be dedicated to one component or shared among multiple components. In at least one embodiment, APU 3700 implements a cache subsystem that includes, without limitation, one or more cache memories (e.g., L2 cache 3828, L3 cache 3730, and L2 cache 3742), each of which can be component private or shared among any number of components (e.g., core 3720, core complex 3710, SIMD unit 3752, compute unit 3750, and graphics complex 3740).

[0384] In at least one embodiment, sender processor 114 and / or receiver processor 126 can each be implemented as an APU 3700. In such an embodiment, sender memory 116 and / or receiver memory 128 can each be implemented as a unified system memory 3790.

[0385] Figure 39 A CPU 3800 according to at least one embodiment is shown. In at least one embodiment, CPU 3800 is developed by AMD Corporation of Santa Clara, California. In at least one embodiment, CPU 3800 can be configured to execute application programs. In at least one embodiment, CPU 3800 is configured to execute host executable code derived from CUDA source code, and an external GPU can be configured to execute device executable code derived from such CUDA source code. In at least one embodiment, CPU 3800 includes, without limitation, any number of core complexes 3810, fabric 3860, I / O interfaces 3870, and memory controllers 3880.

[0386] In at least one embodiment, core complex 3810 includes, without limitation, cores 3820(1)-3820(4) and L3 cache 3830. In at least one embodiment, core complex 3810 can include, without limitation, any number of cores 3820 and any combination and type of caches. In at least one embodiment, cores 3820 are configured to execute instructions of a particular ISA. In at least one embodiment, each core 3820 is a CPU core.

[0387] In at least one embodiment, each core 3820 includes, without limitation, a fetch / decode unit 3822, an integer execution engine 3824, a floating point execution engine 3826, and an L2 cache 3828. In at least one embodiment, fetch / decode unit 3822 fetches instructions, decodes such instructions, generates micro-operations, and dispatches individual micro-instructions to integer execution engine 3824 and floating point execution engine 3826. In at least one embodiment, fetch / decode unit 3822 can simultaneously dispatch one micro-instruction to integer execution engine 3824 and another micro-instruction to floating point execution engine 3826. In at least one embodiment, integer execution engine 3824 executes integer and memory operations, not limited to. In at least one embodiment, floating point engine 3826 executes floating point and vector operations, not limited to. In at least one embodiment, fetch-decode unit 3822 dispatches micro-instructions to a single execution engine in place of both integer execution engine 3824 and floating point execution engine 3826.

[0388] In at least one embodiment, each core 3820(i) has access to an L2 cache 3828(i) included in core 3820(i), where i is an integer representing a particular instance of core 3820. In at least one embodiment, each core 3820 included in core complex 3810(j) is connected to the other cores 3820 in core complex 3810(j) via an L3 cache 3830(j) included in core complex 3810(j), where j is an integer representing a particular instance of core complex 3810. In at least one embodiment, cores 3820 included in core complex 3810(j) have access to all L3 caches 3830(j) included in core complex 3810(j), where j is an integer representing a particular instance of core complex 3810. In at least one embodiment, L3 cache 3830 can include, without limitation, any number of slices.

[0389] In at least one embodiment, structure 3860 is a system interconnect that facilitates data and control transfers across core complexes 3810(1)-3810(N) (where N is a positive integer), I / O interface 3870, and memory controller 3880. In at least one embodiment, in addition to or instead of structure 3860, CPU 3800 may also include, but is not limited to, any number and type of system interconnects that facilitate data and control transfers across any number and type of components that may be directly or indirectly linked, either inside or outside CPU 3800. In at least one embodiment, I / O interface 3870 represents any number and type of I / O interfaces (e.g., PCI, PCI-X, PCIe, GBE, USB, etc.). In at least one embodiment, various types of peripheral devices are coupled to I / O interface 3870. In at least one embodiment, peripheral devices coupled to I / O interface 3870 may include, but are not limited to, displays, keyboards, mice, printers, scanners, joysticks or other types of game controllers, media recording devices, external storage devices, network interface cards, etc.

[0390] In at least one embodiment, memory controller 3880 facilitates data transfer between CPU 3800 and system memory 3890. In at least one embodiment, core complex 3810 and graphics complex 3840 share system memory 3890. In at least one embodiment, CPU 3800 implements a memory subsystem, which includes, but is not limited to, any number and type of memory controllers 3880 and memory devices that may be dedicated to a component or shared among multiple components. In at least one embodiment, CPU 3800 implements a cache subsystem, which includes, but is not limited to, one or more cache memories (e.g., L2 cache 3828 and L3 cache 3830), each cache memory may be component-private or shared among any number of components (e.g., core 3820 and core complex 3810).

[0391] In at least one embodiment, the sender processor 114 and / or the receiver processor 126 may each be implemented as a CPU 3800. In such embodiments, the sender memory 116 and / or the receiver memory 128 may each be implemented as a system memory 3890.

[0392] Figures 39-48 An exemplary accelerator integration slice 3990 according to at least one embodiment is shown. (See reference...) Slice Control RegisterAs used, a“slice” includes a specified portion of the processing resources of an accelerator integration circuit. In at least one embodiment, an accelerator integration circuit provides cache management, memory access, environment management, and interrupt management services on behalf of a plurality of graphics processing engines representing a plurality of graphics acceleration modules. Graphics processing engines can each comprise a separate GPU. Alternatively, graphics processing engines can comprise different types of graphics processing engines within a GPU, such as graphics execution units, media processing engines (e.g., video encoders / decoders), samplers, and blit engines. In at least one embodiment, a graphics acceleration module can be a GPU with a plurality of graphics processing engines. In at least one embodiment, a graphics processing engine can be a GPU integrated on a general purpose package, line card, or chip.

[0393] Application effective address space 3982 within system memory 3914 stores process elements 3983. In one embodiment, process elements 3983 are stored in response to GPU invocations 3981 from applications 3980 executing on processor 3907. Process elements 3983 contain processing state for respective applications 3980. A work descriptor (WD) 3984 contained in process element 3983 can be a single job requested by an application or can contain pointers to a queue of jobs. In at least one embodiment, WD 3984 is a pointer to a job request queue in application effective address space 3982.

[0394] Graphics acceleration module 3946 and / or individual graphics processing engines can be shared by all or a subset of processes in a system. In at least one embodiment, a process element can include infrastructure for setting up processing state and sending WDs 3984 to graphics acceleration module 3946 to start a job in a virtualized environment.

[0395] In at least one embodiment, a dedicated process programming model is implemented for graphics acceleration module 3946. In this model, a single process owns graphics acceleration module 3946 or an individual graphics processing engine. As graphics acceleration module 3946 is owned by a single process, a hypervisor initializes the accelerator integration circuit for the owning partition and an operating system initializes the accelerator integration circuit for the owning partition when graphics acceleration module 3946 is assigned.

[0396] During operation, the WD fetch unit 3991 in the accelerator integrated slice 3990 fetches the next WD 3984, which includes instructions for the work to be performed by one or more graphics processing engines of the graphics acceleration module 3946. Data from the WD 3984 can be stored in register 3945 and used by the memory management unit (MMU) 3939, interrupt management circuitry 3947, and / or environment management circuitry 3948, as shown. For example, the MMU 3939 includes segment / page roaming circuitry for accessing segment / page tables 3986 within the OS virtual address space 3985. The interrupt management circuitry 3947 can handle interrupt events (INT) 3992 received from the graphics acceleration module 3946. When performing graph operations, the effective address 3993 generated by the graphics processing engine is translated into an actual address by the MMU 3939.

[0397] In one embodiment, the same register set 3945 is copied for each graphics processing engine and / or graphics acceleration module 3946 and can be initialized by the hypervisor or operating system. Each of these copied registers can be included in the accelerator integration slice 3990. Exemplary registers that can be initialized by the hypervisor are shown in Table 1.

[0398] Table 1 – Registers for Supervisor Initialization

[0399] 1 Real Address (RA) Plan's Processing Region Pointer 2 Authorization Mask Override Register 3 Interrupt Vector Table Input Offset 4 Interrupt Vector Table Entry Limit 5 Status Register 6 Logical Partition ID 7 Real Address (RA) Hypervisor Accelerator Utilization Log Pointer 8 Storage Description Register 9 Figure 39

[0400] Table 2 shows exemplary registers that can be initialized by the operating system.

[0401] Table 2 – Operating System Initialization Registers

[0402]

[0403]

[0404] In one embodiment, each WD 3984 is specific to a particular graphics acceleration module 3946 and / or a particular graphics processing engine. It contains all the information required for the graphics processing engine to perform its work or to do so, or it may be a pointer to a memory location where the application has established a command queue for the work to be done.

[0405] In at least one embodiment, the sending device 102 and / or the receiving device 104 can each be used. Figure 39 The system is implemented in this way. In such an embodiment, the sender processor 114 and / or the receiver processor 126 may each be implemented as processor 3907, and the sender memory 116 and / or the receiver memory 128 may each be implemented as system memory 3914.

[0406] In at least one embodiment, one or more of Tx host computers 524, 526, and 528 and / or Rx host computer 540 can each be implemented using an embodiment of a system-on-a-chip (SoC) as described in more detail below. Figure 39 In such embodiments, sender processor 114 and / or receiver processor 126 can each be implemented as processor 3907, and sender memory 116 and / or receiver memory 128 can each be implemented as system memory 3914.

[0407] In at least one embodiment, O-DU 812 and / or O-RU 814 can each be implemented using an embodiment of a system-on-a-chip (SoC) as described in more detail below. Figures 40A-40B In such embodiments, sender processor 114 and / or receiver processor 126 can each be implemented as processor 3907, and sender memory 116 and / or receiver memory 128 can each be implemented as system memory 3914.

[0408] Figure 40A An exemplary graphics processor according to at least one embodiment is shown. In at least one embodiment, any of the exemplary graphics processors can be fabricated as one or more IP cores. In addition to the graphics processors illustrated, in at least one embodiment, other logistics and circuitry can be included, including additional graphics processors cores, peripheral interface controllers or general-purpose processor cores. In at least one embodiment, an exemplary graphics processor is fabricated using one or more IP cores.

[0409] Figure 40B An exemplary graphics processor 4010 of an SoC integrated circuit that can be fabricated using one or more IP cores is shown, according to at least one embodiment. Figure 40A An additional exemplary graphics processor 4040 of an SoC integrated circuit that can be fabricated using one or more IP cores is shown, according to at least one embodiment. In at least one embodiment, graphics processor 4010 is a low-power graphics processor core. In at least one embodiment, graphics processor 4040 is a higher performance graphics processor core. In at least one embodiment, each of graphics processors 4010, 4040 can be a variant of the graphics processor of Figure 16. Figure 40B An additional exemplary graphics processor 4040 of an SoC integrated circuit that can be fabricated using one or more IP cores is shown, according to at least one embodiment. In at least one embodiment, graphics processor 4010 is a low-power graphics processor core. In at least one embodiment, graphics processor 4040 is a higher performance graphics processor core. In at least one embodiment, each of graphics processors 4010, 4040 can be a variant of the graphics processor of Figure 16. Figure 40A An additional exemplary graphics processor 4040 of an SoC integrated circuit that can be fabricated using one or more IP cores is shown, according to at least one embodiment. In at least one embodiment, graphics processor 4010 is a low-power graphics processor core. In at least one embodiment, graphics processor 4040 is a higher performance graphics processor core. In at least one embodiment, each of graphics processors 4010, 4040 can be a variant of the graphics processor of Figure 16.

[0410] In at least one embodiment, graphics processor 4010 includes a vertex processor 4005 and one or more fragment processor(s) 4015A-4015N (e.g., 4015A, 4015B, 4015C, 4015D, through 4015N-1, and 4015N). In at least one embodiment, graphics processor 4010 can execute different shader programs via separate logic for vertex processing and / or for fragment / pixel processing, where some shader programs may only be used in one or the other. In at least one embodiment, graphics processor 4010 can be optimized to execute multiple instances of one or more same shader programs simultaneously. In at least one embodiment, graphics processor 4010 may

[0411] In at least one embodiment, graphics processor 4010 additionally includes one or more MMU(s) 4020A-4020B, cache memory 4025A-4025B, and circuit interconnect 4030A-4030B. In at least one embodiment, one or more MMU(s) 4020A-4020B provide for virtual to physical address mapping for graphics processor 4010, including for vertex processor 4005 and / or fragment processor 4015A-4015N, which can reference vertex or image / texture data stored in memory, in addition to vertex or image / texture data stored in one or more cache(s) 4025A-4025B. In at least one embodiment, one or more MMU(s) 4020A-4020B can be synchronized with other MMUs within a system, including one or more MMUs associated with one or more application processor(s), image processor(s), and / or video processor(s), so that each processor can participate in a shared or unified virtual memory system. In at least one embodiment, one or more circuit interconnects 4030A-4030B enable graphics processor 4010 to interface with other IP cores within a SoC, either via an internal bus, as shown, or via a direct connection.

[0412] In at least one embodiment, graphics processor 4040 includes Figure 41Aone or more MMUs 4020A-4020B, caches 4025A-4025B, and circuit interconnect 4030A-4030B of graphics processor 4010. In at least one embodiment, graphics processor 4040 includes one or more shader core(s) 4055A-4055N (e.g., 4055A, 4055B, 4055C, 4055D, 4055E, 4055F, through 4055N-1, and 4055N), which provides for a unified shader core architecture in which a single core or type or core can execute all types of programmable shader code, including shader program code to implement vertex shaders, fragment shaders, and / or compute shaders. In at least one embodiment, a plurality of shader cores can be varied. In at least one embodiment, graphics processor 4040 includes an inter-core task manager 4045, which function as a thread dispatcher to assign tasks to one or more shader cores 4055A-4055N and block unit 4058 for use in accelerated processing of tasks in response to batched tasks from graphics processing engine(s) 420. In at least one embodiment, graphics processor 4040 includes a task distributor 4052 that divides incoming tasks equally between task manager 4045 and a video encode / decode core 4056.

[0413] In at least one embodiment, sender device 102 and / or receiver device 104 can each include graphics processor 4010 and / or graphics processor 4040. In at least one embodiment, one or more of Tx host computers 524, 526, and 528 and / or Rx host computer 540 can each include graphics processor 4010 and / or graphics processor 4040. In at least one embodiment, O-DU 812 and / or O-RU 814 can each include graphics processor 4010 and / or graphics processor 4040.

[0414] Figure 35 Graphics core 4100 is shown in accordance with at least one embodiment. In at least one embodiment, graphics core 4100 can be included within graphics processor 3510. Figure 40B Graphics core 4100 is shown in accordance with at least one embodiment. In at least one embodiment, graphics core 4100 can be included within graphics processor 3510. Figure 41BThe unified shader cores 4055A-4055N. In at least one embodiment, graphics core 4100 includes a shared instruction cache 4102, texture units 4118, and cache / shared memory 4120, which are common to the execution resources within graphics core 4100. In at least one embodiment, graphics core 4100 can include a number of slices 4101A-4101N or partitions of each core, graphics processing units can include multiple instances of graphics core 4100. Slices 4101A-4101N can include support logic including a local instruction cache 4104A-4104N, a thread scheduler 4106A-4106N, a thread dispatcher 4108A-4108N, and a set of registers 4110A-4110N. In at least one embodiment, slices 4101A-4101N can include a set of additional functional units (AFUs) 4112A-4112N, floating point units (FPUs) 4114A-4114N, integer arithmetic logic units (ALUs) 4116A-4116N, address computation units (ACUs) 4113A-4113N, double precision floating point units (DPFPUs) 4115A-4115N, and matrix processing units (MPUs) 4117A-4117N.

[0415] In one embodiment, FPUs 4114A-4114N can perform single precision (32-bit) and half precision (16-bit) floating point operations, while DPFPUs 4115A-4115N can perform double precision (64-bit) floating point operations. In at least one embodiment, ALUs 4116A-4116N can perform variable precision integer operations in 8-bit, 16-bit, and 32-bit precision, and can be configured to perform mixed precision operations. In at least one embodiment, MPUs 4117A-4117N can also be configured to perform mixed precision matrix operations including half-precision floating point operations and 8-bit integer operations. In at least one embodiment, MPUs 4117A-4117N can perform a variety of matrix operations to accelerate CUDA programs, including enabling support for accelerated general matrix to matrix multiplication (GEMM). In at least one embodiment, AFUs 4112A-4112N can perform additional logical operations not supported by floating point or integer units, including trigonometric operations (e.g., Sine, Cosine, etc.).

[0416] Figure 41AA general-purpose graphics processing unit (GPGPU) 4130 is illustrated in at least one embodiment. In at least one embodiment, the GPGPU 4130 is highly parallel and suitable for deployment on a multi-chip module. In at least one embodiment, the GPGPU 4130 can be configured to enable highly parallel computational operations to be performed by a GPU array. In at least one embodiment, the GPGPU 4130 can be directly linked to other instances of the GPGPU 4130 to create a multi-GPU cluster to improve execution time for CUDA programs. In at least one embodiment, the GPGPU 4130 includes a host interface 4132 for connection to a host processor. In at least one embodiment, the host interface 4132 is a PCIe interface. In at least one embodiment, the host interface 4132 can be a vendor-specific communication interface or communication structure. In at least one embodiment, the GPGPU 4130 receives commands from the host processor and uses a global scheduler 4134 to assign execution threads associated with those commands to a set of computational clusters 4136A-4136H. In at least one embodiment, computing clusters 4136A-4136H share cache memory 4138. In at least one embodiment, cache memory 4138 can be used as an advanced cache of cache memory within computing clusters 4136A-4136H.

[0417] In at least one embodiment, the GPGPU 4130 includes memories 4144A-4144B coupled to computing clusters 4136A-4136H via a set of memory controllers 4142A-4142B. In at least one embodiment, memories 4144A-4144B may include various types of memory devices, including dynamic random access memory (DRAM) or graphics random access memory, such as synchronous graphics random access memory (SGRAM), including graphics double data rate (GDDR) memory.

[0418] In at least one embodiment, computing clusters 4136A-4136H each include a set of graphics cores, such as Figure 42A The graphics core 4100 may include various types of integer and floating-point logic units, capable of performing computational operations at various precisions, including computations suitable for CUDA programs. For example, in at least one embodiment, at least a subset of the floating-point units in each computing cluster 4136A-4136H may be configured to perform 16-bit or 32-bit floating-point operations, while different subsets of the floating-point units may be configured to perform 64-bit floating-point operations.

[0419] In at least one embodiment, multiple instances of GPGPU 4130 can be configured to operate as a compute cluster. In at least one embodiment, compute clusters 4136A-4136H can implement any technically feasible communication technology for synchronization and data exchange. In at least one embodiment, multiple instances of GPGPU 4130 communicate over host interface 4132. In at least one embodiment, GPGPU 4130 includes I / O hub 4139 that couples GPGPU 4130 with GPU links 4140 enabling direct connection to other instances of GPGPU 4130. In at least one embodiment, GPU links 4140 couple to a specialized GPU-to-GPU bridge enabling communication and synchronization among multiple instances of GPGPU 4130. In at least one embodiment, GPU links 4140 couple with a high-speed interconnect to transmit and receive data to other GPGPUs or parallel processors. In at least one embodiment, multiple instances of GPGPU 4130 are located in separate data processing systems and communicate via a network device accessible via host interface 4132. In at least one embodiment, GPU links 4140 can be configured to connect to a host processor, in addition to or in lieu of host interface 4132. In at least one embodiment, GPGPU 4130 can be configured to execute a CUDA program.

[0420] In at least one embodiment, sender device 102 and / or receiver device 104 can each include graphics core 4100 and / or GPGPU 4130. In at least one embodiment, one or more of Tx host computers 524, 526, and 528 and / or Rx host computer 540 can each include graphics core 4100 and / or GPGPU 4130. In at least one embodiment, O-DU 812 and / or O-RU 814 can each include graphics core 4100 and / or GPGPU 4130.

[0421] Figure 42A Parallel processor 4200 is shown in accordance with at least one embodiment. In at least one embodiment, various components of parallel processor 4200 can be implemented using one or more integrated circuit devices, such as programmable processor(s), application specific integrated circuit(s) (ASICs), or FPGA(s).

[0422] In at least one embodiment, parallel processor 4200 includes a parallel processing unit 4202. In at least one embodiment, parallel processing unit 4202 includes an I / O unit 4204 that enables communication with other devices, including other instances of parallel processing unit 4202. In at least one embodiment, I / O unit 4204 can be directly connected to other devices. In at least one embodiment, I / O unit 4204 connects with other devices via use of a hub or switch, for example, memory hub 1705. In at least one embodiment, connections between memory hub 1705 and I / O unit 4204 form a communication link. In at least one embodiment, I / O unit 4204 connects with a host interface 4206 and a memory crossbar switch 4216, where host interface 4206 receives commands directed to the processing operations and memory crossbar switch 4216 receives commands directed to memory operations.

[0423] In at least one embodiment, when host interface 4206 receives a command buffer via I / O unit 4204, host interface 4206 can direct work operations to execute those commands to front end 4208. In at least one embodiment, front end 4208 couples with a scheduler 4210, which is configured to assign commands or other work items to processing arrays 4212. In at least one embodiment, scheduler 4210 ensures that processing arrays 4212 are properly configured and in an active state before tasks are assigned to processing arrays 4212 within processing arrays 4212. In at least one embodiment, scheduler 4210 is implemented by firmware logic executing on a microcontroller. In at least one embodiment, microcontroller implemented scheduler 4210 is configurable to perform complex scheduling and work distribution operations, enabling fine grain task scheduling at a thread level and coarse grain task scheduling at a block level, to enable fast pre-emption and context switching for threads executing on processing arrays 4212. In at least one embodiment, host software can prove a workload for scheduling on processing arrays 4212 through one of a number of graphics processing doorbells. In at least one embodiment, workload can then be automatically distributed on processing arrays 4212 by scheduler 4210 logic within microcontroller including scheduler 4210.

[0424] In at least one embodiment, processing array 4212 can include up to “N” processing clusters (e.g., cluster 4214A, 4214B through 4214N). In at least one embodiment, each cluster 4214A-4214N of processing array 4212 can execute a large number of concurrent threads. In at least one embodiment, scheduler 4210 can allocate work to clusters 4214A-4214N of processing array 4212 using various scheduling and / or work distribution algorithms, which can be determined at least in part by workload arriving at processing array 4212, and / or received from other processor cores, processor arrays, or systems, which can be structured as a multi-core computer system, a multiprocessor computer system, or a distributed computer system.

[0425] In at least one embodiment, processing array 4212 can be configured to perform various types of parallel processing operations. In at least one embodiment, processing array 4212 is configured to perform general-purpose parallel compute operations. For example, in at least one embodiment, processing array 4212 can include logic to perform processing tasks including filtering of video and / or audio data, performing modeling operations including physical operations, and performing data transformations.

[0426] In at least one embodiment, processing array 4212 is configured to perform parallel graphics processing operations. In at least one embodiment, processing array 4212 can include additional logic to support performance of such graphics processing operations, including but not limited to texture sampling logic to perform texture operations, tiling logic, and other vertex processing logic. In at least one embodiment, processing array 4212 can be configured to execute graphics processing related shader programs, for example, but not limited to, vertex shaders, tessellation shaders, geometry shaders, and pixel shaders. In at least one embodiment, parallel processing unit 4202 can transfer data to be processed from a system memory over I / O unit 4204. In at least one embodiment, data can be stored to on-chip memory (e.g., parallel processor memory 4222) during processing and then written back to system memory.

[0427] In at least one embodiment, when parallel processing unit 4202 is used to perform graphics processing, scheduler 4210 can be configured to divide incoming workloads into tasks of approximately equal size to better enable distribution of graphics processing operations across multiple clusters 4214A-4214N of processing array 4212. In at least one embodiment, portions of processing array 4212 can be configured to perform different types of processing. For example, in at least one embodiment, a first portion can be configured to perform vertex shading and topology generation, a second portion can be configured to perform tessellation and geometry shading, and a third portion can be configured to perform pixel shading or other screen space operations to produce a rendered image for display on a display device. In at least one embodiment, intermediate data produced by one or more of clusters 4214A-4214N can be stored in buffers to allow transmission of intermediate data between clusters 4214A-4214N for further processing.

[0428] In at least one embodiment, processing array 4212 can receive processing tasks to be executed from scheduler 4210, which receives commands defining the processing tasks from front end 4208. In at least one embodiment, a processing task can include an index into data to be processed, such as can include surface (patch) data, raw data, vertex data, and / or pixel data, as well as state parameters and commands defining how the data is to be processed (e.g., what program is to be executed). In at least one embodiment, scheduler 4210 can be configured to fetch the index corresponding to a task, or can receive the index from front end 4208. In at least one embodiment, front end 4208 can be configured to ensure that processing array 4212 is configured in an effective state before launching a workload specified by an incoming command buffer (e.g., a batch-buffer, a push buffer, etc.).

[0429] In at least one embodiment, each of one or more instances of parallel processing unit 4202 can be coupled to parallel processor memory 4222. In at least one embodiment, parallel processor memory 4222 can be accessed by parallel processing array 4212, as well as I / O unit 4204, via memory crossbar 4216. In at least one embodiment, memory crossbar 4216 can be configured to include a crossbar switch to access the memory. In at least one embodiment, memory crossbar 4216 can be configured from on-die memory crossbar error correction code (ECC) logic 4218 to provide memory crossbar ECC on memory crossbar 4216. In at least one embodiment, memory crossbar 4216 can be configured to support mixed point-to-point and shared memory paradigms. In at least one embodiment, some or all accesses to memory crossbar 4216 can be done via registers. In at least one embodiment, memory crossbar 4216 can be configured to support a variety of memory architectures.

[0430] In at least one embodiment, memory units 4224A-4224N can include various types of memory devices including dynamic random access memory (DRAM) or graphics random access memory, such as synchronous graphics random access memory (SGRAM), including graphics double data rate (GDDR) memory. In at least one embodiment, memory units 4224A-4224N can also include 3D stacked memory, including but not limited to high bandwidth memory (HBM). In at least one embodiment, rendering targets such as frame buffers or texture maps can be stored across memory units 4224A-4224N, allowing partition units 4220A-4220N to write portions of each rendering target in parallel to efficiently use available bandwidth of parallel processor memory 4222. In at least one embodiment, local instances of parallel processor memory 4222 can be excluded to facilitate a unified memory design that utilizes system memory in combination with local cache memory.

[0431] In at least one embodiment, any of clusters 4214A-4214N of processing array 4212 can process data that is to be written into any of memory units 4224A-4224N within parallel processor memory 4222. In at least one embodiment, memory crossbar 4216 can be configured to transmit outputs of each cluster 4214A-4214N to any partition unit 4220A-4220N or another cluster 4214A-4214N, which can perform other processing operations on the outputs. In at least one embodiment, each cluster 4214A-4214N can communicate with memory interface 4218 through memory crossbar 4216 to read from or write to various external memory devices. In at least one embodiment, memory crossbar 4216 has a connection to memory interface 4218 to communicate with I / O unit 4204, as well as a local instance of parallel processor memory 4222, to enable processing units within different processing clusters 4214A-4214N to communicate with system memory or other memory that is not local to the parallel processing units 4202. In at least one embodiment, memory crossbar 4216 can use virtual channels to separate traffic flows between clusters 4214A-4214N and partition units 4220A-4220N.

[0432] In at least one embodiment, multiple instances of parallel processing unit 4202 can be provided on a single add-in card, or multiple add-in cards can be interconnected. In at least one embodiment, different instances of parallel processing unit 4202 can be configured to operate in coordination with each other to enable parallel processing operations. For example, in at least one embodiment, some instances of parallel processing unit 4202 can be configured to perform physics calculations for a game while others instances of parallel processing unit 4202

[0433] In at least one embodiment, sender processor 114 and / or receiver processor 126 can each be implemented as parallel processor 4200. In such embodiments, sender memory 116 and / or receiver memory 128 can each be implemented using any memory described in Figure 42A and / or described with respect to Figure 42B FIG. 20.

[0434] Figure 42AA processing cluster 4294 is shown in accordance with at least one embodiment. In at least one embodiment, processing cluster 4294 is included in a parallel processing unit. In at least one embodiment, processing cluster 4294 is an instance of one of processing clusters 4214A-4214N. In at least one embodiment, processing cluster 4294 can be configured to execute many threads in parallel, where the term “thread” refers to an instance of a particular program executed by a particular set of input data. In at least one embodiment, Single-Instruction, Multiple-Data (SIMD) instruction issue techniques are used to support parallel execution of a large number of threads, without requiring multiple independent instruction units. In at least one embodiment, Single-Instruction, Multiple Thread (SIMT) techniques are used to support parallel execution of a large number of generally synchronous threads, using a common instruction unit configured to issue instructions to a set of processing engines within each processing cluster 4294. Figure 42A

[0435] In at least one embodiment, operation of processing cluster 4294 can be controlled via a pipeline manager 4232 that is assigned to a SIMT parallel processor. In at least one embodiment, pipeline manager 4232 receives instructions from scheduler 4210 of FIG. 4A, and manages execution of those instructions via graphics multiprocessor 4234 and / or texture unit 4236. In at least one embodiment, graphics multiprocessor 4234 is an exemplary instance of a SIMT parallel processor. However, different architectures can be used for various types of SIMT parallel processors within processing cluster 4294. In at least one embodiment, one or more instances of graphics multiprocessor 4234 can be included within processing cluster 4294. In at least one embodiment, graphics multiprocessor 4234 can process data, and data crossbar 4240 can be used to distribute processed data to one of a number of possible destinations, including other shader units. In at least one embodiment, pipeline manager 4232 can facilitate distribution by specifying destinations for processed data as a function of the processed data. Figure 42A

[0436] In at least one embodiment, each graphics multiprocessor 4234 within processing cluster 4294 can include an identical set of functional execution logic (e.g., arithmetic logic units, load store units (LSUs), etc.). In at least one embodiment, functional execution logic can be configured in a pipelined manner, where new instructions can be issued before previous instructions are complete. In at least one embodiment, functional execution logic supports a variety of operations including integer and floating point arithmetic, comparison operations, Boolean operations, shift operations, and the like. In at least one embodiment, same functional-unit hardware can be leveraged to perform ​​

[0437] In at least one embodiment, instructions delivered to processing cluster 4294 constitute a thread. In at least one embodiment, a set of threads executing across a set of parallel processing engines is a thread group. In at least one embodiment, a thread group executes a program on different input data. In at least one embodiment, each thread within a thread group can be assigned to a different processing engine within graphics multiprocessor 4234. In at least one embodiment, a thread group can include fewer threads than are present in a plurality of processing engines. In at least one embodiment, when a thread group includes fewer threads than the number of processing engines present in graphics multiprocessor 4234, one or more of the processing engines can be idle during the execution of the threads in the thread group. In at least one embodiment, a thread group can also include more threads than are present in a plurality of processing engines. In at least one embodiment, when a thread group includes more threads than the number of processing engines present in graphics multiprocessor 4234, multiple threads in a thread group can be executed concurrently on multiple processing engines of graphics multiprocessor 4234. In at least one embodiment, parallel thread execution can be instructed up from a higher level paradigm, for example, a model thread executing on a CPU.

[0438] In at least one embodiment, graphics multiprocessor 4234 includes internal cache memory, to perform load and store operations. In at least one embodiment, graphics multiprocessor 4234 can fall back to using cache memory within processing cluster 4294 (e.g., LI cache 4248) for load and store operations. In at least one embodiment, each graphics multiprocessor 4234 can also have access to L2 cache within partition unit (e.g., partition unit 4220A-4220N) that is shared among all processing clusters 4294 and can be used to transfer data between threads. In at least one embodiment, graphics multiprocessor 4234 can also have access to off-chip global memory, which can include one or more of local parallel processor memory and / or system memory. In at least one embodiment, any memory external to parallel processor 4202 can be used as global memory. In at least one embodiment, processing cluster 4294 includes multiple instances of graphics multiprocessor 4234 that share shared cache memory that can be stored in LI cache 4248. Figure 42A

[0439] In at least one embodiment, each processing cluster 4294 can include an MMU 4245 configured to translate virtual addresses into physical addresses. In at least one embodiment, one or more instances of MMU 4245 can reside within graphics multiprocessor 4234. In at least one embodiment, graphics multiprocessor 4234 includes a cache memory 4248 to cache data stored in on-chip memory 4244. In at least one embodiment, graphics multiprocessor 4234 includes a shared memory 4250 to store data for threads executing on graphics multiprocessor 4234. In at least one embodiment, shared memory 4250 can include a shared pool of global memory that can be accessed by all threads executing on graphics multiprocessor 4234. In at least one embodiment, shared memory 4250 can include one or more shared memory segments within a global memory space. In at least one embodiment, shared memory 4250 can include one or more segments within shared cache memory 4248. In at least one embodiment, shared memory 4250 can include a shared pool of memory that is partitioned into segments that are allocated to different threads for reading and / or writing. Figure 42A ​within memory interface 4218. In at least one embodiment, MMU 4245 includes a set of page table entries (PTEs) for mapping virtual addresses into physical addresses at which tiles (more on tiles below) are stored and optionally into cache line indices. In at least one embodiment, MMU 4245 can include address translation lookaside buffer (TLB) or cache, which can reside within graphics multiprocessor 4234 or within LI cache 4248 or processing cluster 4294. In at least one embodiment, processing physical addresses to allocate surface data access locality for efficient request interweaving between partition units. In at least one embodiment, cache line indices can be used to determine whether a request for a cache line is a hit or miss.

[0440] In at least one embodiment, processing cluster 4294 can be configured such that each graphics multiprocessor 4234 is coupled to a texture unit 4236 for performing texture mapping operations, e.g., determining texture sample positions, reading texture data, and filtering texture data. In at least one embodiment, texture data is read from an internal texture LI cache (not shown) or from an LI cache within graphics multiprocessor 4234 as needed, and texture data is fetched from an L2 cache, local parallel processor memory, or system memory, as needed. In at least one embodiment, each graphics multiprocessor 4234 outputs processed tasks to data crossbar 4240 in order to provide processed task to another processing cluster 4294 for further processing or to store processed task in an L2 cache, local parallel processor memory, or system memory via memory crossbar 4216. In at least one embodiment, a raster operations pre-unit (Pre-ROP) 4242 is configured to receive data from graphics multiprocessor 4234, direct data to ROP unit which can be located within partition unit (e.g., partition units 4220A-4220N) as described herein, in at least one embodiment. In at least one embodiment, Pre-ROP 4242 can perform optimizations for color blending, organize pixel color data, and perform address translation. Figure 42A

[0441] In at least one embodiment, sender processor 114 and / or receiver processor 126 can each be implemented as processing cluster 4294. In such embodiments, sender memory 116 and / or receiver memory 128 can each be implemented using any memory shown in FIG. 29 and / or described with respect to Figure 42B Figure 42B

[0442] Figure 42C A graphics multiprocessor 4296, according to at least one embodiment, is shown. In at least one embodiment, graphics multiprocessor 4296 is a version of graphics multiprocessor 4234 that is described above.​​​Figure 42B graphics processor 4234. In at least one embodiment, graphics processor 4296 couples with the pipeline manager 4232 of processing cluster 4294. In at least one embodiment, graphics processor 4296 has a thread execution pipeline that includes, without limitation, an instruction cache 4252, an instruction unit 4254, an address mapping unit 4256, a register file 4258, one or more GPGPU cores 4262, and one or more LSU’s 4266. GPGPU cores 4262 and LSU’s 4266 couple with cache memory 4272 and shared memory 4270 via a memory and cache interconnect 4268.

[0443] In at least one embodiment, instruction cache 4252 receives a stream of instructions 4250 to execute from pipeline manager 4232. In at least one embodiment, instructions are cached in instruction cache 4252 and dispatched for execution by instruction unit 4254. In one embodiment, instruction unit 4254 can dispatch instructions to the various functional units available on GPGPU cores 4262 including, but not limited to, integer algebraic units, floating point units, and others. In at least one embodiment, instructions are dispatched as groups, e.g., thread groups, which are executed via a thread execution pipeline.

[0444] In at least one embodiment, register file 4258 provides a set of registers for functional units of graphics processor 4296. In at least one embodiment, register file 4258 provides temporary storage for operands of the data paths connected to the functional units (e.g., GPGPU cores 4262, LSU’s 4266) of graphics processor 4296. In at least one embodiment, register file 4258 is split between registers for each functional unit such that, for each functional unit, there is a dedicated portion of the register file 4258. In at least one embodiment, register file 4258 is split between different thread groups being executed by graphics processor 4296.

[0445] In at least one embodiment, GPGPU cores 4262 can each include FPUs and / or ALUs for executing instructions for graphics processing. GPGPU cores 4262 can be similar to each other in architecture or can differ from each other in architecture. In at least one embodiment, a first portion of GPGPU cores 4262 includes single precision FPUs and integer ALUs, while a second portion of GPGPU cores includes double precision FPUs. In at least one embodiment, FPUs can implement IEEE 754-2008 standard for floating point arithmetic or enable variable precision floating point arithmetic. In at least one embodiment, graphics processor 4200 can additionally include one or more fixed function or special purpose logic units to perform specific functions or handle specific data types. In at least one embodiment, one or more of GPGPU cores 4262 can also include fixed or special function logic.

[0446] In at least one embodiment, GPGPU cores 4262 include SIMD logic capable of

[0447] In at least one embodiment, memory and cache interconnect 4268 is an interconnect network that connects each functional unit of graphics multiprocessor 4296 to register file 4258 and shared memory 4270. In at least one embodiment, memory and cache interconnect 4268 is a crossbar interconnect that allows LSUs 4266 to implement load and store operations between shared memory 4270 and register file 4258. In at least one embodiment, register file 4258 can operate at same frequency as GPGPU cores 4262, resulting in very low latency for data transfers between GPGPU cores 4262 and register file 4258. In at least one embodiment, shared memory 4270 can be used to enable communications between threads executing on functional units within graphics multiprocessor 4296. In at least one embodiment, for example, cache memory 4272 can be used as a data cache to cache texture data communicated between the functional units and texture unit 4236. In at least one embodiment, shared memory 4270 can also be used as a program managed cache. In at least one embodiment, in addition to automatically cached data that is stored in cache memory 4272, a thread executing on a GPGPU core 4262 can store data in shared memory at programmed locations that are accessible to other threads executing on other GPGPU cores 4262.

[0448] In at least one embodiment, a parallel processor or GPGPU as described herein is communicatively coupled to a host / processor core to accelerate graphics operations, machine learning operations, pattern analysis operations, and various general purpose GPU (GPGPU) functions. In at least one embodiment, GPU can be communicatively coupled to a host processor / cores by a bus or other interconnect (e.g., a high-speed

[0449] In at least one embodiment, sender processor 114 and / or receiver processor 126 can each be implemented as a graphics multiprocessor 4296. In such embodiments, sender memory 116 and / or receiver memory 128 can each be implemented using any memory as shown in FIG. 13 and / or described with respect to Figure 42C In at least one embodiment, sender processor 114 and / or receiver processor 126 can each be implemented as a graphics multiprocessor 4296. In such embodiments, sender memory 116 and / or receiver memory 128 can each be implemented using any memory as shown in FIG. 13 and / or described with respect to Figure 42C In at least one embodiment, sender processor 114 and / or receiver processor 126 can each be implemented as a graphics multiprocessor 4296. In such embodiments, sender memory 116 and / or receiver memory 128 can each be implemented using any memory as shown in FIG. 13 and / or described with respect to

[0450] General purpose computing

[0451] The following figures set forth, without limitation, example software configurations used in general computing to implement at least one embodiment.

[0452] Figure 43 A software stack of a programming platform is shown, in accordance with at least one embodiment. In at least one embodiment, a programming platform is a platform for utilizing hardware on a computing system to accelerate computational tasks. In at least one embodiment, a software developer can access a programming platform through libraries, compiler directives, and / or extensions to a programming language. In at least one embodiment, a programming platform can be, but is not limited to, CUDA, Radeon Open Compute Platform (“ROCm”), OpenCL (OpenCL TM ), SYCL, or Intel One API.

[0453] In at least one embodiment, a software stack 4300 of a programming platform provides an execution environment for an application 4301. In at least one embodiment, an application 4301 can include any computer software capable of launching on a software stack 4300. In at least one embodiment, an application 4301 can include, without limitation, an artificial intelligence (“AI”) / machine learning (“ML”) application, a high performance computing (“HPC”) application, a virtual desktop infrastructure (“VDI”), or a data center workload.

[0454] In at least one em...

Claims

1. A communication device, comprising: A network interface for receiving multiple packets, each of the multiple packets including at least one packet identifier and at least one payload; and A packet processing circuit for detecting redundant packets and preventing at least some of the redundant packets from being stored repeatedly, comprising: Maintain the group identifier vector; For each arriving packet, the corresponding bit in the packet identifier vector is flipped, and when a bit of a given packet identifier in the packet identifier vector is set and a corresponding packet from another first-order packet arrives, the packet is discarded without further processing; and The packet identifier, along with the base address of the buffer, is used to map the payload of each packet to the corresponding address in the buffer. This ensures that when the payloads of two valid packets from different, corresponding streams in a redundant transmission scheme are mapped to the same address, regardless of whether they have the same or different packet sequence numbers (PSNs), only one of them will ultimately be written to the buffer.

2. The communication apparatus of claim 1, wherein the redundant packets are detected based on a redundant packet identifier in the at least one packet identifier or a redundant payload in the at least one payload.

3. The communication apparatus of claim 1, wherein the network interface includes a plurality of connections for receiving the plurality of packets as a plurality of packet streams on a plurality of different communication channels.

4. The communication apparatus of claim 1, wherein the plurality of packet identifiers includes the at least one packet identifier for each of the plurality of packets, and The packet processing circuit implements an anti-replay engine that prevents at least some of the redundant packets from being stored repeatedly by passing at least one unique packet, the at least one unique packet comprising at most a single packet from the plurality of packets having each unique value of the plurality of packet identifiers.

5. The communication device of claim 4, wherein the packet processing circuit implements a distributed engine. The anti-replay engine passes the at least one unique packet to the distributed engine, and The scatter engine determines the memory address of each unique packet in the at least one unique packet passed to the scatter engine.

6. The communication apparatus of claim 5, wherein each of the plurality of packets includes a header containing the at least one packet identifier. The packet processing circuit implements the packet parsing engine and the bootstrapping engine. The packet parsing engine parses each of the plurality of packets, identifies the header of each of the plurality of packets, and forwards the header of each of the plurality of packets and the at least one payload to the bootstrapping engine. The bootstrap engine identifies at least one packet identifier in the header of each of the plurality of packets and passes the at least one packet identifier and the at least one payload of each of the plurality of packets to the anti-replay engine.

7. The communication apparatus of claim 1, wherein the plurality of packet identifiers includes the at least one packet identifier for each of the plurality of packets, and The packet processing circuit implements an anti-replay engine that prevents at least some of the redundant packets from being stored repeatedly by discarding all packets except for one packet having each of the unique packet identifiers among the plurality of packet identifiers received within a sliding window.

8. The communication device of claim 7, which is used by a user, wherein the size of the sliding window can be provided by the user.

9. The communication apparatus of claim 7, wherein late packets are received outside the sliding window, and The anti-replay engine discards the late packets.

10. The communication apparatus of claim 7, wherein delayed packets are received outside the sliding window, and The anti-replay engine stores the late packets in a queue for later processing.

11. The communication apparatus of claim 1, wherein the plurality of packets comprises one or more transmitted packets and one or more discarded packets, and The packet processing circuit is used to prevent the storage of the one or more discarded packets and to determine the memory address of each of the one or more transmitted packets.

12. The communication device of claim 11, further comprising: An interface for writing each of the one or more transmitted packets to the memory address of the determined transmitted packet.

13. The communication device of claim 12, further comprising: A bus that connects the interface to a memory including the memory address of each of the determined one or more transmitted packets, the interface writing at most a single copy of each of the one or more transmitted packets to the memory address of the determined transmitted packet via the bus.

14. The communication device of claim 13, wherein the bus is a peripheral component interconnection fast PCIe bus.

15. The communication apparatus of claim 11, wherein the packet processing circuitry is configured to determine the memory address of each of the one or more transmitted packets by mapping the at least one packet identifier of the transmitted packet to the memory address.

16. The communication apparatus of claim 11, wherein the packet processing circuitry is configured to determine the memory address of each of the one or more transmitted packets by mapping the at least one packet identifier of the transmitted packets to the memory address using a linear mapping.

17. The communication apparatus of claim 11, wherein the packet processing circuitry is configured to determine the memory address of each of the one or more transmitted packets based on a function of an initial packet identifier, the at least one packet identifier of the transmitted packet, the size of the at least one payload of the transmitted packet, and a base memory address.

18. A system for transmitting and receiving packets, comprising: A packet source that transmits multiple packets, each including multiple payloads, as multiple packet streams on multiple different communication channels, wherein the multiple payloads are associated with multiple packet identifiers. as well as A receiving device, comprising circuitry, the circuitry being used to receive the plurality of packet streams; To prevent duplicate storage of at least some of the plurality of payloads, the method includes maintaining a packet identifier vector, flipping the corresponding bit in the packet identifier vector for each arriving packet, and discarding the packet without further processing when a bit of a given packet identifier in the packet identifier vector is set and a corresponding packet from another first-order arrives. Determining the memory address of each corresponding payload and writing the payload to the corresponding memory address determined by the circuitry includes using a packet identifier along with the base address of the buffer to map the payload of each packet to the corresponding address in the buffer, such that when the payloads of two valid packets from different, corresponding streams in a redundant transmission scheme are mapped to the same address, regardless of whether they have the same or different packet sequence numbers (PSNs), only one of them is ultimately written to the buffer.

19. The system of claim 18, wherein the packet source includes a virtual machine.

20. The system of claim 18, wherein the circuitry implements the anti-replay engine and the dispersion engine. The anti-replay engine prevents at least some of the payloads from being stored repeatedly by passing at most a single payload associated with each unique value of the group identifier to the distribution engine. The distribution engine determines the memory address of each payload passed to the distribution engine, thereby determining the memory address of the respective individual payloads in the payload.

21. The system of claim 18, wherein the circuitry implements an anti-replay engine that prevents at least some of the payload from being stored repeatedly by discarding all but one packet having each unique packet identifier received within a sliding window.

22. The system of claim 21, wherein late packets are received outside the sliding window, and The anti-replay engine performs at least one action: discarding the late packet or storing the late packet in a queue for subsequent processing.

23. The system of claim 18, wherein the payload comprises a plurality of delivered payloads and a plurality of discarded payloads. The circuit prevents the storage of the plurality of discarded payloads, and The circuit determines one of the memory addresses of each of the plurality of transmitted payloads by mapping the group identifier associated with the transmitted payload to one of the memory addresses.

24. The system of claim 23, wherein a linear mapping is used to map the packet identifier associated with the transmitted payload to one of the memory addresses.

25. The system of claim 18, wherein the payload comprises a plurality of delivered payloads and a plurality of discarded payloads. The circuit prevents the storage of the plurality of discarded payloads, and The circuit determines one of the memory addresses of each of the plurality of transmitted payloads based on an initial sequence number, a group identifier associated with the transmitted payload, the size of the transmitted payload, and a base memory address in the memory address set.

26. The system of claim 18, wherein the packet source includes a transmitting device having circuitry, the circuitry adding a unique packet identifier from the plurality of packet identifiers to each of the plurality of initial packets; After the unique group identifier is added to each of the plurality of initial groups, each of the plurality of initial groups is copied to create a plurality of copied groups; And transmit the plurality of initial packets and the plurality of duplicate packets as a plurality of packet streams on the plurality of different communication channels.

27. The system of claim 26, wherein at least one of the transmitting device or the receiving device is an autonomous vehicle.

28. The system of claim 27, wherein the transmitting device is an autonomous vehicle. The autonomous vehicle includes at least one sensor, and Each of the plurality of payloads includes information based on sensor data obtained from the at least one sensor.

29. The system of claim 28, wherein the sending device includes a camera for capturing video data and an interface for packaging the video data into the plurality of initial packets.

30. The system of claim 29, wherein each of the plurality of payloads comprises a slice of the video data.

31. A method for receiving packets, comprising: Receive multiple packet streams transmitted on multiple different communication channels, the multiple packet streams comprising multiple packets, the multiple packets comprising multiple payloads respectively associated with multiple packet identifiers; Identifying a copy of each of at least some of the plurality of payloads, comprising: maintaining a packet identifier vector, for each arriving packet, flipping a corresponding bit in the packet identifier vector, and discarding the packet without further processing when a bit of a given packet identifier in the packet identifier vector is set and a corresponding packet from another first-order arrives; Determining the memory address of each identified payload and writing each identified payload to the determined memory address of the identified payload includes: using a packet identifier along with the base address of the buffer to map the payload of each packet to the corresponding address in the buffer, such that when the payloads of two valid packets from different, corresponding streams in a redundant transmission scheme are mapped to the same address, regardless of whether they have the same or different packet sequence numbers (PSNs), only one of them is ultimately written to the buffer.

32. The method of claim 31, further comprising: Discard all other packets except for one packet associated with each unique packet identifier among the plurality of packet identifiers received within the sliding window.

33. The method of claim 32, wherein late packets are received outside the sliding window, and the method further comprises: Discard the late packet or store the late packet in a queue for later processing.

34. The method of claim 31, wherein determining the memory address of each identified payload comprises: The group identifier associated with each identified payload is mapped to the memory address.

35. The method of claim 34, wherein a linear mapping is used to map the group identifier associated with each identified payload to the memory address.

36. The method of claim 34, wherein the group identifier associated with each identified payload is mapped to the memory address using a function of an initial sequence number, the group identifier associated with each identified payload, the size of each identified payload, and the base memory address.

37. The method of claim 31, further comprising: Add a unique group identifier from the plurality of group identifiers to each of the plurality of initial groups; After adding the unique group identifier to each of the plurality of initial groups, each of the plurality of initial groups is copied to create a plurality of copied groups; as well as The multiple initial packets and the multiple duplicate packets are transmitted as multiple packet streams on the multiple different communication channels.

Images