Method and system for forward error correction coding using binary clustering

Abstract

Embodiments of the present disclosure are directed to binary clustered forward error correction coding schemes. Systems and methods are disclosed for defining binary clustered encodings of media packets from which forward error correction (FEC) packets are computed. Different encodings specify which media packets in a frame are used to compute each FEC packet (a frame includes M media packets). The number of media packets in a frame M≤floor(2 N ) to define different encodings, where each bit of the binary representation of N is associated with a different cluster pair encoding of the media packet. Each cluster pair includes a cluster with a bit of 0 and a cluster with a bit of 1. Using at least two cluster pair encodings to calculate the FEC packet provides redundancy for each media packet, thereby improving the media packet recovery rate.

Description

Background Art

[0001] Packet loss can affect multimedia streaming performance. Forward Error Correction (FEC) is a technique that mitigates network packet loss by recovering lost packets. Conventional XOR-based FEC techniques typically do not provide enough redundancy to recover any packet in a frame (a frame includes M media packets). Redundancy can be improved by covering more media packets in a single FEC packet and / or calculating (and transmitting) more FEC packets. Conventional techniques typically provide limited coverage when multiple lost packets belong to the same media frame. In addition, conventional techniques may simply transmit duplicate media packets to improve media packet recovery. There is a need to address and improve these problems and / or other problems associated with the prior art Summary of the invention

[0002] Embodiments of the present disclosure are directed to forward error correction coding schemes using binary clustering. The present disclosure is directed to systems and methods that define binary cluster encodings of media packets from which FEC packets are computed. Different encodings specify which media packets in a frame are used to compute each FEC packet (a frame includes M media packets). The number of media packets in a frame M≤floor(2 N ) to define different encodings, where each bit of the binary representation of N is associated with a different cluster pair encoding of the media packet. Each cluster pair includes a cluster with a bit of 0 and a cluster with a bit of 1.

[0003] For example, when M=64 and N=6, 6 bits represent the binary value associated with the media packet. For N=6, there are 6 possible cluster pairs (12 cluster codes). Each cluster code selects a group of media packets to calculate the FEC packet. The media packets of the frame are each associated with an N-bit binary value 0 to 2. N -1 (each value is associated with a single media packet). Compared to traditional XOR-based FEC techniques (such as those described above which generally do not provide sufficient redundancy to recover any packet in a frame), each particular media packet may not be covered by more than one FEC packet.

[0004] The present disclosure describes systems, methods, and computer-readable media for binary cluster coding of FEC. In one embodiment, a sequence of media packets is received, wherein for a positive integer N, the number of media packets is less than or equal to 2. N, and each media packet is associated with a different binary value specified by N bits. A first cluster of the media packets is selected, the first cluster consisting of media packets associated with a binary value for asserting a first bit, wherein the first cluster includes a first media packet in the quantity and a second media packet in the quantity. In one embodiment, the first bit can be the least significant bit, the most significant bit, or any bit in between, but is not limited to this. A second cluster of media packets is selected, the cluster consisting of media packets associated with a binary value for asserting a second bit, wherein the first media packet is included in the second cluster and the second media packet is not included in the second cluster. A first FEC packet in a plurality of FEC packets is calculated using the first cluster, and a second FEC packet in a plurality of FEC packets is calculated using the second cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The disclosed system and method for a binary clustered FEC encoding scheme is described in detail below with reference to the accompanying drawings, wherein:

[0006] Figure 1 A block diagram of an example system using a binary cluster coding scheme to generate FEC packets suitable for implementing some embodiments of the present disclosure is shown.

[0007] Figure 2A Media grouping in a frame and cluster coding pair suitable for implementing some embodiments of the present disclosure is shown.

[0008] Figure 2B , 2C 2D and 2D illustrate cluster encoding suitable for implementing some embodiments of the present disclosure.

[0009] Figure 3 A flow chart of a method for a binary clustered FEC encoding scheme according to an embodiment is shown.

[0010] Figure 4 An example parallel processing unit suitable for implementing some embodiments of the present disclosure is shown.

[0011] Figure 5A is suitable for implementing some embodiments of the present disclosure Figure 4 Conceptual diagram of the processing system implemented by the PPU.

[0012] Figure 5B Example systems are shown that may implement the various architecture and / or functionality of various previous embodiments.

[0013] Figure 5C Components of an example system that can be used to train and utilize machine learning in at least one embodiment are shown.

[0014] Figure 6An exemplary streaming system suitable for implementing some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0015] The present disclosure relates to systems and methods related to binary clustered FEC coding schemes for recovering packets lost during transmission. In particular, the binary clustered FEC coding scheme can improve packet recovery for multimedia streams compared to conventional FEC techniques. Depending on the number of FEC packets generated for a frame and which media packets in the frame are used to generate the FEC packets, the ability to recover lost (e.g., discarded) media packets varies. Each FEC packet is generated by combining multiple media packets using an exclusive-or (XOR) logical operator. Conventional XOR-based FEC techniques generally do not provide enough redundancy to recover any media packet in a frame. In some cases, when a particular media packet is only used to calculate one FEC packet, the packet has no redundancy. Redundancy can be improved by using more media packets to calculate each FEC packet, thereby using a particular media packet to calculate multiple FEC packets. Redundancy can also be improved by calculating more FEC packets so that a particular media packet is used to calculate multiple FEC packets. Conventional techniques generally cannot cover a particular media packet with more than one FEC packet. In addition, conventional techniques can simply send duplicate media packets to improve media packet recovery by sending redundant media packets.

[0016] The binary clustered FEC coding scheme selects groups (clusters) of media packets to generate FEC packets to improve redundancy and media packet recovery rates. The number of FEC packets calculated for each frame of media packets can vary based on the packet loss determined by the receiver. For example, when the uniform packet loss is 2%, the traditional FEC coding technology can recover 90.85% of the media packets, while the binary clustered FEC coding scheme can recover 96.73% of the media packets. Similarly, when the uniform packet loss is 4%, the traditional FEC coding technology can recover 85% of the media packets, while the use of the binary clustered FEC coding scheme can recover 96.57% of the media packets. In order to achieve a media packet recovery rate similar to that of the binary clustered FEC coding scheme when the packet loss rate is 2% or 4%, the traditional technology needs to calculate and transmit approximately 5% more FEC packets.

[0017] Figure 1A block diagram of an example system 100 implementing a binary clustered FEC coding scheme suitable for implementing some embodiments of the present disclosure is shown. It should be understood that this arrangement and other arrangements described herein are only set forth as examples. In addition to the arrangements and elements shown, other arrangements and elements (e.g., machines, interfaces, functions, sequences, functional groupings, etc.) may also be used, or other arrangements and elements may be used to replace the arrangements and elements shown, and some elements may be omitted completely. In addition, many of the elements described herein are functional entities that may be implemented as discrete or distributed components, or implemented in combination with other components, and may be implemented in any suitable combination and position. The various functions performed by the entities described herein may be performed by hardware, firmware, and / or software. For example, various functions may be performed by a processor that executes instructions stored in a memory. In addition, it will be understood by those of ordinary skill in the art that any system that performs the operation of the system 100 is within the scope and spirit of the embodiments of the present disclosure.

[0018] like Figure 1 As shown, the system 100 includes an FEC encoder 110, a transmission interface 130, one or more networks 135, a reception interface 140, and an XOR FEC packet decoder 145. In response to receiving a packet loss rate via a packet loss feedback 124, the FEC controller 120 dynamically determines the number of FEC packets 116 to be calculated for each frame of the media packets 105. In one embodiment, the packet loss rate can be expressed as a percentage or count of lost packets per frame. In one embodiment, the packet loss rate can be provided for a single frame, or the packet loss rate can be determined for multiple frames. In one embodiment, the number of FEC packets 116 is determined based on the packet loss rate in the frame and the number of media packets 105. In one embodiment, the number of FEC packets 116 increases in response to an increased packet loss rate and decreases in response to a decreased packet loss rate. A lookup table can be used to determine the number of FEC packets 116 required for each packet loss rate. The number of FEC packets 116 determined by the FEC controller 120 is provided to the media packet selection unit 115 and / or the XOR FEC packet encoder 125. In one embodiment, the number of packets 116 is passed from the media packet selection unit 115 to the XOR FEC packet encoder 125.

[0019] The FEC packets 126 are calculated based on (binary) cluster codes. Specifically, each cluster code is associated with a specific group (cluster) of media packets 105 that are combined by an XOR operation to calculate the FEC packets 126. The media packets 105 in the cluster are selected based on the binary value associated with each media packet. In one embodiment, the media packets 105 of each frame are assigned a binary value in a sequence that matches the order in which the media packets 105 are received starting from zero and increasing to M-1, where M is equal to the number of media packets 105 in each frame. In another embodiment, the binary values ​​are assigned to the media packets 105 of each frame in an order that is not equal to the sequence, where each media packet in the frame is assigned a different binary value between 0 and M-1 (including M-1). As combined Figure 2A As described, each cluster code is associated with a different cluster of media packets 105 .

[0020] Now, more illustrative information about various optional architectures and features that can be used to implement the above framework will be described according to the user's wishes. It should be noted that the following information is for illustration only and should not be interpreted as limiting in any way. Any of the following features can be selected to be combined with the other features described, or the other features described may not be excluded.

[0021] Figure 2A Frame 200 and media packets in clustered coding pairs 210 are shown, which are suitable for implementing some embodiments of the present disclosure. Frame 200 includes M=64 media packets 105, and a binary value is assigned to each media packet 105. Figure 2A As shown, the media packets 105 of the frame 200 may be organized in an 8×8 array, where each media packet 105 occupies a position (decimal number 0-63) associated with a column and a row. For example, the binary value assigned to the media packet 105 for position 12 is the binary value 001100. Note that the media packets 105 need not be associated with consecutive binary values ​​that match the sequence (first, second, third, fourth, etc.) in which the media packets 105 were received.

[0022] In the context of the following description, it can be based on the number of media packets in the frame, M≤floor(2 N ) and N different cluster pair codes of the media packets of each frame to define different cluster codes. The binary value assigned to each media packet 105 is represented by N bits. Each cluster pair code is associated with a different bit in the N bits, wherein the cluster pair code includes a cluster code with an associated bit of 0 and a cluster with an associated bit of 1.

[0023] For example, when M=64 and N=6, 6 bits represent a binary value associated with a media packet. For N=6, there are 6 cluster pair encodings (12 cluster encodings). Each cluster encoding selects a group (cluster) of media packets 105 for use in calculating FEC packets 126. As described above, each media packet 105 in frame 200 is associated with an N-bit binary value 0 to 2. N =1 (each value is associated with a single media packet). For N=6, three bits (N / 2) are column bits and three bits are row bits, each ranging from 0 to 7. In one embodiment, the column bits are the least significant bits of the binary value and the row bits are the most significant bits of the binary value. In another embodiment, the row bits are the least significant bits of the binary value and the column bits are the most significant bits of the binary value. In yet another embodiment, the bits of the binary value alternate between row bits and column bits. In general, each bit of the binary value is associated with either a row bit or a column bit without restriction.

[0024] Cluster code pair 210 includes cluster codes 211 and 212 and is associated with bit 1 of an N-bit binary value, which is bit 1 (center) of three columns of bits (bit 0 is the lsb). The four columns of media packets in frame 200 with bit 1 equal to 1 are colored to indicate the media packet group selected for cluster code 211. The four columns of media packets with bit 1 equal to zero are colored to indicate the media packet group selected for cluster code 212. When selecting a media packet group for cluster code pair 210, bits in the binary value that are not bit 1 are ignored. Although cluster code pair 210 does not provide redundancy when used to calculate the first and second FEC packets, because no single media packet is included in cluster codes 211 and 212. By using at least one additional cluster code to calculate the third FEC packet, redundancy can be provided for at least a portion of the media packets.

[0025] Figure 2B Cluster codes 221, 222, 231, and 232 suitable for implementing some embodiments of the present disclosure are shown. Cluster codes 221 and 222 are a cluster code pair associated with bit 0 of an N-bit binary value that is the lsb of three columns of bits. The four columns of media packets in frame 200 with bit 0 equal to 1 are colored to indicate the media packet group selected for cluster code 221. The four columns of media packets with bit 0 equal to zero are colored to indicate the media packet group selected for cluster code 222. Similar to cluster code pair 210, cluster codes 221 and 222 do not provide redundancy. However, by using cluster codes 211, 212, 221, and 222 to calculate four FEC packets, redundancy is provided for all media packets in frame 200.

[0026] Cluster codes 231 and 232 are a pair of cluster codes associated with bit 2 of an N-bit binary value, which is the MSB of the three columns of bits. The four columns of media packets in frame 200 with bit 2 equal to 1 are colored to indicate the group of media packets selected for cluster code 231. The four columns of media packets in frame 200 with bit 2 equal to zero are colored to indicate the group of media packets selected for cluster code 232. Redundancy is provided for all media packets in frame 200 by calculating four FEC packets using cluster codes 211, 212, 231, and 232. Alternatively, redundancy is provided for all media packets in frame 200 by calculating four FEC packets using cluster codes 221, 222, 231, and 232. In addition, six cluster codes 211, 212, 221, 222, 231, and 232 may be used to provide additional redundancy.

[0027] Figure 2C Cluster codes 241 and 242 suitable for implementing some embodiments of the present disclosure are shown. Cluster codes 241 and 242 are a cluster code pair associated with bit 4 of an N-bit binary value, which is bit 1 (center) of three rows of bits. The four rows of media packets in frame 200 where bit 4 is equal to 1 are colored to indicate the group of media packets selected for cluster code 241. The four rows of media packets in frame 200 where bit 4 is equal to zero are colored to indicate the group of media packets selected for cluster code 242. Similar to cluster code pair 210, cluster codes 241 and 242 do not provide redundancy. However, by using cluster codes 241 and 242 and an additional cluster code pair to calculate four FEC packets, redundancy is provided for all media packets in frame 200.

[0028] Figure 2D Cluster codes 251, 252, 261, and 262 suitable for implementing some embodiments of the present invention are shown. Cluster codes 251 and 252 are cluster code pairs associated with bit 3 of an N-bit binary value as the lsb of three rows of bits. The four rows of media packets in frame 200 with bit 3 equal to 1 are colored to indicate the group of media packets selected for cluster code 251. The four rows of media packets in frame 200 with bit 3 equal to zero are colored to indicate the group of media packets selected for cluster code 252. Similar to cluster code pair 210, cluster codes 251 and 252 do not provide redundancy. However, by using cluster codes 251 and 252 and an additional cluster code pair to calculate four FEC packets, redundancy is provided for all media packets in frame 200.

[0029] Cluster codes 261 and 262 are a cluster code pair associated with bit 5 of an N-bit binary value, which is the MSB of three rows of bits. The four rows of media packets in frame 200 with bit 5 equal to 1 are colored to indicate the group of media packets selected for cluster code 261. The four rows of media packets in frame 200 with bit 5 equal to zero are colored to indicate the group of media packets selected for cluster code 262. Redundancy is provided for all media packets in frame 200 by calculating four FEC packets using cluster codes 261 and 262 and an additional cluster code pair.

[0030] Return to Figure 1 , the media packet selection unit 115 selects a group (cluster) of media packets 114 from the media packets 105 for each cluster encoding, each of which is used for the XOR FEC packet encoder 125 to calculate the FEC packet 126 in the number of FEC packets 116. In one embodiment, the media packet selection unit 115 includes a buffer for storing at least one frame of the media packets 105. One or more cluster encodings can be determined based on the number of FEC packets 116.

[0031] In one embodiment, in addition to determining the number of FEC packets 116, FEC controller 120 also determines one or more cluster codes for selecting one or more groups of media packets to calculate each FEC packet. In one embodiment, media packet selection unit 115 determines one or more cluster codes based on the number of FEC packets 116. For example, when packet loss feedback 124 indicates a first packet loss rate, four FEC packets 126 may be calculated based on four cluster codes. When packet loss feedback 124 indicates a second packet loss rate, five FEC packets 126 may be calculated based on five cluster codes.

[0032] The cluster codes may have a fixed association with the number of FEC packets 116 (or packet losses), or may be programmable (via a lookup table). When the association is fixed, the same four cluster codes are used for the first and second packet loss rates, and a different cluster code is added for the second packet loss rate. When the association is programmable, instead of using the same four cluster codes for the first and second packet loss rates, the second packet loss may specify at least two cluster codes that are not used for the first packet loss rate.

[0033] The selected group of media packets 114 is input to an XOR FEC packet encoder 125 to generate FEC packets 126. The XOR FEC packet encoder 125 also receives the number of FEC packets 116 to be generated for each frame. The XOR FEC packet encoder 125 calculates the FEC packets for each group of media packets 114 and outputs the FEC packets 126 to the transmission interface 130. The transmission interface 130 transmits the FEC packets 126 and the media packets 105 for each frame to one or more networks 135. The receiving interface 140 outputs the received media packets 155 and the received FEC packets 146 to the XOR FEC packet decoder 145. The XOR FEC packet decoder 145 uses the combination of the received FEC packets 146 and the received media packets 155 to generate the recovered media packets 150. The binary cluster encoding scheme is compatible with the conventional transmission interface, the receiving interface and the XOR FEC packet decoder. Therefore, the FEC encoder 110 can be incorporated into an existing system without requiring any modifications to the transmit interface, receive interface, and XOR FEC packet decoder.

[0034] Figure 3 A flow chart of a method 300 for a binary clustered FEC encoding scheme according to an embodiment is shown. Each block of the method 300 described herein includes a computational process that can be performed using any combination of hardware, firmware, and / or software. For example, the various functions can be performed by a processor executing instructions stored in a memory. The method can also be implemented as computer usable instructions stored on a computer storage medium. The method can be provided by a stand-alone application, a service or a hosted service (standalone or in combination with another hosted service), or a plug-in for another product, to name a few. In addition, as an example, for Figure 1 The method 300 is described with reference to the system 100 of FIG. However, the method may be additionally or alternatively performed by any system or any combination of systems, including but not limited to the systems described herein. In addition, a person of ordinary skill in the art will understand that any system that performs the method 300 is within the scope and spirit of the embodiments of the present disclosure.

[0035] In step 305, a sequence of media packets is received, wherein for a positive integer N, the number of media packets is less than or equal to 2. N , and each media packet is associated with a different binary value specified by the N bits. In one embodiment, the binary value associated with each media packet in the number indicates the sequence in which the media packets were received. In one embodiment, the binary value assigned to each media packet in the number is independent of the sequence in which the media packets were received.

[0036] At step 310, a first cluster of media packets is selected, the cluster consisting of media packets associated with a binary value for asserting a first bit, wherein a first media packet in the number and a second media packet in the number are included in the first cluster. In one embodiment, the first bit is the most significant bit of the binary value. In one embodiment, the first bit is the least significant bit of the binary value. In one embodiment, the first bit of the binary value is neither the lsb nor the msb.

[0037] At step 315, a second cluster of media packets is selected, the cluster consisting of media packets associated with the binary value for asserting the second bit, wherein the first media packet is included in the second cluster and the second media packet is not included in the second cluster.

[0038] In step 320, a first FEC packet in the plurality of FEC packets is calculated using the first cluster. In step 325, a second FEC packet in the plurality of FEC packets is calculated using the second cluster. In one embodiment, the number of FEC packets in the plurality of FEC packets is determined based on the packet loss rate and the number. In one embodiment, the first and second FEC packets are calculated using an XOR operation. In one embodiment, the first and second FEC packets are calculated in parallel. In one embodiment, the number of media packets and the first and second FEC packets are sent to a remote device.

[0039] Binary cluster coding provides redundancy to improve media packet recovery using XOR-based FEC. Conventional techniques are often unable to cover a particular media packet with multiple FEC packets. The binary cluster FEC coding scheme selects groups or clusters of media packets to generate FEC packets. Clusters are defined to improve redundancy and media packet recovery rates. In addition, the number of FEC packets calculated for each frame of media packets can vary based on packet loss determined by the receiver.

[0040] Parallel Processing Architecture

[0041] Figure 4A parallel processing unit (PPU) 400 is shown according to an embodiment. The PPU 400 may be used to implement the system 100. The PPU 400 may be used to implement one or more of the FEC encoder 110 and the XOR FEC block decoder 145 within the system 100. In one embodiment, a processor such as the PPU 400 may be configured to implement a neural network model. The neural network model may be implemented as software instructions executed by the processor, or in other embodiments, the processor may include a matrix of hardware elements configured to process a set of inputs (e.g., electrical signals representing values) to generate a set of outputs, which may represent activations of the neural network model. In other embodiments, the neural network model may be implemented as a combination of software instructions and processing performed by a matrix of hardware elements. Implementing the neural network model may include determining a set of parameters for the neural network model by, for example, supervised or unsupervised training of the neural network model, and, or in the alternative, performing inference using the set of parameters to process a new set of inputs.

[0042] In one embodiment, PPU 400 is a multi-threaded processor implemented on one or more integrated circuit devices. PPU 400 is a latency-hiding architecture designed to process many threads in parallel. A thread (e.g., an execution thread) is an instantiation of a set of instructions configured to be executed by PPU 400. In one embodiment, PPU 400 is a graphics processing unit (GPU) configured to implement a graphics rendering pipeline for processing three-dimensional (3D) graphics data to generate two-dimensional (2D) image data for display on a display device. In other embodiments, PPU 400 can be used to perform general-purpose calculations. Although an exemplary parallel processor is provided herein for illustrative purposes, it should be specifically noted that such a processor is described only for illustrative purposes, and any processor may be used to supplement and / or replace the processor.

[0043] One or more PPUs 400 can be configured to accelerate thousands of high performance computing (HPC), data center, cloud computing, and machine learning applications. PPU 400 can be configured to accelerate numerous deep learning systems and applications for autonomous vehicles, simulations, computational graphics such as ray or path tracing, deep learning, high-precision speech, image and text recognition systems, intelligent video analysis, molecular simulations, drug discovery, disease diagnosis, weather forecasting, big data analysis, astronomy, molecular dynamics simulations, financial modeling, robotics, factory automation, real-time language translation, online search optimization, and personalized user recommendations, among others.

[0044] like Figure 4As shown, the PPU 400 includes an input / output (I / O) unit 405, a front-end unit 415, a scheduler unit 420, a work distribution unit 425, a hub 430, a crossbar switch (Xbar) 470, one or more general processing clusters (GPCs) 450, and one or more memory partition units 480. The PPU 400 can be connected to a host processor or other PPUs 400 via one or more high-speed NVLink 410 interconnects. The PPU 400 can be connected to a host processor or other peripheral devices via an interconnect 402. The PPU 400 can also be connected to a local memory 404 including multiple memory devices. In one embodiment, the local memory can include multiple dynamic random access memory (DRAM) devices. The DRAM device can be configured as a high bandwidth memory (HBM) subsystem, in which multiple DRAM dies are stacked within each device.

[0045] The NVLink 410 interconnect enables the system to scale and include one or more PPUs 400 in conjunction with one or more CPUs, supporting cache coherency between the PPU 400 and the CPU, and CPU mastering. Data and / or commands may be sent by the NVLink 410 through the hub 430 to or from other units of the PPU 400, such as one or more copy engines, video encoders, video decoders, power management units, etc. (not explicitly shown). Figure 5B NVLink 410 is described in more detail.

[0046] I / O unit 405 is configured to send and receive communications (e.g., commands, data, etc.) from a host processor (not shown) via interconnect 402. I / O unit 405 may communicate with the host processor directly via interconnect 402, or through one or more intermediate devices such as a memory bridge. In one embodiment, I / O unit 405 may communicate with one or more other processors such as one or more PPUs 400 via interconnect 402. In one embodiment, I / O unit 405 implements a peripheral component interconnect express (PCIe) interface for communicating over a PCIe bus, and interconnect 402 is a PCIe bus. In alternative embodiments, I / O unit 405 may implement other types of known interfaces for communicating with external devices.

[0047] The I / O unit 405 decodes packets received via the interconnect 402. In one embodiment, the packets represent commands configured to cause the PPU 400 to perform various operations. The I / O unit 405 transmits the decoded commands to various other units of the PPU 400 that the commands may specify. For example, some commands may be transmitted to the front end unit 415. Other commands may be transmitted to the hub 430 or other units of the PPU 400, such as one or more copy engines, video encoders, video decoders, power management units, etc. (not explicitly shown). In other words, the I / O unit 405 is configured to route communications between and among the various logical units of the PPU 400.

[0048] In one embodiment, a program executed by a host processor encodes a command stream in a buffer that provides a workload to the PPU 400 for processing. The workload may include a number of instructions and data to be processed by those instructions. A buffer is an area in memory that is accessible (e.g., read / write) by both the host processor and the PPU 400. For example, the I / O unit 405 may be configured to access a buffer in a system memory connected to the interconnect 402 via a memory request transmitted through the interconnect 402. In one embodiment, the host processor writes a command stream to the buffer and then transmits a pointer to the start of the command stream to the PPU 400. The front end unit 415 receives a pointer to one or more command streams. The front end unit 415 manages the one or more streams, reads commands from these streams, and forwards the commands to the various units of the PPU 400.

[0049] The front end unit 415 is coupled to a scheduler unit 420, which configures the various GPCs 450 to process the tasks defined by the one or more streams. The scheduler unit 420 is configured to track state information related to the various tasks managed by the scheduler unit 420. The state may indicate which GPC 450 the task is assigned to, whether the task is active or inactive, a priority associated with the task, etc. The scheduler unit 420 manages the execution of multiple tasks on the one or more GPCs 450.

[0050] Scheduler unit 420 is coupled to work distribution unit 425, which is configured to dispatch tasks for execution on GPCs 450. Work distribution unit 425 may keep track of a number of scheduled tasks received from scheduler unit 420. In one embodiment, work distribution unit 425 manages a pool of pending tasks and a pool of active tasks for each GPC 450. When a GPC 450 completes execution of a task, the task is evicted from the pool of active tasks for GPC 450, and one of the other tasks from the pool of pending tasks is selected and scheduled for execution on GPC 450. If an active task on a GPC 450 has become idle, such as while waiting for a data dependency to be resolved, then the active task may be evicted from GPC 450 and returned to the pool of pending tasks, while another task in the pool of pending tasks is selected and scheduled for execution on GPC 450.

[0051] In one embodiment, the host processor executes a driver kernel that implements an application programming interface (API) that enables one or more applications executed on the host processor to schedule operations for execution on the PPU 400. In one embodiment, multiple computing applications are executed simultaneously by the PPU 400, and the PPU 400 provides isolation, quality of service (QoS), and independent address spaces for the multiple computing applications. The application can generate instructions (e.g., API calls) that cause the driver kernel to generate one or more tasks to be executed by the PPU 400. The driver kernel outputs the tasks to one or more streams being processed by the PPU 400. Each task can include one or more related thread groups, referred to herein as warps. In one embodiment, a warp includes 32 related threads that can be executed in parallel. A cooperative thread can refer to multiple threads that include instructions to execute tasks and can exchange data through a shared memory. These tasks can be assigned to one or more processing units within the GPC 450, and the instructions are scheduled for execution by at least one warp.

[0052] Work distribution unit 425 communicates with the one or more GPCs 450 via XBar 470. XBar 470 is an interconnect network that couples many units of PPU 400 to other units of PPU 400. For example, XBar 470 may be configured to couple work distribution unit 425 to a particular GPC 450. Although not explicitly shown, one or more other units of PPU 400 may also be connected to XBar 470 via hub 430.

[0053] Tasks are managed by a scheduler unit 420 and dispatched to GPCs 450 by a work distribution unit 425. GPCs 450 are configured to process tasks and generate results. The results can be consumed by other tasks within the GPC 450, routed to different GPCs 450 via XBar 470, or stored in memory 404. The results can be written to memory 404 via a memory partition unit 480, which implements a memory interface for reading data from and writing data to memory 404. The results can be transmitted to another PPU 400 or CPU via NVLink 410. In one embodiment, the PPU 400 includes a number U of memory partition units 480, which is equal to the number of separate and distinct memory devices coupled to the memory 404 of the PPU 400. Each GPC 450 may include a memory management unit to provide virtual address to physical address translation, memory protection, and arbitration of memory requests. In one embodiment, the memory management unit provides one or more translation lookaside buffers (TLBs) for performing virtual address to physical address translation in memory 404 .

[0054] In one embodiment, the memory partition unit 480 includes a raster operation (ROP) unit, a second level (L2) cache, and a memory interface coupled to the memory 404. The memory interface can implement a 32-bit, 64-bit, 128-bit, 1024-bit data bus for high-speed data transfer. The PPU 400 can be connected to up to Y memory devices, such as a high bandwidth memory stack or a graphics double data rate, version 5, synchronous dynamic random access memory or other types of persistent storage. In one embodiment, the memory interface implements an HBM2 memory interface and Y is equal to half of U. In one embodiment, the HBM2 memory stack is located on the same physical package as the PPU 400, providing a large amount of power and area savings compared to conventional GDDR5 SDRAM systems. In one embodiment, each HBM2 stack includes four memory dies and Y is equal to 4, wherein each HBM2 stack includes two 129-bit channels per die, a total of 8 channels and a data bus width of 1024 bits.

[0055] In one embodiment, memory 404 supports single error correction double error detection (SECDED) error correction code (ECC) to protect data. ECC provides higher reliability for computing applications that are sensitive to data corruption. Reliability is particularly important in large-scale cluster computing environments where PPU 400 processes very large data sets and / or runs applications for extended periods of time.

[0056] In one embodiment, the PPU 400 implements a multi-level memory hierarchy. In one embodiment, the memory partition unit 480 supports unified memory that provides a single unified virtual address space for the CPU and PPU 400 memory, allowing data sharing between virtual memory systems. In one embodiment, the frequency with which the PPU 400 accesses memory located on other processors is tracked to ensure that memory pages are moved to the physical memory of the PPU 400 that is accessing these pages more frequently. In one embodiment, the NVLink 410 supports address translation services, allowing the PPU 400 to directly access the CPU's page tables and providing the PPU 400 with full access to the CPU's memory.

[0057] In one embodiment, the copy engine transfers data between multiple PPUs 400 or between a PPU 400 and a CPU. The copy engine can generate page faults for addresses that are not mapped in the page table. The memory partition unit 480 can then repair the page faults, map these addresses into the page table, and then the copy engine can perform the transfer. In conventional systems, memory is fixed (e.g., non-pageable) for multiple copy engine operations between multiple processors, greatly reducing the amount of available storage. In the event of a hardware page fault, the address can be passed to the copy engine without worrying about whether the memory page is resident, and the copy process is transparent.

[0058] Data from memory 404 or other system memory can be retrieved by memory partition unit 480 and stored in L2 cache 460 located on-chip and shared between various GPCs 450. As shown, each memory partition unit 480 includes a portion of the L2 cache associated with the corresponding memory 404. Then, low-level caches can be implemented in various units within GPC 450. For example, each of the processing units within GPC 450 can implement a level 1 (L1) cache. The L1 cache is a private memory dedicated to a specific processing unit. The L2 cache 460 is coupled to the memory interface 470 and the XBar 470, and data from the L2 cache can be retrieved and stored in each of the L1 caches for processing.

[0059] In one embodiment, the processing unit within each GPC 450 implements a SIMD (single instruction multiple data) architecture, wherein each thread in a thread group (e.g., a warp) is configured to process a different set of data based on the same set of instructions. All threads in a thread group execute the same instructions. In another embodiment, the processing unit implements a SIMT (single instruction multiple thread) architecture, wherein each thread in a thread group is configured to process a different set of data based on the same set of instructions, but wherein individual threads in a thread group are allowed to diverge during execution. In one embodiment, a program counter, call stack, and execution state are maintained for each warp, allowing concurrency between warps and serial execution within a warp when threads within the warp diverge. In another embodiment, a program counter, call stack, and execution state are maintained for each individual thread, allowing equal concurrency between all threads within and between warps. When execution state is maintained for each individual thread, threads executing the same instruction can be aggregated and executed in parallel to achieve maximum efficiency.

[0060] Cooperative Groups is a programming model for organizing groups of communicating threads that allows developers to express the granularity at which threads are communicating, allowing for richer and more efficient expressions of parallel decompositions. The cooperative launch API supports synchronization between thread blocks used to execute parallel algorithms. Conventional programming models provide a single simple construct for synchronizing cooperative threads: a barrier across all threads of a thread block (e.g., the syncthreads() function). However, programmers often prefer to define thread groups smaller than the granularity of a thread block in the form of a collective group-wide function interface, and synchronize within the defined group to allow for greater performance, design flexibility, and software reuse.

[0061] Cooperative Groups enables programmers to explicitly define groups of threads (as small as a single thread) at sub-block and multi-block granularity, and perform collective operations such as synchronization on threads in cooperative groups. The programming model supports clean composition across software boundaries, allowing libraries and utility functions to safely synchronize within their local context without having to make assumptions about aggregation. Cooperative Groups primitives allow the implementation of new cooperative parallelism patterns, including producer-consumer parallelism, opportunistic parallelism, and global synchronization across an entire grid of thread blocks.

[0062] Each processing unit includes a large number (e.g., 128, etc.) of different processing cores (e.g., functional units) that can be fully pipelined, single-precision, double-precision, and / or mixed-precision, and include a floating-point arithmetic logic unit and an integer arithmetic logic unit. In one embodiment, the floating-point arithmetic logic unit implements the IEEE 754-2008 standard for floating-point arithmetic. In one embodiment, the cores include 64 single-precision (32-bit) floating-point cores, 64 integer cores, 32 double-precision (64-bit) floating-point cores, and 8 tensor cores.

[0063] The tensor cores are configured to perform matrix operations. In particular, the tensor cores are configured to perform deep learning matrix arithmetic, such as GEMM (matrix-matrix multiplication), for convolution operations during neural network training and inference. In one embodiment, each tensor core operates on a 4x4 matrix and performs matrix multiplication and accumulation operations, D = A × B + C, where A, B, C, and D are 4x4 matrices.

[0064] In one embodiment, the matrix multiplication inputs A and B can be integer, fixed-point or floating-point matrices, and the accumulation matrices C and D can be integer, fixed-point or floating-point matrices of equal or higher bit width. In one embodiment, the tensor core uses 32-bit integer accumulation to operate on 1-bit, 4-bit or 8-bit integer input data. 8-bit integer matrix multiplication requires 1024 operations and results in full-precision products, which are then accumulated using 32-bit integer additions with other intermediate products for 8x8x16 matrix multiplication. In one embodiment, the tensor core uses 32-bit floating-point accumulation to operate on 16-bit floating-point input data. 16-bit floating-point multiplication requires 64 operations and results in full-precision products, which are then accumulated using 32-bit floating-point additions with other intermediate products for 4x4x4 matrix multiplication. In practice, the tensor core is used to perform much larger two-dimensional or higher-dimensional matrix operations composed of these smaller elements. APIs such as the CUDA 9 C++ API expose specialized matrix load, matrix multiply and accumulate, and matrix store operations for efficient use with tensor cores from CUDA-C++ programs. At the CUDA level, the warp-level interface takes a 16x16 size matrix across all 32 threads of the warp.

[0065] Each processing unit may also include M special function units (SFUs) that perform special functions (e.g., attribute evaluation, inverse square root, etc.). In one embodiment, the SFU may include a tree traversal unit configured to traverse a hierarchical tree data structure. In one embodiment, the SFU may include a texture unit configured to perform texture map filtering operations. In one embodiment, the texture unit is configured to load texture maps (e.g., 2D texture element arrays) from memory 404 and sample these texture maps to generate sampled texture values ​​for shader programs executed by the processing unit. In one embodiment, the texture map is stored in a shared memory that may include or contain an L1 cache. The texture unit uses multi-level textures (e.g., texture maps with different levels of detail) to implement texture operations such as filtering operations. In one embodiment, each processing unit includes two texture units.

[0066] Each processing unit also includes N load-store units (LSUs) that implement load and store operations between the shared memory and the register file. Each processing unit includes an interconnect network that connects each of the cores to the register file and connects the LSUs to the register file and the shared memory. In one embodiment, the interconnect network is a crossbar switch that can be configured to connect any of the cores to any register in the register file and connect the LSUs to the register file and memory locations in the shared memory.

[0067] Shared memory is an on-chip memory array that allows data storage and communication between processing units and threads within a processing unit. In one embodiment, the shared memory includes 128KB of storage capacity and is on the path from each of the processing units to the memory partition unit 480. The shared memory can be used for cache reads and writes. One or more of the shared memory, L1 cache, L2 cache, and memory 404 are backup warehouses.

[0068] Combining data cache and shared memory functionality into a single memory block provides the best overall performance for both types of memory access. The capacity can be used as a cache by programs that do not use the shared memory. For example, if the shared memory is configured to use half of the capacity, then texture and load / store operations can use the remaining capacity. Integration into shared memory enables the shared memory to be used as a high throughput conduit for streaming data while providing high bandwidth and low latency access to frequently reused data.

[0069] When configured for general parallel computing, a simpler configuration can be used than for graphics computing. In particular, the fixed-function graphics processing unit is bypassed to create a much simpler programming model. In this general parallel computing configuration, the work distribution unit 425 dispatches and distributes thread blocks directly to the processing units within the GPC 450. The threads execute the same program using a unique thread ID in the calculation to ensure that each thread generates a unique result using the processing unit that executes the program and performs the calculation, the shared memory that communicates between the threads, and the LSU that reads and writes the global memory through the shared memory and the memory partition unit 480. When configured for general parallel computing, the processing unit can also write commands, and the scheduler unit 420 can use the commands to start new work on the processing unit.

[0070] Each of the PPUs 400 may include one or more processing cores and / or components thereof such as a tensor core (TC), a tensor processing unit (TPU), a pixel vision core (PVC), a ray tracing (RT) core, a vision processing unit (VPU), a graphics processing cluster (GPC), a texture processing cluster (TPC), a streaming multiprocessor (SM), a tree traversal unit (TTU), an artificial intelligence accelerator (AIA), a deep learning accelerator (DLA), an arithmetic logic unit (ALU), an application specific integrated circuit (ASIC), a floating point unit (FPU), an input / output (I / O) element, a peripheral component interconnect (PCI) or a peripheral component interconnect express (PCIe) element, and / or the like, and / or be configured to perform its functions.

[0071] The PPU 400 may be included in a desktop computer, a laptop computer, a tablet computer, a server, a supercomputer, a smartphone (e.g., a wireless, handheld device), a personal digital assistant (PDA), a digital camera, a vehicle, a head-mounted display, a handheld electronic device, etc. In one embodiment, the PPU 400 is implemented on a single semiconductor substrate. In another embodiment, the PPU 400 is included in a system on a chip (SoC) along with one or more other devices such as additional PPUs 400, memory 404, a reduced instruction set computer (RISC) CPU, a memory management unit (MMU), a digital-to-analog converter (DAC), etc.

[0072] In one embodiment, PPU 400 may be included on a graphics card including one or more memory devices. The graphics card may be configured to interface with a PCIe slot on a motherboard of a desktop computer. In yet another embodiment, PPU 400 may be an integrated graphics processing unit (iGPU) or a parallel processor included in a chipset of a motherboard. In yet another embodiment, PPU 400 may be implemented in reconfigurable hardware. In yet another embodiment, portions of PPU 400 may be implemented in reconfigurable hardware.

[0073] Exemplary Computing System

[0074] As developers expose and exploit more parallelism in applications such as artificial intelligence computing, systems with multiple GPUs and CPUs are used in a wide range of industries. High-performance GPU-accelerated systems with tens to thousands of computing nodes are deployed in data centers, research facilities, and supercomputers to solve increasingly large problems. As the number of processing devices within high-performance systems increases, communication and data transmission mechanisms need to scale to support the increased bandwidth.

[0075] Figure 5A For use according to one embodiment Figure 4 A conceptual diagram of a processing system 500 implemented by a PPU 400 of FIG. 5 . The exemplary system 500 may be configured to implement Figure 3 The processing system 500 includes a CPU 530 , a switch 510 , and a plurality of PPUs 400 and respective memories 404 .

[0076] NVLink 410 provides a high-speed communication link between each of the PPUs 400. Figure 5B 402 connections, but the number of connections to each PPU 400 and CPU 530 may vary. Switch 510 forms an interface between interconnect 402 and CPU 530. PPU 400, memory 404, and NVLink 410 may be located on a single semiconductor platform to form a parallel processing module 525. In one embodiment, switch 510 supports two or more protocols to interface between various connections and / or links.

[0077] In another embodiment (not shown), NVLink 410 provides one or more high-speed communication links between each PPU 400 and CPU 530, and switch 510 forms an interface between interconnect 402 and each PPU 400. PPU 400, memory 404, and interconnect 402 may be located on a single semiconductor platform to form a parallel processing module 525. In yet another embodiment (not shown), interconnect 402 provides one or more communication links between each PPU 400 and CPU 530, and switch 510 forms an interface between each PPU 400 using NVLink 410 to provide one or more high-speed communication links between PPUs 400. In another embodiment (not shown), NVLink 410 provides one or more high-speed communication links between PPU 400 and CPU 530 through switch 510. In yet another embodiment (not shown), interconnect 402 directly provides one or more communication links between each PPU 400. One or more of the NVLink 410 high-speed communication links may be implemented as a physical NVLink interconnect or an on-chip or on-die interconnect using the same protocol as NVLink 410.

[0078] In the context of this specification, a single semiconductor platform may refer to a single semiconductor-based integrated circuit manufactured on a bare die or chip. It should be noted that the term single semiconductor platform may also refer to a multi-chip module with increased connectivity, which simulates on-chip operation and makes significant improvements over conventional bus implementations. Of course, the various different circuits or devices may also be located separately in the semiconductor platform or in various combinations thereof as the user wishes. Alternatively, the parallel processing module 525 may be implemented as a circuit board substrate, and each PPU 400 and / or memory 404 may be a packaged device. In one embodiment, the CPU 530, the switch 510, and the parallel processing module 525 are located on a single semiconductor platform.

[0079] In one embodiment, the signaling rate of each NVLink 410 is 20-25 Gbit / s, and each PPU 400 includes six NVLink 410 interfaces (e.g., Figure 5A As shown in FIG. 4 , each PPU 400 includes five NVLink 410 interfaces. Each NVLink 410 provides a data transfer rate of 25 GBytes / sec in each direction, and six links provide 400 GBytes / sec. NVLink 410 can be configured as follows: Figure 5A Shown is used exclusively for PPU-to-PPU communications, or for some combination of PPU-to-PPU and PPU-to-CPU when the CPU 530 also includes one or more NVLink 410 interfaces.

[0080] In one embodiment, NVLink 410 allows direct load / store / atomic access from CPU 530 to memory 404 of each PPU 400. In one embodiment, NVLink 410 supports coherency operations, allowing data read from memory 404 to be stored in the cache hierarchy of CPU 530, reducing cache access latency of CPU 530. In one embodiment, NVLink 410 includes support for address translation services (ATS), allowing PPU 400 to directly access page tables within CPU 530. One or more of NVLink 410 may also be configured to operate in a low power mode.

[0081] Figure 5B An exemplary system 565 is shown in which various architectures and / or functions of various previous embodiments can be implemented. The exemplary system 565 can be configured to implement Figure 3 The method 300 is shown in FIG.

[0082] As shown, a system 565 is provided, which includes at least one central processing unit 530 connected to a communication bus 575. The communication bus 575 can directly or indirectly couple one or more of the following devices: main memory 540, network interface 535, CPU 530, display device 545, input device 560, switch 510 and parallel processing system 525. The communication bus 575 can be implemented using any appropriate protocol and can represent one or more links or buses, such as an address bus, a data bus, a control bus or a combination thereof. The communication bus 575 can include one or more bus or link types, such as an industry standard architecture (ISA) bus, an extended industry standard architecture (EISA) bus, a video electronics standard association (VESA) bus, a peripheral component interconnect (PCI) bus, a peripheral component interconnect high speed (PCIe) bus, a super transport and / or another type of bus or link. In some embodiments, there is a direct connection between components. As an example, the CPU 530 can be directly connected to the main memory 540. In addition, the CPU 530 can be directly connected to the parallel processing system 525. Where there is a direct or point-to-point connection between components, communication bus 575 may include a PCIe link to implement the connection. In these examples, a PCI bus need not be included in system 565.

[0083] Although the use of lines Figure 5BThe different blocks of are shown as being connected via communication bus 575, but this is not intended to be limiting and is provided for clarity only. For example, in some embodiments, presentation components such as display device 545 may be considered I / O components, such as input device 560 (e.g., if the display is a touch screen). As another example, CPU 530 and / or parallel processing system 525 may include memory (e.g., main memory 540 may represent storage devices in addition to parallel processing system 525, CPU 530, and / or other components). In other words, Figure 5B The computing devices described herein are merely illustrative. No distinction is made between categories such as "workstations," "servers," "laptops," "desktops," "tablets," "client devices," "mobile devices," "handheld devices," "game consoles," "electronic control units (ECUs)," "virtual reality systems," and / or other device or system types, as all are expected to be in the Figure 5B within the range of computing devices.

[0084] The system 565 also includes a main memory 540. Control logic (software) and data are stored in the main memory 540, which can take the form of a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by the system 565. Computer-readable media can include volatile and non-volatile media and removable and non-removable media. By way of example and not limitation, computer-readable media can include computer storage media and communication media.

[0085] Computer storage media may include volatile and nonvolatile media and / or removable and non-removable media implemented in any method or technology for storing information such as computer readable instructions, data structures, program modules, and / or other data types. For example, main memory 540 may store computer readable instructions (e.g., representing programs and / or program elements) such as an operating system. Computer storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by system 565. As used herein, computer storage media does not include the signals themselves.

[0086] Computer storage media may contain computer readable instructions, data structures, program modules, or other data types in a modulated data signal such as a carrier wave or other transport mechanism, and include any information delivery media. The term "modulated data signal" may refer to a signal that has one or more of its characteristics set or changed in such a way that information is encoded into the signal. By way of example and not limitation, computer storage media may include wired media such as a wired network or a direct wired connection, and wireless media such as sound, RF, infrared, and other wireless media. Any combination of the above should also be included within the scope of computer readable media.

[0087] When executed, the computer program enables the system 565 to perform various functions. The CPU 530 may be configured to execute at least some of the computer-readable instructions to control one or more components of the system 565 to perform one or more of the methods and / or processes described herein. Each of the CPUs 530 may include one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of simultaneously processing numerous software threads. Depending on the type of system 565 implemented, the CPU 530 may include any type of processor, and may include different types of processors (e.g., processors with fewer cores for mobile devices, and processors with more cores for servers). For example, depending on the type of system 565, the processor may be an Advanced RISC Machine (ARM) processor implemented using Reduced Instruction Set Computing (RISC) or an x86 processor implemented using Complex Instruction Set Computing (CISC). In addition to one or more microprocessors or supplementary coprocessors such as mathematical coprocessors, the system 565 may include one or more CPUs 530.

[0088] In addition to or alternatively to the CPU 530, the parallel processing module 525 can be configured to execute at least some of the computer-readable instructions to control one or more components of the system 565 to perform one or more of the methods and / or processes described herein. The parallel processing module 525 can be used by the system 565 to render graphics (e.g., 3D graphics) or perform general-purpose computing. For example, the parallel processing module 525 can be used for general-purpose computing on a GPU (GPGPU). In an embodiment, the CPU 530 and / or the parallel processing module 525 can perform any combination of the methods, processes, and / or portions thereof separately or jointly.

[0089] System 565 also includes input device 560, parallel processing system 525, and display device 545. Display device 545 may include a display (e.g., a monitor, a touch screen, a television screen, a head-up display (HUD), other display types, or a combination thereof), speakers, and / or other presentation components. Display device 545 may receive data from other components (e.g., parallel processing system 525, CPU 530, etc.) and output the data (e.g., images, video, sound, etc.).

[0090] The network interface 535 can enable the system 565 to be logically coupled to other devices, including input devices 560, display devices 545, and / or other components, some of which can be embedded (e.g., integrated) into the system 565. Illustrative input devices 560 include microphones, mice, keyboards, joysticks, game pads, game controllers, satellite dishes, scanners, printers, wireless devices, and the like. The input device 560 can provide a natural user interface (NUI) that processes air gestures, voice, or other physiological inputs generated by the user. In some instances, the input can be transmitted to an appropriate network element for further processing. The NUI can implement voice recognition, stylus recognition, facial recognition, biometric recognition, gesture recognition on and adjacent to the screen, air gestures, head and eye tracking, and touch recognition associated with the display of the system 565 (as described in more detail below). The system 565 may include a depth camera for gesture detection and recognition, such as a stereo camera system, an infrared camera system, an RGB camera system, touch screen technology, and combinations thereof. Additionally, system 565 may include an accelerometer or gyroscope that enables motion detection (e.g., as part of an inertial measurement unit (IMU)). In some examples, the output of the accelerometer or gyroscope may be used by system 565 to render immersive augmented reality or virtual reality.

[0091] In addition, the system 565 can be coupled to a network (e.g., a telecommunications network, a local area network (LAN), a wireless network, a wide area network (WAN) such as the Internet, a peer-to-peer network, a cable network, etc.) for communication purposes through the network interface 535. The system 565 can be included in a distributed network and / or cloud computing environment.

[0092] The network interface 535 may include one or more receivers, transmitters, and / or transceivers that enable the system 565 to communicate with other computing devices via an electronic communication network including wired and / or wireless communications. The network interface 535 may be implemented as a network interface controller (NIC) that includes one or more data processing units (DPUs) to perform operations such as, for example, but not limited to, packet parsing and accelerated network processing and communications. The network interface 535 may include components and functionality that allow for communication over any of a number of different networks, such as wireless networks (e.g., Wi-Fi, Z-wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., communications over Ethernet or InfiniBand), low power wide area networks (e.g., LoRaWAN, SigFox, etc.), and / or the Internet.

[0093] The system 565 may also include an auxiliary storage device (not shown). The auxiliary storage device includes, for example, a hard disk drive and / or a removable storage drive representing a floppy disk drive, a tape drive, a compact disk drive, a digital versatile disk (DVD) drive, a recording device, a universal serial bus (USB) flash memory. The removable storage drive reads from and / or writes to the removable storage unit in a well-known manner. The system 565 may also include a hard-wired power supply, a battery power supply, or a combination thereof (not shown). The power supply can supply power to the system 565 to enable the components of the system 565 to operate.

[0094] Each of the aforementioned modules and / or devices may even be located on a single semiconductor platform to form system 565. Alternatively, the various modules may also be placed individually or in various combinations of semiconductor platforms as desired by the user. Although various embodiments are described above, it should be understood that they are given by way of example only and not by way of limitation. Therefore, the breadth and scope of the preferred embodiments should not be limited to any exemplary embodiment described above, but should be limited only in accordance with the following claims and their equivalents.

[0095] Example network environment

[0096] A network environment suitable for implementing embodiments of the present disclosure may include one or more client devices, servers, network attached storage (NAS), other backend devices, and / or other device types. The client devices, servers, and / or other device types (e.g., each device) may be configured to communicate with the network. Figure 5A The processing system 500 and / or Figure 5B The exemplary system 565 may be implemented on one or more instances of the exemplary system 565 , for example, each device may include similar components, features, and / or functionality of the processing system 500 and / or the exemplary system 565 .

[0097] The components of the network environment can communicate with each other via a network, which can be wired, wireless, or both. The network can include multiple networks or networks of networks. For example, the network can include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks, such as the Internet, and / or a public switched telephone network (PSTN), and / or one or more private networks. In the case where the network includes a wireless telecommunications network, components such as base stations, communication towers, or even access points (and other components) can provide wireless connectivity.

[0098] Compatible network environments may include one or more peer-to-peer network environments, in which case the server may not be included in the network environment, and one or more client-server network environments, in which case one or more servers may be included in the network environment. In a peer-to-peer network environment, the functionality described herein with respect to the server may be implemented on any number of client devices.

[0099] In at least one embodiment, the network environment may include one or more cloud-based network environments, distributed computing environments, combinations thereof, and the like. A cloud-based network environment may include a framework layer, a job scheduler, a resource manager, and a distributed file system implemented on one or more servers, which may include one or more core network servers and / or edge servers. The framework layer may include a framework of one or more applications of the software supporting the software layer and / or the application layer. The software or application may include web-based service software or application programs, respectively. In an embodiment, one or more of the client devices may use web-based service software or application programs (e.g., by accessing the service software and / or application programs via one or more application programming interfaces (APIs)). The framework layer may be, but is not limited to, a free open source software web application framework type that may, for example, use a distributed file system for large-scale data processing (e.g., "big data").

[0100] A cloud-based network environment can provide cloud computing and / or cloud storage that implements the computing and / or data storage functions (or one or more parts thereof) described herein. Any of these different functions can be distributed in multiple locations from a central or core server (e.g., a central or core server in one or more data centers, which can be distributed across states, regions, countries, the world, etc.). If the connection to a user (e.g., a client device) is relatively close to an edge server, the core server can assign at least a portion of the functions to the edge server. A cloud-based network environment can be private (e.g., limited to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment).

[0101] Client devices may include Figure 5A The example processing system 500 and / or Figure 5B At least some of the components, features, and functions of the exemplary system 565 of the present invention. By way of example and not limitation, the client device may be implemented as a personal computer (PC), a laptop computer, a mobile device, a smart phone, a tablet computer, a smart watch, a wearable computer, a personal digital assistant (PDA), an MP3 player, a virtual reality headset, a global positioning system (GPS) or device, a video player, a video camera, a surveillance device or system, a vehicle, a ship, a spacecraft, a virtual machine, a drone, a robot, a handheld communication device, a hospital device, a gaming device or system, an entertainment system, a vehicle computer system, an embedded system controller, a remote control, an appliance, a consumer electronic device, a workstation, an edge device, any combination of these delineated devices, or any other suitable device.

[0102] Machine Learning

[0103] Deep Neural Networks (DNNs) developed on processors such as the PPU 400 have been used in a wide variety of use cases ranging from self-driving cars to faster drug development, from automatic image captioning in online image databases to intelligent real-time language translation in video chat applications. Deep learning is a technique that models the neural learning process of the human brain, which continuously learns, continuously gets smarter, and delivers more accurate results faster over time. A child is initially taught by an adult to correctly identify and classify a variety of different shapes, and eventually is able to identify shapes without any guidance. Similarly, a deep learning or neural learning system needs to be trained in object recognition and classification so that it becomes smarter and more efficient in identifying basic objects, occluded objects, and so on, while also giving context to the objects.

[0104] At the simplest level, neurons in the human brain look at various inputs received, importance levels are assigned to each of these inputs, and outputs are passed to other neurons to react. An artificial neuron, or perceptron, is the most basic model of a neural network. In one example, a perceptron can receive one or more inputs representing various features of the object that the perceptron is being trained to recognize and classify, and each of these features is assigned a certain weight based on the importance of that feature in defining the shape of the object.

[0105] Deep neural network (DNN) models include multiple layers of many connected nodes (e.g., perceptrons, Boltzmann machines, radial basis functions, convolutional layers, etc.) that can be trained with huge amounts of input data to quickly solve complex problems with high accuracy. In one example, the first layer of a DNN model breaks down an input image of a car into different segments and looks for basic patterns such as lines and angles. The second layer assembles these lines to look for higher-level patterns such as wheels, windshields, and mirrors. The next layer identifies the type of vehicle, and the final layers generate tags for the input image, identifying the model of a specific car brand.

[0106] Once trained, a DNN can be deployed and used to identify and classify objects or patterns in a process called inference. Examples of inference (the process by which a DNN extracts useful information from a given input) include identifying handwritten digits on a check deposited into an ATM, identifying images of friends in photos, delivering movie recommendations to over 50 million users, identifying and classifying different types of cars, pedestrians, and road hazards in self-driving cars, or translating human language in real time.

[0107] During training, data flows through the DNN in a forward propagation phase until a prediction is produced indicating a label that corresponds to the input. If the neural network does not label the input correctly, the error between the correct label and the predicted label is analyzed, and the weights are adjusted for each feature during a backpropagation phase until the DNN correctly labels the input as well as other inputs in the training data set. Training complex neural networks requires a large amount of parallel computing performance, including floating-point multiplication and addition supported by the PPU400. Inference is less computationally intensive than training and is a latency-sensitive process in which the trained neural network is applied to new inputs it has not seen before in order to classify images, detect emotions, identify recommendations, recognize and translate languages, and generally reason about new information.

[0108] Neural networks rely heavily on matrix math operations, and complex, multi-layer networks require massive amounts of floating-point performance and bandwidth for both efficiency and speed. With thousands of processing cores optimized for matrix math operations and delivering tens to hundreds of TFLOPS of performance, the PPU 400 is a computing platform capable of delivering the performance required for deep neural network-based artificial intelligence and machine learning applications.

[0109] In addition, images generated using one or more of the techniques disclosed herein can be used to train, test, or certify DNNs for recognizing objects and environments in the real world. Such images can include scenes of roadways, factories, buildings, urban environments, rural environments, humans, animals, and any other physical objects or real-world environments. Such images can be used to train, test, or certify DNNs that are used in machines or robots to manipulate, process, or modify physical objects in the real world. In addition, such images can be used to train, test, or certify DNNs that are used in autonomous vehicles to navigate and move vehicles in the real world. In addition, images generated using one or more of the techniques disclosed herein can be used to convey information to users of such machines, robots, and vehicles.

[0110] Figure 5C Components of an example system 555 that can be used to train and utilize machine learning in accordance with at least one embodiment are shown. As will be discussed, various components can be provided by a single computing system or various combinations of computing devices and resources that can be under the control of a single entity or multiple entities. In addition, various aspects can be triggered, initiated, or requested by different entities. In at least one embodiment, the training of the neural network can be directed by a vendor associated with the vendor environment 506, while in at least one embodiment, the training can be requested by a customer or other user who can access the vendor environment through a client device 502 or other such resource. In at least one embodiment, the training data (or data to be analyzed by the trained neural network) can be provided by the vendor, the user, or the third-party content provider 524. In at least one embodiment, the client device 502 can be, for example, a vehicle or object to be navigated on behalf of a user, who can submit requests and / or receive instructions to facilitate navigation of the device.

[0111] In at least one embodiment, the request can be submitted via at least one network 504 for receipt by the provider environment 506. In at least one embodiment, the client device can be any suitable electronic and / or computing device that enables a user to generate and send such a request, such as, but not limited to, a desktop computer, a laptop computer, a computer server, a smart phone, a tablet computer, a game console (portable or otherwise), a computer processor, computing logic, and a set-top box. The one or more networks 504 may include any suitable network for transmitting the request or other such data, such as the Internet, an intranet, an Ethernet network, a cellular network, a local area network (LAN), a wide area network (WAN), a personal area network (PAN), an ad hoc network with direct wireless connections between peers, and the like.

[0112] In at least one embodiment, the request may be received at the interface layer 508, which in this example may forward the data to the training and reasoning manager 532. The training and reasoning manager 532 may be a system or service including hardware and software for managing services and requests corresponding to data or content. In at least one embodiment, the training and reasoning manager 532 may receive a request to train a neural network and may provide data for the request to the training module 512. In at least one embodiment, if the request is not specified, the training module 512 may select an appropriate model or neural network to use and may use the relevant training data to train the model. In at least one embodiment, the training data may be a batch of data stored in the training data repository 514, received from the client device 502, or obtained from a third-party supplier 524. In at least one embodiment, the training module 512 may be responsible for the training data. The neural network may be any appropriate network, such as a recurrent neural network (RNN) or a convolutional neural network (CNN). Once the neural network is trained and successfully evaluated, the trained neural network may be stored, for example, in a model repository 516, which may store different models or networks for users, applications, services, etc. In at least one embodiment, there may be multiple models for a single application or entity, which may be utilized based on a number of different factors.

[0113] In at least one embodiment, at a later point in time, a request for content (e.g., path determination) or data determined or influenced at least in part by a trained neural network may be received from the client device 502 (or another such device). The request may include, for example, input data to be processed using the neural network to obtain one or more inferences or other output values, classifications, or predictions, or for at least one embodiment, the input data may be received by the interface layer 508 and directed to the inference module 518, although different systems or services may also be used. In at least one embodiment, if not already locally stored to the inference module 518, the inference module 518 may obtain an appropriate trained network from the model repository 516, such as a trained deep neural network (DNN) discussed herein. The inference module 518 may provide data as input to the trained network, which may then generate one or more inferences as output. For example, this may include a classification of the input data instance. In at least one embodiment, the inference may then be transmitted to the client device 502 for display to the user or other communication with the user. In at least one embodiment, the user's context data may also be stored in a user context data repository 522, which may include data about the user that may be used as a network input for generating inferences or determining data to be returned to the user after obtaining an instance. In at least one embodiment, related data that may include at least some of the input or inference data may also be stored in a local database 534 for processing future requests. In at least one embodiment, a user may use account information or other information to access resources or functions of a provider environment. In at least one embodiment, if allowed and available, user data may also be collected and used to further train the model to provide more accurate inferences for future requests. In at least one embodiment, a request for a machine learning application 526 executed on a client device 502 may be received through a user interface, and the results may be displayed through the same interface. The client device may include resources such as a processor 528 and a memory 562 for generating requests and processing results or responses, and at least one data storage element 552 for storing data for the machine learning application 526.

[0114] In at least one embodiment, the processor 528 (or the processor of the training module 512 or the reasoning module 518) will be a central processing unit (CPU). However, as described above, resources in such an environment may utilize GPUs to process data for at least some types of requests. GPUs such as PPU 400 have thousands of cores and are designed to handle large parallel workloads, and therefore have become popular in deep learning for training neural networks and generating predictions. While using GPUs for offline construction allows larger and more complex models to be trained faster, generating predictions offline means that request-time input features cannot be used, or predictions must be generated for all feature permutations and stored in lookup tables to serve real-time requests. If the deep learning framework supports CPU mode, and the model is small and simple enough that feedforward can be performed on the CPU with reasonable latency, a service on a CPU instance can host the model. In this case, training can be performed offline on the GPU and reasoning can be performed in real time on the CPU. If the CPU approach is not feasible, the service can run on a GPU instance. However, because GPUs have different performance and cost characteristics than CPUs, running a service that offloads runtime algorithms to a GPU may require that it be designed differently from a CPU-based service.

[0115] In at least one embodiment, video data may be provided from client device 502 for enhancement in vendor environment 506. In at least one embodiment, video data may be processed for enhancement on client device 502. In at least one embodiment, video data may be streamed from third party content provider 524 and enhanced by third party content provider 524, vendor environment 506, or client device 502. In at least one embodiment, video data may be provided from client device 502 for use as training data in vendor environment 506.

[0116] In at least one embodiment, supervised and / or unsupervised training can be performed by the client device 502 and / or the vendor environment 506. In at least one embodiment, a set of training data 514 (e.g., classified or labeled data) is provided as input for use as training data. In at least one embodiment, the training data can include instances of at least one type of object for which the neural network is to be trained, and information identifying the type of object. In at least one embodiment, the training data can include a set of images, each of which includes a representation of a type of object, wherein each image also includes or is associated with a label, metadata, classification, or other information identifying the type of object represented in the corresponding image. A variety of other types of data can also be used as training data, which can include text data, audio data, video data, and the like. In at least one embodiment, the training data 514 is provided as training input to the training module 512. In at least one embodiment, the training module 512 can be a system or service including hardware and software, such as one or more computing devices executing a training application for training a neural network (or other model or algorithm, etc.). In at least one embodiment, the training module 512 receives an instruction or request indicating the type of model to be used for training, which in at least one embodiment can be any suitable statistical model, network or algorithm useful for such a purpose, which can include artificial neural networks, deep learning algorithms, learning classifiers, Bayesian networks, etc. In at least one embodiment, the training module 512 can select an initial model or other untrained model from an appropriate repository and train the model using training data 514 to generate a trained model (e.g., a trained deep neural network) that can be used to classify similar types of data or generate other such inferences. In at least one embodiment in which training data is not used, an initial model can still be selected for training the input data of each training module 512.

[0117] In at least one embodiment, the model can be trained in several different ways, which can depend in part on the type of model selected. In at least one embodiment, a training data set can be provided to a machine learning algorithm, where the model is a model artifact created by a training process. In at least one embodiment, each instance of the training data contains a correct answer (e.g., a classification) that can be referred to as a target or target attribute. In at least one embodiment, the learning algorithm finds patterns in the training data that map input data attributes to targets - the answers to be predicted - and the machine learning model is the output that captures these patterns. In at least one embodiment, the machine learning model can then be used to obtain predictions for new data for which a target is not specified.

[0118] In at least one embodiment, the training and inference manager 532 can select from a set of machine learning models including binary classification, multi-class classification, generative and regression models. In at least one embodiment, the type of model to be used can depend at least in part on the type of target to be predicted.

[0119] In one embodiment, PPU 400 includes a graphics processing unit (GPU). PPU 400 is configured to receive commands specifying a shader program for processing graphics data. Graphics data may be defined as a set of primitives, such as points, lines, triangles, quadrilaterals, triangle strips, etc. Typically, a primitive includes data specifying the number of vertices of the primitive (e.g., in a model space coordinate system) and attributes associated with each vertex of the primitive. PPU 400 may be configured to process graphics primitives to generate a frame buffer (e.g., pixel data for each pixel of a display).

[0120] The application writes the model data (e.g., a collection of vertices and attributes) of the scene to a memory, such as system memory or memory 404. The model data defines each object that may be visible on the display. The application then makes an API call to the driver kernel to request that the model data be rendered and displayed. The driver kernel reads the model data and writes commands to one or more streams to perform operations that process the model data. These commands may reference different shader programs to be implemented on the processing units within the PPU 400, including one or more of a vertex shader, a hull shader, a domain shader, a geometry shader, and a pixel shader. For example, one or more processing units may be configured to execute a vertex shader program that processes multiple vertices defined by the model data. In one embodiment, different processing units may be configured to execute different shader programs simultaneously. For example, a first subset of processing units may be configured to execute a vertex shader program, while a second subset of processing units may be configured to execute a pixel shader program. The first subset of processing units processes the vertex data to generate processed vertex data, and writes the processed vertex data to the secondary cache 460 and / or the memory 404. After the processed vertex data is rasterized (e.g., converted from three-dimensional data to two-dimensional data in screen space) to generate fragment data, a second subset of processing units executes a pixel shader to generate processed fragment data, which is then blended with other processed fragment data and written to a frame buffer in memory 404. The vertex shader program and the pixel shader program can be executed simultaneously, processing different data from the same scene in a pipelined manner until all model data for the scene has been rendered to the frame buffer. The contents of the frame buffer are then sent to a display controller for display on a display device.

[0121] The image generated by applying one or more of the techniques disclosed herein can be displayed on a monitor or other display device. In some embodiments, the display device can be directly coupled to a system or processor that generates or renders an image. In other embodiments, the display device can be indirectly coupled to the system or processor, for example, via a network. Examples of such networks include the Internet, mobile telecommunications networks, WIFI networks, and any other wired and / or wireless network systems. When the display device is indirectly coupled, the image generated by the system or processor can be transmitted to the display device via a network stream. For example, such a stream allows a rendered image to be transmitted and displayed on a server, a data center, or in a cloud-based computing environment, and on one or more user devices (such as computers, video game consoles, smartphones, other mobile devices, etc.) that are physically separated from a server or data center. Therefore, the technology disclosed herein can be used to enhance the image of streaming, and enhance the service of streaming images, such as NVIDIA GeForce Now (GFN), Google Stadia, etc.

[0122] Example streaming system

[0123] Figure 6 FIG. 6 is a schematic diagram of an example system of a streaming system 605 according to some embodiments of the present disclosure. Figure 6 Includes server 603 (which may include Figure 5A The example processing system 500 and / or Figure 5B ), client 604 (which may include components, features and / or functions similar to the exemplary system 565 of Figure 5A The example processing system 500 and / or Figure 5B In some embodiments of the present disclosure, system 605 may be implemented.

[0124] In one embodiment, the streaming system 605 is a game streaming system, and the server 603 is a game server. In the system 605, for a game session, the client device 604 can only receive input data in response to input from the input device 626, send the input data to the server 603, receive encoded display data from the server 603, and display the display data on the display 624. In this way, more computationally intensive calculations and processing are offloaded to the server 603 (e.g., rendering of the graphical output of the game session, particularly ray or path tracing, is performed by the GPU 615 of the server 603). In other words, the game session is streamed from the server 603 to the client device 604, thereby reducing the requirements for graphics processing and rendering on the client device 604.

[0125] For example, with respect to instantiation of a game session, client device 604 may be displaying frames of the game session on display 624 based on receiving display data from server 603. Client device 604 may receive input from one of input devices 626 and generate input data in response. Client device 604 may send the input data to server 603 via communication interface 621 and over network 606 (e.g., the Internet), and server 603 may receive the input data via communication interface 618. CPU 608 may receive the input data, process the input data, and send the data to GPU 615, which causes GPU 615 to generate a rendering of the game session. For example, the input data may represent the movement of a user character in the game, firing a weapon, reloading, passing a ball, steering a vehicle, and the like. Rendering component 612 may render the game session (e.g., representing the result of the input data), and rendering capture component 614 may capture the rendering of the game session as display data (e.g., as image data capturing a frame of the rendering of the game session). The rendering of the game session may include ray or path-traced lighting and / or shadow effects calculated using one or more parallel processing units (e.g., GPUs, which may further employ the use of one or more dedicated hardware accelerators or processing cores to perform ray or path tracing techniques) of the server 603. The encoder 616 may then encode the display data to generate encoded display data, and the encoded display data may be sent to the client device 604 via the network 606 via the communication interface 618. The client device 604 may receive the encoded display data via the communication interface 621, and the decoder 622 may decode the encoded display data to generate display data. The client device 604 may then display the display data via the display 624.

[0126] Server 603 may include FEC encoder 110 of system 100 , which receives one or more source video streams (media packet frames) that are transmitted to client device 604 along with calculated FEC packets via communication interface 618 and network 606 .

[0127] It should be noted that the techniques described herein may be included in executable instructions stored in a computer-readable medium, which is used by or in conjunction with a processor-based instruction execution machine, system, device, or apparatus. Those skilled in the art will appreciate that for some embodiments, various different types of computer-readable media may be included for storing data. When used herein, "computer-readable media" includes one or more of any suitable media for storing executable instructions of a computer program, so that an instruction execution machine, system, device, or apparatus can read (or obtain) instructions from the computer-readable medium and execute these instructions to implement the described embodiments. Suitable storage formats include one or more of electronic, magnetic, optical, and electromagnetic formats. A non-exhaustive list of conventional exemplary computer-readable media includes: portable computer disks; random access memory (RAM); read-only memory (ROM); erasable programmable read-only memory (EPROM); flash memory devices; and optical storage devices, including portable compact disks (CDs), portable digital video disks (DVDs), and the like.

[0128] It should be understood that the arrangement of the components shown in the drawings is for illustrative purposes, and other arrangements are possible. For example, one or more of the elements described herein can be implemented as electronic hardware components in whole or in part. Other elements can be implemented in software, hardware, or a combination of software and hardware. Moreover, some or all of these other elements can be combined, some can be completely omitted, and additional components can be added while still realizing the functions described herein. Therefore, the subject matter described herein can be implemented in many different variations, and all such variations are expected to be within the scope of the claims.

[0129] For the ease of understanding the subject matter described herein, many aspects are described according to the action sequence.Those skilled in the art will recognize that various actions can be performed by a dedicated circuit or circuit system, by the program instruction of one or more processors or by a combination of the two.This paper does not expect to mean that the specific order described for performing this sequence must be followed for the description of any action sequence.All methods described herein can be performed in any appropriate order, unless otherwise specified herein or otherwise clearly contradictory with context.

[0130] The use of the terms "one", "an", "the", and similar designations in the context of describing the subject matter (especially in the context of the following claims) should be interpreted as covering both the singular and the plural, unless otherwise specified herein or clearly contradicted by the context. The use of the term "at least one" (e.g., at least one of A and B) followed by a list of one or more items should be interpreted as indicating an item (A or B) selected from the listed items, or any combination of two or more of the listed items (A and B), unless otherwise specified herein or clearly contradicted by the context. In addition, the foregoing description is only for illustrative purposes and not for limiting purposes, because the scope of protection sought is defined by the claims set forth hereafter together with their equivalents. The use of any and all examples or exemplary language (e.g., "such as") provided herein is intended only to better illustrate the subject matter and does not constitute a limitation on the scope of the subject matter, unless otherwise stated. The use of "based on" and other similar phrases indicating conditions that bring about a result in both the claims and the written description is not intended to exclude any other conditions that bring about the result. The language in the description should not be interpreted as indicating that any element that is not claimed for protection is essential to the implementation of the claimed invention.

Claims

1. A computer-implemented method comprising: Receive a sequence of media packets where for a positive integer N, the number of media packets is less than or equal to 2 N , and each media packet is associated with a different binary value represented by N bits; selecting a first cluster of the media packets from the sequence of media packets, the first cluster consisting of one or more media packets associated with the binary value having a first bit asserted, wherein a first one of the number of media packets and a second one of the number of media packets are included in the first cluster; selecting a second cluster of the media packets from the sequence of media packets, the second cluster consisting of one or more media packets associated with the binary value having its second bit asserted, wherein the first media packet is included in the second cluster and the second media packet is not included in the second cluster; Calculating a first FEC packet among a plurality of forward error correction FEC packets using the first clustering; as well as A second FEC packet in the plurality of FEC packets is calculated using the second cluster. 2 . The computer-implemented method of claim 1 , wherein the first bit is a most significant bit of the binary value. 3 . The computer-implemented method of claim 1 , wherein the first bit of the binary value is not a least significant bit of the binary value.

4. The computer-implemented method of claim 1, wherein the binary value associated with each media packet in the number indicates the sequence in which the media packet was received.

5. The computer-implemented method of claim 1, wherein the first and second FEC packets are calculated using an exclusive-OR operation.

6. The computer-implemented method of claim 1 , further comprising: The number of the media packets and the first and second FEC packets are sent to a remote device.

7. The computer-implemented method of claim 1, wherein the first and second FEC packets are computed in parallel.

8. The computer-implemented method of claim 1, wherein a number of FEC packets in the plurality of FEC packets is determined based on a packet loss rate and the quantity.

9. The computer-implemented method of claim 1 , wherein: At least one of the steps of selecting the first cluster, selecting the second cluster, calculating the first FEC packet, or calculating the second FEC packet is performed on a server or in a data center to stream the number of media packets and the first and second FEC packets to a user device.

10. The computer-implemented method of claim 1, wherein at least one of said selecting the first cluster, said selecting the second cluster, said calculating the first FEC packet, or said calculating the second FEC packet is performed within a cloud computing environment.

11. The computer-implemented method of claim 1 , wherein at least one of said selecting said first cluster, said selecting said second cluster, said calculating said first FEC packet, or said calculating said second FEC packet is performed for at least one of training, testing, or proving of a neural network.

12. The computer-implemented method of claim 11, wherein the neural network comprises a neural network for at least one of a machine, a robot, or an autonomous vehicle.

13. A system comprising: Transmission interface; as well as A forward error correction (FEC) encoder coupled to the transmission interface, the FEC encoder being configured to: Receive a sequence of media packets, wherein for a positive integer N, the number of media packets is less than or equal to 2 N , and each media packet is associated with a different binary value represented by N bits; selecting a first cluster of the media packets from the sequence of media packets, the first cluster consisting of one or more media packets associated with the binary value having a first bit asserted, wherein a first one of the number of media packets and a second one of the number of media packets are included in the first cluster; selecting a second cluster of the media packets from the sequence of media packets, the second cluster consisting of one or more media packets associated with the binary value having its second bit asserted, wherein the first media packet is included in the second cluster and the second media packet is not included in the second cluster; calculating a first FEC packet among a plurality of FEC packets using the first cluster; as well as A second FEC packet in the plurality of FEC packets is calculated using the second cluster.

14. The system of claim 13, wherein the first bit of the binary value is a most significant bit.

15. The system of claim 13, wherein the first bit of the binary value is not a least significant bit of the binary value.

16. The system of claim 13, wherein the binary value associated with each media packet in the number indicates the sequence in which the media packet was received.

17. The system of claim 13, wherein the number of the plurality of FEC packets is determined based on a packet loss rate.

18. The system of claim 13, wherein the FEC encoder is further configured to The number of the media packets and the first and second FEC packets are output to a network through the transmission interface.

19. A non-transitory computer readable medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the following steps: Receive a sequence of media packets, wherein for a positive integer N, the number of media packets is less than or equal to 2 N , and each media packet is associated with a different binary value represented by N bits; selecting a first cluster of the media packets from the sequence of media packets, the first cluster consisting of one or more media packets associated with the binary value having a first bit asserted, wherein a first one of the number of media packets and a second one of the number of media packets are included in the first cluster; selecting a second cluster of the media packets from the sequence of media packets, the second cluster consisting of one or more media packets associated with the binary value having its second bit asserted, wherein the first media packet is included in the second cluster and the second media packet is not included in the second cluster; Calculating a first FEC packet among a plurality of forward error correction FEC packets using the first clustering; as well as A second FEC packet in the plurality of FEC packets is calculated using the second cluster.

20. The non-transitory computer-readable medium of claim 19, further comprising sending the number of the media packets and the first and second FEC packets to a remote device.

Images