SCHEDULING DOWNSTREAM TRAFFIC OF A VIRTUAL CCAP

Abstract

Non-transitory computer-readable storage medium that stores one or more sequences of instructions for scheduling packets to be forwarded to Data Over Cable Service Interface Specification (DOCSIS) downstream channels, the execution of which causes: Inserting a packet that is to be forwarded on a DOCSIS downstream channel into a service stream queue, wherein packets stored in the service stream queue belong to a single service stream; Passing a request up a hierarchy of scheduling elements to a scheduler process to schedule the packet for delivery; where the scheduler process determines an allocation that specifies how much data traffic is to be offered to the DOCSIS downstream channel; wherein the allocation allocated for the DOCSIS downstream channel is expressed in units of symbols rather than bytes, and wherein the number of bits represented by each of the symbols differs depending on a modulation of the symbol; and wherein the scheduler process performs an extension of a special authorization to the service stream queue by transmitting symbols in the authorization using the modulation of each of the symbols, the modulation of each symbol being based on the service stream queue that issued the request.

Description

CLAIMING A PRIORITY

[0001] This application claims priority over provisional US patent application No. 62 / 360,851, entitled “Virtual CCAP Downstream Traffic Scheduling”, which was filed on July 11, 2016. AREA OF INVENTION

[0002] Embodiments of the invention relate to the scheduling of packets where forwarding on DOCSIS downstream channels is to be carried out as part of a virtual CCAP (Converged Cable Access Platform). BACKGROUND

[0003] A CCAP (Converged Cable Access Platform) is a project led by CableLabs that combines two projects technically and operationally: CMAP (Converged Multiservice Access Platform), led by Comcast Corp., and CESAR (Converged Edge Services Access Router), led by Time Warner Cable Inc.

[0004] A virtual CCAP (Converged Cable Access Platform) is software that performs the functions of a hardware-based CCAP. The virtual CCAP can run on hardware components that include a COTS switch / router (COTS = Commercial Off-the-Shelf) and one or more COTS compute servers. A commercial example of a virtual CCAP is CableOS, available from Harmonic, Inc. of San Jose, California.

[0005] DOCSIS (Data Over Cable Service Interface Specification) is a telecommunications standard used to provide internet access over a cable modem. Version 3.0 of DOCSIS introduced the concept of a bonding group. A single data stream for a cable modem (CM) can be transmitted over two or more channels within a single bonding group (BG). Conversely, two or more BGs can share a single channel. Consequently, given the loose correlation between BGs and RF channels, scheduling packets while adhering to Quality of Service (QoS) requirements can be complex.

[0006] WO 2012 / 178117 A2 discloses a parameterized scheduling system that considers end-user applications and is used with scheduling groups containing data streams from heterogeneous applications. Data packets are analyzed at multiple protocol levels to detect features related to the packets' communication. The data packets are filtered to enable efficient feature detection. The detected features can be used to schedule packet transmission. These features can also be used to dynamically modify scheduling parameters. The dynamic scheduling parameters can maximize the quality of experience (QoE) in response to recurring network patterns, one-off events, application features, protocol characteristics, device characteristics, service level agreements, or combinations thereof.The planning parameters can also take into account the aspects of “neglect of duration” and “repetition effect” in the end user’s perception of video quality in order to manage video traffic during periods of congestion. SUMMARY

[0007] The object of the present invention is to provide an advantageous approach for scheduling packets forwarded on downstream channels of the Data Over Cable Service Interface Specification (DOCSIS). This object is achieved by the features of the main and dependent claims using time-based scheduling. The dependent claims relate to optional features of some embodiments of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Embodiments of the invention are shown by way of example and without limitation in the figures of the accompanying drawings, in which the same reference numerals denote similar elements; in these are: Fig. 1 a flowchart of the functional steps associated with the assignment of a QOS class according to an embodiment of the invention; Fig. 2 a representation of a topology for scheduling DOCSIS packages with WFQ (= Weighted-Fair-Queueing = weighted fair queuing) of overlapping QCls according to an embodiment of the invention; Fig. 3 a representation of an “ice cube tray” algorithm for distributing a concurrent reserved total bandwidth of all DCSs between shared channels, according to an embodiment of the invention; Fig.4 a representation of a topology for an SP scheduling (SP = Strict-Priority) of QOS classes according to an embodiment of the invention; Fig. 5 a block diagram representing a computer system on which an embodiment of the invention may be implemented; and Fig. 6 a diagram of a logical representation of a QOS hierarchy according to an embodiment of the invention. DETAILED DESCRIPTION OF THE INVENTION

[0009] This document presents solutions for scheduling packets where forwarding on DOCSIS downstream channels is to take place as part of a virtual CCAP (Converged Cable Access Platform). Numerous specific details are provided in the following description for explanatory purposes, to ensure a basic understanding of the embodiments of the invention described herein. It is understood, however, that the embodiments of the invention described herein may be implemented without these specific details. In other cases, generally known structures and devices are depicted in the form of a block diagram or discussed at a higher level to avoid unnecessarily obscuring the teachings of the embodiments of the invention. FUNCTION OVERVIEW

[0010] Embodiments of the invention relate to the scheduling of packets where forwarding on DOCSIS downstream channels is to take place as part of a virtual CCAP consisting of a COTS-CRE (COTS = Commercial Off-the-Shelf; CRE = Core Routing Engine). Packet scheduling can be performed entirely in software using multiple CPU cores in the core servers. Software-based scheduling allows the topology of the scheduling tree diagram to be easily modified to introduce new features and requirements.

[0011] A downstream packet scheduler of one implementation can be implemented in a TM thread (TM = Traffic Manager) operating within the Data Plane Development Kit (DPDK) environment, which is a set of open-source libraries and drivers for "fast packet processing." While the DPDK itself includes a packet scheduling module, the DPDK scheduler is not optimal for scheduling DOCSIS packets for several reasons. In particular, the DPDK packet scheduler lacks the concept of bundling groups arranged overlapping on channels.

[0012] A TM thread consists of two main functions: queuing incoming packets into queues within each data stream, and scheduling the removal of these packets from the queue for sending. A more detailed description of the TM thread, and especially its scheduling function, follows.

[0013] Embodiments of the invention relate to and overcome the inherent limitations of the DPDK packet scheduler, as will be explained in more detail later. QOS classes

[0014] In embodiments of the invention, an entity called a QOS class is assigned to all DOCSIS service data streams. A QOS class is an entity that defines common access control and scheduling parameters shared by all DOCSIS service data streams assigned to that QOS class. For illustrative purposes, a non-restrictive set of default QOS classes according to one embodiment of the invention is shown in Table 1 below. TABLE 1 Name of the QOS class Description of the associated DOCSIS SF (SF = Service Stream) DsMaps a single SF per primary downstream channel for forwarding DOCSIS maps DsVoice a SF for the payload of each normal priority voice call DsVoiceEm a SF for the user data of each "emergency" voice call DsAdmin Administrative SFs for transporting network management traffic, such as ARP and DHCP. DSMulticast DOCSIS "group" SFs for forwarding IP multicast sessions DsReserved CM-Default-SFs, whose signaling is done with an MRTR (minimum reserved data rate). DsSignal CM non-default SFs whose signaling is done with an MRTR (minimum reserved data rate). DsBestEffort all other SFs

[0015] The QOS classes DsVoice and DsVoiceEm are defined separately solely for access control; the scheduling of service data streams assigned to a respective QOS class is performed using the same WFQ scheduling element (WFQ = Weighted Fair Queueing). The WFQ scheduling element is described in more detail below.

[0016] Each DOCSIS SF typically has a token bucket rate limiter defined by the parameters "Maximum Sustained Traffic Rate" and "Maximum Traffic Burst". Examples of this rate limiter are shown in Fig. 2 represented as rate limiters 210 and 212, which are shown above the service data stream 214.

[0017] A DCS (Downstream Channel Set) represents either a single downstream channel or a bonding group consisting of multiple downstream channels. A DOCSIS CMTS assigns a single DCS to each SF for scheduling purposes. For the QoS classes DsMaps, DsVoice, DsVoiceEm, and DsAdmin, service data streams are assigned only a single channel. All other QoS classes have SFs that can be assigned to either single-channel or multi-channel DCSs. QOS CLASS ASSIGNMENT

[0018] Fig. Figure 1 is a flowchart of the functional steps associated with the assignment of QoS classes according to one embodiment of the invention. In step 110, it is determined whether the DOCSIS downstream service data stream was generated by a dynamic DOCSIS service add message. If the determination in step 110 is positive, then in step 112, it is determined whether its MRTR (Minimum Reserved Traffic Rate), signaled by TLV 25.10, is equal to its MSTR (Maximum Sustained Traffic Rate), signaled by TLV 25.8.

[0019] If the determination in step 112 is affirmative, then step 114 is performed. In step 114, a check is carried out to determine whether a so-called "gate" of DOCSIS PacketCable is authorized for the service data stream. If it is not authorized, then in step 118 the data stream is classified as a normal priority QoS class "DsVoice". If the data stream is authorized with PacketCable, in step 116 it is checked whether the PacketCable gate has a "high" priority (for PacketCable 1.0), or a priority greater than or equal to 4 (out of 7) for PCMM (PacketCable Multimedia). If the determination in step 116 is affirmative, then the data stream is a constant bitrate data stream, which is classified as QoS class DsVoiceEm in step 120. Otherwise, if the determination in step 116 is negative, the data stream will be signaled at step 118 with a normal priority QoS class DsVoice.

[0020] If the determination in step 110 was incorrect, the DOCSIS traffic priority TLV 25.7 of the data stream is checked. If the TLV exists and a CCAP table is configured to assign this traffic priority to a specific QoS class, then the data stream is assigned to the corresponding QoS class, as shown in step 124.

[0021] Otherwise, step 126 determines whether the data stream is a DOCSIS group service stream for forwarding multicast packets. If the determination in step 126 is positive, the data stream is assigned to the QoS class DsMulticast, as shown in step 128.

[0022] Otherwise, if the determination in step 126 is negative, then in step 130 the data stream is checked to identify whether the data stream is configured with an MRTR of TLV 25.10. If the determination in step 130 is positive, then in step 132 a determination is made as to whether the data stream is the so-called "default" downstream service data stream for its cable modem, i.e., the first downstream service data stream in the cable modem's configuration file (CM).

[0023] If the determination in step 132 is affirmative, the data stream is assumed to be intended for relatively high and bursty throughput and is assigned the QoS class DsReserved, as shown in step 134. Otherwise, if the determination in step 132 is negative, the data stream is a secondary downstream data stream and is assumed to be intended for relatively infrequent, low-bandwidth traffic, and is assigned the QoS class DsSignal, as shown in step 136. Note that the use of the default position, as opposed to a secondary position of the service data stream, which are typically assigned high-throughput or low-throughput QoS classes, is unique in the industry.Using a standard configuration rule for this distinction simplifies things for an MSO operator, as it avoids other manual configuration mechanisms, such as explicit service class names or a table used to assign a traffic priority to a QOS class.

[0024] If none of the previously described conditions apply, then the data stream is assigned to the QOS class DsBestEffort, as shown in step 138. SETS FROM DOWNSTREAM CHANNELS

[0025] A DCS (Downstream Channel Set) is a set of downstream channels in the same DOCSIS MAC domain. A DCS can contain a single downstream channel or it can be a downstream bundling group with multiple channels. A given downstream channel can belong to multiple DCSs. A DOCSIS CMTS assigns a service stream to exactly one DCS at a time. While an assignment to a DCS exists, packets from that service stream are only scheduled on the channels contained within that DCS. Informally, a DCS is sometimes referred to as an "interface," although the DOCSIS standard does not require that bundling groups be assigned an "interface index" for SNMP reporting purposes.

[0026] As used here, the term QCI (QCI = QOS Class Interface = "interface of a QOS class") refers to the scheduling of packets of the service data streams of a single QOS class within a single DCS, i.e., a single interface. Since channels can belong to multiple DCSs, packets belonging to the different QCls must compete against each other for scheduling on the channel.

[0027] Embodiments of the invention advantageously enable a dynamic, flexible topology configuration, whereby multi-channel QCIs sharing a channel compete against each other using either WFQ (Weighted-Fair-Queueing) or SP (Strict-Priority). Note that in the examples in Table 1, the flexibility of WFQ or SP scheduling is only valid for the QOS classes DsMulticast, DsSignal, DsReserved, and DsBestEffort, since these have QCIs that can overlap on a channel; service data streams of the other QOS classes listed in Table 1 are always assigned to a single channel and therefore never overlap.

[0028] In embodiments of the invention, no intermediate queue with a DCS is used. Prior art approaches involved transferring packets from a service stream queue to a DCS queue when scheduling packets. This prior art approach was the only way to support the QoS hierarchy used in the prior art (i.e., a hierarchy in which higher layers always fully aggregate lower layers) and the common use of bonding channels and overlapping bonding groups (DCSs) in the cable industry. In contrast, embodiments of the invention support any configuration of DCSs (including overlapping DCSs) without requiring any intermediate DCS queues.For this purpose, embodiments use a non-aggregating hierarchy in which elements of the upper layer do not contain all elements of the next lower layer that are associated with the element of the upper layer. DOWNSTREAM TRAFFIC DATA PROFILES

[0029] In embodiments of the invention, a "traffic profile" is used, which defines a set of access control and scheduling parameters for operating a QoS class on a single DCS. The format of an exemplary downstream traffic profile, in a configuration for a specific example, is shown below:

[0030] A "ds-traffic-profile" is a named set of "ds-qc-profiles". A ds-qc-profile defines a set of parameters for access control and packet scheduling for a single QCI, i.e., a single QoS class operating on a single DCS. The parameters of a ds-qc-profile are described below in Table 2. Table 2 parameter Description scheduling-priority Selecting SP for approvals that are offered to QCI data streams. reserved-bandwidth Specifies the bandwidth of the DCS capacity that is permissible for the CIR of SFs in the QCI. Advantageously, this is specified either directly as a speed, such as kilobits per second, or indirectly as a percentage of the RF capacity of the DCS. concurrent percent Specifies the proportion of permissible CIR traffic that is assumed to be active simultaneously, thereby effectively controlling CIR overbooking; e.g., a concurrency of 25% allows a 4-to-1 overbooking of CIR traffic. class-rate-limit (class rate limit) Specifies the maximum permissible rate of the sum of all SFs in the QCI. This is advantageously configured directly as a speed, for example in kilobits per second, indirectly as a proportion of the RF capacity of the DCS, or in the same way as the reserved bandwidth. flow-queue-limit-msec (Data stream queue limit in ms) Indirectly configures the maximum number of packets allowed in a service data stream queue. The millisecond count for the configured value of "Data Stream Queue Limit in msec" is sufficient to insert a packet into the data stream job queue, which consists of packets of the fixed size "assumed packet byte count", whose scheduling is performed during the MSTR (Main Stream Manager) of the data stream. assumed-packet-bytes The assumed number of packet bytes for the total data traffic for the purpose of calculating the overhead present at DOCSIS and the physical layer, as well as the maximum number of packets queued for the service data stream.

[0031] Configuring "reserved bandwidth" and "class rate limit" as absolute speeds (e.g., kilobits per second) ensures that the bitrate is directly visible to the operator and is suitable for a traffic profile assigned to a DCS with a known number of channels. Configuring these parameters as relative percentages allows for the simpler approach of defining a single traffic profile that can then be applied to multiple (or even all) DCSs, regardless of the number of channels. Customers requiring a minimum numerical bitrate for an application (e.g., 2.0 Mbps per program for an HDTV video) can configure using the absolute bitrate without having to perform mathematical calculations to configure percentages.The option to use either an absolute or a relative percentage configuration of bandwidth for QOS profiles, as enabled by implementations, is certainly unique within the CMTS industry, and indeed, the user considers this to be unique among all computer networking elements. WFQ SCHEDULING OF OVERLAPKING QCIS

[0032] Fig. Figure 2 is a representation of a topology for scheduling DOCSIS packets with WFQ (Weighted-Fair-Queueing) of overlapping QCls according to an embodiment of the invention.

[0033] A TM thread of the DPDK process repeatedly alternates between reading incoming packets and scheduling outgoing packets. At this point, incoming packets have already been mapped to a DOCSIS service stream. The TM thread queues the packets for that specific stream. The TM thread maintains a maximum queue size for the service stream (for taildrop operation) and can advantageously implement an AQM (Active Queue Management) process to reduce the average queue length.

[0034] The TM scheduler operates using a "request / approval" mechanism. Incoming packets generate requests that are passed upwards through the scheduling elements of a tree structure. For each channel, approvals originate at the top of the tree structure, and their downward distribution is handled by scheduling elements that utilize SP (Strict Priority) and WFQ (Weighted-Fair Queueing).

[0035] The approval granted to a scheduler at a higher level is then offered, according to the concept, to lower scheduling elements, as in Fig.Figure 2 shows the process until the authorization reaches a specific DOCSIS SF. Packets are only queued into the service stream queues; the scheduling elements do not contain queues. In the software-based scheduling process of one embodiment, the process of offering an authorization to a lower scheduling element involves calling a software subroutine to allocate the authorization to a lower level. FORWARDING A REQUEST TO ABOVE

[0036] Upon arrival of packets at the TM thread, their assignment to a service stream has already taken place. The TM thread adds each packet to a queue available for each service stream, as shown in the lower part of Fig.Figure 2 illustrates this. Each SF queue is assigned a maximum length designed to hold a configurable number of milliseconds of packets of a configured size of "assumed packet bytes" arriving at the SF's maximum sustained traffic rate (MSTR). The use of a configured size of "assumed packet bytes" when configuring a maximum SF queue size is likely unique in the CMTS industry. Using a default size of 1518 bytes for "assumed packet bytes" prevents over-allocation of packet resources to best-effort service streams, while using a smaller size prevents under-allocation of traffic queues for voice, map, and signaling control.

[0037] In general, each DOCSIS data stream can have a Minimum Reserved Traffic Rate (MRTR) and a Maximum Sustained Traffic Rate (MSTR). The configured MRTR is referred to as the Committed Information Rate (CIR) of the data stream. The scheduler calculates the Excess Information Rate (EIR) of a service data stream as the difference between the maximum and minimum rates, i.e., MSTR - MRTR. The scheduler implements separate token bucket rate limiters for the CIR and EIR requests of the data stream. In any DCS, the scheduler routes requests matching the EIR token bucket to the QCI for that DCS's DsBestEffort QoS class. A service data stream can also be associated with more than two QoS classes: DsBestEffort and DsReserved.

[0038] A CIR request for a data stream (i.e., corresponding to its CIR token bucket) is routed to the QCI scheduling element (QCI = QOS class interface) for that data stream. Note that a data stream can contain both CIR and EIR requests, each potentially pending for the same packet. The burst size of the CIR request token bucket is ideally one SMLP; the burst size of the EIR request token bucket is the configured maximum traffic burst of the DOCSIS service data stream. When a respective service data stream is visited in the scheduler, its CIR and EIR token buckets are updated.

[0039] When a queued data stream receives a request that exceeds its CIR and / or EIR rate limit, the stream is "announced" to the associated FlowWfq scheduling element and qualifies to receive permissions offered to that element. The FlowWfq element then "passes" the request up the scheduling hierarchy. Each scheduling element above the FlowWfq level maintains a bitmask of the scheduling elements below it that are queued for processing.

[0040] For example, a query that uses QCI DsReserved FlowWfq for channel D0 in Fig.Step 2 involves a logical pass to the "DCS-SP1-WFQ" element. The scheduler sets a bit corresponding to the QoS class DsReserved in a bitmask of the DCS-SP1-WFQ. The DCS-SP1-WFQ element, in turn, logically passes this request to the CHAN-SP1-WFQ of the respective channel in the DCS. For each SP level of a MAC domain, the scheduler maintains a bitmask with one position for each channel. Finally, the CHAN-SP1-WFQ logically passes this request to the MAC domain level by setting a bit for SP1 in a mask present in each MD. OFFERING A DOWNWARD APPROVAL

[0041] For each MAC domain, the scheduler maintains an 8-bit mask, with a single position for each SP level. A 1 bit indicates that a qualified request for any scheduling element for SP1 is present in the MD. For each combination of MAC domain and SP, the scheduler maintains a 256-bit mask, with a single position for each possible channel in the MAC domain. A 1 bit indicates that a qualified pending request for any QCI is present at the SP level and can be fulfilled by approving that request on the channel. The scheduler of an embodiment can search for qualified channels using a round-robin approach. When searching for a channel, the scheduler calculates an approval amount that specifies how much traffic should be offered on that channel.

[0042] Channel authorizations (which can be expressed in authorization units called symbols, as will be explained later) are limited by a token-bucket ARL function (ARL = Aggregate Rate Limiter) for each channel, which limits the authorization rate so that the channel capacity is not exceeded. The ARL function is defined in Fig. The ARL is shown at the very top of the topology. It further limits the burst of pending packets, preventing overflow of the internal buffers of the RPDs (Remote PHY devices) that handle the physical transmission of the SCQAM or OFDM RF signals. The default burst time is 1 millisecond of symbols.

[0043] How to configure Fig.As shown in Figure 2, all other QoS classes are configured with the same SP1 (SP = Strict Priority). SP1 is the second-lowest of the SP scheduling levels, from SP1 to SP7. The lowest level, SP0, is reserved for a "worse than best-effort" service, i.e., for background or secondary traffic. Fig. 2. Downstream channel 0 is shared by the single-channel DCS for channel 0, a four-channel bundling group, and an eight-channel bundling group. The ARL-limited authorization for channel 0 at SP1 is offered to a "channel WFQ" node designated CHAN-WFQ-(SP1), which implements a WFQ process such as DRR (Deficit Round Robin) to allocate authorizations to the SP1 QCIs of a given DCS sharing the channel.

[0044] The weighting of each DCS at the CHAN-SPx-WFQ scheduling node can be calculated based on an "ice cube tray" distribution of the total (concurrent) reserved bandwidth of all DCSs sharing that channel. The total concurrently served bandwidth for a DCS is the sum of the concurrently reserved bandwidth for each QCI on that DCS. The concurrently reserved bandwidth for a single QCI is obtained by multiplying the "percent concurrency" by the "reserved bandwidth" for a downstream traffic profile configured with QoS classes for the DCS. The process for distributing the total concurrently reserved bandwidth of all DCSs across shared channels is referred to here as the "ice cube tray" algorithm, as it is analogous to the way water flows into an ice cube tray to cover ice cubes of varying heights.

[0045] Fig.Figure 3 is a representation of an “ice cube tray” algorithm for dividing the total concurrent reserved bandwidth of all DCSs between shared channels according to one embodiment of the invention. Step 2 of Fig.Step 3 can be interpreted as pouring water, using the bandwidth currently reserved by a DCS, into certain ice cube tray positions of the pending channels. The water then levels the water to a new average level. If any previously frozen allocation exceeds this new average, an ice cube will protrude above the average water level. Therefore, this position is removed, and the pouring process in step 2 is repeated with a smaller number of channels. Otherwise, the "Ice Cube Tray" algorithm freezes the ice cubes at their current average levels. The "Ice Cube Tray" algorithm is executed whenever a configuration change occurs that affects the distribution of the concurrent reserved bandwidth.in the event of changes to downstream traffic profiles or DCIs (= Downstream Channel Sets = sets of downstream channels).

[0046] The scheduler allows a packet of any length to be sent when any authorization is received for a data stream, causing its token bucket count to become negative. The scheduler handles this when an authorization for CIR or EIR requests is offered to a data stream with no pending packets.

[0047] This results in the scheduling of all pending single-channel QCI packets for DsMaps in SP7 (Strict Priority 7), DsVoice+DsVoiceEm in SP6, and DsAdmin in SP6, in that order. The scheduling of the DsVoice and DsVoiceEm packets is performed using the same WFQ scheduling node. RF SYMBOL UNITS

[0048] To implement both SCQAM and OFDM channels (SCQAM = Single Carrier QAM; OFDM = Orthogonal Frequency Division Multiplexing), one embodiment calculates ARL channel permissions in symbolic units, either SCQAM symbols or OFDM subcarrier symbols. Using symbols as the unit of permissions is a unique advantage of this embodiment, as all other known industry literature describes scheduling of requests and permissions in terms of byte units. Using symbols for permissions is particularly useful for OFDM channel scheduling, where the number of bits per unit of time varies for a given service data stream, or rather, can vary for each packet.When channel symbols are passed to lower scheduling elements, each request is translated into the equivalent symbols for the specific service data stream through which the request is made.

[0049] An RF channel transmits a fixed number of symbols per unit of time; however, the number of bits represented by each symbol varies depending on the symbol's modulation. A question for nodes performing WFQ scheduling is whether weighting should be based on the number of bits processed during scheduling or on the time elapsed during scheduling (with respect to RF symbols). With bit-based scheduling, SFs of the same weight send the same number of bits per second in the event of congestion, regardless of their OFDM modulation profile. With time-based scheduling, SFs of the same weight send the same number of symbols per second.

[0050] Consider, for example, two data streams with different OFDM modulation rates of 1*B and 2*B bits / s, which are otherwise considered "equal" with respect to priority, minimum rate, and maximum rate of DOCSIS traffic. In a one-second period where both data streams have unlimited capacity, a bit-based scheduler will transmit for the slower data stream for 2 / 3 of the second, resulting in 2B / 3 bits being sent, while transmitting for the faster data stream for 1 / 3 of the second, also resulting in 2B / 3 bits being sent. The total throughput would be 4B / 3 bits.

[0051] With a time-based WFQ scheduler, the scheduler assigns equal symbol times to data streams with equal WFQ weights. Specifically, the scheduler would send data for half a second for each data stream with the same weight, or it would send B / 2 bits for the slower data stream and B bits for the faster data stream, totaling 3B / 2 bits. In this case, the total number of bits sent is higher.

[0052] One advantage of a software-based scheduler is that bit-based or time-based WFQ scheduling can be configured as an operator configuration. The scheduler maps each service data stream to a specific OFDM profile, i.e., modulation. Since each symbol allocation involves a comparison with an SF request, the SF's OFDM profile is used to convert a (byte-based) data stream request into the allocated number of symbols. The remaining symbols are then offered to other scheduling elements. The units from the WFQ decision, such as the actual value derived in a DRR process, can be either the symbols or the size in bytes of the packets processed during forwarding. SP-QCI SCHEDULING (SP = STRICT PRIORITY)

[0053] One advantage of software-based scheduling is the ease with which the scheduling topology can be dynamically changed based on a customized configuration or even dynamic load conditions. Fig. Figure 2 shows an alternative topology for scheduling DOCSIS packets based on SP scheduling of the entirety of QOS classes.

[0054] Fig. Figure 4 is a representation of a topology for SP scheduling of QOS classes according to an embodiment of the invention. In the topology of Fig. In section 4, each QOS class has its own SP level. The "DCS-WFQ" scheduling node handles only a single QOS class. This case can be detected, and the process can be modified to eliminate DRR processing for the DCS-WFQ level. This demonstrates the flexibility of software-based scheduling.

[0055] With SP-WFQ scheduling, oversubscribed CIR traffic is always allowed to take over bandwidth from lower-priority QCls. To prevent lower-priority QCls from being "starved," the QCI's "class rate limiter" parameter can be configured with "as-reserved" to limit a high-priority QCI's scheduling only to its un-oversubscribed, configured reserved rate. LOGICAL REPRESENTATION OF A QOS HIERARCHY

[0056] Fig. Figure 6 is a diagram of a logical representation of a QOS hierarchy according to one embodiment of the invention. As in Fig.As shown in Figure 6, each channel has multiple priorities (usually 0-7). Under each SP (Speedport), there is a priority WFQ (Wideband Frequency Qualifier) ​​that distributes bandwidth among QoS classes at a channel level according to their weights. The purpose of this WFQ is to support the instantiation of more than one QoS class under the same SP. If there is only a single QoS class under the channel WFQ, weights are not needed to determine how bandwidth is distributed.

[0057] For each QoS class at a channel level, bandwidth is allocated to the corresponding QoS class among all connected DCSs in the manner of WFQ. For each QoS class of DCS, bandwidth is distributed to all connected SFs according to their weighting. Non-restrictive examples of QoS classes include DsVoice, DsReserved, DsAdmin, DsBestEffort, and DsMulticast; however, the hierarchy structure is not limited to these specific services or this number of QoS classes. In one implementation, it is also possible for a user to define or configure a QoS class.

[0058] A service provider (SF) can be connected to more than one QoS class and thus receive bandwidth of two priorities: EIR and CIR. In some implementations, it can be supported that an SF is connected to any number of QoS classes, including three or more.

[0059] The in Fig.The six shape elements depicted as black hourglasses represent shapers that enable control of maximum rate and maximum burst for (a) channel (designated as ARL (Aggregate Rate Limiter)), (b) priority according to channel, (c) QOS class of a channel, (d) SF - separate shaper of SF for a bandwidth source of a respective QOS class, and (e) QOS class of a DCS layer.

[0060] The logical representation of Fig. Option 6 can be used to configure CIR percentages per channel instead of per QOS class. IMPLEMENTATION MECHANISMS

[0061] Embodiments of the invention enable packet scheduling to be performed entirely by software, using multiple CPU cores or processors. In one embodiment, packet scheduling can be implemented using a computer system. Fig.Figure 5 is a block diagram representing a computer system 500 on which an embodiment of the invention may be implemented. In one embodiment, the computer system 500 includes a processor 504, main memory 506, ROM 508, a storage device 510, and a communication interface 518. The computer system 500 includes at least one processor 504 for processing information. The computer system 500 also includes main memory 506, such as RAM (Random Access Memory) or another dynamic storage device, for storing information and instructions to be executed by the processor 504. The main memory 506 can also be used for storing temporary variables or other intermediate information during the execution of instructions to be carried out by the processor 504.The Computer System 500 further includes a ROM (Read Only Memory) 508 or other static storage device for storing static information and instructions for the Processor 504. A storage device 510, such as a magnetic disk or an optical disk, is provided for storing information and instructions.

[0062] Embodiments of the invention relate to the use of a computer system 500 for implementing the methods described herein. According to one embodiment of the invention, these methods are carried out by the computer system 500 in response to the processor 504 executing one or more sequences of one or more instructions located in the main memory 506. Such instructions can be read into the main memory 506 from another machine-readable medium, such as the storage device 510. Execution of the sequences of instructions contained in the main memory 506 causes the processor 504 to execute the process steps described herein. In alternative embodiments, a hard-wired circuit can be used instead of, or in combination with, software instructions to implement embodiments of the invention.Thus, embodiments of the invention are not limited to any specific combination of hardware circuits and software.

[0063] The term “non-transitory machine-readable storage medium,” as used here, refers to any physically tangible medium that participates in storing instructions that can be supplied to the 504 processor for execution. Non-restrictive, illustrative examples of machine-readable media include, for example, a floppy disk, a flexible disk, a hard disk, magnetic tape or any other magnetic medium, a CD-ROM, any other optical medium, RAM, a PROM, an EPROM, a FLASH EPROM, any other memory chip or cartridge, or any other medium from which a computer can read.

[0064] Various forms of machine-readable media can be involved in transporting one or more sequences of instructions to the processor 504 for execution. For example, the instructions may initially reside on a magnetic disk of a remote computer. The remote computer can load the instructions into its dynamic memory and send them to the computer system 500 via a network connection 520.

[0065] The communication interface 518 provides a two-way data communication link to a network interface 520 connected to a local area network (LAN). For example, a communication interface 518 can be an ISDN card (ISDN = Integrated Services Digital Network) or a modem to provide a data communication link to a corresponding type of telephone line. Another example is a LAN card (LAN = Local Area Network) to provide a data communication link to a compatible LAN. Wireless connections can also be implemented. In each such implementation, the communication interface 518 performs the sending and receiving of electrical, electromagnetic, or optical signals carrying digital data streams representing various types of information.

[0066] The 520 network interface typically provides data communication over one or more networks to other data devices. For example, the 520 network interface can provide a connection over a local network to a host computer or to data facilities operated by an Internet service provider (ISP).

[0067] The computer system 500 can send messages and receive data, including program code, via the network(s), the network connection 520, and the communication interface 518. For example, a server could transmit requested code for an application program via the internet, a local ISP, a local network, and then to the communication interface 518. The received code can be executed by the processor 504 upon receipt and / or stored in a memory device 510 or other non-volatile memory for later execution.

[0068] The preceding description described embodiments of the invention with reference to numerous specific details that may vary from implementation to implementation. Therefore, the sole and exclusive indicator of what the invention is and what the applicant intends to be the invention is the set of claims contained in this application, in the specific form in which such claims are to be drafted, including any subsequent amendments. Any definitions expressly set forth herein for terms contained in such claims shall be authoritative for the meaning of such terms as used in the claims. Thus, no limitation, element, property, feature, advantage, or attribute not expressly set forth in a claim shall in any way limit the scope of protection of that claim.The description and drawings are therefore to be understood in an illustrative and not in a restrictive sense.

Claims

[1] Non-transitory computer-readable storage medium that stores one or more sequences of instructions for scheduling packets to be forwarded to Data Over Cable Service Interface Specification (DOCSIS) downstream channels, the execution of which causes: Inserting a packet that is to be forwarded on a DOCSIS downstream channel into a service stream queue, wherein packets stored in the service stream queue belong to a single service stream; Passing a request up a hierarchy of scheduling elements to a scheduler process to schedule the packet for delivery; where the scheduler process determines an allocation that specifies how much data traffic is to be offered to the DOCSIS downstream channel; wherein the allocation allocated for the DOCSIS downstream channel is expressed in units of symbols rather than bytes, and wherein the number of bits represented by each of the symbols differs depending on a modulation of the symbol; and wherein the scheduler process performs an extension of a special authorization to the service stream queue by transmitting symbols in the authorization using the modulation of each of the symbols, the modulation of each symbol being based on the service stream queue that issued the request. [2] Non-transitory computer-readable storage medium according to claim 1, wherein the scheduler process performs an extension of the authorization to the service data stream queue using a Quality of Service (QoS) hierarchy structure, according to a set of weights and priorities specified therein. [3] Non-transient computer-readable storage medium according to claim 1, where passing a request up a hierarchy of scheduling elements to a scheduler process to schedule the packet for delivery includes: passing both a CIR (Committed Information Rate) request and an EIR (Excess Information Rate) request for that packet up a hierarchy of scheduling elements to a scheduler process to schedule the packet for delivery; where both the CIR request and the EIR request for the packet are pending at the same time; and the transfer of symbols in the authorization is carried out in different ways, based on whether the authorization has satisfied the CIR request or the EIR request. [4] Non-transitory computer-readable storage medium that stores one or more sequences of instructions for scheduling packets to be forwarded to Data Over Cable Service Interface Specification (DOCSIS) downstream channels, the execution of which causes: Assign to a DOCSIS service stream a specific Quality of Service (QoS) class that defines common access control and scheduling parameters shared by all DOCSIS service streams assigned to it; Assigning a DCS (Downstream Channel Set) to the DOCSIS service data stream, where the DCS represents either a single DOCSIS downstream channel or a bonding group consisting of multiple downstream channels; Inserting a packet that needs to be forwarded on the DCS into a service stream queue, wherein packets stored in the service stream queue belong to the DOCSIS service stream; and where a scheduler process performs scheduling of the packet on the DCS, based on the access control and scheduling parameters defined by the specific QOS class. [5] Non-transitory computer-readable storage medium according to claim 4, wherein the scheduler process that performs the scheduling of the packet comprises: that the scheduler process determines which QOS class, from a plurality of different QOS classes that can send packets on the DCS, should be the next one for sending packets on the DCS during a scheduling process. [6] Non-transitory computer-readable storage medium according to claim 5, wherein the scheduler process determines which QOS class, from the plurality of different QOS classes that can send packets on the DCS, should be the next one for sending packets on the DCS during a scheduling, based either on a WFQ or an SP solution approach (WFQ = Weighted Fair Queueing; SP = Strict Priority). [7] Non-transitory computer-readable storage medium according to claim 4, wherein the special QOS class can be configured to define the common access control and scheduling parameters using both relative measures in the sense of percentages and absolute measures in the sense of bytes / unit of time. [8] Device (500) for scheduling packets where forwarding to Data Over Cable Service Interface Specification (DOCSIS) downstream channels is to be performed, comprising: one or more processors (504); and one or more non-transitory computer-readable storage media that store one or more sequences of instructions which, when executed, cause: Inserting a packet that is to be forwarded on a DOCSIS downstream channel into a service stream queue, wherein packets stored in the service stream queue belong to a single service stream; Passing a request up a hierarchy of scheduling elements to a scheduler process to schedule the packet for delivery; where the scheduler process determines an allocation that specifies how much data traffic is to be offered to the DOCSIS downstream channel; wherein the allocation allocated for the DOCSIS downstream channel is expressed in units of symbols rather than bytes, and wherein the number of bits represented by each of the symbols differs depending on a modulation of the symbol; and wherein the scheduler process performs an extension of a special authorization to the service stream queue by transmitting symbols in the authorization using the modulation of each of the symbols, the modulation of each symbol being based on the service stream queue that issued the request. [9] Device (500) according to claim 8, wherein the scheduler process extends the special authorization to the service data stream queue using a Quality of Service (QoS) hierarchy structure, according to a set of weights and priorities specified therein. [10] Device (500) according to claim 8, where passing a request up a hierarchy of scheduling elements to a scheduler process to schedule the packet for delivery includes: passing both a CIR request and an EIR request for this packet up a hierarchy of scheduling elements to a scheduler process to schedule the packet for delivery; where both the CIR request and the EIR request for the packet are pending at the same time; and the transfer of symbols in the authorization is carried out in different ways, based on whether the authorization has satisfied the CIR request or the EIR request. [11] Device (500) for scheduling packets where forwarding to Data Over Cable Service Interface Specification (DOCSIS) downstream channels is to be performed, comprising: one or more processors (504); and one or more non-transitory computer-readable storage media that store one or more sequences of instructions which, when executed, cause: Assign to a DOCSIS service stream a specific Quality of Service (QoS) class that defines common access control and scheduling parameters shared by all DOCSIS service streams assigned to it; Assign to the DOCSIS service data stream of a DCS, where the DCS represents either a single DOCSIS downstream channel or a bundling group of multiple downstream channels; Inserting a packet that needs to be forwarded on the DCS into a service stream queue, wherein packets stored in the service stream queue belong to the DOCSIS service stream; and where a scheduler process performs scheduling of the packet on the DCS, based on the access control and scheduling parameters defined by the specific QOS class. [12] Device (500) according to claim 11, wherein the scheduler process that performs the scheduling of the packet includes: that the scheduler process determines which QOS class, from a plurality of different QOS classes that can send packets on the DCS, should be the next one for sending packets on the DCS during a scheduling process. [13] Device (500) according to claim 12, wherein the scheduler process determines which QOS class, from the plurality of different QOS classes that can send packets on the DCS, should be the next one for sending packets on the DCS during a scheduling, based either on a WFQ or an SP solution approach. [14] Device (500) according to claim 11, wherein the special QOS class can be configured to define the common access control and scheduling parameters using both relative measures in the sense of percentages and absolute measures in the sense of bytes / unit of time.

Images