NETWORK OVERLOAD CONTROL SYSTEM

Abstract

This paper describes systems, computer program products, and procedures for advanced congestion control with multiple congestion indicators in a network environment. An example system might include an intelligent agent configured to learn congestion control policies. The agent can interact with real or simulated environments that replicate real-world benchmarks. Congestion indicators such as telemetry information, packet loss metrics, congestion notification packet rate, pause frame rate, port utilization metrics, and / or the like provide a comprehensive representation of the network's state, thus enabling the assessment of the network environment's congestion condition. The intelligent agent evaluates these conditions using a reward function to optimize network performance.The intelligent agent can then implement a behavioral policy in response to the detected overload indicators, thereby changing the overload state of the network environment.

Description

TECHNICAL AREA

[0001] Exemplary embodiments of the present invention relate to a system for controlling network congestion. BACKGROUND

[0002] Network congestion occurs in computer networks when a node receives traffic faster than it can process or transmit it. This imbalance between incoming and outgoing traffic leads to a buildup of data packets within the network node, resulting in delays, packet loss, and reduced service quality. The consequences of network congestion are significant for both network performance and user experience. Congestion can lead to increased latency, reduced data throughput, and decreased performance for applications that rely on real-time data transmission.

[0003] Traditional solutions for resolving network congestion typically rely on manually defined behaviors. These solutions are often tailored to function optimally under specific conditions, but they are not robust enough to handle changes such as the introduction of additional data flows, modifications to the network topology, or varying noise levels. Furthermore, these algorithms frequently require adjustments to meet specific user needs—some may prioritize bandwidth maximization, while others focus on latency minimization.

[0004] The applicant has identified a number of shortcomings and problems associated with overload control based on tunable parameters in a network environment. Many of these identified problems have been solved by developing solutions contained in embodiments of the present disclosure, many examples of which are described in detail herein. SUMMARY

[0005] Therefore, systems, procedures, and computer program products are provided for overload control in a network environment.

[0006] The invention is defined by the claims. To illustrate the invention, aspects and embodiments are described herein that may or may not fall within the scope of the claims.

[0007] One aspect introduces a congestion control unit for network congestion control. The congestion control unit is configured to: deploy an intelligent agent in a network environment; detect, using the intelligent agent, congestion indicators representing a congestion state of the network environment, where the congestion indicators include at least telemetry information associated with one or more network devices in the network environment; and implement, using the intelligent agent, a behavioral policy in response to the detected congestion indicators, thereby changing the congestion state of the network environment.

[0008] In some embodiments, implementing the behavioral policy further includes: determining, based on behavioral policy parameters, actions to be executed in response to the detected overload indicators; and executing the actions in the network environment.

[0009] In some embodiments, the overload control unit is further configured to: determine a reward associated with the implementation of the behavior policy; and iteratively update the behavior policy parameters to maximize a cumulative value of the reward.

[0010] In some embodiments, determining the reward further includes: executing a reward function for at least one subset of the detected overload indicators based on the corresponding subset of actions performed; and calculating a reward value for the subset of actions performed based on the executed reward function.

[0011] In some embodiments, the reward function for telemetry information associated with one or more network devices in the network environment includes the following: r(c1) = -(qlen * transmissionrate - target) 2 - (maximumutil - util) 2, where qlen is a queue length indicating the total size of data packets waiting in an output queue at each network device, where transmissionrate is a rate at which data packets are transmitted within the network environment, where target is a predefined value representing a desired product of qlen and transmissionrate, where maximumutil is a maximum potential port utilization indicating full bandwidth utilization, and where util is an actual port utilization associated with the one or more network devices, indicating network traffic as a percentage of the maximum potential bandwidth.

[0012] In some embodiments, qlen is the queue length, which specifies the total size of data packets waiting in the output queue of a network device at maximum congestion, and util is the current port utilization associated with a network device, which specifies the maximum bandwidth usage.

[0013] In some embodiments, the detected congestion indicator includes packet drop metrics, and the reward function for the packet drop metrics includes: r(c2) = -(packetdroprate) 2 + transmission rate, where packet drop rate includes the rate at which data packets are lost in the network environment.

[0014] In some embodiments, the packet loss metrics include at least one of OOO (Out-of-Order) NACKs (Negative Acknowledgements), three consecutive ACKs (Acknowledgements), or explicit and / or intentional loss indications.

[0015] In some embodiments, the detected overload indicator includes a pause frame rate, where the reward function for the pause frame rate includes: r(c3) = -(pauserate) 2 + transmission rate, where pause rate is the number of received pause frames.

[0016] In some embodiments, the detected congestion indicator includes a congestion notification packet rate, where the reward function for the congestion notification packet rate includes: r(c4) = -(CNPrate) 2 + transmission rate, where CNPrate is the number of received congestion notification packets.

[0017] In some embodiments, the congestion notification packet rate includes a congestion notification type, the congestion notification type being based on the network environment.

[0018] In some embodiments, the detected congestion indicator includes a port utilization metric associated with each target network device and a round-trip time associated with data packets sent and / or received by each target network device, and wherein the reward function for the port utilization metric includes: r(c5) = (networkportutilization) 2 + transmission rate - (RTTsample - targetRTT) 2 , where networkportutilization is a port utilization rate of each target network device, RTTsample is a measured sample of round-trip time associated with data packets sent and / or received by each target network device, and targetRTT is a predefined target value for round-trip time associated with data packets sent and / or received by each target network device.

[0019] In another aspect, a method for network congestion control is presented. This method comprises: deploying an intelligent agent in a network environment; using the intelligent agent, capturing congestion indicators that represent a congestion state of the network environment, where the congestion indicators include at least telemetry information associated with one or more network devices in the network environment; and using the intelligent agent, implementing a behavioral policy in response to the captured congestion indicators, thereby changing the congestion state of the network environment.

[0020] In some embodiments, implementing the behavioral policy further includes: determining, based on behavioral policy parameters, actions to be executed in response to the detected overload indicators; and executing the actions in the network environment.

[0021] In some embodiments, the method further includes: determining a reward associated with the implementation of the behavioral policy; and iteratively updating the behavioral policy parameters to maximize a cumulative value of the reward.

[0022] In some embodiments, determining the reward further includes: executing a reward function for at least one subset of the detected overload indicators based on the corresponding subset of actions performed; and calculating a reward value for the subset of actions performed based on the executed reward function.

[0023] In some embodiments, the reward function for telemetry information associated with one or more network devices in the network environment includes the following: r(c1) = -(qlen * transmissionrate - target) 2 - (maximumutil - util) 2, where qlen is a queue length indicating the total size of data packets waiting in an output queue at each network device, where transmissionrate is a rate at which data packets are transmitted within the network environment, where target is a predefined value representing a desired product of qlen and transmissionrate, where maximumutil is a maximum potential port utilization indicating full bandwidth utilization, and where util is an actual port utilization associated with the one or more network devices, indicating network traffic as a percentage of the maximum potential bandwidth.

[0024] In yet another aspect, a computer program product for network congestion control is presented. The computer program product comprises a non-volatile, machine-readable medium containing code configured to cause a device to: deploy an intelligent agent in a network environment; detect, by means of the intelligent agent, congestion indicators representing a congestion state of the network environment, wherein the congestion indicators include at least telemetry information associated with one or more network devices in the network environment; and implement, by means of the intelligent agent, a behavioral policy in response to the detected congestion indicators, thereby changing the congestion state of the network environment.

[0025] In some embodiments, the code for implementing the behavior policy further causes the device to: determine, based on behavior policy parameters, actions to be executed in response to the detected overload indicators, and execute the actions in the network environment.

[0026] In some embodiments, the code further causes the device to: determine a reward associated with the implementation of the behavioral policy; and iteratively update the behavioral policy parameters to maximize a cumulative value of the reward.

[0027] This paper describes systems, computer program products, and procedures for advanced congestion control using multiple congestion indicators in a network environment. An example system might include an intelligent agent configured to learn congestion control policies. The agent can interact with real or simulated environments that replicate real-world benchmarks. Congestion indicators such as telemetry information, packet loss metrics, congestion notification packet rate, pause frame rate, port utilization metrics, and / or similar data provide a comprehensive representation of the network's state, thus enabling the assessment of the network environment's congestion condition. The intelligent agent evaluates these conditions using a reward function to optimize network performance.The intelligent agent can then implement a behavioral policy in response to the detected overload indicators, thereby changing the overload state of the network environment.

[0028] Each feature of an aspect or embodiment can be applied to other aspects or embodiments in any suitable combination. In particular, each feature of a process aspect or embodiment can be applied to an equipment aspect or embodiment, and vice versa.

[0029] The above summary is intended only to provide an overview of some exemplary embodiments in order to give a basic understanding of some aspects of the present disclosure. Accordingly, it should be noted that the embodiments described above are merely examples and should not be interpreted as limiting the scope or nature of the disclosure in any way. It should be noted that the scope of the present disclosure includes many other potential embodiments in addition to those summarized here, some of which are described in more detail below. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Having described above certain exemplary embodiments of the present disclosure in general terms, reference is now made to the accompanying drawings. The components illustrated in the figures may or may not be present in certain embodiments described herein. Some embodiments may include fewer (or more) components than shown in the figures. Fig. Figure 1 illustrates an example of a system environment for advanced overload control according to an embodiment of the present invention; Fig. 2 illustrates an example of an overload control unit for overload control according to an embodiment of the present invention; Fig. Figure 3 illustrates an example of a network environment according to an embodiment of the invention; and Fig. Figure 4 shows an example of an advanced overload control method according to an embodiment of the invention. DETAILED DESCRIPTION Overview

[0031] In scenarios where multiple network devices, each equipped with a NIC (Network Interface Card), transmit data through a single switch toward a receiving server, network congestion can occur. Each NIC can transmit data at rates of up to 100 Gbps, resulting in a combined input rate to the switch of up to 400 Gbps. However, the switch, acting as the congestion point, typically has a maximum output rate of 100 Gbps. This imbalance between input and output rates can lead to network congestion. To address this, congestion control algorithms can be implemented to manage the data transmission rates of the NICs. Such algorithms can be used to adjust the transmission rates, thereby preventing congestion and ensuring efficient network operation while minimizing latency and packet loss.

[0032] Congestion control algorithms can utilize various network indicators to adjust the transmission rates of network interface cards (NICs). These indicators include dropped packet notifications, round-trip delay sampling, in-band flow telemetry (IFA), explicit congestion marking of data packets passing through a switch, and / or similar indicators. By using these congestion indicators, congestion control algorithms can gather comprehensive data about the congestion state of network paths, enabling more accurate calculations of transmission rates and improving the algorithm's ability to effectively manage and mitigate congestion.

[0033] Accordingly, embodiments of the invention introduce an advanced congestion control algorithm that utilizes multiple congestion indicators within the network environment. An exemplary system may include an algorithmic (e.g., reinforcement) learning agent that employs a deep neural network to learn congestion control policies. The agent can interact with a distributed training component and operate across various simulated environments that replicate real-world benchmarks and hardware. The simulation may include generating and collecting congestion indicators such as telemetry probes, packet loss indicators, congestion notification packets, received pause / transmit wait signals, and port utilization metrics. The collected congestion indicators can be used to form a comprehensive representation of the network's congestion state.The congestion state representation can include real-time network congestion metrics that the agent uses to understand current network conditions. The agent's actions can be evaluated based on a reward function that considers congestion indicators, such as minimizing queue lengths and bandwidth usage (from telemetry probes), reducing the frequency of packet loss indications, decreasing the number of congestion notification packets, limiting pauses and transmission wait times, and / or balancing port utilization metrics to ensure efficient use of network resources. The agent can employ a reinforcement learning algorithm, such as a policy gradient deep neural network, that uses the state representation and the reward function to update its policy.The policy dictates the agent's actions to optimize network performance and balances goals such as maximizing throughput, minimizing latency, and ensuring fairness.

[0034] Embodiments of the present disclosure are now described in more detail with reference to the accompanying drawings, which show some, but not all, embodiments of the present disclosure. In fact, the present disclosure can be implemented in many different forms and should not be interpreted as being limited to the embodiments set forth herein; rather, these embodiments are given so that the present disclosure meets the applicable legal requirements.Therefore, it is to be understood that each block of the block diagrams and flowchart illustrations can be implemented in the form of a computer program product, a fully hardware-based embodiment, a fully firmware-based embodiment, a combination of hardware, computer program products and / or firmware; and / or devices, systems, computing devices, computing units and / or the like, which execute instructions, operations, steps and similar terms used interchangeably (e.g., executable instructions, commands to execute, program code and / or the like) on a computer-readable storage medium for execution. For example, fetching, loading and executing code can be performed sequentially, such that one instruction is fetched, loaded and executed in turn.In some exemplary embodiments, fetching, loading, and / or executing can be performed in parallel, allowing multiple instructions to be fetched, loaded, and / or executed simultaneously. Thus, such embodiments can create specially configured machines that perform the steps or operations specified in the block diagrams and flowchart illustrations. Accordingly, the block diagrams and flowchart illustrations support various combinations of embodiments for executing the specified instructions, operations, or steps.

[0035] Where possible, all terms expressed herein in the singular form should also include the plural form, and vice versa, unless explicitly stated otherwise. Furthermore, the term "a" and / or "an" as used herein means "one or more," even when the phrase "one or more" is also used herein. When it is stated herein that something is "based" on something else, it may also be based on one or more other things. In other words, unless explicitly stated otherwise, "based on" as used herein means "at least partly based on" or "at least partially based on." The same numbers throughout refer to the same elements.

[0036] “Operationally coupled,” as used herein, can mean that the components are electronically or optically coupled and / or in electrical or optical communication with each other. Furthermore, “operationally coupled” can mean that the components are integrally formed or separately formed and coupled together. Additionally, “operationally coupled” can mean that the components are directly connected or connected via one or more components (e.g., connectors) located between the operationally coupled components. Finally, “operationally coupled” can mean that the components are detachable from each other or that they are permanently coupled.

[0037] As used herein, “interconnected” can imply that each component is directly or indirectly linked to every other component or switch in the network, enabling seamless data transfer and communication between all components.

[0038] As used herein, "determine" can encompass a variety of different actions. For example, "determine" can include calculating, computing, processing, deriving, investigating, ascertaining, and / or the like. Furthermore, "determine" can also include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), and / or the like. "Determine" can also include resolving, selecting, calculating, setting, and / or the like. Determining can also include ascertaining that a parameter meets a predetermined criterion, including that a threshold has been reached, exceeded, satisfied, etc.

[0039] It is understood that the word "exemplary" is used here in the sense of "serving as an example, case, or illustration." An implementation described herein as "exemplary" is not necessarily to be understood as advantageous compared to other implementations.

[0040] Furthermore, it is obvious to the average person skilled in the art, in view of the disclosure provided, that the terms “essentially” and “approximately” indicate that the referenced element or associated description is accurate within the applicable technical tolerances. Example system environment

[0041] Fig. Figure 1 shows an example of a system environment 100 for advanced overload control according to an embodiment of the present invention. As shown in Fig. As shown in Figure 1, the system environment 100 can include an overload control unit 102 and a network environment 104.

[0042] The overload control unit 102 can be operationally coupled with the network environment 104 and configured to manage and mitigate network congestion within the network environment 104, as described in relation to Fig. 2 is described in more detail. The congestion control unit 102 can collect congestion indicators 108 from the network environment 104. The congestion indicators 108 can include, but are not limited to, telemetry information, packet loss metrics, congestion notification packet rates, pause frame rates, and port utilization metrics. Based on the congestion indicators 108, the congestion control unit 102 can determine actions 106 to be performed to reduce congestion within the network environment 104. These actions 106 can include adjusting the transmission rates of NICs, rerouting traffic over alternative paths, temporarily pausing data transmission, reallocating network resources for more effective load balancing, and / or the like.By implementing these actions 106, the congestion control unit 102 can prevent data bottlenecks and ensure smooth data flow in the network environment 104. After executing the actions 106, the congestion control unit 102 can receive feedback 110 from the network environment 104. This feedback 110 can include updated congestion indicators and performance metrics, which the congestion control unit 102 can use to evaluate the effectiveness of the implemented actions 106. Based on this feedback 110, the congestion control unit 102 can modify future actions to continuously optimize network performance. This iterative process ensures that the congestion control unit 102 adapts to changing network conditions and maintains an optimal balance between data throughput and network stability.

[0043] The term "network environment 104" can refer to an integrated system configured to support data transmission across a variety of interconnected devices. This environment can encompass a wide range of network devices, including servers, switches, routers, and other embedded components such as network interface cards (NICs). These devices work together to facilitate seamless data exchange and ensure robust network performance and reliability. In addition to core network devices, the network environment 104 can also include various ancillary components such as firewalls, load balancers, and network management systems.Firewalls can protect the network from unauthorized access and cyber threats, while load balancers distribute network traffic evenly across servers to prevent overload and ensure high availability. Network management systems can provide real-time monitoring and analysis, allowing administrators to monitor network performance, identify potential problems, and implement corrective actions immediately.

[0044] The Network Environment 104 can also include endpoint devices that serve as user input devices, enabling users to access the Network Environment. These endpoint devices can include, but are not limited to, personal computers, laptops, tablets, smartphones, and other mobile devices. Such devices allow users to interact with the Network Environment 104 and access applications, data, and services hosted on the servers. These endpoint devices are typically equipped with their own NICs, which provide connectivity to the network via either wired connections such as Ethernet® or wireless connections such as Wi-Fi®. Endpoint devices can also include specialized input devices such as keyboards, mice, touchscreens, and other peripherals that enhance the user's interaction with the network.By providing various access points and input methods, the network environment 104 ensures that users can seamlessly connect to and utilize network resources from different locations and contexts, thereby supporting a wide range of user requirements, including network accessibility and ease of use.

[0045] It should be noted that the description of the network environment 104 contained herein is for illustrative purposes only and is not intended as a limitation. The scope of the network environment 104 is not limited to the specific devices, configurations, or applications discussed above. Variations and modifications are possible without deviating from the essence and scope of the invention. The network environment 104 may include additional components, configurations, and functions not expressly mentioned but falling within the general framework and objectives described. All equivalent implementations or adaptations that achieve substantially the same results as the disclosed embodiments are deemed to be included within the scope of the present invention.Furthermore, it should be understood that the structure of System Environment 100 and its components, connections, and relationships, as well as their functions, serve only as examples and are not intended to limit implementations of the disclosures described and / or claimed in this document. In one example, System Environment 100 may include more, fewer, or different components. In another example, some or all parts of System Environment 100 may be combined into a single part, or all parts of Environment 100 may be divided into two or more distinct parts. Example of an overload control unit circuit

[0046] Fig. Figure 2 illustrates an example of an overload control unit 102 for overload control according to an embodiment of the present invention. As shown in Fig. As shown in Figure 2, the overload control unit 102 can include a processor 112, a memory 114, an input / output circuit 116, a communication circuit 118 and a machine learning circuit 120.

[0047] Although the term “circuit” as used herein in reference to components 112–120 is in some cases described using functional language, it is to be understood that the particular implementations necessarily involve the use of specific hardware configured to perform the functions associated with the respective circuit as described herein. It is also to be understood that certain of these components 112–120 may include similar or common hardware. For example, two circuit sets may both use the same processor, network interface, storage medium, or the like to perform their associated functions, so that duplicate hardware is not required for each circuit set.In this context, it should be understood that some of the components described in connection with the overload control unit 102 may be housed together, while other components are housed separately (e.g., a controller communicating with the overload control unit 102). While the term "circuit" is to be understood broadly as including hardware, in some embodiments the term "circuit" may also include software for configuring the hardware. For example, in some embodiments, "circuit" may include processing circuitry, storage media, network interfaces, input / output devices, and the like. In some embodiments, other elements of the overload control unit 102 may provide or supplement the functionality of certain circuitry.For example, the processor 112 can provide processing functionality, the main memory 114 can provide storage functionality, the communication circuit 118 can provide network interface functionality, and so on.

[0048] In some embodiments, the processor 112 (and / or a coprocessor or any other processing circuitry supporting or otherwise associated with the processor) can communicate with the main memory 114 via a bus to pass information between components, for example, the overload control unit 102. The main memory 114 can be non-volatile and may, for example, comprise one or more volatile and / or non-volatile memory units or a combination thereof. In other words, the main memory 114 can, for example, be an electronic storage device (e.g., a non-volatile, computer-readable storage medium). The main memory 114 can be configured to store information, data, content, applications, instructions, or the like, to power a device, e.g., a computer.the overload control unit 102, to enable it to perform various functions according to exemplary embodiments of the present disclosure.

[0049] Although in Fig. While represented as a single memory unit, the memory unit 114 can comprise multiple memory components. These multiple memory components can be implemented on a single computing device or distributed across multiple computing devices. In various embodiments, the memory unit 114 can, for example, comprise a hard disk, random-access memory, cache memory, flash memory, a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), an optical disk, circuitry configured to store information, or a combination thereof. The memory unit 114 can be configured to store information, data, applications, instructions, or the like, enabling the overload control unit 102 to perform various functions according to the exemplary embodiments discussed herein.For example, in at least some embodiments, the main memory 114 can be configured to buffer data for processing by the processor 112. Additionally or alternatively, in at least some embodiments, the main memory 114 can be configured to store program instructions for execution by the processor 112. The main memory 114 can store information in the form of static and / or dynamic information. This stored information can be stored and / or used by the overload control unit 102 during the execution of its functions.

[0050] The Processor 112 can be configured in various ways and may, for example, include one or more processing units configured for independent operation. Additionally or alternatively, the Processor 112 may include one or more processors configured in tandem via a bus to enable independent instruction execution, pipelining, and / or multithreading.The processor 112, for example, can be implemented as various means, including one or more microprocessors with associated digital signal processors, one or more processors without an associated digital signal processor, one or more coprocessors, one or more multi-core processors, one or more controllers, processing circuitry, one or more computers, various other processing elements, including integrated circuits such as an ASIC (application-specific integrated circuit) or an FPGA (field-programmable gate array), or a combination thereof. The term "processing circuitry" can be understood to include a single-core processor, a multi-core processor, multiple processors within the device, and / or remote or "cloud" processors. Although in . Fig. While Figure 2 illustrates a single processor, the processor 112 may, in some embodiments, comprise multiple processors. These multiple processors may be configured on a single computing device or distributed across multiple such devices, which together are configured to function as the overload control unit 102. The multiple processors may communicate operationally with one another and may be configured together to perform one or more functions of the overload control unit 102 as described herein.

[0051] In one exemplary embodiment, the processor 112 can be configured to execute instructions stored in the main memory 114 or otherwise accessible to the processor 112. Alternatively or additionally, the processor 112 can be configured to execute hard-coded functions. Thus, regardless of whether it is configured by hardware or software methods, or by a combination thereof, the processor 112 can represent an entity (e.g., physically implemented in circuitry) capable of performing operations according to an embodiment of the present disclosure when configured accordingly. Alternatively, as a further example, if the processor 112 is configured as an executor of software instructions, the instructions can specifically configure the processor 112 to execute one or more algorithms and / or operations described herein when the instructions are executed.For example, when executed by processor 112, these instructions can cause the overload control unit 102 to perform one or more of its functions described here.

[0052] In some embodiments, the overload control unit 102 may further comprise input / output circuitry 116, which in turn may communicate with the processor 112 to provide an acoustic, visual, mechanical, or other output and / or, in some embodiments, to receive a display of input from a user or other source. In this sense, the input / output circuitry 116 may include means for performing analog-to-digital and / or digital-to-analog data conversions. For example, the input / output circuitry 116 may include support for a display, touchscreen, keyboard, mouse, image capture device (e.g., a camera), microphone, and / or other input / output mechanisms. The input / output circuitry 116 may include a user interface and may be a web interface, a mobile application, a kiosk, or the like.

[0053] The processor 112 and / or the user interface circuitry comprising the processor 112 can be configured to control one or more functions of a display or one or more user interface elements via computer program instructions (e.g., software and / or firmware) stored in a working memory accessible to the processor 112 (e.g., the working memory 114 and / or the like). In some embodiments, aspects of the input / output circuitry 116 can be reduced compared to embodiments in which the overload control unit 102 can be implemented as an end-user machine or other type of device designed for complex user interactions. In some embodiments (like other components described herein), the input / output circuitry 116 can be removed from the overload control unit 102.The input / output circuit 116 can be connected to the main memory 114, the communication circuit 118, and / or other components, for example via a bus. Although more than one input / output circuit and / or other component may be included in the overload control unit 102, it is stated in . Fig. 2 only one is shown, so as not to complicate the disclosure (e.g. as with the other components described herein).

[0054] In some embodiments, the communication circuit 118 comprises any means, such as a device or circuit, implemented either in hardware, software, firmware, or a combination thereof, and configured to receive and / or transmit data to / from a network and / or another associated device, circuit, or module. In this respect, the communication circuit 118 may, for example, include a network interface to enable communication with a wired or wireless communication network. For example, in some embodiments, the communication circuit 118 may be configured to receive and / or transmit any data that can be stored by the working memory 114 using any protocol that can be used for communication between computing devices.For example, the communication circuit 118 may include one or more communication ports, network interface cards, antennas, transmitters, receivers, buses, switches, routers, modems, and supporting hardware and / or software and / or firmware / software, or any other device suitable for enabling communication over a network. Additionally or alternatively, in some embodiments, the communication circuit 118 may include circuitry for interacting with the antenna(s) to effect the transmission of signals via the antenna(s) or to handle the reception of signals received via the antenna(s). These signals may be transmitted by the overload control unit 102 using any number of wireless PAN (Personal Area Network) technologies, such as Bluetooth® v1.0 to v5.0, BLE (Bluetooth Low Energy), wireless infrared transmission (e.g., infrared), or other wireless technologies.IrDA), UWB (Ultra-Wideband), wireless induction transmission, or the like. Furthermore, it is understood that these signals can be transmitted using Wi-Fi, NFC (Near Field Communications), WiMAX (Worldwide Interoperability for Microwave Access), or other proximity-based communication protocols. The communication circuit 118 can additionally or alternatively be connected to the working memory 114, the input / output circuit 116, and / or another component of the overload control unit 102, for example, via a bus. The communication circuit 118 of the overload control unit 102 can also be configured to receive and transmit information to and from the various associated components.

[0055] The congestion control unit 102 can also include a machine learning circuit 120 to enhance its ability to predict and manage network congestion. In this respect, the machine learning circuit 120 can include an intelligent agent 122. The intelligent agent 122 can employ advanced machine learning techniques to analyze the congestion state of the network environment 104 and make informed decisions to manage the congestion. The representation of the congestion state can be generated using real-time metrics derived from congestion indicators (e.g., congestion indicators 108 in Fig. 1) are obtained, such as telemetry information, packet loss metrics, congestion notification packet rate, pause frame rate, and port utilization metrics. The intelligent agent 122 can use this representation of the congestion state to accurately understand the current conditions of the network environment 104 and subsequently take actions (e.g., actions 106 in Fig. 1) to implement in order to change the congestion state of network environment 104 (e.g. to reduce the congestion within network environment 104).

[0056] The intelligent agent 122 can include a behavioral policy 124, which is embedded within the intelligent agent 122 and serves as a decision framework specifying the actions (e.g., actions 106) that the intelligent agent 122 can perform in response to the representation of the congestion state. The behavioral policy 124 can be formulated based on the analysis of the representation of the congestion state, which is generated using congestion indicators. The behavioral policy 124 can assign specific actions to the observed congestion state, aimed at managing and mitigating network congestion. The intelligent agent 122 can be configured to be adaptive and continuously learn and update its behavioral policy 124 based on real-time feedback (e.g., feedback 110) from the network environment 104.

[0057] In one example, the intelligent agent 122 can use a deep reinforcement learning framework that combines deep learning with reinforcement learning principles. The deep learning component, typically implemented as a deep neural network (DNN), processes high-dimensional input data from the network environment 104 (e.g., congestion indicators 108) to extract relevant features and create a comprehensive representation of the congestion state of the network environment 104. The reinforcement learning component can enable the intelligent agent 122 to optimize its behavioral policy 124 by trial and error, guided by a reward function that evaluates the effectiveness of its actions (e.g., actions 106). The reward function can consider various metrics, such as...Minimizing queue lengths, reducing packet loss, decreasing congestion notification packets, balancing port load, and / or similar actions. By iteratively refining its behavior policy (e.g., behavior policy 124) based on received rewards, the intelligent agent 122 can learn to perform actions to reduce congestion and improve network performance over time.

[0058] The intelligent agent 122 can be trained by deploying it in various real-world network environments and / or simulated network environments (not shown) that replicate real-world conditions and congestion scenarios and are collectively referred to as training environments. These training environments can be configured to expose the intelligent agent 122 to a wide range of network states and congestion events, allowing it to gain diverse experience. Such training environments can include different network topologies, varying traffic loads, diverse congestion patterns, multiple network protocols, and / or the like. During the training phase, the intelligent agent 122 can interact with the training environments by performing actions based on an initial behavioral policy.The results of these actions can be observed, and intelligent agent 122 can receive feedback in the form of rewards or penalties. The reward function can evaluate the effectiveness of intelligent agent 122's actions by considering metrics such as queue lengths, packet loss rates, congestion notification packet rates, pause frame rates, and port utilization. Positive rewards can be given for actions that improve the performance of the training environments, while negative rewards can be given for actions that worsen congestion. As intelligent agent 122 gains experience through its interactions with the training environments, it can update its behavioral policy to improve its decision-making capabilities.The process of updating the behavioral policy can involve adjusting the parameters of the underlying machine learning model, such as the weights of a deep neural network in the case of deep reinforcement learning. The intelligent agent 122 can use algorithms like policy gradients or Q-learning to iteratively refine the behavioral policy 124 based on the rewards received, thereby progressively improving its ability to manage overload. The training process can be iterative and continuous, allowing the intelligent agent 122 to learn from a large set of scenarios and gradually develop a robust behavioral policy (e.g., behavioral policy 124). The performance of the intelligent agent 122 can be regularly evaluated in the training environments to ensure progress toward optimal overload management.In some cases, the training environments can be adjusted as needed to introduce new challenges or to focus on specific areas where the performance of the intelligent agent 122 needs improvement.

[0059] Once the intelligent agent 122 has demonstrated sufficient competence in the training environments (e.g., based on benchmarks and KPIs such as reduced average queue lengths, reduced packet loss rates, improved overall network throughput, and consistent low latency maintenance), it can be deployed in a live network environment (e.g., network environment 104). Even after deployment, the intelligent agent 122 can continue to learn and adapt its behavior policy 124 based on real-time feedback from the network environment (e.g., network environment 104) in which it is deployed. This continuous learning process ensures that the intelligent agent 122 remains effective at congestion management under changing network conditions and evolving traffic patterns.

[0060] It is important to understand that the intelligent agent 122 is not limited to the use of a deep reinforcement learning framework. Within the scope of the present invention, any machine learning model capable of analyzing network state data and making informed decisions to manage congestion can be used. Examples of such models include, but are not limited to, supervised learning models, unsupervised learning models, other reinforcement learning models, decision tree-based models, support vector machines, Bayesian networks, and ensemble learning methods. Furthermore, the intelligent agent 122 can use any combination of these models to optimize its decision-making process.

[0061] In some embodiments, the overload control unit 102 may comprise hardware, software, firmware, and / or a combination of such components configured to support various aspects of the machine learning circuit 120 as described herein. It should be noted that in some embodiments, the machine learning circuit 120 may perform one or more of these exemplary actions in combination with another circuit of the overload control unit 102, such as the memory 114, the processor 112, the input / output circuit 116, and / or the communication circuit 118. For example, in some embodiments, the machine learning circuit 120 may use the processing circuit, such as the processor 112 and / or the like, to form a self-contained subsystem for performing one or more of its corresponding operations.In another example and in some embodiments, some or all functions of the machine learning circuit 120 can be executed by the processor 112. In this respect, some or all of the exemplary processes and algorithms discussed herein can be executed by at least one processor 112 and / or the machine learning circuit 120. It should also be noted that in some embodiments, the machine learning circuit 120 may include a separate processor, a specially configured FPGA, or an ASIC for executing its respective functions. Additionally or alternatively, in some embodiments, the machine learning circuit 120 may use the main memory 114 to store collected information.For example, in some implementations the machine learning circuitry may include hardware, software, firmware and / or a combination thereof, which interacts with the main memory to send, retrieve, update and / or store data.

[0062] Accordingly, a non-volatile, computer-readable storage medium, which may be, for example, the main memory 114, can be configured to store firmware, one or more application programs, and / or other software containing instructions and / or other computer-readable program code segments that can be executed to control the operation of the overload control unit 102 to perform various operations, including the examples described herein. As such, a set of computer-readable program code segments can be designed in one or more computer program products and used with a device, an overload control unit 102, a database, and / or other programmable device to generate the machine-implemented processes discussed herein.It is also noted that all or some of the information discussed herein may be based on data received, generated, and / or maintained by one or more components of the overload control unit 102. In some embodiments, one or more external systems (such as a remote cloud computing and / or data storage system) may also be used to provide at least some of the functions discussed herein.

[0063] It should be noted that the structure of the overload control unit 102 described herein represents only one embodiment among a multitude of possible configurations. This particular structure of the overload control unit 102 is described to demonstrate a specific arrangement and interaction of its components, which collectively contribute to its comprehensive network capabilities. However, this outlined configuration is not definitive or limiting. Variations and modifications to the configuration, connection, and interaction of the components within the overload control unit 102 may be made without departing from the scope of the invention. The described embodiment serves only for illustrative purposes, and all equivalent structures or methods that perform essentially the same function as the described embodiment are considered to be included within the scope of the present invention.The attached claims are intended to cover such variations and modifications as would occur to a person skilled in the art. Example network environment

[0064] Fig. Figure 3 illustrates an example of a network environment 104 according to an embodiment of the invention. As shown in Fig. As shown in Figure 3, the network environment 104 can include several transmitting NICs (Network Interface Cards), NIC_1 202, NIC_2 204, NIC_3 206 and NIC_4 208, a switch, SWITCH_1 210, a buffer 212 and a receiving NIC, NIC_5 214.

[0065] NICs can be specialized components embedded in various network devices such as servers, workstations, and networked storage devices, and are responsible for facilitating data transmission and reception in the network environment. In an exemplary embodiment, NIC_1 202, NIC_2 204, NIC_3 206, and NIC_4 208 can be transmitting NICs embedded in different servers or other network devices, each capable of transmitting data at a rate of up to 100 Gbit / s. The transmitting NICs can operate in tandem to send data via SWITCH_1 210 to the receiving NIC, NIC_5 214.

[0066] SWITCH_1 210 can act as an intermediary device that manages the data flow. SWITCH_1 210 can receive data from the transmitting NICs, NIC_1 202, NIC_2 204, NIC_3 206, and NIC_4 208, and forward the data to the appropriate destination, for example, the receiving NIC, NIC_5 214. In certain embodiments, SWITCH_1 210 can use a buffer 212 to temporarily store data packets as needed to manage traffic and prevent data loss. In one exemplary embodiment, SWITCH_1 210 can have a combined input capacity of up to 400 Gbit / s from the four transmitting NICs, but its maximum output rate may be limited to 100 Gbit / s. Such a configuration can create a potential congestion point within the network environment 104.

[0067] It is understandable that the in Fig. The illustrated network environment 104 is for illustrative purposes only and is not intended to limit the scope of the invention. The specific configuration and components shown, including the transmitting NICs (NIC_1 202, NIC_2 204, NIC_3 206, and NIC_4 208), the switch (SWITCH_1 210), the buffer 212, and the receiving NIC (NIC_5 214), represent only one of many possible embodiments of the network environment. Variations in the arrangement, number, and types of components used in the network environment 104 are possible without deviating from the essence and scope of the invention. Alternative configurations may include additional or different network devices, varying numbers of NICs, or alternative methods for handling data flow and congestion. Exemplary procedure for advanced overload control

[0068] Fig. Figure 4 illustrates an exemplary method for advanced congestion control according to an embodiment of the invention. As shown in block 302, an intelligent agent can be deployed in a network environment. The intelligent agent can be used within the network environment to monitor and manage network congestion. As described herein, the intelligent agent can employ advanced machine learning techniques to analyze the congestion state of the network environment and make informed decisions regarding congestion management.

[0069] As shown in Block 304, congestion indicators can be collected, representing the congestion state of the network environment. Once deployed, the intelligent agent can collect congestion indicators within the network environment. Congestion indicators can be various metrics and signals that reflect the current traffic and congestion state within a network environment. These indicators can include telemetry information, packet loss metrics, congestion notification packet rates, pause frame rates, port utilization metrics, and / or similar data.

[0070] Telemetry information can relate to a range of signals emanating from network devices (e.g., SWITCH_1 210 in Fig. 3) can be extracted and observed only by these devices. Examples of telemetry information may include port utilization metrics, queue lengths, queue delays, link throughput, estimated number of concurrent flows, packet drop counters, flow control pause counters, congestion topology location, and / or the like.

[0071] Port utilization metrics relate to how many ports of a given network device are in use at any given time. Port utilization can provide a snapshot of the network device's activity level and capacity utilization. High port utilization can indicate that a significant portion of the network device's ports are active, which may point to potential congestion issues. Monitoring port utilization can identify bottlenecks where too many ports are in use simultaneously, potentially exceeding the network device's processing capacity and leading to congestion. Port utilization can be measured in various ways, including the number of active ports relative to the total number of available ports, the bandwidth consumed by each port, and the data throughput processed by the network device.Thus, port utilization can provide a comprehensive overview of network resource usage and serve as a basis for decisions regarding scaling, traffic management, and QoS (Quality of Service) policies.

[0072] Queue metrics can refer to the total size of data packets waiting in the input queue of a particular network device. In implementations where data transmission encounters multiple network devices, the queue metric can be a combined value of the total size of data packets waiting in each queue of every network device. The combined value can be determined using methods such as maximum, average, median, or other statistical measures. For example, the maximum queue metric can reflect the size of the largest queue along the data transmission path, thus providing insight into potential bottlenecks. The average queue metric can offer an overall perspective on network congestion by calculating the average of the sizes of all queues.The median queue metric, on the other hand, can represent the mean of the queue sizes, thereby reducing the impact of outliers and providing a more robust measure of central tendency. Other methods for determining the combined queue metric include weighted averages, where certain network devices can be assigned higher significance due to their role or importance within the network. Other techniques, such as moving averages or exponential smoothing, can be used to account for temporal variations and enable a dynamic assessment of network conditions.

[0073] Queue delay refers to the time data packets spend in a queue before being processed by a network device. Queue delay metrics can provide insights into network congestion and performance by measuring latency at various points in the network. Queue delay can be monitored on individual network devices or aggregated across multiple devices to understand the overall delay experienced by data packets.

[0074] Link throughput, measured in bytes per second, represents the actual rate of data transfer across a network link. Unlike link utilization, which is expressed as a percentage, link throughput provides a quantifiable measure of the volume of successfully transferred data. Link throughput can be used to evaluate the efficiency and capacity of network links, identify potential bottlenecks, and optimize data flow. A flow can be defined by a 5-tuple that includes the source and destination IP addresses, source and destination ports, and protocol type. Monitoring the number of concurrent flows can reveal network utilization and usage patterns, aiding in capacity planning and the detection of anomalies or potential security threats.Packet drop counters can track the number of data packets dropped by network devices for various reasons, such as buffer overflow, errors, or policy enforcement. Flow control pause counters can record instances where flow control mechanisms are activated to temporarily halt data transmission. Congestion location topology information can identify where congestion occurs within the network topology, such as in the uplink relative to the downlink or at specific switch levels in a fat-tree topology. The format of this information can vary depending on the network topology and provides a detailed overview of congestion patterns, enabling targeted troubleshooting and optimization efforts.

[0075] Packet loss metrics measure the number of data packets lost or dropped while traversing the network. They can be used to assess network reliability and performance, as high packet loss rates can indicate problems such as network congestion, hardware failures, or suboptimal routing. Packet loss metrics can include one out-of-order (OOO) NACK (Negative Acknowledgement), three consecutive ACKs (Acknowledgements), explicit and / or intentional loss indications, and similar indicators. Out-of-order packets can refer to packets that arrive at their destination in a different order than the one in which they were sent. This can occur due to varying paths taken by packets or due to processing delays within network devices (e.g., due to network congestion).Out-of-order (OOO) packets can lead to retransmissions if the receiving system interprets them as lost, impacting network efficiency. Monitoring OOO packets can aid in diagnosing path variability issues and ensuring data sequence integrity. Non-lock-through errors (NACKs) refer to signals sent by the receiving end to indicate that a packet was not received correctly or was lost. NACKs can prompt the sender to retransmit the specific packet. A high number of NACKs can be a strong indicator of poor network performance, network congestion, instability, and / or similar issues. By analyzing NACK metrics, network administrators can identify patterns or specific conditions that lead to packet loss.In many transmission protocols, receiving three consecutive ACKs for the same packet can signal to the sender that a packet is likely lost. Three consecutive ACKs can trigger a retransmission of the presumably lost packet. Explicit and / or intentional loss indicators can occur when network devices intentionally discard packets for reasons such as congestion, policy enforcement, or prioritization of critical traffic. Explicit loss indicators can be valuable for understanding how network policies and configurations affect packet transmission. Intentional loss metrics can reveal whether the network is discarding packets as intended to maintain performance, or whether there are unintended side effects that lead to unnecessary packet loss.

[0076] Congestion notification packet rates refer to the rate at which congestion notification packets are sent to signal congestion. The rate at which these packets are sent can provide a clear indication of the degree of congestion. The congestion notification packet rate can include a congestion notification type, such as an Explicit Congestion Notification (ECN). ECNs can be specialized packets used by various network protocols to signal congestion. The congestion notification type can be based on the network environment. In other words, the specific type of congestion notification packet used, as well as its corresponding transmission rate, can be specific to the characteristics and operational requirements of the network.For example, a different ECN configuration might be used in a high-throughput environment than in a latency-sensitive network. Pause frame rates can be used to control data flow and prevent buffer overflows in various network environments (e.g., Ethernet networks). The frequency of pause frames can indicate the level of network congestion. Port utilization metrics at a receiving network device (e.g., NIC_5 214 in . Fig. 3) can measure the utilization rate of the network ports of the receiving network device, which shows how much of the available bandwidth is being used at any given time.

[0077] Other congestion indicators can include RTT (Round-Trip Time) and latency, IFA (In-Band Flow Analysis), queue lengths and buffer occupancy, and / or similar metrics. RTT and latency measure the time it takes for a data packet to travel from its source to its destination and back. Elevated RTT and latency values ​​often indicate network congestion, as packets take longer to traverse the network due to queue delays. IFA can involve analyzing data packets as they move through the network to gather information about flow characteristics and performance. IFA can help identify congestion points and traffic patterns that contribute to network slowdowns. Queue lengths and buffer occupancy (e.g., buffer occupancy in buffer 212 in...) are examples of such indicators. Fig. 2) They can provide information about congestion levels. Long queues and high buffer occupancy generally indicate that the network is struggling to efficiently handle traffic. Congestion indicators can be used to create a comprehensive picture of the network environment's congestion state, which can be used to diagnose network performance problems and as a basis for decisions regarding congestion reduction.

[0078] As shown in Block 306, a behavioral policy can be implemented in response to the captured congestion indicators, thereby changing the congestion state of the network environment. As discussed here in relation to Fig.As described in section 2, the behavioral policy can be configured to serve as a decision framework, specifying the actions to be taken in response to the detected overload indicators. In one exemplary embodiment, the actions to be taken can be determined based on behavioral policy parameters. These parameters can refer to specific rules, criteria, or models associated with the behavioral policy and define how the intelligent agent interprets the overload indicators and determines appropriate actions to manage and mitigate overload.

[0079] The behavioral policy parameters can be fine-tuned and updated during the intelligent agent's training phase. As described herein, the intelligent agent can be trained by deploying it in both real network environments and simulated environments that replicate real-world conditions and congestion scenarios. These training environments can be configured to expose the intelligent agent to a wide range of network states and congestion events, allowing it to gain diverse experience. During training, the intelligent agent can interact with these training environments by performing actions based on an initial behavioral policy with an initial set of behavioral policy parameters.As the intelligent agent gains experience through its interactions with the training environments, it can update its behavior policy and behavior policy parameters to improve its decision-making capabilities.

[0080] Implementing a behavioral policy can involve determining, based on policy parameters, actions to be taken in response to detected congestion indicators, as well as executing these actions within the network environment. The implementation process may include analyzing the detected congestion indicators to determine the current state of the network environment. Based on this analysis, the policy parameters can facilitate the determination of specific actions to manage and mitigate congestion. These actions may include adjusting network resource allocation, modifying packet routing paths, prioritizing certain network traffic types, employing congestion control mechanisms, and / or similar measures.The intelligent agent can continuously monitor the effectiveness of these actions and update its behavioral policy in real time based on feedback from the network environment. This adaptive approach allows the intelligent agent to dynamically respond to network changes, maintain optimal performance, and prevent overload from impacting service quality. The ongoing learning process enables the intelligent agent to refine its decision-making framework.

[0081] As shown in Block 308, a reward can be set for implementing the behavioral policy. Determining the reward can involve executing a reward function for at least one subset of the collected congestion indicators (e.g., telemetry information, packet loss metrics, and / or the like) based on the corresponding subset of actions performed. This process can include selecting relevant congestion indicators that reflect network performance aspects and applying a predefined reward function to evaluate the impact of the actions performed on these indicators. Such an evaluation can measure the extent to which each action has affected network congestion, either by improving or worsening it.

[0082] In one example, the reward function for telemetry information (c1) can be = -(qlen * transmissionrate - target)2 - (maximumutil - util) 2In one aspect, `qlen` can be the queue length, indicating the total size of data packets waiting in an output queue at each network device. In another aspect, `qlen` can be the queue length, indicating the total size of data packets waiting in the output queue of a network device experiencing maximum congestion. `transmissionrate` can be the rate at which data packets are transmitted within the network environment. `target` can be a predefined value representing a desired product of `qlen` and `transmissionrate`. `maximumutil` can be the maximum potential port utilization, indicating full bandwidth usage. In another aspect, `util` can be the current port utilization associated with the one or more network devices, indicating network traffic as a percentage of the maximum potential bandwidth.In another aspect, util can be the current port utilization associated with a network device, indicating the maximum bandwidth usage.

[0083] In another example, the reward function for packet loss metrics can be r(c2) = -(packetdroprate) 2 + transmission rate. Here, packet drop rate can include the rate at which data packets are lost in the network environment. In another example, the reward function for pause frame rate can be r(c3) = -(pause rate). 2 + transmission rate. Here, pause rate can be the number of received pause frames. In further examples, the reward function for congestion notification packet rate can be r(c4) = -(CNPrate) 2+ transmission rate. Here, CNPrate can be a number of received congestion notification packets. In further examples, the reward function for the port utilization metric associated with each destination network device, and a round-trip time associated with data packets sent and / or received by each destination network device, can be r(c5) = (networkportutilization) 2 + transmission rate - (RTTsample - targetRTT) 2 Here, networkportutilization can be a port utilization rate of each target network device; RTTsample can be a measured sample of round-trip time associated with data packets sent and / or received by each target network device; and targetRTT can be a predefined target value for round-trip time associated with data packets sent and / or received by each target network device.

[0084] By calculating the reward value for the congestion indicators, the observed effects of actions taken using the behavioral policy can be translated into a numerical reward value, thus providing a quantifiable metric for evaluating the effectiveness of actions in reducing network congestion. The reward value derived in this way can empower network administrators to make informed decisions for optimizing network management strategies.

[0085] As shown in Block 310, behavioral policy parameters can be iteratively updated to maximize a cumulative reward value. Iteratively updating the behavioral policy parameters can involve continuously adjusting the parameters that govern the network's behavior based on feedback obtained from reward values. Each iteration can utilize reward values ​​calculated based on the effectiveness of previously executed actions, as described herein, to refine and optimize the policy parameters. The iterative update process can employ optimization algorithms such as gradient descent or other machine learning techniques to evaluate the current behavioral policy parameters against the observed rewards.These algorithms can identify potential improvements by analyzing the impact of actions taken and adjusting parameters to enhance performance. The primary goal of this process can be to progressively maximize the reward for each action and ensure the network continuously learns and adapts to optimize its performance. Through repeated iterations, the network's behavioral policy can become more adept at managing traffic, reducing congestion, and improving data transmission efficiency. The cumulative reward can reflect the overall success of these strategies and lead the system to sustained improvements in network management and performance.

[0086] The reward functions provided herein are presented as examples only and are not intended to be exhaustive or limiting. It is expressly acknowledged that other reward functions may be used in accordance with the principles set forth herein, and such alternative reward functions also fall within the scope of this disclosure. Accordingly, the specific examples given should not be interpreted as limiting the broad applicability of the reward function framework to other network metrics or performance indicators that the average professional might consider appropriate.

[0087] The person skilled in the art, to whom the embodiments set forth herein relate, will, taking into account the teachings set forth in the preceding descriptions and the associated drawings, be able to conceive of many modifications and other embodiments of the present disclosure. Although the figures show only certain components of the methods and systems described herein, it is understood that various other components may also be part of the disclosure herein. Furthermore, the method described above may in some cases comprise fewer steps, while in other cases it may comprise additional steps. The steps and modifications of the steps of the method described above may, in some cases, be carried out in any order and in any combination.

[0088] Therefore, it should be understood that the present disclosure is not limited to the specific embodiments disclosed and that modifications and other embodiments are to be included within the scope of the appended claims. Although specific terms are used here, they are used only in a generic and descriptive sense and not for the purpose of limitation.

[0089] It is understood that the aspects and embodiments described above serve only as examples and that detailed modifications can be made within the scope of the claims.

[0090] Each device, method and feature disclosed in the description and (where applicable) in the claims and drawings can be provided independently or in any suitable combination.

[0091] The reference numbers listed in the claims are for illustrative purposes only and do not restrict the scope of the claims.

Claims

[1] Congestion control unit for network congestion control, wherein the congestion control unit is configured to: Deploying an intelligent agent in a network environment; Detect, by means of the intelligent agent, congestion indicators that represent a congestion state of the network environment, wherein the congestion indicators include telemetry information associated with at least one or more network devices in the network environment; and Implement, using the intelligent agent, a behavioral policy in response to the detected congestion indicators, thereby changing the congestion state of the network environment. [2] Overload control unit according to claim 1, wherein implementing the behavioral policy further comprises: Determine, based on behavioral policy parameters, actions to be taken in response to the recorded overload indicators; and Executing actions in the network environment. [3] Overload control unit according to claim 2, wherein the overload control unit is further configured to: Determining a reward associated with the implementation of the behavioral guideline; and Iteratively updating the behavioral policy parameters to maximize the cumulative value of the reward. [4] Overload control unit according to claim 3, wherein determining the reward further comprises: Executing a reward function for at least one subset of the recorded overload indicators based on the corresponding subset of actions performed; and Calculating a reward value for the subset of performed actions based on the executed reward function. [5] Overload control unit according to claim 4, wherein the reward function for the telemetry information associated with one or more network devices in the network environment comprises the following: r(c1)=−(qlen∗transmissionrate−target)2−(maximumutil−util)2 where qlen is a queue length that specifies the total size of data packets waiting in an output queue at each network device, where transmissionrate is a rate at which data packets are transmitted within the network environment, where target is a predefined value that represents a desired product of qlen and transmissionrate, where maximumutil is a maximum potential port utilization that specifies full bandwidth utilization, and where util is an actual port utilization associated with the one or more network devices and specifies network traffic as a percentage of the maximum potential bandwidth. [6] Congestion control unit according to claim 5, wherein qlen is the queue length indicating the total size of the data packets waiting in the output queue of a network device experiencing maximum congestion, and wherein util is the current port utilization associated with a network device indicating maximum bandwidth utilization. [7] Overload control unit according to claim 5 or 6, wherein the detected overload indicator comprises packet loss metrics and wherein the reward function for the packet loss metrics comprises: r(c2)=−(packetdrop rate)2+transmission rate, where packet drop rate encompasses the rate at which data packets are dropped into the network environment. [8] Overload control unit according to claim 7, wherein the packet loss metrics include at least one OOO (Out-of-Order) NACKs (Negative Acknowledgements), three consecutive ACKs (Acknowledgements), or explicit and / or intentional loss indications. [9] Overload control unit according to one of claims 5-8, wherein the detected overload indicator comprises a pause frame rate, wherein the reward function for the pause frame rate comprises: r(c3)=−(pause rate)2+transmission rate where pauserate is the number of received pause frames. [10] Overload control unit according to one of claims 5-9, wherein the detected overload indicator comprises overload notification packet rate, and wherein the reward function for the overload notification packet rate comprises: r(c4)=−(CNPrate)2+transmission rate, where pauserate is the number of received congestion notification packets. [11] Congestion control unit according to claim 10, wherein the congestion notification packet rate comprises a congestion notification type, the congestion notification type being based on the network environment. [12] Congestion control unit according to any one of claims 5-11, wherein the detected congestion indicator comprises a port utilization metric associated with each target network device and a round-trip time associated with data packets transmitted and / or received by each target network device, and wherein the reward function for the port utilization metric comprises: r(c5) = (networkportutilization) 2 + transmission rate - (RTTsample - targetRTT) 2, where networkportutilization is a port utilization rate of each target network device, RTTsample is a measured sample of a round-trip time associated with data packets transmitted and / or received by each target network device, and targetRTT is a predefined target value for the round-trip time associated with data packets transmitted and / or received by each target network device. [13] Methods for network congestion control, the method comprising: Deploying an intelligent agent in a network environment; Detect, by means of the intelligent agent, congestion indicators that represent a congestion state of the network environment, wherein the congestion indicators include at least telemetry information associated with one or more network devices in the network environment; and Implement, using the intelligent agent, a behavioral policy in response to the detected congestion indicators, thereby changing the congestion state of the network environment. [14] Method according to claim 13, wherein implementing the code of conduct further comprises: Determine, based on behavioral policy parameters, actions to be taken in response to the recorded overload indicators; and Executing actions in the network environment. [15] The method of claim 14, wherein the method further comprises: Determining a reward associated with the implementation of the behavioral guideline; and Iteratively updating the behavioral policy parameters to maximize the cumulative value of the reward. [16] The method of claim 15, wherein determining the reward further comprises: Executing a reward function for at least one subset of the recorded overload indicators based on the corresponding subset of actions performed; and Calculating a reward value for the subset of performed actions based on the executed reward function. [17] Method according to claim 16, wherein the reward function for the telemetry information associated with one or more network devices in the network environment comprises the following: r(c1)=−(qlen∗transmissionrate−target)2−(maximumutil−util)2 where qlen is a queue length that specifies the total size of data packets waiting in an output queue at each network device, where transmissionrate is a rate at which data packets are transmitted within the network environment, where target is a predefined value that represents a desired product of qlen and transmissionrate, where maximumutil is a maximum potential port utilization that specifies full bandwidth utilization, and where util is an actual port utilization associated with the one or more network devices that specifies network traffic as a percentage of the maximum potential bandwidth. [18] Computer program product for network congestion control, wherein the computer program product comprises a non-volatile, machine-readable medium containing code configured to cause a device to: Deploying an intelligent agent in a network environment; Detect, by means of the intelligent agent, congestion indicators that represent a congestion state of the network environment, wherein the congestion indicators include at least telemetry information associated with one or more network devices in the network environment; and Implement, using the intelligent agent, a behavioral policy in response to the detected congestion indicators, thereby changing the congestion state of the network environment. [19] Computer program product according to claim 18, wherein the code for implementing the code of conduct further causes the device to: Determine, based on behavioral policy parameters, actions to be taken in response to the recorded overload indicators; and Executing actions in the network environment. [20] Computer program product according to claim 19, wherein the code further causes the device to: Determining a reward associated with the implementation of the behavioral guideline; and Iteratively updating the behavioral policy parameters to maximize the cumulative value of the reward.

Images