Bandwidth allocation method and device, electronic equipment and nonvolatile storage medium

Abstract

The invention discloses a bandwidth allocation method and device, electronic equipment and a nonvolatile storage medium. The method comprises the following steps: acquiring a network data link bandwidth of a target network cluster at a first moment; the shared data link is divided into a plurality of different data planes based on the service requirements of different network flows of the target network cluster, each data plane corresponds to the service requirement of one network flow, and the shared data link is a network data link commonly used by all network flows in the target network cluster; and performing network data link bandwidth allocation on each data plane based on a token bucket algorithm and the network data link bandwidth at the first moment, the token bucket algorithm being used for adjusting the network data link bandwidth of each data plane. According to the method and the device, the technical problems that the key service has no response and bandwidth resources cannot be fully utilized due to the fact that network traffic of multiple services mixedly uses a shared data link in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the field of computer network technology, and in particular to a bandwidth allocation method, device, electronic device and non-volatile storage medium. Background Art

[0002] Network traffic management is the key to ensure the stability and performance of network clusters. In the architectural design of network clusters (such as Kubernetes, referred to as K8s), multiple network data planes usually share the same physical data link for communication. When multiple business traffic uses the same physical path, such as high-frequency interaction of internal services, storage synchronization of large amounts of data, frequent external load balancing requests, and high-speed forwarding of external data, these business activities may increase sharply in a short period of time, resulting in data congestion on the data link. Data congestion will not only reduce network transmission efficiency and fail to fully utilize bandwidth resources, but also have a negative impact on the stability of the network cluster. In particular, when key business traffic is not processed in a timely manner, it may lead to a decline in the system service level agreement (SLA) and quality of service (QoS), or even unresponsiveness of key business services, which greatly threatens the operational efficiency and user experience of the network cluster.

[0003] Therefore, there are problems in the related art such as non-responsiveness of key services and inability to fully utilize bandwidth resources due to the mixed use of a common data link by network traffic of multiple services.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a bandwidth allocation method, device, electronic device and non-volatile storage medium to at least solve the technical problems in the related art of non-responsiveness of key services and inability to fully utilize bandwidth resources due to mixed use of a common data link by network traffic of multiple services.

[0006] According to one aspect of an embodiment of the present application, a method for bandwidth allocation is provided, including: obtaining the network data link bandwidth of a target network cluster at a first moment; dividing a common data link into multiple different data planes based on the business needs of different network traffic of the target network cluster, wherein each data plane corresponds to a business need of a network traffic, and the common data link is a network data link commonly used by all network traffic in the target network cluster; allocating network data link bandwidth to each data plane based on a token bucket algorithm and the network data link bandwidth at the first moment, wherein the token bucket algorithm is used to adjust the network data link bandwidth of each data plane.

[0007] In some embodiments of the present invention, the shared data link is divided into the following data planes: a storage data plane, an internal communication data plane, a load balancing network data plane, and a data forwarding network data plane, wherein the storage data plane is used for communication within the internal storage area network of the target network cluster, the internal communication data plane is used for communication of internal data of the target network cluster, the load balancing network data plane is used for communication between services deployed in the target network cluster and external clients, and the data forwarding network data plane is used to forward service requests from external clients to access the target network cluster.

[0008] In some embodiments of the present application, network data link bandwidth is allocated to each data plane based on a token bucket algorithm and the network data link bandwidth at a first moment, including: allocating a speed limit token to each data plane based on the network data link bandwidth at the first moment, wherein the speed limit token is used to control the network data link bandwidth of the data plane, wherein the speed limit token is an identifier of the network data link bandwidth of the data plane controlled in the token bucket algorithm, and each speed limit token represents a preset transmission amount; and adjusting the network data link bandwidth of each data plane based on the number of speed limit tokens.

[0009] In some embodiments of the present application, the network data link bandwidth of each data plane is adjusted based on the number of speed limit tokens, including: when the number of first speed limit tokens of the first data plane is less than or equal to a first preset threshold, obtaining the number of second speed limit tokens of the second data plane, wherein the first data plane is any data plane and the second data plane is any data plane different from the first data plane; when the number of second speed limit tokens is greater than the first preset threshold, allocating part of the speed limit tokens in the second speed limit tokens to the first data plane; and adjusting the network data link bandwidth of the first data plane based on the allocated part of the speed limit tokens.

[0010] In some embodiments of the present application, the method also includes: when the number of second speed limit tokens is less than or equal to a first preset threshold, reallocating part of the speed limit tokens allocated by the second data plane to the first data plane to the second data plane; and adjusting the network data link bandwidth of the second data plane based on the part of the speed limit tokens reallocated to the second data plane.

[0011] In some embodiments of the present invention, the method also includes: for each data plane, dividing the data bandwidth of each data plane into multiple subclasses, each subclass represents a single data link, and for the single data link of each subclass, limiting the traffic speed in the following manner: limiting the traffic of the single data link of each subclass based on the quintuple: obtaining the quintuple information of the target data stream, wherein the quintuple information includes the source IP, destination IP, source port, destination port, and communication protocol type; based on the quintuple information, mapping the target data stream to the corresponding target subclass; limiting the traffic speed of the target subclass based on the configuration information, wherein the configuration information contains the traffic speed limit rules for all subclasses.

[0012] In some embodiments of the present application, the method also includes: when the number of data links in the third data plane exceeds a second preset threshold, obtaining the priority of the data link in the third data plane and the network data link bandwidth requirement of each data link, wherein the third data plane is any data plane; in each time slice period, allocating time slices to the data links according to the priority of the data links, wherein the time slices are used to allocate periodic network data link bandwidth usage rights to the data links; allocating bandwidth that meets the network data link bandwidth requirement to the data links that obtain the time slices, and allocating the minimum bandwidth to the data links that do not obtain the time slices.

[0013] In some embodiments of the present application, the method also includes, according to another aspect of the embodiments of the present application, a bandwidth allocation device is also provided, including: an acquisition module, used to obtain the network data link bandwidth of the target network cluster at the first moment; a division module, used to divide the common data link into multiple different data planes based on the business needs of different network traffic of the target network cluster, wherein each data plane corresponds to a business need of a network traffic, and the common data link is a network data link commonly used by all network traffic in the target network cluster; a bandwidth allocation module, used to allocate network data link bandwidth to each data plane based on a token bucket algorithm and the network data link bandwidth at the first moment, wherein the token bucket algorithm is used to adjust the network data link bandwidth of each data plane.

[0014] According to another aspect of an embodiment of the present application, a non-volatile storage medium is provided, in which a program is stored, wherein when the program is running, a device where the non-volatile storage medium is located is controlled to execute the above-mentioned bandwidth allocation method.

[0015] According to another aspect of an embodiment of the present application, an electronic device is further provided, including: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the above-mentioned bandwidth allocation method is executed when the program is run.

[0016] According to another aspect of an embodiment of the present application, a computer program product is also provided, including computer instructions, and when the computer instructions are executed by a processor, the above-mentioned bandwidth allocation method is implemented.

[0017] In an embodiment of the present application, the network data link bandwidth of the target network cluster at the first moment is obtained; the shared data link is divided into multiple different data planes based on the business needs of different network traffic of the target network cluster, wherein each data plane corresponds to a business need of a network traffic, and the shared data link is a network data link commonly used by all network traffic in the target network cluster; the network data link bandwidth is allocated to each data plane based on the token bucket algorithm and the network data link bandwidth at the first moment, wherein the token bucket algorithm is used to adjust the network data link bandwidth of each data plane, by obtaining the network data link bandwidth of the target network cluster at the first moment, the network data link bandwidth is divided into multiple different data planes, and the network data link bandwidth is allocated to each data plane based on the token bucket algorithm and the network data link bandwidth at the first moment, the network traffic of multiple services uses the corresponding data planes respectively, and the bandwidth is allocated respectively, so that the bandwidth resources are fully utilized while ensuring that the services can be responded to, thereby solving the technical problems in the related art of non-responsiveness of key services and inability to fully utilize bandwidth resources due to the mixed use of shared data links by network traffic of multiple services. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 It is a hardware structure block diagram of a computer terminal for implementing a method for bandwidth allocation provided in an embodiment of the present application;

[0020] Figure 2 It is a flowchart of a bandwidth allocation method provided according to an embodiment of the present application;

[0021] Figure 3 is a data plane diagram provided according to an embodiment of the present application;

[0022] Figure 4 It is a schematic diagram of a process of dynamically balancing data plane token buckets provided according to an embodiment of the present application;

[0023] Figure 5 A data plane lending diagram is provided according to an embodiment of the present application;

[0024] Figure 6It is a schematic diagram of traffic shaping for multiple data flows in a data plane according to an embodiment of the present application;

[0025] Figure 7 A storage data plane partition diagram provided according to an embodiment of the present application;

[0026] Figure 8 It is a schematic diagram of scheduling data stream bandwidth by time slice according to an embodiment of the present application;

[0027] Fig. 9 It is a flowchart of another bandwidth allocation method provided according to an embodiment of the present application;

[0028] Fig.10 It is a structural diagram of a bandwidth allocation device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0030] The information collected in the embodiments of the present application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or reject automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0033] Service Level Agreement (SLA): defines the agreement between the service provider and the customer, and specifies the performance and reliability indicators of the service, such as response time, uptime, data transfer rate, etc.

[0034] Quality of Service (QoS): refers to providing a predetermined level of service performance in a communication network to ensure that specific types of traffic can be given priority or meet specific latency, bandwidth and packet loss requirements. QoS mechanisms are often used in network communications to support real-time applications, mission-critical applications and other services that require guaranteed performance.

[0035] K8s: short for Kubernetes, Kubernetes is an open source container orchestration and management platform used to automate the deployment, expansion and management of container applications. K8s provides an efficient, scalable and flexible mechanism that enables developers and operation and maintenance personnel to deploy containerized applications in a multi-host cluster environment without manually managing the life cycle of each container.

[0036] In the related art, in the architectural design of a network cluster (such as Kubernetes, referred to as K8s), multiple network data planes usually share the same physical data link for communication. When multiple business traffic flows are mixed using the same physical path, such as high-frequency interaction of internal services, storage synchronization of large amounts of data, frequent external load balancing requests, and high-speed forwarding of external data, these business activities may increase dramatically in a short period of time, resulting in data congestion on the data link. There are problems of unresponsiveness of key businesses and inability to fully utilize bandwidth resources due to the mixed use of shared data links by network traffic of multiple businesses. In order to solve this problem, a relevant solution is provided in the embodiments of the present application.

[0037] According to an embodiment of the present application, a method embodiment of bandwidth allocation is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0038] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a method for bandwidth allocation. Figure 1As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0039] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0040] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the method of bandwidth allocation in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the above-mentioned method of bandwidth allocation is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0041] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0042] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0043] In the above operating environment, the present application embodiment provides a flow chart of a method for bandwidth allocation, such as Figure 2 As shown, the method comprises the following steps:

[0044] Step S202, obtaining the network data link bandwidth of the target network cluster at the first moment. This method of obtaining bandwidth information in real time can ensure that the allocation of network resources is based on the latest network conditions, improving the accuracy and efficiency of bandwidth allocation. Application scenarios include data centers, cloud computing platforms, and other environments that require dynamic adjustment of network resources.

[0045] In the technical solution provided in step S202, the network data link bandwidth of the target network cluster (for example, K8s, K8s uses redundant physical links to provide a bearer network for all data planes in the cluster) at the first moment (for example, the current moment) is obtained. The following is a specific embodiment:

[0046] Use Ethernet tools (for example, Linux ethtool) to check the data link bandwidth of the physical link data used by K8s at the current moment (that is, the above-mentioned network data link bandwidth). Linux ethtool is a command-line tool used to query and set network device properties. It can obtain detailed information about the network interface, obtain the theoretical maximum bandwidth and duplex mode of the interface, and manage and troubleshoot network devices.

[0047] Step S204, based on the business requirements of different network traffic of the target network cluster, the shared data link is divided into multiple different data planes, wherein each data plane corresponds to a business requirement of a network traffic, and the shared data link is a network data link commonly used by all network traffic in the target network cluster. By dividing the data plane, it is possible to achieve refined management of network traffic with different business requirements, effectively avoiding the impact of key services due to bandwidth competition, and is suitable for a multi-tenant, multi-service parallel network architecture.

[0048] In the technical solution provided in step S204, the shared data link is divided into the following data planes: storage data plane, internal communication data plane, load balancing network data plane, and data forwarding network data plane, wherein the storage data plane is used for the communication of the internal storage area network of the target network cluster, the internal communication data plane is used for the communication of the internal data of the target network cluster, the load balancing network data plane is used for the communication between the services deployed by the target network cluster and the external clients, and the data forwarding network data plane is used to forward the service requests of the external clients to access the target network cluster. This classification method can provide customized bandwidth management strategies for different types of network traffic, improve the stability and response speed of network services, and is particularly suitable for scenarios with high requirements on network performance, such as high-performance computing and big data processing.

[0049] The following is a specific embodiment:

[0050] The target network cluster supports single-link data paths, dual-link data paths, and multi-link data paths. In the physical link, according to the business requirements of different network traffic within the target network cluster (for example, K8s), the shared network resources (that is, the above-mentioned shared data links) are divided into multiple logical data planes, such as Figure 3 As shown, a data plane diagram is provided. It can be seen that the physical link bandwidth (i.e. the above-mentioned shared data link) is divided into a storage data plane: the communication of the storage area network (SAN) within the K8s cluster, the storage dedicated network bandwidth; a load balancing data plane (i.e. the above-mentioned load balancing network data plane): the load balancing network data plane for the deployed K8s to communicate externally, which is used for the communication between the services deployed in the target network cluster and the external clients; a K8s communication data plane (i.e. the above-mentioned internal communication data plane): the K8s internal communication data plane, the internal data communication of the K8s cluster, the application programming interface (API) service data network (the bandwidth capacity of the network connection used when the API Server in the K8s cluster exchanges data with its components); a bandwidth external communication plane (i.e. the above-mentioned data forwarding network data plane): an externally accessible data forwarding network data plane, which is used to forward service requests from external clients to access the target network cluster.

[0051] Step S206: Allocate network data link bandwidth to each data plane based on a token bucket algorithm and the network data link bandwidth at the first moment, wherein the token bucket algorithm is used to adjust the network data link bandwidth of each data plane.

[0052] In the technical solution provided in step S206, the allocation of network data link bandwidth to each data plane based on the token bucket algorithm and the network data link bandwidth at the first moment can be achieved in a variety of ways, for example: allocating speed limit tokens to each data plane based on the network data link bandwidth at the first moment, wherein the speed limit token is used to control the network data link bandwidth of the data plane, wherein the speed limit token is an identifier of the network data link bandwidth of the control data plane in the token bucket algorithm, and each speed limit token represents a preset transmission quota; adjusting the network data link bandwidth of each data plane based on the number of speed limit tokens. Through speed limit token control, it is possible to dynamically adjust the bandwidth usage of each data plane to ensure fair allocation of network resources, which is suitable for scenarios where network traffic fluctuates greatly and bandwidth allocation needs to be adjusted in real time. The adjustment mechanism based on the number of tokens can flexibly respond to changes in network traffic and ensure efficient use of network resources. Each token represents a preset transmission quota (i.e., each token can send a certain amount of data). Before a data packet is sent through a network link, the corresponding speed limit token must be consumed to achieve bandwidth quota control of the data plane, thereby ensuring the rational use of network resources and the priority of traffic for key services. In short, the speed limit token is a virtual mark that quantifies the network bandwidth usage rights and is the core mechanism for implementing bandwidth allocation and control.

[0053] In the above steps, there are many ways to implement the adjustment of the network data link bandwidth of each data plane based on the number of speed limit tokens. For example, the following methods can be used: when the number of first speed limit tokens of the first data plane is less than or equal to the first preset threshold, the number of second speed limit tokens of the second data plane is obtained, wherein the first data plane is any data plane, and the second data plane is any data plane different from the first data plane; when the number of second speed limit tokens is greater than the first preset threshold, part of the speed limit tokens in the second speed limit tokens are allocated to the first data plane; based on the allocated part of the speed limit tokens, the network data link bandwidth of the first data plane is adjusted. By monitoring the number of tokens, abnormalities in network traffic can be discovered in a timely manner, providing a basis for the dynamic adjustment of bandwidth. The dynamic token reallocation mechanism can effectively balance the bandwidth usage between different data planes, avoid idleness and waste of bandwidth resources, and is particularly suitable for network environments with multiple businesses running in parallel and limited bandwidth resources. By adjusting the bandwidth, it can ensure that the network requirements of key businesses are met, and the overall performance of network services is improved.

[0054] When the number of the second speed limit tokens is less than or equal to the first preset threshold, part of the speed limit tokens allocated by the second data plane to the first data plane are reallocated to the second data plane; and the network data link bandwidth of the second data plane is adjusted based on the part of the speed limit tokens reallocated to the second data plane. This dynamic recycling and reallocation mechanism of tokens can realize dynamic optimization of network resources and ensure high availability and efficiency of the network.

[0055] The following is a specific embodiment:

[0056] Based on the token bucket algorithm (for example, Linux HTB (hierarchical token bucket traffic scheduling algorithm)), bandwidth restrictions are imposed on different data planes to perform dynamic traffic speed limiting. When the speed limit tokens of a data plane are exhausted, but speed limit tokens of other data planes are left over, speed limit tokens can be lent from the data plane that is allowed to lend speed limit tokens to the exhausted data plane to temporarily increase the network bandwidth of the data plane that has exhausted speed limit tokens, thereby ensuring maximum utilization of the total physical link bandwidth. The hierarchical token bucket traffic scheduling algorithm of the present application adds dynamic traffic token analysis, and in response to the situation of token exhaustion, the communication data is balanced across token buckets for the data traffic across the HTB hierarchical token buckets on different data planes according to the configured schedulable ratio.

[0057] like Figure 4 As shown in the figure, a schematic diagram of the process of dynamically balancing the data plane token bucket is provided, which shows the operation mechanism of the token bucket algorithm:

[0058] First, perform token statistics on the K8s communication data plane (i.e., the above-mentioned internal communication data plane): count the number of tokens (i.e., the above-mentioned speed limit tokens) of the K8s communication data plane in real time; upon detecting that the tokens of the K8s communication data plane are exhausted (i.e., the number of the above-mentioned first speed limit tokens is less than or equal to the first preset threshold, and the first preset threshold is 0 at this time), send them to the data plane token monitoring of the token bucket algorithm; then perform storage data plane token statistics: count the number of tokens (i.e., the above-mentioned speed limit tokens) of the storage data plane in real time, and upon checking that the remaining tokens (i.e., the number of the above-mentioned second speed limit tokens is greater than the first preset threshold), send them to the data plane token monitoring of the token bucket algorithm, and the data plane token monitoring performs storage data plane token configuration and K8s communication data plane token configuration, and rebalances the data plane token bucket (i.e., allocating part of the speed limit tokens in the second speed limit token to the first data plane), such as Figure 5As shown, a data plane lending graph is provided. When it is detected that an area for storing tokens lent to the K8s communication data plane is divided in the storage data plane (for storing tokens lent to the K8s communication data plane, i.e., the above-mentioned partial speed limit tokens), the partial speed limit tokens are lent to the K8s communication data plane, and the network data link bandwidth of the first data plane is adjusted based on the allocated partial speed limit tokens, thereby achieving traffic speed limiting.

[0059] When the data traffic plane that lent out the tokens also faces token exhaustion, the tokens previously lent to other data planes are gradually recovered through dynamic configuration to prioritize the communication bandwidth requirements of this data plane. Specifically: when the number of second speed limit tokens is less than or equal to the first preset threshold, part of the speed limit tokens allocated by the second data plane to the first data plane are reallocated to the second data plane; and the network data link bandwidth of the second data plane is adjusted based on the part of the speed limit tokens reallocated to the second data plane.

[0060] For each data plane, the data bandwidth of each data plane is divided into multiple subclasses, each subclass represents a single data link, and for a single data link of each subclass, for example, traffic speed limiting can be performed in the following manner: traffic speed limiting of a single data link of each subclass based on a quintuple: obtaining quintuple information of a target data stream, wherein the quintuple information includes source IP, destination IP, source port, destination port, and communication protocol type; based on the quintuple information, mapping the target data stream to a corresponding target subclass; and traffic speed limiting of the target subclass based on the configuration information, wherein the configuration information contains traffic speed limiting rules for all subclasses. For example, preset traffic rules refer to a set of rules used to control and manage the transmission rate of data flows in the network, including the following key elements: the maximum bandwidth that each data flow or data plane can use, the handling strategy for burst traffic: such as setting the size of the "Burst Buffer" to allow data flows to exceed their average rate limit for a short period of time to handle instantaneous high loads, and the minimum bandwidth guarantee: ensuring that each data flow or data plane has at least the minimum bandwidth available. When bandwidth resources are tight, the time slice method can be used to allocate bandwidth to different data flows on a periodic basis to ensure that each data flow has the opportunity to transmit data. This method is suitable for network environments that require speed limits on specific traffic. By configuring traffic speed limits through configuration information, the allocation strategy of network resources can be flexibly adjusted to meet different business needs. It is suitable for scenarios where the network resource allocation strategy needs to be flexibly adjusted.

[0061] For different data planes, you can also prioritize data speed limits on the selected data planes according to the configuration: allocate network data bandwidth to any data plane selected by the user according to the configuration information, thereby giving priority to the data line speed (traffic speed limit).

[0062] like Figure 6 As shown, a schematic diagram of traffic shaping for multiple data flows in a data plane is provided. Traffic shaping is performed on multiple data flows in the data plane based on quintuples to obtain unified (Network File System, referred to as NFS) data. For sub-data planes (subclasses) that affect system stability, fixed bandwidth guarantees can be further used to ensure the minimum bandwidth of the traffic, such as the cross-device storage synchronization data bandwidth configuration of the storage system.

[0063] A specific service within a single data plane may have multiple data flows (data links). The data within a single data plane can be further limited according to the service traffic priority. According to the service priority, the data flows that affect the operation stability are given priority, such as Figure 7 As shown, a storage data plane partition diagram is provided, which further divides the storage data plane into multiple data links: network storage accesses the network file system (NFS), network storage accesses block devices, and cross-device storage synchronization. According to the priority of the services of the three data links, the network data link bandwidth is allocated to the data link with the highest priority first, and the data flow that affects the stable operation is guaranteed first.

[0064] For client access data traffic, the client-server access data can be divided into the same sub-data plane according to the actual access sharing situation, and the traffic of a single data stream can be shaped according to the five-tuple. In the K8s physical cluster, the actual business data of each data plane is unbalanced. When the data traffic of some data planes surges, some data planes have surplus bandwidth (that is, the bandwidth demand of the network data link mentioned above). At this time, the bandwidth can be lent for a short time (the data plane with surplus bandwidth lends bandwidth to the data plane with no surplus bandwidth) to maximize the use of the physical link bandwidth.

[0065] When the number of data flows in the data plane is huge and each path (data link) has bandwidth, the time slice method is used to periodically allocate bandwidth to all data links according to the configured time period (i.e. the above-mentioned network data link bandwidth requirements) according to the time slice. The data flow (data link) that obtains the time slice can obtain the bandwidth actually accessed and used within the time slice, and the data flow that does not obtain the time slice is only guaranteed to keep the bandwidth alive within the minimum bandwidth. This ensures that when the bandwidth is insufficient, the borrowed bandwidth cannot meet the total data flow, and the evenly allocated data link cannot guarantee the minimum access bandwidth requirements, the minimum guaranteed data flow bandwidth can be allocated to each link according to the time slice method.

[0066] When the number of data links in the third data plane exceeds the second preset threshold, the priority of the data link in the third data plane and the network data link bandwidth requirement of each data link are obtained, wherein the third data plane is any data plane; in each time slice period, time slices are allocated to the data links according to the priority of the data links, wherein the time slices are used to allocate periodic network data link bandwidth usage rights to the data links; the bandwidth that meets the network data link bandwidth requirement is allocated to the data links that obtain the time slices, and the minimum bandwidth is allocated to the data links that do not obtain the time slices.

[0067] The bandwidth of data streams (data links) can be rotated according to time slices. When a single data link needs the first bandwidth guarantee, the network data link bandwidth can be allocated to the single data stream in a periodic rotation according to the minimum bandwidth guarantee requirement. For example, the following methods can also be combined: Adjustment of the Transmission Control Protocol (TCP) Slow Start and Sliding Window mechanisms: By modifying the TCP slow start threshold, the speed at which the data stream (data link) increases the transmission rate at the beginning of the time slice can be controlled to avoid occupying too much bandwidth instantly. In each time slice, the sliding window size is limited to ensure that the data stream does not exceed its allocated bandwidth share, and does not over-compete for bandwidth resources even when network conditions permit. The status of the data stream is continuously monitored, including the transmission rate, congestion, and network delay, so as to dynamically adjust the time slice allocation and TCP parameters to ensure efficient and fair use of bandwidth. In the time slice rotation, the drastic changes in the data stream rate between different time slices are mitigated by a smoothing mechanism (such as using a cache or queue) to avoid network fluctuations and data packet loss. According to the priority of data flow and the minimum bandwidth guarantee requirements, the bandwidth allocation within the time slice is dynamically adjusted to give priority to the transmission of key data flows.

[0068] like Figure 8 As shown, a schematic diagram of scheduling data flow bandwidth by time slice is provided. For the data links of the same storage data plane, traffic shaping is performed according to the data flow to share the bandwidth of the entire data plane. Each data link in the figure can use a specific bandwidth (i.e., the above-mentioned network data link bandwidth) within the allocated time slice to prevent a single data link from exhausting the bandwidth of the entire data plane, thereby causing other data links in the entire data plane to be unable to obtain data and cause link denial of service (Denial of Service, referred to as DoS).

[0069] Through the above steps, it can be achieved that different data planes of the target network cluster (for example, K8s) can share a data link composed of a group of physical links, and on this basis, dynamically allocate data plane bandwidth within the bandwidth provided by the shared redundant physical link, and perform traffic shaping on a single data flow within the data plane, and dynamically balance data flows across data planes. It is ensured that when the physical link is shared, the target network cluster (for example, K8s) cluster can guarantee stable operation, and will not cause DoS of the main business of the entire cluster due to bandwidth competition for data transmission. It is suitable for a target network cluster (for example, K8s) with busy data, and dynamically shares physical bandwidth. Improve the overall resource utilization of the target network cluster (for example, K8s), and each data plane can ensure that a busy target network cluster (for example, K8s) can work stably on a shared physical link (shared data link) because of precise data plane traffic control and SLA guarantee of each data flow within the data plane.

[0070] The present application embodiment provides a flow chart of another bandwidth allocation method, such as Fig. 9 As shown,

[0071] First, load the traffic engineering configuration, check the node physical link bandwidth (that is, obtain the network data link bandwidth of the target network cluster at the first moment), configure the data plane token bucket current limiting according to the configuration and physical link bandwidth (that is, allocate network data link bandwidth to each data plane based on the token bucket algorithm and the network data link bandwidth at the first moment), create a token monitor for each data plane, and the token monitor is used for data plane monitoring. The process of configuring data flow flow control within the plane for each data plane is as follows: the data plane monitor configures the flow control (network data link bandwidth) of the data flow (data link) within the data plane for each data plane one by one, and returns the result to the token monitor.

[0072] Calculate and summarize the monitoring data of each data plane in a cycle according to the time slot, reconfigure the data plane token bucket flow limit according to the remaining tokens, and send the result to the data platform monitoring to perform flow control of the data plane data link. Specifically: when the number of first speed limit tokens of the first data plane is less than or equal to the first preset threshold, obtain the number of second speed limit tokens of the second data plane, wherein the first data plane is any data plane, and the second data plane is any data plane different from the first data plane; when the number of second speed limit tokens is greater than the first preset threshold, allocate part of the speed limit tokens in the second speed limit token to the first data plane; adjust the network data link bandwidth of the first data plane based on the allocated part of the speed limit tokens. When the number of second speed limit tokens is less than or equal to the first preset threshold, reallocate part of the speed limit tokens allocated by the second data plane to the first data plane to the second data plane; adjust the network data link bandwidth of the second data plane based on the part of the speed limit tokens reallocated to the second data plane.

[0073] The present application embodiment provides a schematic diagram of the structure of a bandwidth allocation device, such as Fig.10 As shown, including:

[0074] The acquisition module 1002 is used to acquire the network data link bandwidth of the target network cluster at a first moment.

[0075] The division module 1004 is used to divide the shared data link into multiple different data planes based on the business requirements of different network traffic in the target network cluster, wherein each data plane corresponds to the business requirement of a network traffic, and the shared data link is a network data link commonly used by all network traffic in the target network cluster.

[0076] The shared data link is divided into the following data planes: storage data plane, internal communication data plane, load balancing network data plane, and data forwarding network data plane. The storage data plane is used for communication within the internal storage area network of the target network cluster, the internal communication data plane is used for communication within the target network cluster, the load balancing network data plane is used for communication between services deployed in the target network cluster and external clients, and the data forwarding network data plane is used to forward service requests from external clients to access the target network cluster.

[0077] The bandwidth allocation module 1006 is used to allocate network data link bandwidth to each data plane based on a token bucket algorithm and the network data link bandwidth at the first moment, wherein the token bucket algorithm is used to adjust the network data link bandwidth of each data plane.

[0078] The bandwidth allocation module 1006 is also used to obtain the number of second speed limit tokens of the second data plane when the number of first speed limit tokens of the first data plane is less than or equal to a first preset threshold, wherein the first data plane is any data plane and the second data plane is any data plane different from the first data plane; when the number of second speed limit tokens is greater than the first preset threshold, allocate part of the speed limit tokens in the second speed limit tokens to the first data plane; and adjust the network data link bandwidth of the first data plane based on the allocated part of the speed limit tokens.

[0079] The bandwidth allocation module 1006 is also used to reallocate part of the speed limit tokens allocated by the second data plane to the first data plane to the second data plane when the number of the second speed limit tokens is less than or equal to the first preset threshold; and adjust the network data link bandwidth of the second data plane based on the part of the speed limit tokens reallocated to the second data plane.

[0080] The bandwidth allocation module 1006 is also used to divide the data bandwidth of each data plane into multiple subclasses for each data plane, each subclass represents a single data link, and for a single data link of each subclass, the traffic speed is limited in the following manner: based on the quintuple, the traffic speed of the single data link of each subclass is limited: the quintuple information of the target data stream is obtained, wherein the quintuple information includes the source IP, the destination IP, the source port, the destination port, and the communication protocol type; based on the quintuple information, the target data stream is mapped to the corresponding target subclass; based on the configuration information, the traffic speed is limited for the target subclass, wherein the configuration information contains the traffic speed limit rules for all subclasses.

[0081] The bandwidth allocation module 1006 is also used to obtain the priority of the data link of the third data plane and the network data link bandwidth requirements of each data link when the number of data links in the third data plane exceeds the second preset threshold, wherein the third data plane is any data plane; in each time slice period, allocate time slices to the data links according to the priority of the data links, wherein the time slices are used to allocate periodic network data link bandwidth usage rights to the data links; allocate bandwidth that meets the network data link bandwidth requirements to the data links that obtain the time slices, and allocate the minimum bandwidth to the data links that do not obtain the time slices.

[0082] The bandwidth allocation module 1006 is also used to obtain the priority of the data link of the third data plane and the network data link bandwidth requirements of each data link when the number of data links in the third data plane exceeds the second preset threshold, wherein the third data plane is any data plane; in each time slice period, allocate time slices to the data links according to the priority of the data links, wherein the time slices are used to allocate periodic network data link bandwidth usage rights to the data links; allocate bandwidth that meets the network data link bandwidth requirements to the data links that obtain the time slices, and allocate the minimum bandwidth to the data links that do not obtain the time slices.

[0083] It should be noted that Fig.10 The bandwidth allocation device shown is used to perform Figure 2 The bandwidth allocation method shown is therefore Figure 2 The relevant explanations in the bandwidth allocation method in are also applicable to the bandwidth allocation device and will not be repeated here.

[0084] It should be noted that the various modules in the above-mentioned bandwidth allocation device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of the above-mentioned modules is a processor, or the functions of the above-mentioned modules are implemented by a processor.

[0085] The embodiment of the present application also provides a non-volatile storage medium, the non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the above bandwidth allocation method. For example, the network data link bandwidth of the target network cluster at the first moment is obtained; based on the business needs of different network traffic of the target network cluster, the shared data link is divided into multiple different data planes, wherein each data plane corresponds to a business need of a network traffic, and the shared data link is a network data link used by all network traffic in the target network cluster; the network data link bandwidth is allocated to each data plane based on the token bucket algorithm and the network data link bandwidth at the first moment, wherein the token bucket algorithm is used to adjust the network data link bandwidth of each data plane.

[0086] The embodiment of the present application also provides an electronic device, the electronic device includes a processor, the processor is used to run a program, wherein the above bandwidth allocation method is executed when the program is running. For example, the network data link bandwidth of the target network cluster at the first moment is obtained; based on the business needs of different network traffic of the target network cluster, the shared data link is divided into multiple different data planes, wherein each data plane corresponds to a business need of a network traffic, and the shared data link is a network data link commonly used by all network traffic in the target network cluster; the network data link bandwidth is allocated to each data plane based on the token bucket algorithm and the network data link bandwidth at the first moment, wherein the token bucket algorithm is used to adjust the network data link bandwidth of each data plane.

[0087] According to another aspect of the embodiment of the present application, a computer program product is also provided, including a computer program, which implements the above bandwidth allocation method when executed by a processor. For example, the network data link bandwidth of the target network cluster at the first moment is obtained; based on the business needs of different network traffic of the target network cluster, the shared data link is divided into multiple different data planes, wherein each data plane corresponds to a business need of a network traffic, and the shared data link is a network data link used by all network traffic in the target network cluster; the network data link bandwidth is allocated to each data plane based on the token bucket algorithm and the network data link bandwidth at the first moment, wherein the token bucket algorithm is used to adjust the network data link bandwidth of each data plane.

[0088] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0089] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0090] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0091] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0092] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0093] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for allocating bandwidth, characterized in that: include: Obtain the network data link bandwidth of the target network cluster at the first moment; Dividing the shared data link into a plurality of different data planes based on the business requirements of different network flows of the target network cluster, wherein each data plane corresponds to a business requirement of a network flow, and the shared data link is a network data link commonly used by all network flows in the target network cluster; The network data link bandwidth is allocated to each of the data planes based on a token bucket algorithm and the network data link bandwidth at the first moment, wherein the token bucket algorithm is used to adjust the network data link bandwidth of each of the data planes.

2. The method according to claim 1, characterized in that The common data link is divided into the following data planes: Storage data plane, internal communication data plane, load balancing network data plane, data forwarding network data plane, wherein the storage data plane is used for communication within the internal storage area network of the target network cluster, the internal communication data plane is used for communication within the target network cluster, the load balancing network data plane is used for communication between services deployed by the target network cluster and external clients, and the data forwarding network data plane is used to forward requests from external clients to access services within the target network cluster.

3. The method according to claim 1, characterized in that Allocating network data link bandwidth to each of the data planes based on a token bucket algorithm and the network data link bandwidth at the first moment includes: Allocating a speed limit token to each of the data planes based on the network data link bandwidth at the first moment, wherein the speed limit token is used to control the network data link bandwidth of the data plane, wherein the speed limit token is an identifier of the network data link bandwidth of the data plane controlled in the token bucket algorithm, and each speed limit token represents a preset transmission quota; The network data link bandwidth of each of the data planes is adjusted based on the number of the rate limit tokens.

4. The method according to claim 3, characterized in that Adjusting the network data link bandwidth of each of the data planes based on the number of the rate limit tokens includes: When the number of first speed limit tokens of the first data plane is less than or equal to the first preset threshold, obtaining the number of second speed limit tokens of the second data plane, wherein the first data plane is any data plane, and the second data plane is any data plane different from the first data plane; When the number of the second speed limit tokens is greater than the first preset threshold, allocating some of the second speed limit tokens to the first data plane; The network data link bandwidth of the first data plane is adjusted based on the allocated portion of the rate limit tokens.

5. The method according to claim 4, characterized in that The method further comprises: When the number of the second speed limit tokens is less than or equal to the first preset threshold, reallocate part of the speed limit tokens allocated by the second data plane to the first data plane to the second data plane; The network data link bandwidth of the second data plane is adjusted based on the portion of the rate limit tokens reallocated to the second data plane.

6. The method according to claim 1, characterized in that The method further includes: for each of the data planes, dividing the data bandwidth of each of the data planes into a plurality of subclasses, each subclass representing a single data link, and for a single data link of each of the subclasses, performing flow rate limiting in the following manner: The traffic rate of a single data link of each subclass is limited based on the five-tuple: Obtain quintuple information of the target data stream, wherein the quintuple information includes source IP, destination IP, source port, destination port, and communication protocol type; Based on the five-tuple information, mapping the target data stream to a corresponding target subclass; The target subclass is subjected to flow rate limiting based on configuration information, wherein the configuration information stores flow rate limiting rules for all subclasses.

7. The method according to claim 1, characterized in that The method further comprises: When the number of data links of the third data plane exceeds the second preset threshold, Obtaining the priority of the data link of the third data plane and the network data link bandwidth requirement of each data link, wherein the third data plane is any data plane; In each time slice period, allocating time slices to the data link according to the priority of the data link, wherein the time slices are used to allocate the periodic network data link bandwidth usage rights to the data link; The bandwidth that meets the bandwidth requirement of the network data link is allocated to the data link that obtains the time slice, and the minimum bandwidth is allocated to the data link that does not obtain the time slice.

8. A bandwidth allocation device, characterized in that: include: An acquisition module, used to acquire the network data link bandwidth of the target network cluster at the first moment; A division module, used to divide the common data link into a plurality of different data planes based on the business requirements of different network flows of the target network cluster, wherein each data plane corresponds to a business requirement of a network flow, and the common data link is a network data link commonly used by all network flows in the target network cluster; The bandwidth allocation module is used to allocate network data link bandwidth to each of the data planes based on a token bucket algorithm and the network data link bandwidth at the first moment, wherein the token bucket algorithm is used to adjust the network data link bandwidth of each of the data planes.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the bandwidth allocation method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the method for allocating bandwidth according to any one of claims 1 to 7 is executed when the program is run.

11. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the bandwidth allocation method described in any one of claims 1 to 7 is implemented.

Images