Bandwidth limiting system and method, electronic device, storage medium and program product

Abstract

The invention provides a bandwidth limiting system and method, electronic equipment, a storage medium and a program product, which are applied to any node in a cluster, the cluster is used for processing data packets with multiple priorities, and the system comprises a bandwidth acquisition module used for acquiring current actual bandwidths corresponding to the priorities, adjusting the current limited bandwidth corresponding to each priority according to the current actual bandwidth corresponding to the plurality of priorities, the pre-configured total limited bandwidth of the node and the limited bandwidth range corresponding to each priority; and the bandwidth limiting module is used for determining a target priority corresponding to the to-be-sent data packet, and determining whether to release the data packet according to the current actual bandwidth and the current limited bandwidth corresponding to the target priority. According to the method and the device, the bandwidth control of the hybrid scene can be effectively supported, the bandwidth control process and the configuration mode are simple and efficient, the maintenance is easy, and the overall processing effect of the multi-priority task is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a bandwidth limiting system, method, electronic device, storage medium, and program product. Background Art

[0002] With the continuous development of cloud computing technology, more and more users choose to use cloud computing to process tasks. In order to achieve higher resource utilization, online tasks and offline tasks can be mixed and deployed in the nodes of the cloud cluster. The mixed deployment scenario poses higher challenges to bandwidth control. How to effectively isolate the traffic of online tasks and offline tasks and improve the processing effect of online tasks and offline tasks as much as possible has become a key issue.

[0003] Traditional bandwidth control solutions have complex processes, are prone to performance bottlenecks, and are difficult to meet the requirements of the mixed deployment scenario. Summary of the Invention

[0004] This application provides a bandwidth limiting system, method, electronic device, storage medium, and program product to improve the bandwidth control effect in the mixed deployment scenario.

[0005] In a first aspect, an embodiment of this application provides a bandwidth limiting system, which is applied to any node in a cluster, and the cluster is used to process packets of multiple priorities; the system includes:

[0006] A bandwidth collection module, configured to collect the current actual bandwidth corresponding to each priority among multiple priorities, and adjust the current limiting bandwidth corresponding to each priority according to the current actual bandwidth corresponding to the multiple priorities, the total limiting bandwidth of the node pre-configured, and the limiting bandwidth range corresponding to each priority;

[0007] A bandwidth limiting module, configured to determine the target priority corresponding to the packet to be sent, and determine whether to release the packet according to the current actual bandwidth and the current limiting bandwidth corresponding to the target priority, where the target priority is one of the multiple priorities.

[0008] Optionally, the bandwidth limiting system further includes:

[0009] A configuration module, configured to obtain the total limiting bandwidth and the limiting bandwidth range corresponding to each priority sent by the application programming interface server.

[0010] Optionally, at least one container is deployed on the node, and the at least one container is used to process packets;

[0011] The configuration module is further configured to: obtain the priority corresponding to each container;

[0012] When determining the target priority corresponding to the data packet to be sent, the bandwidth limiting module is specifically configured to:

[0013] Determine the container corresponding to the data packet according to the characteristic information of the data packet, where the target priority of the data packet is the priority of the corresponding container.

[0014] Optionally, the node includes at least one network card; the bandwidth acquisition module and the bandwidth limiting module are used to be mounted on the at least one network card to perform bandwidth acquisition and bandwidth limiting on the data packets corresponding to the at least one network card.

[0015] Optionally, the at least one network card all corresponds to an input queue and an output queue, where the input queue of any network card includes the data packets sent by the network card to the container, and the output queue includes the data packets sent by the container to the network card;

[0016] The bandwidth acquisition module is specifically configured to perform bandwidth acquisition on the input queue and the output queue respectively; the bandwidth limiting module is specifically configured to perform bandwidth limiting on the input queue and the output queue respectively.

[0017] Optionally, the total limited bandwidth of the node includes the total limited bandwidth of the entrance and the total limited bandwidth of the exit, and the limited bandwidth ranges corresponding to each priority include the limited bandwidth range of the entrance and the limited bandwidth range of the exit, so as to perform bandwidth acquisition and bandwidth limiting on the input queue according to the total limited bandwidth of the entrance and the limited bandwidth ranges of the entrances corresponding to each priority, and perform bandwidth acquisition and bandwidth limiting on the output queue according to the total limited bandwidth of the exit and the limited bandwidth ranges of the exits corresponding to each priority.

[0018] Optionally, when the bandwidth acquisition module adjusts the current limited bandwidth corresponding to each priority according to the current actual bandwidths corresponding to the multiple priorities, the total limited bandwidth of the node pre-configured, and the limited bandwidth ranges corresponding to each priority, it is specifically configured to:

[0019] If the sum of the current actual bandwidths corresponding to the multiple priorities is less than the total limited bandwidth of the node, then upwardly adjust the current limited bandwidth corresponding to at least some priorities; and / or,

[0020] If the sum of the current actual bandwidths corresponding to the multiple priorities is greater than the total limited bandwidth of the node, then downwardly adjust the current limited bandwidth corresponding to at least some priorities;

[0021] Wherein, the current limited bandwidth before and after the adjustment of any priority is within the corresponding limited bandwidth range.

[0022] Optionally, the limited bandwidth range includes a maximum limited bandwidth and a minimum limited bandwidth;

[0023] When the bandwidth acquisition module upward adjusts the current limited bandwidth corresponding to at least some priorities, it specifically is used for: subtracting the sum of the current actual bandwidths of the multiple priorities from the total limited bandwidth of the node to obtain the idle bandwidth; upward adjusting the current limited bandwidth corresponding to each priority in order from high to low until the total upward adjustment amplitude reaches the idle bandwidth;

[0024] Among them, for any priority other than the last adjusted priority in at least some priorities that have been upward adjusted, the adjusted current limited bandwidth is the corresponding highest limited bandwidth;

[0025] When the bandwidth acquisition module downward adjusts the current limited bandwidth corresponding to at least some priorities, it specifically is used for: subtracting the total limited bandwidth of the node from the sum of the current actual bandwidths of the multiple priorities to obtain the contention bandwidth; downward adjusting the current limited bandwidth corresponding to each priority in order from low to high until the total downward adjustment amplitude reaches the contention bandwidth;

[0026] Among them, for any priority other than the last adjusted priority in at least some priorities that have been downward adjusted, the adjusted current limited bandwidth is the corresponding highest limited bandwidth.

[0027] Optionally, when the bandwidth limiting module determines whether to release the data packet according to the current actual bandwidth and the current limited bandwidth corresponding to the target priority, it specifically is used for:

[0028] If the current actual bandwidth corresponding to the target priority is less than the current limited bandwidth, release the data packet;

[0029] If the current actual bandwidth corresponding to the target priority is not less than the current limited bandwidth, discard the data packet.

[0030] Optionally, when the bandwidth acquisition module acquires the current actual bandwidth corresponding to each priority among multiple priorities, it specifically is used for:

[0031] Within a preset period, for the data packets currently received or sent by the container, determine the priority corresponding to the data packet, and accumulate the size of the data packet to the traffic size corresponding to the priority;

[0032] Determine the current actual bandwidth corresponding to each priority according to the traffic size of each priority within the preset period.

[0033] Optionally, the bandwidth acquisition module is further used for:

[0034] After accumulating the size of the data packet to the traffic size corresponding to the priority, update the time stamp corresponding to the priority to the time stamp corresponding to the data packet;

[0035] When determining the current actual bandwidth of each priority according to the traffic sizes of each priority within a preset period, the bandwidth acquisition module is specifically configured to:

[0036] For any priority, calculate the bandwidth size of each sampling point among multiple sampling points within the preset period according to the timestamp and traffic size corresponding to the priority, and determine the current actual bandwidth of the priority within the preset period according to the bandwidth sizes of the multiple sampling points.

[0037] Optionally, the bandwidth acquisition module and the bandwidth limiting module are implemented by the extended Berkeley Packet Filtering (eBPF) technology.

[0038] Optionally, the total limited bandwidth of the node and the limited bandwidth ranges corresponding to each priority are recorded in the data storage structure of eBPF;

[0039] The bandwidth acquisition module is further configured to: read the total limited bandwidth of the node and the limited bandwidth ranges corresponding to each priority from the data storage structure of eBPF, and record the current actual bandwidth corresponding to each priority and the adjusted current limited bandwidth into the data storage structure of eBPF;

[0040] The bandwidth limiting module is further configured to: read the current actual bandwidth and the current limited bandwidth corresponding to the target priority from the data storage structure of eBPF.

[0041] In a second aspect, an embodiment of the present application provides a bandwidth limiting method, which is applied to any node in a cluster, and the cluster is used to process data packets of multiple priorities; the method includes:

[0042] Acquire the current actual bandwidth corresponding to each priority among multiple priorities, and adjust the current limited bandwidth corresponding to each priority according to the current actual bandwidths corresponding to the multiple priorities, the pre-configured total limited bandwidth of the node, and the limited bandwidth ranges corresponding to each priority;

[0043] Determine the target priority corresponding to the data packet to be sent, and determine whether to release the data packet according to the current actual bandwidth and the current limited bandwidth corresponding to the target priority, where the target priority is one of the multiple priorities.

[0044] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0045] At least one processor; and

[0046] A memory communicatively connected to the at least one processor;

[0047] Among them, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to cause the electronic device to execute the method described in the second aspect.

[0048] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, and when a processor executes the computer-executable instructions, the method described in the second aspect is implemented.

[0049] In a fifth aspect, an embodiment of the present application provides a computer program product including a computer program, and when the computer program is executed by a processor, the method described in the second aspect is implemented.

[0050] The bandwidth limiting system, method, electronic device, storage medium, and program product provided by the present application are applied to any node in a cluster, and the cluster is used to process data packets of multiple priorities. The system includes: a bandwidth acquisition module configured to acquire the current actual bandwidth corresponding to each priority among multiple priorities, and adjust the current limiting bandwidth corresponding to each priority according to the current actual bandwidths corresponding to the multiple priorities, the total limiting bandwidth of the node pre-configured, and the limiting bandwidth range corresponding to each priority; a bandwidth limiting module configured to determine the target priority corresponding to the data packet to be sent, and determine whether to release the data packet according to the current actual bandwidth and the current limiting bandwidth corresponding to the target priority, where the target priority is one of the multiple priorities. Thus, by configuring the limiting bandwidth range for each priority, it is possible to support dynamically adjusting the current limiting bandwidth for each priority respectively, and controlling the data packets of each priority respectively to meet the current limiting bandwidth of each priority, achieving effective isolation of the bandwidths of each priority. Moreover, when adjusting, referring to the current actual bandwidths of each priority and the total limiting bandwidth of the node, it is possible to perform dynamic allocation of bandwidth from the overall machine dimension according to the current actual usage situation, support bandwidth borrowing and return for different priorities, achieve effective support for the co-location scenario, and the bandwidth control process and configuration method are simple, efficient, and easy to maintain, improving the overall processing effect of multi-priority tasks. Description of the Drawings

[0051] The drawings here are incorporated into the description and form a part of this description, showing embodiments consistent with the present application and used together with the description to explain the principles of the present application.

[0052] Figure 1 It is a scenario diagram provided by an embodiment of the present application;

[0053] Figure 2 It is a schematic diagram of a bandwidth limiting system provided by an embodiment of the present application;

[0054] Figure 3 Schematic diagram of a user - end configuration interface provided by an embodiment of the present application;

[0055] Figure 4 Schematic diagram of the working process of a bandwidth acquisition module provided by an embodiment of the present application;

[0056] Figure 5 Schematic diagram of the working process of a bandwidth limiting module provided by an embodiment of the present application;

[0057] Figure 6 Schematic diagram of the process of a bandwidth limiting method provided by an embodiment of the present application;

[0058] Figure 7 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application.

[0059] Through the above - mentioned drawings, specific embodiments of the present application have been shown, and there will be more detailed descriptions hereinafter. These drawings and text descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Specific Embodiments

[0060] Here, exemplary embodiments will be described in detail, and their examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0061] It should be noted that the user information (including but not limited to user device information, user attribute information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with relevant laws, regulations, and standards, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0062] First, the nouns involved in the present application are explained:

[0063] eBPF: extended Berkeley Packet Filter, an extended Berkeley packet filter, a Linux operating system kernel function extension technology that can embed custom programs to execute custom logic in the Linux system.

[0064] TC: Traffic Control, which is a traffic rate limiting, shaping, and policy control mechanism provided by the Linux kernel.

[0065] edt: Earliest Departure Time, which is a timestamp-based bandwidth limiting technology.

[0066] Qdisc: Queue discipline, which is a queuing rule in the Linux system for bandwidth control.

[0067] Mixed deployment means that online tasks and offline tasks are deployed on the same node simultaneously.

[0068] Node refers to an individual computer or server in a cluster.

[0069] Online tasks generally refer to tasks with high real-time requirements, such as online queries, instant messaging, etc.

[0070] Offline tasks generally refer to tasks with low real-time requirements that can be processed offline, such as offline training, batch computing, etc.

[0071] In the cloud native scenario, containers can run on any node in the cluster, and there is a situation of mixed deployment of online tasks and offline tasks on the node. The embodiments of this application can be used to control the bandwidth of nodes in the cluster. Among them, a node can also be called a node device, which is a device in the cluster used to process online or offline tasks. Each node can include multiple network cards. At the same time, one or more containers can be deployed on the node, and the application programs in the one or more containers can use multiple network cards to process online or offline tasks.

[0072] Among them, containers can be created or destroyed, and the number of containers in a node can change. The relationship between containers and network cards is not restricted. Usually, one container can use one network card, and in some cases, it can also use multiple network cards. Different containers can be used to process different task types. For example, some containers are used to process online tasks, and some are used to process offline tasks. Different containers can all use the network cards of the node, so that online tasks and offline tasks can share the underlying infrastructure of the node. At the same time, due to the limited bandwidth resources of the node, there may be resource contention between online tasks and offline tasks. Therefore, bandwidth can be restricted for the data packets of different tasks. How to effectively isolate online and offline bandwidth in the case of resource contention and meet the offline task bandwidth as much as possible in the case of no resource contention has become a challenge.

[0073] In some technologies, the Qdisc mechanism of the Linux kernel can be used to achieve bandwidth control. The Qdisc mechanism configures a Qdisc scheduler for each network card and realizes bandwidth control through methods such as HTB (Hierarchical Token Bucket). For example, a tree structure can be adopted to classify the data packets transmitted by the network card, and each category is queued separately. The same global lock is used for different queues to achieve bandwidth limitation. When applying the Qdisc solution to the hybrid deployment scenario, there will be many problems and it is difficult to meet the requirements of the hybrid deployment scenario.

[0074] Specifically, in the case of container hybrid deployment, containers of different task types will use multiple network cards on the node to send and receive data packets simultaneously. The Qdisc solution can only perform bandwidth limitation on one network card, and multiple network cards cannot share the speed limit policy. Therefore, dynamic bandwidth allocation cannot be completed on multiple network cards of the node, and dynamic bandwidth borrowing and returning at the whole machine (node) dimension cannot be supported, which is not applicable to the hybrid deployment scenario. Moreover, bandwidth limitation depends on the global lock mechanism, and in high-traffic scenarios, performance bottlenecks will occur due to the global lock.

[0075] In addition, the bandwidth limitation policy may need to be dynamically adjusted. For example, users expect to be able to adjust the bandwidth allocation ratio of different task types according to actual needs. In the Qdisc solution, the tree structure needs to be configured through specific commands, and the configuration rules are complex and difficult to maintain.

[0076] In view of this, the embodiments of the present application provide a bandwidth limitation system to solve the problem that the traditional bandwidth limitation solution cannot support bandwidth control in the hybrid deployment scenario.

[0077] Figure 1 The figure is a schematic diagram of an application scenario provided by the embodiments of the present application. As Figure 1 shown, the system may include: a configuration module, a bandwidth collection module, and a bandwidth limitation module. Users can interact with the configuration module through the API (Application Programming Interface) server to implement the configuration of the bandwidth limitation policy, so as to control the bandwidth collection module and the bandwidth limitation module by using the configuration module. The bandwidth collection module and the bandwidth limitation module are mounted on the network card and are used to control the ingress and egress queues of the network card respectively. Among them, the ingress queue and the egress queue are used to maintain the traffic in the ingress direction and the egress direction respectively. The ingress direction is from the network card to the container, and the egress direction is from the container to the network card. There are multiple priorities for the data packets transmitted between the network card and the container, and the bandwidth collection module and the bandwidth limitation module can control the data packets with different priorities respectively.

[0078] Specifically, the configuration module can configure the total limited bandwidth for the entire node. For example, the total limited bandwidth of the node is 100M. At the same time, tasks are classified according to priorities, and task containers with different priorities are mixed and deployed on the node, supporting the configuration of different limited bandwidth ranges for tasks with different priorities. For example, the limited bandwidth range for priority L1 is 10M - 20M. In practical applications, the current limited bandwidth of each priority can be adjusted in real time according to the total limited bandwidth and the limited bandwidth ranges of each priority, so as to control the actual bandwidth of each priority below the current limited bandwidth.

[0079] The bandwidth acquisition module focuses on the acquisition of the current actual bandwidth of each priority and the dynamic adjustment of the current limited bandwidth. Specifically, the bandwidth acquisition module can acquire the current actual bandwidth corresponding to each priority in real time, and dynamically adjust the current limited bandwidth of each priority according to the preset total limited bandwidth, the limited bandwidth ranges corresponding to each priority, and the currently acquired actual bandwidth, so that the current limited bandwidth of each priority meets its respective limited bandwidth range, and the overall current actual bandwidth meets the total limited bandwidth of the node.

[0080] The bandwidth limitation module can perform actual bandwidth control according to the dynamically adjusted current limited bandwidth of each priority. For example, for any priority, the data packets of this priority can be allowed to pass or discarded according to the current limited bandwidth of this priority.

[0081] The embodiments of the present application can configure the limited bandwidth ranges of each priority, support the dynamic adjustment of the current limited bandwidth of each priority respectively, effectively isolate the online bandwidth and the offline bandwidth, and when adjusting, refer to the current actual bandwidth of each priority and the total limited bandwidth of the node, and can perform dynamic allocation of bandwidth from the whole machine dimension according to the current actual usage situation, support the borrowing and return of bandwidth for different priorities, and effectively support the hybrid deployment scenario, thereby improving the overall processing effect of online tasks and offline tasks.

[0082] On the premise of better meeting the actual task requirements, the solution provided by the embodiments of the present application has at least the following effects compared with the Qdisc solution: The embodiments of the present application can dynamically adjust the bandwidth usage of different priorities according to the node dimension, and support the bandwidth limitation strategy of multi-network card sharing. The bandwidth acquisition module focuses on dynamically adjusting the current limited bandwidth, and the bandwidth limitation module focuses on allowing or discarding data packets according to the current limited bandwidth, without using a tree structure to classify data packets and using a global lock to control data packets of different categories, effectively improving the processing efficiency and reducing performance bottlenecks. Moreover, the embodiments of the present application configure the total limited bandwidth of the node and the limited bandwidth ranges of each priority, without configuring a complex tree structure, and the configuration method is simple and efficient, and easy to maintain.

[0083] The following will describe in detail some embodiments of the present application in conjunction with the accompanying drawings. Without conflict between the embodiments, the following embodiments and the features in the embodiments can be combined with each other. In addition, the step timings in the following method embodiments are only examples and are not strictly limited.

[0084] Figure 2 The figure is a schematic diagram of a bandwidth limiting system provided by an embodiment of the present application. The system can be implemented in the form of software, hardware, or a combination of software and hardware, which is not limited in this embodiment. The system can be applied to any node in a cluster, and the cluster is used to process packets of multiple priorities. The cluster can be a computing cluster in a cloud-native scenario or other scenarios. As Figure 2 shown, the system may include: a bandwidth collection module 201 and a bandwidth limiting module 202.

[0085] The bandwidth collection module 201 is configured to collect the current actual bandwidth corresponding to each priority among multiple priorities, and adjust the current limiting bandwidth corresponding to each priority according to the current actual bandwidths corresponding to the multiple priorities, the total limiting bandwidth of the node pre-configured, and the limiting bandwidth ranges corresponding to each priority.

[0086] Among them, the multiple priorities can be set according to actual needs to distinguish different types of traffic. Optionally, multiple priorities can be set according to real-time requirements. For example, two priorities, online and offline, can be set, or three priorities, L0, L1, and L2, can be set. L0 has the highest priority, indicating the highest real-time requirement, L1 is the second, and L2 is the lowest. Or, the priorities can also be set in other ways, which is not limited here.

[0087] Each priority corresponds to a current actual bandwidth, a limiting bandwidth range, and a current limiting bandwidth. Among them, the current actual bandwidth is used to represent the actual bandwidth of the current priority, that is, the actual traffic size generated by the packets corresponding to the priority. The limiting bandwidth range can include a minimum limiting bandwidth and / or a maximum limiting bandwidth. The current limiting bandwidth is used to limit the actual traffic of the priority and can be regarded as a threshold. Moreover, the current limiting bandwidth is adjusted in real time, but it is within the limiting bandwidth range before and after the adjustment, that is, the current limiting bandwidth cannot be less than the minimum limiting bandwidth and / or cannot be greater than the maximum limiting bandwidth.

[0088] In addition, the node has a total limiting bandwidth, which is used to limit the total bandwidth of multiple priorities corresponding to the node.

[0089] The bandwidth acquisition module 201 can determine the current actual bandwidth corresponding to each priority among multiple priorities according to the priority and size of each data packet transmitted by the network card.

[0090] Optionally, when the bandwidth acquisition module 201 adjusts the current limit bandwidth corresponding to the multiple priorities according to the current actual bandwidth corresponding to the multiple priorities, the total limit bandwidth of the node pre-configured, and the limit bandwidth range corresponding to each priority, it is specifically used for:

[0091] If the sum of the current actual bandwidths corresponding to the multiple priorities is less than the total limit bandwidth of the node, then upwardly adjust the current limit bandwidth corresponding to at least some of the priorities; and / or,

[0092] If the sum of the current actual bandwidths corresponding to the multiple priorities is greater than the total limit bandwidth of the node, then downwardly adjust the current limit bandwidth corresponding to at least some of the priorities;

[0093] Wherein, the current limit bandwidth before and after the adjustment of any priority is within the corresponding limit bandwidth range.

[0094] Specifically, the bandwidth acquisition module 201 first adds up the current actual bandwidths corresponding to the multiple priorities. If the sum of the current actual bandwidths corresponding to the multiple priorities is less than the total limit bandwidth of the node, it means that there is no resource contention and some resources are idle, and the current limit bandwidth of some or all of the priorities can be adjusted upward.

[0095] Exemplarily, there are three priorities, namely LO, L1, and L2. It is calculated that the current actual bandwidths corresponding to LO, L1, and L2 are 10M, 20M, and 30M respectively, and the total limit bandwidth of the node is 70M. 10M + 20M + 30M < 70M. Therefore, the current limit bandwidth of some or all of the three priorities can be adjusted upward. For example, the current limit bandwidth corresponding to LO before adjustment is also 10M, and the limit bandwidth range corresponding to LO is 10 - 30M. The current limit bandwidth corresponding to LO can be adjusted upward to 20M.

[0096] If the sum of the current actual bandwidths corresponding to the multiple priorities is greater than the total limit bandwidth of the node, it means that there is resource contention and some resources are in a contended state, and the current limit bandwidth of some or all of the priorities can be adjusted downward.

[0097] Exemplarily, there are three priorities in total, namely LO, L1, and L2. The currently actual bandwidths corresponding to LO, L1, and L2 are calculated to be 30M, 20M, and 30M respectively, and the total limited bandwidth of the node is 70M. Since 30M + 20M + 30M > 70M, the currently limited bandwidths of some or all of the three priorities can be down-regulated. For example, before adjustment, the currently limited bandwidth corresponding to L2 is 35M, and the range of the limited bandwidth corresponding to L2 is 10 - 50M. The currently limited bandwidth corresponding to L2 can be down-regulated to 15M.

[0098] It can be set that only when the sum of the currently actual bandwidths corresponding to multiple priorities is less than the total limited bandwidth of the node, the currently limited bandwidths of some or all of the priorities are up-regulated. It can also be set that only when the sum of the currently actual bandwidths corresponding to multiple priorities is greater than the total limited bandwidth of the node, the currently limited bandwidths of some or all of the priorities are down-regulated. It can also be set that when the sum of the currently actual bandwidths corresponding to multiple priorities is less than the total limited bandwidth of the node, the currently limited bandwidths of some or all of the priorities are up-regulated, and when the sum of the currently actual bandwidths corresponding to multiple priorities is greater than the total limited bandwidth of the node, the currently limited bandwidths of some or all of the priorities are down-regulated.

[0099] However, for any priority, the currently limited bandwidth before adjustment and the currently limited bandwidth after adjustment should both be within the range of the limited bandwidth corresponding to this priority.

[0100] In this way, according to the sum of the currently actual bandwidths and the total limited bandwidth, it can be determined whether the multiple priorities of the node are currently in a contention state, and the currently limited bandwidth is adjusted according to whether it is in a contention state. Thus, when resource contention occurs, the currently limited bandwidths corresponding to some or all of the priorities are down-regulated to reduce contention and improve the overall performance. When resources are idle, the currently limited bandwidths corresponding to some or all of the priorities are up-regulated to make the most of the resources as much as possible, so as to achieve the reasonable allocation of resources, improve the utilization rate of node resources, and improve the processing efficiency of the node.

[0101] Optionally, it can be set to adjust different priorities in sequence. For example, when up-regulating the currently limited bandwidth of one or more priorities, the currently limited bandwidth of the online task can be preferentially up-regulated to ensure the priority transmission of online task data packets as much as possible. When the online task is at a low ebb, the currently limited bandwidth of the offline task can be up-regulated so that the offline task can occupy the remaining bandwidth as much as possible. When down-regulating the currently limited bandwidth of one or more priorities, the currently limited bandwidth of the offline task can be preferentially down-regulated to avoid affecting the online task as much as possible. Thus, it can effectively isolate the online and offline bandwidths in the case of resource contention and satisfy the offline task bandwidth as much as possible in the case of no resource contention.

[0102] Optionally, the restricted bandwidth range includes a maximum restricted bandwidth and a minimum restricted bandwidth.

[0103] When the bandwidth acquisition module 201 upward adjusts the current restricted bandwidth corresponding to at least some of the priorities, it specifically is used for: subtracting the sum of the current actual bandwidths of the multiple priorities from the total restricted bandwidth of the node to obtain the idle bandwidth; sequentially upward adjusting the current restricted bandwidths corresponding to each priority in descending order of priority until the total upward adjustment amplitude reaches the idle bandwidth; wherein, for any priority other than the last adjusted priority among the at least some priorities that have been upward adjusted, the adjusted current restricted bandwidth is the corresponding maximum restricted bandwidth.

[0104] When the bandwidth acquisition module 201 downward adjusts the current restricted bandwidth corresponding to at least some of the priorities, it specifically is used for: subtracting the total restricted bandwidth of the node from the sum of the current actual bandwidths of the multiple priorities to obtain the contention bandwidth; sequentially downward adjusting the current restricted bandwidths corresponding to each priority in ascending order of priority until the total downward adjustment amplitude reaches the contention bandwidth; wherein, for any priority other than the last adjusted priority among the at least some priorities that have been downward adjusted, the adjusted current restricted bandwidth is the corresponding minimum restricted bandwidth.

[0105] Specifically, the restricted bandwidth range includes a maximum restricted bandwidth and a minimum restricted bandwidth, the minimum restricted bandwidth is the minimum value of the restricted bandwidth range, and the maximum restricted bandwidth is the maximum value of the restricted bandwidth range.

[0106] When the bandwidth acquisition module 201 upward adjusts the current restricted bandwidth corresponding to some or all of the priorities, the difference obtained by subtracting the sum of the current actual bandwidths of the multiple priorities from the total restricted bandwidth of the node is the idle bandwidth, and then, in descending order of priority, sequentially upward adjusts the current restricted bandwidth corresponding to each priority until the upward adjustment amplitude is the same as the size of the idle bandwidth. For any priority other than the last adjusted priority among the some or all priorities that have been upward adjusted, the adjusted current restricted bandwidth is the corresponding maximum restricted bandwidth.

[0107] Exemplarily, the priorities are sorted from high to low. First, adjust the priority ranked first. Subtract the current restricted bandwidth corresponding to this priority from the highest restricted bandwidth corresponding to this priority. Compare the obtained difference b with the size a of the idle bandwidth. If the obtained difference b is greater than the size a of the idle bandwidth, the upward adjustment amplitude of the priority ranked first is equal to the size a of the idle bandwidth, and the priority ranked second will no longer be adjusted; if the obtained difference b is less than the size a of the idle bandwidth, the current restricted bandwidth after adjustment of the priority ranked first is the highest restricted bandwidth corresponding to this priority. At this time, the remaining size of the idle bandwidth is a - b. Then, continue to adjust the current restricted bandwidth of the priority ranked second. Subtract the current restricted bandwidth corresponding to this priority from the highest restricted bandwidth corresponding to this priority. Compare the obtained difference c with a - b. If it is greater, the upward adjustment amplitude of the priority ranked second is equal to the size a - b of the idle bandwidth at this time, and the priority ranked third will no longer be adjusted; if the obtained difference c is less than the size a - b of the idle bandwidth, the current restricted bandwidth after adjustment of the priority ranked second is the highest restricted bandwidth corresponding to this priority. At this time, the remaining size of the idle bandwidth is a - b - c. Then, continue to increase the current restricted bandwidths of the priorities ranked third, fourth... The principle is the same and will not be elaborated here.

[0108] For example, there are three priorities, which are arranged in descending order as LO, L1, and L2. The corresponding current actual bandwidths are 10M, 20M, and 30M respectively. The total restricted bandwidth of the node is 70M, and 10M + 20M + 30M < 70M. The size of the idle bandwidth is 70M - (10M + 20M + 30M) = 10M. Therefore, it is necessary to increase the current restricted bandwidths of some or all of the priorities.

[0109] The specific method is as follows: The current restricted bandwidths corresponding to LO, L1, and L2 are 10M, 25M, and 35M respectively, and the corresponding maximum restricted bandwidths are 30M, 30M, and 45M respectively. First, adjust the current restricted bandwidth of LO. Since 30M - 10M = 20M, which is greater than the size 10M of the idle bandwidth, the upward adjustment amplitude of the current restricted bandwidth of LO is equal to the size 10M of the idle bandwidth.

[0110] If the size of the idle bandwidth is 30M, since 20M is less than 30M, the adjustment range of the current limit bandwidth of LO is 20M, and the current limit bandwidth of LO after adjustment is 30M, which is equal to the highest limit bandwidth corresponding to LO. At this time, the size of the idle bandwidth becomes 30M - 20M = 10M. Therefore, the adjustment of the current limit bandwidth of L1 is started. Since 30M - 25M = 5M, which is less than the current idle bandwidth of 10M, the adjustment range of the current limit bandwidth of L1 is 5M, and the current limit bandwidth of L1 after adjustment is 30M, which is equal to the highest limit bandwidth corresponding to L1. At this time, the size of the idle bandwidth becomes 10M - 5M = 5M. Then, the adjustment of the current limit bandwidth of L2 is carried out. Since 45M - 35M = 10M, which is greater than the size of the idle bandwidth of 5M at this time, the adjustment range of the current limit bandwidth of L2 is the size of the idle bandwidth of 5M.

[0111] When the bandwidth acquisition module 201 downward adjusts the current limit bandwidth corresponding to some or all of the priorities, the sum of the current actual bandwidths of multiple priorities is subtracted from the total limit bandwidth of the node, and the obtained difference is the contention bandwidth. Then, in the order from the lowest priority to the highest priority, the current limit bandwidth corresponding to each priority is successively downregulated until the total downregulation range is the same as the size of the contention bandwidth. Among the part or all of the priorities that have been downregulated, for any priority other than the last adjusted priority, the current limit bandwidth after adjustment is the corresponding lowest limit bandwidth.

[0112] Exemplarily, the priorities are sorted from the lowest to the highest. First, the priority ranked first is adjusted. The difference e obtained by subtracting the lowest limit bandwidth corresponding to this priority from the current limit bandwidth corresponding to this priority is compared with the size f of the contention bandwidth. If the obtained difference e is greater than the size f of the contention bandwidth, the downregulation range of the priority ranked first is equal to the size f of the contention bandwidth, and the priority ranked second is no longer adjusted; if the obtained difference e is less than the size f of the contention bandwidth, the current limit bandwidth of the priority ranked first after adjustment is the lowest limit bandwidth corresponding to this priority. At this time, the remaining size of the contention bandwidth is f - e. Then, the adjustment of the current limit bandwidth of the priority ranked second is continued. The difference g obtained by subtracting the lowest limit bandwidth corresponding to this priority from the current limit bandwidth corresponding to this priority is compared with f - e. If it is greater, the downregulation range of the priority ranked second is equal to the size f - e of the contention bandwidth at this time, and the priority ranked third is no longer adjusted; if the obtained difference g is less than the size f - e of the contention bandwidth, the current limit bandwidth of the priority ranked second after adjustment is the lowest limit bandwidth corresponding to this priority. At this time, the remaining size of the contention bandwidth is f - e - g. Then, the downregulation of the current limit bandwidths of the priorities ranked third, fourth... is continued. The principle is the same and will not be elaborated here.

[0113] For example, there are three priorities, which are arranged in ascending order as L2, L1, and L0. The corresponding current actual bandwidths are 30M, 20M, and 30M respectively. The total limited bandwidth of the node is 70M. Since 30M + 20M + 30M > 70M, the size of the bandwidth contention is (30M + 20M + 30M) - 70M = 20M. Therefore, it is necessary to lower the current limited bandwidth of some or all priorities.

[0114] The specific method is as follows: The current limited bandwidths corresponding to L0, L1, and L2 are 10M, 25M, and 35M respectively, and the corresponding minimum limited bandwidths are 5M, 15M, and 20M respectively. First, adjust the current limited bandwidth of L2. Since 35M - 20M = 15M, and since 15M is less than the bandwidth contention of 20M, the adjustment range of the current limited bandwidth of L2 is 15M. After adjustment, the current limited bandwidth of L0 is 20M, which is equal to the minimum limited bandwidth corresponding to L2. At this time, the size of the bandwidth contention becomes 20M - 15M = 5M. Therefore, start to adjust the current limited bandwidth of L1. Since 25M - 15M = 10M, which is greater than the current bandwidth contention of 5M, the adjustment range of the current limited bandwidth of L1 is the size of the bandwidth contention of 5M. After adjustment, the current limited bandwidth of L1 is 20M, and no longer adjust the current limited bandwidth of L0.

[0115] In this way, when there is idle bandwidth, the current limited bandwidths corresponding to some or all priorities can be increased in descending order of priority, and the current limited bandwidths after adjustment for each priority except the last adjusted priority can reach the maximum limited bandwidth, which can improve the processing speed of higher-priority tasks as much as possible; when there is bandwidth contention, the current limited bandwidths corresponding to some or all priorities are decreased in ascending order of priority, and the current limited bandwidths after adjustment for the priorities except the last adjusted priority are the minimum limited bandwidths, which can avoid affecting higher-priority tasks due to the contention of lower-priority tasks as much as possible.

[0116] The bandwidth limiting module 202 is used to determine the target priority corresponding to the data packet to be sent, and determine whether to release the data packet according to the current actual bandwidth and the current limited bandwidth corresponding to the target priority, where the target priority is one of multiple priorities.

[0117] Optionally, when the bandwidth limiting module 202 determines whether to release the data packet according to the current actual bandwidth and the current limited bandwidth corresponding to the target priority, it is specifically used for:

[0118] If the current actual bandwidth corresponding to the target priority is less than the current limited bandwidth, release the data packet;

[0119] If the current actual bandwidth corresponding to the target priority is not less than the current restricted bandwidth, discard the data packet.

[0120] In summary, the bandwidth restriction system provided in this embodiment includes: a bandwidth collection module 201, configured to collect the current actual bandwidth corresponding to each priority among multiple priorities, and adjust the current restricted bandwidth corresponding to each priority according to the current actual bandwidth corresponding to the multiple priorities, the total restricted bandwidth of the node pre-configured, and the restricted bandwidth range corresponding to each priority; a bandwidth restriction module 202, configured to determine the target priority corresponding to the data packet to be sent, and determine whether to release the data packet according to the current actual bandwidth and the current restricted bandwidth corresponding to the target priority. The configuration rules are simple and easy to maintain, the node processing efficiency is high, and in the case of mixed deployment of multiple priorities, dynamic and precise control of the bandwidth corresponding to each priority can be achieved, improving the utilization rate of node resources.

[0121] Optionally, the bandwidth restriction system further includes: a configuration module, configured to obtain the total restricted bandwidth and the restricted bandwidth range corresponding to each priority sent by an application programming interface (API) server.

[0122] Optionally, the configuration module may specifically be configured to obtain the total restricted bandwidth and the restricted bandwidth range corresponding to each priority configured by a user through the API server.

[0123] Exemplarily, Figure 3 is a schematic diagram of a user-side configuration interface provided by an embodiment of the present application. As Figure 3 shown, to configure the total restricted bandwidth of the node and the restricted bandwidth ranges corresponding to priorities L0, L1, and L2 for node 1, the restricted bandwidth range includes the maximum restricted bandwidth and the minimum restricted bandwidth. After the user enters the set values in the corresponding input boxes and clicks the confirmation button, the user side sends the total restricted bandwidth of the node and the restricted bandwidth ranges corresponding to each priority to the configuration module through the API server.

[0124] In this way, the user can flexibly configure the total restricted bandwidth of the node and the restricted bandwidth ranges corresponding to each priority according to requirements, improving the flexibility of configuration and modification.

[0125] Optionally, at least one container is deployed on the node, and the at least one container is used to process data packets;

[0126] The configuration module is further configured to: obtain the priority corresponding to each container;

[0127] When determining the target priority corresponding to the data packet to be sent, the bandwidth restriction module 202 is specifically configured to:

[0128] Determine the container corresponding to the data packet according to the characteristic information of the data packet, where the target priority of the data packet is the priority of the corresponding container.

[0129] Specifically, the configuration module can also obtain the priorities corresponding to each container configured by the user through the application programming interface server.

[0130] Multiple containers are running in the node, and the user can divide the containers into multiple priorities according to the types of tasks processed by the applications running in the containers.

[0131] In an example, when the type of task corresponding to the container is an online task, the priority can be set to L0, and when the type of task corresponding to the container is an offline task, the priority can be set to L1, where the priority L0 is higher than L1.

[0132] In another example, the online tasks and offline tasks can be more finely divided to obtain more priority types. For example, the online tasks are divided into live broadcast tasks and voice call tasks. The priority corresponding to the live broadcast task is set to L0, and the priority corresponding to the voice call task is set to L1. The offline tasks are divided into model training tasks and data analysis tasks. The priority corresponding to the model training task is set to L2, and the priority corresponding to the data analysis task is set to L3, where the priority L0 > L1 > L2 > L3.

[0133] The bandwidth limit module 202 can determine the container corresponding to the data packet according to the characteristic information of the data packet, and the priority of the container is the target priority of the data packet.

[0134] Among them, the characteristic information can be IP (Internet Protocol) information, port information or cgroup (control group) id (Identity document). IP and port can be used in combination, that is, the container corresponding to the data packet is determined jointly according to the IP and port of the data packet. The cgroup id can be used alone, that is, the container corresponding to the data packet can be determined only according to the cgroup id of the data packet.

[0135] This application can determine the container corresponding to the data packet according to the characteristic information of the data packet, and determine the priority corresponding to the data packet according to the container, without relying on a complex tree structure, which can improve the efficiency of priority judgment. Moreover, in the cloud native scenario, when there are multiple network namespaces on the same node and the Qdisc scheme is adopted, there will be a problem that the classification id is lost when the data packet traverses different network namespaces, resulting in the inability to classify the data packet. In this embodiment, the priority is determined through the container corresponding to the data packet. Even if the node corresponds to multiple different network namespaces, it does not affect the priority judgment, effectively improving the accuracy of determining the priority.

[0136] Figure 4 This is a schematic diagram of the working process of a bandwidth acquisition module 201 provided by an embodiment of the present application. As Figure 4 shown, when a data packet is detected to enter, first determine the priority corresponding to the data packet according to the IP and / or port of the data packet. For example, when the IP is 127.0.0.1, the corresponding priority is L2, and when the IP is 127.0.0.2 and the port (Dport) is 60, the corresponding priority is L0. Classify all data packets entering the bandwidth acquisition module 201 within a preset period according to the priority, calculate the current actual bandwidth (current_bps) corresponding to priorities L0, L1, and L2 and store them. Sum up the current actual bandwidths corresponding to priorities L0, L1, and L2, and compare with the total node limit bandwidth. According to the comparison result and the limit bandwidth ranges corresponding to priorities L0, L1, and L2 configured by the user, the minimum value of the limit bandwidth range is the minimum limit bandwidth (min_bps), and the maximum value is the maximum limit bandwidth (max_bpx). Adjust the current limit bandwidth (max_bps) of L0, L1, and L2, and store the adjusted current limit bandwidth.

[0137] Figure 5 This is a schematic diagram of the working process of a bandwidth limit module 202 provided by an embodiment of the present application. As Figure 5 shown, when a data packet is detected to enter, first judge the priority of the data packet, and then obtain the current actual bandwidth (current_bps) and the current limit bandwidth (max_bps) corresponding to the priority according to the priority of the data packet. Judge whether the current actual bandwidth corresponding to the priority is less than the current limit bandwidth. If it is less, release the data packet. If it is not less, discard the data packet.

[0138] Optionally, the node includes at least one network card; the bandwidth acquisition module 201 and the bandwidth limit module 202 are used to be mounted on the at least one network card to perform bandwidth acquisition and bandwidth limitation on the data packets corresponding to the at least one network card.

[0139] Specifically, the node includes at least one network card, and each network card's TC module in the at least one network card mounts a bandwidth acquisition module 201 and a bandwidth limit module 202. The bandwidth acquisition module 201 can perform bandwidth acquisition on the data packets in the corresponding network card, and the bandwidth limit module 202 can perform bandwidth limitation on the data packets in the corresponding network card.

[0140] Optionally, the configuration module is further configured to, in response to the addition of a node network card, automatically mount the bandwidth acquisition module 201 and the bandwidth limit module 202 on the TC module of the newly added network card. The bandwidth acquisition module 201 and the bandwidth limit module 202 can be mounted on multiple physical network cards, and the program operation does not depend on the Qdisc lock, and multiple processor cores are fully utilized for operation.

[0141] Optionally, each of the at least one network card corresponds to an input queue and an output queue, wherein the input queue of any network card includes data packets sent from the network card to the container, and the output queue includes data packets sent from the container to the network card;

[0142] The bandwidth acquisition module 201 is specifically configured to perform bandwidth acquisition on the input queue and the output queue respectively; the bandwidth limit module 202 is specifically configured to perform bandwidth limitation on the input queue and the output queue respectively.

[0143] Specifically, each of the at least one network card corresponds to an input queue and an output queue, wherein the input queue includes data packets sent from the network card to the container, and the output queue includes data packets sent from the container to the network card. The bandwidth acquisition module 201 can perform bandwidth acquisition on the input queue and the output queue respectively, and the bandwidth limit module 202 can perform bandwidth limitation on the input queue and the output queue respectively.

[0144] Optionally, in the egress direction, the EDT method can be used to set the earliest departure time for the data packets in the output queue, and the time-based scheduling policy is used to send the data packets, which can effectively reduce the occupancy of the buffer and the increase of latency under high traffic. In the ingress direction, methods such as the token bucket implemented by the extended Berkeley Packet Filter eBPF can be used to send the data packets, and the bandwidth acquisition module 201 and the bandwidth limit module 202 are used to determine whether to release the data packets to be sent.

[0145] In this way, by distinguishing the traffic in the ingress direction and the egress direction of each network card and performing bandwidth acquisition and bandwidth limitation respectively, the resource utilization rate of the node and the data packet processing efficiency can be improved.

[0146] Optionally, the bandwidth acquisition module 201 and the bandwidth limit module 202 can perform bandwidth acquisition and bandwidth limitation only on the input queue, or only on the output queue, or perform bandwidth acquisition and bandwidth limitation on both the input queue and the output queue at the same time. This application does not make any restrictions on this.

[0147] Optionally, the total limited bandwidth of the node includes the total limited bandwidth of the ingress and the total limited bandwidth of the egress. The limited bandwidth ranges corresponding to each priority include the limited bandwidth range of the ingress and the limited bandwidth range of the egress, so as to perform bandwidth collection and bandwidth limitation on the ingress queue according to the total limited bandwidth of the ingress and the limited bandwidth ranges of the ingress corresponding to each priority, and perform bandwidth collection and bandwidth limitation on the egress queue according to the total limited bandwidth of the egress and the limited bandwidth ranges of the egress corresponding to each priority.

[0148] Specifically, the network card corresponds to the ingress queue and the egress queue. The ingress queue corresponds to the ingress, and the egress queue corresponds to the egress. The total limited bandwidth of the node includes the total limited bandwidth of the ingress and the total limited bandwidth of the egress. The limited bandwidth range corresponding to each priority in each of the priorities also includes the limited bandwidth range of the ingress and the limited bandwidth range of the egress. The bandwidth collection module 201 and the bandwidth limitation module 202 can perform bandwidth collection and bandwidth limitation on the ingress queue according to the total limited bandwidth of the ingress and the limited bandwidth range of the ingress corresponding to each priority; the bandwidth collection module 201 and the bandwidth limitation module 202 can perform bandwidth collection and bandwidth limitation on the egress queue according to the total limited bandwidth of the egress and the limited bandwidth range of the egress corresponding to each priority.

[0149] In this way, for the ingress and the egress, the total limited bandwidth and the limited bandwidth ranges corresponding to each priority are set respectively, and bandwidth collection is performed on the ingress queue and the egress queue respectively to determine the current actual bandwidth of each priority, and the current limited bandwidth is determined according to the current actual bandwidth, the total limited bandwidth and the limited bandwidth range of each priority, so that different bandwidth limitation strategies are adopted for the ingress queue and the egress queue, improving the accuracy of the determined current limited bandwidth.

[0150] Optionally, when the bandwidth collection module 201 collects the current actual bandwidth corresponding to each priority among multiple priorities, it is specifically used for:

[0151] Within a preset period, for the data packets currently received or sent by the container, determine the priority corresponding to the data packet, and accumulate the size of the data packet to the traffic size corresponding to the priority;

[0152] According to the traffic sizes of each priority within the preset period, determine the current actual bandwidth of each priority.

[0153] Among them, the preset period can be flexibly set according to actual needs, and this is not limited herein.

[0154] Specifically, within the preset period, the network card sends multiple data packets to the container, or the container sends multiple data packets to the network card. For each data packet, determine the priority corresponding to the data packet. For each priority, accumulate the sizes of the data packets of this priority to obtain the traffic size corresponding to this priority, and determine the current actual bandwidth of this priority according to the traffic size of this priority within the preset period.

[0155] Exemplarily, the node includes two containers, namely container 1 and container 2. The priority corresponding to container 1 is L0, and the priority corresponding to container 2 is L1. The preset period is set to 1 s. Within 1 s, the network card sends 3 data packets to container 1, namely data packet 1, data packet 2, and data packet 3. The priorities corresponding to these three data packets are all L0; 2 data packets are sent to container 2, namely data packet 4 and data packet 5. The priorities corresponding to these two data packets are both L1. Therefore, the sizes of data packet 1, data packet 2, and data packet 3 are accumulated, and based on the accumulation result, the current actual bandwidth corresponding to priority L0 can be determined. The sizes of data packet 4 and data packet 5 are accumulated, and based on the accumulation result, the current actual bandwidth corresponding to priority L1 can be determined.

[0156] In this way, first classify the data packets received or sent by the container within the preset period according to the priority, accumulate the sizes of the data packets belonging to the same priority, and based on the accumulation result, the current actual bandwidth of this priority can be judged, which can improve the accuracy of the current actual bandwidth of each priority calculated.

[0157] Optionally, the bandwidth acquisition module 201 is further configured to:

[0158] After accumulating the size of the data packet to the traffic size corresponding to the priority, update the timestamp corresponding to the priority to the timestamp corresponding to the data packet.

[0159] Specifically, for each data packet, after accumulating the size of the data packet to the traffic size corresponding to the priority of this data packet, update the timestamp corresponding to this priority, and update the timestamp corresponding to this priority to the timestamp corresponding to the data packet.

[0160] When the bandwidth acquisition module 201 determines the current actual bandwidth of each priority according to the traffic sizes of each priority within the preset period, it is specifically configured to:

[0161] Calculate the bandwidth size of each sampling point among multiple sampling points within the preset period according to the timestamp and traffic size corresponding to the priority, and determine the current actual bandwidth of the priority within the preset period according to the bandwidth sizes of multiple sampling points.

[0162] Specifically, for each priority level, when calculating the current actual bandwidth within a preset period, multiple sampling points can be selected within the preset period. The current actual bandwidth is calculated at each sampling point. The current actual bandwidth at each sampling point is calculated based on the timestamp and traffic volume corresponding to this priority level. Then, the average value of the current actual bandwidths corresponding to multiple sampling points is calculated to determine the current actual bandwidth corresponding to this priority level within the preset period. Specifically, for any priority level, the cumulative size and timestamp of the data packets corresponding to this priority level may change continuously. A preset period is divided into multiple sampling points, and each sampling point corresponds to a moment. For any sampling point, based on the change in the cumulative size of the data packets at different timestamps, the cumulative size of the data packets at the moment corresponding to this sampling point can be determined. Thus, based on the cumulative size of the data packets at the moment corresponding to this sampling point, the current actual bandwidth corresponding to this sampling point is calculated. Determining the current actual bandwidth of this priority level within the preset period based on the current actual bandwidths of multiple sampling points can reduce the probability of inaccurate measurement of the current actual bandwidth caused by traffic jitter and improve the accuracy of the current actual bandwidth measurement.

[0163] In the embodiments of the present application, the current restricted bandwidth can be adjusted according to the pre-configured information and the actually collected information, so that the adjusted current restricted bandwidth better meets the actual requirements. For example, if it is determined that the current state is relatively idle based on the current actual bandwidths of each priority level, the current restricted bandwidth of one or more priority levels can be increased to make the most of the bandwidth resources as much as possible. At the same time, the degree of increase can consider the total restricted bandwidth of the node to avoid excessive actual bandwidth after the increase, which may affect the overall performance. If it is determined that the current state is relatively busy based on the current actual bandwidths of each priority level, the current restricted bandwidth of one or more priority levels can be decreased to reduce bandwidth contention.

[0164] Optionally, the bandwidth acquisition module 201 and the bandwidth restriction module 202 are implemented through the extended Berkeley Packet Filtering (eBPF) technology.

[0165] Specifically, the bandwidth acquisition module 201 and the bandwidth restriction module 202 can be obtained through programming with the extended Berkeley Packet Filtering (eBPF) technology.

[0166] Traditional speed limiting solutions all rely on existing kernel modules in the kernel. When it is necessary to adjust the policy or add new functions, the kernel modules need to be modified, which is very inflexible. Through the programmable method of eBPF in the present application, the execution logic in the Linux kernel can be modified without replacing the kernel module, and it can take effect in real time.

[0167] Optionally, the total restricted bandwidth of the node and the restricted bandwidth ranges corresponding to each priority level are recorded in the data storage structure (eBPF map) of eBPF;

[0168] The bandwidth acquisition module 201 is further configured to: read the total limited bandwidth of the node and the limited bandwidth ranges corresponding to each priority level from the data storage structure of the eBPF, and record the current actual bandwidths corresponding to each priority level and the adjusted current limited bandwidths into the data storage structure of the eBPF;

[0169] The bandwidth limitation module 202 is further configured to: read the current actual bandwidth and the current limited bandwidth corresponding to the target priority level from the data storage structure of the eBPF.

[0170] Specifically, the configuration module is further configured to record the total limited bandwidth of the node configured by the user received and the limited bandwidth ranges corresponding to each of the multiple priority levels into the eBPF map.

[0171] Specifically, the bandwidth acquisition module 201 acquires the current actual bandwidths corresponding to each priority level, then sums up the current actual bandwidths corresponding to each priority level, compares the summation result with the total limited bandwidth of the node, adjusts the current limited bandwidths corresponding to each priority level according to the comparison result, and records the adjusted current limited bandwidths corresponding to each priority level and the current actual bandwidths of each priority level into the eBPF map. The bandwidth limitation module 202 reads the current limited bandwidths corresponding to each priority level and the current actual bandwidths of each priority level from the eBPF map, and decides whether to forward the data packet based on the read current limited bandwidths corresponding to each priority level and the current actual bandwidths of each priority level.

[0172] Optionally, for the case where the bandwidth acquisition module 201 is respectively mounted on each of multiple network cards, each bandwidth acquisition module 201 among the multiple bandwidth acquisition modules 201 may calculate the current actual bandwidths of each priority level corresponding to the corresponding network card every preset period and write them into the eBPF map. Since the periodic recording method is adopted, the contention occurring during reading and writing the eBPF map can be effectively reduced. For each priority level, the current actual bandwidths of the corresponding network card of the priority level acquired by each bandwidth acquisition module 201 are accumulated to obtain the current actual bandwidth corresponding to the priority level. The current actual bandwidths corresponding to each priority level are summed up, the summation result is compared with the total limited bandwidth of the node, the current limited bandwidths corresponding to each priority level are adjusted according to the comparison result, and the adjusted current limited bandwidths corresponding to each priority level and the current actual bandwidths of each priority level are recorded into the eBPF map. The bandwidth limitation module 202 reads the current limited bandwidths corresponding to each priority level and the current actual total bandwidths of each priority level from the eBPF map and decides whether to forward the data packet.

[0173] In this way, the dynamic borrowing and returning of multi-network card traffic can be achieved by using the eBPF map, improving the overall resource utilization rate of the node and the processing efficiency of the node. Through the eBPF map as the configuration management entry, the speed limit policy can be set efficiently and conveniently, and the configuration is simple.

[0174] In summary, the embodiment of the present application provides a bandwidth limitation system, which can solve the problem that the traditional bandwidth limitation scheme cannot support the bandwidth contention, borrowing and returning of multiple tasks in the hybrid deployment scenario, support any number of network cards of the node, and has the ability to dynamically adjust the bandwidth limitation policy. It can automatically perform bandwidth allocation and adjustment according to the real-time task requirements and bandwidth resource conditions to achieve better bandwidth utilization and task performance.

[0175] In addition, the embodiment of the present application can be particularly applied to a cluster with hybrid resource deployment in the cloud native scenario, seamlessly integrated with cloud native technologies such as containerization and microservices. By docking with the interface of the cloud platform, precise monitoring, allocation and management of online and offline bandwidth can be realized, improving the resource utilization rate and performance in the cloud native environment, supporting large bandwidths above 10G, and realizing stable control and management of high-speed data streams.

[0176] The embodiment of the present application also provides a bandwidth limitation method. Figure 6 It is a schematic flowchart of a bandwidth limitation method provided by the embodiment of the present application. As Figure 6 shown, the execution subject can be any node in the cluster, and the cluster is used to process packets of multiple priorities. The method includes:

[0177] Step 601, collect the current actual bandwidth corresponding to each priority among multiple priorities, and adjust the current limitation bandwidth corresponding to each priority according to the current actual bandwidth corresponding to the multiple priorities, the total limitation bandwidth of the node pre-configured, and the limitation bandwidth range corresponding to each priority;

[0178] Step 602, determine the target priority corresponding to the packet to be sent, and determine whether to release the packet according to the current actual bandwidth and the current limitation bandwidth corresponding to the target priority, where the target priority is one of the multiple priorities.

[0179] Optionally, step 601 can be implemented by a bandwidth collection module, and step 602 can be implemented by a bandwidth limitation module.

[0180] Optionally, the bandwidth limitation method provided by the present application further includes:

[0181] Obtain the total limitation bandwidth and the limitation bandwidth range corresponding to each priority sent by the application program interface server.

[0182] Optionally, at least one container is deployed on the node, and the at least one container is used to process data packets. The bandwidth limiting method provided in this application further includes:

[0183] Obtain the priorities corresponding to the containers;

[0184] Determine the target priority corresponding to the data packet to be sent, including:

[0185] According to the characteristic information of the data packet, determine the container corresponding to the data packet, where the target priority of the data packet is the priority of the corresponding container.

[0186] Optionally, the node includes at least one network card; the bandwidth collection module and the bandwidth limiting module are used to be mounted on the at least one network card to perform bandwidth collection and bandwidth limiting on the data packets corresponding to the at least one network card.

[0187] Optionally, each of the at least one network card corresponds to an input queue and an output queue, where the input queue of any network card includes the data packets sent by the network card to the container, and the output queue includes the data packets sent by the container to the network card;

[0188] The bandwidth collection module is specifically used to perform bandwidth collection on the input queue and the output queue respectively; the bandwidth limiting module is specifically used to perform bandwidth limiting on the input queue and the output queue respectively.

[0189] Optionally, the total limited bandwidth of the node includes the total limited bandwidth of the entrance and the total limited bandwidth of the exit. The limited bandwidth ranges corresponding to each priority include the limited bandwidth range of the entrance and the limited bandwidth range of the exit, so as to perform bandwidth collection and bandwidth limiting on the input queue according to the total limited bandwidth of the entrance and the limited bandwidth ranges of the entrances corresponding to each priority, and perform bandwidth collection and bandwidth limiting on the output queue according to the total limited bandwidth of the exit and the limited bandwidth ranges of the exits corresponding to each priority.

[0190] Optionally, according to the current actual bandwidths corresponding to the multiple priorities, the total limited bandwidth of the node pre-configured, and the limited bandwidth ranges corresponding to each priority, adjust the current limited bandwidths corresponding to each priority, including:

[0191] If the sum of the current actual bandwidths corresponding to the multiple priorities is less than the total limited bandwidth of the node, then upward-adjust the current limited bandwidths corresponding to at least some priorities; and / or,

[0192] If the sum of the current actual bandwidths corresponding to the multiple priorities is greater than the total limited bandwidth of the node, then downward-adjust the current limited bandwidths corresponding to at least some priorities;

[0193] Among them, the current restricted bandwidth before and after the adjustment of any priority is within the corresponding restricted bandwidth range.

[0194] Optionally, the restricted bandwidth range includes the highest restricted bandwidth and the lowest restricted bandwidth; adjusting the current restricted bandwidth corresponding to at least some priorities upward includes: subtracting the sum of the current actual bandwidths of the multiple priorities from the total restricted bandwidth of the node to obtain the idle bandwidth; sequentially increasing the current restricted bandwidth corresponding to each priority in descending order of priority until the total increase reaches the idle bandwidth;

[0195] Among them, for any priority other than the last adjusted priority among the at least some priorities that have been increased, the current restricted bandwidth after adjustment is the corresponding highest restricted bandwidth;

[0196] Correspondingly, adjusting the current restricted bandwidth corresponding to at least some priorities downward includes: subtracting the total restricted bandwidth of the node from the sum of the current actual bandwidths of the multiple priorities to obtain the contention bandwidth; sequentially decreasing the current restricted bandwidth corresponding to each priority in ascending order of priority until the total decrease reaches the contention bandwidth;

[0197] Among them, for any priority other than the last adjusted priority among the at least some priorities that have been decreased, the current restricted bandwidth after adjustment is the corresponding lowest restricted bandwidth.

[0198] Optionally, determining whether to forward the data packet according to the current actual bandwidth and the current restricted bandwidth corresponding to the target priority includes:

[0199] If the current actual bandwidth corresponding to the target priority is less than the current restricted bandwidth, then forward the data packet;

[0200] If the current actual bandwidth corresponding to the target priority is not less than the current restricted bandwidth, then discard the data packet.

[0201] Optionally, collecting the current actual bandwidth corresponding to each priority among the multiple priorities includes:

[0202] Within a preset period, for the data packets currently received or sent by the container, determine the priority corresponding to the data packet, and accumulate the size of the data packet to the traffic size corresponding to the priority;

[0203] Determine the current actual bandwidth of each priority according to the traffic size of each priority within the preset period.

[0204] Optionally, the method further includes:

[0205] After accumulating the size of the data packet to the traffic size corresponding to the priority, update the timestamp corresponding to the priority to the timestamp corresponding to the data packet;

[0206] Determine the current actual bandwidth of each priority according to the traffic size of each priority within a preset period, including:

[0207] For any priority, calculate the bandwidth size of each sampling point among multiple sampling points within the preset period according to the timestamp and traffic size corresponding to the priority, and determine the current actual bandwidth of the priority within the preset period according to the bandwidth sizes of multiple sampling points.

[0208] Optionally, the method is implemented by the extended Berkeley Packet Filtering (eBPF) technology.

[0209] Optionally, the total limited bandwidth of the node and the limited bandwidth ranges corresponding to each priority are recorded in the data storage structure of eBPF; the method further includes:

[0210] Read the total limited bandwidth of the node and the limited bandwidth ranges corresponding to each priority from the data storage structure of eBPF, and record the current actual bandwidth and the adjusted current limited bandwidth corresponding to each priority into the data storage structure of eBPF;

[0211] Read the current actual bandwidth and the current limited bandwidth corresponding to the target priority from the data storage structure of eBPF.

[0212] For the specific implementation principle and effect of the bandwidth limiting method provided in the embodiments of the present application, reference can be made to the technical solutions of the above embodiments, which will not be elaborated here.

[0213] Figure 7 This is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 7 shown, the electronic device of this embodiment may include:

[0214] At least one processor 701; and

[0215] A memory 702 communicatively connected to the at least one processor;

[0216] Wherein, the memory 702 stores instructions executable by the at least one processor 701, and when the instructions are executed by the at least one processor 701, the electronic device is caused to execute the method as described in any of the above embodiments.

[0217] Optionally, the memory 702 can be either independent or integrated with the processor 701.

[0218] The implementation principles and technical effects of the electronic device provided in this embodiment can be referred to the foregoing embodiments, and will not be elaborated herein.

[0219] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the method described in any of the foregoing embodiments is implemented.

[0220] An embodiment of the present application further provides a computer program product, including a computer program that implements the method described in any of the foregoing embodiments when executed by a processor.

[0221] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0222] The integrated modules implemented in the form of software function modules described above can be stored in a computer-readable storage medium. The above software function modules are stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute some steps of the methods described in various embodiments of the present application.

[0223] It should be understood that the above processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the application can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor. The memory may include high-speed random access memory (RAM), and may also include non-volatile memory (NVM), such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disc, etc.

[0224] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically-Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable read-only memory (PROM), Read-Only Memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk. The storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0225] An exemplary storage medium is coupled to a processor so that the processor can read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic device or a master device.

[0226] It should be noted that in this article, the terms "including", "comprising", or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article, or device. Without further limitations, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article, or device including the element.

[0227] The serial numbers of the embodiments of the present application above are only for description and do not represent the superiority or inferiority of the embodiments.

[0228] Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0229] The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present application.

Claims

1. A bandwidth limiting system, characterized in that, Applied to any node in a cluster, where the cluster is used to process data packets of multiple priorities; the system includes: A bandwidth acquisition module, configured to acquire the current actual bandwidth corresponding to each priority among multiple priorities, and adjust the current restricted bandwidth corresponding to each priority according to the current actual bandwidths corresponding to the multiple priorities, the total restricted bandwidth of the node pre-configured, and the restricted bandwidth ranges corresponding to each priority; A bandwidth restriction module, configured to determine the target priority corresponding to the data packet to be sent, and determine whether to release the data packet according to the current actual bandwidth and the current restricted bandwidth corresponding to the target priority, where the target priority is one of the multiple priorities.

2. The system according to claim 1, wherein The bandwidth restriction system further includes: A configuration module, configured to obtain the total restricted bandwidth and the restricted bandwidth ranges corresponding to each priority sent by the application programming interface server.

3. The system according to claim 2, wherein At least one container is deployed on the node, and the at least one container is used to process data packets; The configuration module is further configured to: obtain the priorities corresponding to each container; When determining the target priority corresponding to the data packet to be sent, the bandwidth restriction module specifically is configured to: Determine the container corresponding to the data packet according to the characteristic information of the data packet, where the target priority of the data packet is the priority of the corresponding container.

4. The system according to claim 1, wherein The node includes at least one network card; the bandwidth acquisition module and the bandwidth restriction module are used to be mounted on the at least one network card to perform bandwidth acquisition and bandwidth restriction on the data packets corresponding to the at least one network card.

5. The system according to claim 4, wherein Each of the at least one network cards corresponds to an input queue and an output queue, where the input queue of any network card includes the data packets sent from the network card to the container, and the output queue includes the data packets sent from the container to the network card; The bandwidth acquisition module is specifically configured to perform bandwidth acquisition on the input queue and the output queue respectively; the bandwidth restriction module is specifically configured to perform bandwidth restriction on the input queue and the output queue respectively.

6. The system according to claim 5, wherein The total restricted bandwidth of the node includes the total restricted bandwidth of the entrance and the total restricted bandwidth of the exit, and the restricted bandwidth ranges corresponding to each priority include the restricted bandwidth range of the entrance and the restricted bandwidth range of the exit, so as to perform bandwidth acquisition and bandwidth restriction on the input queue according to the total restricted bandwidth of the entrance and the restricted bandwidth ranges of each priority of the entrance, and perform bandwidth acquisition and bandwidth restriction on the output queue according to the total restricted bandwidth of the exit and the restricted bandwidth ranges of each priority of the exit.

7. The system according to any one of claims 1-6, characterized in that, When adjusting the current restricted bandwidth corresponding to each priority according to the current actual bandwidths corresponding to the multiple priorities, the total restricted bandwidth of the node pre-configured, and the restricted bandwidth ranges corresponding to each priority, the bandwidth acquisition module specifically is configured to: If the sum of the current actual bandwidths corresponding to the multiple priorities is less than the total restricted bandwidth of the node, then upward-adjust the current restricted bandwidths corresponding to at least some priorities; and / or, If the sum of the current actual bandwidths corresponding to the multiple priorities is greater than the total restricted bandwidth of the node, then downward-adjust the current restricted bandwidths corresponding to at least some priorities; Wherein, the current restricted bandwidths before and after adjustment of any priority are both within the corresponding restricted bandwidth range.

8. The system according to claim 7, wherein The described restricted bandwidth range includes the maximum restricted bandwidth and the minimum restricted bandwidth; When the bandwidth acquisition module adjusts the current restricted bandwidth corresponding to at least some priorities upward, it specifically is used for: subtracting the sum of the current actual bandwidths of the multiple priorities from the total restricted bandwidth of the node to obtain the idle bandwidth; sequentially increasing the current restricted bandwidth corresponding to each priority in descending order of priority until the total increase amplitude reaches the idle bandwidth; Among them, for any priority other than the last adjusted priority in at least some priorities that have been increased, the adjusted current restricted bandwidth is the corresponding maximum restricted bandwidth; When the bandwidth acquisition module adjusts the current restricted bandwidth corresponding to at least some priorities downward, it specifically is used for: subtracting the total restricted bandwidth of the node from the sum of the current actual bandwidths of the multiple priorities to obtain the contention bandwidth; sequentially decreasing the current restricted bandwidth corresponding to each priority in ascending order of priority until the total decrease amplitude reaches the contention bandwidth; Among them, for any priority other than the last adjusted priority in at least some priorities that have been decreased, the adjusted current restricted bandwidth is the corresponding minimum restricted bandwidth.

9. The system according to any one of claims 1-6, characterized in that, When the bandwidth restriction module determines whether to release the data packet according to the current actual bandwidth and the current restricted bandwidth corresponding to the target priority, it specifically is used for: If the current actual bandwidth corresponding to the target priority is less than the current restricted bandwidth, release the data packet; If the current actual bandwidth corresponding to the target priority is not less than the current restricted bandwidth, discard the data packet.

10. The system according to any one of claims 1-6, characterized in that, When the bandwidth acquisition module acquires the current actual bandwidth corresponding to each priority among multiple priorities, it specifically is used for: Within a preset period, for the data packets currently received or sent by the container, determine the priority corresponding to the data packet, and accumulate the size of the data packet to the traffic size corresponding to the priority; Determine the current actual bandwidth of each priority according to the traffic size of each priority within the preset period.

11. The system according to claim 10, wherein The bandwidth acquisition module is also used for: After accumulating the size of the data packet to the traffic size corresponding to the priority, update the timestamp corresponding to the priority to the timestamp corresponding to the data packet; When the bandwidth acquisition module determines the current actual bandwidth of each priority according to the traffic size of each priority within the preset period, it specifically is used for: For any priority, calculate the bandwidth size of each sampling point among multiple sampling points within the preset period according to the timestamp and the traffic size corresponding to the priority, and determine the current actual bandwidth of the priority within the preset period according to the bandwidth sizes of the multiple sampling points.

12. The system according to any one of claims 1-6, characterized in that, The bandwidth acquisition module and the bandwidth restriction module are implemented through the extended Berkeley Packet Filtering eBPF technology.

13. The system according to claim 12, wherein, The total restricted bandwidth of the node and the restricted bandwidth range corresponding to each priority are recorded in the data storage structure of eBPF; The bandwidth acquisition module is further configured to: read the total limited bandwidth of the node and the limited bandwidth ranges corresponding to each priority from the data storage structure of the eBPF, and record the current actual bandwidth and the adjusted current limited bandwidth corresponding to each priority into the data storage structure of the eBPF; The bandwidth limitation module is further configured to: read the current actual bandwidth and the current limited bandwidth corresponding to the target priority from the data storage structure of the eBPF.

14. A bandwidth limiting method, characterized in that, Applied to any node in a cluster, the cluster is used for processing packets of multiple priorities, the method includes: Acquire the current actual bandwidth corresponding to each priority among multiple priorities, and adjust the current limited bandwidth corresponding to each priority according to the current actual bandwidths corresponding to the multiple priorities, the pre-configured total limited bandwidth of the node, and the limited bandwidth ranges corresponding to each priority; Determine the target priority corresponding to the packet to be sent, and determine whether to release the packet according to the current actual bandwidth and the current limited bandwidth corresponding to the target priority, where the target priority is one of the multiple priorities.

15. An electronic device, characterized in that, Including: At least one processor; And A memory communicatively connected to the at least one processor; Wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the electronic device executes the method according to claim 14.

16. A computer-readable storage medium, characterized in that, Computer-executable instructions are stored in the computer-readable storage medium, and when the processor executes the computer-executable instructions, the method according to claim 14 is implemented.

17. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the method according to claim 14 is implemented.

Images