Traffic control method, device and equipment and computer readable storage medium

Abstract

The application relates to the technical field of distributed storage, and discloses a traffic control method and device, equipment and a computer readable storage medium. The method comprises the following steps: a client determines a target process corresponding to an I / O request for accessing a remote storage node; an executable file of the target process is determined, and a priority label configured in advance for the executable file by a user is acquired; the target process is allocated to a resource control group corresponding to the priority label, the resource control group is pre-configured with a resource limit parameter corresponding to the priority label, the resource limit parameter is used for representing an upper limit of traffic, including at least one of an I / O bandwidth upper limit or an IOPS upper limit, of an I / O request initiated by the corresponding resource control group to the remote storage node; and the target process in the resource control group is controlled based on the corresponding resource limit parameter. The application can finely control the target process based on the corresponding resource limit parameter, so as to solve the problem of coarse granularity of the existing traffic control mode.

Description

Technical Field

[0001] This application relates to the field of distributed storage technology, and in particular to a traffic control method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] A distributed storage system primarily consists of business clients that issue data access requests and storage nodes that actually store the data. Clients read and write data using storage volumes provided by the storage nodes. With the widespread use of distributed storage technology, a single distributed storage system often needs to run multiple services simultaneously. Some are real-time services with high response speed requirements, while others are background tasks that are less sensitive to latency. While this hybrid approach improves resource utilization, it can also easily lead to different services competing for storage resources.

[0003] Existing solutions mainly rely on setting fixed quality of service parameters in advance, or performing simple I / O rate limiting locally. Administrators will pre-set fixed IOPS and bandwidth limits for storage volumes or clients using storage services, or perform simple traffic control based on the storage node's own local disk.

[0004] However, existing resource control methods are coarse-grained and cannot identify specific business types. They can only apply the same strategy to all traffic or control only at the storage volume level. This makes it easy for low-value businesses such as background backups to preempt the bandwidth and resources of core businesses, resulting in unstable operation of critical businesses and failure to achieve reasonable resource allocation. Summary of the Invention

[0005] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide a traffic control method, apparatus, device and computer-readable storage medium that can set a priority label in the executable file corresponding to the target process and assign the target process to the resource control group corresponding to the priority label, and perform fine-grained resource control on the target process based on the resource limit parameters configured in the resource control group, thereby solving the problem that the existing traffic control method is too coarse-grained, resulting in low-value services occupying core business resources and unreasonable resource allocation.

[0006] In a first aspect, this application provides a traffic control method applied to a distributed storage system, the system including a client and a remote storage node connected by communication, the method including:

[0007] The client determines the target process corresponding to the I / O request to access the remote storage node;

[0008] Identify the executable file of the target process and obtain the priority label that the user has pre-configured for the executable file;

[0009] The target process is assigned to the resource management group corresponding to the priority label. The resource management group is pre-configured with resource limit parameters corresponding to the priority label. The resource limit parameters are used to characterize the traffic limit of the corresponding resource management group to initiate I / O requests to the remote storage node, including at least one of the I / O bandwidth limit or IOPS limit.

[0010] Traffic control is applied to target processes within the resource management group based on the corresponding resource limitation parameters.

[0011] In conjunction with the first aspect, in one possible implementation, there are multiple target processes, and the priority label corresponding to each target process is any one of avoidance level, normal level or core level.

[0012] Each target process is assigned to a resource control group that corresponds to its priority label, which can be any one of the avoidance group, normal group, or core group.

[0013] In conjunction with the first aspect, in one possible implementation, after performing traffic control on the target processes within the resource control group based on the corresponding resource limit parameters, it also includes:

[0014] The target process of the core group is recorded as the core process. The first total latency information of multiple core processes on the client and the second total latency information on the remote storage node are obtained within the current collection period.

[0015] The first total delay information and the second total delay information are weighted and fused to obtain the end-to-end delay information;

[0016] Based on the end-to-end latency information, it is determined whether congestion has occurred. If so, the resource limit parameters of at least one other resource control group outside the core group are lowered within the current collection cycle until the congestion is relieved or the resource limit parameters of all resource control groups that have been lowered reach the corresponding lower limit threshold, and then the next collection cycle begins.

[0017] If not, proceed to the next data collection cycle.

[0018] In conjunction with the first aspect, one possible implementation method is to determine whether congestion has occurred based on end-to-end latency information, specifically as follows:

[0019] Determine whether the end-to-end latency information is greater than a preset first threshold. If yes, it is determined that congestion has occurred; otherwise, it is determined that no congestion has occurred or the congestion has been resolved.

[0020] In conjunction with the first aspect, in one possible implementation, the resource limit parameters of at least one other resource control group besides the core group are lowered during the current collection cycle until congestion is relieved or the resource limit parameters of all lowered resource control groups reach their corresponding lower thresholds, at which point the next collection cycle begins. Specifically:

[0021] The time delay deviation is obtained by subtracting the end-link time delay information from the first threshold.

[0022] Determine the reduction step size, which is used to characterize the extent of the reduction in resource constraint parameters for any other resource control group;

[0023] Based on the step size reduction, the resource limit parameters of at least one other resource control group are reduced at least once, and the end-to-end latency information in the current collection cycle is updated after each reduction, until the end-to-end latency information after the last update is less than or equal to the first threshold, or the resource limit parameters of all resource control groups that have been reduced reach the corresponding lower limit threshold, and then the next collection cycle begins.

[0024] In conjunction with the first aspect, in one possible implementation, the resource limit parameters of at least one other resource control group are lowered at least once based on the reduction step size, and the end-to-end latency information within the current collection period is updated after each reduction, until the end-to-end latency information after the last update is less than or equal to a first threshold, or the resource limit parameters of all the resource control groups that have been lowered reach their corresponding lower limit thresholds, before entering the next collection period, specifically:

[0025] Based on the reduction step size, the resource limit parameters of the avoidance group are reduced in the current round. After the reduction, the end-to-end latency information in the current collection period is updated, and it is determined whether the updated end-to-end latency information is less than or equal to the first threshold.

[0026] If not, determine whether the resource limit parameter of the avoidance group after the reduction in the current round is less than the second threshold;

[0027] If so, the resource limit parameters of the ordinary group will be reduced at least once based on the reduction step size. After each reduction of the resource limit parameters of the ordinary group, the end-to-end latency information in the current collection cycle will be updated. If, after the current number of reductions, the reduced resource limit parameters of the ordinary group are less than the preset third threshold and the updated end-to-end latency information is greater than the first threshold, then the current resource limit parameters of each resource control group will be maintained, and the next collection cycle will begin.

[0028] If not, the resource limit parameters of the avoidance group are lowered again based on the reduction step size, and the end-to-end latency information in the current collection cycle is updated again. It is determined whether the end-to-end latency information after the second update is less than or equal to the first threshold. If not, it is determined whether the resource limit parameters of the avoidance group after the second reduction are less than the second threshold. This process continues until the end-to-end latency information after the last update is less than or equal to the first threshold, or the resource limit parameters of all resource control groups that have been lowered reach the corresponding lower limit threshold, and then the next collection cycle begins.

[0029] In conjunction with the first aspect, one possible implementation includes, after entering the next acquisition cycle, the following:

[0030] Determine the recovery step size, and based on the recovery step size, restore each resource limit parameter that was downgraded in the previous period until each downgraded resource limit parameter is restored to its initial value before downgrading.

[0031] Secondly, this application also provides a flow control device, the device comprising:

[0032] The determination module is used to determine the target process corresponding to the I / O request to access the remote storage node;

[0033] The tag module is used to determine the executable file of the target process and obtain the priority tags that the user has pre-configured for the executable file;

[0034] The grouping module is used to assign the target process to the resource management group corresponding to the priority label. The resource management group is pre-configured with resource limit parameters corresponding to the priority label. The resource limit parameters are used to characterize the traffic limit of the corresponding resource management group to initiate I / O requests to the remote storage node, including at least one of the I / O bandwidth limit or IOPS limit.

[0035] The flow control module is used to control the flow of target processes within the resource control group based on the corresponding resource limit parameters.

[0036] Thirdly, this application also provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0037] The memory stores the instructions that the computer executes;

[0038] The processor executes computer-executable instructions stored in memory to implement any of the flow control methods in the first aspect.

[0039] Fourthly, this application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the flow control method of any of the first aspects.

[0040] Compared with the prior art, this application has the following beneficial effects:

[0041] This application addresses the problem of low-value businesses preempting core business resources and unreasonable resource allocation caused by the coarse granularity of existing traffic control methods by setting a priority label in the executable file corresponding to the target process and assigning the target process to the resource control group corresponding to the priority label, and by performing fine-grained resource control on the target process based on the resource limit parameters configured in the resource control group. Attached Figure Description

[0042] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0043] Figure 1 This is a flowchart illustrating the method described in this application;

[0044] Figure 2 This is another flowchart illustrating the method described in this application;

[0045] Figure 3 A flowchart illustrating the method for determining the congestion type in this application;

[0046] Figure 4 This is another flowchart illustrating the method described in this application;

[0047] Figure 5 This is a structural block diagram of the device of this application in one embodiment. Detailed Implementation

[0048] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0049] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present application will now be described in detail with reference to the accompanying drawings and embodiments. Furthermore, the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The terms "first" and "second," etc., in the specification and claims of the embodiments of this application are used to distinguish different objects, not to describe a specific order of objects.

[0050] The following explains the terms used in this application:

[0051] Distributed storage: A technology that distributes data across multiple independent physical nodes and forms a clustered storage resource through unified scheduling, which can improve system capacity, concurrency and reliability.

[0052] VFS (Virtual File System) hook mechanism: An event interception and extension mechanism based on the Linux virtual file system, used to capture, identify and intervene when I / O requests are submitted to the file system layer.

[0053] System calls are standard service interfaces provided by the operating system kernel to applications. Applications request low-level functions such as file operations, I / O operations, and process management from the kernel by initiating system calls. System calls are the core way for applications to interact with the kernel.

[0054] An I / O request (Input / Output Request) is a data read or write operation request initiated by a process or business program to a file system or storage device. It is the basic unit of resource usage and traffic control in a storage system.

[0055] xattr (Extended Attributes): In this application, it is used to set and persistently save priority tags for the executable binary files corresponding to business processes, providing a data basis for process priority identification and resource control group allocation.

[0056] PID control (Proportional-Integral-Derivative Control): In this application, it means that the step size for adjustment or restoration is dynamically calculated based on the delay deviation between the actual end-to-end delay and the target delay, so as to achieve flexible dynamic control of the resource management group.

[0057] Cgroups: A part of the Linux kernel used to limit, record, and isolate the physical resources (such as CPU, memory, disk I / O, etc.) used by process groups.

[0058] IOPS (Input / Output Operations Per Second): A core metric for measuring the read / write performance of a storage system, representing the number of I / O requests that can be processed per second. It is an important basis for resource management and rate limiting in this application.

[0059] Bandwidth limiting: By limiting the data transmission speed that can be read and written within a unit of time, the usage of storage and network resources by a business can be controlled, thus preventing a single business from excessively consuming system resources.

[0060] Netlink mechanism: A mechanism provided by the Linux kernel for asynchronous communication between kernel space and user space, enabling efficient transmission of process information, resource status, and management policies.

[0061] End-to-end latency: The total time taken from the start of an I / O request by the business process, through the file system, local client, and network transmission to the remote storage node, to the completion of request processing and the return of a response. It is used to reflect the end-to-end operating status of the distributed storage system and is an important basis for congestion judgment and pressure reduction in this application.

[0062] With the widespread application of distributed storage technology in critical infrastructure, a single storage cluster often needs to support multiple different types of business operations simultaneously, forming a mixed workload scenario. On the same physical node, there are often online transaction systems with extremely high latency requirements, as well as offline backup or big data analytics tasks with high data throughput requirements. While this mixed deployment improves the overall utilization of system resources, it also leads to significant resource contention between different business operations.

[0063] Traditional solutions to resource contention issues primarily rely on static quality of service (QoS) configurations or simple local I / O rate limiting. In practice, administrators pre-set fixed IOPS and bandwidth limits for designated storage volumes or business clients, or implement simple traffic limiting based on the request queue length of the local disk. These technologies can be effective in environments with simple business types and relatively stable data traffic, preventing a single business from unduly consuming local system resources.

[0064] However, in more complex distributed storage mixed-load scenarios, traditional technologies have significant shortcomings, specifically in the following aspects:

[0065] First, the granularity of resource control is relatively coarse, making it impossible to identify the specific type and importance of a business. It can only apply the same restriction rules to all business traffic or manage it only at the storage volume level. This makes it easy for non-core businesses such as background backups to occupy the bandwidth and resources required by core transaction businesses, affecting the normal operation of core businesses.

[0066] Second, the end-to-end operating status of the system cannot be uniformly perceived. Traditional service quality mechanisms can only monitor the I / O request queue and latency of local nodes, and cannot obtain the actual load status of remote storage nodes. When there is concentrated access or network congestion on remote nodes, local nodes continue to send data requests, which will further aggravate the congestion of the remote system.

[0067] Third, static rate limiting strategies lack flexibility. When business traffic reaches the preset limit threshold, traditional solutions typically use forced interruption or packet dropping to limit traffic, which can easily cause significant fluctuations in business processing volume, thereby affecting the continuity and stability of business operations.

[0068] Based on this, embodiments of this application provide a traffic control method, apparatus, device, and computer-readable storage medium, which can set a priority label in the executable file corresponding to the target process and allocate the target process to the resource control group corresponding to the priority label. Based on the resource limit parameters configured in the resource control group, the target process can be finely controlled for resources, thereby solving the problem that the existing traffic control method is too coarse-grained, resulting in low-value services occupying core business resources and unreasonable resource allocation.

[0069] Firstly, such as Figure 1 As shown, this application provides a traffic control method applied to a distributed storage system. The system includes a client and a remote storage node connected by communication. The method includes:

[0070] S101. The client determines the target process corresponding to the I / O request to access the remote storage node;

[0071] In Example 1, there are multiple target processes, and the priority label corresponding to each target process is any one of avoidance level, normal level or core level.

[0072] "Multiple target processes" refers to the simultaneous existence and running of multiple independent program execution instances in a distributed storage system. For example, multiple applications, such as database applications, backup applications, and log services, may be running simultaneously on a client. Each application's I / O request to the remote storage node can be considered a target process.

[0073] In one possible implementation, this application embeds a path-aware hook function in the kernel VFS layer. When the target process needs to initiate an I / O request to a remote storage node, it will first execute the open system call on the distributed storage virtual block device or the file under its mount directory. The triggering of this system call will directly activate the aforementioned embedded path-aware hook function. After the hook function is triggered, it will feed back the identification information of the target process corresponding to the system call to the kernel, thereby enabling the kernel to determine the target process.

[0074] S102. Determine the executable file of the target process and obtain the priority label that the user has pre-configured for the executable file;

[0075] In one possible implementation, the user pre-configures the extended attributes of the executable file (such as / usr / bin / oracle or / usr / bin / backup_agent) for each process in user space using the setfattr tool and the setfmac command or a similar command, writing priority labels ("Level 0" (avoidance level), "Level 1" (normal level), or "Level 2" (kernel level). Among these, kernel level has the highest priority, followed by normal level, and avoidance level has the lowest priority.

[0076] After identifying the target process, the kernel automatically backtracks to the current target process context and reads the extended attributes of the corresponding binary file. Regardless of whether a valid priority label is read, the kernel sends an asynchronous notification to the user-space daemon via the Netlink mechanism, containing the PID and priority label information. If no valid label is read, the kernel carries the default Level 1 label information in the asynchronous notification to ensure that all target processes can complete the mapping of resource control groups.

[0077] In one possible implementation, if no label is read, the priority label is set to Level 1 by default.

[0078] The traffic control method proposed in this application achieves identification and control at the operating system kernel and file attribute level. It does not rely on the business program itself to provide interfaces or carry priority information. Administrators only need to set extended attribute tags on the binary file of the business program. Without modifying the source code, recompiling or restarting the program, the system can identify the business priority and perform traffic control. Therefore, it can seamlessly include old businesses in the scope of traffic control.

[0079] S103. Assign the target process to the resource management group corresponding to the priority label. The resource management group is pre-configured with resource limit parameters corresponding to the priority label. The resource limit parameters are used to characterize the upper limit of the traffic of the corresponding resource management group to initiate I / O requests to the remote storage node.

[0080] A resource management group is a logical grouping used in an operating system or virtualization environment to uniformly manage and limit the resources of a group of processes. Users can pre-configure different resource limit parameters for each resource management group, including at least one of I / O bandwidth limits or IOPS limits. For example, a backup process assigned to a "low-priority resource management group" may have its I / O bandwidth limited to a lower level to prevent it from consuming too many system resources.

[0081] In one possible implementation, each resource control group in this application corresponds to an independent IO Cgroup under the Linux system, and the I / O resource limit parameters of the resource control group are implemented through the configuration file of the IO Cgroup.

[0082] In Example 1, the resource management group to which each target process is assigned corresponds to the priority label of that target process, and can be any one of the avoidance group, normal group or core group.

[0083] Specifically, when the priority label is core level (Level 2): ​​the corresponding resource management group is the core group, also known as a high-priority Cgroup, which has absolute bandwidth preemption rights and I / O submission queue priority, and is suitable for latency-sensitive businesses such as databases and online transactions.

[0084] When the priority label is Level 1, the corresponding resource management group is the normal group, also known as the default Cgroup. It is controlled by the fair scheduling algorithm and is suitable for throughput-oriented businesses such as log collection and batch computing.

[0085] When the priority label is Level 0, the corresponding resource control group is the Scavenger Class, which only allows the use of fragmented idle resources in the system and is suitable for back-end services such as cold backup and archiving.

[0086] In S103, after receiving the asynchronous notification through the user-space daemon process, this application immediately writes the PID in the asynchronous notification into the corresponding IO Cgroup (such as / sys / fs / cgroup / blkio / level2 / tasks), thus completing the automatic mapping from extended attributes to process resources.

[0087] Furthermore, this application uses hook functions in the kernel to perceive the I / O behavior of processes in real time, automatically traces the priority label of the executable file corresponding to the process, and completes process grouping through user-space daemons. This eliminates the need for manual PID lookup and manual configuration of binding relationships, saving tedious operations and reducing the probability of human configuration errors. At the same time, the kernel can perceive the start and stop of business processes in real time. When a process initiates I / O, it automatically completes grouping through Cgroup technology, which automatically expires after the process exits. This allows it to adapt to scenarios of dynamic start and stop of business without manual intervention, and continuously and stably achieves process priority identification and resource management.

[0088] S104. Perform traffic control on the target processes within the resource control group based on the corresponding resource limit parameters.

[0089] Specifically, once a target process is assigned to a specific resource management group, the resource limits configured for that group take effect. For example, if a resource management group is configured with an I / O bandwidth limit, then when the total I / O request rate of all target processes in the group exceeds that limit, the system will queue or delay the I / O operations of multiple target processes in the group to ensure that their I / O usage does not exceed the preset limit.

[0090] This application implements priority-based hierarchical control of business processes. Specifically, it dynamically reduces the total resource usage of low-priority resource control groups in a step-by-step, non-rigid manner, rather than using traditional rigid rate limiting methods such as circuit breaking and hard limiting. While ensuring that the latency of core business is not disturbed and resource demand is met first, it dynamically identifies idle resources in the system and allows low-priority business to flexibly reuse the remaining resources. This solves the problems of core business being easily disturbed and unreasonable allocation of system resources in traditional resource control methods.

[0091] like Figure 2 The diagram shown is another flowchart of the method described in this application. The specific steps are as follows:

[0092] S201. The user pre-sets a corresponding priority label for each executable file;

[0093] S202. The target process initiates an open system call on a distributed storage virtual block device or a file in its mount directory.

[0094] S203. If the kernel VFS path-aware hook is triggered, proceed to step S204; if the kernel VFS path-aware hook is not triggered, proceed to step S209.

[0095] S204. Backtrack the target process context and read the binary file xattr;

[0096] S205. Send asynchronous notifications via Netlink;

[0097] S206, Receive asynchronous notification;

[0098] S207. Write the PID of the target process into the corresponding resource control group. The resource control group is pre-configured with corresponding resource limit parameters.

[0099] S208. Perform traffic control on all target processes in the resource control group based on the corresponding resource limit parameters;

[0100] S209, End.

[0101] In one possible implementation, after S104, it also includes:

[0102] S105. Record the target process of the core group as the core process, and obtain the first total latency information of multiple core processes on the client and the second total latency information on the remote storage node within the current collection period.

[0103] Core processes typically handle critical business operations with extremely high latency and throughput requirements, such as database transactions and real-time data analysis.

[0104] In one possible implementation, this application achieves real-time collection of end-to-end latency information through a perception layer. Firstly, the core process initiates an I / O request to the remote storage node through the virtual block device on the client side. During the processing of the I / O request, the virtual block device generates and reports the first total latency information of all core processes. The first total latency information includes the latency r_await / w_await of each core process submitted by the local distributed storage virtual block device. Secondly, the perception layer collects the second total latency information through a bypass channel. The second total latency information includes the response latency of the remote storage node.

[0105] The specific process of S105 is as follows: The client injects probes or hook functions into the I / O request path of the operating system to record the local time of each core process's I / O request from being sent to receiving the response from the remote storage node, and accumulates or averages the data to obtain the first total latency information.

[0106] The remote storage node can record the time taken from receiving all I / O requests to processing them and returning a response, and periodically transmit this latency information back to the client via a specific communication protocol. The client then aggregates this information to obtain a second total latency. The collection period can be a fixed time interval (e.g., every second, every 5 seconds) or dynamically adjusted, such as adaptively adjusting based on system load or congestion levels.

[0107] S106. The first total delay information and the second total delay information are weighted and fused to obtain the end-to-end delay information;

[0108] End-to-end latency information can be determined using the following formula:

[0109] End-to-end latency = W1 × first total latency information + W2 × second total latency information

[0110] In the formula, W1 and W2 are weighting coefficients, and W1 + W2 = 1. The weighting coefficients can be preset or dynamically adjusted by the user based on the actual system architecture, network conditions, and the respective impact of the client and remote storage nodes on the total latency.

[0111] S107. Based on the end-link latency information, determine whether congestion has occurred. If so, in the current collection cycle, lower the resource limit parameters of at least one other resource control group other than the core group until the congestion is relieved or the resource limit parameters of all resource control groups that have been lowered reach the corresponding lower limit threshold, and enter the next collection cycle.

[0112] If not, proceed to the next data collection cycle.

[0113] This application constructs a full-link, three-dimensional monitoring system that coordinates local clients and remote storage nodes. It can perceive and synchronize the local I / O operation status and the actual load and congestion of remote storage nodes in real time. It can identify congestion caused by hotspots of remote storage nodes that cannot be detected by traditional monitoring methods. This solves the technical problem that traditional resource management mechanisms only focus on local status and cannot perceive the remote load status, thus continuously sending requests and aggravating system congestion.

[0114] In one possible implementation, such as Figure 3 As shown, this application also determines the current congestion type based on the first total latency information, the second total latency information, and the end-to-end latency information. For example, when only the first total latency information is greater than the fourth threshold, it is determined to be local queue congestion; when only the second total latency information is greater than the fifth threshold, it is determined to be that the remote storage node has a processing capacity bottleneck, which leads to remote congestion; when the first total latency information is greater than the fourth threshold and the second total latency information is greater than the fifth threshold, it is determined to be end-to-end congestion.

[0115] It is worth mentioning that the first to fifth thresholds in this application were all set by those skilled in the art.

[0116] This application enables the precise identification of congestion types, thereby triggering corresponding differentiated processing strategies for different types of congestion and avoiding blindly implementing traffic control on remote storage nodes or clients.

[0117] In one possible implementation, congestion is determined based on end-to-end latency information, specifically as follows:

[0118] Determine whether the end-to-end latency information is greater than a preset first threshold. If yes, it is determined that congestion has occurred; otherwise, it is determined that no congestion has occurred or the congestion has been resolved.

[0119] The first threshold is a pre-set upper limit for a performance metric, representing the maximum latency of the system under normal or acceptable load.

[0120] This technical feature refers to the proactive restriction of I / O resource usage by non-core services (i.e., avoidance groups and normal groups) after the system determines that congestion has occurred, in order to reduce system load and prioritize the normal operation of core services. Pressure reduction measures may include reducing the I / O bandwidth of these processes, reducing the concurrency of I / O operations, and increasing the queuing time for I / O requests. Its purpose is to alleviate overall system congestion by sacrificing the performance of non-core services, thereby protecting the service level agreement (SLA) of core services. For example, the I / O priority of non-core processes can be reduced by adjusting the I / O scheduler parameters in the operating system kernel, putting them at a disadvantage in resource contention. Alternatively, the resource limit configuration of non-core resource management groups can be dynamically modified in the user-space or kernel-space flow control module, such as reducing their maximum allowed IOPS or bandwidth, or increasing the latency of their I / O requests. The specific reduction amount can be set by those skilled in the art.

[0121] In one possible implementation, during the current data collection period, the resource limit parameters of at least one other resource control group besides the core group are lowered until congestion is relieved or the resource limit parameters of all lowered resource control groups reach their corresponding lower thresholds. Specifically:

[0122] The time delay deviation is obtained by subtracting the end-link time delay information from the first threshold.

[0123] Determine the reduction step size, which is used to characterize the extent of the reduction in resource constraint parameters for any other resource control group;

[0124] Based on the step size, the resource limit parameters of at least one other resource control group are reduced at least once, and the end-to-end latency information in the current collection period is updated after each reduction, until the end-to-end latency information after the last update is less than or equal to the first threshold, or the resource limit parameters of all resource control groups that have been reduced reach the corresponding lower limit threshold.

[0125] Because the acquisition period in this solution is set to be short and the latency calculation and data processing flow has high execution efficiency, the latency information of the entire link can be updated and refreshed multiple times in real time within a single current acquisition period, ensuring that the latency data can reflect the actual operating status of the I / O path in real time and accurately.

[0126] In one possible implementation, the step size can be a preset fixed value.

[0127] In another possible implementation, the step size can also be dynamically adjusted, specifically:

[0128] Adjustment step size = Latest latency deviation (Error) / Latest end-to-end latency information (Current)

[0129] According to the above formula, after each downward adjustment of the resource limit parameters of the corresponding resource control group (such as the avoidance group and the normal group), the end-to-end latency information in the current collection period will be updated immediately. Based on the updated end-to-end latency information and the preset first threshold, the latest latency deviation is recalculated. Then, based on the ratio of the updated end-to-end latency information to the recalculated latency deviation, the downward adjustment step size used in the next round of downward adjustment is dynamically determined to ensure that the downward adjustment magnitude in each round can accurately match the current I / O path load status.

[0130] In one possible implementation, the resource limit parameters of at least one other resource control group are lowered at least once based on the reduction step size, and the end-to-end latency information for the current collection period is updated after each reduction, until the end-to-end latency information after the last update is less than or equal to a first threshold, or the resource limit parameters of all the resource control groups that have been lowered reach their corresponding lower thresholds, and then the next collection period begins. Specifically:

[0131] Based on the reduction step size, the resource limit parameters of the avoidance group are reduced in the current round. After the reduction, the end-to-end latency information in the current collection period is updated, and it is determined whether the updated end-to-end latency information is less than or equal to the first threshold.

[0132] If not, determine whether the resource limit parameter of the avoidance group after the reduction in the current round is less than the second threshold;

[0133] If so, the resource limit parameters of the ordinary group will be reduced at least once based on the reduction step size. After each reduction of the resource limit parameters of the ordinary group, the end-to-end latency information within the current collection period will be updated. If, after the current number of reductions, the reduced resource limit parameters of the ordinary group are less than the preset third threshold and the updated end-to-end latency information is greater than the first threshold, the current resource limit parameters of each resource control group will be maintained, and the next collection period will begin. If, after the current number of reductions, the updated end-to-end latency information is less than or equal to the first threshold, the next collection period will begin. If the updated end-to-end latency information is greater than the first threshold and the reduced resource limit parameters of the ordinary group are greater than the preset third threshold, the resource limit parameters of the ordinary group will be reduced again according to the reduction step size.

[0134] If not, the resource limit parameters of the avoidance group are lowered again based on the reduction step size, and the end-to-end latency information in the current collection cycle is updated again. It is determined whether the end-to-end latency information after the second update is less than or equal to the first threshold. If not, it is determined whether the resource limit parameters of the avoidance group after the second reduction are less than the second threshold. This process continues until the end-to-end latency information after the last update is less than or equal to the first threshold, or the resource limit parameters of all resource control groups that have been lowered reach the corresponding lower limit threshold, and then the next collection cycle begins.

[0135] like Figure 4 The diagram illustrates the process of determining end-to-end latency information and adjusting resource limitation parameters based on that information, as described in this application. The process includes the following steps:

[0136] S401. Record the target process of the core group as the core process, and obtain the first total latency information of multiple core processes on the client and the second total latency information on the remote storage node within the current collection period.

[0137] S402. The first total delay information and the second total delay information are weighted and fused to obtain the end-to-end delay information;

[0138] S403. Determine a first threshold value, which is used to characterize the preset upper limit value of the end-link latency information;

[0139] S404. Determine whether the end-to-end latency information is greater than the first threshold.

[0140] If not, proceed to step S4055;

[0141] S4055, Enter the next acquisition cycle.

[0142] If so, proceed to step S4051;

[0143] S4051. The time delay information of the entire link is calculated by subtracting the first threshold to obtain the time delay deviation.

[0144] S4052. Determine the adjustment step size based on the time delay deviation, and adjust the resource limit parameters of the avoidance group for the current round based on the adjustment step size;

[0145] S4053. Update the end-to-end latency information for the current acquisition period and determine whether the updated end-to-end latency information is less than or equal to the first threshold.

[0146] If so, proceed to step S40541;

[0147] S40541, Enter the next acquisition cycle.

[0148] If not, proceed to step S40542;

[0149] S40542. Determine whether the resource limit parameter of the avoidance group is less than the second threshold after the adjustment in the current round.

[0150] If not, proceed to step S405432;

[0151] S405432. Based on the reduction step size, the resource limit parameters of the avoidance group are reduced again and the process returns to step S4053.

[0152] If so, proceed to step S405431;

[0153] S405431. Based on the reduction step size, the resource limit parameters of the ordinary group are reduced for the current number of times, and the end-to-end latency information is updated.

[0154] S40544. Determine whether the updated end-link latency information is less than or equal to the first threshold.

[0155] If so, proceed to step S40545;

[0156] S40545, Entering the next data acquisition cycle;

[0157] If not, proceed to step S40546;

[0158] S40546. Determine whether the resource limit parameter of the ordinary group is less than the third threshold after the current number of reductions.

[0159] If not, return to step S405431;

[0160] If so, proceed to step S40547;

[0161] S40547. Maintain the resource limit parameters of each resource control group corresponding to the current number of times, and enter the next collection cycle.

[0162] This application achieves flexible control over I / O traffic by dynamically calculating and adjusting the resource constraint parameters of the avoidance group based on PID control theory and the time delay fluctuation state of the core business.

[0163] In one possible implementation, after entering the next acquisition cycle, the following is also included:

[0164] S108. Determine the recovery step size based on the time delay deviation and the first threshold, and restore each resource limit parameter that was downgraded in the previous period based on the recovery step size until each downgraded resource limit parameter is restored to its initial value before downgrading. The recovery step size is used to characterize the value or proportion that the downgraded resource limit parameter can be restored in each recovery process.

[0165] In other embodiments, the resource limitation parameters of each group can be restored after a preset number of collection cycles are delayed.

[0166] In one possible implementation:

[0167] Recovery step size = absolute value of the latest delay deviation / first threshold

[0168] As shown in the above formula, similar to the downsizing step size, the recovery step size also adopts a dynamic adjustment mechanism to achieve smooth recovery of resource constraint parameters. Specifically, after the end-link latency information is updated and becomes less than or equal to the first threshold, i.e., after the congestion state is resolved, the system enters the recovery process for the downsized resource constraint parameters. During each recovery operation, the recovery step size is dynamically calculated and determined based on the current latency deviation and the preset first threshold, so that the recovery process can adapt to real-time latency changes and avoid congestion recurring due to overly rapid recovery.

[0169] In one possible implementation, the recovery step size can be a fixed resource increment value, such as increasing by 100 IOPS or 10MB / s bandwidth with each recovery; or it can be a percentage value, such as 5% or 10% of the current limit value with each recovery.

[0170] Specifically, when performing the recovery operation for the downgraded resource limit parameters after congestion is relieved, the system can choose to synchronously restore the resource limit parameters of all downgraded resource control groups based on the actual system load. Alternatively, it can restore the resource limit parameters of ordinary groups first, in descending order of priority. After the ordinary groups are restored to their initial values ​​before downgrading, the resource limit parameters of the avoidance groups are then restored. The execution principle of this recovery process is similar to the process of downgrading resource limit parameters. Both adjust the resource limit parameters of the corresponding resource control groups round by round based on a preset or dynamically calculated recovery step size until all downgraded resource limit parameters are restored to their initial values ​​before downgrading.

[0171] Secondly, such as Figure 5 As shown, this application also provides a flow control device 10, comprising:

[0172] Module 11 is used to determine the target process corresponding to the I / O request to access the remote storage node;

[0173] Tag module 12 is used to determine the executable file of the target process and obtain the priority tags that the user has pre-configured for the executable file;

[0174] Grouping module 13 is used to assign the target process to the resource management group corresponding to the priority label. The resource management group is pre-configured with resource limit parameters corresponding to the priority label. The resource limit parameters are used to characterize the upper limit of the traffic of the corresponding resource management group to initiate I / O requests to the remote storage node, including at least one of I / O bandwidth limit or IOPS limit.

[0175] The flow control module 14 is used to control the flow of target processes within the resource control group based on the corresponding resource limit parameters.

[0176] It should be noted that although the operations of the method of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. On the contrary, the steps depicted in the flowchart may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0177] In another aspect, this application also provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0178] The memory stores the instructions that the computer executes;

[0179] The processor executes computer-executable instructions stored in memory to implement any of the flow control methods in the first aspect.

[0180] On the other hand, this application also provides a computer-readable storage medium, which may be included in an electronic device or exist independently without being assembled into the electronic device. The aforementioned computer-readable storage medium stores one or more programs that, when used by one or more processors, execute the methods described in this application. For example, it may execute... Figure 1 The steps of the method shown.

[0181] This application provides a computer program product including instructions that, when executed, cause the method described in this application to be performed. For example, it can execute... Figure 1 The steps of the method shown.

[0182] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0183] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A traffic control method applied to a distributed storage system, the system comprising a client and a remote storage node connected by communication, characterized in that, The method includes: The client determines the target process corresponding to the I / O request to access the remote storage node; The executable file of the target process is determined, and the priority label pre-configured by the user for the executable file is obtained; The target process is assigned to a resource management group corresponding to the priority label. The resource management group is pre-configured with resource limit parameters corresponding to the priority label. The resource limit parameters are used to characterize the upper limit of the traffic for the corresponding resource management group to initiate I / O requests to the remote storage node, including at least one of I / O bandwidth limit or IOPS limit. Traffic control is performed on the target processes within the resource control group based on the corresponding resource restriction parameters.

2. The method according to claim 1, characterized in that, There are multiple target processes, and the priority label corresponding to each target process is any one of avoidance level, normal level or core level; The resource management group to which each target process is assigned corresponds to the priority label of that target process, and can be any one of the avoidance group, normal group or core group.

3. The method according to claim 2, characterized in that, After performing traffic control on the target processes within the resource control group based on the corresponding resource limitation parameters, the method further includes: The target process of the core group is denoted as the core process. The first total latency information of multiple core processes on the client and the second total latency information on the remote storage node are obtained within the current collection period. The first total latency information and the second total latency information are weighted and fused to obtain the end-to-end latency information; Based on the end-to-end latency information, it is determined whether congestion has occurred. If so, the resource limit parameters of at least one other resource management group other than the core group are lowered within the current collection cycle until the congestion is relieved or the resource limit parameters of all resource management groups that have been lowered reach the corresponding lower limit threshold, and then the next collection cycle begins. If not, proceed to the next data collection cycle.

4. The method according to claim 3, characterized in that, Determining whether congestion has occurred based on the aforementioned end-link latency information involves the following steps: Determine whether the end-to-end latency information is greater than a preset first threshold. If yes, it is determined that congestion has occurred; otherwise, it is determined that no congestion has occurred or the congestion has been resolved.

5. The method according to claim 4, characterized in that, Within the current data collection period, the resource limit parameters of at least one other resource management group besides the core group are lowered until congestion is relieved or the resource limit parameters of all lowered resource management groups reach their corresponding lower thresholds, at which point the next data collection period begins. Specifically: The delay deviation is obtained by subtracting the end-link delay information from the first threshold. Determine the reduction step size, which is used to characterize the extent of the reduction of the resource limit parameters of any of the other resource control groups; Based on the reduction step size, the resource limit parameters of at least one other resource control group are reduced at least once, and the end-to-end latency information in the current collection cycle is updated after each reduction, until the end-to-end latency information after the last update is less than or equal to the first threshold, or the resource limit parameters of all resource control groups that have been reduced reach the corresponding lower limit threshold, and then the next collection cycle begins.

6. The method according to claim 5, characterized in that, Based on the aforementioned reduction step size, the resource limit parameters of at least one other resource control group are reduced at least once, and the end-to-end latency information within the current collection cycle is updated after each reduction, until the end-to-end latency information after the last update is less than or equal to the first threshold, or the resource limit parameters of all reduced resource control groups reach their corresponding lower limit thresholds, before entering the next collection cycle. Specifically: Based on the reduction step size, the resource limit parameters of the avoidance group are reduced in the current round. After the reduction, the end-to-end latency information in the current collection period is updated, and it is determined whether the updated end-to-end latency information is less than or equal to the first threshold. If not, determine whether the resource limit parameter of the avoidance group after the reduction in the current round is less than the second threshold; If so, the resource limitation parameters of the ordinary group are reduced at least once based on the reduction step size, and after each reduction of the resource limitation parameters of the ordinary group, the end-to-end latency information in the current collection period is updated. If, after the current number of reductions, the reduced resource limitation parameters of the ordinary group are less than a preset third threshold and the updated end-to-end latency information is greater than the first threshold, then the current resource limitation parameters of each resource control group are maintained, and the next collection period begins. If not, the resource limit parameters of the avoidance group are lowered again based on the lowering step size, and the end-to-end latency information in the current collection cycle is updated again. It is determined whether the end-to-end latency information after the second update is less than or equal to the first threshold. If not, it is determined whether the resource limit parameters of the avoidance group after the second lowering are less than the second threshold. This process continues until the end-to-end latency information after the last update is less than or equal to the first threshold, or the resource limit parameters of all resource control groups that have been lowered reach the corresponding lower limit threshold, and then the next collection cycle begins.

7. The method according to claim 5 or 6, characterized in that, After entering the next acquisition cycle, it also includes: Determine the recovery step size, and restore each of the resource limit parameters that were downgraded in the previous period based on the recovery step size, until each of the downgraded resource limit parameters is restored to its initial value before downgrading.

8. A flow control device, the device comprising: The determination module is used to determine the target process corresponding to the I / O request to access the remote storage node; The tag module is used to determine the executable file of the target process and obtain the priority tags that the user has pre-configured for the executable file; A grouping module is used to assign the target process to a resource management group corresponding to the priority label. The resource management group is pre-configured with resource limit parameters corresponding to the priority label. The resource limit parameters are used to characterize the upper limit of the traffic for the corresponding resource management group to initiate I / O requests to the remote storage node, including at least one of I / O bandwidth limit or IOPS limit. The flow control module is used to control the flow of target processes within the resource control group based on the corresponding resource limit parameters.

9. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the traffic control method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the traffic control method as described in any one of claims 1 to 7.

Images