Method and apparatus for bandwidth allocation of storage devices in dual active storage systems

Abstract

The application provides a bandwidth allocation method and device for storage devices in a dual-active storage system, which can be applied to the technical field of storage. The method comprises the following steps: analyzing load data of the storage devices in a predetermined period, and determining a load change rate in the predetermined period; based on the load change rate, determining one of a first bandwidth allocation mode and a second bandwidth allocation mode as a target bandwidth allocation mode, the first bandwidth allocation mode being bandwidth allocation based on a bandwidth adjustment model obtained through machine learning, and the second bandwidth allocation mode being bandwidth allocation based on dynamic weights; and based on the target bandwidth allocation mode, determining available bandwidths of a service path and a control path of the storage device, the service path being used for transmitting service traffic, the control path being used for transmitting control traffic, and the service path and the control path sharing a transmission channel of the storage device.

Description

Technical Field

[0001] This application relates to the field of storage technology, and more specifically to a bandwidth allocation method and device for storage devices in a dual-active storage system. Background Technology

[0002] Active-active storage systems typically utilize multiple storage devices deployed independently in different locations to share the workload.

[0003] However, in traditional active-active storage systems, a static bandwidth allocation method is usually used, which allocates a fixed amount of available bandwidth to the service path and control path of the storage device. This results in a mismatch between the available bandwidth of the service path and control path and the actual required bandwidth, which in turn leads to an increase in the overall latency of the active-active storage system. Summary of the Invention

[0004] In view of the above problems, this application provides a bandwidth allocation method and electronic device for storage devices in a dual-active storage system.

[0005] According to a first aspect of this application, a bandwidth allocation method for a storage device in a dual-active storage system is provided, comprising: analyzing load data of the storage device within a predetermined time period to determine the load change rate within the predetermined time period; determining, based on the load change rate, one of a first bandwidth allocation method and a second bandwidth allocation method as a target bandwidth allocation method, wherein the first bandwidth allocation method is bandwidth allocation based on a bandwidth adjustment model obtained through machine learning, and the second bandwidth allocation method is bandwidth allocation based on dynamic weights; and determining, based on the target bandwidth allocation method, the available bandwidth of the service path and the control path of the storage device, wherein the service path is used to transmit service traffic, the control path is used to transmit control traffic, and the service path and the control path share the transmission channel of the storage device.

[0006] A second aspect of this application provides a bandwidth allocation device for a storage device in a dual-active storage system, comprising: a load analysis module for analyzing load data of the storage device within a predetermined time period and determining the load change rate within the predetermined time period; a mode determination module for determining, based on the load change rate, one of a first bandwidth allocation mode and a second bandwidth allocation mode as a target bandwidth allocation mode, wherein the first bandwidth allocation mode is based on a bandwidth adjustment model obtained through machine learning, and the second bandwidth allocation mode is based on dynamic weights; and a bandwidth allocation module for determining the available bandwidth of the service path and the control path of the storage device based on the target bandwidth allocation mode, wherein the service path is used to transmit service traffic, the control path is used to transmit control traffic, and the service path and the control path share the transmission channel of the storage device.

[0007] A third aspect of this application provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0008] A fourth aspect of this application also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0009] The fifth aspect of this application also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0010] According to the embodiments of this application, considering the impact of load change rate on the accuracy of bandwidth allocation method, one of the first bandwidth allocation method and the second bandwidth allocation method is determined as the target bandwidth allocation method, so that the target bandwidth allocation method can adapt to load fluctuations and dynamically allocate the available bandwidth of the service path and the control path, thereby enabling the available bandwidth of the service path and the control path to meet the actual bandwidth requirements, realizing the rational utilization of bandwidth resources and reducing the processing latency of the dual-active storage system. Attached Figure Description

[0011] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments of this application with reference to the accompanying drawings.

[0012] Figure 1 The illustration shows a bandwidth allocation method for a storage device in a dual-active storage system according to an embodiment of this application, and an application scenario diagram of the device.

[0013] Figure 2 A flowchart illustrating a bandwidth allocation method for a storage device in a dual-active storage system according to an embodiment of this application is shown.

[0014] Figure 3 A system architecture diagram for implementing a bandwidth allocation method according to an embodiment of this application is shown.

[0015] Figure 4 A flowchart illustrating bandwidth allocation according to an embodiment of this application is shown.

[0016] Figure 5 A structural block diagram of a bandwidth allocation device for a storage device in a dual-active storage system according to an embodiment of this application is shown.

[0017] Figure 6A block diagram of an electronic device suitable for implementing a bandwidth allocation method for a storage device in a dual-active storage system, according to an embodiment of this application, is shown. Detailed Implementation

[0018] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0019] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0020] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0021] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0022] A dual-active storage system is a high-availability storage architecture design that typically includes multiple storage devices deployed independently in different locations. This allows for workload distribution across multiple storage devices and ensures that in the event of a primary storage device failure, services can be seamlessly migrated to the backup storage device, guaranteeing service continuity.

[0023] In traditional active-active storage systems, static bandwidth allocation is typically used, allocating a larger amount of available bandwidth to the service path. Alternatively, under high service load, the priority service path may occupy the entire bandwidth of the transmission channel. This can cause storage devices to be unable to transmit control traffic in a timely manner through the control path, increasing the transmission latency of control traffic. Consequently, the overall latency of the active-active storage system may increase, and even cause multiple storage devices to have difficulty communicating, thus affecting storage consistency.

[0024] Embodiments of this application provide a bandwidth allocation method for storage devices in a dual-active storage system, comprising: analyzing load data of the storage device within a predetermined time period to determine the load change rate within the predetermined time period; determining one of a first bandwidth allocation method and a second bandwidth allocation method as a target bandwidth allocation method based on the load change rate, wherein the first bandwidth allocation method is bandwidth allocation based on a bandwidth adjustment model obtained through machine learning, and the second bandwidth allocation method is bandwidth allocation based on dynamic weights; and determining the available bandwidth of the service path and control path of the storage device based on the target bandwidth allocation method, wherein the service path is used to transmit service traffic, the control path is used to transmit control traffic, and the service path and control path share the transmission channel of the storage device.

[0025] The embodiments of this application take into account the impact of load change rate on the accuracy of bandwidth allocation method, and determine one of the first bandwidth allocation method and the second bandwidth allocation method as the target bandwidth allocation method. This allows the target bandwidth allocation method to adapt to load fluctuations and dynamically allocate the available bandwidth of the service path and the control path, thereby ensuring that the available bandwidth of the service path and the control path can meet the actual bandwidth requirements, achieve reasonable utilization of bandwidth resources, and reduce the processing latency of the dual-active storage system.

[0026] Figure 1 The illustration shows a bandwidth allocation method for a storage device in a dual-active storage system according to an embodiment of this application, and an application scenario diagram of the device.

[0027] like Figure 1 As shown, the application scenario according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a server 105, and a dual-active storage system 106. Network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0028] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0029] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0030] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). Server 105 can forward business requests to the dual-active storage system 106, so that the dual-active storage system 106 can process the stored data according to the business requests.

[0031] The dual-active storage system 106 may include multiple storage devices, such as a first storage device 106_1 and a second storage device 106_2, which can communicate with each other.

[0032] Storage devices can include data paths and control paths. Data paths are typically used to transmit business data; for example, when a server needs to read data, it can transmit read requests or read results through the data path. Data paths can also be used for data synchronization between multiple storage devices. Control paths are typically used to transmit management and control information. They can be paths for control operations such as status monitoring, fault detection, failover decisions, and arbitration mechanisms between servers and storage devices, and between multiple storage devices.

[0033] It should be noted that the bandwidth allocation method for storage devices in a dual-active storage system provided in this application embodiment can generally be executed by the first storage device 106_1 or the second storage device 106_2 in the dual-active storage system 106. Accordingly, the bandwidth allocation device for storage devices in a dual-active storage system provided in this application embodiment can generally be set in the first storage device 106_1 or the second storage device 106_2 in the dual-active storage system 106.

[0034] It should be understood that Figure 1 The number of the first terminal device, second terminal device, third terminal device, server, dual-active storage system, first storage device, and second storage device in the diagram is merely illustrative. Depending on the implementation requirements, any number of the first terminal device, second terminal device, third terminal device, server, dual-active storage system, first storage device, and second storage device can be included.

[0035] The following will be based on Figure 1 The described scene, through Figures 2-4The bandwidth allocation method for storage devices in a dual-active storage system according to the embodiments of the application will be described in detail.

[0036] Figure 2 A flowchart illustrating a bandwidth allocation method for a storage device in a dual-active storage system according to an embodiment of this application is shown.

[0037] like Figure 2 As shown, the bandwidth allocation method for storage devices in a dual-active storage system in this embodiment includes operations S210 to S230.

[0038] In operation S210, the load data of the storage device during a predetermined period is analyzed to determine the load change rate during the predetermined period.

[0039] In operation S220, based on the load change rate, one of the first bandwidth allocation method and the second bandwidth allocation method is determined as the target bandwidth allocation method. The first bandwidth allocation method is based on the bandwidth adjustment model obtained through machine learning, and the second bandwidth allocation method is based on dynamic weights.

[0040] In operation S230, based on the target bandwidth allocation method, the available bandwidth of the service path and control path of the storage device is determined. The service path is used to transmit service traffic, and the control path is used to transmit control traffic. The service path and control path share the transmission channel of the storage device.

[0041] Active-active storage systems typically utilize multiple storage devices deployed independently in different locations to share the workload, placing high demands on both service performance and communication reliability. Service traffic latency can degrade service performance, while control traffic latency can reduce communication reliability. When the service and control paths share a transmission channel, the total bandwidth of the transmission channel needs to be rationally allocated between the service and control paths to ensure both service performance and communication reliability.

[0042] When allocating bandwidth to the service path and control path, the load on the storage device changes in real time, which in turn causes the service traffic and control traffic of the storage device to change in real time. Consequently, the bandwidth requirements of the service path and control path change in real time. Therefore, a dynamic first bandwidth allocation method or a second bandwidth allocation method can be adopted to adjust the available bandwidth of the service path and control path based on the real-time service traffic status information and control traffic status information.

[0043] In the embodiments of this application, considering that the load change rate of the storage device affects the accuracy of the bandwidth allocation method, the load change rate of the storage device in a predetermined period of time can be determined first, and then the bandwidth allocation method with higher accuracy under the current load change rate can be determined as the target bandwidth allocation method among the first bandwidth allocation method and the second bandwidth allocation method.

[0044] When determining the load change rate of a storage device, statistical analysis can be performed on the load data of the storage device within a predetermined time period. For example, the load change rate can be determined based on the maximum and minimum loads in the last 5 minutes, or the load data in the last 5 minutes can be fitted into a load data curve, and the load change rate can be determined based on the load data curve.

[0045] When determining the target bandwidth allocation method, the target bandwidth allocation method can be determined based on the comparison between the load change rate and the load change rate threshold, or the bandwidth allocation method that has a mapping relationship with the load change rate can be determined as the target bandwidth allocation method based on the pre-configured mapping relationship between the load change rate and the bandwidth allocation method.

[0046] For example, when the load change rate is large, the first bandwidth allocation method uses a bandwidth adjustment model obtained through machine learning, which can adaptively learn the constantly changing load pattern and is relatively more accurate than the second bandwidth allocation method. The first bandwidth allocation method can be set as the allocation method when the load change rate is large.

[0047] In some embodiments, when the target bandwidth allocation method is the first bandwidth allocation method, real-time service traffic status information and control traffic status information can be input into the bandwidth adjustment model obtained through machine learning to allocate bandwidth using the bandwidth adjustment model. When the target bandwidth allocation method is the second bandwidth allocation method, the dynamic weights of the service path and control path can be determined based on the real-time service traffic status information and control traffic status information to allocate bandwidth to the service path and control path using the dynamic weights.

[0048] According to the embodiments of this application, considering the impact of load change rate on the accuracy of bandwidth allocation method, one of the first bandwidth allocation method and the second bandwidth allocation method is determined as the target bandwidth allocation method, so that the target bandwidth allocation method can adapt to load fluctuations and dynamically allocate the available bandwidth of the service path and the control path, thereby enabling the available bandwidth of the service path and the control path to meet the actual bandwidth requirements, realizing the rational utilization of bandwidth resources and reducing the processing latency of the dual-active storage system.

[0049] Figure 3 A system architecture diagram for implementing a bandwidth allocation method according to an embodiment of this application is shown.

[0050] like Figure 3As shown, the system architecture for implementing the bandwidth allocation method includes a monitoring layer, a decision layer, and an execution layer. The monitoring layer is used to acquire load data within a preset time period, the decision layer is used to determine the target bandwidth allocation method, and the execution layer is used to allocate bandwidth according to the target bandwidth allocation method.

[0051] According to an embodiment of this application, analyzing the load data of a storage device within a predetermined time period to determine the load change rate within the predetermined time period includes: determining a target fitting method for fitting the load data based on the distribution characteristics of the load data within the predetermined time period; using the target fitting method to perform curve fitting on the load data to obtain a load change curve; and determining the load change rate based on the load change curve.

[0052] When determining the load change rate, since the load changes of storage devices are complex, the load change rate determined solely by the ratio of the maximum load to the minimum load cannot reflect the load changes. Therefore, the load data can be curve-fitted, and the load change rate can be determined based on the load change curve obtained from the curve fitting.

[0053] To improve the accuracy of curve fitting for load data, a target fitting method that matches the load data variation can be selected based on the distribution characteristics of the load data, and then curve fitting can be performed on the load data.

[0054] For example, when the distribution characteristics indicate that the load data is linearly distributed, the target fitting method can be determined as linear regression; when the distribution characteristics indicate that the load data is periodically changing, the target fitting method can be determined as Fourier series fitting.

[0055] In some embodiments, in order to ensure the accuracy of the load change curve, after obtaining the load change curve by curve fitting the load data, the load change curve can be verified. If the load change curve passes the verification, the load change rate can be determined using the load change curve. If the load change curve fails the verification, other preset fitting methods can be used to perform curve fitting on the load data until the load change curve passes the verification.

[0056] After obtaining the load change curve, the load change curve can be differentiated to obtain the load change rate function. Then, based on the load change rate function, the instantaneous load change rate at each moment within a preset time period can be obtained. Finally, the load change rate within the preset time period can be determined based on multiple instantaneous load change rates.

[0057] According to embodiments of this application, by fitting the load data using a target fitting method that matches the load data distribution characteristics, the fitted load change curve can accurately represent the actual load change, thereby improving the accuracy of the load change rate determined based on the load change curve, and further improving the accuracy of the target bandwidth allocation method.

[0058] According to an embodiment of this application, determining one of a first bandwidth allocation method and a second bandwidth allocation method as a target bandwidth allocation method based on the load change rate includes: determining the first bandwidth allocation method as the target bandwidth allocation method in response to determining that the load change rate is greater than a load change rate threshold; and determining the second bandwidth allocation method as the target bandwidth allocation method in response to determining that the load change rate is less than or equal to the load change rate threshold.

[0059] When determining the target bandwidth allocation method, the load change rate can be compared with the load change rate threshold. When the load change rate is greater than the load change rate threshold, it indicates that the current load is fluctuating greatly. Since the first bandwidth allocation method uses a bandwidth adjustment model obtained through machine learning, it can adaptively learn the constantly changing load pattern. Therefore, when the load change rate is greater than the load change rate threshold, the first bandwidth allocation method can be determined as the target bandwidth allocation method.

[0060] When the load change rate is less than or equal to the load change rate threshold, it indicates that the current load fluctuation is small. At this time, using the first bandwidth allocation method will lead to a waste of computing resources. Therefore, when the load change rate is less than or equal to the load change rate threshold, the second bandwidth allocation method can be determined as the target bandwidth allocation method. This not only allows for bandwidth allocation for service paths and control paths based on real-time service traffic status information and control traffic status information, but also reduces the consumption of computing resources.

[0061] According to the embodiments of this disclosure, by determining the first bandwidth allocation method as the target bandwidth allocation method when the load change rate is greater than the change rate threshold, and determining the second bandwidth allocation method as the target bandwidth allocation method when the load change rate is less than or equal to the change rate threshold, not only can the accuracy of bandwidth allocation using the target bandwidth allocation method be guaranteed, but the consumption of computing resources caused by bandwidth allocation can also be reduced.

[0062] According to an embodiment of this application, the load change rate threshold is determined as follows: the storage device is tested using multiple test strategies determined based on different load change rates to simulate the operation of the storage device under different load change rates; the performance indicators of the storage device under each test strategy are obtained, and the load change rate threshold is determined based on multiple performance indicators.

[0063] For example, based on performance metrics, the load change rate that causes a significant increase in packet loss rate can be determined, and this load change rate can be set as a threshold.

[0064] According to embodiments of this application, by utilizing multiple test strategies determined based on different load change rates, a change rate threshold for the storage device is determined, so that the change rate threshold matches the actual performance of the storage device, thereby improving the accuracy of the determined target bandwidth allocation method.

[0065] According to an embodiment of this application, determining the available bandwidth of the service path and control path of the storage device based on the target bandwidth allocation method includes: in response to determining the target bandwidth allocation method as the first bandwidth allocation method, inputting the current service traffic status information, the current control traffic status information, and the current transmission channel status information into the bandwidth adjustment model to obtain the bandwidth adjustment amount; and adjusting the available bandwidth of the service path and the available bandwidth of the control path based on the bandwidth adjustment amount.

[0066] The bandwidth adjustment model used in the first bandwidth allocation method can be obtained through reinforcement learning. In the embodiments of this application, the actor-critic algorithm can be used to perform reinforcement learning on the bandwidth adjustment model, so that the bandwidth adjustment model can adaptively learn the constantly changing load patterns.

[0067] When using the actor-critic algorithm to perform reinforcement learning on a bandwidth adjustment model, the bandwidth adjustment model can include an actor network and a critic network, which can include multiple fully connected layers.

[0068] When using the bandwidth adjustment model for bandwidth adjustment, the current service traffic status information, current control traffic status information, and current transmission channel status information can be input into the actor network of the bandwidth adjustment model to output the bandwidth adjustment amount. The current service traffic status information, current control traffic status information, and current transmission channel status information can be input into the critic network to estimate the value of the current state and obtain the current state evaluation value.

[0069] After obtaining the bandwidth adjustment amount, the available bandwidth of the service path and the available bandwidth of the control path can be adjusted separately based on the bandwidth adjustment amount to obtain the adjusted available bandwidth of each service path and the control path.

[0070] In some embodiments, available bandwidth can represent the proportion of the total bandwidth of the transmission channel, and bandwidth adjustment amount can represent the amount of adjustment to the proportion. For example, if the original available bandwidth of the service path is 60% of the total bandwidth of the transmission channel, and the original available bandwidth of the control path is 40% of the total bandwidth of the transmission channel, the bandwidth adjustment amount can represent reducing the proportion of the available bandwidth of the service path in the total bandwidth by 10%, so that after the adjustment, the available bandwidth of the service path accounts for 50% of the total bandwidth of the transmission channel, and the available bandwidth of the control path accounts for 50% of the total bandwidth of the transmission channel.

[0071] Because the available bandwidth of the service path and control path has been adjusted, the service traffic state, control traffic state, and transmission channel state of the storage device have all changed. The next service traffic state information, the next control traffic state information, and the next transmission channel state information after the bandwidth adjustment can be obtained. The next service traffic state information, the next control traffic state information, and the next transmission channel state information are input into the critic network. The critic network is used to estimate the value of the next state and obtain the next state evaluation value.

[0072] After obtaining the next state evaluation value, the reward function can be used to calculate the current reward value based on the service traffic delay, heartbeat delay, and transmission channel bandwidth utilization. The reward function can be represented by the following formula (1):

[0073] rt =0.6×(1 / max(data_latency,0.1))+0.3×(1-min(heartbeat_delay / 500,1)) +0.1× (1 - abs(bandwidth_usage - 0.7)) (1);

[0074] Where rt represents the current reward value, data_latency represents the service latency, heartbeat_delay represents the heartbeat latency, and bandwidth_usage represents the bandwidth utilization rate.

[0075] After obtaining the reward value, the reward value, the current state evaluation value, and the next state evaluation value can be processed using the Temporal-Difference (TD) error function to obtain the current TD error. The Temporal-Difference error function can be represented by the following formula (2):

[0076] δt=rt+θV(st+1)- V(st) (2);

[0077] Where δt represents the current TD error, θ represents the discount factor, V(st+1) represents the next state evaluation value, and V(st) represents the current state evaluation value.

[0078] After obtaining the TD error, the parameters of the actor network and the critic network are adjusted based on the TD error to achieve iterative updates of the bandwidth adjustment model.

[0079] According to embodiments of this application, bandwidth allocation is performed using a bandwidth adjustment model obtained through reinforcement learning. This allows the parameters of the bandwidth adjustment model to change according to changes in state information, thereby enabling the bandwidth adjustment model to adapt to constantly changing load patterns and improving the accuracy of bandwidth allocation.

[0080] According to embodiments of this application, the current service traffic status information includes at least one of service processing rate, service latency, and pending service volume. In one specific embodiment, the service processing rate can be the input / output per second (IOPS) of the storage device, the service latency can be the 99th percentile latency (P99), and the pending service volume can be the service queue depth.

[0081] According to embodiments of this application, the current control flow status information includes at least one of heartbeat delay and the amount of metadata to be synchronized. In one specific embodiment, the heartbeat delay may be the average delay of the last 10 heartbeats, and the amount of metadata to be synchronized may characterize the backlog of metadata synchronization.

[0082] According to embodiments of this application, the current transmission channel status information includes transmission channel bandwidth utilization and cross-storage device latency. In one specific embodiment, the dual-active storage system includes multiple data centers, and cross-storage device latency may include cross-data center latency.

[0083] According to embodiments of this application, by utilizing state data from multiple dimensions to determine the bandwidth adjustment amount, the data input to the bandwidth adjustment model becomes more comprehensive, improving the accuracy of the bandwidth adjustment amount output by the bandwidth adjustment model, and thus improving the accuracy of the available bandwidth allocated to the service path and control path.

[0084] According to embodiments of this application, the output of the bandwidth adjustment model further includes a control path bandwidth limit threshold; the bandwidth allocation method further includes: in response to the current available bandwidth of the control path being less than the control path bandwidth limit threshold, triggering synchronous metadata compression and / or heartbeat aggregation.

[0085] The bandwidth adjustment model can output a control path bandwidth limit threshold that matches the current status information based on real-time status information. However, under heavy workloads, the current available bandwidth of the adjusted control path may be less than the control path bandwidth limit threshold, leading to high control traffic latency or even loss of control signaling, which may affect communication between multiple storage devices.

[0086] To ensure both service performance and communication reliability, synchronous metadata compression and / or heartbeat aggregation can be triggered when the current available bandwidth of the control path is less than the control path bandwidth limit threshold. This reduces the amount of data in the control traffic, thereby reducing the transmission pressure on the control path and ensuring that critical control signaling is not lost.

[0087] According to embodiments of this application, synchronous metadata compression and / or heartbeat aggregation are triggered in response to the current available bandwidth of the control path being less than the control path bandwidth limit threshold, while ensuring service performance and communication reliability.

[0088] According to an embodiment of this application, determining the available bandwidth of the service path and control path of the storage device based on the target bandwidth allocation method includes: in response to determining the target bandwidth allocation method as the second bandwidth allocation method, determining the service path weight based on service traffic status information using a preset service weight determination function; determining the control path weight based on control traffic status information using a preset control weight determination function; and determining the available bandwidth of the service path and control path of the storage device based on the total bandwidth of the transmission channel of the storage device, the service path weight, and the control path weight.

[0089] When using the second bandwidth allocation method, the dynamic weights of the service path and the control path can be determined based on the service traffic status information and the control traffic status information, and then bandwidth can be allocated based on the dynamic weights of the service path and the control path.

[0090] In the embodiments of this application, a preset service weight determination function is used to determine the service path weight based on service traffic status information. The preset service weight determination function can be shown in the following formula (3):

[0091] W_data=α×(IOPS / IOPS_max)+β×(Latency_actual / Latency_SLA)+γ×(Business_Priority) (3);

[0092] Where W_data represents the business path weight, IOPS represents the input / output per second, IOPS_max represents the maximum input / output per second of the storage device, Latency_SLA represents the maximum allowable latency of the business, Latency_actual represents the business latency of the storage device, Business_Priority represents the business priority of the business to be processed, ranging from 0.1 to 1.0, and α, β, and γ represent adjustable parameters.

[0093] In the embodiments of this application, a preset control weight determination function is used to determine the control path weight based on control flow status information. The preset control weight determination function can be represented by the following formula (4):

[0094] W_ctrl=μ×(Heartbeat_Urgency)+λ×(Metadata_Update_Rate)+ω×(Network_Jitter) (4);

[0095] Where W_ctrl represents the control path weight, Heartbeat_Urgency represents the heartbeat delay, Metadata_Update_Rate represents the metadata change frequency, Network_Jitter represents network jitter, and μ, λ, and ω represent adjustable parameters.

[0096] After determining the service path weight and control path weight, the bandwidth ratio of each service path and control path can be determined based on the ratio of their respective weights, thus determining the available bandwidth of each. The process of determining the available bandwidth of each service path and control path of the storage device can be illustrated by the following formulas (5) to (6):

[0097] BW_data = (W_data / (W_data + W_ctrl)) × Total_BW (5);

[0098] BW_ctrl = Total_BW - BW_data (6);

[0099] Wherein, BW_data represents the available bandwidth of the service path, BW_ctrl represents the available bandwidth of the control path, and Total_BW represents the total bandwidth of the transmission channel.

[0100] According to embodiments of this application, the service path weight and control path weight are dynamically determined based on service traffic status information and control traffic status information. This allows for the adjustment of the available bandwidth of the service path and control path based on the service path weight and control path weight, so that the allocated available bandwidth matches the actual bandwidth demand and improves bandwidth utilization.

[0101] Figure 4 A flowchart illustrating bandwidth allocation according to an embodiment of this application is shown.

[0102] like Figure 4 As shown, bandwidth allocation includes operations S410 to S460.

[0103] When operating S410, obtain service traffic status information.

[0104] When operating the S420, obtain control flow status information.

[0105] The S430 is used to obtain transmission channel status information.

[0106] In operation S440, determine whether the target bandwidth allocation method is the first bandwidth allocation method. If the target bandwidth allocation method is the first bandwidth allocation method, execute operation S450; otherwise, execute operation S460.

[0107] When operating the S450, bandwidth allocation is performed based on a bandwidth adjustment model obtained through machine learning, utilizing service traffic status information, control traffic status information, and transmission channel status information.

[0108] When operating S460, bandwidth allocation is performed based on dynamic weights using service traffic status information and control traffic status information.

[0109] According to embodiments of this application, the available bandwidth of the service path and control path of the storage device is determined based on the total bandwidth of the transmission channel, the service path weight, and the control path weight. This includes: determining the initial service bandwidth ratio and the initial control bandwidth ratio based on the service path weight and the control path weight; determining the available bandwidth of the service path based on the service bandwidth ratio threshold and the total bandwidth of the transmission channel when the initial service bandwidth ratio is greater than the service bandwidth ratio threshold; and determining the available bandwidth of the control path based on the total bandwidth of the transmission channel and the available bandwidth of the service path.

[0110] Under heavy workloads, the initial service bandwidth ratio and the initial control bandwidth ratio may be severely unbalanced, with the initial service bandwidth ratio being extremely high and the initial control bandwidth ratio being extremely low. This results in high control traffic latency, affecting communication between multiple storage devices.

[0111] To ensure normal communication between multiple storage devices, a service bandwidth ratio threshold can be preset. If the initial service bandwidth ratio is greater than the service bandwidth ratio threshold, the available bandwidth of the service path is determined based on the service bandwidth ratio threshold to avoid difficulties in normal communication between multiple storage devices due to insufficient available bandwidth of the control path.

[0112] According to the embodiments of this application, when the initial service bandwidth ratio is greater than the service bandwidth ratio threshold, the available bandwidth of the service path is determined based on the service bandwidth ratio threshold and the total bandwidth of the transmission channel, thereby limiting the available bandwidth of the service path and avoiding excessively low available bandwidth of the control path, thus ensuring communication reliability.

[0113] According to an embodiment of this application, the bandwidth allocation method further includes: determining the priority of each of the multiple pending services based on their respective service attribute information; determining the total priority of the multiple pending services based on their respective priorities; and adjusting the initial service bandwidth ratio threshold based on the total priority to obtain the service bandwidth ratio threshold.

[0114] Since multiple services have different priorities, the latency requirements for low-priority services are lower. Therefore, a service bandwidth ratio threshold can be set and changed in real time according to the priority of the service to be processed. When the priority is low, the service bandwidth ratio threshold can be appropriately reduced.

[0115] When determining the priority of multiple pending services, the priority of each service can be determined based on its respective service attribute information, such as its service type. The average of the multiple priorities can then be used to determine the total priority.

[0116] After determining the total priority, the total priority can be compared with the priority threshold. If the total priority is greater than the priority threshold, a preset percentage threshold adjustment amount can be added to the initial service bandwidth percentage threshold. If the total priority is less than or equal to the priority threshold, the preset percentage threshold adjustment amount can be reduced from the initial service bandwidth percentage threshold to obtain the service bandwidth percentage threshold.

[0117] According to the embodiments of this application, by adjusting the service bandwidth ratio threshold in real time according to the priority of the service to be processed, it is possible to avoid the backlog of control traffic caused by the long-term low availability of control path bandwidth under high service load, thereby ensuring service performance requirements while improving communication reliability.

[0118] Based on the bandwidth allocation method for storage devices in a dual-active storage system described above, this application also provides a bandwidth allocation device for storage devices in a dual-active storage system. The following will be combined with... Figure 5 The device is described in detail.

[0119] Figure 5 A structural block diagram of a bandwidth allocation device for a storage device in a dual-active storage system according to an embodiment of this application is shown.

[0120] like Figure 5 As shown, the bandwidth allocation device 500 for storage devices in a dual-active storage system in this embodiment includes a load analysis module 510, a mode determination module 520, and a bandwidth allocation module 530.

[0121] The load analysis module 510 is used to analyze the load data of the storage device within a predetermined time period and determine the load change rate within the predetermined time period. In one embodiment, the load analysis module 510 can be used to perform the operation S210 described above, which will not be repeated here.

[0122] The mode determination module 520 is used to determine one of a first bandwidth allocation mode and a second bandwidth allocation mode as the target bandwidth allocation mode based on the load change rate. The first bandwidth allocation mode is based on a bandwidth adjustment model obtained through reinforcement learning, and the second bandwidth allocation mode is based on dynamic weights. In one embodiment, the mode determination module 520 can be used to perform the operation S220 described above, which will not be repeated here.

[0123] The bandwidth allocation module 530 is used to determine the available bandwidth for the service path and control path of the storage device based on the target bandwidth allocation method. The service path is used to transmit service traffic, and the control path is used to transmit control traffic. The service path and control path share the transmission channel of the storage device. In one embodiment, the bandwidth allocation module 530 can be used to perform the operation S230 described above, which will not be repeated here.

[0124] According to an embodiment of this application, the bandwidth allocation module 530 includes an adjustment determination submodule and a bandwidth adjustment submodule.

[0125] The adjustment determination submodule is used to respond to the determination that the target bandwidth allocation method is the first bandwidth allocation method. It inputs the current service traffic status information, the current control traffic status information, and the current transmission channel status information into the bandwidth adjustment model to obtain the bandwidth adjustment amount.

[0126] The bandwidth adjustment submodule is used to adjust the available bandwidth of the service path and the available bandwidth of the control path based on the bandwidth adjustment amount.

[0127] According to embodiments of this application, the current service traffic status information includes at least one of service processing rate, service latency, and pending service volume; the current control traffic status information includes at least one of heartbeat latency and pending metadata volume; and the current transport channel status information includes transport channel bandwidth utilization and cross-storage device latency.

[0128] According to an embodiment of this application, the output of the bandwidth adjustment model further includes a control path bandwidth limit threshold; the bandwidth allocation module 530 also includes a control trigger module.

[0129] The control trigger module is used to trigger synchronous metadata compression and / or heartbeat aggregation in response to the current available bandwidth of the control path being less than the control path bandwidth limit threshold.

[0130] According to an embodiment of this application, the bandwidth allocation module 530 further includes a service determination submodule, a control determination submodule, and a bandwidth determination submodule.

[0131] The service determination submodule is used to determine the service path weight based on service traffic status information in response to the determination that the target bandwidth allocation method is the second bandwidth allocation method.

[0132] The control determination submodule is used to determine the control path weights based on the control flow status information using a preset control weight determination function.

[0133] The bandwidth determination submodule is used to determine the available bandwidth of the service path and control path of the storage device based on the total bandwidth of the transmission channel, the service path weight, and the control path weight of the storage device.

[0134] According to an embodiment of this application, the bandwidth determination submodule includes an initial determination unit, a threshold determination unit, and a bandwidth determination unit.

[0135] The initial determination unit is used to determine the initial service bandwidth ratio and the initial control bandwidth ratio based on the service path weight and the control path weight.

[0136] The threshold determination unit is used to determine the available bandwidth of the service path based on the service bandwidth percentage threshold and the total bandwidth of the transmission channel when the initial service bandwidth percentage is greater than the service bandwidth percentage threshold.

[0137] The bandwidth determination unit is used to determine the available bandwidth of the control path based on the total bandwidth of the transmission channel and the available bandwidth of the service path.

[0138] According to an embodiment of this application, the bandwidth allocation device 500 for a storage device in a dual-active storage system further includes a priority determination module, a total priority determination module, and a percentage threshold determination module.

[0139] The priority determination module is used to determine the priority of each of the multiple pending services based on their respective service attribute information.

[0140] The overall priority determination module is used to determine the overall priority of multiple pending services based on their individual priorities.

[0141] The percentage threshold determination module is used to adjust the initial service bandwidth percentage threshold according to the total priority to obtain the service bandwidth percentage threshold.

[0142] According to an embodiment of this application, the method determination module 520 includes a first determination submodule and a second determination submodule.

[0143] The first determining submodule is used to determine the first bandwidth allocation method as the target bandwidth allocation method in response to determining that the load change rate is greater than the load change rate threshold.

[0144] The second determining submodule is used to determine the second bandwidth allocation method as the target bandwidth allocation method in response to determining that the load change rate is less than or equal to the load change rate threshold.

[0145] According to an embodiment of this application, the load analysis module 510 includes a mode determination submodule, a load fitting submodule, and a change determination submodule.

[0146] The method determination submodule is used to determine the target fitting method for fitting the load data based on the distribution characteristics of the load data within a predetermined time period.

[0147] The load fitting submodule is used to perform curve fitting on the load data using a target fitting method to obtain the load change curve.

[0148] The Change Determination submodule is used to determine the load change rate based on the load change curve.

[0149] According to embodiments of this application, any plurality of modules among the load analysis module 510, mode determination module 520, and bandwidth allocation module 530 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this application, at least one of the load analysis module 510, mode determination module 520, and bandwidth allocation module 530 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any appropriate combination of any of these three implementation methods. Alternatively, at least one of the load analysis module 510, mode determination module 520, and bandwidth allocation module 530 can be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0150] Figure 6 A block diagram of an electronic device suitable for implementing a bandwidth allocation method for a storage device in a dual-active storage system, according to an embodiment of this application, is shown.

[0151] like Figure 6As shown, an electronic device 600 according to an embodiment of this application includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0152] RAM 603 stores various programs and data required for the operation of electronic device 600. Processor 601, ROM 602, and RAM 603 are interconnected via bus 604. Processor 601 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 602 and / or RAM 603. It should be noted that programs may also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in one or more memories.

[0153] According to embodiments of this application, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to a bus 604. The electronic device 600 may also include one or more of the following components connected to the input / output (I / O) interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0154] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0155] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 602 and / or RAM 603 and / or one or more memories other than ROM 602 and RAM 603 described above.

[0156] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this application.

[0157] When the computer program is executed by the processor 601, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0158] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0159] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, it performs the functions defined in the system of this application embodiment. According to the embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0160] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0161] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0162] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

[0163] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of this application, those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. A bandwidth allocation method for storage devices in a dual-active storage system, characterized in that, The bandwidth allocation method includes: Analyze the load data of the storage device within a predetermined time period to determine the load change rate within the predetermined time period; Based on the load change rate, one of the first bandwidth allocation method and the second bandwidth allocation method is determined as the target bandwidth allocation method. The first bandwidth allocation method is based on a bandwidth adjustment model obtained through machine learning, and the second bandwidth allocation method is based on dynamic weights. Based on the target bandwidth allocation method, the available bandwidth of the service path and control path of the storage device is determined. The service path is used to transmit service traffic, and the control path is used to transmit control traffic. The service path and the control path share the transmission channel of the storage device.

2. The bandwidth allocation method according to claim 1, characterized in that, The step of determining the available bandwidth for the service path and control path of the storage device based on the target bandwidth allocation method includes: In response to determining that the target bandwidth allocation method is the first bandwidth allocation method, the current service traffic status information, the current control traffic status information, and the current transmission channel status information are input into the bandwidth adjustment model to obtain the bandwidth adjustment amount; Based on the bandwidth adjustment amount, the available bandwidth of the service path and the available bandwidth of the control path are adjusted respectively.

3. The bandwidth allocation method according to claim 2, characterized in that, The current service traffic status information includes at least one of service processing rate, service latency, and pending service volume; The current control flow status information includes at least one of heartbeat delay and the amount of metadata to be synchronized; The current transmission channel status information includes transmission channel bandwidth utilization and cross-storage device latency.

4. The bandwidth allocation method according to claim 2, characterized in that, The output of the bandwidth adjustment model also includes a control path bandwidth limit threshold; the bandwidth allocation method further includes: In response to the current available bandwidth of the control path being less than the control path bandwidth limit threshold, synchronous metadata compression and / or heartbeat aggregation are triggered.

5. The bandwidth allocation method according to claim 1, characterized in that, The step of determining the available bandwidth for the service path and control path of the storage device based on the target bandwidth allocation method includes: In response to determining that the target bandwidth allocation method is the second bandwidth allocation method, the service path weight is determined based on the service traffic status information using a preset service weight determination function. The control path weights are determined using a preset control weight determination function based on control flow status information. Based on the total bandwidth of the transmission channel of the storage device, the service path weight, and the control path weight, the available bandwidth of the service path and control path of the storage device is determined.

6. The bandwidth allocation method according to claim 5, characterized in that, The determination of the available bandwidth of the service path and control path of the storage device based on the total bandwidth of the transmission channel of the storage device, the service path weight, and the control path weight includes: Based on the service path weight and the control path weight, the initial service bandwidth ratio and the initial control bandwidth ratio are determined. If the initial service bandwidth ratio is greater than the service bandwidth ratio threshold, the available bandwidth of the service path is determined based on the service bandwidth ratio threshold and the total bandwidth of the transmission channel. The available bandwidth of the control path is determined based on the total bandwidth of the transmission channel and the available bandwidth of the service path.

7. The bandwidth allocation method according to claim 6, characterized in that, The bandwidth allocation method further includes: Based on the business attribute information of each of the multiple pending services, the priority of each of the multiple pending services is determined. The overall priority of the multiple pending services is determined based on the individual priorities of each service. Based on the total priority, the initial service bandwidth percentage threshold is adjusted to obtain the service bandwidth percentage threshold.

8. The bandwidth allocation method according to claim 1, characterized in that, The step of determining one of the first bandwidth allocation method and the second bandwidth allocation method as the target bandwidth allocation method based on the load change rate includes: In response to determining that the load change rate is greater than the load change rate threshold, the first bandwidth allocation method is determined to be the target bandwidth allocation method; In response to determining that the load change rate is less than or equal to the load change rate threshold, the second bandwidth allocation method is determined to be the target bandwidth allocation method.

9. The bandwidth allocation method according to claim 1, characterized in that, The step of analyzing the load data of the storage device within a predetermined time period to determine the load change rate within the predetermined time period includes: Based on the distribution characteristics of the load data within the predetermined time period, a target fitting method for fitting the load data is determined. Using the target fitting method, curve fitting is performed on the load data to obtain the load change curve; The load change rate is determined based on the load change curve.

10. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the bandwidth allocation method according to any one of claims 1 to 9.

Images