Data balancing method and system for distributed storage online scaling

Abstract

The application relates to the technical field of distributed storage, and discloses a data balancing method and system for online expansion and contraction of distributed storage, which comprises the following steps: extracting shard read-write load, time delay, address concentration, hot and cold attenuation and bandwidth occupation from block device request logs and node monitoring data, and obtaining a main feature sequence through mutual information screening; splitting total request load into trend, period and residual components, calculating a burst correction based on impulse response when an upstream residual anomaly occurs, and obtaining a comprehensive prediction load; calculating an imbalance cost from capacity deviation and overload risk, simulating changes in the overload probability of a source node through counterfactual migration, and generating a migration task list by deducting migration cost from causal benefits; adjusting token rate based on predicted bandwidth and confidence interval, and suspending or resuming migration when time delay exceeds a limit; and adjusting prediction parameters and conservative coefficients according to load error and bandwidth out-of-limit during migration, and forming a closed-loop feedback.

Description

Technical Field

[0001] This invention relates to the field of distributed storage technology, and specifically to a data balancing method and system for online scaling up and down of distributed storage. Background Technology

[0002] When adding or removing nodes in a distributed storage cluster, the existing data shards need to be redistributed across the storage nodes to maintain a balance in capacity and access pressure among the nodes. Existing data balancing methods typically generate migration tasks based on consistent hashing, capacity differences, or fixed migration thresholds, which can complete basic data redistribution when node capacity changes.

[0003] In scaling up and down scenarios where online services are continuously running, migrating solely based on capacity differences can easily overlook factors such as actual read / write load, service latency, network bandwidth usage, data hot / cold variations, and access locality. While some nodes may have near-average capacity usage, they can still become performance bottlenecks due to concentrated hotspot sharding, periodic service peaks, or sudden access propagation, leading to a discrepancy between capacity balancing and performance balancing.

[0004] Furthermore, migration execution typically employs a fixed migration rate or simple remaining bandwidth ratio control. When service input / output loads are at their peak, the migrated input / output may compete with online services for network bandwidth and storage resources, causing service latency fluctuations. If the migration plan lacks feedback and correction based on actual load, actual bandwidth usage, and migration completion status after generation, the migration plan will struggle to adapt to load changes during scaling up or down in a timely manner.

[0005] Therefore, this application provides a data balancing method and system for online scaling up and down of distributed storage to solve the above-mentioned technical problems. Summary of the Invention

[0006] The purpose of this invention is to provide a data balancing method and system for online scaling of distributed storage, in order to solve the technical problems of existing distributed storage causing significant interference to online services during the online scaling data balancing process due to the single balancing decision dimension, the disconnect between migration rate control and business load fluctuations, and the lack of migration execution feedback.

[0007] To address the aforementioned technical problems, this invention provides a data balancing method for online scaling up and down of distributed storage, comprising: Extract the read / write load, tail latency, address concentration, hot / cold attenuation, and node bandwidth usage of data shards from block device request logs and node resource monitoring data. Sort and filter according to the mutual information of each feature and bottleneck marker to obtain the node main feature sequence. The total request load in the node main feature sequence is divided into trend component, periodic component and residual component. The trend component and periodic component are extrapolated, and when the current residual deviation of the upstream associated node identified by the residual causality test exceeds the historical fluctuation threshold, the burst correction amount is calculated using the impulse response function to obtain the comprehensive predicted load. The node imbalance cost is formed based on the capacity deviation and the risk of the comprehensive predicted load exceeding the safety threshold. Candidate migration shards are generated for the imbalance source node. The load change after the candidate migration shards migrate out of the source node and into the target node is simulated. The migration task list is obtained by subtracting the migration cost from the decrease in the overload probability of the source node. Migration is performed by consuming migration token buckets in the causal benefit order of migration tasks. The token generation rate is calculated based on the service bandwidth usage and the predicted confidence interval half-width. Migration inputs and outputs are suspended or resumed when service latency triggers suspension or resumption conditions. The prediction parameters are re-estimated based on the load error during the migration, the token bucket conservatism coefficient is adjusted based on the bandwidth out-of-bounds status, and the control parameters for the next migration round are output.

[0008] In some specific embodiments, the node main feature sequence is obtained by sorting and filtering according to the mutual information between each feature and the bottleneck marker, and further includes: The number of fragment read / write requests, the number of requested bytes, and the tail latency are counted according to the time window. The Gini coefficient or information entropy of the requested logical block address is calculated as the address concentration. Estimate the hot and cold decay constants based on the most recent access time and the historical access interval, calculate the half-life of the heat of each segment, and form the hot and cold decay amount; After concatenating the sharding features with the node bandwidth usage and input / output ratios, mutual information calculation is performed on each feature and the bottleneck marker, and a preset number of features with the highest ranking are retained as the constituent fields of the node's main feature sequence.

[0009] In some specific embodiments, the total request load in the node master feature sequence is split into a trend component, a periodic component, and a residual component. Extrapolation of the trend component and the periodic component further includes: Perform seasonal trend decomposition on the total request load of each node within a continuous time window to obtain trend components, periodic components, and residual components. Local weighted regression smoothing is performed on the trend component and linear extrapolation is performed along the prediction window; Fourier series fitting is performed on the periodic component and harmonic extrapolation is performed along the prediction window. The deterministic forecasting benchmark is obtained by adding the trend extrapolation value and the period extrapolation value.

[0010] In some specific embodiments, when the current residual deviation of the upstream associated node identified by the residual causality test exceeds the historical fluctuation threshold, a sudden correction amount is calculated using an impulse response function to obtain the comprehensive predicted load, further including: A vector autoregression relationship is established for the residual components of any two storage nodes, and the Granger causality test is used to screen upstream related nodes; When the current residual deviation of the upstream associated node exceeds the historical fluctuation threshold, the current residual deviation is multiplied by the impulse response function estimated from the historical residual sequence by the prediction step size to obtain the burst correction amount of the target node within the prediction step size. The burst correction amount is added to the deterministic prediction baseline to obtain the comprehensive prediction load.

[0011] In some specific embodiments, the node imbalance cost is formed based on capacity deviation and the risk of the overall predicted load exceeding the safety threshold. Candidate migration shards are then generated for the imbalance source node, further including: The normalized deviation of the node's used capacity relative to the cluster's average used capacity is used to obtain the capacity deviation; The probability and magnitude of the comprehensive predicted load exceeding the node safety threshold within the prediction window are statistically analyzed to obtain the overload risk. The node imbalance cost is obtained by assigning weights to capacity deviation and overload risk according to the business service level constraints and then summing them up. When the node imbalance cost exceeds the preset trigger condition, the data shards managed by the triggering node are removed, and the data shards whose cold and hot attenuation is lower than the preset cooling condition and whose request load in the source node's total request load is lower than the preset contribution condition are removed, thus obtaining the candidate migration shards.

[0012] In some specific embodiments, the load change after candidate migration shards migrate out of the source node and into the target node is simulated, and the migration task list is obtained by subtracting the migration cost from the decrease in the source node overload probability. This further includes: For each candidate migration shard, read the shard request load, shard bandwidth contribution, and shard data volume. Subtract the shard request load and shard bandwidth contribution from the source node prediction sequence and add the shard request load and shard bandwidth contribution to the target node prediction sequence. By comparing the overload probability of the source node before and after the migration, the causal benefits can be obtained. Calculate the migration cost using the amount of data in the fragment and the unit transmission overhead, and select the candidate migration fragments whose causal benefits are greater than the migration cost and whose causal benefits are ranked highest to add them to the migration task list. If the node imbalance cost of the target node exceeds the preset backtracking condition after being added, the most recently added migration task is removed and a new candidate migration shard is selected.

[0013] In some specific embodiments, calculating the token generation rate based on service bandwidth usage and the predicted confidence interval half-width further includes: Read the source node's total network bandwidth security limit, the service bandwidth usage within the prediction window, the prediction confidence interval half-width, and the token bucket conservatism coefficient; The token generation rate is obtained by deducting the service bandwidth usage from the total network bandwidth security limit and compensating for the half-width of the predicted confidence interval according to the token bucket conservatism coefficient. When the deducted token generation rate is lower than the preset minimum rate, the token generation rate is set to the pause rate and the unsent data blocks are kept in the migration task queue.

[0014] In some specific embodiments, suspending or resuming migration input / output when service latency triggers a suspension or resumption condition further includes: During the migration execution, business input and output latency is collected according to time windows, and the deviation of business input and output latency from historical mean and historical standard deviation is calculated. When the deviation exceeds the preset suspension condition, stop retrieving new data blocks from the migration task queue and temporarily store the retrieved but unsent data blocks. When the deviation falls back to the preset recovery condition, data blocks are retrieved according to the causal benefit order of the migration task list and the storage order of the migration task queue, and migration input and output are submitted according to the number of tokens.

[0015] In some specific embodiments, reestimating prediction parameters based on load errors during migration and adjusting the token bucket conservatism coefficient based on bandwidth out-of-bounds status further includes: By comparing the actual node load with the comprehensive predicted load over a continuous time window, a prediction error sequence is obtained. When the prediction error sequence exceeds the preset re-estimation condition, the decomposition parameters of the trend component and the periodic component are re-estimated, or the causal coefficients between nodes are recalculated using the updated residual sequence. By comparing the actual bandwidth usage with the predicted confidence interval boundary, the token bucket conservatism coefficient is increased when the actual bandwidth usage exceeds the upper limit of the predicted confidence interval, and the token bucket conservatism coefficient is decreased when the actual bandwidth usage remains below the predicted trough and the migration task queue is not cleared, thus obtaining the control parameters for the next migration round.

[0016] Based on the same concept, the present invention also provides a distributed storage online scaling data balancing system, comprising: The profile generation module is configured to extract the read / write load, tail latency, address concentration, hot / cold attenuation, and node bandwidth usage of data shards from block device request logs and node resource monitoring data, sort and filter them according to the mutual information between each feature and bottleneck marker, and obtain the node main feature sequence. The load prediction module is configured to split the total request load in the node main feature sequence into trend component, periodic component and residual component, extrapolate the trend component and periodic component, and calculate the burst correction amount using the impulse response function when the current residual deviation of the upstream associated node identified by the residual causality test exceeds the historical fluctuation threshold, so as to obtain the comprehensive predicted load. The migration planning module is configured to generate node imbalance costs based on capacity deviation and the risk of the comprehensive predicted load exceeding the safety threshold. It generates candidate migration shards for the imbalanced source node, simulates the load change after the candidate migration shards migrate out of the source node and into the target node, and obtains the migration task list by subtracting the migration cost from the decrease in the overload probability of the source node. The migration execution module is configured to consume migration token buckets to perform migrations in the causal benefit order of migration tasks, calculate the token generation rate based on service bandwidth usage and the half-width of the predicted confidence interval, and suspend or resume migration input and output when service latency triggers suspension or resumption conditions. The feedback correction module is configured to re-estimate prediction parameters based on load errors during migration, adjust the token bucket conservatism coefficient based on bandwidth out-of-bounds status, and output control parameters for the next migration round.

[0017] Compared with existing technologies, its advantages are as follows: This invention discloses a data balancing method and system for online scaling up and down of distributed storage. It extracts read / write load, tail latency, address concentration, hot / cold attenuation, and node bandwidth usage from block device request logs and node resource monitoring data. It also forms a node main feature sequence through mutual information filtering. The data balancing judgment can simultaneously receive capacity, access locality, hot / cold changes, and node resource usage information, reducing the omission of performance hotspots when migration is triggered solely based on capacity differences.

[0018] The total request load is broken down into trend components, periodic components, and residual components. When a sudden change occurs in the upstream related node identified by the residual causality test, the burst correction amount is calculated using the impulse response function. The comprehensive predicted load can simultaneously express the periodic business changes and the impact of sudden propagation between nodes, so that the subsequent node imbalance cost and migration plan are based on the load changes within the prediction window.

[0019] The node imbalance cost is formed based on the risk of capacity deviation and comprehensive prediction of load exceeding the safety threshold. The causal benefit is calculated by simulating the load change after candidate migration shards are moved out of the source node and into the target node. The migration task list can exclude shards that contribute little to the performance risk of the source node or cause imbalance of the target node after migration, so that the selection of migration objects forms a consistent processing link with hotspot mitigation, target node carrying capacity and migration cost.

[0020] The token generation rate of the migration token bucket is calculated based on the service bandwidth usage and the half-width of the predicted confidence interval. Migration inputs and outputs are suspended or resumed when service latency triggers suspension or resumption conditions. The migration execution can adjust the migration submission rhythm according to the fluctuation of service bandwidth, reducing resource competition between migration inputs and outputs and online service inputs and outputs in high-load windows.

[0021] Because this invention re-estimates the prediction parameters based on the load error during migration and adjusts the token bucket conservatism coefficient according to the bandwidth overrun status, the actual load and actual bandwidth usage during the migration process can be fed back to the prediction and control parameters of subsequent migration rounds, forming a closed-loop processing process of prediction, migration, feedback and re-control, reducing the balance deviation caused by the rigid execution of the migration plan. Attached Figure Description

[0022] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating some specific embodiments of the data balancing method for online scaling up and down of distributed storage according to the present invention; Figure 2 This is a schematic diagram of the structure of a distributed storage online scaling and shrinking data balancing system according to some specific embodiments of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms "a," "the," and "the" as used in the embodiments of this application are also intended to include the plural forms, unless the context clearly indicates otherwise, and "multiple" generally includes at least two.

[0025] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0026] It should be understood that although the terms first, second, third, etc., may be used in the embodiments of this application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, first may also be referred to as second without departing from the scope of the embodiments of this application, and similarly, second may also be referred to as first.

[0027] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0028] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0029] It should be noted that any symbols and / or numbers present in the specification that are not marked in the accompanying drawings are not reference numerals.

[0030] Reference Figure 1 A data balancing method for online scaling up and down of distributed storage, comprising: S101, extract the read / write load, tail latency, address concentration, hot / cold attenuation, and node bandwidth usage of data fragments from the block device request log and node resource monitoring data, sort and filter them according to the mutual information of each feature and bottleneck marker, and obtain the node main feature sequence. S102, the total request load in the node main feature sequence is divided into trend component, periodic component and residual component, the trend component and periodic component are extrapolated, and when the current residual deviation of the upstream associated node identified by the residual causality test exceeds the historical fluctuation threshold, the burst correction amount is calculated using the impulse response function to obtain the comprehensive predicted load. S103, based on the capacity deviation and the risk of the comprehensive predicted load exceeding the safety threshold, a node imbalance cost is formed. Candidate migration shards are generated for the imbalance source node. The load change after the candidate migration shards migrate out of the source node and into the target node is simulated. The migration cost is subtracted by the decrease in the overload probability of the source node to obtain the migration task list. S104: Execute migration by consuming migration token buckets in the causal benefit order of migration tasks. Calculate the token generation rate based on service bandwidth usage and the predicted confidence interval half-width. Suspend or resume migration input and output when service latency triggers suspension or resumption conditions. S105 re-estimates the prediction parameters based on the load error during the migration, adjusts the token bucket conservatism coefficient based on the bandwidth out-of-bounds status, and outputs the control parameters for the next migration round.

[0031] To provide a clearer explanation, the steps in the embodiments of the present invention are described in detail below: S101, extract the read / write load, tail latency, address concentration, hot / cold attenuation, and node bandwidth usage of data fragments from the block device request log and node resource monitoring data, sort and filter them according to the mutual information of each feature and bottleneck marker, and obtain the node main feature sequence. In this embodiment, the distributed storage cluster comprises multiple storage nodes, each managing multiple data shards. Block device request logs are generated by the storage node's block layer tracker, storage engine logs, or I / O scheduling layer records. Node resource monitoring data is periodically collected by the node monitoring agent. The block device request logs include at least the request time, data shard identifier, logical block address, read / write type, requested byte size, request completion latency, and the node to which the request belongs. Node resource monitoring data includes at least the node's network transmit bandwidth usage, node's network receive bandwidth usage, processor I / O wait percentage, and node's used capacity. The above data is aligned to the same time window during collection, for example, using a 10-minute statistical window. The window length can also be configured to 1 minute, 5 minutes, or 15 minutes to adapt to the frequency of cluster load changes. For missing log entries, if the missing proportion is lower than a preset missing proportion threshold, the statistical values ​​of adjacent windows of the same data shard are used for interpolation; if the missing proportion exceeds the preset missing proportion threshold, the corresponding data shard is marked as a low-confidence input, and the ranking weight of the corresponding feature is reduced during subsequent mutual information filtering.

[0032] Furthermore, the number of fragment read / write requests, the number of requested bytes, and the tail latency are counted according to the time window, and the Gini coefficient or information entropy of the requested logical block address is calculated as the address concentration. Estimate the hot and cold decay constants based on the most recent access time and the historical access interval, calculate the half-life of the heat of each segment, and form the hot and cold decay amount; After concatenating the sharding features with the node bandwidth usage and input / output ratios, mutual information calculation is performed on each feature and the bottleneck marker, and a preset number of features with the highest ranking are retained as the constituent fields of the node's main feature sequence.

[0033] In this embodiment, the number of read requests, the number of write requests, the number of read requests in bytes, the number of write requests in bytes, and the request completion latency for each data shard within the current statistical window are aggregated to obtain read / write load and tail latency. Tail latency can be taken as the 99th percentile of the request completion latency within the window, or the 95th percentile, or a preset high percentile. Address concentration is used to characterize the degree of concentration of requested logical block addresses within the data shard. When using the Gini coefficient, the logical address range of the data shard is divided into multiple address buckets, the number of requests within each address bucket is counted, and the Gini coefficient is calculated based on the number of requests per address bucket. When using information entropy, the proportion of the number of requests per address bucket to the total number of requests in the window is obtained, and the information entropy is calculated using the entropy formula. A larger Gini coefficient or a smaller information entropy indicates that access is concentrated in a smaller address region. The address concentration, along with the load release of the source node after shard migration, participates in the subsequent candidate shard migration selection.

[0034] In this embodiment, a popularity value is maintained for each data shard. Data shards At any moment The heat value is calculated using the following formula: ; in, Data sharding At any moment Popularity value; Data sharding The initial heat of the most recent access can be a fixed constant or proportional to the number of data fragment requests or the number of bytes requested in the most recent access window. Data sharding The thermal decay constant; The duration of silence from the last access time to the current time; It is a natural constant.

[0035] Cold and heat attenuation constant Data can be sharded The historical access interval sequence is obtained through maximum likelihood estimation, or it can be approximated by the reciprocal of the mean of the historical access intervals. Fragment popularity half-life. Calculate using the following formula: ; in, Data sharding The heat half-life; It is the natural logarithm of 2; Data sharding The thermal decay constant.

[0036] The amount of thermal attenuation can be taken as You can also take The normalized result. If the number of historical access interval samples is insufficient, the median decay constant of the same type of data shard within the corresponding business volume or storage pool is used as the normalized result. The initial value.

[0037] In this embodiment, sharding features and node-level features are concatenated into a candidate feature set. Sharding features include at least read / write load, tail latency, address concentration, and hot / cold attenuation; node-level features include at least node bandwidth occupancy and input / output waiting ratio. Bottleneck markers can be generated by whether a node experiences tail latency exceeding a safety threshold, service request queue length exceeding a queue threshold, or bandwidth occupancy exceeding a bandwidth threshold within a historical window. For candidate features and bottleneck markers, the mutual information value is calculated by taking the logarithmic weighted sum of the joint probability and the ratio of their respective probabilities within their value ranges. Continuous features can be binned using equal frequency or equal width binning before calculation. During sorting and filtering, a preset number of features with high mutual information ranking are retained to form a node master feature sequence. The node master feature sequence is arranged in time window order and serves as input for total request load splitting, residual causality testing, and comprehensive predicted load calculation in subsequent steps.

[0038] In this embodiment, feature stability constraints can be added to the mutual information ranking results. Specifically, the mutual information ranking of candidate features is calculated in multiple consecutive historical windows. When a candidate feature enters the top ranking of a preset number of historical windows within a preset proportion, the candidate feature is retained. When a candidate feature only ranks highly in a single abnormal window, the candidate feature is transferred to the observation set and awaits confirmation in subsequent windows. With the feature stability constraints, the node's main feature sequence can avoid the introduction of unstable fields by occasional log spikes, while still maintaining its connection with the input of subsequent comprehensive prediction load calculations.

[0039] S102, the total request load in the node main feature sequence is divided into trend component, periodic component and residual component, the trend component and periodic component are extrapolated, and when the current residual deviation of the upstream associated node identified by the residual causality test exceeds the historical fluctuation threshold, the burst correction amount is calculated using the impulse response function to obtain the comprehensive predicted load. In this embodiment, the node master feature sequence is received. For each storage node, the total request load time series is read from the node master feature sequence. The total request load can be the sum of the number of read requests and the number of write requests, the service bandwidth usage converted from the number of bytes of read and write requests, or the comprehensive request load obtained by weighting multiple input and output load fields. The time series of each node is arranged in statistical window order, and the prediction window can be set to 6, 12, or 24 future windows. This step separates the stable predictable trend changes and periodic changes, and assigns sudden residuals that are difficult to explain by the time regularity of a single node to the causal relationship between nodes for processing. Through the above decomposition method, subsequent steps can obtain the prediction basis for both capacity and performance risk dimensions when generating the migration plan.

[0040] Furthermore, seasonal trend decomposition is performed on the total request load of each node within a continuous time window to obtain trend components, periodic components, and residual components. Local weighted regression smoothing is performed on the trend component and linear extrapolation is performed along the prediction window; Fourier series fitting is performed on the periodic component and harmonic extrapolation is performed along the prediction window. The deterministic forecasting benchmark is obtained by adding the trend extrapolation value and the period extrapolation value; A vector autoregression relationship is established for the residual components of any two storage nodes, and the Granger causality test is used to screen upstream related nodes; When the current residual deviation of the upstream associated node exceeds the historical fluctuation threshold, the current residual deviation is multiplied by the impulse response function estimated from the historical residual sequence by the prediction step size to obtain the burst correction amount of the target node within the prediction step size. The burst correction amount is added to the deterministic prediction baseline to obtain the comprehensive prediction load.

[0041] In this embodiment, for nodes At any moment The total request load undergoes seasonal trend decomposition, breaking it down into a trend component, a periodic component, and a residual component. The trend component reflects the slow changes in load due to scaling up, scaling down, or changes in business scale; the periodic component reflects the periodic fluctuations caused by intraday peaks, scheduled tasks, or batch processing tasks; and the residual component reflects sudden changes not explained by the trend and periodic components. Seasonal trend decomposition can be implemented using robust local regression. When abnormal peaks exist, load points exceeding historical percentile thresholds are truncated before decomposition to reduce the impact of individual sudden points on the trend component. If the number of historical windows is insufficient to support complete periodic decomposition, a moving average is used as the trend component, and the total request load minus the moving average is used as the residual component.

[0042] In this embodiment, local weighted regression smoothing is performed on the trend component to obtain the future prediction step size. Extrapolate the trend value; perform Fourier series fitting on the periodic components to obtain the future prediction step size. The periodic extrapolation value on the node. The deterministic prediction benchmark is calculated using the following formula: ; in, For nodes Future prediction step size Deterministic prediction benchmark; For nodes The extrapolated value of the trend component; For nodes Extrapolated values ​​of periodic components.

[0043] The harmonic order of the Fourier series can be an integer between 1 and 5. If fitting higher-order harmonics causes the change in adjacent prediction windows to exceed the historical fluctuation limit, the harmonic order is reduced and the period extrapolation value is recalculated. The deterministic prediction benchmark is written to the prediction cache. The prediction cache is indexed by node identifier, prediction start time, and prediction step size for reading when calculating sudden corrections and for subsequent calculations of node imbalance costs.

[0044] In this embodiment, for any two storage nodes and A vector autoregression relationship is established for the residual components. The maximum lag order is set, and the actual lag order is selected based on the Akaike Information Criterion. Then, the nodes are... Do historical residuals help explain the nodes? Perform a Granger causality test on the current residuals. If the significance value obtained from the test is less than the preset significance level, the node will be... Record as a node The upstream associated nodes. The set of upstream associated nodes can be updated with the feedback correction steps; when a node fails the significance test for multiple consecutive update cycles, the node is removed from the set of upstream associated nodes of the target node. The set of upstream associated nodes serves as the basis for calculating the burst correction amount.

[0045] In this embodiment, upstream associated nodes are read in real time. The residual deviation in the current window. When this deviation exceeds the node... Trigger the target node when the historical residual standard deviation is a preset multiple. Calculation of sudden correction amount. Target node. Future prediction step size The sudden correction amount is calculated according to the following formula: ; in, For the target node Future prediction step size Sudden correction amount; For nodes To node The impulse response function in the prediction step size The value of is estimated by calculating the relationship of vector autoregression of the historical residual sequence; upstream related nodes The residual deviation in the current window; This represents the summation over all upstream associated nodes that triggered the mutation.

[0046] Comprehensive forecast load Calculate using the following formula: ; in, For nodes Future prediction step size Comprehensive predicted load; For nodes Deterministic prediction benchmark; For nodes The sudden correction amount is 0 if no upstream associated node triggers the mutation.

[0047] The comprehensive predicted load output is used to calculate the subsequent node imbalance cost and migration causal benefits, and also to calculate the token generation rate of the migration token bucket.

[0048] In this embodiment, confidence boundaries can also be set for sudden corrections. These boundaries are based on the standard deviation of historical prediction errors and the prediction step size. The confidence interval half-width is generated by multiplying the confidence coefficient by the standard deviation and then by the square root of the prediction step size. The confidence interval half-width is output along with the comprehensive prediction load. If there are many upstream associated nodes, they can be sorted by the absolute value of the impulse response function, and a predetermined number of upstream associated nodes with the highest contribution ranking can be retained for the burst correction calculation. The above processing can control the online computational overhead and provide input for conservative bandwidth reservation in subsequent steps.

[0049] S103, based on the capacity deviation and the risk of the comprehensive predicted load exceeding the safety threshold, a node imbalance cost is formed. Candidate migration shards are generated for the imbalance source node. The load change after the candidate migration shards migrate out of the source node and into the target node is simulated. The migration cost is subtracted by the decrease in the overload probability of the source node to obtain the migration task list. In this embodiment, the system receives both the predicted load and data sharding characteristics. The node imbalance cost reflects both capacity imbalance and future performance risk. Capacity imbalance is used to prevent data shards from being concentrated on a few nodes in terms of storage capacity after scaling up or down. Future performance risk is used to prevent seemingly balanced capacity from creating hotspots due to concentrated business requests on a few nodes. This step determines whether migration is necessary at the node level and identifies migration targets at the data shard level. The selection of migration targets considers not only shard size but also the reduction in the overload probability of the source node after shards are moved out, the impact of the imbalance cost on the target node after migration into the target node, and migration transmission overhead.

[0050] Furthermore, the normalized deviation of the node's used capacity relative to the cluster's average used capacity is calculated to obtain the capacity deviation; The probability and magnitude of the comprehensive predicted load exceeding the node safety threshold within the prediction window are statistically analyzed to obtain the overload risk. The node imbalance cost is obtained by assigning weights to capacity deviation and overload risk according to the business service level constraints and then summing them up. When the node imbalance cost exceeds the preset triggering condition, the data shards managed by the triggering node are removed, and the data shards whose cold and hot attenuation is lower than the preset cooling condition and whose request load in the source node's total request load is lower than the preset contribution condition are removed, thus obtaining the candidate migration shards. For each candidate migration shard, read the shard request load, shard bandwidth contribution, and shard data volume. Subtract the shard request load and shard bandwidth contribution from the source node prediction sequence and add the shard request load and shard bandwidth contribution to the target node prediction sequence. By comparing the overload probability of the source node before and after the migration, the causal benefits can be obtained. Calculate the migration cost using the amount of data in the fragment and the unit transmission overhead, and select the candidate migration fragments whose causal benefits are greater than the migration cost and whose causal benefits are ranked highest to add them to the migration task list. If the node imbalance cost of the target node exceeds the preset backtracking condition after being added, the most recently added migration task is removed and a new candidate migration shard is selected.

[0051] In this embodiment, the node Capacity deviation at node The absolute value of the difference between the current used capacity and the cluster average used capacity is then divided by the cluster average used capacity. Overload risk is calculated based on the comprehensive predicted load within the prediction window. Crossing the node The average percentage of the safety threshold exceeded within the prediction window is calculated by taking the larger of the ratio of the exceeded portion to the safety threshold for each prediction step and zero, and then averaging the ratios over all prediction steps. (Node) The cost of node imbalance is calculated using the following formula: ; in, For nodes The cost of node imbalance; For nodes Capacity deviation; For nodes Overload risk; Weighting for capacity imbalance; This is the overload risk weight.

[0052] For latency-sensitive services It can be greater than The value; for expansion scenarios that prioritize capacity consolidation, It can be taken as not less than The value of . When the node imbalance cost exceeds the preset trigger condition, the node is identified as the imbalance source node and enters the candidate migration shard generation.

[0053] In this embodiment, the hot / cold attenuation, read / write load, and address concentration of each data shard managed by the unbalanced source node are read. If the hot / cold attenuation of a data shard is lower than a preset cooling condition and the proportion of the data shard's request load in the total request load of the source node is lower than a preset contribution condition, it indicates that the data shard contributes little to mitigating the source node's hotspot within the prediction window and can be removed from the candidate migration shards. If the number of candidate migration shards is lower than a preset minimum number of candidates, the contribution condition is lowered or candidate migration shards are added from shards with large capacity deviations. When outputting candidate migration shards, the shard identifier, shard data volume, source node identifier, candidate target node set, read / write load, bandwidth contribution, hot / cold attenuation, and address concentration are retained.

[0054] In this embodiment, for each candidate migration fragment and candidate target nodes Predicted sequences before migration and simulated predicted sequences after migration are constructed respectively. For the source node Candidate migration shards are subtracted from the predicted sequence of the source node. Contribution of request load and bandwidth; for the target node Add candidate migration shards to the target node prediction sequence Contributing request load and bandwidth contribution. Candidate migration shards. The causal benefit is calculated according to the following formula: ; in, For candidate migration partitioning causal benefits; For the source node before migration The probability of overload exceeding the safety threshold within the prediction window; To simulate the source node after migration The probability of overload exceeding the safety threshold within the prediction window.

[0055] If the same candidate migration shard can be migrated into multiple target nodes, then calculate the causal benefit for each combination of target nodes separately, and retain the combination with the highest causal benefit ranking where the target node imbalance cost has not exceeded the backtracking condition.

[0056] In this embodiment, the migration cost is the product of the data volume of the shard and the unit transmission cost. The unit transmission cost is weighted by network bandwidth usage, storage read / write overhead, and migration task metadata overhead. During selection, candidate migration shards with causal benefits not exceeding the migration cost are filtered out. From the remaining candidate migration shards, the data shard with the highest causal benefit is added to the migration task list. After addition, the node imbalance cost of the target node is recalculated. If the node imbalance cost of the target node exceeds a preset backtracking condition, the most recently added migration task is removed, the corresponding target node is removed from the target node set of the candidate migration shard, and a new candidate migration shard or candidate target node is selected. Each migration task in the migration task list includes at least a shard identifier, a source node identifier, a target node identifier, an estimated data volume, a causal benefit, a migration cost, and a task sort number. The migration task list is output in descending order of causal benefit to subsequent migration execution steps.

[0057] Furthermore, when the data volume of a candidate migration fragment exceeds the preset receiving capacity of the target node, the candidate migration fragment is divided into multiple candidate sub-fragments according to the continuous segment of the access address. Calculate the decrease in the overload probability of the source node and the increase in the imbalance cost of the target node after each candidate sub-shard migrates out of the source node and into the target node; Retain candidate sub-shards whose decrease in the overload probability of the source node is greater than the sum of the increase in the imbalance cost of the target node and the migration cost, and generate migration tasks based on the retained candidate sub-shards.

[0058] In this embodiment, the preset receiving capacity of the target node can be determined by the remaining capacity of the target node, the safety capacity level, and the current migration concurrency. When candidate migration fragments... When the amount of data exceeds the target node's preset receiving capacity, candidate migration fragments are divided according to the continuous segments of logical block addresses. Multiple candidate sub-shards are obtained. Each candidate sub-shard retains the original data shard identifier, address range, sub-shard data volume, and access load contribution. For each candidate sub-shard, the source node overload probability reduction calculation is re-executed, and the target node imbalance cost increment is calculated. If the source node overload probability reduction is greater than the sum of the target node imbalance cost increment and the migration cost, the candidate sub-shard is retained; otherwise, the candidate sub-shard is left at the source node or awaits recalculation in the next round. This in-depth processing is suitable for scenarios where a single data shard is too large, the target node's receiving capacity is limited, or the target node experiences short-term bandwidth pressure within the prediction window. When the retained candidate sub-shards enter the migration task list, the task sorting follows the causal benefit sorting, and within the same original data shard, they are arranged according to address range order or load contribution order.

[0059] In this embodiment, the address contiguous segmentation can also be replaced by segmentation by object number range or by storage block group, as long as the candidate sub-shards after segmentation can retain the calculation relationship of the source node overload probability decrease, the target node imbalance cost increment and migration cost.

[0060] S104: Execute migration by consuming migration token buckets in the causal benefit order of migration tasks. Calculate the token generation rate based on service bandwidth usage and the predicted confidence interval half-width. Suspend or resume migration input and output when service latency triggers suspension or resumption conditions. In this embodiment, the system receives the migration task list and the comprehensive predicted load, service bandwidth usage, and predicted confidence interval half-width output from the load prediction step. Each source node maintains a migration token bucket. The migration token bucket uses bytes or data blocks as the unit of token measurement. Migration tasks are retrieved from the migration task queue in causal benefit order, and migration input / output is submitted when the number of tokens meets the size of the data block to be sent. The token generation rate varies with the service bandwidth prediction waveform. When the predicted service bandwidth usage is close to the network's total bandwidth safety limit, the token generation rate decreases; when the predicted service bandwidth usage is below the predicted trough, the token generation rate increases. This execution method ensures that more migration data is sent during windows with low service load and is sent later during windows with high service load.

[0061] Furthermore, the total network bandwidth security limit of the source node, the service bandwidth usage within the prediction window, the half-width of the prediction confidence interval, and the token bucket conservatism coefficient are read. The token generation rate is obtained by deducting the service bandwidth usage from the total network bandwidth security limit and compensating for the half-width of the predicted confidence interval according to the token bucket conservatism coefficient. When the deducted token generation rate is lower than the preset minimum rate, the token generation rate is set to the pause rate and the unsent data blocks are kept in the migration task queue. During the migration execution, business input and output latency is collected according to time windows, and the deviation of business input and output latency from historical mean and historical standard deviation is calculated. When the deviation exceeds the preset suspension condition, stop retrieving new data blocks from the migration task queue and temporarily store the retrieved but unsent data blocks. When the deviation falls back to the preset recovery condition, data blocks are retrieved according to the causal benefit order of the migration task list and the storage order of the migration task queue, and migration input and output are submitted according to the number of tokens.

[0062] In this embodiment, the source node At any moment The token generation rate is calculated using the following formula: ; in, For the source node At any moment The token generation rate; For the source node The total network bandwidth security limit can be taken as the product of the node's physical network bandwidth and the security ratio, with the security ratio set to 0.9. The time obtained for the load forecasting step Service bandwidth usage; To predict the half-width of the confidence interval; The conservatism coefficient for the token bucket; This indicates taking the larger of 0 and the value within the parentheses.

[0063] The initial value of the token bucket conservatism coefficient can be 1, or a value between 0.5 and 2 depending on the service level. The calculated token generation rate represents the bandwidth allowed for migration of inputs and outputs within the current window. The migration token bucket accumulates tokens according to the token generation rate. If the data block size is BlockSize, a data block migration request is submitted when the number of tokens is not less than BlockSize, and the corresponding number of tokens is deducted.

[0064] In this embodiment, when the token generation rate is lower than a preset minimum rate, the token generation rate is set to a pause rate. The pause rate can be 0 or a very small rate used for metadata heartbeats. Unsent data blocks are retained in the migration task queue, and migration tasks and their causal reward order are preserved. If there are tokens in the token bucket but not enough to send a complete data block, the existing tokens are retained and the accumulation continues in the next window; if the migration task queue is empty, the token bucket stops accumulating migration tokens. For migration inputs and outputs that have been committed but not yet completed, the storage layer can be allowed to complete the current block write and subsequent block commits can be paused to maintain the consistency of the data block replication state.

[0065] In this embodiment, the business input / output latency is collected from the business request completion log or storage engine latency statistics. Let the current window's business input / output latency be... The historical average is The historical standard deviation is Then the deviation of the business input and output delay is taken as minus Difference after The preset suspension condition can be configured as a deviation greater than 3, and the preset recovery condition can be configured as a deviation less than 1.5 that is met consecutively for two windows. When the deviation exceeds the preset suspension condition, the migration execution module stops retrieving new data blocks from the migration task queue and places the retrieved but uncommitted migration data blocks into a temporary queue. The temporary queue records the task identifier, fragment identifier, data block offset, data block length, and verification status. When the deviation falls back to the preset recovery condition, the module reads the data blocks in the temporary queue, reads the subsequent data blocks in the migration task queue in causal benefit order according to the migration task list, and submits the migration input and output according to the number of tokens. If the data block verification in the temporary queue fails, the same address range in the source node is reread and resubmitted.

[0066] Furthermore, read the business input / output delay deviation and token under-issuance amount within the previous migration window, increase the adjustment step size of the token bucket conservative coefficient according to the delay deviation, and decrease the adjustment step size of the token bucket conservative coefficient according to the token under-issuance amount. The bandwidth reserve is obtained by multiplying the adjustment step size (increased by delay deviation) and the adjustment step size (decreased by token under-issuance) with the half-width of the predicted confidence interval. The token generation rate is calculated by subtracting the service bandwidth usage and bandwidth reservation from the total network bandwidth security limit.

[0067] In this embodiment, the token under-issuance amount represents the difference between the planned number of tokens issued and the actual number of tokens consumed in the previous migration window. If the service input / output latency deviation is large, the increase step of the token bucket conservatism coefficient is set to a larger value, allowing the next window to reserve more bandwidth for service input / output; if the token under-issuance amount is large and the service input / output latency deviation has not exceeded the suspension condition, the decrease step of the token bucket conservatism coefficient is set to a larger value, allowing the next window to release more migration bandwidth. The conservatism coefficient adjustment step can be calculated based on the weighted difference between the latency deviation and the under-issuance amount, and is limited to a preset range. The bandwidth reservation amount is the updated token bucket conservatism coefficient. and The product of these factors. The aforementioned in-depth processing incorporates both business latency fluctuations and migration progress into the token generation rate calculation, forming a closed loop with the token bucket conservative coefficient adjustment in the feedback correction step. In this embodiment, the token under-issue amount can also be replaced with the migration task queue waiting time or the proportion of remaining migration data. The replaced variable still participates in the conservative coefficient adjustment step size calculation and is output to the token generation rate calculation.

[0068] S105 re-estimates the prediction parameters based on the load error during the migration, adjusts the token bucket conservatism coefficient based on the bandwidth out-of-bounds status, and outputs the control parameters for the next migration round.

[0069] In this embodiment, during the migration execution process, the actual node load, actual bandwidth usage, migration completion time, and service input / output latency are continuously received. The actual node load comes from the node resource monitoring agent, the actual bandwidth usage comes from network interface statistics or storage layer input / output counts, and the migration completion time comes from the migration task status record. This step compares the actual observed values ​​with the predicted values ​​output by the load prediction step. If the prediction deviation continuously exceeds the preset reassessment condition, the trend cycle parameter or inter-node causality coefficient is reassessed. If the actual bandwidth usage continuously exceeds the boundary of the prediction confidence interval or continuously falls below the predicted trough, the token bucket conservatism coefficient is adjusted. The output control parameters include at least the reassessed trend cycle parameter, inter-node causality coefficient, impulse response function, token bucket conservatism coefficient, and the migration execution window for the next migration round.

[0070] Furthermore, the actual node load and the comprehensive predicted load are compared according to a continuous time window to obtain a prediction error sequence; When the prediction error sequence exceeds the preset re-estimation condition, the decomposition parameters of the trend component and the periodic component are re-estimated, or the causal coefficients between nodes are recalculated using the updated residual sequence. By comparing the actual bandwidth usage with the predicted confidence interval boundary, the token bucket conservatism coefficient is increased when the actual bandwidth usage exceeds the upper limit of the predicted confidence interval, and the token bucket conservatism coefficient is decreased when the actual bandwidth usage remains below the predicted trough and the migration task queue is not cleared, thus obtaining the control parameters for the next migration round.

[0071] In this embodiment, the node At any moment The prediction error is taken from the actual node load. Combined predicted load with the output of the load forecasting step The difference is as follows. To reduce misjudgments caused by a single abnormal window, a moving average or the mean of absolute errors can be taken from the prediction errors of multiple consecutive windows. Preset reassessment conditions can be configured as a preset proportion where the mean of absolute prediction errors within multiple consecutive windows exceeds the node's safety threshold, or a preset multiple of the standard deviation of historical prediction errors. The prediction error sequence is written to the prediction error cache, which is stored by node identifier and window time, and used to select reassessment objects.

[0072] In this embodiment, when the prediction error sequence mainly exhibits an overall shift, the locally weighted regression parameters of the trend component are re-estimated first; when the prediction error sequence shows a periodic shift with the intraday cycle or batch processing window, the Fourier harmonic parameters of the periodic component are re-estimated first; when the prediction error sequence appears in a concentrated manner after a sudden burst at an upstream associated node, the causal coefficients and impulse response functions between nodes are recalculated first using the updated residual sequence. The re-estimation process can be executed online or by the control node in a background window and then the updated parameters are issued. Before the re-estimated trend periodic parameters and causal coefficients between nodes overwrite the old parameters, a replay verification is performed using the most recent windows; if the error obtained from the replay verification is lower than the error corresponding to the old parameters, it is written to the parameter cache and output to the next migration round; if the replay verification fails, the old parameters are retained and the system waits for the next round of error triggering.

[0073] In this embodiment, the upper bound of the prediction confidence interval can be represented as the comprehensive prediction load. With confidence interval half-width The sum of these values, along with the predicted trough, can be represented as a local low-value window in the predicted service bandwidth usage sequence. When the actual bandwidth usage... When the token bucket exceeds the upper bound of the prediction confidence interval, it indicates that the current service bandwidth usage is higher than the conservative estimate, and the token bucket conservatism coefficient should be increased. When actual bandwidth usage remains below the predicted trough and the migration task queue is not emptied, it indicates that there is still usable migration bandwidth, thus reducing the token bucket conservatism factor. . The value can be adjusted between a preset maximum and minimum. The updated token bucket conservatism coefficient, together with the re-estimated prediction parameters, constitutes the control parameters for the next migration round. If actual bandwidth usage sampling fails, the current token bucket conservatism coefficient remains unchanged, and the migration token bucket is switched to a more conservative default rate until valid bandwidth sampling is obtained again for several consecutive windows.

[0074] In this embodiment, detectable operation records can also be configured. At the end of each migration round, the control node outputs an operation record, which includes at least a summary of the node's main feature sequence, a summary of the comprehensive predicted load, the node imbalance cost, the number of candidate migration shards, a summary of the migration task list, a token generation rate sequence, service latency deviation, a prediction error sequence, an updated value of the token bucket conservative coefficient, and control parameters for the next migration round. The operation record can be stored in the control node's local log, cluster metadata service, or monitoring database, and can be read by querying through the management interface or by periodically exporting. The operation record is used to review the input, judgment, intermediate results, and output status of each migration task during online scaling up and down, keeping the main process of the above steps unchanged.

[0075] The following description, using specific examples, elaborates on this embodiment.

[0076] Suppose a distributed block storage cluster includes node A, node B, and a newly added node C, with a statistical window of 10 minutes.

[0077] Read shards from the block device request log of node A. , and The request record was obtained. The current window has 12,000 read / write requests, 48 ​​GB of requested bytes, a tail latency of 18 ms, and an address concentration Gini coefficient of 0.72. The number of read / write requests is 3000, the requested byte size is 10GB, the tail latency is 7ms, and the address concentration Gini coefficient is 0.31. According to... The estimated hot / cold decay constant based on the most recent access time and historical access interval is 0.08. If the silence duration is 2 windows, the heat value is 0.852, and the half-life is 8.66 windows. After sorting the fragmentation features, node bandwidth usage, and input / output waiting ratio using mutual information, the following data are retained: number of read / write requests, request byte size, tail latency, address concentration, hot / cold decay, network transmit bandwidth usage, network receive bandwidth usage, and input / output waiting ratio, forming the node master feature sequence of node A.

[0078] Seasonal trend decomposition is performed on the total request load sequence of node A, yielding trend extrapolation values ​​of 9000, 9200, and 9400 for the current forecast window, and periodic extrapolation values ​​of 1500, 2200, and 1800 for the same period. The deterministic forecast baselines are 10500, 11400, and 11200 for the same period. Vector autoregression is established for the residual components of nodes B and A. Granger causality tests show that node B is an upstream related node of node A. The residual deviation of node B in the current window is 600. If the impulse response function from node B to node A in the first future window is 0.3, then the burst correction for node A in the first future window is 180, and the overall forecast load is 10680. If the impulse response function in the second future window is 0.2, then the burst correction is 120, and the overall forecast load is 11520. The overall forecast load and the half-width of the forecast confidence interval are output together to subsequent steps.

[0079] Node A currently has a used capacity of 82TB, and the average used capacity of the expanded cluster is 70TB, resulting in a capacity deviation of 0.171. The probability of Node A exceeding the safety threshold 10,000 times within the prediction window is 1.0, with an average exceedance of 0.113 over the window, resulting in an overload risk of 0.113. If the capacity imbalance weight is 0.4 and the overload risk weight is 0.6, the node imbalance cost is 0.1362. If the preset trigger condition is 0.08, then Node A is the source node of the imbalance. During candidate migration shard generation... Because the thermal attenuation is 0.852 and the data shard's request load accounts for a relatively high proportion of the source node's total request load, it is retained. Because the cold / hot attenuation is low and the request load of the data shard accounts for a small percentage of the total request load of the source node, it is excluded. The simulation will... After migrating from node A to node C, the overload probability of node A decreased from 1.0 to 0.25, and the causal benefit was 0.75. With a data volume of 48GB and a unit transfer cost of 0.003, the migration cost is 0.144. The causal benefit of 0.75, after deducting the migration cost of 0.144, remains positive. Therefore, [the data is considered positive]. Add to the migration task list. If node C only has a receiving capacity of 32GB, then... Divided into contiguous address segments and The decrease in the overload probability of the source node and the increase in the imbalance cost of the target node are calculated respectively. Candidate sub-shards that meet the conditions are retained and migration tasks are generated.

[0080] Node A has a physical network bandwidth of 10Gbps, and the total network bandwidth safety limit is set at 9Gbps. The predicted service bandwidth usage is 6.8Gbps, the predicted confidence interval half-width is 0.5Gbps, and the token bucket conservatism coefficient is set to 1, resulting in a token generation rate of 1.7Gbps. Migration tasks consume tokens and send data blocks in causal benefit order. When the current service input / output latency is 30ms, the historical average is 15ms, and the historical standard deviation is 4ms, the deviation is 3.75. If the preset suspension condition is a deviation greater than 3, then new data blocks are stopped being retrieved, and retrieved but unsent data blocks are placed in a temporary queue. Subsequently, when the service input / output latency drops back to 20ms, the deviation is 1.25, and it remains below the recovery condition for two consecutive windows, migration input / output continues to be submitted in the order of the temporary queue and the migration task queue. If the token under-issuance amount in the previous migration window is large and the service input / output latency deviation has not exceeded the suspension condition, then the adjustment step size of the token bucket conservatism coefficient is reduced, causing the token generation rate in the next window to increase accordingly.

[0081] After the migration is executed, the actual node load of node A is compared with the comprehensive predicted load. If the actual node load is 1300, 1100, and 1250 times higher than the comprehensive predicted load for three consecutive windows, respectively, and the mean absolute prediction error exceeds the preset re-estimation condition, then the decomposition parameters of the trend component and the periodic component are re-estimated. If the error concentration occurs after the burst at node B, then the causality coefficient and impulse response function from node B to node A are recalculated. If the actual bandwidth usage exceeds the upper bound of the prediction confidence interval, then the token bucket conservatism coefficient is adjusted from 1 to 1.1. If the actual bandwidth usage remains below the predicted trough and the migration task queue is not yet cleared, then the token bucket conservatism coefficient is adjusted from 1.1 to 1.0 or 0.9. Finally, the control parameters for the next migration round are output. These control parameters include the updated trend and periodic parameters, the inter-node causality coefficient, the token bucket conservatism coefficient, and the migration execution window, so that the next migration round continues to execute according to the above steps.

[0082] For the purpose of simplicity, the method steps disclosed in the above embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0083] like Figure 2 As shown, the present invention also provides a distributed storage online scaling data balancing system, comprising: The profile generation module 201 is configured to extract the read / write load, tail latency, address concentration, cold / hot attenuation, and node bandwidth usage of data shards from block device request logs and node resource monitoring data, sort and filter them according to the mutual information of each feature and bottleneck marker, and obtain the node main feature sequence. The load prediction module 202 is configured to split the total request load in the node main feature sequence into a trend component, a periodic component and a residual component, extrapolate the trend component and the periodic component, and calculate the burst correction amount using the impulse response function when the current residual deviation of the upstream associated node identified by the residual causality test exceeds the historical fluctuation threshold, so as to obtain the comprehensive predicted load. The migration planning module 203 is configured to generate node imbalance cost based on capacity deviation and the risk of the comprehensive predicted load exceeding the safety threshold, generate candidate migration shards for the imbalanced source node, simulate the load change after the candidate migration shards migrate out of the source node and into the target node, and obtain the migration task list by subtracting the migration cost from the decrease in the overload probability of the source node. The migration execution module 204 is configured to consume migration token buckets to perform migration in the causal benefit order of migration tasks, calculate the token generation rate based on service bandwidth usage and predicted confidence interval half-width, and suspend or resume migration input and output when service latency triggers suspension or resumption conditions. The feedback correction module 205 is configured to re-estimate the prediction parameters based on the load error during the migration, adjust the token bucket conservatism coefficient based on the bandwidth out-of-bounds status, and output the control parameters for the next migration round.

[0084] It is worth noting that although only some basic functional modules are disclosed in the embodiments of this invention, it does not mean that the composition of this system is limited to the above-mentioned basic functional modules. On the contrary, based on the above-mentioned basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with existing technology to form an infinite number of embodiments or technical solutions. That is to say, this system is open rather than closed. The fact that this embodiment only discloses a few basic functional modules should not be considered as the scope of protection of this invention being limited to the disclosed basic functional modules. At the same time, for the convenience of description, the above devices are described separately according to their functions as various units and modules. Of course, in implementing this invention, the functions of each unit and module can be implemented in one or more software and / or hardware.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data balancing method for online scaling up and down of distributed storage, characterized in that, include: Extract the read / write load, tail latency, address concentration, hot / cold attenuation, and node bandwidth usage of data shards from block device request logs and node resource monitoring data. Sort and filter according to the mutual information of each feature and bottleneck marker to obtain the node main feature sequence. The total request load in the node main feature sequence is divided into trend component, periodic component and residual component. The trend component and periodic component are extrapolated, and when the current residual deviation of the upstream associated node identified by the residual causality test exceeds the historical fluctuation threshold, the burst correction amount is calculated using the impulse response function to obtain the comprehensive predicted load. The node imbalance cost is formed based on the capacity deviation and the risk of the comprehensive predicted load exceeding the safety threshold. Candidate migration shards are generated for the imbalance source node. The load change after the candidate migration shards migrate out of the source node and into the target node is simulated. The migration task list is obtained by subtracting the migration cost from the decrease in the overload probability of the source node. Migration is performed by consuming migration token buckets in the causal benefit order of migration tasks. The token generation rate is calculated based on the service bandwidth usage and the predicted confidence interval half-width. Migration inputs and outputs are suspended or resumed when service latency triggers suspension or resumption conditions. The prediction parameters are re-estimated based on the load error during the migration, the token bucket conservatism coefficient is adjusted based on the bandwidth out-of-bounds status, and the control parameters for the next migration round are output.

2. The data balancing method for online scaling up and down of distributed storage according to claim 1, characterized in that, The node main feature sequence is obtained by sorting and filtering according to the mutual information between each feature and the bottleneck marker, which further includes: The number of fragment read / write requests, the number of requested bytes, and the tail latency are counted according to the time window. The Gini coefficient or information entropy of the requested logical block address is calculated as the address concentration. Estimate the hot and cold decay constants based on the most recent access time and the historical access interval, calculate the half-life of the heat of each segment, and form the hot and cold decay amount; After concatenating the sharding features with the node bandwidth usage and input / output ratios, mutual information calculation is performed on each feature and the bottleneck marker, and a preset number of features with the highest ranking are retained as the constituent fields of the node's main feature sequence.

3. The data balancing method for online scaling up and down of distributed storage according to claim 1, characterized in that, The total request load in the node's main feature sequence is split into trend components, periodic components, and residual components. Extrapolation of the trend components and periodic components further includes: Perform seasonal trend decomposition on the total request load of each node within a continuous time window to obtain trend components, periodic components, and residual components. Local weighted regression smoothing is performed on the trend component and linear extrapolation is performed along the prediction window; Fourier series fitting is performed on the periodic component and harmonic extrapolation is performed along the prediction window. The deterministic forecasting benchmark is obtained by adding the trend extrapolation value and the period extrapolation value.

4. The data balancing method for online scaling up and down of distributed storage according to claim 1, characterized in that, When the current residual deviation of the upstream associated node identified by the residual causality test exceeds the historical fluctuation threshold, the sudden correction amount is calculated using the impulse response function to obtain the comprehensive predicted load, which further includes: A vector autoregression relationship is established for the residual components of any two storage nodes, and the Granger causality test is used to screen upstream related nodes; When the current residual deviation of the upstream associated node exceeds the historical fluctuation threshold, the current residual deviation is multiplied by the impulse response function estimated from the historical residual sequence by the prediction step size to obtain the burst correction amount of the target node within the prediction step size. The burst correction amount is added to the deterministic prediction baseline to obtain the comprehensive prediction load.

5. The data balancing method for online scaling up and down of distributed storage according to claim 1, characterized in that, The node imbalance cost is calculated based on the risk of capacity deviation and overall predicted load exceeding the safety threshold. Candidate migration shards are generated for the imbalance source nodes, and further include: The normalized deviation of the node's used capacity relative to the cluster's average used capacity is used to obtain the capacity deviation; The probability and magnitude of the comprehensive predicted load exceeding the node safety threshold within the prediction window are statistically analyzed to obtain the overload risk. The node imbalance cost is obtained by assigning weights to capacity deviation and overload risk according to the business service level constraints and then summing them up. When the node imbalance cost exceeds the preset trigger condition, the data shards managed by the triggering node are removed, and the data shards whose cold and hot attenuation is lower than the preset cooling condition and whose request load in the source node's total request load is lower than the preset contribution condition are removed, thus obtaining the candidate migration shards.

6. The data balancing method for online scaling up and down of distributed storage according to claim 1, characterized in that, The load changes after candidate migration shards are migrated out of the source node and into the target node are simulated. The migration task list is obtained by subtracting the migration cost from the decrease in the source node overload probability, and further includes: For each candidate migration shard, read the shard request load, shard bandwidth contribution, and shard data volume. Subtract the shard request load and shard bandwidth contribution from the source node prediction sequence and add the shard request load and shard bandwidth contribution to the target node prediction sequence. By comparing the overload probability of the source node before and after the migration, the causal benefits can be obtained. Calculate the migration cost using the amount of data in the fragment and the unit transmission overhead, and select the candidate migration fragments whose causal benefits are greater than the migration cost and whose causal benefits are ranked highest to add them to the migration task list. If the node imbalance cost of the target node exceeds the preset backtracking condition after being added, the most recently added migration task is removed and a new candidate migration shard is selected.

7. The data balancing method for online scaling up and down of distributed storage according to claim 1, characterized in that, The token generation rate is calculated based on service bandwidth usage and the predicted confidence interval half-width, and further includes: Read the source node's total network bandwidth security limit, the service bandwidth usage within the prediction window, the prediction confidence interval half-width, and the token bucket conservatism coefficient; The token generation rate is obtained by deducting the service bandwidth usage from the total network bandwidth security limit and compensating for the half-width of the predicted confidence interval according to the token bucket conservatism coefficient. When the deducted token generation rate is lower than the preset minimum rate, the token generation rate is set to the pause rate and the unsent data blocks are kept in the migration task queue.

8. A data balancing method for online scaling up and down of distributed storage according to claim 1, characterized in that, Suspend or resume migration input / output when business delays trigger suspension or resumption conditions, further including: During the migration execution, business input and output latency is collected according to time windows, and the deviation of business input and output latency from historical mean and historical standard deviation is calculated. When the deviation exceeds the preset suspension condition, stop retrieving new data blocks from the migration task queue and temporarily store the retrieved but unsent data blocks. When the deviation falls back to the preset recovery condition, data blocks are retrieved according to the causal benefit order of the migration task list and the storage order of the migration task queue, and migration input and output are submitted according to the number of tokens.

9. A data balancing method for online scaling up and down of distributed storage according to claim 1, characterized in that, Re-estimating prediction parameters based on load errors during migration, adjusting the token bucket conservatism coefficient based on bandwidth out-of-bounds status, and further including: By comparing the actual node load with the comprehensive predicted load over a continuous time window, a prediction error sequence is obtained. When the prediction error sequence exceeds the preset re-estimation condition, the decomposition parameters of the trend component and the periodic component are re-estimated, or the causal coefficients between nodes are recalculated using the updated residual sequence. By comparing the actual bandwidth usage with the predicted confidence interval boundary, the token bucket conservatism coefficient is increased when the actual bandwidth usage exceeds the upper limit of the predicted confidence interval, and the token bucket conservatism coefficient is decreased when the actual bandwidth usage remains below the predicted trough and the migration task queue is not cleared, thus obtaining the control parameters for the next migration round.

10. A distributed storage online scaling data balancing system, characterized in that, include: The profile generation module is configured to extract the read / write load, tail latency, address concentration, hot / cold attenuation, and node bandwidth usage of data shards from block device request logs and node resource monitoring data, sort and filter them according to the mutual information between each feature and bottleneck marker, and obtain the node main feature sequence. The load prediction module is configured to split the total request load in the node main feature sequence into trend component, periodic component and residual component, extrapolate the trend component and periodic component, and calculate the burst correction amount using the impulse response function when the current residual deviation of the upstream associated node identified by the residual causality test exceeds the historical fluctuation threshold, so as to obtain the comprehensive predicted load. The migration planning module is configured to generate node imbalance costs based on capacity deviation and the risk of the comprehensive predicted load exceeding the safety threshold. It generates candidate migration shards for the imbalanced source node, simulates the load change after the candidate migration shards migrate out of the source node and into the target node, and obtains the migration task list by subtracting the migration cost from the decrease in the overload probability of the source node. The migration execution module is configured to consume migration token buckets to perform migrations in the causal benefit order of migration tasks, calculate the token generation rate based on service bandwidth usage and the half-width of the predicted confidence interval, and suspend or resume migration input and output when service latency triggers suspension or resumption conditions. The feedback correction module is configured to re-estimate prediction parameters based on load errors during migration, adjust the token bucket conservatism coefficient based on bandwidth out-of-bounds status, and output control parameters for the next migration round.

Images