A storage decision method and apparatus

Abstract

The application discloses a storage decision method and device, relates to the technical field of storage, and determines a trend factor of a data unit according to the access times of the data unit, determines a seasonal factor of the data unit according to the historical same-period average access times and the historical overall average access times of the data unit, and determines a basic prediction value according to the trend factor and the seasonal factor, so that the active prediction of the future access trend of the data unit is realized. In addition, the method introduces a risk penalty factor, determines an expected access intensity adjusted by risk according to the basic prediction value and the risk penalty factor, and then selects the data unit according to the expected access intensity adjusted by risk, and migrates the selected data unit from a mechanical hard disk to a solid-state hard disk, so that the invalid migration caused by the flash hot spot can be effectively inhibited.

Description

Technical Field

[0001] This application relates to the field of storage technology, and in particular to a storage decision-making method and device. Background Technology

[0002] Currently, single-media storage struggles to balance performance and cost. To achieve this balance, hybrid storage architectures are increasingly adopted, combining solid-state drives (SSDs) and hard disk drives (HDDs) to form multi-layered storage systems. Layered control mechanisms are typically deployed on the storage system's control plane, where the system periodically collects data access behavior and controls data migration between different media layers based on a pre-defined algorithm model. However, these technologies struggle to maintain consistently optimal migration performance between different media layers. Therefore, addressing these technical shortcomings has become a pressing issue for those skilled in the art. Summary of the Invention

[0003] This application provides a storage decision-making method and apparatus to at least solve the problem of maintaining stable optimal migration performance in related technologies.

[0004] This application provides a storage decision method, including: The trend factor of the data unit is determined based on the number of times the data unit is accessed; the trend factor is used to measure the direction and magnitude of the change in the data access popularity of the data unit. The seasonality factor of the data unit is determined based on the historical average number of visits during the same period and the historical overall average number of visits; the seasonality factor is used to measure the degree of deviation of the access popularity of the data unit from the historical overall average level. The basic forecast value is determined based on the trend factor and the seasonal factor; the basic forecast value is a quantitative estimate of the expected future access intensity of the data unit. Determine the risk penalty factor for the data unit; the risk penalty factor is used to adjust the risk of the base forecast value. The effective popularity is determined based on the baseline forecast value and the risk penalty factor; the effective popularity is the expected visit intensity after risk adjustment. Data units are selected based on the effective heat level, and the selected data units are migrated from mechanical hard drives to solid-state drives.

[0005] This application also provides a storage decision-making device, including: The first determining module is used to determine the trend factor of the data unit based on the number of times the data unit is accessed; the trend factor is used to measure the direction and magnitude of the change in the data access popularity of the data unit. The second determining module is used to determine the seasonality factor of the data unit based on the historical average number of visits during the same period and the historical overall average number of visits; the seasonality factor is used to measure the degree of deviation of the access popularity of the data unit from the historical overall average level. The third determining module is used to determine the basic forecast value based on the trend factor and the seasonal factor; the basic forecast value is a quantitative estimate of the expected future access intensity of the data unit. The fourth determining module is used to determine the risk penalty factor of the data unit; the risk penalty factor is used to adjust the risk of the base prediction value. The fifth determining module is used to determine the effective popularity based on the basic predicted value and the risk penalty factor; the effective popularity is the expected access intensity after risk adjustment. The migration module is used to select data units based on the effective heat level and migrate the selected data units from the mechanical hard drive to the solid-state drive.

[0006] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing any of the above-described storage decision methods when executing the computer program.

[0007] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described storage decision methods.

[0008] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described storage decision methods.

[0009] The beneficial effects are as follows: This application identifies trend and seasonal factors for data units, and determines the basic forecast value based on these factors. This enables proactive prediction of future access trends for data units, solving the problem of migration being quickly abandoned due to historical lag, and maintaining a stable and optimal migration effect. Furthermore, this application introduces a risk penalty factor; based on the basic forecast value and the risk penalty factor, the expected access intensity after risk adjustment is determined, and data units are selected accordingly. The selected data units are then migrated from mechanical hard drives to solid-state drives, effectively suppressing invalid migrations caused by flash hotspots and improving the reliability of storage decisions. Attached Figure Description

[0010] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A schematic flowchart illustrating a storage decision method provided in an embodiment of this application; Figure 2 This application provides a schematic diagram of the overall process of storage decision-making. Figure 3 A schematic diagram of a storage decision device provided in an embodiment of this application; Figure 4 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0013] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0014] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0015] The embodiments of this application provide a storage decision method. The method is described in detail below in conjunction with the execution flow of the method.

[0016] refer to Figure 1 As shown, an embodiment of this application provides a storage decision method including: S101: Determine the trend factor of the data unit based on the number of times the data unit is accessed; the trend factor is used to measure the direction and magnitude of the change in the data access popularity of the data unit.

[0017] The purpose of this application embodiment is to determine which data should be placed on the solid-state drive. To this end, this application embodiment quantifies the future access intensity of data units based on trend factors and seasonality factors. Step S101 aims to determine the trend factor of the data unit. The trend factor of the data unit is used to measure the direction and magnitude of changes in the data access popularity of the data unit.

[0018] In some embodiments, determining the trend factor of a data unit based on the number of times the data unit is accessed includes: The first popularity of a data unit is determined based on the number of times the data unit is accessed at the first target's historical access time point; the first target's historical access time point is located in the first observation window. The second popularity of the data unit is determined based on the number of times the data unit is accessed at the second target's historical access time point; the second target's historical access time point is located in the second observation window; the second observation window is larger than the first observation window; The trend factor of the data unit is determined based on the first popularity and the second popularity.

[0019] Let the data unit be , abbreviated as data unit i.

[0020] Extent represents a contiguous block of data, where The size is . Too small a value will lead to excessively high management costs, while too large a value will affect scheduling accuracy. Therefore, The value range can be 1MB-64MB.

[0021] The initial heat of data unit i at time t, i.e., short-term heat. The calculation method is as follows: .

[0022] in, Indicates the time point of historical access Number of visits; This indicates the short-term observation window, i.e., the first observation window (e.g., the past 24 hours). This represents the short-term decay coefficient, which controls the rate at which past data decays. A larger value indicates that only recent visits are considered, while a smaller value indicates that historical visits over a longer period are considered. The short-term decay coefficient can range from [0.001, 0.1]. The value of the short-term decay coefficient can initially be preset empirically, and can be updated online through adaptive parameter adjustments.

[0023] The second heat of data unit i at time t, i.e., the long-term heat. The calculation method is as follows: .

[0024] in, Indicates the time point of historical access Number of visits; This indicates the long-term observation window, also known as the second observation window (e.g., the past 48 hours). This represents the long-term attenuation coefficient.

[0025] Overall short-term popularity With long-term popularity The trend factor for data unit i is determined as follows: .

[0026] Trend factors This is used to measure the direction and magnitude of changes in data access popularity at the current moment. Among them, This indicates that the data unit has been accessed more recently than the historical average, the business is about to enter a peak period, or hot data is forming; This indicates that the recent access to this data unit is consistent with its historical access levels, suggesting that this data block is a stable hotspot. This indicates that the recent access to this data unit is lower than the historical average access level, meaning that the peak business period for this data unit has passed and it is cooling down. To prevent the division by zero small constant, the order of magnitude is The denominator in the above formula achieves normalization, making the trend factor reflect the relative rate of change and better capturing the qualitative change of data units from cold to hot.

[0027] S102: Determine the seasonality factor of the data unit based on the historical average number of visits during the same period and the historical overall average number of visits; the seasonality factor is used to measure the degree of deviation of the access popularity of the data unit from the historical overall average level.

[0028] Step S102 aims to determine the seasonality factor of the data unit. The seasonality factor is a parameter that measures how much the access popularity of data unit i at the current time point t deviates from its historical overall average level.

[0029] The seasonality factor of data unit i is determined based on the historical average number of visits during the same period and the historical overall average number of visits. The calculation method is as follows: .

[0030] in, Indicates the current time in hours. This indicates the current day of the week; each data unit maintains a 24-hour log. A matrix of 7: .

[0031] This represents the historical average number of visits to data unit i in the h-th hour of week d.

[0032] In some embodiments, it also includes: If the seasonality factor is greater than the preset maximum value, then the seasonality factor is taken as the maximum value; If the seasonality factor is less than a preset minimum value, then the seasonality factor is taken as the minimum value.

[0033] when If the value is too large (e.g., >10) or too small (e.g., <0.1), truncation can be performed. That is, when the seasonality factor reaches its maximum value, Values ;when That is, when the seasonality factor reaches its minimum value, Values .

[0034] S103: Determine the basic forecast value based on the trend factor and the seasonal factor; the basic forecast value is a quantitative estimate of the expected future access intensity of the data unit.

[0035] The baseline forecast is a quantitative estimate of the expected access intensity of data unit i over a future period. The baseline forecast incorporates at least trend factors and seasonality factors.

[0036] In some embodiments, determining the base forecast value based on the trend factor and the seasonality factor includes: The base forecast value is determined based on the trend factor, the seasonality factor, the first popularity, and the trend enhancement weight; the trend enhancement weight is used to control the degree of influence of the trend factor.

[0037] In this embodiment, the basic forecast value integrates a comprehensive indicator that combines trend factors, seasonal factors, and short-term popularity information from three different time scales. The calculation method is as follows: .

[0038] in, This indicates the trend enhancement weight, used to control the degree of influence of the trend factor. This embodiment achieves prediction of access behavior by simultaneously considering the current state (short-term popularity), the direction of change (trend factor), and the cyclical pattern (seasonal factor).

[0039] S104: Determine the risk penalty factor for the data unit; the risk penalty factor is used to adjust the risk of the base forecast value.

[0040] The risk penalty factor is used to adjust the baseline forecast value for risk. The effective heat determined based on the baseline forecast value and the risk penalty factor is the risk-adjusted expected visit intensity.

[0041] In some embodiments, determining the risk penalty factor for the data unit includes: Determine the variance of the number of accesses to the data unit; The coefficient of variation is determined based on the variance of the number of visits; the coefficient of variation is used to measure the degree of fluctuation relative to the average level. Determine the prediction confidence level of the data unit; The risk penalty factor is determined based on the prediction confidence level and the coefficient of variation.

[0042] Determining the prediction confidence of the data unit includes: The average absolute percentage error is determined based on the historical prediction frequency, historical prediction value, and corresponding actual value of the data unit. The prediction confidence level is determined based on the average percentage error.

[0043] The variance of the number of visits is calculated as follows: .

[0044] in, Indicates the variance of the number of visits; Indicates the number of statistical periods; This represents the average number of visits, and can be used to... Perform an approximate substitution.

[0045] The coefficient of variation is calculated as follows: .

[0046] in, This represents the standard deviation of the number of visits. To prevent the division by zero small constant, the order of magnitude is . It is the coefficient of variation, used to measure the degree of fluctuation relative to the average level.

[0047] The prediction confidence level is calculated as follows: .

[0048] in, This represents the confidence decay coefficient. It represents the mean absolute percentage error. The calculation method is as follows: .

[0049] M represents the number of historical predictions in the statistics. This represents the m-th prediction value. This represents the corresponding actual value. The smaller the value, the more accurate the prediction; the larger the value, the less reliable the prediction. To prevent the division by zero small constant, the order of magnitude is .

[0050] The risk penalty factor is calculated as follows: .

[0051] in, It is a risk penalty coefficient, representing the overall level of risk aversion. The lower the value, the more stable the business risk.

[0052] S105: Determine the effective popularity based on the basic predicted value and the risk penalty factor; the effective popularity is the expected visit intensity after risk adjustment.

[0053] The effective heat can be calculated as follows: .

[0054] Effective heat It is the risk-adjusted expected access intensity, representing the level of access that can be reasonably expected considering the possibility of forecast errors.

[0055] S106: Select a data unit based on the effective heat level, and migrate the selected data unit from the mechanical hard disk to the solid-state drive.

[0056] In some embodiments, selecting a data unit based on the effective heat intensity includes: The expected performance gains from migrating the data unit from the hard disk drive to the solid-state drive are determined based on the effective heat. Determine the migration cost of moving the data unit from a mechanical hard drive to a solid-state drive; Select data units based on the expected performance gains and the migration costs.

[0057] This embodiment considers the expected performance gains and migration costs of migrating data units from mechanical hard drives to solid-state drives. Migrating data units can achieve a precise trade-off between performance improvement and resource consumption.

[0058] In some embodiments, determining the expected performance gains from migrating the data unit from a hard disk drive to a solid-state drive based on the effective heatsink includes: The expected performance gains from migrating the data unit from the mechanical hard drive to the solid-state drive are determined based on the effective heat, effective latency gains, and access type weights; the effective latency gains represent the expected latency savings that can be obtained after migrating the data unit from the mechanical hard drive to the solid-state drive; and the access type weights reflect the latency sensitivity of different access methods.

[0059] The expected performance gain from migrating data unit i from a mechanical hard drive to a solid-state drive (SSD) is one of the core bases for storage decisions in this application embodiment. The goal of the benefit model is to quantify the expected performance gain so that it can be compared with the migration cost, thereby making the optimal decision. In this embodiment, the expected performance gain depends on three core factors: 1. Future access intensity: the more data is accessed, the greater the gain; 2. Latency improvement: how much faster the SSD is than the mechanical hard drive determines the time saved per access; 3. Access type: different access types (synchronous / asynchronous / read-ahead) have different sensitivities to latency, and the expected performance gain should also be different. Therefore, the benefit model is as follows: .

[0060] in, This is expressed as effective heat. This represents the effective latency benefit, indicating the expected latency savings that can be achieved per access after migrating data units from mechanical hard drives to solid-state drives. Here, latency refers to the time from initiating a request to retrieve data to actually retrieving the data.

[0061] In some embodiments, the effective latency gain is calculated based on the average read latency of the hard disk drive, the average read latency of the solid-state drive, and the average queue depth of the solid-state drive.

[0062] In this embodiment The calculation method is as follows: .

[0063] in, This indicates the current average read latency of the HDD, which is due to seek and rotational latency caused by the mechanical hard drive. This indicates the current average read latency of the SSD; This indicates the current average queue depth of the SSD, which refers to the number of I / O requests waiting to be processed in the SSD controller, reflecting the current load pressure on the storage device; The rate at which queue depth diminishes the effect of latency gains is a parameter that reflects the system's sensitivity to load. The larger the queue depth, the more significant its impact; that is, for the same queue depth, the returns decay more rapidly. The smaller the value, the more tolerant the system is of high load, believing that even with queuing, SSDs can still provide significant benefits.

[0064] This represents the access type weight, reflecting the latency sensitivity of different I / O operations. Since different I / O operations have varying latency sensitivities, the business value derived from latency savings per access also differs. The access type weight reflects this difference. Synchronous read / write operations can typically be configured. Asynchronous read / write Read-ahead / write-ahead .

[0065] In some embodiments, determining the migration cost of migrating the data unit from a hard disk drive to a solid-state drive includes: Determine the bandwidth cost, write amplification cost, wear cost, risk cost, and performance impact cost of migrating the data unit from a mechanical hard disk to a solid-state drive; The migration cost is obtained by weighted summation of the bandwidth cost, write amplification cost, wear and tear cost, risk cost, and performance impact cost.

[0066] The bandwidth cost is calculated as follows: .

[0067] Bandwidth cost measures the system transmission resources consumed by migration operations. Among these, Indicates the size of the data unit; This indicates the maximum bandwidth available for migration per unit of time. This represents the bandwidth cost coefficient.

[0068] The calculation method for write amplification cost is as follows: .

[0069] in, This represents the write amplification factor, which is estimated based on the access mode. It can be set to 1.1 for sequential writes, 2.0 for mixed reads and writes, and 3.0 for random writes. The specific value can be dynamically adjusted according to the requirements. This indicates the additional write amplification resulting from migrating one unit of data. This represents the unit write cost (RMB / GB), which can be calculated based on the SSD model (total write life) and price.

[0070] Write amplification refers to the ratio of the actual physical data written to an SSD to the logical data written to the host. Due to SSD mechanisms such as garbage collection and wear leveling, the actual write volume is often greater than the logical write volume.

[0071] The wear and tear cost is calculated as follows: .

[0072] in, Indicates the total write lifespan of the SSD. This refers to the purchase cost of the SSD, while wear and tear costs focus on the basic lifespan of the SSD consumed by the writing process itself.

[0073] The risk cost is calculated as follows: .

[0074] in, This represents the risk cost coefficient, which can be preset based on experience and adjusted online. This represents an uncertainty indicator (coefficient of variation). Risk cost reflects the potential wasted migration due to the high volatility of data access. Even with high forecasts, highly volatile data may cool down rapidly after migration, making migration costs unrecoverable.

[0075] The calculation method for performance impact on cost is as follows: .

[0076] in, This represents the influence coefficient, which can be preset based on experience and adjusted online. This represents PCIe bandwidth utilization (here, PCIe refers to the physical link) and bandwidth cost. The focus is on the usage of bandwidth resources themselves; The concern is the actual interference of bandwidth usage on front-end services, because migration operations share the same physical link as front-end services. When the link is already highly busy, migration will intensify competition, leading to increased latency for front-end services.

[0077] The total cost, or migration cost, is calculated as follows: .

[0078] in, The weights of the cost items are summed to 1.

[0079] In some embodiments, selecting a data unit based on the expected performance gains and the migration costs includes: The net value score for migrating the data unit from the hard disk drive to the solid-state drive is determined based on the expected performance gains and the migration costs. Select data units based on the net value score.

[0080] The net value score measures the net migration value of a data unit. The net value score is calculated as follows: .

[0081] In some embodiments, selecting a data unit based on the net value score includes: Under capacity and bandwidth constraints, with the optimization objective of maximizing total migration benefits, data units are selected based on the unit capacity benefit; the unit capacity benefit is determined based on the net value score and the size of the data unit.

[0082] This embodiment models storage decision-making as a constrained optimization problem. Under capacity and bandwidth constraints, the optimization objective is to maximize the total migration benefit, which can ensure the optimal allocation of SSD capacity and bandwidth resources.

[0083] The capacity constraint is expressed as follows: .

[0084] The bandwidth constraint is expressed as follows: .

[0085] in, For decision variables, 1 indicates migration, and 0 indicates no migration; This indicates the maximum allowed migration time.

[0086] The optimization objective is expressed as follows: .

[0087] Using a greedy algorithm for approximation, according to (Revenue per unit capacity) Sort in descending order, and select sequentially until the capacity is full. ).

[0088] In some embodiments, it also includes: The target parameters corresponding to the target indicator are updated using the prediction error of the target indicator as the objective function; the target indicator includes any one or more of the following: first popularity, second popularity, basic predicted value, and effective delayed revenue.

[0089] This embodiment aims to update parameters adaptively online. The workload of a storage system is dynamically evolving: application patterns change, data access patterns shift, and hardware performance ages. If model parameters remain fixed, prediction accuracy and decision-making effectiveness will degrade over time. Therefore, this embodiment introduces an online adaptive parameter update mechanism that can continuously optimize parameters based on real-time feedback, enabling the self-evolution of the storage tiering strategy.

[0090] Using prediction error as the objective function, .

[0091] The F function can be represented as , , , Any one of the following. Based on the loss function, the gradient can be calculated (the gradient can be calculated using the chain rule, but since the prediction formula involves operations such as exponential decay, analytical differentiation or numerical approximation is preferred), to obtain the parameter update formula: .

[0092] in, This indicates an adjustable parameter. It can be... , , , Any one of the following (corresponding to the target indicator). The learning rate is expressed using time decay. ,in To learn the attenuation coefficient, This is the initial learning rate.

[0093] Parameters are adjusted based on Bayesian updates. Assume a reliability decay coefficient is set. The prior distribution is After observing the new data D, the posterior distribution is: .

[0094] Conjugate priors are often chosen to simplify computation. For example, if To control for exponential decay, we can assume that it follows a Gamma distribution.

[0095] According to the confidence formula, after observing the actual prediction error, we hope to adjust... This makes the confidence level more accurate. Define the likelihood function: .

[0096] in, This represents the actual observed accuracy or reliability of the predictions after comparing the actual prediction results with the actual access data. The specific calculation formula is as follows: .

[0097] in, This is an indicator function; it returns 1 if the condition is true, and 0 otherwise. The error tolerance threshold can be preset to 0.2 (meaning that a relative error of 20% is allowed).

[0098] Using conjugate priors, an analytical update formula for the posterior mean can be derived. For simplified engineering implementation, an exponential moving average approximation can be used. .

[0099] in, To update the weights (also known as the smoothing factor or learning rate); Obtained from the current error estimate . The target confidence level reflects the system's tolerance or expected value for prediction reliability. It is not a directly observable statistic, but a configurable target parameter. This represents the overall average of the average absolute percentage error of all data units within the current statistical window.

[0100] .

[0101] In some embodiments, it also includes: The weight parameters are dynamically adjusted based on the system status and business objectives. The weight parameters include any one or more of the following: cost weight, access type weight, maximum migration bandwidth per unit time, and maximum allowed migration duration.

[0102] This embodiment aims to perform rule-based weight adaptation. Rule-based weight adaptation refers to dynamically adjusting the weight parameters in the model through explicit rule logic based on the current system state (such as resource utilization, hardware health, and workload) and preset business objectives. Unlike methods based on gradient descent or Bayesian updates, rule adaptation does not rely on historical data or gradient calculations, but rather on historical experience or system design principles. By dynamically adjusting the weight parameters, the storage tiering strategy can achieve self-evolution.

[0103] Weight parameters can include cost weights I / O access type weight Maximum bandwidth for migration per unit time Maximum allowed migration time .

[0104] Cost weights include: (Bandwidth cost) (Write the scaling cost) (Wear and tear costs) (Risk Cost) (Performance impacts cost). The default weight (normal state) is: =0.20, =0.20, =0.20, =0.20, =0.20.

[0105] When SSD utilization exceeds 85%, increase the bandwidth cost weight α1 and wear cost weight α3 by 20% each (i.e., multiply by 1.2), while decreasing other weights proportionally, keeping the total sum at 1. When SSD utilization is less than 50%, revert to the default weights. When the remaining lifespan of the SSD is less than 20%, increase the wear cost weight α3 to 0.40, while increasing the write amplification cost weight α2 to 0.30, while decreasing the remaining weights proportionally, keeping the total sum at 1. This also applies to average queue depth. When the value is greater than 10, the performance impact cost weight α5 is increased to 0.35, and the remaining weights are reduced proportionally to suppress migration under high load.

[0106] By default: synchronous read / write Asynchronous read / write Read-ahead / write-ahead During peak business hours ( If the system throughput exceeds 80% of the threshold, increase the synchronous read / write weight to 1.2 and decrease the asynchronous and pre-read weights to 0.4 and 0.2 respectively, prioritizing critical business operations. This can be implemented during off-peak periods ( The synchronous I / O weight is reduced to 0.8, the asynchronous and read-ahead weights are increased to 0.6 and 0.4 respectively, and background task migration is allowed.

[0107] Default value: 30% of the total system bandwidth (i.e.) =0.3× ); can be used when PCIe bandwidth utilization >0.7, at this time =0.15× (Reduced by 50%); can be used When <0.3 and there are no peak business periods, at this time =0.4× (Increased to 40%, making full use of idle bandwidth); can be used during peak business periods and When >0.5, at this time =0.1× (Further reduce performance and protect foreground performance).

[0108] Maximum allowed migration time default value: =600 seconds, in this embodiment of the application, the migration decision is based on a fixed time period. (Default 10 minutes) Execution. Within each cycle, the system collects data and executes a complete prediction, evaluation, and optimization process. This indicates the maximum allowed migration duration within a single cycle, ensuring that migration operations do not span cycles and avoid conflicts with decisions in the next cycle. Peak business periods ( ), =300 seconds (to avoid prolonged resource occupation); off-peak period ( (and SSD utilization > 80%) =1200 seconds (too much migration was completed during idle time), urgent downgrade requirement (SSD utilization >95%). =1800 seconds (forced extension to free up space as soon as possible).

[0109] The above rules are evaluated and applied at the beginning of each decision cycle. If multiple rules are triggered simultaneously, they are handled according to the following priority: emergency (SSD > 95% or lifetime < 10%) > peak business period > bandwidth overload > others. The final parameter values ​​must be within a reasonable range (e.g., weight sum is 1, and the bandwidth limit does not exceed 50% of the total bandwidth).

[0110] The organization process of the storage decision method provided in the above embodiments can be referred to Figure 2 As shown, it mainly includes: data acquisition, parameter prediction, uncertainty modeling, revenue calculation, cost calculation, net value scoring, constraint optimization solution, migration execution, feedback collection, and model uncertainty parameter update.

[0111] In summary, this application identifies trend and seasonality factors for data units and determines base forecast values ​​based on these factors, enabling proactive prediction of future access trends for data units and resolving the issue of data becoming unavailable immediately after migration due to historical lag. Furthermore, this application introduces a risk penalty factor; based on the base forecast value and the risk penalty factor, the risk-adjusted expected access intensity is determined, and data units are selected accordingly. The selected data units are then migrated from mechanical hard drives to solid-state drives, effectively suppressing invalid migrations caused by flash hotspots and improving the reliability of storage decisions.

[0112] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0113] Embodiments of this application also provide a storage decision-making device, with reference to Figure 3 As shown, the device includes: The first determining module 10 is used to determine the trend factor of the data unit based on the number of times the data unit is accessed; the trend factor is used to measure the direction and magnitude of the change in the data access popularity of the data unit. The second determining module 20 is used to determine the seasonality factor of the data unit based on the historical average number of visits during the same period and the historical overall average number of visits; the seasonality factor is used to measure the degree of deviation of the access popularity of the data unit from the historical overall average level. The third determining module 30 is used to determine a basic forecast value based on the trend factor and the seasonal factor; the basic forecast value is a quantitative estimate of the expected future access intensity of the data unit. The fourth determining module 40 is used to determine the risk penalty factor of the data unit; the risk penalty factor is used to adjust the risk of the basic prediction value. The fifth determining module 50 is used to determine the effective popularity based on the basic predicted value and the risk penalty factor; the effective popularity is the expected access intensity after risk adjustment. The migration module 60 is used to select data units based on the effective heat level and migrate the selected data units from the mechanical hard disk to the solid-state drive.

[0114] In some embodiments, the migration module 60 includes: The first determining unit is used to determine the expected performance gains from migrating the data unit from the hard disk drive to the solid-state drive based on the effective heat. The second determining unit is used to determine the migration cost of migrating the data unit from the mechanical hard disk to the solid-state drive; The selection unit is used to select a data unit based on the expected performance gains and the migration costs.

[0115] In some embodiments, the first determining unit is configured to: The expected performance gains from migrating the data unit from the mechanical hard drive to the solid-state drive are determined based on the effective heat, effective latency gains, and access type weights; the effective latency gains represent the expected latency savings that can be obtained after migrating the data unit from the mechanical hard drive to the solid-state drive; and the access type weights reflect the latency sensitivity of different access methods.

[0116] In some embodiments, the effective latency gain is calculated based on the average read latency of the hard disk drive, the average read latency of the solid-state drive, and the average queue depth of the solid-state drive.

[0117] In some embodiments, the second determining unit is used for: Determine the bandwidth cost, write amplification cost, wear cost, risk cost, and performance impact cost of migrating the data unit from a mechanical hard disk to a solid-state drive; The migration cost is obtained by weighted summation of the bandwidth cost, write amplification cost, wear and tear cost, risk cost, and performance impact cost.

[0118] In some embodiments, the selection unit includes: The net value rating determination subunit is used to determine the net value rating for migrating the data unit from the hard disk drive to the solid-state drive based on the expected performance gains and the migration costs. Select sub-units for selecting data units based on the net value score.

[0119] In some embodiments, the selection subunit is used for: Under capacity and bandwidth constraints, with the optimization objective of maximizing total migration benefits, data units are selected based on the unit capacity benefit; the unit capacity benefit is determined based on the net value score and the size of the data unit.

[0120] In some embodiments, the first determining module 10 includes: The first popularity determination unit is used to determine the first popularity of the data unit based on the number of times the data unit is accessed at the first target's historical access time point; the first target's historical access time point is located in the first observation window. The second popularity determination unit is used to determine the second popularity of the data unit based on the number of times the data unit is accessed at the second target's historical access time point; the second target's historical access time point is located in the second observation window; the second observation window is larger than the first observation window; A trend factor determination unit is used to determine the trend factor of the data unit based on the first popularity and the second popularity.

[0121] In some embodiments, the third determining module 30 is used to: The base forecast value is determined based on the trend factor, the seasonality factor, the first popularity, and the trend enhancement weight; the trend enhancement weight is used to control the degree of influence of the trend factor.

[0122] In some embodiments, the fourth determining module 40 includes: Access frequency variance determination unit, used to determine the access frequency variance of the data unit; A coefficient of variation determination unit is used to determine the coefficient of variation based on the variance of the number of visits; the coefficient of variation is used to measure the degree of fluctuation relative to the average level. A prediction confidence determination unit is used to determine the prediction confidence of the data unit; A risk penalty factor determination unit is used to determine the risk penalty factor based on the prediction confidence level and the coefficient of variation.

[0123] In some embodiments, the prediction confidence determination unit is used for: The average absolute percentage error is determined based on the historical prediction frequency, historical prediction value, and corresponding actual value of the data unit. The prediction confidence level is determined based on the average percentage error.

[0124] In some embodiments, it also includes: The parameter update module is used to update the target parameters corresponding to the target indicator with the prediction error of the target indicator as the objective function; the target indicator includes any one or more of the following: first popularity, second popularity, basic predicted value, and effective delayed revenue.

[0125] In some embodiments, it also includes: The weight update module is used to dynamically adjust weight parameters according to system status and business objectives. The weight parameters include any one or more of the following: cost weight, access type weight, maximum migration bandwidth per unit time, and maximum allowed migration duration.

[0126] In some embodiments, it also includes: The first truncation processing module is used to take the seasonal factor as the maximum value if the seasonal factor is greater than the preset maximum value. The second truncation processing module is used to take the seasonal factor as the minimum value if the seasonal factor is less than the preset minimum value.

[0127] For a description of the features in the embodiment corresponding to the storage decision device, please refer to the relevant description of the embodiment corresponding to the storage decision method, which will not be repeated here.

[0128] Embodiments of this application also provide an electronic device, with reference to Figure 4 As shown, the electronic device includes a memory 1 and a processor 2. The memory 1 stores a computer program, and the processor 2 is configured to run the computer program to perform the steps in any of the above-described storage decision method embodiments.

[0129] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described storage decision method embodiments when it runs.

[0130] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0131] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described storage decision method embodiments.

[0132] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described storage decision method embodiments.

[0133] Any of the components, modules, units, parts, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Alternatively or additionally, any functionality described herein can be executed at least in part by one or more hardware logic components, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system-on-a-chip (SoC), a complex programmable logic device (CPLD), a microprocessor (MCU), etc. The terms "system," "computing device," or "apparatus" as used herein encompass various means, devices, and machines for processing data, including, for example, one or more programmable processors, computers, SoCs, or combinations thereof. The apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The aforementioned computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for a computing environment.

[0134] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0135] The storage decision-making method and device provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A storage decision-making method, characterized in that, include: The trend factor of the data unit is determined based on the number of times the data unit is accessed; The trend factor is used to measure the direction and magnitude of changes in the data access popularity of the data unit. The seasonality factor of the data unit is determined based on the historical average number of visits during the same period and the historical overall average number of visits; the seasonality factor is used to measure the degree of deviation of the access popularity of the data unit from the historical overall average level. The basic forecast value is determined based on the trend factor and the seasonal factor; the basic forecast value is a quantitative estimate of the expected future access intensity of the data unit. Determine the risk penalty factor for the data unit; The risk penalty factor is used to adjust the base forecast value for risk. The effective popularity is determined based on the baseline forecast value and the risk penalty factor; the effective popularity is the expected visit intensity after risk adjustment. Data units are selected based on the effective heat level, and the selected data units are migrated from mechanical hard drives to solid-state drives.

2. The storage decision method according to claim 1, characterized in that, The data unit selected based on the effective heat intensity includes: The expected performance gains from migrating the data unit from the hard disk drive to the solid-state drive are determined based on the effective heat. Determine the migration cost of moving the data unit from a mechanical hard drive to a solid-state drive; Select data units based on the expected performance gains and the migration costs.

3. The storage decision method according to claim 2, characterized in that, Based on the effective heat, the expected performance gains from migrating the data unit from the hard disk drive to the solid-state drive include: The expected performance gains from migrating the data unit from the mechanical hard drive to the solid-state drive are determined based on the effective heat, effective latency gains, and access type weights; the effective latency gains represent the expected latency savings that can be obtained after migrating the data unit from the mechanical hard drive to the solid-state drive; and the access type weights reflect the latency sensitivity of different access methods.

4. The storage decision method according to claim 3, characterized in that, The effective latency benefit is calculated based on the average read latency of mechanical hard drives, the average read latency of solid-state drives, and the average queue depth of solid-state drives.

5. The storage decision method according to claim 2, characterized in that, The migration cost of moving the data unit from a hard disk drive (HDD) to a solid-state drive (SSD) includes: Determine the bandwidth cost, write amplification cost, wear cost, risk cost, and performance impact cost of migrating the data unit from a mechanical hard disk to a solid-state drive; The migration cost is obtained by weighted summation of the bandwidth cost, write amplification cost, wear and tear cost, risk cost, and performance impact cost.

6. The storage decision method according to claim 2, characterized in that, The data unit selected based on the expected performance gains and the migration costs includes: The net value score for migrating the data unit from the hard disk drive to the solid-state drive is determined based on the expected performance gains and the migration costs. Select data units based on the net value score.

7. The storage decision method according to claim 6, characterized in that, The data units selected based on the net value score include: Under capacity and bandwidth constraints, with the optimization objective of maximizing total migration benefits, data units are selected based on the unit capacity benefit; the unit capacity benefit is determined based on the net value score and the size of the data unit.

8. The storage decision method according to claim 1, characterized in that, The trend factors for determining the data unit based on the number of times the data unit is accessed include: The first popularity of a data unit is determined based on the number of times the data unit is accessed at the first target's historical access time point; the first target's historical access time point is located in the first observation window. The second popularity of the data unit is determined based on the number of times the data unit is accessed at the second target's historical access time point; the second target's historical access time point is located in the second observation window; the second observation window is larger than the first observation window; The trend factor of the data unit is determined based on the first popularity and the second popularity.

9. The storage decision method according to claim 1, characterized in that, Determining the basic forecast value based on the trend factor and the seasonal factor includes: The base forecast value is determined based on the trend factor, the seasonality factor, the first popularity, and the trend enhancement weight; the trend enhancement weight is used to control the degree of influence of the trend factor.

10. The storage decision method according to claim 1, characterized in that, Determining the risk penalty factor for the data unit includes: Determine the variance of the number of accesses to the data unit; The coefficient of variation is determined based on the variance of the number of visits; the coefficient of variation is used to measure the degree of fluctuation relative to the average level. Determine the prediction confidence level of the data unit; The risk penalty factor is determined based on the prediction confidence level and the coefficient of variation.

11. The storage decision method according to claim 10, characterized in that, Determining the prediction confidence of the data unit includes: The average absolute percentage error is determined based on the historical prediction frequency, historical prediction value, and corresponding actual value of the data unit. The prediction confidence level is determined based on the mean absolute percentage error.

12. The storage decision method according to claim 1, characterized in that, Also includes: The target parameters corresponding to the target indicator are updated using the prediction error of the target indicator as the objective function; the target indicator includes any one or more of the following: first popularity, second popularity, basic predicted value, and effective delayed revenue.

13. The storage decision method according to claim 1, characterized in that, Also includes: The weight parameters are dynamically adjusted based on the system status and business objectives. The weight parameters include any one or more of the following: cost weight, access type weight, maximum migration bandwidth per unit time, and maximum allowed migration duration.

14. The storage decision method according to claim 1, characterized in that, Also includes: If the seasonality factor is greater than the preset maximum value, then the seasonality factor is taken as the maximum value; If the seasonality factor is less than a preset minimum value, then the seasonality factor is taken as the minimum value.

15. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for implementing the steps of the storage decision method as described in any one of claims 1 to 14 when executing the computer program.

Images