Mass data multilayer hierarchical storage method fusing access frequency and life cycle weight

Abstract

The invention relates to the technical field of data storage, in particular to a mass data multilayer hierarchical storage method fusing access frequency and life cycle weight, which comprises the following steps: calculating the value life cycle weight of a data object and the physical life cycle health degree of a target storage medium in parallel, and outputting a composite weight factor through a coupling function; calculating a net benefit score value based on the composite weight factor and the data real-time access frequency in combination with a migration I / O loss estimated value of cross-layer migration; when the net benefit score value breaks through a preset migration threshold value, data migration is executed, and physical storage positions are dynamically distributed on the target storage layer according to the physical life cycle health degree. According to the method, the migration threshold is dynamically adjusted through the business key factors, the core data migration responsiveness is guaranteed, and meanwhile, high-cost migration behaviors are prevented from being triggered by temporary or edge data by mistake.

Description

Technical Field

[0001] The present invention relates to the field of data storage technology, and in particular to a multi-layer hierarchical storage method for massive data that integrates access frequency and lifecycle weight. Background Art

[0002] With the widespread deployment of cloud computing, edge computing, and big data platforms, the demand for massive data storage is becoming more diversified and dynamic. Modern data centers typically adopt a tiered storage architecture, dividing media of different performance levels (such as NVMe SSD, SASSSD, HDD, tapes, etc.) according to access latency and cost to achieve the optimal cost-effective configuration of storage resources.

[0003] Most existing multi-layer tiered storage methods rely on static rules or access frequency thresholds to classify data as hot or cold. Traditional solutions often use access frequency as the sole indicator, which cannot reflect the value fluctuations of data throughout its entire life cycle, resulting in cooling misjudgments or delayed migration of hot data. Most existing systems only judge the availability of the target storage layer based on performance indicators, ignoring the changes in the health status of the media itself, such as degradation characteristics such as the exhaustion of SSD erase and write cycles or the appearance of bad sectors on HDDs. Data migration across layers is often triggered only by access intensity, failing to systematically evaluate the resource loss and potential benefits brought about by migration, which can easily lead to resource waste or excessive migration. Migrated data is usually randomly located or balanced by capacity, lacking a refined allocation mechanism based on the physical status of the physical media, reducing the overall health utilization of the storage system. Summary of the Invention

[0004] The present invention provides a multi-layer hierarchical storage method for massive data that integrates access frequency and lifecycle weight. It also provides an intelligent multi-layer hierarchical storage method that integrates multidimensional data value, medium health status, and system real-time load perception. It can dynamically evaluate the rationality of data migration and achieve fine-grained control of physical resources, thereby improving the stability, performance, and life management capabilities of the storage system in large-scale data processing scenarios.

[0005] A multi-layer hierarchical storage method for massive data that integrates access frequency and lifecycle weights includes the following steps:

[0006] S1, dual life cycle weight synthesis:

[0007] Parallel calculation of the value lifecycle weight of the data object and the physical lifecycle health of the target storage medium, and output of the composite weight factor through the coupling function;

[0008] S2, migration cost perception score:

[0009] Calculating a net benefit score based on the composite weight factor and the real-time data access frequency, combined with an estimated migration I / O loss for cross-tier migration;

[0010] S3, dynamic level rebalancing:

[0011] When the net benefit score exceeds the preset migration threshold, data migration is performed and physical storage locations are dynamically allocated on the target storage tier based on the physical lifecycle health.

[0012] Optionally, the calculation of the value lifecycle weight in S1 includes real-time capture of the business event stream of the data object, matching the preset value stage transition rules according to the event type, and outputting a dynamic weight value W v ;

[0013] The calculation of the physical life cycle health in S1 includes monitoring the degradation characteristic indicators of the target storage medium and outputting the normalized health value W through the health mapping function. p , the target storage medium includes SSD media and HDD media:

[0014] For SSD media: Calculate the physical lifecycle health based on the ratio of used times to the maximum number of times it can be erased and written to.

[0015] For HDD media: The physical lifecycle health is calculated based on the bad sector ratio as the attenuation factor and the vibration amplitude coefficient.

[0016] Optionally, the value stage transition rules include:

[0017] If a critical activation event is detected, then

[0018] If the number of consecutive event-free cycles reaches the threshold, then

[0019] Among them, δ is the attenuation factor of the value weight, N represents the number of consecutive event-free cycles; key activation events include user access, editing, sharing, auditing, downloading, and model calling.

[0020] Optionally, the S1 further includes executing a dynamic coupling function, including: v With W p Input the piecewise coupling function and calculate the composite weight factor W c :

[0021]

[0022] Among them, W c is the composite weight factor, and k is the health gain coefficient.

[0023] Optionally, the S2 specifically includes:

[0024] S21, Calculation of expected benefits: Combine the composite weight factor CW with the real-time access frequency Fa Input the income calculation model to calculate the expected storage income value V exp :

[0025] V exp =CW×ln(F a +1)×T retain ; Where CW is the composite weight factor, F a is the data access frequency per unit time (average number of daily visits), T retain The expected retention period for the data.

[0026] S22, Migration loss estimation calculation: Calculate the I / O loss estimation C based on the cross-layer migration path. mig :

[0027] C mig =α×Size×(1+|L src -L dst |)+β×Lat diff ;

[0028] Among them, Size represents the size of the data object, L src ,L dst The performance levels of the source and target storage tiers are defined as follows:

[0029] High-speed layer: L=3, typical media is NVMe;

[0030] Medium-speed tier: L=2, typical media are SAS and SSD;

[0031] Low-speed layer: L=1, typical media are HDD and magnetic tape;

[0032] L src ,L dst The specific value of is determined by the level of the current data. If the data is currently stored in HDD, then L src =1.

[0033] α represents the migration data loss coefficient that is dynamically adjusted based on the system I / O load rate, β represents the delay loss coefficient that is dynamically adjusted based on the network bandwidth occupancy rate, and Lat diff Indicates the access latency difference between the target storage layer and the source storage layer. Use the system's built-in I / O detection instructions (read / writeping) to regularly perform small data block read and write operations on the target storage layer and the source storage layer:

[0034] Lat src =Avg.Latency fromsourcelayer ;

[0035] Lat dst =Avg.Latencyfromtargetlayer ;

[0036] The difference is calculated as: Lat diff =|Lat dst -Lat src |;

[0037] If the source storage layer is HDD (read and write latency 8ms) and the target is NVMeSSD (latency 0.3ms), then Lat diff =|0.3-8.0|=7.7ms.

[0038] I / O loss coefficient α is obtained as follows: α = α0 × (1 + ρ IO ); α0 is the basic migration loss coefficient (preset according to the device type, with a value of 0.05-0.2), ρ IO The current system I / O load rate (IOPS usage per unit time) is collected through the operating system or storage controller.

[0039] Delay loss coefficient β is obtained as follows: β = β0 × (1 + ρ BW ); β0 is the basic delay loss coefficient (range 0.005 to 0.05), ρ BW For the current network or bus bandwidth usage, use the network monitoring module or storage bus interface to calculate bandwidth usage:

[0040] S23, net benefit score generation: Comprehensively evaluate the cost-effectiveness of migration and generate a net benefit score N eff :

[0041] N eff =V exp -γ critical ×C mig ; Among them, N eff is the net benefit score, which is used as the final indicator to determine whether to migrate. critical It is the business criticality factor and its value is as follows:

[0042] Core business data: γ = 0.5;

[0043] Normal business data: γ = 1.0;

[0044] Temporary / low priority data: γ = 1.5.

[0045] Optionally, the preset migration threshold in S3 is calculated based on the real-time load status and physical lifecycle health distribution of the target storage layer, and is expressed as:

[0046]

[0047] Among them, T mig is the preset migration threshold, Load src Load is the I / O load rate of the source storage layer. dst is the I / O load rate of the target storage layer, H avg is the average physical life cycle health of the target layer medium, H min It is the minimum value of the physical life cycle health in the target layer.

[0048] Optionally, the S3 specifically includes:

[0049] S31, setting of preset migration threshold: dynamically preset the migration threshold T according to the migration direction mig :

[0050]

[0051] Among them, Load src Load is the I / O load rate of the source storage layer (normalized value range: [0,1]). dst is the I / O load rate of the target storage layer (normalized value range: [0,1]), H avg is the average physical life cycle health of the target layer medium (normalized value range: [0,1]), H min The lowest value of the physical life cycle health in the target layer. Here H is the W in S1 above. p The value is health.

[0052] S32, migration triggering and execution:

[0053] When the net benefit score N eff >T mig When , the following strategy is executed:

[0054] If the migration is an upgrade (e.g. HDD to SSD): the migration is started immediately, using the real-time migration channel.

[0055] If it is a down-tier migration (e.g. SSD to HDD): Write to the migration queue and press N eff Execute item by item in descending order;

[0056] If the number of access requests increases by ≥200% during the migration process and the migration traffic occupies ≥40% of the system bandwidth, the migration will be suspended and the system will enter the waiting state or differential synchronization mode.

[0057] S33, health-driven physical allocation:

[0058] (1) Health partitioning: The target storage medium is divided into health partitions according to its health H (i.e., W in S1 above). p Values) are divided into three categories:

[0059] High health area: H≥0.8;

[0060] Middle and healthy area: 0.5≤H<0.8;

[0061] Low health area: H<0.5;

[0062] (2) Data object allocation rule: Storage allocation is performed based on the composite weight factor CW of the data:

[0063]

[0064] (3) Intra-area allocation strategy:

[0065] High Health Zone: Health Weighted Poll: Health-weighted round-robin refers to the resource allocation strategy in the high-health zone. When allocating data objects, devices with higher health are given priority, and health is used as the weight for polling scheduling. i represents the probability that storage medium i is selected for writing (weighted polling probability), H i is the health of storage medium i, which is equivalent to the aforementioned W p , the value range is [0,1], ∑ j∈高健区 H j Indicates the sum of the health of all available media in the current high health area;

[0066] Medium / low health area: Remaining life expectancy distribution: W i Indicates the allocation weight of storage medium i, which is used to select the target medium in the medium / low health zone. Indicates the maximum square value of the remaining life of all devices in the current medium or low health zone, used for normalization, H i Here, it is directly used as the remaining life of storage medium i. Therefore, H is used to measure the current physical state of the storage medium and also participates in resource allocation and scheduling decisions as a remaining life indicator in the distribution of the remaining life ratio in the medium / low health area.

[0067] Beneficial effects of the present invention:

[0068] The present invention uses dual-channel parallel calculations of the value lifecycle weight of data objects and the physical health of the target storage medium, and adopts a three-segment coupling function to output a composite weight factor, effectively preventing high-value data from being allocated to inferior media or low-priority space. Compared with the traditional single-dimensional scoring strategy, the proposed "health-driven partitioning + value factor strong binding" mechanism improves the survival quality and reliability of key data in storage resources.

[0069] The present invention introduces logarithmic compression of access frequency and lifecycle weighting to form a benefit function, and integrates cross-layer I / O loss, latency difference and business criticality factors to calculate the net benefit score of migration. This mechanism can quantify the cost-effectiveness of migrating different types of data between different storage layers. In particular, it dynamically adjusts the migration threshold through business criticality factors to ensure the responsiveness of core data migration, while avoiding temporary or marginal data from mistakenly triggering high-cost migration behavior.

[0070] This invention proposes dynamically setting migration thresholds based on the target layer's real-time load and health. Combined with a three-level health partitioning strategy and a weighted allocation strategy within the partition, this ensures that migrated data is accurately placed in the most appropriate physical block. It also incorporates burst access detection and migration interruption recovery mechanisms to prevent migration from impacting business continuity. This approach effectively alleviates data accumulation in high-load areas, data sinking on low-health media, and performance jitter caused by mis-migrations, significantly improving data layout efficiency and global lifespan management capabilities in massive storage environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0072] Figure 1 A schematic diagram of calculating the net benefit score according to an embodiment of the present invention;

[0073] Figure 2 Schematic diagram of the health distribution mechanism according to an embodiment of the present invention. DETAILED DESCRIPTION

[0074] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Those skilled in the art may also implement some known technologies in other alternative ways. The accompanying drawings are only for describing the embodiments in more detail and are not intended to limit the present invention in any specific way.

[0075] like Figure 1-Figure 2 As shown, a multi-layer hierarchical storage method for massive data that integrates access frequency and lifecycle weight includes the following steps:

[0076] S1, dual life cycle weight synthesis:

[0077] Parallel calculation of the value lifecycle weight of the data object and the physical lifecycle health of the target storage medium, and output of the composite weight factor through the coupling function;

[0078] S2, migration cost perception score:

[0079] The net benefit score is calculated based on the composite weight factor and the real-time data access frequency, combined with the estimated migration I / O loss of cross-tier migration.

[0080] S3, dynamic level rebalancing:

[0081] When the net benefit score exceeds the preset migration threshold, data migration is performed and physical storage locations are dynamically allocated on the target storage tier based on the physical lifecycle health.

[0082] S1 specifically includes:

[0083] S11, parallel calculation of value lifecycle weight: real-time capture of the business event stream of data objects, matching the preset value stage transition rules according to the event type, and outputting the dynamic weight value W v .

[0084] The rules for value stage transition are as follows:

[0085] If a critical activation event is detected, then

[0086] If the number of consecutive event-free cycles reaches the threshold, then

[0087] Where, δ=1-0.1×N, and δ≥0.3, W v It represents the value lifecycle weight of the data object, ranging from [0.1, 1.0]. δ is the decay factor of the value weight, and N is the number of consecutive event-free cycles. Key activation events refer to important business operations that significantly improve the value of data usage, including user access, editing, sharing, auditing, downloading, model calling, etc. A consecutive event-free cycle refers to the period during which no business access or operation events related to the data object are detected within several consecutive monitoring cycles.

[0088] S12, parallel calculation of physical health weight: real-time monitoring of the degradation characteristic index of the target storage medium, and output of the normalized health value W based on the degradation characteristic index p The following health indicators are defined for different media types:

[0089] SSD media indicators:

[0090] HDD media indicators:

[0091] Among them, W p Indicates the health weight of the physical life cycle, ranging from [0,1], C max Indicates the maximum number of times the SSD is erased and written. used Indicates the number of erase and write times used, B errIndicates the number of HDD bad sectors detected, B total represents the total number of HDD sectors, ξ represents the HDD vibration amplitude coefficient, ranging from [0.8, 1.0], and represents the inhibitory factor of physical vibration on health. The number of bad sectors on the HDD refers to the number of data storage sectors on the disk that have been detected to be unable to read or write normally, reflecting the degree of physical damage to the disk.

[0092] S13, dynamic coupling function execution: W v With W p Input the piecewise coupling function and calculate the composite weight factor W c :

[0093]

[0094] Among them, W c is a composite weight factor that serves as the basis for subsequent migration decisions. k is the health gain coefficient, with a value range of [0.2, 0.5]. It is dynamically configured based on the data type. For critical business data (such as logs and image archives), a high value, k = 0.5, is used to emphasize media reliability. For temporary caches or low-value data, a low value, k = 0.2, is used to reduce the impact of health on migration.

[0095] The above-mentioned "value life cycle weight" and "physical health weight" adopt a dual-channel asynchronous parallel computing architecture. The two computing channels run in parallel driven by data streams from different sources, and use a multi-threaded or multi-process computing framework (task distribution based on message queues) to achieve computing logic decoupling and parallelism, avoiding the update efficiency of one party being affected by the waiting blockage of the other party.

[0096] S2 specifically includes:

[0097] S21, Calculation of expected benefits: Combine the composite weight factor CW with the real-time access frequency F a Input the income calculation model to calculate the expected storage income value V exp :

[0098] V exp =CW×ln(F a +1)×T retain ; Where CW is the composite weight factor, F a is the data access frequency per unit time (average number of daily visits), T retain The expected retention period for the data.

[0099] S22, Migration loss estimation calculation: Calculate the I / O loss estimation C based on the cross-layer migration path. mig :

[0100] C mig =α×Size×(1+|L src -Ldst |)+β×Lat diff ;

[0101] Among them, Size represents the size of the data object, L src ,L dst The performance levels of the source and target storage tiers are defined as follows:

[0102] High-speed layer: L=3, typical media is NVMe;

[0103] Medium-speed tier: L=2, typical media are SAS and SSD;

[0104] Low-speed layer: L=1, typical media are HDD and magnetic tape;

[0105] L src ,L dst The specific value of is determined by the level of the current data. If the data is currently stored in HDD, then L src =1.

[0106] α represents the migration data loss coefficient that is dynamically adjusted based on the system I / O load rate, β represents the delay loss coefficient that is dynamically adjusted based on the network bandwidth occupancy rate, and Lat diff Indicates the access latency difference between the target storage layer and the source storage layer. Use the system's built-in I / O detection instructions (read / writeping) to regularly perform small data block read and write operations on the target storage layer and the source storage layer:

[0107] Lat src =Avg.Latency fromsourcelayer ;

[0108] Lat dst =Avg.Latency fromtargetlayer ;

[0109] The difference is calculated as: Lat diff =|Lat dst -Lat src |;

[0110] If the source storage layer is HDD (read and write latency 8ms) and the target is NVMeSSD (latency 0.3ms), then Lat diff =|0.3-8.0|=7.7ms.

[0111] I / O loss coefficient α is obtained as follows: α = α0 × (1 + ρ IO ); α0 is the basic migration loss coefficient (preset according to the device type, with a value of 0.05-0.2), ρ IOThe current system I / O load rate (IOPS usage per unit time) is collected through the operating system or storage controller.

[0112] Delay loss coefficient β is obtained as follows: β = β0 × (1 + ρ BW ); β0 is the basic delay loss coefficient (range 0.005 to 0.05), ρ BW For the current network or bus bandwidth usage, use the network monitoring module or storage bus interface to calculate bandwidth usage:

[0113] S23, net benefit score generation: Comprehensively evaluate the cost-effectiveness of migration and generate a net benefit score N eff :

[0114] N eff =V exp -γ critical ×C mig ; Among them, N eff is the net benefit score, which is used as the final indicator to determine whether to migrate. critical It is the business criticality factor and its value is as follows:

[0115] Core business data: γ = 0.5;

[0116] Normal business data: γ = 1.0;

[0117] Temporary / low priority data: γ = 1.5.

[0118] S3 specifically includes:

[0119] S31, setting of preset migration threshold: dynamically preset the migration threshold T according to the migration direction mig :

[0120]

[0121] Among them, Load src Load is the I / O load rate of the source storage layer (normalized value range: [0,1]). dst is the I / O load rate of the target storage layer (normalized value range: [0,1]), H avg is the average physical life cycle health of the target layer medium (normalized value range: [0,1]), H min The lowest value of the physical life cycle health in the target layer. Here H is the W in S1 above. p The value is health.

[0122] S32, migration triggering and execution:

[0123] When the net benefit score N eff >T mig When , the following strategy is executed:

[0124] If the migration is an upgrade (e.g. HDD to SSD): the migration is started immediately, using the real-time migration channel.

[0125] If it is a down-tier migration (e.g. SSD to HDD): Write to the migration queue and press N eff Execute item by item in descending order;

[0126] If the number of access requests increases by ≥200% during the migration process and the migration traffic occupies ≥40% of the system bandwidth, the migration will be suspended and the system will enter the waiting state or differential synchronization mode.

[0127] S33, health-driven physical allocation:

[0128] (1) Health partitioning: The target storage medium is divided into health partitions according to its health H (i.e., W in S1 above). p Values) are divided into three categories:

[0129] High health area: H≥0.8;

[0130] Middle and healthy area: 0.5≤H<0.8;

[0131] Low health area: H<0.5;

[0132] (2) Data object allocation rule: Storage allocation is performed based on the composite weight factor CW of the data:

[0133]

[0134] (3) Intra-area allocation strategy:

[0135] High Health Zone: Health Weighted Poll: Health-weighted round-robin refers to the resource allocation strategy in the high-health zone. When allocating data objects, devices with higher health are given priority, and health is used as the weight for polling scheduling. i represents the probability that storage medium i is selected for writing (weighted polling probability), H i is the health of storage medium i, which is equivalent to the aforementioned W p , the value range is [0,1], ∑ j∈高健区 H j Indicates the sum of the health of all available media in the current high health area;

[0136] Medium / low health area: Remaining life expectancy distribution: W i Indicates the allocation weight of storage medium i, which is used to select the target medium in the medium / low health zone. Indicates the maximum square value of the remaining life of all devices in the current medium or low health zone, used for normalization, H i Here, it is directly used as the remaining life of storage medium i. Therefore, H is used to measure the current physical state of the storage medium and also participates in resource allocation and scheduling decisions as a remaining life indicator in the distribution of the remaining life ratio in the medium / low health area.

[0137] The present invention encompasses any alternatives, modifications, equivalents, and solutions that fall within the spirit and scope of the present invention. To provide a thorough understanding of the present invention, specific details are described in detail below in connection with the preferred embodiments of the present invention, but those skilled in the art will be able to fully understand the present invention without these detailed descriptions. Furthermore, to avoid unnecessary confusion regarding the essence of the present invention, well-known methods, processes, procedures, components, and circuits have not been described in detail.

[0138] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A multi-layer hierarchical storage method for massive data that integrates access frequency and lifecycle weight, characterized in that: The following steps are involved: S1, dual life cycle weight synthesis: Parallel calculation of the value lifecycle weight of the data object and the physical lifecycle health of the target storage medium, and output of the composite weight factor through the coupling function; S2, migration cost perception score: Calculating a net benefit score based on the composite weight factor and the real-time data access frequency, combined with an estimated migration I / O loss for cross-tier migration; S3, dynamic level rebalancing: When the net benefit score exceeds the preset migration threshold, data migration is performed and physical storage locations are dynamically allocated on the target storage tier based on the physical lifecycle health.

2. The method for multi-layer hierarchical storage of massive data integrating access frequency and lifecycle weight according to claim 1 is characterized in that: The calculation of the value life cycle weight in S1 includes real-time capture of the business event flow of the data object, matching the preset value stage transition rules according to the event type, and outputting the dynamic weight value W v ; The calculation of the physical life cycle health in S1 includes monitoring the degradation characteristic indicators of the target storage medium and outputting the normalized health value W through the health mapping function. p , the target storage medium includes SSD media and HDD media: For SSD media: Calculate the physical lifecycle health based on the ratio of used times to the maximum number of times it can be erased and written to. For HDD media: The physical lifecycle health is calculated based on the bad sector ratio as the attenuation factor and the vibration amplitude coefficient.

3. The method for multi-layered hierarchical storage of massive data integrating access frequency and lifecycle weight according to claim 2, characterized in that: The value stage transition rules include: If a critical activation event is detected, then If the number of consecutive event-free cycles reaches the threshold, then Among them, δ is the attenuation factor of the value weight, N represents the number of consecutive event-free cycles; key activation events include user access, editing, sharing, auditing, downloading, and model calling.

4. The method for multi-layered hierarchical storage of massive data integrating access frequency and lifecycle weight according to claim 3 is characterized in that: Said S1 also includes the execution of the dynamic coupling function, including W v With W p Input the piecewise coupling function and calculate the composite weight factor W c : Among them, W c is the composite weight factor, and k is the health gain coefficient.

5. The method for multi-layer hierarchical storage of massive data integrating access frequency and lifecycle weight according to claim 1 is characterized in that: The net benefit score is calculated as: N eff =V exp -γ critical ×C mig ; Among them, N eff is the net benefit score, which is used as the final indicator to determine whether to migrate. critical is the business criticality factor, which is determined by different business data. exp represents the expected storage benefit value, C mig Estimate the migration I / O loss.

6. The method for multi-layered hierarchical storage of massive data integrating access frequency and lifecycle weight according to claim 5, characterized in that: The expected storage return value is calculated as: V exp =CW×ln(F a +1)×T retain ; Where CW is the composite weight factor, F a is the data access frequency per unit time, T retain The expected retention period for the data.

7. The method for multi-layered hierarchical storage of massive data integrating access frequency and lifecycle weight according to claim 5, characterized in that: The migration I / O loss estimate is calculated as: C mig =α×Size×(1+|L src -L dst |)+β×Lat diff ; Among them, Size represents the size of the data object, L src ,L dst Lat is the performance level of the source storage layer and the target storage layer. diff represents the access latency difference between the target storage tier and the source storage tier, α represents the migration data loss coefficient that is dynamically adjusted based on the system I / O load rate, and β represents the latency loss coefficient that is dynamically adjusted based on the network bandwidth usage.

8. The method for multi-layer hierarchical storage of massive data integrating access frequency and lifecycle weight according to claim 7, characterized in that: The storage tier performance level is defined as: High-speed tier: Performance level 3, including NVMe storage media; Medium-speed tier: Performance level is 2, including storage media such as SAS and SSD; Low-speed tier: The performance level is 1, including storage media such as HDDs and tapes.

9. The method for multi-layer hierarchical storage of massive data integrating access frequency and lifecycle weight according to claim 1, characterized in that: The preset migration threshold in S3 is calculated based on the real-time load status and physical lifecycle health distribution of the target storage layer, and is expressed as: Among them, T mig is the preset migration threshold, Load src Load is the I / O load rate of the source storage layer. dst is the I / O load rate of the target storage layer, H avg is the average physical life cycle health of the target layer medium, H min It is the minimum value of physical life cycle health in the target layer.

10. The method for multi-layer hierarchical storage of massive data integrating access frequency and lifecycle weight according to claim 9, characterized in that: The S3 specifically includes: S31, when the net benefit score N eff >T mig When , the following strategy is executed: If it is a layer-up migration: Start the migration immediately and use the real-time migration channel; If it is a down-level migration: Write to the migration queue and press N eff Execute item by item in descending order; If the number of access requests increases suddenly during the migration process and the migration traffic occupies too much system bandwidth, the migration will be suspended and switched to waiting state or differential synchronization mode; S32, health-driven allocation: S321, health zoning: dividing the target storage medium into high health zone, medium health zone, and low health zone according to the physical life cycle health; S322, data object allocation rule: allocate storage to high-health area, medium-health area and low-health area according to the composite weight factor of the data; S323, Intra-area allocation strategy: High Health Zone: Executes health-weighted polling. In the resource allocation strategy within the high health zone, devices with higher health are prioritized when allocating data objects, and polling scheduling is performed using health as the weight. Medium / low health area: perform proportional allocation of remaining life.

Images