A hierarchical storage data migration method and device based on data block correlation

Abstract

This application discloses a hierarchical storage data migration method and apparatus based on data block correlation, comprising: creating and maintaining a data block association graph for each storage volume; sampling IO accesses of each storage volume in real time and generating IO event records in a sliding time window circular buffer corresponding to the storage volume; periodically analyzing IO events, identifying data block pairs that have been accessed together within a specified time window, calculating their association weights based on the difference in the number of accesses of the two data blocks in the data block pair within the sliding time window, and updating the association weights of the corresponding edges in the data block association graph according to the association weights; when a data block is detected to meet preset cross-layer migration conditions, querying the data block association graph to obtain all strongly correlated data blocks with the target data block and migrating them together. This application can achieve batch migration based on the access correlation between data blocks, improve migration efficiency, reduce access latency and avoid migration jitter, and improve overall performance.

Description

Technical Field

[0001] This application relates to the field of data migration technology, specifically to a hierarchical storage data migration method and apparatus based on data block correlation. Background Technology

[0002] As digital transformation deepens, the amount of data in enterprises and organizations is growing exponentially, and the value, access frequency, and performance requirements of data change dynamically throughout its lifecycle. On the one hand, businesses have extremely high latency requirements for accessing hot data (such as online transaction databases and real-time analytics), necessitating high-performance storage media to ensure this. On the other hand, massive amounts of warm and cold data (such as backups, archives, and historical records) occupying expensive high-performance storage resources for extended periods will lead to significant cost waste.

[0003] Against this backdrop, data tiered storage technology has emerged. Its core concept is to automatically place data on storage media or storage tiers of different costs and performance levels based on data value, access frequency, and performance requirements, thereby optimizing the total cost of storage (TCO) while meeting business performance needs. This is a typical cost-performance balance strategy of "storing the right data on the right medium at the right time."

[0004] Currently, common data tiering storage technologies include: automatic tiering based on RAID groups or storage pools and cache-accelerated tiering for specific workloads.

[0005] 1. Automatic tiering based on RAID groups or storage pools

[0006] With the increasing intelligence of storage arrays, automatic tiering technology has emerged within a single storage system. A typical implementation involves dividing disks into tiers based on their performance. Within a single storage device, solid-state drives (SSDs) form a "high-performance tier" (Tier 3 / 2), while large-capacity hard disk drives (such as NL-SAS) or high-density hard drives form a "capacity tier" (Tier 1 / 0). Data migration is then performed according to a specific migration strategy. The system monitors the access frequency (I / O frequency) of data blocks and, according to a preset strategy (such as the Least Recently Used (LRU) algorithm), automatically migrates "hot data" blocks to higher tiers and demotes "cold data" blocks to lower tiers in the background.

[0007] 2. Load-specific cache acceleration tiering

[0008] Load-specific caching acceleration tiering does not perform a full data migration; instead, it caches the hottest subset of data on the fastest media (such as SSDs or SCM storage-class memory). This is a special form of tiering that focuses on read performance acceleration, essentially a caching technique. It only involves continuous data migration between cold devices.

[0009] In summary, traditional migration strategies primarily rely on data access frequency, recent access time, or access pattern prediction. When a data block is accessed frequently, the system migrates it from a cold layer to a hot layer; conversely, it degrades it from a hot layer to a cold layer. However, this strategy, based solely on individual access history, has significant drawbacks:

[0010] 1. Insufficient foresight: It can only react to accesses that have already occurred, and cannot prevent potential delays in related accesses in the future, ignoring the access relationships between data blocks.

[0011] 2. Insufficient Local Amplification: Data access in computer applications (such as database queries, scientific computing, and video editing) often has strong spatial, temporal, or semantic relevance. After accessing data block A, it is highly likely that related data blocks B and C will be accessed within a short period of time. If B and C are still in the cold stack, it will cause a surge in latency for subsequent requests, resulting in "migration jitter".

[0012] 3. Low migration efficiency: Frequent fine-grained migration of individual data blocks may generate a large amount of metadata overhead and migration fragmentation, failing to leverage data correlation to achieve efficient batch migration. Summary of the Invention

[0013] To address this issue, this application provides a hierarchical storage data migration method and apparatus based on data block correlation, thereby solving the problems of delayed migration decisions, migration jitter, and low migration efficiency caused by the inability to predict the access correlation between data blocks in the prior art.

[0014] To achieve the above objectives, this application provides the following technical solution:

[0015] In a first aspect, a hierarchical storage data migration method based on data block correlation is provided, which pre-divides the device into a cold device layer, a hot device layer, and several storage volumes, further divides the cold device layer and the hot device layer into several data blocks, and creates a sliding time window circular buffer for each storage volume; the method includes:

[0016] Step 1: Create and maintain a data block association graph for each storage volume; the data block association graph uses data blocks as nodes and access correlation between data blocks as edges, and each edge has a dynamically updated association weight, which is used to quantify the association strength between two data blocks based on the difference in the number of accesses within a time window;

[0017] Step 2: Use an IO thread to sample the IO access of each storage volume in real time and generate IO events. Record the IO events in the sliding time window circular buffer corresponding to the storage volume.

[0018] Step 3: Use the correlation analysis thread to periodically analyze the IO events in the sliding time window circular buffer, identify the data block pairs that have been accessed together within the specified time window, and calculate their correlation weight based on the difference in the number of accesses of the two data blocks in the data block pair within the sliding time window.

[0019] Step 4: Update the association weights of the corresponding edges in the data block association graph according to the calculated association weights;

[0020] Step 5: When any data block is detected to meet the preset cross-layer migration conditions due to its access popularity or coldness, the data block is determined as the target data block, and the target storage layer for this migration is determined.

[0021] Step 6: Query the data block association graph to obtain all strongly associated data blocks whose association weight with the target data block is greater than a preset association threshold;

[0022] Step 7: Migrate the target data block and all its corresponding strongly associated data blocks to the target storage layer.

[0023] Preferably, in step 1, creating and maintaining a data block association graph for each storage volume specifically includes: if a persistent data block association graph already exists for the storage volume, loading the data block association graph from the disk metadata area; otherwise, creating an empty data block association graph.

[0024] Preferably, the data block association graph is an undirected graph, and the metadata stored in the nodes of the undirected graph includes: data block ID, last access timestamp, and current storage layer; the metadata stored in the edges of the undirected graph includes: source node ID, target node ID, and association weight.

[0025] Preferably, the data block association graph is stored using an array and an adjacency matrix. The array is used to store node information, and the array index is the order of the data blocks in the storage volume. The adjacency matrix is ​​used to store edge information, and the value of each matrix element is the association weight of the corresponding edge.

[0026] Preferably, the sliding time window circular buffer is an IO event queue allocated based on the IO queue depth. Each IO event record includes a data block ID, access time, and access count. When multiple consecutive IO events correspond to the same data block ID, only the last access time and access count of the data block ID are updated, without inserting a new event record.

[0027] Preferably, the window length of the sliding time window circular buffer is adjustable between 1 second and 60 seconds.

[0028] Preferably, in step 3, the correlation weight is calculated based on the difference in the number of accesses of the two data blocks in the data block pair within the sliding time window. The specific calculation formula is as follows:

[0029] W = (count1 + count2) / (|count1 - count2| + 1)

[0030] Here, count1 and count2 represent the number of times the two data blocks are accessed within the sliding time window, and count1 ≥ count2; when the number of accesses of either data block within the time window is 0 or the two data blocks do not currently belong to the same storage layer, the association weight is set to 1.

[0031] Preferably, in step 5, the preset cross-layer migration conditions include:

[0032] When the access frequency of a data block exceeds the first threshold within a preset time period, it is determined to be hot data, triggering a migration from the cold device layer to the hot device layer.

[0033] When the access frequency of a data block is lower than the second threshold within a preset time period, it is determined to be cold data, triggering a migration from the hot device layer to the cold device layer.

[0034] Preferably, in step 7, a separate migration thread is used when migrating the target data block and all its corresponding strongly associated data blocks to the target storage layer.

[0035] Secondly, a hierarchical storage data migration device based on data block correlation includes:

[0036] The association graph module is used to create and maintain a data block association graph for each storage volume. The data block association graph uses data blocks as nodes and access correlations between data blocks as edges. Each edge has a dynamically updated association weight, which is used to quantify the association strength between two data blocks based on the difference in the number of accesses within a time window.

[0037] The IO sampling module is used to sample the IO access of each storage volume in real time using the IO thread, generate IO events, and record the IO events in the sliding time window circular buffer corresponding to the storage volume;

[0038] The correlation analysis module is used to periodically analyze the IO events in the sliding time window circular buffer using the correlation analysis thread, identify data block pairs that have been accessed together within a specified time window, and calculate their correlation weight based on the difference in the number of accesses of the two data blocks in the data block pair within the sliding time window.

[0039] The weight calculation module is used to update the association weight of the corresponding edge in the data block association graph according to the calculated association weight;

[0040] The migration monitoring module is used to determine the data block as the target data block and the target storage layer for this migration when it detects that any data block meets the preset cross-layer migration conditions due to its access popularity or coldness.

[0041] The associated block query module is used to query the data block association graph and obtain all strongly associated data blocks whose association weight with the target data block is greater than a preset association threshold.

[0042] The migration module is used to migrate the target data block and all its corresponding strongly associated data blocks to the target storage layer.

[0043] Compared with the prior art, this application has at least the following beneficial effects:

[0044] This application provides a hierarchical storage data migration method based on data block correlation, including: creating and maintaining a data block association graph for each storage volume; using an IO thread to sample IO accesses of each storage volume in real time and generating IO event records in a sliding time window circular buffer corresponding to the storage volume; periodically analyzing IO events in the sliding time window circular buffer, identifying data block pairs that have been accessed together within a specified time window, and calculating their association weights based on the difference in access frequency between the two data blocks in the data block pair within the sliding time window, and updating the association weights of the corresponding edges in the data block association graph based on the association weights; when any data block is detected to meet the preset cross-layer migration conditions due to its access popularity or coldness, determining the data block as the target data block and determining the target storage layer for this migration; querying the data block association graph to obtain all strongly correlated data blocks with an association weight greater than a preset association threshold with the target data block; and migrating the target data block and all its corresponding strongly correlated data blocks to the target storage layer. This application enables proactive batch migration based on the access correlation between data blocks, improving migration efficiency, effectively reducing access latency, and avoiding migration jitter, thereby significantly improving the overall performance of the tiered storage system. Attached Figure Description

[0045] To more intuitively illustrate the prior art and this application, exemplary drawings are provided below. It should be understood that the specific shapes and structures shown in the drawings should not generally be regarded as limiting conditions for implementing this application; for example, based on the technical concept disclosed in this application and the exemplary drawings, those skilled in the art are able to easily make conventional adjustments or further optimizations to the addition / reduction / classification, specific shapes, positional relationships, connection methods, size ratios, etc. of certain units (components).

[0046] Figure 1 This is a schematic diagram of the cold equipment layer, hot equipment layer, and storage volume structure provided in Embodiment 1 of this application;

[0047] Figure 2 A flowchart of a hierarchical storage data migration method based on data block correlation is provided in Embodiment 1 of this application;

[0048] Figure 3 This is a schematic diagram of a hierarchical storage data migration method based on data block correlation provided in Embodiment 1 of this application;

[0049] Figure 4 This is a schematic diagram of the data block association graph structure provided in Embodiment 1 of this application;

[0050] Figure 5 This is a schematic diagram of the sliding time window circular buffer structure provided in Embodiment 1 of this application. Detailed Implementation

[0051] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] In the description of this application: unless otherwise stated, "a plurality of" means two or more. The terms "first," "second," "third," etc., in this application are intended to distinguish the objects referred to and do not have any special meaning in terms of technical connotation (e.g., they should not be construed as an emphasis on importance or order). Expressions such as "including," "comprising," and "having" also mean "not limited to" (certain units, components, materials, steps, etc.).

[0053] The terms used in this application, such as "upper," "lower," "left," "right," and "middle," are generally used to indicate the general relative positional relationship for the purpose of intuitive understanding by referring to the accompanying drawings, and are not absolute limitations on the positional relationship in the actual product.

[0054] Example 1

[0055] This embodiment provides a hierarchical storage data migration method based on data block correlation. This method can deeply understand the inherent relationship between data and make a forward-looking, batch migration strategy to optimize overall I / O performance.

[0056] Its core idea is that when migrating hot and cold data, it not only migrates data based on its own hot or cold status, but also identifies other data blocks that are strongly related to the current hot (or cold) data. Even if these data blocks are not currently hot or cold data, they are also migrated together, thereby enabling prediction and preheating of future accesses, significantly reducing access latency, or timely migration to cold devices to provide more performance space and improve the overall performance of the storage system.

[0057] This embodiment provides a hierarchical storage data migration method based on data block correlation. It requires initializing the correlation graph, i.e., pre-dividing the device into a cold device layer, a hot device layer, and several storage volumes. The cold device layer and the hot device layer are each further divided into several data blocks, and a sliding time window circular buffer is created for each storage volume. Each storage volume consists of several data blocks, such as... Figure 1 As shown.

[0058] Please see Figure 2 and Figure 3 This embodiment provides a hierarchical storage data migration method based on data block correlation, which includes:

[0059] S1: Create and maintain a data block association graph for each storage volume; the data block association graph uses data blocks as nodes and access correlations between data blocks as edges. Each edge has a dynamically updated association weight, which is used to quantify the association strength between two data blocks based on the difference in the number of accesses within a time window.

[0060] Specifically, this step creates and maintains a data block association graph for each storage volume, including: if a persistent data block association graph already exists for the storage volume, loading the data block association graph from the disk metadata area; otherwise, creating an empty data block association graph.

[0061] A data block association graph is used to represent the correlations between data blocks. The core principle of correlation is that if two or more data blocks are accessed consecutively or nearly consecutively within a time window and belong to the same layer, then there is an "access correlation" between them. They typically have strong spatial, temporal, or semantic correlations. This correlation strength is quantified by "weights." If a data block is migrated because its popularity reaches a threshold, its related data blocks need to be migrated even if their popularity / coldness does not reach the threshold.

[0062] like Figure 4 As shown, the data block association graph is represented by an undirected graph G = (V, E), which consists of a set of vertices (i.e., nodes) and edges:

[0063] Node: A storage volume consists of several data blocks, each of which is 256MB or a multiple of 256MB in size. Each data block is called a node. These data blocks are related and contain the following metadata: data block ID, last access timestamp, and current storage tier.

[0064] Edge: An undirected weighted edge representing the access correlation between data blocks. If data block A is accessed, and data block B is also accessed within a preset time window T, then an undirected edge (A - B) is established between A and B. It includes the following metadata: source node ID, target node ID, and association weight W.

[0065] This embodiment uses an array to store node information. The array index represents the order of the data block in the storage volume, and the array elements are the node information. An adjacency matrix g[u][v] is used to store graph information, where u and v are indices in the node array and also the block IDs of the data blocks. The values ​​of the elements are weights W, which are dynamically updated based on the differences in the number of visits within a specified window period.

[0066] Within a time window, if the number of visits to two data blocks is close and they belong to the same layer, the weight value W is higher. If one of the data blocks has 0 visits or does not belong to the same layer, the weight value is 1. Otherwise, the formula W = (count1 + count2) / (count1 - count2 + 1) is used, where count1 is the number of visits to the node with the larger number of visits, count2 is the number of visits to the node with the smaller number of visits, and 1 is added to prevent the denominator from being zero. In other words, the closer the correlation and the number of visits of two data blocks are, the stronger the correlation.

[0067] It should be noted that this embodiment establishes associations between data blocks in a disk or storage volume, rather than between independent data blocks; an undirected association graph is used to maintain the association relationships between data blocks, and the association weights are determined based on the similarity of access frequencies within a time window to determine the association strength.

[0068] S2: Use the IO thread to sample the IO access of each storage volume in real time and generate IO events, and record the IO events in the sliding time window circular buffer corresponding to the storage volume;

[0069] Specifically, this embodiment creates a fixed-size sliding time window (1-60 seconds) circular buffer for each storage volume. The window size is adjustable (i.e., the window length is adjustable between 1 second and 60 seconds). The most recently occurring I / O access events are recorded in chronological order. Each event record contains: {data block ID, access time, access count}.

[0070] When processing an I / O request, an I / O event is created for that request and added to the sliding time window circular buffer. The tail of the queue is checked to see if there are consecutive events with the same data block ID. If they are, only the access count and access time are updated; no new access events are inserted. The sliding time window circular buffer structure is as follows: Figure 5 As shown.

[0071] It should be noted that the sliding time window circular buffer is an IO event queue allocated based on three times the IO queue depth, stored in an array, with one window recorded in each period T. In other words, this embodiment uses a time sliding window to provide analysis samples for correlation analysis.

[0072] S3: Use the correlation analysis thread to periodically analyze the IO events in the sliding time window circular buffer, identify the data block pairs that have been accessed together within the specified time window, and calculate their correlation weight based on the difference in the number of accesses of the two data blocks in the data block pair within the sliding time window.

[0073] Specifically, in this embodiment, a separate thread (association analysis thread) is run to periodically perform association detection and analysis: From the sliding time window circular buffer, all data blocks accessed within each time window T are retrieved in chronological order, and the access information for each data block is counted. If they are at the same level, the data block set is [(A, t=10, count=5), (B, t=30, count=4), (C, t=50, count=4)], and A, B, and C are defined as having an association. If the time window is set to 60 seconds, then data block A is accessed 5 times, B is accessed 4 times, and C is accessed 4 times. The association weight W is then calculated according to the formula (see step S1):

[0074] W(AB) = (5+4) / (5-4+1) = 4.5;

[0075] W(BC) = (4+4) / (4-4+1) = 8.0;

[0076] W(AC) = (5+4) / (5-4+1) = 4.5.

[0077] S4: Update the association weights of the corresponding edges in the data block association graph based on the calculated association weights;

[0078] Specifically, this step updates the association weights of the corresponding edges in the data block association graph based on the calculated association weights, that is, updates the adjacency matrix g[u][v] and persists it.

[0079] S5: When any data block is detected to meet the preset cross-layer migration conditions due to its access popularity or coldness, the data block is determined as the target data block, and the target storage layer for this migration is determined.

[0080] Specifically, the preset cross-layer migration conditions include:

[0081] When the access frequency of a data block exceeds the first threshold within a preset time period, it is determined to be hot data, triggering a migration from the cold device layer to the hot device layer.

[0082] When the access frequency of a data block is lower than the second threshold within a preset time period, it is determined to be cold data, triggering a migration from the hot device layer to the cold device layer.

[0083] S6: Query the data block association graph to obtain all strongly associated data blocks whose association weight with the target data block is greater than the preset association threshold;

[0084] S7: Migrate the target data block and all its strongly associated data blocks to the target storage layer.

[0085] Specifically, this step uses a separate migration thread to migrate the target data block and all its strongly associated data blocks to the target storage layer.

[0086] It should be noted that steps S1 to S4 above are the process of building and maintaining a dynamic "data block association graph", which is the core of formulating association migration strategies.

[0087] Steps S5 to S7 above are the migration process based on the association graph. There are two main scenarios for data migration: one is data migration from the cold device layer to the hot device layer, and the other is data migration from the hot device layer to the cold device layer.

[0088] Scenario 1: "Association Preheating" during Hot Data Boosting

[0089] When a data block D is marked as hot data due to being accessed and needs to be migrated from the cold layer to the hot layer, not only is data block D itself migrated, but the data block association graph is also immediately scanned to find the set S = {R | W(DR) > mix} of all data blocks strongly related to data block D. `mix` is the set minimum weight value. All data blocks R in set S (even if they are currently in the cold layer and not hot data individually) are also migrated to the hot layer. This is because historical patterns predict that they will be accessed soon. For example, in the data block set [(A, t=10, count=5), (B, t=30, count=4), (C, t=50, count=4)], if `mix` is set to 5, data block A will be migrated along with data block D. If `mix` is set to 4, A, B, and C will be migrated along with data block D.

[0090] Scenario 2: "Association Degradation" During Data Cooling

[0091] When a data block D is deemed cold data due to prolonged inactivity and needs to be downgraded from the hot layer to the cold layer, the data block association graph is scanned to find the set S = {R | W(D->R) > mix} of all data blocks strongly correlated with data block D. If these data blocks are no longer hot data, they can still be migrated in advance, even if they haven't reached the cold data threshold. In this case, {D} ∪ {R} can be downgraded to the cold layer as a single association group. This aligns with the principle that "a group of correlated data often becomes cold simultaneously," avoiding fragmented migrations.

[0092] This embodiment provides a hierarchical storage data migration method based on data block correlation. It can predict in advance whether a data block is about to become hot data or cold data, and preheat or move the data block out of the performance layer in advance to make room for hot data. This allows users to read cold data without waiting, thus improving performance.

[0093] Example 2

[0094] This embodiment provides a hierarchical storage data migration device based on data block correlation, including:

[0095] The association graph module is used to create and maintain a data block association graph for each storage volume. The data block association graph uses data blocks as nodes and access correlations between data blocks as edges. Each edge has a dynamically updated association weight, which is used to quantify the association strength between two data blocks based on the difference in the number of accesses within a time window.

[0096] The IO sampling module is used to sample the IO access of each storage volume in real time using the IO thread, generate IO events, and record the IO events in the sliding time window circular buffer corresponding to the storage volume.

[0097] The correlation analysis module is used to periodically analyze the IO events in the sliding time window circular buffer using the correlation analysis thread, identify data block pairs that have been accessed together within a specified time window, and calculate their correlation weight based on the difference in the number of accesses of the two data blocks in the data block pair within the sliding time window.

[0098] The weight calculation module is used to update the association weight of the corresponding edge in the association graph of the data block according to the calculated association weight;

[0099] The migration monitoring module is used to determine the target data block and the target storage layer for this migration when it detects that any data block meets the preset cross-layer migration conditions due to its access popularity or coldness.

[0100] The associated block query module is used to query the data block association graph and obtain all strongly associated data blocks whose association weight with the target data block is greater than a preset association threshold.

[0101] The migration module is used to migrate the target data block and all its strongly associated data blocks to the target storage layer.

[0102] For details on the specific implementation of each module in a hierarchical storage data migration device based on data block correlation, please refer to the above description of the limitations of a hierarchical storage data migration method based on data block correlation, which will not be repeated here.

[0103] The technical features of the above embodiments can be combined in any way (as long as there is no contradiction in the combination of these technical features). For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described; these embodiments not explicitly written should also be considered to be within the scope of this specification.

Claims

1. A hierarchical storage data migration method based on data block correlation, characterized in that, The device is pre-divided into a cold device layer, a hot device layer, and several storage volumes. The cold device layer and the hot device layer are each further divided into several data blocks, and a sliding time window circular buffer is created for each storage volume. The method includes: Step 1: Create and maintain a data block association graph for each storage volume; the data block association graph uses data blocks as nodes and access correlation between data blocks as edges, and each edge has a dynamically updated association weight, which is used to quantify the association strength between two data blocks based on the difference in the number of accesses within a time window; Step 2: Use an IO thread to sample the IO access of each storage volume in real time and generate IO events. Record the IO events in the sliding time window circular buffer corresponding to the storage volume. Step 3: Use the correlation analysis thread to periodically analyze the IO events in the sliding time window circular buffer, identify the data block pairs that have been accessed together within the specified time window, and calculate their correlation weight based on the difference in the number of accesses of the two data blocks in the data block pair within the sliding time window. Step 4: Update the association weights of the corresponding edges in the data block association graph according to the calculated association weights; Step 5: When any data block is detected to meet the preset cross-layer migration conditions due to its access popularity or coldness, the data block is determined as the target data block, and the target storage layer for this migration is determined. Step 6: Query the data block association graph to obtain all strongly associated data blocks whose association weight with the target data block is greater than a preset association threshold; Step 7: Migrate the target data block and all its corresponding strongly associated data blocks to the target storage layer.

2. The hierarchical storage data migration method based on data block correlation according to claim 1, characterized in that, In step 1, creating and maintaining a data block association graph for each storage volume specifically includes: if a persistent data block association graph already exists for the storage volume, then loading the data block association graph from the disk metadata area; otherwise, creating an empty data block association graph.

3. The hierarchical storage data migration method based on data block correlation according to claim 1, characterized in that, The data block association graph is an undirected graph. The metadata stored in the nodes of the undirected graph includes: data block ID, last access timestamp, and current storage layer. The metadata stored in the edges of the undirected graph includes: source node ID, target node ID, and association weight.

4. The hierarchical storage data migration method based on data block correlation according to claim 3, characterized in that, The data block association graph is stored using an array and an adjacency matrix. The array is used to store node information, and the array index is the order of the data blocks in the storage volume. The adjacency matrix is ​​used to store edge information, and the value of the matrix element is the association weight of the corresponding edge.

5. The hierarchical storage data migration method based on data block correlation according to claim 1, characterized in that, The sliding time window circular buffer is an IO event queue allocated based on the IO queue depth. Each IO event record includes a data block ID, access time, and access count. When multiple consecutive IO events correspond to the same data block ID, only the last access time and access count of the data block ID are updated, without inserting a new event record.

6. The hierarchical storage data migration method based on data block correlation according to claim 1, characterized in that, The window length of the sliding time window circular buffer is adjustable between 1 second and 60 seconds.

7. The hierarchical storage data migration method based on data block correlation according to claim 1, characterized in that, In step 3, the association weight is calculated based on the difference in the number of accesses of the two data blocks in the data block pair within the sliding time window. The specific calculation formula used is as follows: W = (count1 + count2) / (|count1 - count2| + 1) Here, count1 and count2 represent the number of times the two data blocks are accessed within the sliding time window, and count1 ≥ count2; when the number of accesses of either data block within the time window is 0 or the two data blocks do not currently belong to the same storage layer, the association weight is set to 1.

8. The hierarchical storage data migration method based on data block correlation according to claim 1, characterized in that, In step 5, the preset cross-layer migration conditions include: When the access frequency of a data block exceeds the first threshold within a preset time period, it is determined to be hot data, triggering a migration from the cold device layer to the hot device layer. When the access frequency of a data block is lower than the second threshold within a preset time period, it is determined to be cold data, triggering a migration from the hot device layer to the cold device layer.

9. The hierarchical storage data migration method based on data block correlation according to claim 1, characterized in that, In step 7, a separate migration thread is used to migrate the target data block and all its corresponding strongly associated data blocks to the target storage layer.

10. A hierarchical storage data migration device based on data block correlation, characterized in that, include: The association graph module is used to create and maintain a data block association graph for each storage volume; The data block association graph uses data blocks as nodes and access correlations between data blocks as edges. Each edge has a dynamically updated association weight, which is used to quantify the association strength between two data blocks based on the difference in the number of accesses within a time window. The IO sampling module is used to sample the IO access of each storage volume in real time using the IO thread, generate IO events, and record the IO events in the sliding time window circular buffer corresponding to the storage volume; The correlation analysis module is used to periodically analyze the IO events in the sliding time window circular buffer using the correlation analysis thread, identify data block pairs that have been accessed together within a specified time window, and calculate their correlation weight based on the difference in the number of accesses of the two data blocks in the data block pair within the sliding time window. The weight calculation module is used to update the association weight of the corresponding edge in the data block association graph according to the calculated association weight; The migration monitoring module is used to determine the data block as the target data block and the target storage layer for this migration when it detects that any data block meets the preset cross-layer migration conditions due to its access popularity or coldness. The associated block query module is used to query the data block association graph and obtain all strongly associated data blocks whose association weight with the target data block is greater than a preset association threshold. The migration module is used to migrate the target data block and all its corresponding strongly associated data blocks to the target storage layer.

Images