Entity and method for managing data stored in multiple data storage tiers

By using counter tables and histogram algorithms in multiple data storage layers, the problems of performance degradation and excessive resource consumption in existing data storage systems are solved, achieving efficient data layer management and performance improvement.

CN118984986APending Publication Date: 2024-11-19HUAWEI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202280094058.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2022-06-02
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

Existing technologies for managing data across multiple data storage layers suffer from issues such as input/output activity discrepancies, performance degradation, excessive resource consumption, and suboptimal layer allocation. In particular, they struggle to effectively track file/object access frequencies and allocations in large-scale storage systems.

Method used

An entity and method are adopted to track the access frequency of data segments by using a table with multiple counters in multiple data storage layers. The distribution of data in different storage layers is determined by using counter values ​​and histogram algorithms, thereby achieving efficient data hierarchical management.

Benefits of technology

It improves the overall performance of the data storage system, reduces resource consumption, ensures efficient allocation of data segments, and enhances the system's scalability and access speed.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118984986A_ABST
    Figure CN118984986A_ABST
Patent Text Reader

Abstract

The invention relates to an entity for managing data stored in a plurality of data storage tiers. The entity includes a table storing a plurality of counters. Each counter of the plurality of counters is to count access to one or more data segments of data stored in the plurality of data storage layers. The entity is configured to use the table to determine in which data storage layer of the plurality of data storage layers the data segment of the data is stored. Further, a method for managing data stored in a plurality of data storage layers using a table storing a plurality of counters is provided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an entity and method for managing data stored in multiple data storage tiers. The present invention relates to the field of tiering using multiple data storage tiers. Background Art

[0002] In a large-scale file / object based storage controller that stores multiple data storage types with different density, latency, bandwidth, and cost, it is necessary to optimize data storage allocation based on application activity and / or user preference. The term "media" may be used as a synonym for the term "data storage". The multiple data storage types with different density, latency, bandwidth, and cost may be referred to as multiple data storage tiers. A tiering algorithm may be used to store data in the multiple data storage tiers. Summary of the invention

[0003] With respect to storing data in multiple data storage tiers (i.e., tiering of data), there may be a skew in input-output activity (I / O activity) such that a small portion of files / objects are highly active (e.g., frequently read). When considering large files, this behavior is reflected by file fragments where the first file fragment is read the same number of times as the rest of the file, or many times more than the rest of the file.

[0004] By identifying popular / hot files / objects or their popular fragments and moving them from lower (slower) data storage tiers (which may be referred to as media tiers) to higher (faster) data storage tiers, the overall performance in terms of, for example, the number of read / write operations per second (IOPS) can be maximized. By subtracting the number of accesses from the activity density and dividing the difference by the size ([activity density – number of accesses] / size), a fragment popularity score can be defined. Allocating popular data fragments in faster tiers may improve overall performance.

[0005] Tracking accesses without significantly interrupting the input / output flow (I / O flow) results in performance degradation, thus requiring efficient RAM-resident data structures. However, due to the large number of files / objects, a single storage node may not be able to hold the mapping from files / objects to access counters.

[0006] The problem may be even greater when considering file / object fragments, many of which are small (<32 kiloBytes (kB)) and others with huge address spaces (GigaBytes (GB) to TeraBytes (TB)). Tracking access to every file / object at a detailed granularity (e.g., media block or a fixed number of such blocks) may be infeasible.

[0007] In a distributed storage controller that shares a common medium (i.e., a common data storage or data storage layer), each storage node may have a partial view of files / objects (e.g., sharded by name / id / directory) that may be biased toward certain files / objects. This raises another global tiering decision problem with respect to the common medium.

[0008] Methods for managing data stored in multiple data storage tiers may have limited scalability in one or more of the following points. Such methods may limit the number of files / objects that can be managed, and / or may not track activity at a detailed granularity (e.g., they may only track entire files / objects). Such methods may degrade input / output performance (I / O performance) in order to track all files / objects, and / or may require a large amount of random-access memory (RAM) in order to track accurate file / object activity, and / or may not correctly capture file popularity that affects read performance.

[0009] Local information and decisions can lead to suboptimal tier assignments. Gathering activity from all nodes and sorting it can be resource intensive, leading to long tier cycles and / or degraded performance. Iteration over all inodes is slow and resource intensive, leading to long tier cycles and / or degraded performance.

[0010] In view of the above, the present invention is directed to providing an improved entity and method for managing data stored in multiple data storage tiers.

[0011] These and other objects are achieved by the solutions of the invention as described in the independent claims. Advantageous implementations are further defined in the dependent claims.

[0012] A first aspect of the present invention provides an entity for managing data stored in multiple data storage layers. The entity includes a table storing multiple counters. Each of the multiple counters is used to count accesses to one or more data segments of the data stored in the multiple data storage layers. The entity is used to use the table to determine in which data storage layer of the multiple data storage layers to store the data segment of the data.

[0013] Since each counter in the table is used to count accesses to one or more data segments, a large number of data segments can be counted (e.g., billions of data segments that may exist in today's enterprises, cloud storage, etc.). Therefore, entities are not limited to a small number of data segments.

[0014] Since an entity can count a large number of data segments, the entity has good activity visibility into data stored in multiple data storage tiers. This allows the entity to perform improved tiering, i.e., deciding in which of the multiple data storage tiers to store a data segment.

[0015] The paragraph "the entity is used to determine using a table" can be understood as "the entity is used to determine using a plurality of counters in a table (i.e., the values ​​of the plurality of counters)". Deciding in which of a plurality of data storage tiers a data segment is stored can be a tiering process or can be a part of a tiering process. In other words, the entity can be used to perform tiering of data stored in a plurality of data storage tiers using a table (i.e., using a plurality of counters in a table).

[0016] The entity may be part of a storage node or may be configured as part of a storage node. The storage node may be part of a storage system or may be configured as part of a storage system. The plurality of data storage tiers may be part of a storage system or may be configured as part of a storage system. The entity may be a storage controller (e.g., for a storage node). The plurality of data storage tiers may be part of a storage node. The entity may include or at least partially include the plurality of data storage tiers.

[0017] The entity may be implemented by hardware and / or software. The entity may be or may include a processor or processing circuit that is used to perform, implement or initiate various operations of the entity described herein. The processing circuit may include hardware and / or the processing circuit may be controlled by software. The hardware may include analog circuits or digital circuits, or both. The digital circuit may include components such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or a multi-purpose processor.

[0018] The entity may include a memory circuit that stores one or more instructions that can be executed by a processor or processing circuit, optionally under the control of software. For example, the memory circuit may include a non-transitory storage medium that stores executable software code that, when executed by the processor or processing circuit, causes the entity to perform various operations. In one embodiment, the processing circuit includes one or more processors and a non-transitory memory connected to the one or more processors. The non-transitory memory may carry an executable program code that, when executed by the one or more processors, causes the entity to perform, implement, or initiate the operations or methods described herein.

[0019] Data may be or may include one or more files and / or one or more objects. One or more data segments of data may be one or more files and / or one or more objects. Additionally or alternatively, one or more data segments of data may be one or more fragments of one or more files and / or one or more fragments of one or more objects. In other words, a data segment may be a fragment of a file or an object. The term "data segment" may be used as a synonym of the term "data segment". The terms "fragment" and "segment" may be used as synonyms. The terms "data file" and "data object" may be used as synonyms of the terms "file" and "object", respectively.

[0020] The multiple data storage layers may be multiple different data storage types. The term "media layer" or "layer" may be used as a synonym for the term "data storage layer". The multiple data storage layers may include two or more different data storage types. For example, the multiple data storage layers may be or may include at least two of a hard disk drive (HDD), a solid-state drive (SSD), a non-volatile memory express storage (NVMe), a tape memory, an optical memory, a memory, a storage class memory (SCM), etc.

[0021] The table may be a RAM resident structure, which may be referred to as a "RAM based table" or a "RAM resident active structure." The table may be referred to as an "active table."

[0022] Access to one or more data segments (counted by corresponding counters) may be or may include reading data from multiple data storage tiers (e.g., reading objects / files and / or fragments of objects / files) and / or writing data to multiple data storage tiers (e.g., writing objects / files and / or fragments of objects / files).

[0023] In an implementation of the first aspect, the plurality of data storage tiers differ from one another in terms of storage capacity and performance regarding writing and / or reading data.

[0024] The multiple data storage tiers may differ from one another in terms of density, latency, bandwidth, and / or cost.

[0025] In an implementation of the first aspect, at least two counters of the plurality of counters are used to count accesses to different numbers of data segments of one or more data segments of data stored in the plurality of data storage layers.

[0026] For example, in the case where the data stored in the plurality of data storage layers are files, the number of files and / or file fragments that can be counted by a first counter in the plurality of counters can be different from the number of files and / or file fragments that can be counted by at least another counter in the plurality of counters. Data segment access can be a file access and / or an object access.

[0027] In an implementation of the first aspect, at least one of the plurality of counters is used to count accesses to a plurality of data segments of data stored in a plurality of data storage layers by using a hash function that maps the plurality of data segments to the at least one counter.

[0028] In other words, at least one of the multiple counters can be used to count accesses to two or more data segments of data stored in multiple data storage layers by using a hash function that maps the two or more data segments to the at least one counter. The counter used to count accesses to multiple data segments can be called a "shared counter". Optionally, at least one counter can be used to count accesses to two or more different data segments selected by hashing the inode number (using a hash function). For example, in the case where the data stored in multiple data storage layers is a file, at least one counter can be used to count accesses to two or more different files and / or file fragments selected by hashing the file id / inode number (using a hash function).

[0029] The entity may be used to determine the popularity of data stored in multiple data storage tiers. For example, the entity may be used to approximate the popularity of data stored in multiple data storage tiers. For tiering purposes (i.e., storing data in multiple data storage tiers), it is sufficient to approximate the data popularity (e.g., file / object popularity) as long as highly active data segments of the data (e.g., highly active files / objects and / or highly active file / object segments) are identified and there is not much deviation from the actual data activity distribution (e.g., file / object activity distribution).

[0030] In an implementation of the first aspect, an entity is configured to count a number of one or more unique data segments counted by a counter of a plurality of counters in a table.

[0031] The paragraph "one or more unique data segments counted by the counter" may be understood as "one or more unique data segments accesses of which are counted by the counter". The entity is used to calculate the number of one or more unique data segments counted by a counter in a plurality of counters in a table by using an algorithm for approximating access counts, i.e., for approximating the number of one or more unique data segments accesses of which are counted by the counter. For example, the algorithm may be a count-min-sketch algorithm.

[0032] In an implementation of the first aspect, the entity is configured to calculate a number of one or more unique data segments using a HyperLogLog algorithm.

[0033] For example, an entity may be used to estimate the number of one or more unique data segments of data stored in multiple data storage tiers using a HyperLogLog algorithm and data structure. The HyperLogLog data structure may be referred to as an HLL. An entity may be used to add a hash of an inode number / file id / object id to the HLL when tracking accesses to the unique data segments. The HLL may be used to answer queries for the number of unique elements present in a collection that is equivalent to the number of unique data segments counted by a counter.

[0034] For example, a counter that counts accesses to multiple data segments with a value of 10 may represent accesses to a single data segment 10 times, or may represent accesses to each of the 10 data segments once. These situations will cause an entity to make different decisions regarding tier allocation (i.e., allocation of data to multiple data storage tiers). Thus, counting the number of one or more unique data segments counted by the counter may distinguish between the above exemplary described situations (at least on average).

[0035] In one implementation of the first aspect, an entity is used to calculate a counter value threshold for each of a plurality of data storage tiers for a table. The entity can be used to manage storing data in the plurality of data storage tiers by comparing the counter values ​​of a plurality of counters in the table with the counter value threshold of at least one of the plurality of data storage tiers.

[0036] The term "tiered threshold" may be used to refer to a counter value threshold. The counter value thresholds for multiple data storage tiers may be capacity percentiles, such as the 99th percentile (P99), the 95th percentile (P95), the 90th percentile (P90), etc. The capacity percentiles may assign data segments (e.g., files / objects and / or file / object segments) to a collection such that the capacity of all data segments in the collection is equal to a percentage of the capacity of the entire storage covered by the multiple data storage tiers. For example, the 99th percentile (P99) may be a collection of 1% (100% - 99% = 1%) of the capacity of the entire storage covering the multiple data storage tiers (e.g., the top data storage tier may be 1% of the capacity of the entire storage).

[0037] The counter value threshold (tier threshold) of the data storage tier may be the number of one or more counters in the plurality of counters in the table that, when used to filter the sorted list of data segments assigned to the data storage tier, would result in a complete data storage tier. That is, after filtering the list of data segments by the counter value threshold, the capacity of the remaining data segments may be equal to the capacity of the data storage tier.

[0038] In one implementation of the first aspect, an entity is used to: determine a data storage layer among multiple data storage layers having a maximum counter value threshold for a counter among multiple counters in a table, wherein the maximum counter value threshold is less than or equal to the counter value of the counter; and store one or more data segments whose accesses are counted by the counter in the determined data storage layer.

[0039] In an implementation of the first aspect, an entity is configured to generate a histogram for a table. The histogram includes: sorted different counter values ​​of a plurality of counters in the table; and for each of the different counter values, a number of one or more counters in the plurality of counters having the counter value. The different counter values ​​are indexed such that a minimum counter value has a minimum index and a maximum counter value has a maximum index.

[0040] In other words, the x-axis (horizontal axis) of the histogram may indicate the sorted different counter values ​​of the plurality of counters in the table. The y-axis (vertical axis) of the histogram may indicate the number of one or more counters in the plurality of counters. The histogram may include a bar graph for each of the sorted different counter values ​​of the plurality of counters, wherein, for a corresponding counter value in the different counter values, the height of the bar graph may indicate the number of one or more counters in the plurality of counters having the corresponding counter value. The histogram may be referred to as a "visit histogram".

[0041] In an implementation of the first aspect, an entity is configured to calculate a counter value threshold for each data storage layer in a plurality of data storage layers for a table using a histogram by iteratively performing the following steps:

[0042] - in the first iteration setting the capacity parameter to zero and setting the counter index to the index of the maximum counter value in the histogram;

[0043] - Check if the capacity parameter is less than or equal to the capacity of the data storage layer;

[0044] - if the capacity parameter is greater than the capacity of the data storage layer, setting the counter value threshold to the counter value indexed by the counter index;

[0045] - If the capacity parameter is less than or equal to the capacity of the data storage layer:

[0046] - a second capacity parameter is calculated by multiplying the number of one or more counters associated with the counter value indexed by the counter index, the sharing factor in the table and the average size of the data segments whose accesses are counted by the table, wherein

[0047] The sharing factor is equal to the number of unique data segments counted by the plurality of counters in the table divided by the number of counters in the table;

[0048] - setting the capacity parameter to a second capacity parameter; and

[0049] - Decrement the counter index by one.

[0050] In other words, the sharing factor may be defined as the average number of unique data segments that share a counter in the table.The number of counters in the table may be or may indicate the size of the table.

[0051] In one implementation of the first aspect, the entity includes a plurality of tables storing a plurality of counters, wherein a table in the plurality of tables is used to count accesses to data segments of different size ranges of data stored in a plurality of data storage layers, such that each table in the plurality of tables counts accesses to data segments of the same size range.

[0052] Each of the plurality of tables may be configured according to the above table.The term "size class" may be used to refer to a range of sizes.

[0053] In other words, accesses to data segments of different sizes (e.g., different size ranges) may be counted in different tables. For example, the different size ranges may include a first range less than or equal to 8 KB, a second range greater than 8 KB and less than or equal to 64 KB, a third range greater than 64 KB and less than or equal to 4 MB, and a fourth range greater than 4 MB.

[0054] The popularity score (access density) of a data segment (e.g., file / object or file / object segment) may be affected by the size of the data segment (e.g., size of the file / object or file / object segment). Tracking the average size of each counter in the table may result in additional RAM usage and high complexity. The above-mentioned multiple tables for different size ranges of data stored in the data storage layer (i.e., data segment size segmentation of the table including multiple counters) can reduce or avoid complexity. In addition, this also improves the density and accuracy of the counters. The RAM size of the table can be determined by the sharing factor of the table and the maximum number of data segments whose accesses are counted by the counters in the table. Since there may be a different number of data segments for each size range, adjusting the table size according to the size range may improve RAM utilization (number of bits per data segment).

[0055] If the data stored in the data storage tier includes or is a file, large files may be significantly reduced. Entities can be used to split large files into file fragments (which may be called file chunks) and count activities for the file fragments separately, thereby reducing or avoiding conflicts. Splitting large files and counting their fragments separately enables identification of file ranges that are read more often than other files. This enables moving only these file ranges to a higher data storage tier (e.g., a data storage tier with better performance in writing and / or reading data).

[0056] When the file size is unknown at access time (e.g., read time), using offset-based counting may enable updating counters (e.g., read counters). For example, an 8KB file fragment in a 1GB file may be considered a small file or micro file. Performing tiering decisions independently for this fragment can achieve a direct relationship between access and tiering selection.

[0057] In an implementation of the first aspect, an entity is configured to generate a histogram for each of a plurality of tables. The histogram includes: sorted different counter values ​​of a plurality of counters in the table; and for each of the different counter values, a number of one or more counters in the plurality of counters having the counter value. The different counter values ​​are indexed such that a minimum counter value has a minimum index and a maximum counter value has a maximum index.

[0058] In one implementation of the first aspect, an entity is used to normalize different counter values ​​in each histogram by: calculating a normalized value, the normalized value being equal to the product of a sharing factor of a table (for which the histogram includes different counter values) and an average size of data segments whose accesses are counted by the table; and dividing the different counter values ​​in the histogram by the normalized value.

[0059] The sharing factor may be equal to the number of unique data segments counted by the plurality of counters in the table divided by the number of counters in the table.

[0060] In an implementation of the first aspect, an entity is configured to calculate a counter value threshold for each of a plurality of data storage tiers for a plurality of tables using a plurality of histograms by performing the following steps:

[0061] - Sort all normalized counter values ​​in multiple histograms;

[0062] - Calculate capacity percentiles for multiple data storage tiers; and

[0063] - For each table, using the sorted normalized counter values ​​in the plurality of histograms and the calculated capacity percentiles, calculating a counter value threshold for each of the plurality of data storage tiers.

[0064] In an implementation of the first aspect, the entity is configured to determine one or more data segments of data stored in a data storage layer for moving to an adjacent data storage layer with a larger capacity by performing the following operations:

[0065] -determining that the capacity of the data storage tier is full or has reached a capacity threshold;

[0066] - receiving an instruction to move one or more data segments to an adjacent data storage tier; and / or

[0067] - Receiving information regarding future reduced access activity for one or more data segments.

[0068] A data storage tier with a larger capacity may have worse performance in writing and / or reading data, and vice versa.

[0069] In one implementation of the first aspect, an entity is used to: maintain a database for data storage layers, the database including an inode number of each data segment of data stored in the data storage layer; and determine one or more data segments for moving to an adjacent data storage layer by scanning the database.

[0070] The database may be referred to as an index structure. For each data segment of the data stored in the data storage layer, in addition to the inode number, the database may also include additional information. The additional information may include a segment offset and / or length.

[0071] In an implementation of the first aspect, the entity is configured to move the data segment to an adjacent data storage layer with larger capacity when a counter value of a counter counting data segments of data stored in the data storage layer is less than a counter value threshold of the data storage layer.

[0072] In one implementation of the first aspect, an entity is used to determine one or more data segments of data stored in a data storage tier for movement to an adjacent data storage tier with smaller capacity by determining information about access activity of the one or more data segments that is greater than an access activity threshold.

[0073] A data storage tier with smaller capacity may have better performance in writing and / or reading data, and vice versa.

[0074] In an implementation of the first aspect, the entity is configured to move the data segment to an adjacent data storage layer with smaller capacity when a counter value for counting the data segment of the data stored in the data storage layer is greater than or equal to a counter value threshold of the adjacent data storage layer.

[0075] In order to implement the entity according to the first aspect of the present invention, some or all of the implementations and optional features of the first aspect may be combined with each other.

[0076] A second aspect of the present invention provides a method for managing data stored in multiple data storage layers using a table storing multiple counters. The method includes: each of the multiple counters counts accesses to one or more data segments of the data stored in the multiple data storage layers; and using the table to determine in which data storage layer of the data storage layer the data segment of the data is stored.

[0077] In an implementation of the second aspect, the plurality of data storage tiers differ from one another in terms of storage capacity and performance regarding writing and / or reading data.

[0078] In an implementation of the second aspect, the method includes: at least two counters of the plurality of counters count accesses to different numbers of data segments of one or more data segments of data stored in the plurality of data storage layers.

[0079] In an implementation of the second aspect, the method includes: at least one of the multiple counters counts accesses to multiple data segments of data stored in multiple data storage layers by using a hash function, and the hash function maps the multiple data segments to at least one counter.

[0080] In an implementation of the second aspect, the method includes calculating a number of one or more unique data segments counted by a counter of a plurality of counters in a table.

[0081] In an implementation of the second aspect, the method includes: using a HyperLogLog algorithm to calculate a number of one or more unique data segments.

[0082] In an implementation of the second aspect, the method includes: calculating a counter value threshold for each of the plurality of data storage tiers for a table. The method may include: managing storage of data in the plurality of data storage tiers by comparing the counter values ​​of the plurality of counters in the table with the counter value threshold of at least one of the plurality of data storage tiers.

[0083] That is, the method may include managing storage of data in the plurality of data storage tiers by comparing counter values ​​of the plurality of counters in the table with a counter value threshold of at least one of the plurality of data storage tiers.

[0084] In an implementation of the second aspect, the method may include: determining a data storage layer with a maximum counter value threshold among multiple data storage layers for a counter among multiple counters in a table, the maximum counter value threshold being less than or equal to the counter value of the counter; and storing one or more data segments in the determined data storage layer, the counter counting accesses to the one or more data segments.

[0085] In an implementation of the second aspect, the method may include: generating a histogram for a table. The histogram includes: sorted different counter values ​​of a plurality of counters in the table; and for each of the different counter values, a number of one or more counters in the plurality of counters having the counter value. The different counter values ​​are indexed such that a minimum counter value has a minimum index and a maximum counter value has a maximum index.

[0086] In an implementation of the second aspect, the method includes: calculating a counter value threshold for each data storage layer in the plurality of data storage layers for a table using a histogram by iteratively performing the following steps:

[0087] - in the first iteration setting the capacity parameter to zero and setting the counter index to the index of the maximum counter value in the histogram;

[0088] - Check if the capacity parameter is less than or equal to the capacity of the data storage layer;

[0089] - if the capacity parameter is greater than the capacity of the data storage layer, setting the counter value threshold to the counter value indexed by the counter; and

[0090] - If the capacity parameter is less than or equal to the capacity of the data storage layer:

[0091] - a second capacity parameter is calculated by multiplying the number of one or more counters associated with the counter value indexed by the counter index, the sharing factor in the table and the average size of the data segments whose accesses are counted by the table, wherein

[0092] The sharing factor is equal to the number of unique data segments counted by the plurality of counters in the table divided by the number of counters in the table;

[0093] - setting the capacity parameter to a second capacity parameter; and

[0094] - Decrement the counter index by one.

[0095] In an implementation of the second aspect, a method may use multiple tables storing multiple counters to manage data stored in multiple data storage layers. The method may include: a table in the multiple tables counts accesses to data segments of different size ranges of the data stored in the multiple data storage layers, so that each table in the multiple tables counts accesses to data segments of the same size range.

[0096] In an implementation of the second aspect, the method includes: generating a histogram for each of a plurality of tables. The histogram includes: sorted different counter values ​​of a plurality of counters in the table; and for each of the different counter values, a number of one or more counters in the plurality of counters having the counter value. The different counter values ​​are indexed such that a minimum counter value has a minimum index and a maximum counter value has a maximum index.

[0097] In an implementation of the second aspect, the method includes normalizing different counter values ​​in each histogram by: calculating a normalized value, the normalized value being equal to the product of a sharing factor of a table (for which the histogram includes different counter values) and an average size of a data segment for which the table counts accesses; and dividing the different counter values ​​in the histogram by the normalized value.

[0098] In an implementation of the second aspect, the method includes calculating a counter value threshold for each of the plurality of data storage tiers for the plurality of tables using the plurality of histograms by performing the following steps:

[0099] - Sort all normalized counter values ​​in multiple histograms;

[0100] - Calculate capacity percentiles for multiple data storage tiers; and

[0101] - For each table, using the sorted normalized counter values ​​in the plurality of histograms and the calculated capacity percentiles, calculating a counter value threshold for each of the plurality of data storage tiers.

[0102] In an implementation of the second aspect, the method includes determining one or more data segments of data stored in a data storage layer for moving to an adjacent data storage layer with a larger capacity by:

[0103] -determining that the capacity of the data storage tier is full or has reached a capacity threshold;

[0104] - receiving an instruction to move one or more data segments to an adjacent data storage tier; and / or

[0105] - Receiving information regarding future reduced access activity for one or more data segments.

[0106] In an implementation of the second aspect, the method includes: maintaining a database for data storage layers, the database including an inode number of each data segment of data stored in the data storage layer; and determining one or more data segments for moving to an adjacent data storage layer by scanning the database.

[0107] In an implementation of the second aspect, the method includes: when a counter value of a counter counting data segments of data stored in a data storage layer is less than a counter value threshold of the data storage layer, moving the data segments to an adjacent data storage layer with larger capacity.

[0108] In an implementation of the second aspect, the method includes: determining one or more data segments of data stored in a data storage layer for moving to an adjacent data storage layer with smaller capacity by determining information about access activity of the one or more data segments that is greater than an access activity threshold.

[0109] In an implementation of the second aspect, the method includes: when a counter value of a counter counting data segments of data stored in a data storage layer is greater than or equal to a counter value threshold of an adjacent data storage layer, moving the data segment to an adjacent data storage layer with smaller capacity.

[0110] The above description of the entity according to the first aspect applies correspondingly to the method according to the second aspect.

[0111] The method according to the second aspect and its implementations and optional features has the same advantages as the entity according to the first aspect and its corresponding implementations and corresponding optional features.

[0112] In order to implement the method according to the second aspect of the present invention, part or all of the implementation modes and optional features of the second aspect described above may be combined with each other.

[0113] A third aspect of the present invention provides a computer program, the computer program comprising program code, which, when implemented on a processor, is used to execute the method according to the second aspect or any implementation thereof. The processor can control a table storing multiple counters. The processor can access the table. Optionally, the processor can include the table. The processor can access and optionally include multiple data storage layers or at least a portion of multiple data storage layers.

[0114] A fourth aspect of the present invention provides a computer program, the computer program comprising program code, the program code being used to execute the method according to the second aspect or any implementation manner thereof.

[0115] A fifth aspect of the present invention provides a computer comprising a memory and a processor, the memory and the processor being used to store and execute program code to execute the method according to the second aspect or any implementation thereof. A computer (e.g., a processor) can control a table storing multiple counters. A computer (e.g., a processor) can access a table. Optionally, a computer can include a table. A computer can access and optionally include multiple data storage layers or at least a portion of multiple data storage layers.

[0116] A sixth aspect of the present invention provides a non-transient storage medium storing an executable program code, which, when executed by a processor, causes the method according to the second aspect or any implementation thereof to be executed. The processor may control a table storing a plurality of counters. The processor may access the table. Optionally, the processor may include the table. The processor may access and optionally include a plurality of data storage layers or at least a portion of a plurality of data storage layers.

[0117] A seventh aspect of the present invention provides a computer-readable storage medium storing an executable program code, which, when executed by a processor, causes the method according to the second aspect or any implementation thereof to be executed. The processor can control a table storing multiple counters. The processor can access the table. Optionally, the processor can include the table. The processor can access and optionally include multiple data storage layers or at least a portion of multiple data storage layers.

[0118] The computer program according to the third aspect, the computer program according to the fourth aspect, the computer according to the fifth aspect, the non-transitory storage medium according to the sixth aspect, and the computer-readable storage medium according to the seventh aspect each have the same advantages as the entity according to the first aspect and its corresponding implementation methods and corresponding optional features.

[0119] All steps performed by various entities described in this application and the functions to be performed by various entities described are intended to refer to the corresponding entities being suitable for or used to perform the corresponding steps and functions. Although in the following description of specific embodiments, the specific functions or steps to be performed by external entities are not reflected in the description of the specific detailed elements of the entities that perform the specific steps or functions, it should be clear to the technician that these methods and functions can be implemented with corresponding hardware or software elements or any combination thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0120] In conjunction with the accompanying drawings, the following description of specific embodiments illustrates the above aspects and implementation methods, in which:

[0121] Figure 1 An example of an entity for managing data stored in multiple data storage tiers according to an embodiment of the present invention is shown;

[0122] Figure 2 It shows that Figure 1 Examples of processes performed by entities;

[0123] Figure 3 Shows Figure 1 Optional functionality for entities;

[0124] Figure 4 Shown by Figure 3 An example of counting a table of

[0125] Figure 5 It shows that it can be Figure 1 Examples of different size ranges of data that entities may optionally use;

[0126] Figure 6 It shows that it can be Figure 1 An entity optionally generates an example of the values ​​of a histogram;

[0127] Figure 7 It shows that it can be Figure 1 Flowcharts of examples of processes performed by entities;

[0128] Figure 8 An example of a sorted normalized histogram entry list according to an embodiment of the present invention is shown;

[0129] Fig. 9 It shows that it can be Figure 1 A flowchart of an example of a process performed by an entity; and

[0130] Fig.10 A flow chart showing an example of a method for managing data stored in multiple data storage tiers according to an embodiment of the present invention.

[0131] In the figures, corresponding elements are provided with the same reference numerals. DETAILED DESCRIPTION

[0132] Figure 1 An example of an entity for managing data stored in multiple data storage tiers according to an embodiment of the present invention is shown. Figure 1 The entity is an example of the entity according to the first aspect of the present invention. Therefore, the description of the entity according to the first aspect applies accordingly to Figure 1 Entity.

[0133] Figure 1 The entity 1 is an entity for managing data stored in a plurality of data storage layers 3. The entity 1 includes a table 2 storing a plurality of counters 2a. Each of the plurality of counters 2a is used to count accesses to one or more data segments of data stored in the plurality of data storage layers 3. The entity 1 is used to use the table 2 to decide in which data storage layer of the plurality of data storage layers 3 to store the data segment of the data.

[0134] Figure 1 The number of counters 2a in Table 2 is shown as an example only and does not limit the present invention. That is, Table 2 may include two or more counters 2a. Figure 1 The number of data storage layers shown (ie, two data storage layers 3a and 3b) is only for example and does not limit the present invention. That is, the entity 1 can be used to manage data stored in two or more data storage layers 3.

[0135] Entity 1 can be part of a storage node or can be configured as part of a storage node. A storage node can be part of a storage system or can be configured as part of a storage system ( Figure 1 Multiple data storage layers may be part of the storage system or may be configured as part of the storage system ( Figure 1 Entity 1 may include or at least partially include multiple data storage layers 3 ( Figure 1 Entity 1 may be implemented by hardware and / or software.

[0136] For further details of the entity 1 (eg, optional features or implementations), reference is made to the description of the entity according to the first aspect of the present invention and Figures 2 to 10 Description.

[0137] Figure 2 It shows that Figure 1 An example of a process performed by an entity.

[0138] like Figure 2 As shown, entity 1 may use a hash function 4 that maps multiple data segments (e.g., files / objects and / or file / object segments) of data stored in multiple data storage layers 3 to counters 2a in table 2. This enables each counter 2a in table 2 to count accesses to two or more data segments of data stored in data storage layers 3.

[0139] Figure 2 The number of counters 2a and the values ​​of counters 2a in Table 2 are shown only as examples and are not limiting of the present invention (ie, they may be different). Figure 2 The number of data storage layers 3 shown is for example only and does not limit the present invention (i.e., it may be different). If the data segment of the data stored in the data storage layer 3 is a file, the hash function may hash the file using a file id (file_id) (e.g., an inode number (inode_num)). The inode number is a unique identifier for file data. If the data segment of the data stored in the data storage layer 3 is an object, the hash function may hash the object using an object id (object_id).

[0140] Figure 2 The examples of values ​​of counter 2a shown indicate examples of the number of accesses to one or more data segments counted by the corresponding counter, optionally using hash function 4. For example, counter 2a at the top of Table 2 has a value of 3. This may mean that the counter has counted a single access to three different data segments (e.g., three different files or objects) or three accesses to one data segment. In order to distinguish between the above two cases, entity 1 may be used to count the number of one or more unique data segments (e.g., files / objects and / or file / object segments) counted by the corresponding counter in counter 2a in Table 2.

[0141] according to Figure 2In the example of, there may be three different data storage tiers 3a, 3b, and 3c for storing data. As indicated by the three blocks of different sizes representing the three different data storage tiers 3a, 3b, and 3c, the data storage tiers 3 (e.g., data storage tiers 3a, 3b, and 3c) may differ from each other in terms of storage capacity and performance regarding writing and / or reading data. For example, data storage tier 3a may be a non-volatile memory express storage (NVMe), data storage tier 3b may be a solid-state driver (SSD), and data storage tier 3c may be a hard disk drive (HDD). Therefore, data storage tier 3a may have better performance than the other two data storage tiers 3b and 3c; data storage tier 3b may have better performance than data storage tier 3c. The better the performance of a data storage tier, the higher or greater the cost of the data storage tier may be for a certain storage capacity. Therefore, the better the performance of a data storage tier, the smaller the storage capacity of the data storage tier may be, such as Figure 2 This can save costs.

[0142] In order to access data segments of data stored in the data storage layer 3, the entity 1 may include a filtering and sorting unit 5. The entity 1 may use the filtering and sorting unit 5 to segment data of different sizes (eg, file / object size segmentation).

[0143] Figure 3 Shows Figure 1 Optional functionality for entities. Figure 3 In the example, it is assumed that the files are counted as data segments by counter 2a in Table 2. The number of files, the number of counter 2a in Table 2, and Figure 3 The value of the counter 2a is only an example and does not limit the present invention.

[0144] like Figure 3 As shown, entity 1 can use hash function 4 so that one of counters 2a in table 2 can count two or more files. Such counters can be called "shared counters". That is, file activities can be tracked by shared counters. This is advantageous when there are too many files / objects (e.g., small files / objects) in a limited table (i.e., the capacity of table 2 is limited) that need to be tracked independently (i.e., the access to each file / object is counted by the corresponding counter). For example, table 2 can be a RAM-resident structure, i.e., a RAM-based table.

[0145] like Figure 3As shown, the hash function 4 can map the files "File A", "File B" and "File C" to the same counter 21 in the counter 2a in Table 2, and the hash function 4 can map the files "File D" and "File E" to the same counter 22 in the counter 2a in Table 2. In other words, the counter 21 can count the accesses to the files "File A", "File B" and "File C" of the data stored in the data storage layer 3. The counter 22 can count the accesses to the files "File D" and "File E" of the data stored in the data storage layer 3.

[0146] Figure 4 Shown by Figure 3 An example of counting a table.

[0147] Figure 4 The left side of Figure 3 Which of the five files "File A", "File B", "File C", "File D", and "File E" shown is accessed (e.g., read) at consecutive time points T0 to T6. For example, it is assumed that the file "File D" is read (i.e., accessed) at time point T1, and the file "File C" is read (i.e., accessed) at time point T5. Figure 4 The right side shows Figure 3 The values ​​of counters 21 and 22 in Table 2 are due to Figure 4 . Since counter 21 counts files "File A", "File B", and "File C", the value of counter 21 changes only at the time point when one of the three files is accessed (e.g., time points T0, T2, T3, T4, and T5). Since counter 22 counts files "File D" and "File E", the value of counter 22 changes only at the time point when one of the two files is accessed (e.g., time points T1 and T6). Periodic decay can adapt counter 2a in Table 2 of entity 1 to changing workloads.

[0148] Multiple data segments (e.g., files / objects and / or file / object segments) sharing a single counter in a table (i.e., multiple data segments are counted by a common counter) may be referred to as a conflict. Conflicts between data segments (e.g., files / objects) may increase as the number of data segments tracked (e.g., counted) by Table 2 increases. The sharing factor in Table 2 may be the average number of data segments (e.g., files / objects) that share a counter. The sharing factor in Table 2 may be equal to the number of unique data segments counted by multiple counters 2a in Table 2 divided by the number of counters 2a in Table 2.

[0149] If conflicting data segments (e.g., files / objects) have the same popularity score, the conflict will not have an adverse effect. However, without proper "guidance", data segments with different popularity scores may conflict and be incorrectly assigned to the same data storage tier. Such incorrect associations, such as conflicts between active small data segments (e.g., small files / objects) and inactive large data segments (e.g., files / objects), may cause tier space waste in higher data storage tiers (i.e., tiers with better performance), may reduce the effective utilization of data storage tier capacity, and may degrade performance.

[0150] For example, in the above read request, the files "File A" and "File B" are counted in the same counter. File "File A" may be very small, such as 4 KB. File "File B" may be very large, such as 1 GB. File "File H" ( Figure 3 and Figure 4 ) can have the same hash value but cannot be read more than once.

[0151] Figure 5 It shows that it can be Figure 1 Examples of data of different size ranges that the entity may optionally use.

[0152] For example, four different size classes (class 1, class 2, class 3, and class 4) may be defined. Class 1, class 2, class 3, and class 4 may correspond to micro data segments, small data segments, medium data segments, and large data segments, respectively. For example, class 1 may correspond to a first size range less than or equal to 8 KB. Class 2 may correspond to a second size range greater than 8 KB and less than or equal to 64 KB. Class 3 may correspond to a third size range greater than 64 KB and less than or equal to 4 MB, and class 4 may correspond to a fourth size range greater than 4 MB.

[0153] Entity 1 may include multiple tables storing multiple counters ( Figure 5 (not shown). Figures 1 to 4 The description of Table 2 of entity 1 may be correspondingly applicable to each of the multiple tables. The multiple tables are used to count accesses to data segments of different size ranges of data stored in multiple data storage layers, so that each of the multiple tables counts accesses to data segments of the same size range. The terms "size range" and "size range" may be used as synonyms. Figure 5 The four size categories of may be examples of different size ranges. In this case, the entity may include four tables, a table for each size category, wherein the respective table counts data segments having a size that falls within the respective size category.

[0154] Figure 6 It shows that it can be Figure 1 An entity optionally generates an example of the values ​​of the histogram.

[0155] Entity 1 may be used to generate a histogram for table 2. The histogram includes: sorted different counter values ​​of a plurality of counters in the table; and for each of the different counter values, the number of one or more counters in the plurality of counters having the counter value. The different counter values ​​are indexed such that a minimum counter value has a minimum index and a maximum counter value has a maximum index. The term "bucket" may be used to refer to a counter.

[0156] like Figure 6 As shown in the table of FIG. 1 , the x-axis (horizontal axis) 62 of the histogram may indicate the sorted different counter values ​​of the plurality of counters in the table. Each counter value corresponds to the number of accesses counted by the corresponding counter. The y-axis (vertical axis) 61 of the histogram may indicate the number of one or more of the plurality of counters (which may be abbreviated as parameter “num_buckets”). For example, Figure 6 As shown, in Table 2 of entity 1, there may be a counter 2a that counts zero accesses (access = 0, num_buckets = 1); two counters 2a count one access (access = 1, num_buckets = 2); two counters 2a count two accesses (access = 2, num_buckets = 2); one counter 2a counts three accesses (access = 3, num_buckets = 1), and so on. Figure 6 The values ​​are only examples and thus do not limit the present invention.

[0157] The histogram may include a bar graph for each counter value (i.e., each access count) in the sorted different counter values ​​of the plurality of counters, wherein, for a corresponding counter value in the different counter values, the height of the bar graph may indicate the number (i.e., num_buckets) of one or more counters in the plurality of counters having the corresponding counter value. The entity may be used to calculate a histogram of Table 2 by counting the frequency or number of counters having a specific counter value indicating a specific access count in Table 2. For example, where counter 2a in Table 2 is an 8-bit counter, there may be 256 values ​​on the x-axis of the histogram. Other counters, such as 16-bit counters, may be used.

[0158] Entity 1 can be used to determine an access threshold, beyond which the entity can decide whether to move a data segment (e.g., a file / object or a file / object segment) from a current data storage tier among multiple data storage tiers to another data storage tier among multiple data storage tiers.

[0159] Entity 1 may use a popularity tiering algorithm to assign data segments (e.g., files / objects or segments of files / objects) to multiple data storage tiers so that the most popular data segments reside in the top data storage tier (i.e., the data storage tier with the best performance in writing and / or reading data). For example, the algorithm may iterate over all data (e.g., files / objects) stored in multiple data storage tiers, sort the data by popularity, and assign the data segments of the data to multiple data storage tiers based on the capacity occupied by the data segments (e.g., files / objects or segments of files / objects). When the top data storage tier (i.e., the data storage tier with the best performance in writing and / or reading data) is full (i.e., has no free storage capacity), the algorithm may select the next data storage tier (i.e., the data storage tier with the second best performance in writing and / or reading data) until all data segments are assigned to multiple data storage tiers.

[0160] The above histogram is a compact representation of Table 2 of Entity 1, which is sufficient to calculate the capacity-based counter value threshold.

[0161] The entity 1 may be used to calculate a counter value threshold for each of the plurality of data storage layers 3 for the table 2. The entity 1 may be used to manage the storage of data in the plurality of data storage layers 3 by comparing the counter values ​​of the plurality of counters 2a in the table 2 with the counter value threshold of at least one of the plurality of data storage layers 3. The counter value of the plurality of counters 2a may be understood as the counter value of one of the plurality of counters 2a.

[0162] For example, the entity 1 may manage storing data in the plurality of data storage tiers 3 by comparing each counter value of the plurality of counters 2 a in the table 2 with a counter value threshold of the plurality of data storage tiers 3 .

[0163] The entity 1 may be used to: determine, for a counter in the plurality of counters 2a in Table 2, a data storage layer in the plurality of data storage layers 3 having a maximum counter value threshold, the maximum counter value threshold being less than or equal to the counter value of the counter; and store one or more data segments in the determined data storage layer, the counter counting accesses to the one or more data segments. For example, the entity 1 may perform the following operations: determine, for each counter in the plurality of counters 2a in Table 2, a data storage layer in the plurality of data storage layers 3 having a maximum counter value threshold, the maximum counter value threshold being less than or equal to the counter value of the corresponding counter; and store the data segments in the determined data storage layer, the corresponding counter counting accesses to the one or more data segments.

[0164] The method that can be performed by entity 1 for using the histogram of Table 2 (e.g., Figure 6The histogram shown in Table 2) is a process of calculating the counter value threshold for each of the multiple data storage layers. The process can be defined by the following pseudo code (assuming that the data is a file):

[0165]

[0166] According to the above pseudo code, the sharing factor (sharing_factor) in Table 2 can be calculated by dividing the number of unique files counted by multiple counters 2a in Table 2 (estimated_number_of_files) by the number of counters 2a in Table 2 (total_num_buckets). Entity 1 can be used to determine or estimate the number of unique files counted by multiple counters 2a in Table 2 using the HyperLogLog algorithm. The parameter "category_size" of the above pseudo code can be the average size of the files of Table 2 of Entity 1. For example, a fixed value in the middle of the size range of files counted by Table 2 of Entity 1 can be used as the average size (i.e., it can be a valid estimated value). Entity 1 can be used to calculate the average size (e.g., the exact value of the average size) by iterating a layer file database that can be associated with Table 2 of Entity 1.

[0167] For example, entity 1 may be used to maintain a layer file database for each data storage layer, the layer file database including the inode number of each file or file segment of the data stored in the data storage layer. For each file of the data stored in the data storage layer, in addition to the inode number, the database may also include additional information. The additional information may include a segment offset and / or length.

[0168] According to the above pseudo code for calculating the counter value threshold of the desired data storage layer in the data storage layer 3, entity 1 can perform the following steps. First, entity 1 can set the capacity parameter (cap_par) to zero and set the counter index (bucket_index) to the index of the maximum counter value in the histogram (max_histogram_x_axis_value). Next, as long as the capacity parameter (cap_par) is less than or equal to the capacity (capacity_of_desired_tier) of the desired data storage layer, entity 1 can perform a calculation loop. In each iteration of the calculation loop, the second capacity parameter (cap_par_2) is calculated by multiplying the number (num_buckets) of one or more counters associated with the counter value indexed by the counter index (bucket_index), the sharing factor (sharing_factor) in Table 2, and the average size (category_size) of the file whose access is counted by Table 2. The capacity parameter (cap_par) is set to the second capacity parameter (cap_par_2), and the counter index (bucket_index) is reduced by one. When the capacity parameter (cap_par) is greater than the capacity of the desired data storage tier (capacity_of_desired_tier), the calculation loop is not executed, but the counter value threshold (threshold) is set to the counter value indexed by the counter index (bucket_index). For any type of data, such as files / objects, this process is also Figure 7 It is shown as an example in FIG.

[0169] Figure 7 It shows that it can be Figure 1 A flowchart of an example of a process performed by an entity.

[0170] Figure 7 The process is an example of a process for calculating the counter value threshold for each of the multiple data storage layers 3 in Table 2 using the histogram of Table 2 by iteratively performing the following steps: In a first iteration, the capacity parameter is set to zero and the counter index is set to the index of the maximum counter value in the histogram. This corresponds to the first step S71 of the process. In the next step S72, it is checked whether the capacity parameter is less than or equal to the capacity of the data storage layer. If the capacity parameter is greater than the capacity of the data storage layer ("No" in step S72), the process proceeds to step S73, in which the counter value threshold is set to the counter value (of the histogram) indexed by the counter index, and the process ends.

[0171] If the capacity parameter is less than or equal to the capacity of the data storage layer ("yes" in step S72), the following steps may be performed: In step S74, a second capacity parameter may be calculated by multiplying the number of one or more counters associated with the counter value indexed by the counter index, the sharing factor in Table 2, and the average size of the data segments whose accesses are counted by Table 2. The sharing factor is equal to the number of unique data segments counted by the plurality of counters 2a in Table 2 divided by the number of counters 2a in Table 2. In the next step S75, the capacity parameter is set to the second capacity parameter. In the next step S76, the counter index is decremented by one. The process continues to step S72.

[0172] The sharing factor (sharing_factor) in Table 2 and the average size (category_size) of the data segments whose accesses are counted by Table 2 are parameters associated with the table. The entity 1 may include multiple tables storing multiple counters for managing data stored in multiple data storage layers 3, wherein the tables in the multiple tables are used to count accesses to data segments of different size ranges of the data stored in the multiple data storage layers, so that each table in the multiple tables counts accesses to data segments of the same size range. In this optional case, different parameters may be used for each table entity.

[0173] Entity 1 can be used to generate a histogram for each table in a plurality of tables. The histogram includes: sorted different counter values ​​of a plurality of counters in the table; and for each of the different counter values, the number of one or more counters in the plurality of counters having the counter value. The different counter values ​​are indexed such that a minimum counter value has a minimum index and a maximum counter value has a maximum index.

[0174] Entity 1 can be used to normalize the different counter values ​​in each histogram by: calculating a normalized value, the normalized value being equal to the product of the sharing factor of the table (for which the histogram includes the different counter values) and the average size of the data segments whose accesses are counted by the table; and dividing the different counter values ​​in the histogram by the normalized value.

[0175] At a single node level (i.e., the entity is a single storage node, or is part of a single storage node), each table of different size categories may have different access patterns and different histograms. Entities can be used to perform normalization to combine multiple histograms. To this end, the values ​​of the x-axis of each histogram (the different counter values ​​of multiple counters in the corresponding table) can be divided by the product of the sharing factor in the corresponding table and the average size of the data segment whose access is counted by the corresponding table (i.e., size category). This calculates a normalized score (normalized_score) for each value of the x-axis of each histogram. This enables the calculation of access density.

[0176] For the above normalization, a normalized histogram entry (NHE) with the following fields can be defined, where: Figure 8 An example of a sorted normalized histogram entry list (i.e., a sorted NHE list) is shown:

[0177] The parameter "normalized_score" corresponds to the counter value in the histogram of the corresponding table (the value of the x-axis) divided by the product of the sharing factor (sharing_factor) in the corresponding table and the average size (category_size) of the data segment whose accesses are counted by the corresponding table (normalized_score:=accesses_counter_value / (sharing_factor*category_size)). The parameter "-B_reads" corresponds to the negative value of the counter value in the histogram of the corresponding table (the value of the x-axis) (-B_reads:=–accesses_counter_value). The parameter "num_buckets" corresponds to the number of counters having the corresponding counter value in the histogram of the corresponding table (the value of the y-axis). The parameter "B_capacity" corresponds to the above-mentioned second capacity parameter (cap_par_2) in the histogram of the corresponding table (B_capacity:=num_buckets*sharing_factor*category_size). The parameter "histogram_factor" in the histogram of the corresponding table corresponds to the product of the sharing factor (sharing_factor) in the corresponding table and the average size (category_size) of the data segments whose accesses are counted by the corresponding table (histogram_factor:=sharing_factor*category_size). The parameter "histogram_id" can be a unique identifier (histogram id) in the histogram of the corresponding table.

[0178] Entity 1 can be used to track the number of counters and parameters in each histogram of a histogram of multiple tables using NHE to track the capacity reflected by the current global threshold until the data storage layer is full.

[0179] Entity 1 can be used to calculate a list of NHEs for each histogram, cascade the calculated lists, and sort the cascaded lists in descending order by the parameter "normalized_score". This list of entries can be in popularity preference order, and Entity 1 can assign this list of entries to the data storage layer (using a calculation loop similar to the loop in the pseudocode above) by, for example, the matching capacity percentile of the data storage layer, thereby generating an index in the sorted list that points to the final element to be assigned to the data storage layer.

[0180] Since each histogram and the corresponding table in the multiple tables have different activity patterns and scores, the entity can assign a matching threshold to each of them (e.g., their corresponding maximum bucket index (relative to their histogram)). An example of this can be described exemplarily with respect to Figure 8 the sorted list of normalized histogram entries.

[0181] Figure 8 An example of the sorted list of normalized histogram entries according to an embodiment of the present invention is shown.

[0182] Assume that Entity 1 includes four tables (e.g., for data segments of four different size ranges): Table a, Table b, Table c, and Table d, then generate Figure 8 the NHE entries in the sorted list. The histogram ids "histogram_id" of the histograms of Table a, Table b, Table c, and Table d can be "a", "b", "c", and "d" respectively (as shown in the second to last rows of the table in Figure 8 .

[0183] For example, with respect to Figure 8 , select index 5 as the counter value threshold for the top data storage layer (i.e., the data storage layer with the optimal performance among multiple data storage layers) (e.g., due to its capacity Ctop, where this capacity Ctop can be in the range of 80 < Ctop < 170), which may result in the following thresholds for each table: for Table a, the threshold can be 4 (a: 4); for Table b, the threshold can be 2 (b: 2); for Table d, the threshold can be 2 (d: 2); and for Table c, the threshold can be infinity (c: infinity). Query access to data segments (e.g., files) that generate more or equal access counts may result in their assignment to the top data storage layer (i.e., the data storage layer with the optimal performance among multiple data storage layers).

[0184] Fig. 9 It shows that it can be Figure 1 A flowchart of an example of a process performed by an entity.

[0185] like Fig. 9 As shown, assuming that entity 1 includes multiple tables storing multiple counters (as described above), entity 1 can be used to calculate the counter value threshold for each of the multiple data storage layers 3 for the multiple tables using the multiple histograms by performing the following steps: In a first step S91, entity 1 can sort all normalized counter values ​​in the multiple histograms. In a next step S92, entity 1 can calculate the capacity percentiles of the multiple data storage layers 3. In a next step S93, for each table, entity 1 can calculate the counter value threshold for each of the multiple data storage layers 3 using the sorted normalized counter values ​​in the multiple histograms and the calculated capacity percentiles.

[0186] An entity can be used to perform a globally distributed algorithm: Entity 1 can be used to calculate a global threshold by sharing its optional multiple tables' histograms with one or more other corresponding entities (e.g., each entity includes multiple tables). In this case, the entity and the other one or more corresponding entities can be storage nodes. Each storage node can be used to calculate its threshold independently. Alternatively, a cluster leader can collect the histograms, calculate, and broadcast the results.

[0187] For example, the algorithm for calculating the threshold by the cluster leader may include: collecting access histograms of all size categories from all storage nodes. Normalizing based on the size category (category_size) and the sharing factor (e.g., category-unique-files-estimate) to calculate the average access (access density) to each block. Next, sorting by size normalized counter (normalized_score); calculating the capacity percentile (e.g., P99, P95, P90) that matches the desired data storage tier; and calculating the threshold for each table in multiple tables and assigning it to the corresponding storage node.

[0188] The exact threshold may depend on how it is used, such as which decision to make about a data segment (e.g., file / object) (up-tiering vs. down-tiering), or whether to promote (move to a higher data storage tier, i.e., a tier with better performance) the entire data segment or only a portion of it (e.g., only the block that is accessed the next time it is accessed). These decisions may have corresponding weight factors that modify the thresholds. For example, if T99 is the threshold for the top data storage tier, then 0.9*T99 may be the threshold for up-tiering, and 0.8*T99 may be the threshold for down-tiering. Under the term "up-tiering", it may be understood as moving a data segment from a data storage tier to a data storage tier with better performance. Under the term "down-tiering", it may be understood as moving a data segment from a data storage tier to a data storage tier with worse performance.

[0189] An example of steps for estimating tier candidates in data stored in a plurality of data storage tiers is described below, wherein the entity 1 may be used to perform one or more of these steps.

[0190] Entity 1 can use access activity of data stored in multiple data storage tiers 3 as a trigger to select candidates for upward tiering of the data to estimate whether to move a data segment (e.g., a file segment) in the data from a current data storage tier in which the data segment is stored to a higher data storage tier.

[0191] To select candidates for down-tiering, activity triggers for specific data segments (e.g., files / objects and / or file / object segments) may not be used because such data segments are typically inactive data segments (i.e., they are not accessed). Entity 1 may be configured to periodically scan data (e.g., all files / objects and / or file / object segments) stored in one of the plurality of data storage tiers.

[0192] Because data segments (e.g., files / objects) stored in high data storage tiers have faster access speeds, there is generally no reason to move them to a slower tier unless the current data storage tier in which they are located is full or reaches a certain space utilization threshold (e.g., 97%). Other reasons for down-tiering may be user preferences or future usage prompts. Regardless of the reason, entity 1 can be used to scan data (e.g., files / objects). Based on the above, entity 1 can be used to determine one or more data segments of data stored in a data storage tier for moving to an adjacent data storage tier with larger capacity by: determining that the capacity of a data storage tier is full or has reached a capacity threshold; receiving instructions for moving one or more data segments to an adjacent data storage tier; and / or receiving information about future reduced access activity for one or more data segments.

[0193] To iterate over files (e.g., inactive files), entity 1 may be used to maintain an index called a layer file database that includes the inode numbers (and optionally the fragment offset lengths) of those files that have fragments in the source data storage layer (i.e., the data storage layer from which the down-layering is performed). Entity 1 may be used to track a list of all fragments for each data storage layer in this structure. For example, by limiting the number of layer fragments, a list of all fragments for each data storage layer may be tracked in this structure.

[0194] Entity 1 can be used to update the tier file database when a file is created / deleted / extended (e.g., whenever the inode changes). For example, the tier file database can use a log-structured merge-tree (LSM) structure, which can reside on a data memory (e.g., a small write buffer different from RAM) and can be optimized for random writes and sequential reads. Scanning this index structure is much faster than iterating over the file system's internal inode table, which may include more information / metadata that is not used for tiering decisions.

[0195] Entity 1 may periodically perform a scan (loop read) to estimate tier candidates (exhaustive / greedy). For each file / object fragment, entity 1 may query table 2 storing multiple counters 2 and obtain the number of access operations recorded in counter 2a (shared counter) tracking the file / object. If the value of the counter is below (less than) the threshold of the table, entity 1 may allocate the file / object for downward tiering.

[0196] When a file / object is being accessed, entity 1 can query table 2 storing multiple counters 2a and obtain the number of access operations recorded in the counter (shared counter) tracking the file / object. If the value of the counter is equal to or higher than (greater than) the threshold of the table (e.g., calculated in the last scan), the entity can allocate the file / object allocation for upward tiering.

[0197] Fig.10 A flow chart showing an example of a method for managing data stored in multiple data storage tiers according to an embodiment of the present invention. Fig.10 The method is an example of the method according to the second aspect of the present invention. Therefore, the description of the method according to the second aspect applies accordingly to Fig.10 method.

[0198] Fig.10The method is a method for managing data stored in multiple data storage layers using a table storing multiple counters. The method includes the following steps: In a first step S101, the method may include: each of the multiple counters counts accesses to one or more data segments of data stored in the multiple data storage layers. In a next step S102, the method may include: using the table to decide in which data storage layer of the data storage layer to store the data segment of the data. Figure 1 The entity can be used to perform Fig.10 method.

[0199] In the present invention, the description about files can be correspondingly applied to the case where data is or includes objects in addition or alternatively. The description about objects can be correspondingly applied to the case where data is or includes files in addition or alternatively.

[0200] The present invention has been described in conjunction with various embodiments as examples and in conjunction with implementations. However, a person skilled in the art and practicing the subject matter for which protection is sought will be able to understand and implement other variations by studying the drawings, the present invention and the independent claims. In the claims and in the specification, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single element or other unit may fulfil the functions of several entities or items described in the claims. The enumeration of certain measures in mutually different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation.

Claims

1. An entity for managing data stored in multiple data storage layers, characterized in that The entity includes a table storing a plurality of counters; Each counter of the plurality of counters is used to count accesses to one or more data segments of the data stored in the plurality of data storage layers; as well as The entity is configured to use the table to determine in which data storage tier of the plurality of data storage tiers to store the data segment of the data.

2. The entity according to claim 1, characterized in that The plurality of data storage tiers differ from one another in storage capacity and performance with respect to writing and / or reading data.

3. The entity according to claim 1 or 2, characterized in that At least two counters of the plurality of counters are used to count accesses of different numbers of data segments of one or more data segments of the data stored in the plurality of data storage layers.

4. Entity according to any of the preceding claims, characterized in that At least one of the plurality of counters is used to count accesses to a plurality of data segments of the data stored in the plurality of data storage layers by using a hash function that maps the plurality of data segments to the at least one counter.

5. Entity according to any of the preceding claims, characterized in that The entity is for counting a number of one or more unique data segments counted by a counter of the plurality of counters in the table.

6. The entity according to claim 5, characterized in that The entity is used to calculate the number of one or more unique data segments using a HyperLogLog algorithm.

7. Entity according to any of the preceding claims, characterized in that the entity for calculating, for the table, a counter value threshold for each of the plurality of data storage tiers; and The entity is used to manage storing the data in the plurality of data storage tiers by comparing the counter values ​​of the plurality of counters in the table with a counter value threshold of at least one of the plurality of data storage tiers.

8. The entity according to claim 7, characterized in that The entity is configured to: determine, for a counter among the plurality of counters in the table, a data storage tier among the plurality of data storage tiers having a maximum counter value threshold, the maximum counter value threshold being less than or equal to a counter value of the counter; And the one or more data segments are stored in a determined data storage layer, and the counter counts accesses to the one or more data segments.

9. The entity according to claim 7 or 8, characterized in that The entity is used to generate a histogram for the table, where The histogram includes: sorted different counter values ​​of the plurality of counters in the table; and for each of the different counter values, a number of one or more counters of the plurality of counters having the counter value; and The different counter values ​​are indexed such that a minimum counter value has a minimum index and a maximum counter value has a maximum index.

10. The entity according to claim 9, characterized in that The entity is configured to calculate a counter value threshold for each of the plurality of data storage tiers for the table using the histogram by iteratively performing the following steps: in a first iteration setting a capacity parameter to zero and setting a counter index to the index of the maximum counter value in the histogram; Checking whether the capacity parameter is less than or equal to the capacity of the data storage layer; If the capacity parameter is greater than the capacity of the data storage layer, setting the counter value threshold to a counter value indexed by the counter index; as well as If the capacity parameter is less than or equal to the capacity of the data storage layer: calculating a second capacity parameter by multiplying a number of one or more counters associated with the counter value indexed by the counter index, a sharing factor in the table, and an average size of data segments whose accesses are counted by the table, wherein the sharing factor is equal to a number of unique data segments counted by the plurality of counters in the table divided by the number of counters in the table; setting the capacity parameter to the second capacity parameter; and Decrement the counter index by one.

11. Entity according to any of the preceding claims, characterized in that The entity includes a plurality of tables storing a plurality of counters; and A table among the plurality of tables is used to count accesses to data segments of different size ranges of the data stored in the plurality of data storage layers, such that each table among the plurality of tables counts accesses to data segments of the same size range.

12. The entity according to claim 11, characterized in that The entity is used to generate a histogram for each of the plurality of tables, wherein: The histogram includes: sorted different counter values ​​of the plurality of counters in the table; and for each of the different counter values, a number of one or more counters of the plurality of counters having the counter value; and The different counter values ​​are indexed such that a minimum counter value has a minimum index and a maximum counter value has a maximum index.

13. The entity according to claim 12, characterized in that The entity is used to normalize the different counter values ​​in each histogram by: calculating a normalized value equal to a product of a sharing factor of a table for which the histogram includes the different counter values ​​and an average size of data segments whose accesses are counted by the table; and The different counter values ​​in the histogram are divided by the normalized value.

14. The entity according to claim 13, characterized in that The entity is configured to calculate a counter value threshold for each of the plurality of data storage tiers for the plurality of tables using the plurality of histograms by performing the following steps: sorting all normalized counter values ​​in the plurality of histograms; Calculating capacity percentiles of the plurality of data storage tiers; as well as For each table, a counter value threshold for each of the plurality of data storage tiers is calculated using the sorted normalized counter values ​​in the plurality of histograms and the calculated capacity percentiles.

15. Entity according to any of the preceding claims, characterized in that The entity is used to determine one or more data segments of data stored in a data storage layer for moving to an adjacent data storage layer with a larger capacity by: Determining that the capacity of the data storage layer is full or has reached a capacity threshold; receiving an instruction to move the one or more data segments to the adjacent data storage layer; and / or Information regarding future reduced access activity for the one or more data segments is received.

16. The entity according to claim 15, characterized in that The entity is used to: maintaining a database for the data storage layer, the database comprising an inode number of each data segment of the data stored in the data storage layer; and The one or more data segments for moving to the adjacent data storage tier are determined by scanning the database.

17. Entity according to any of the preceding claims, characterized in that The entity is used to move the data segment to an adjacent data storage layer with a larger capacity when a counter value of a counter counting data segments of data stored in the data storage layer is less than a counter value threshold of the data storage layer.

18. Entity according to any of the preceding claims, characterized in that The entity is for determining one or more data segments of data stored in a data storage tier for movement to an adjacent data storage tier with smaller capacity by determining information about access activity of the one or more data segments being greater than an access activity threshold.

19. Entity according to any of the preceding claims, characterized in that The entity is used to move the data segment to the adjacent data storage layer with smaller capacity when the counter value of the counter counting the data segment of the data stored in the data storage layer is greater than or equal to the counter value threshold of the adjacent data storage layer.

20. A method for managing data stored in a plurality of data storage layers using a table storing a plurality of counters, characterized in that, Each counter of the plurality of counters counts accesses to one or more data segments of the data stored in the plurality of data storage tiers; as well as The table is used to determine in which of the plurality of data storage tiers the data segment of the data is stored.