Data storage method and device and electronic equipment

Abstract

The invention relates to a data storage method and device and electronic equipment. The data storage method comprises the following steps: respectively counting access frequency information of each data item stored on a first storage node; periodically summarizing the access frequency information of each data item stored on the first storage node through the coordination node, and determining an access heat value of each data item stored on the first storage node; according to the access heat value of each data item stored on the first storage node, determining a to-be-migrated target data item and a target storage hierarchy adapted to the target data item; and migrating and storing the target data item to a second storage node of the target storage hierarchy, wherein the plurality of storage nodes comprise the first storage node and the second storage node. The method can improve the access efficiency of the stored data.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data storage method, apparatus, and electronic device. Background Technology

[0002] With the rapid development of technologies such as artificial intelligence and the Internet of Things, the amount of statistical data generated and processed by various industries is exploding. How to store and manage this massive amount of data is crucial. Related technologies mainly employ tiered data storage, dividing storage media into different performance levels. For example, high-speed but expensive solid-state drives (SSDs) are used as the hot data layer, while low-speed but inexpensive hard disk drives (HDDs) are used as the cold data layer. The basic idea is to store frequently accessed data (hot data) in the high-performance storage layer and infrequently accessed data (cold data) in the low-cost storage layer. This solution uses a timestamp-based static or semi-static strategy; for example, data older than 90 days is defined as cold data. The reliability of this solution heavily depends on the stability of business access patterns. Once data access patterns change unexpectedly—for example, if old event data is accessed frequently again due to sudden demand—this rigid strategy cannot respond in time, causing hot data to be incorrectly placed in the low-performance cold data layer, resulting in low data access efficiency. Summary of the Invention

[0003] The present invention aims to provide a data storage method, apparatus and electronic device to overcome the shortcomings of the prior art. The technical problem to be solved by the present invention is achieved through the following technical solutions.

[0004] According to a first aspect of this disclosure, a data storage method is provided, applied to a distributed storage system including multiple storage nodes and a coordinating node, wherein the multiple storage nodes belong to multiple storage tiers with different performance levels, the method comprising: The access frequency information of each data item stored on the first storage node is statistically analyzed. The coordination node periodically summarizes the access frequency information of each data item stored on the first storage node to determine the access popularity value of each data item stored on the first storage node. Based on the access popularity values ​​of each data item stored on the first storage node, the target data item to be migrated and the target storage level that matches the target data item are determined. The target data item is migrated and stored to a second storage node of the target storage level, wherein the plurality of storage nodes include the first storage node and the second storage node.

[0005] Further, determining the target data item to be migrated and the target storage level adapted to the target data item based on the access popularity value of each data item stored on the first storage node includes: Based on the access popularity value, data size, and time decay factor of each data item stored on the first storage node, the migration score of each data item stored on the first storage node is determined. Based on the migration scores of each data item stored on the first storage node, the capacity limits of each storage level, and the heat threshold, the target data item to be migrated and the target storage level that matches the target data item are determined.

[0006] Furthermore, the step of separately calculating the access frequency information of each data item stored on the first storage node includes...

[0007] The sliding window algorithm is used to statistically analyze the access frequency information of each data item stored on the first storage node within a preset time window.

[0008] Furthermore, the heat threshold is periodically determined based on the access heat value of each data item stored on the multiple storage nodes within the preset time window.

[0009] Furthermore, the multiple storage tiers with different performance levels include a memory cache layer, a solid-state drive storage layer, and a hard disk drive storage layer, and the method further includes: Receive data access requests; If the pre-accessed data item in the data access request is stored in the mechanical hard disk storage layer, the pre-accessed data item is asynchronously promoted to the solid-state drive storage layer or the memory cache layer; Based on the data access request, the pre-accessed data item is returned.

[0010] Furthermore, the memory caching layer is implemented based on Caffeine, and after returning the pre-accessed data item based on the data access request, the method further includes: Update the access frequency information of the pre-accessed data item in the memory cache layer.

[0011] According to a second aspect of this disclosure, a data storage device is provided, the device comprising: The first statistics module is used to count the access frequency information of each data item stored on the first storage node. The first determining module is used to periodically summarize the access frequency information of each data item stored on the first storage node through the coordination node, and determine the access popularity value of each data item stored on the first storage node. The second determining module is used to determine the target data item to be migrated and the target storage level that matches the target data item based on the access popularity value of each data item stored on the first storage node. A first storage module is used to migrate and store the target data item to a second storage node of the target storage level, wherein the plurality of storage nodes include the first storage node and the second storage node.

[0012] According to a third aspect of this disclosure, an electronic device is provided. The electronic device includes a memory and a processor, wherein a computer program is stored on the memory, and the processor executes the computer program to implement the method.

[0013] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method.

[0014] According to a fifth aspect of this disclosure, a computer program product is provided, including computer instructions that, when executed by a processor, implement the method.

[0015] The embodiments of the present invention have the following advantages: The data storage method provided in this invention is applied to a distributed storage system comprising multiple storage nodes and a coordinating node. The multiple storage nodes belong to multiple storage tiers with different performance levels. The method includes: statistically analyzing the access frequency information of each data item stored on a first storage node; periodically summarizing the access frequency information of each data item stored on the first storage node through the coordinating node to determine the access popularity value of each data item stored on the first storage node; determining the target data item to be migrated and the target storage tier adapted to the target data item based on the access popularity value of each data item stored on the first storage node; and migrating the target data item to a second storage node of the target storage tier. The multiple storage nodes include the first storage node and the second storage node. This method, by not relying on fixed time thresholds to define hot and cold data, but by statistically analyzing the real-time access frequency of data items on the first storage node and combining it with the periodically summarized access popularity value by the coordinating node, dynamically reflects the current popularity of data and can capture sudden access events. Based on real-time heat values, the data to be migrated and the target level are determined. When the data access pattern changes unexpectedly, the storage level and the real-time heat of the data can be accurately matched, thereby improving the flexibility and access efficiency of storage management of massive data. Attached Figure Description

[0016] Figure 1 This is one of the step flowcharts of an embodiment of the data storage method of the present invention; Figure 2 This is a second flowchart of an embodiment of a data storage method of the present invention; Figure 3 This is the third step in a flowchart of an embodiment of a data storage method of the present invention; Figure 4 This is the fourth step of a flowchart illustrating an embodiment of a data storage method of the present invention; Figure 5 This is a schematic diagram of the structure of an embodiment of a data storage device according to the present invention; Figure 6 This is a schematic diagram of the structure of an electronic device according to the present invention. Detailed Implementation

[0017] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0018] Figure 1 A flowchart of a data storage method 100 according to an embodiment of the present disclosure is shown. This data storage method is applied to a distributed storage system including multiple storage nodes and a coordinating node. The multiple storage nodes belong to multiple storage tiers with different performance levels. Specifically, the method 100 includes: Step 101: Calculate the access frequency information of each data item stored on the first storage node; In some embodiments, the step of separately calculating the access frequency information of each data item stored on the first storage node includes... The sliding window algorithm is used to statistically analyze the access frequency information of each data item stored on the first storage node within a preset time window.

[0019] In the above embodiments, taking the first storage node among multiple storage nodes as an example, the access frequency information can be understood as the number of times a certain data item is read or modified by a user or business system within a certain period of time, which is the core indicator for judging the "hotness" or "coldness" of data. The sliding window algorithm is a method for calculating a dynamic statistical time range. It does not fix the statistical time period such as "0:00-24:00 on a certain day", but like a sliding window, it continuously updates the statistical range at fixed time intervals (such as 1 minute) to ensure that the statistical results can reflect the real-time situation of "recent period of time" and avoid judgment bias due to outdated time range.

[0020] In a distributed storage system, a sliding window algorithm is used to count the access frequency of data on the first storage node. A fixed time range (e.g., 1 hour) is initially set as the window. As time progresses, the window continuously moves to the right (e.g., once per minute), constantly counting the number of times each data item on the first storage node has been accessed within the most recent time period.

[0021] See Figure 2 , Figure 2 This demonstrates the complete process of processing access frequency information for each data item in a distributed storage system. The core objective is to address the insufficient accuracy and synchronization latency issues in access frequency statistics within a distributed environment, ensuring that migration decisions are supported by accurate global data. The specific process consists of four stages: Local statistics phase: Node 1, Node 2, and Node 3 in the node cluster (corresponding to storage nodes at different storage levels in the system) each maintain local access counts for data items (such as access frequency counted through a sliding window algorithm), avoiding statistical dependence on cross-node interaction and reducing real-time synchronization pressure.

[0022] Scheduled reporting phase: Each node reports local statistical summaries to the coordinating node at fixed intervals (e.g., every 10 minutes). The summaries are not full data, which can reduce network transmission volume and ensure that statistical data can be summarized periodically, balancing timeliness and resource consumption.

[0023] Coordination node processing phase: After receiving the digest, the coordination node first verifies the validity of the data, such as checking the timestamp, removing expired data, and then temporarily storing it according to the time window; then it merges and counts, that is, deduplicating duplicate key values, aggregating counts, completing missing node data to achieve time window alignment, generating a global access statistics view, and performing hotspot analysis, that is, calculating the TOP-N frequently accessed resources and persisting historical statistical data.

[0024] Global synchronization phase: The coordinating node broadcasts the generated global view to all nodes, enabling each node to grasp the overall data access popularity of the system, providing a unified basis for subsequent migration decisions, cache eviction, etc., and ensuring that the statistical accuracy in a distributed environment is not affected by the number of nodes.

[0025] In this embodiment, the sliding window can reflect the latest access frequency of data in real time, avoiding window updates over time and the lag of static statistics. Statistics are performed separately for each data item, ensuring that the popularity assessment of each data item is independent and accurate. By employing the sliding window algorithm to separately count the access frequency information of each data item stored on the first storage node within a preset time window, a reliable basis can be provided for subsequent calculation of migration scores and judgment of data popularity, allowing for more rational allocation of storage resources.

[0026] Step 102: The access frequency information of each data item stored on the first storage node is periodically summarized by the coordination node to determine the access popularity value of each data item stored on the first storage node. In some embodiments, the coordinating node acts as the global manager of the distributed storage system, responsible for collecting and integrating the access frequencies of various data items scattered across the first storage node. Periodicity refers to the aggregation operation being performed at fixed time intervals. The access popularity value can be understood as a quantitative standard for the "hotness" or "coldness" of data. The coordinating node periodically collects access frequency information for each data item on the first storage node and calculates the access popularity value for each data item through aggregation.

[0027] For example, the first storage node is an SSD node in an e-commerce system, storing data for three products. The coordinating node summarizes this node's data every morning at midnight: Product A was accessed 200 times in the past 24 hours, Product B was accessed 50 times, and Product C was accessed 5 times. Based on preset rules (such as directly mapping access counts to popularity values), popularity values ​​are calculated: A = 200, B = 50, and C = 5. These values ​​will be used to determine which storage level the data should be stored at.

[0028] Step 103: Based on the access popularity value of each data item stored on the first storage node, determine the target data item to be migrated and the target storage level that matches the target data item; In some embodiments, determining the target data item to be migrated and the target storage level adapted to the target data item based on the access popularity value of each data item stored on the first storage node includes: Based on the access popularity value, data size, and time decay factor of each data item stored on the first storage node, the migration score of each data item stored on the first storage node is determined. Based on the migration scores of each data item stored on the first storage node, the capacity limits of each storage level, and the heat threshold, the target data item to be migrated and the target storage level that matches the target data item are determined.

[0029] See Figure 3 Access popularity value represents the current popularity of a data item; the lower the access popularity value, the lower the migration score. Data size represents the storage resources occupied by a data item; the larger the data, the lower the migration score. The time decay factor is a parameter used in distributed storage systems to reduce the influence of historical access popularity on the current value judgment of data. That is, the older the data access behavior, the lower its reference value for current storage decisions. The older the data popularity, the larger the time decay factor, and the higher the decayed value (i.e., w3). The lower the (1-time decay factor), the lower the migration score. You can obtain the access popularity value, data size, and time decay factor for each data item, and then calculate the weighted sum of these factors to obtain the migration score for each data item. For example: Migration score = w1 Popularity value -w2 Data size + w3 (1-Time decay factor). Among them, the higher the access popularity value and the smaller the data, the more likely it is to migrate to the hot layer.

[0030] In some embodiments, the migration score for each data item can be calculated by acquiring its access popularity value, data size, time decay factor, and business priority tag, and then weighting and summing these parameters. The "business priority tag" refers to a pre-defined priority identifier or value for each data item based on business needs and importance. Essentially, it is a manually or system-defined label used to reflect the importance of a data item at the business level. When calculating the migration score, this tag is converted into specific weights to participate in the calculation, allowing data that is more important to the business to receive a higher migration priority.

[0031] A popularity threshold is a core standard used to determine which storage level data belongs to, such as a hot data threshold or a warm / cold data threshold. For example, if the popularity threshold is set to 80, then data with a popularity value greater than 80 is classified as hot data, and data with a popularity value less than 80 is classified as cold data.

[0032] Compare the migration score of each data item on the first storage node with the popularity threshold. If the migration score is greater than or equal to the popularity threshold, the data popularity meets the standard and is eligible to migrate to a higher popularity level. For data that meets the migration eligibility criteria, check whether the remaining capacity of each high-population level is greater than or equal to the data size (i.e., does not exceed the capacity limit). If the remaining capacity of a certain level is sufficient, then that level becomes the target storage level; if the target level is full (i.e., the capacity limit has been reached), then the next highest popularity level is selected as the next highest popularity level.

[0033] For example, taking a video storage scenario, the distributed storage system includes a memory caching layer (hot layer), a solid-state drive storage layer (warm layer), and a hard disk drive storage layer (cold layer). Wherein: Hot layer: Capacity limit is 100GB, currently 85GB used (15GB remaining), heat-warmth threshold is 80; Temperature layer: Capacity limit is 500GB, 400GB currently used (100GB remaining), temperature-cold threshold is 30; Cold layer: Capacity limit is 2TB, currently 1.2TB is used (800GB remaining); First storage node: Node X of the cold layer (currently storing 3 video data: A, B, C).

[0034] First, clarify the transfer scores for each data point: The system calculates the migration scores of the three videos on node X (higher scores indicate higher popularity and better fit to higher-level content): Video A: Migration score 88 (Recent surge in visits, high popularity); Video B: Migration score 45 (occasional visits, moderate popularity); Video C: Migration score 25 (Long period of inactivity, low popularity).

[0035] Secondly, filter the data to be migrated: Video A (88) ≥ thermal-temperature threshold (80): qualifies for migration to thermal layer; Video B (45) ≥ warm-cold threshold (30) and < hot-warm threshold (80): qualified for migration to the thermosphere; Video C (25) < warm-cold threshold (30): No need to migrate upwards, continue to stay in the cold layer.

[0036] Finally, determine the target storage tier: Video A: The target layer is selected as the hot layer. If the remaining capacity is 15GB or more and the size of Video A is 10GB, and it does not exceed the 100GB capacity limit, the target layer is determined to be the hot layer. Video B: The target layer is the Wen layer. The remaining capacity is 100GB or more, which is greater than or equal to 8GB of the size of Video B, and does not exceed the 500GB capacity limit. The target layer is determined to be the Wen layer.

[0037] In this embodiment, by combining access popularity, data size, and time decay factor (to avoid interference from outdated popularity, such as the natural decrease in popularity of old data), misjudgment based on a single dimension is avoided, thus improving the accuracy of migration score calculation. Furthermore, by comprehensively considering migration score, capacity limit, and popularity threshold, a migration strategy for data items is determined. This ensures that hot data can be migrated to the high-speed layer to improve access efficiency, while avoiding resource waste and preventing cold data from occupying expensive storage, thus balancing system performance, cost, and resource utilization.

[0038] In some embodiments, the heat threshold is periodically determined based on the access heat value of each data item stored on the plurality of storage nodes within the preset time window.

[0039] In the above embodiments, to avoid the threshold from becoming disconnected from actual access conditions due to relying on fixed values, the popularity threshold is dynamically adjusted in real time. Specifically, it can be determined based on the access popularity values ​​of each data item stored on multiple storage nodes within a preset time window of the current time period. Then, the popularity values ​​of all collected data items are sorted, quantiles are calculated, or means are analyzed. Based on the storage level requirements, a dividing line is extracted from the statistical results as the popularity threshold. For example, if the distributed storage system includes three storage levels, then two popularity thresholds need to be determined.

[0040] In this embodiment, to avoid the inability of fixed thresholds to cope with business fluctuations, the thresholds are updated periodically to align with real-time heat distribution, ensuring that hot data is stored in the high-speed layer and cold data in the low-cost layer. Based on the global heat value statistical threshold, data layers can be accurately divided, reducing the occupation of the high-speed layer by low-frequency data, while avoiding the idleness of the cold layer, thus balancing performance and cost.

[0041] Step 104: Migrate and store the target data item to the second storage node of the target storage level, wherein the plurality of storage nodes include the first storage node and the second storage node.

[0042] In this step, please refer to [link / reference]. Figure 3 The distributed storage system will, according to the migration plan, completely replicate the target data item from its current storage node (i.e., the first storage node mentioned above) to the second storage node at the target storage level via network transmission. After the target data item is completely written to the second storage node and verified to be correct, the distributed storage system will update the metadata of the target data item (recording that the data has been migrated to the second storage node) and delete the old data on the original storage node (i.e., the first storage node) as needed, completing the entire migration and deployment process.

[0043] In some embodiments, the plurality of storage tiers with different performance levels include a memory cache tier, a solid-state drive (SSD) storage tier, and a hard disk drive (HDD) storage tier, and the method further includes: Receive data access requests; If the pre-accessed data item in the data access request is stored in the mechanical hard disk storage layer, the pre-accessed data item is asynchronously promoted to the solid-state drive storage layer or the memory cache layer; Based on the data access request, the pre-accessed data item is returned.

[0044] In the above embodiments, see Figure 4 After receiving an external data access request, the distributed storage system prioritizes retrieving data from high-speed storage layers to improve response speed, querying pre-accessed data items from various storage levels. Specifically: it first checks the memory cache layer; if the pre-accessed data item is found, it is returned directly to the requester, and the access frequency of that pre-accessed data item is updated. If the memory cache layer is not found, it queries the solid-state drive (SSD) storage layer; if found, the data is returned and the access frequency is updated. If the SSD storage layer is still not found, it queries the hard disk drive (HDD) storage layer. If a match is found, it means that previously infrequently accessed cold data has been accessed. If previously infrequently accessed cold data has been accessed frequently, in addition to returning the data, an asynchronous data upscaling process is initiated. This involves migrating the data to a more frequently accessed layer according to a preset strategy, such as to the SSD storage layer or directly to the memory cache layer, to ensure that it can be retrieved from the high-speed layer on the next access, reducing query latency.

[0045] In this embodiment, hierarchical querying ensures efficient access to hot data, while dynamic optimization of storage hierarchy through cold data warming balances performance and resource utilization.

[0046] In some embodiments, the memory caching layer is implemented based on Caffeine, and after returning the pre-accessed data item based on the data access request, the method further includes: Update the access frequency information of the pre-accessed data item in the memory cache layer.

[0047] In the above embodiments, the memory cache layer of the distributed storage system is implemented using the Caffeine framework. When the system responds to a data access request and returns a pre-accessed data item from the memory cache, it will immediately update the access frequency information of the data item in the memory cache layer.

[0048] In this embodiment, Caffeine relies on real-time access frequency to determine the cache eviction policy. Timely updated frequency information allows the cache layer to accurately identify "real hot data," avoiding the mistaken eviction of high-frequency access data, while promptly cleaning up low-frequency access cold data to improve memory utilization.

[0049] According to embodiments of this disclosure, the access frequency information of each data item stored on a first storage node is statistically analyzed; the access frequency information of each data item stored on the first storage node is periodically summarized by the coordination node to determine the access heat value of each data item stored on the first storage node; based on the access heat value of each data item stored on the first storage node, the target data item to be migrated and the target storage level adapted to the target data item are determined; the target data item is migrated and stored to a second storage node of the target storage level, wherein the plurality of storage nodes includes the first storage node and the second storage node. In this embodiment, by not relying on fixed time thresholds to define hot and cold data, but by statistically analyzing the real-time access frequency of data items on the first storage node and combining the access heat value periodically summarized by the coordination node, the current heat of data is dynamically reflected, which can capture sudden access situations. Based on the real-time heat value, the data to be migrated and the target level are determined. When the data access pattern changes unexpectedly, the storage level and the real-time heat of data can be accurately matched, thereby improving the flexibility and access efficiency of storage management of massive data.

[0050] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0051] The above is an introduction to the method embodiments. The following describes the solution described in this disclosure further through device embodiments.

[0052] Figure 5 A block diagram of a data storage device 500 according to an embodiment of the present disclosure is shown. The device 500 includes: The first statistics module 501 is used to count the access frequency information of each data item stored on the first storage node. The first determining module 502 is used to periodically summarize the access frequency information of each data item stored on the first storage node through the coordination node, and determine the access popularity value of each data item stored on the first storage node. The second determining module 503 is used to determine the target data item to be migrated and the target storage level that is compatible with the target data item based on the access popularity value of each data item stored on the first storage node. The first storage module 504 is used to migrate and store the target data item to a second storage node of the target storage level, wherein the plurality of storage nodes include the first storage node and the second storage node.

[0053] Further, the second determining module 503 includes: The first determining unit is used to determine the migration score of each data item stored on the first storage node based on the access popularity value, data size and time decay factor of each data item stored on the first storage node. The second determining unit is used to determine the target data item to be migrated and the target storage level that is compatible with the target data item based on the migration score of each data item stored on the first storage node, the capacity limit value of each storage level and the heat threshold.

[0054] Furthermore, the first statistical module 501 includes

[0055] The first statistical unit is used to use a sliding window algorithm to count the access frequency information of each data item stored on the first storage node within a preset time window.

[0056] Furthermore, the heat threshold is periodically determined based on the access heat value of each data item stored on the multiple storage nodes within the preset time window.

[0057] Furthermore, the plurality of storage tiers with different performance levels include a memory cache tier, a solid-state drive (SSD) storage tier, and a hard disk drive (HDD) storage tier, and the device further includes: The first receiving module is used to receive data access requests; The first boosting module is used to asynchronously boost the pre-accessed data item in the data access request to the solid-state drive storage layer or the memory cache layer when the pre-accessed data item in the data access request is stored in the mechanical hard disk storage layer. The first return module is used to return the pre-accessed data item based on the data access request.

[0058] Furthermore, the memory cache layer is implemented based on Caffeine, and the device further includes: The first update module is used to update the access frequency information of the pre-accessed data item in the memory cache layer.

[0059] Figure 6 This example illustrates a schematic diagram of the physical structure of an electronic device, which can be a smart terminal. Its internal structure diagram can be as follows: Figure 6 As shown, this electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the aforementioned data storage method.

[0060] On the other hand, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the above-described data storage method.

[0061] In another aspect, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and when the processor executes the computer instructions, it implements the aforementioned data storage method.

[0062] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0063] By way of illustration and not limitation, Random Access Memory (RAM) is available in a variety of forms, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus Direct RAM (RDRAM), Direct Rambus Dynamic RAM (DRDRAM), and Rambus Dynamic RAM (RDRAM), etc.

[0064] It should be noted that the above detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0065] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0066] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that the embodiments of this application described herein can be implemented in sequences other than those illustrated or described herein.

[0067] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or apparatus.

[0068] For ease of description, spatial relative terms such as "above," "on top of," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the figures to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation beyond the orientation of the device as described in the figures. For example, if the device in the figures were inverted, a device described as "above" or "on top of" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways, such as rotated 90 degrees or in other orientations, and the spatial relative descriptions used herein will be interpreted accordingly.

[0069] In the detailed description above, reference has been made to the accompanying drawings, which form part of this document. In the drawings, similar symbols typically identify similar parts unless the context otherwise indicates otherwise. The illustrated embodiments described in the detailed specification, drawings, and claims are not intended to be limiting. Other embodiments may be used and other changes may be made without departing from the spirit or scope of the subject matter presented herein.

[0070] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A data storage method, characterized in that, The method, applied to a distributed storage system comprising multiple storage nodes and a coordinating node, wherein the multiple storage nodes belong to multiple storage tiers with different performance levels, includes: The access frequency information of each data item stored on the first storage node is statistically analyzed. The coordination node periodically summarizes the access frequency information of each data item stored on the first storage node to determine the access popularity value of each data item stored on the first storage node. Based on the access popularity values ​​of each data item stored on the first storage node, the target data item to be migrated and the target storage level that matches the target data item are determined. The target data item is migrated and stored to a second storage node of the target storage level, wherein the plurality of storage nodes include the first storage node and the second storage node.

2. The data storage method according to claim 1, characterized in that, The step of determining the target data item to be migrated and the target storage level adapted to the target data item based on the access popularity value of each data item stored on the first storage node includes: Based on the access popularity value, data size, and time decay factor of each data item stored on the first storage node, the migration score of each data item stored on the first storage node is determined. Based on the migration scores of each data item stored on the first storage node, the capacity limits of each storage level, and the heat threshold, the target data item to be migrated and the target storage level that matches the target data item are determined.

3. The data storage method according to claim 1, characterized in that, The method of separately calculating the access frequency information of each data item stored on the first storage node includes... The sliding window algorithm is used to statistically analyze the access frequency information of each data item stored on the first storage node within a preset time window.

4. The data storage method according to claim 3, characterized in that, The popularity threshold is determined periodically based on the access popularity value of each data item stored on the multiple storage nodes within the preset time window.

5. The data storage method according to claim 1, characterized in that, The multiple storage tiers with different performance levels include a memory cache layer, a solid-state drive storage layer, and a hard disk drive storage layer; the method further includes: Receive data access requests; If the pre-accessed data item in the data access request is stored in the mechanical hard disk storage layer, the pre-accessed data item is asynchronously promoted to the solid-state drive storage layer or the memory cache layer; Based on the data access request, the pre-accessed data item is returned.

6. The data storage method according to claim 4, characterized in that, The memory caching layer is implemented based on Caffeine. After returning the pre-accessed data item based on the data access request, the method further includes: Update the access frequency information of the pre-accessed data item in the memory cache layer.

7. A data storage device, characterized in that, The device includes: The first statistics module is used to count the access frequency information of each data item stored on the first storage node. The first determining module is used to periodically summarize the access frequency information of each data item stored on the first storage node through the coordination node, and determine the access popularity value of each data item stored on the first storage node. The second determining module is used to determine the target data item to be migrated and the target storage level that matches the target data item based on the access popularity value of each data item stored on the first storage node. A first storage module is used to migrate and store the target data item to a second storage node of the target storage level, wherein the plurality of storage nodes include the first storage node and the second storage node.

8. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the data storage method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the data storage method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps of the data storage method as described in any one of claims 1 to 6.

Images