Cold and hot data layering method and system based on heterogeneous storage medium

Abstract

The invention provides a cold and hot data layering method and system based on a heterogeneous storage medium, and relates to the technical field of data storage.The cold and hot data layering method comprises the steps that storage data of all storage nodes in a heterogeneous storage environment and operation parameters of all the storage nodes in the operation process are obtained; obtaining a load state of each storage node by utilizing a static cache mechanism; determining an attribute division rule and a storage matching rule based on the load state of each storage node and the access frequency information of the cache data; generating a migration signal based on the attribute division rule and the storage matching rule; obtaining a target migration signal by using a pulse regulation and control technology; and according to the target migration signal, migrating the hot target data conforming to the hot data attribute in the cache data to the high-performance storage node, thereby realizing hot and cold data layering based on the heterogeneous storage medium, and realizing intelligent identification and efficient layered storage of the hot and cold data in the heterogeneous storage environment.

Description

A Cold and Hot Data Tiling Method and System Based on Heterogeneous Storage Media Technical Field

[0001] This application relates to the field of data storage technology, and in particular to a method and system for cold and hot data tiering based on heterogeneous storage media. Background Technology

[0002] In heterogeneous storage environments on cloud computing platforms, high-performance storage nodes based on high-speed solid-state drives (SSDs) and low-performance storage nodes based on hard disk drives (HDDs) or object storage are typically deployed simultaneously to balance performance and cost. Therefore, there is an urgent need for a technical solution that can intelligently tier data management by combining the real-time operating status of storage nodes with data access characteristics, in order to improve overall read / write performance and reduce long-term storage overhead.

[0003] One type of data tiering method based on access frequency thresholds and preset strategies has emerged in existing mainstream solutions. It triggers migration decisions based on the number of times a file has been accessed recently or the time interval between accesses. It also provides a hierarchical storage mechanism based on a rule engine, allowing administrators to manually configure migration strategies.

[0004] However, existing solutions have certain drawbacks. For example, they lack the ability to dynamically perceive the real-time load status of storage nodes in actual operation, relying solely on static decisions based on historical access frequencies. This can easily trigger hot data writes even when nodes are already under high load, exacerbating system bottlenecks. Furthermore, migration control processes often employ fixed time windows or simple queue scheduling, lacking fine-tuning of migration timing and pace. This can lead to the migration process competing with front-end services for input or output resources, affecting system response stability. Summary of the Invention

[0005] The purpose of this application is to provide a method and system for hot and cold data tiering based on heterogeneous storage media, in order to solve the problem that in the prior art, hot data writing is easily triggered even when the node is already under high load, which exacerbates the system bottleneck.

[0006] Firstly, this application provides a method for hot and cold data stratification based on heterogeneous storage media, comprising: acquiring the storage data of each storage node in a heterogeneous storage environment and the operating parameters of each storage node during operation, wherein the storage nodes include high-performance storage nodes using high-speed storage media and low-performance storage nodes using low-speed storage media, and are deployed based on heterogeneous storage media; caching each storage data using a static caching mechanism to obtain cached data, monitoring and analyzing the operating parameters to obtain the load status of each storage node, wherein the load status includes overload status, normal status, and idle status; determining attribute partitioning rules and storage matching rules based on the load status of each storage node and the access frequency information of the cached data; associating and matching the cached data and load status of each storage node based on the attribute partitioning rules and storage matching rules to generate a migration signal; regulating the migration signal using pulse modulation technology to obtain a target migration signal; and migrating hot target data in the cached data that conforms to hot data attributes to high-performance storage nodes, and migrating cold target data in the cached data that conforms to cold data attributes to low-performance storage nodes, thereby realizing hot and cold data stratification based on heterogeneous storage media.

[0007] Optionally, a static caching mechanism is used to cache the stored data to obtain cached data. Operating parameters are monitored and analyzed to obtain the load status of each storage node. This includes: collecting data access records from each storage node to extract access frequency information for each stored data, and calculating the access popularity value of each stored data based on a preset time period; allocating cache space to each stored data based on the maximum cache capacity of the static caching mechanism and the access popularity value to obtain cache allocation results; storing the stored data in the storage area of ​​the static caching mechanism based on the cache allocation results to obtain cached data; setting multiple statistical periods based on the operating parameter characteristics of each storage node to calculate the parameter statistics of each operating parameter within each statistical period; setting a load judgment threshold based on the parameter statistics and a preset strategy, and determining the load status of each storage node according to the load judgment threshold.

[0008] Optionally, based on the load status of each storage node and the access frequency information of cached data, attribute partitioning rules and storage matching rules are determined, including: generating access frequency information of cached data based on the data access records and corresponding access heat values ​​of each storage node; extracting the number of accesses within a preset unit time from the access frequency information; arranging the access counts to obtain the ranking result; and using the target access count located at a preset quantile position in the ranking result as the access reference value; setting the data capacity that storage nodes in an overloaded state can bear to zero; calculating the remaining allocable resources based on the storage resource utilization rate, maximum storage capacity, and stored data volume of storage nodes in a normal state; using the remaining allocable resources as the data capacity that can bear; and using the difference between the maximum storage capacity and the stored data volume of storage nodes in an idle state as the data capacity that can bear; and determining attribute partitioning rules based on the access reference value, and determining storage matching rules by combining the data capacity that each storage node can bear and the performance characteristics of heterogeneous storage media.

[0009] Optionally, based on attribute partitioning rules and storage matching rules, the cached data and load status of each storage node are correlated and matched to generate a migration signal. This includes: determining the attributes of the cached data of each storage node according to the attribute partitioning rules, designating cached data with hot data attributes as hot target data, and cached data with cold data attributes as cold target data; integrating high-performance storage nodes with normal load status and idle load status into target high-performance nodes according to the storage matching rules, and integrating low-performance storage nodes with normal load status and idle load status into target low-performance nodes; based on the data capacity of each storage node, associating and matching the hot target data with the target high-performance nodes to obtain a first association relationship, and associating and matching the cold target data with the target low-performance nodes to obtain a second association relationship; and generating a migration signal based on the first and second association relationships and a preset format.

[0010] Optionally, based on the first and second association relationships and combined with a preset format, a migration signal is generated, including: generating a first migration instruction based on the first identification information of each hot target data in the first association relationship and the load status of the corresponding high-performance node in the first association relationship; generating a second migration instruction based on the second identification information of each cold target data in the second association relationship and the load status of the corresponding low-performance node in the second association relationship; filling the first identification information of each hot target data and the corresponding first source storage node information, first target storage node information, and first migration instruction into the corresponding fields in the preset format to generate a first migration signal for each hot target data; filling the second identification information of each cold target data and the corresponding second source storage node information, second target storage node information, and second migration instruction into the corresponding fields in the preset format to generate a second migration signal for each cold target data; and removing missing information fields from the first migration signal of each hot target data and the second migration signal of each cold target data to generate a migration signal.

[0011] Optionally, pulse modulation technology is used to modulate the migration signal to obtain the target migration signal, including: marking the migration signal of the target storage node with an idle load state as the first priority signal, and marking the migration signal of the target storage node with a normal load state as the second priority signal; setting modulation parameters according to the carrying requirements of each target storage node using pulse modulation technology, the modulation parameters including the first interval duration of the first priority signal, the second interval duration of the second priority signal, and the transmission offset value of each target storage node; and adjusting the initial transmission time of the first priority signal and the second priority signal based on the modulation parameters to obtain the target migration signal.

[0012] Optionally, based on the control parameters, the initial transmission times of the first priority signal and the second priority signal are controlled to obtain the target migration signal, including: associating the initial transmission time of the first priority signal with the corresponding target storage node information, and associating the initial transmission time of the second priority signal with the corresponding target storage node information to form an initial signal sequence; adjusting the first transmission interval between adjacent first priority signals under the same target storage node in the initial signal sequence based on the first interval duration in the control parameters to obtain a first transmission time; adjusting the second transmission interval between adjacent second priority signals under the same target storage node in the initial signal sequence based on the second interval duration in the control parameters to obtain a second transmission time; offsetting and correcting the first transmission time and the second transmission time corresponding to the same target storage node based on the transmission offset value in the control parameters to form a corrected first transmission time and a corrected second transmission time; associating the first priority signal with the corrected first transmission time, and associating the second priority signal with the corrected second transmission time to form the target migration signal.

[0013] Secondly, this application provides a hot and cold data tiering system based on heterogeneous storage media, comprising: an acquisition module for acquiring storage data of each storage node in a heterogeneous storage environment and operating parameters of each storage node during operation, wherein the storage nodes include high-performance storage nodes using high-speed storage media and low-performance storage nodes using low-speed storage media, and are deployed based on heterogeneous storage media; an analysis module for caching each storage data using a static caching mechanism to obtain cached data, monitoring and analyzing operating parameters to obtain the load status of each storage node, wherein the load status includes overload status, normal status, and idle status; and a determination module for determining based on The system uses the load status of each storage node and the access frequency information of cached data to determine attribute partitioning rules and storage matching rules. The association module is used to associate and match the cached data and load status of each storage node based on the attribute partitioning rules and storage matching rules to generate a migration signal. The control module is used to control the migration signal using pulse control technology to obtain the target migration signal. The migration module is used to migrate hot target data that meets the hot data attributes in the cached data to high-performance storage nodes and cold target data that meets the cold data attributes in the cached data to low-performance storage nodes according to the target migration signal, so as to realize hot and cold data stratification based on heterogeneous storage media.

[0014] Thirdly, this application provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the cold and hot data stratification method based on heterogeneous storage media as described in the first aspect above.

[0015] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps of the cold and hot data stratification method based on heterogeneous storage medium described in the first aspect above.

[0016] This application provides a method for hot and cold data stratification based on heterogeneous storage media. The method acquires the storage data of each storage node in a heterogeneous storage environment and the operating parameters of each storage node during operation. The storage nodes include high-performance storage nodes using high-speed storage media and low-performance storage nodes using low-speed storage media, and are deployed based on heterogeneous storage media. A static caching mechanism is used to cache the storage data, obtaining cached data. The operating parameters are monitored and analyzed to obtain the load status of each storage node, including overload, normal, and idle states. Based on the load status of each storage node and the access frequency information of the cached data, attribute partitioning rules and storage matching rules are determined. Based on the attribute partitioning rules and storage matching rules, the cached data and load status of each storage node are correlated and matched to generate migration signals. Pulse modulation technology is used to regulate the migration signals to obtain target migration signals. According to the target migration signals, hot target data in the cached data that meets the hot data attributes is migrated to high-performance storage nodes, and cold target data in the cached data that meets the cold data attributes is migrated to low-performance storage nodes, thereby realizing hot and cold data stratification based on heterogeneous storage media.

[0017] By acquiring the storage data and operating parameters of each storage node in a heterogeneous storage environment, the system accurately identifies whether each storage node is overloaded, normal, or idle. This provides a dual basis for hot and cold data partitioning and migration strategies, avoiding misjudgment issues caused by static threshold strategies in complex load scenarios. It achieves precise matching between data popularity and node load capacity, ensuring that migration decisions conform to data access characteristics and are adapted to the target node's carrying capacity, thus avoiding performance bottlenecks caused by migrating hot data to high-load nodes. This makes the data migration process time-controllable and resource-coordinated, reducing interference with front-end business input or output resources, ensuring high response performance for hot data, and achieving low-performance storage for cold data, thereby improving overall storage resource utilization efficiency and system stability.

[0018] Furthermore, the target storage nodes are divided into different priorities based on their load status. Combined with the carrying capacity requirements of each target storage node, control parameters are set, including the first interval duration of the first priority signal, the second interval duration of the second priority signal, and the transmission offset value of each node. This dynamically controls the initial transmission time of the two priority signals, generating target migration signals. This solves the problem of migration tasks being concentrated during high-load periods, causing them to compete with for-end services for input or output resources. By utilizing a strategy of prioritizing the migration of idle nodes, the system's idle resource window is fully utilized, improving migration efficiency and overall system response stability. This overcomes the technical barriers of performance fluctuations and resource conflicts caused by the lack of dynamic load awareness and migration rhythm control in existing solutions. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 is a flowchart illustrating a method for cold and hot data stratification based on heterogeneous storage media provided in an embodiment of this application;

[0021] Figure 2 is a schematic diagram of a process for obtaining a target migration signal according to an embodiment of this application;

[0022] Figure 3 is a schematic diagram of a cold and hot data tiering system based on heterogeneous storage media provided in an embodiment of this application. Detailed Implementation

[0023] To address the issues that existing hot and cold data tiering schemes in heterogeneous storage environments of cloud computing platforms rely solely on historical access frequencies for static decision-making, lack dynamic awareness of real-time load status of storage nodes, and have crude migration control mechanisms that easily compete with front-end services for input or output resources.

[0024] This application collects storage data and operating parameters from each storage node, uses a static caching mechanism to uniformly cache the data, and identifies the overload, normal, or idle load state of the nodes in real time based on the operating parameters, thereby making up for the lack of dynamic load perception in existing solutions. It jointly analyzes the load state and the access frequency information of cached data to dynamically generate attribute partitioning rules and storage matching rules that match the current system operating conditions, avoiding blindly writing hot data on high-load nodes. It generates migration signals by associating cached data with node load states and introduces pulse control technology to finely adjust the triggering timing and execution rhythm of migration signals.

[0025] Based on the adjusted migration signals, hot target data is accurately migrated to high-performance storage nodes and cold target data is migrated to low-performance storage nodes. While avoiding resource conflicts, this achieves efficient and stable stratification of hot and cold data between heterogeneous media, improving the overall system response performance and storage resource utilization efficiency.

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] The core of this application is to provide a method for hot and cold data stratification based on heterogeneous storage media. A flowchart of a specific implementation is shown in Figure 1. The method includes: Step 101: Obtain the storage data of each storage node in the heterogeneous storage environment and the operating parameters of each storage node during operation. The storage nodes include high-performance storage nodes using high-speed storage media and low-performance storage nodes using low-speed storage media, and are deployed based on heterogeneous storage media.

[0028] In this step, heterogeneous storage environment refers to the environment in which a storage system composed of various types of storage media exists; each storage node refers to an independent storage unit in the heterogeneous storage environment used to store data.

[0029] High-speed storage media refers to storage media with fast read and write speeds; high-performance storage nodes refer to storage nodes that use high-speed storage media and have high overall performance; low-speed storage media refers to storage media with slow read and write speeds; low-performance storage nodes refer to storage nodes that use low-speed storage media and have low cost; heterogeneous storage media refers to a collection of various types of storage media composed of both high-speed and low-speed storage media.

[0030] In this embodiment of the application, the storage data stored in each storage node can be obtained, and the operating parameters generated by each storage node during operation can be collected in real time through the node monitoring module. It is also clear that among each storage node, some are high-performance storage nodes using high-speed storage media, and some are low-performance storage nodes using low-speed storage media. All storage nodes are deployed based on heterogeneous storage media composed of high-speed storage media and low-speed storage media.

[0031] Step 102: Use a static caching mechanism to cache the data in each storage node, obtain the cached data, monitor and analyze the operating parameters, and obtain the load status of each storage node, including overload status, normal status and idle status.

[0032] In this step, the static caching mechanism refers to a caching processing mechanism that pre-sets the caching strategy and does not dynamically adjust the caching space allocation; the overload state refers to the state in which the storage node load exceeds the preset upper limit; the normal state refers to the state in which the storage node load is within the preset reasonable range; and the idle state refers to the state in which the storage node load is below the preset lower limit.

[0033] Step 103: Based on the load status of each storage node and the access frequency information of cached data, determine the attribute partitioning rules and storage matching rules.

[0034] In this step, access frequency information refers to the frequency-related characteristics of cached data being accessed within a certain period of time; attribute partitioning rules refer to the rules used to determine whether cached data belongs to hot data attributes or cold data attributes; storage matching rules refer to the rules used to match different attribute data with corresponding storage nodes.

[0035] Step 104: Based on attribute partitioning rules and storage matching rules, perform correlation matching on the cached data and load status of each storage node to generate migration signals.

[0036] In this step, the migration signal refers to the signal used to control the migration of cached data between different storage nodes.

[0037] Step 105: Use pulse modulation technology to modulate the migration signal to obtain the target migration signal.

[0038] Step 106: Based on the target migration signal, migrate the hot target data in the cache data that meets the hot data attributes to the high-performance storage node, and migrate the cold target data in the cache data that meets the cold data attributes to the low-performance storage node, so as to realize the hot and cold data tiering based on heterogeneous storage media.

[0039] In this step, hot data attributes refer to attributes in cached data that are accessed frequently, and hot target data refers to cached data that matches hot data attributes; cold data attributes refer to attributes in cached data that are accessed less frequently, and cold target data refers to cached data that matches cold data attributes.

[0040] In this embodiment, a target migration signal is first received, and the migration object and target node information included in the signal are parsed. Then, based on the parsing results, hot target data that meets the hot data attributes is selected from the cached data and migrated to a high-performance storage node. At the same time, cold target data that meets the cold data attributes is selected and migrated to a low-performance storage node. Finally, through this data migration operation, hot and cold data tiering based on heterogeneous storage media is realized.

[0041] This application embodiment realizes hot and cold data tiering based on heterogeneous storage media, which improves the space utilization and data access efficiency of the storage system, while avoiding the overload problem of storage nodes due to centralized data migration.

[0042] This application provides a specific embodiment. Step 102: use a static caching mechanism to cache each storage data to obtain cached data, monitor and analyze the running parameters to obtain the load status of each storage node, specifically including the following steps: Step 201: collect data access records of each storage node to extract access frequency information of each storage data, and calculate the access popularity value of each storage data in combination with a preset time period.

[0043] In this step, data access records refer to the logs in each storage node that record the time when stored data is accessed, the accessing entity, etc.; access popularity value refers to the numerical value that reflects the frequency of access to each stored data.

[0044] In this embodiment, the log collection module of each storage node can collect all data access records of each storage node within a certain period. From the collected data access records, entries corresponding to each stored data are filtered out, and the number of times each stored data is accessed within the recorded period is counted to obtain the access count information of each stored data. Then, combined with a preset time period, the access popularity value of each stored data can be obtained by calculating the quotient of the access count information and the preset time period. The calculated access popularity value will be used for subsequent cache space allocation operations.

[0045] Step 202: Based on the maximum cache capacity of the static caching mechanism and combined with the access frequency value, allocate cache space to each stored data to obtain the cache allocation result.

[0046] In this step, the maximum cache capacity refers to the maximum space value that the static caching mechanism can provide for storing cached data; the cache allocation result refers to the specific cache space size allocated to each piece of stored data based on the access frequency value of the stored data and the maximum cache capacity.

[0047] In this embodiment, the maximum cache capacity is first obtained from the configuration information of the static caching mechanism. Then, the access popularity values ​​of each stored data are normalized to obtain the proportion of each stored data's access popularity value to the sum of all stored data's access popularity values. Following this proportion and considering the maximum cache capacity, a corresponding cache space is allocated to each stored data, i.e., the cache space for a specific stored data. The cache space for a stored data can be obtained by multiplying the maximum cache capacity parameter of the static caching mechanism by the ratio of the stored data's access popularity value to the sum of all stored data's access popularity values. This results in the cache allocation result for each stored data.

[0048] Step 203: Based on the cache allocation result, store the data to the storage area of ​​the static caching mechanism to obtain cached data.

[0049] In this embodiment, based on the cache allocation result, a dedicated sub-region matching the allocated space size is partitioned within the storage area of ​​the static caching mechanism for each piece of stored data. Then, each piece of stored data is copied from the original storage node to its corresponding dedicated sub-region, completing the caching process and ultimately obtaining the cached data stored within the static caching mechanism's storage area.

[0050] Step 204: Based on the operating parameter characteristics of each storage node, set multiple statistical periods to calculate the parameter statistics of each operating parameter within each statistical period.

[0051] In this step, the operating parameter characteristics refer to the patterns or characteristics of the changes in the operating parameters of each storage node over time; the statistical period refers to the pre-set time interval for performing periodic statistics on the operating parameters; and the parameter statistics data refer to the statistical results of each operating parameter within each statistical period.

[0052] In this embodiment, the operating parameter characteristics of each storage node are first analyzed, and multiple statistical periods of different durations are set based on the analysis results. Then, according to the set statistical periods, the operating parameters of each storage node are statistically analyzed in segments, and the average value, maximum value, etc. of each operating parameter in each statistical period are calculated to obtain the parameter statistics of each operating parameter in each statistical period.

[0053] Step 205: Based on parameter statistics and preset strategies, set load judgment thresholds, and determine the load status of each storage node according to the load judgment thresholds.

[0054] In this step, the preset strategy refers to the pre-set rules or standards used to determine the load judgment threshold by combining parameter statistics.

[0055] In this embodiment, a pre-set strategy needs to be invoked. This strategy specifies that the load judgment threshold should be determined based on the average and maximum values ​​in the parameter statistics. Combining the parameter statistics of each storage node, the overload threshold and idle threshold corresponding to each operating parameter are calculated according to the pre-set strategy to obtain the load judgment threshold. Then, the current operating parameters of each storage node are compared with the load judgment threshold. If the parameters exceed the overload threshold, the node is in an overload state; if they are below the idle threshold, the node is in an idle state; and if they are in between, the node is in a normal state. Finally, the load state corresponding to each storage node is determined.

[0056] For example, a preset policy is invoked. This policy specifies that the CPU utilization overload threshold is the statistical average × 1.2, and the idle threshold is the statistical average × 0.8. Assuming that, based on the parameter statistics, the calculated CPU utilization overload threshold for node B is 102% (since the maximum CPU utilization is 100%, it is actually calculated as 100%), and the idle threshold is 68%. The overload threshold for node C is 60%, and the idle threshold is 40%. The overload threshold for node D is 24%, and the idle threshold is 16%. Thus, the load judgment thresholds for each node can be obtained.

[0057] Subsequently, the current operating parameters of each node were compared. Node B's current CPU utilization was 98%, which, although not exceeding 102%, was overloaded due to its storage space utilization reaching 95%, exceeding the storage space overload threshold of 90%. Node C's current CPU utilization was 55%, between 40% and 60%, thus it was considered normal. Node D's current CPU utilization was 15%, below 16%, thus it was considered idle. The load status of each storage node was thus determined.

[0058] The embodiments of this application realize reasonable caching of cached data and accurate determination of the load status of storage nodes, laying the foundation for subsequent data migration based on attribute partitioning rules and storage matching rules, and improving the utilization rate of cache resources and the accuracy of load status judgment.

[0059] This application provides a specific embodiment. Step 103 involves determining attribute partitioning rules and storage matching rules based on the load status of each storage node and the access frequency information of cached data. This specifically includes the following steps:

[0060] Step 301: Based on the data access records and corresponding access popularity values ​​of each storage node, generate access frequency information of cached data, extract the number of accesses within a preset unit time from the access frequency information, arrange the access counts to obtain the arrangement result, and use the target access count located at the preset quantile position in the arrangement result as the access reference value.

[0061] In this step, the preset quantile position refers to the quantile point set in advance to determine the target number of accesses; the access reference value refers to using the target number of accesses as a reference standard to judge the frequency of access to cached data.

[0062] In this embodiment, historical data access records of each storage node are first retrieved. Combined with the access frequency values ​​of each stored data, the access frequency information of cached data is generated by statistically analyzing the trend of access frequency changes and the distribution of access frequency per unit time within different time intervals. Then, from this access frequency information, the access frequency of each cached data corresponding to a preset unit time is selected. These access frequencies are then arranged in ascending order to obtain the sorting result. Finally, a preset quantile position is set, and the target access frequency corresponding to that position is extracted from the sorting result and used as an access reference value.

[0063] Step 302: Set the amount of data that can be carried by the storage node in the overload state to zero. Calculate the remaining allocable resources based on the storage resource utilization, maximum storage capacity, and amount of stored data of the storage node in the normal load state. Use the remaining allocable resources as the amount of data that can be carried. Use the difference between the maximum storage capacity and the amount of stored data of the storage node in the idle load state as the amount of data that can be carried.

[0064] In this embodiment, the current load status of each storage node is first determined. For storage nodes in an overloaded state, their data capacity is directly set to zero to prevent additional data from further increasing the node load. For storage nodes in a normal state, the storage resource utilization rate, maximum storage capacity, and amount of stored data are obtained. The remaining allocable resources of the normal state storage node, i.e., the amount of data the node can handle, are then calculated. This can be obtained by calculating the difference between the product of the node's maximum storage capacity and the remaining storage resource utilization rate, and the amount of stored data. The remaining storage resource utilization rate is obtained by calculating the difference between the value 1 and the storage resource utilization rate.

[0065] For storage nodes that are idle, the difference between their maximum storage capacity and the amount of data already stored is calculated, and this difference is taken as the amount of data they can handle. This yields the total amount of data each storage node can handle under its current load condition.

[0066] For example, assume that in heterogeneous storage environment A, node B is overloaded, node C is in normal operation, and node D is idle. For node B, its data capacity is set to 0. For node C, in normal operation, with a maximum storage capacity of 1000GB and a storage resource utilization rate of 80%, the calculated remaining allocable resource is 200GB, which is used as the data capacity that node C can handle. For node D, in idle operation, with a maximum storage capacity of 1000GB and 200GB of already stored data, the calculated data capacity is 800GB. The final result is that node B can handle 0GB of data, node C 200GB, and node D 800GB.

[0067] Step 303: Based on the access reference value, determine the attribute partitioning rules, and combine the data volume that each storage node can bear and the performance characteristics of heterogeneous storage media to determine the storage matching rules.

[0068] In this step, performance characteristics refer to the inherent performance of different types of storage nodes in heterogeneous storage media in terms of data read / write speed, stability, and load capacity.

[0069] In this embodiment, the access reference value is first used as the core basis to establish the correspondence between the number of cached data accesses and attributes, thereby determining the attribute partitioning rules. Then, combining the data capacity of each storage node and the performance characteristics of heterogeneous storage media, the matching logic between different attribute data and storage nodes is set, thereby determining the storage matching rules.

[0070] The embodiments of this application avoid the problems of ambiguous hot and cold data division and mismatch between data storage and node carrying capacity, and improve the rationality of hot and cold data stratification and the utilization efficiency of storage resources in heterogeneous storage environments.

[0071] This application provides a specific embodiment. Step 104 involves associating and matching the cached data and load status of each storage node based on attribute partitioning rules and storage matching rules to generate a migration signal. This specifically includes the following steps:

[0072] Step 401: According to the attribute partitioning rules, perform attribute determination on the cached data of each storage node, and use the cached data determined to be hot data attributes as hot target data, and use the cached data determined to be cold data attributes as cold target data.

[0073] In this embodiment, the previously determined attribute partitioning rules are first retrieved, and the cached data of each storage node is traversed. For each cached data, the attribute is determined according to the judgment criteria in the attribute partitioning rules. If the access frequency of a cached data meets the judgment criteria for hot data attribute, the cached data is marked as hot target data; if it meets the judgment criteria for cold data attribute, the cached data is marked as cold target data. Through this traversal and judgment operation, all cached data is clearly distinguished into two categories: hot target data and cold target data.

[0074] Step 402: According to the storage matching rules, integrate high-performance storage nodes with normal load status and idle load status into target high-performance nodes, and integrate low-performance storage nodes with normal load status and idle load status into target low-performance nodes.

[0075] In this step, the target high-performance node refers to the set of nodes formed by integrating high-performance storage nodes in normal and idle states, which can be used to carry hot target data; the target low-performance node refers to the set of nodes formed by integrating low-performance storage nodes in normal and idle states, which can be used to carry cold target data.

[0076] In this embodiment, the first step, based on the storage matching rules that match hot target data with high-performance storage nodes and cold target data with low-performance storage nodes, and only selects nodes in non-overloaded states, first filters out all high-performance storage nodes in normal and idle states, and groups these nodes into the same set to obtain target high-performance nodes. The second step then filters out all low-performance storage nodes in normal and idle states, and also groups them into the same set to obtain target low-performance nodes. This ensures that all subsequently matched nodes have the basic conditions to carry the corresponding data. The obtained target high-performance nodes and target low-performance nodes will serve as the matching objects for hot and cold target data.

[0077] Step 403: Based on the amount of data each storage node can carry, associate and match hot target data with target high-performance nodes to obtain the first association relationship, and associate and match cold target data with target low-performance nodes to obtain the second association relationship.

[0078] In this step, the first association relationship refers to the correspondence formed after matching the hot target data with the target high-performance nodes and the data capacity of each storage node; the second association relationship refers to the correspondence formed after matching the cold target data with the target low-performance nodes and the data capacity of each storage node.

[0079] In this embodiment, the total capacity of all hot target data is first calculated, then the data capacity that each target high-performance node can carry is checked, and the hot target data is allocated to the target high-performance nodes according to the principle of prioritizing allocation to nodes with sufficient data capacity and better load conditions. It is ensured that the hot target data capacity allocated to a single target high-performance node does not exceed the data capacity that node can carry. The data capacity allocated to a single target high-performance node is the smaller value between the data capacity that the available high-performance node can carry and the current remaining total capacity of hot target data. After the allocation is completed, a first association relationship is formed.

[0080] Simultaneously, the total capacity of all cold target data is calculated and allocated to low-performance target nodes according to the same principle. Specifically, the cold target data capacity allocated to a single available low-cost node is the smaller of the available low-cost node's data capacity and the current remaining total capacity of cold target data, forming a second association. These two types of associations will clearly define the migration target node for each piece of data.

[0081] Step 404: Based on the first and second association relationships and combined with the preset format, generate a migration signal.

[0082] In this step, the preset format refers to a fixed format that is pre-set and includes key information fields required for data migration.

[0083] In this embodiment, a preset format is first retrieved. From the first association relationship, the source storage node information, target storage node information, and data identifier information of the hot target data corresponding to each associated record are extracted. This information is then filled into the corresponding fields of the preset format, and a data migration instruction is added to the migration trigger instruction field. Simultaneously, the source storage node information, target storage node information, and data identifier information of the cold target data are extracted from the second association relationship, and similarly filled into the corresponding fields of the format, with a migration trigger instruction added. All the filled single migration instructions are sorted and integrated according to the data identifier to obtain a migration signal including complete migration information.

[0084] The embodiments of this application avoid problems such as confusion of migration objects, incorrect selection of target nodes, or missing migration instruction information, and ensure the orderliness and accuracy of hierarchical migration of cold and hot data in heterogeneous storage environments.

[0085] This application provides a specific embodiment. Step 404 involves generating a migration signal based on the first and second association relationships and a preset format, specifically including the following steps:

[0086] Step 411: Based on the first identification information of each hot target data in the first association relationship and the load status of the corresponding high-performance node in the first association relationship, generate a first migration instruction; based on the second identification information of each cold target data in the second association relationship and the load status of the corresponding low-performance node in the second association relationship, generate a second migration instruction.

[0087] In this step, the first identification information refers to the unique identifier of each thermal target data in the first association relationship, which is used to distinguish different thermal target data; the second identification information refers to the unique identifier of each cold target data in the second association relationship, which is used to distinguish different cold target data.

[0088] In this embodiment, firstly, the first identification information of each hot target data is extracted from the first association relationship, and simultaneously, the load status of the target high-performance nodes corresponding to these hot target data is obtained. Then, the uniqueness of the hot target data is confirmed based on the first identification information, and the load status of the corresponding target high-performance node is used to determine whether the node still has the capacity to carry the hot target data, thereby generating a first migration instruction including the data identifier and the target node status. Similarly, second identification information of each cold target data is extracted from the second association relationship, the load status of the corresponding target low-performance node is obtained, the uniqueness of the cold target data is confirmed, and the node's carrying capacity is determined, before generating a second migration instruction including the data identifier and the target node status.

[0089] Step 412: Fill the first identification information of each hot target data and the corresponding first source storage node information, first target storage node information, and first migration instruction into the corresponding fields in the preset format to generate the first migration signal of each hot target data.

[0090] In this step, the first source storage node information refers to the storage node information where each hot target data in the first association is currently located; the first migration signal refers to a single signal used to control the migration of hot target data after filling the relevant information of each hot target data into a preset format.

[0091] In this embodiment, the first source storage node information and the first target storage node information of each hot target data are first extracted from the first association relationship. Then, a preset format is retrieved, and the first identifier information, the corresponding first source storage node information, and the first target storage node information of each hot target data are filled into the corresponding fields of the format. Then, the first migration instruction corresponding to each hot target data is filled into the first migration instruction field of the format. Each hot target data corresponds to one format, and after filling, the first migration signal of each hot target data is obtained.

[0092] Step 413: Fill the corresponding fields in the preset format with the second identification information of each cold target data, the corresponding second source storage node information, the second target storage node information, and the second migration instruction, so as to generate the second migration signal of each cold target data.

[0093] In this step, the second source storage node information refers to the storage node information where each cold target data in the second association is currently located; the second migration signal refers to a single signal used to control the migration of cold target data after filling the relevant information of each cold target data into a preset format.

[0094] In this embodiment, the second source storage node information of each cold target data is first extracted from the second association relationship. Then, following the preset format, the second identifier information, the corresponding second source storage node information, and the second target storage node information of each cold target data are filled into the corresponding fields of the format. Then, the second migration instruction corresponding to each cold target data is filled into the second migration instruction field of the format. Each cold target data corresponds to one format, and after filling, the second migration signal of each cold target data is obtained.

[0095] Step 414: Remove missing information fields from the first migration signal of each hot target data and the second migration signal of each cold target data to generate a migration signal.

[0096] In this step, missing information fields refer to fields in the first or second migration signal that are not filled in completely according to the preset format.

[0097] In this embodiment, all first migration signals and all second migration signals are first collected, and each field of each signal is checked to see if it is completely filled in according to a preset format. If a signal has a missing field, the signal is discarded; if all fields of a signal are completely filled, the signal is retained. After all signals have been checked, all the retained complete first migration signals and complete second migration signals are integrated in the order of data identification information to obtain a migration signal that includes all valid migration information.

[0098] The embodiments of this application avoid migration failures or node overload problems caused by incomplete migration instruction information or mismatch with node status.

[0099] This application provides a specific embodiment, as shown in Figure 2. Step 105 involves using pulse modulation technology to modulate the migration signal to obtain the target migration signal, specifically including the following steps:

[0100] Step 501: Mark the migration signal of the target storage node whose load state is idle as the first priority signal, and mark the migration signal of the target storage node whose load state is normal as the second priority signal.

[0101] In this step, the first priority signal refers to the migration signal whose target storage node load status is idle; the second priority signal refers to the migration signal whose target storage node load status is normal.

[0102] In this embodiment, all individual migration signals included in the migration signal are first parsed, and the target storage node corresponding to each migration signal is extracted. Then, the current load status of each target storage node is obtained. Individual migration signals whose target storage node load status is idle are marked as first priority signals, and individual migration signals whose target storage node load status is normal are marked as second priority signals. This marking operation then clarifies the priority transmission order of different migration signals, providing a basis for subsequently setting control parameters, resulting in the first priority signal and the second priority signal.

[0103] Step 502: Based on the carrying requirements of each target storage node, set the control parameters using pulse control technology. The control parameters include the first interval duration of the first priority signal, the second interval duration of the second priority signal, and the transmission offset value of each target storage node.

[0104] In this step, the carrying requirement refers to the maximum number of migration signals that the target storage node can receive per unit time and the signal transmission interval requirement; the control parameter refers to the set of parameters used to control the transmission timing of migration signals; the first interval duration refers to the transmission time interval between first priority signals; the second interval duration refers to the transmission time interval between second priority signals; and the transmission offset value refers to the signal transmission time offset set for the node to avoid the simultaneous transmission of different migration signals received by the same target storage node.

[0105] In this embodiment, the carrying requirements of each target storage node are first analyzed. Based on these requirements, pulse modulation technology is used to set a first interval duration for the first priority signal and a second interval duration for the second priority signal. Simultaneously, a transmission offset value is set for each target storage node. This operation then generates modulation parameters including the first interval duration, the second interval duration, and the transmission offset values ​​for each target storage node.

[0106] Step 503: Based on the control parameters, the initial transmission times of the first priority signal and the second priority signal are controlled to obtain the target migration signal.

[0107] In this step, the initial transmission time refers to the planned transmission time before the first priority signal and the second priority signal are adjusted.

[0108] In this embodiment, the initial transmission times of the first priority signal and the second priority signal are first extracted. Then, based on the first interval duration in the control parameters, the initial transmission times of all first priority signals are adjusted so that the first first priority signal is transmitted at its initial time, and each subsequent first priority signal is transmitted sequentially delayed according to the first interval duration. Simultaneously, based on the second interval duration, the initial transmission times of all second priority signals are adjusted so that the first second priority signal is transmitted at its initial time, and each subsequent second priority signal is transmitted sequentially delayed according to the second interval duration.

[0109] Then, by combining the transmission offset values ​​of each target storage node, the transmission time of the migration signal of the corresponding node is adjusted to correct the offset; after all signal times are adjusted, the adjusted first priority signal and second priority signal are integrated according to the new transmission time order to obtain the target migration signal.

[0110] The embodiments of this application realize the adaptation of migration signal transmission timing to the carrying capacity of the target node, avoid the node overload problem caused by simultaneous transmission of multiple signals, and ensure the stability and orderliness of migration signal transmission.

[0111] This application provides a specific embodiment. Step 503 involves adjusting the initial transmission times of the first priority signal and the second priority signal based on the adjustment parameters to obtain the target migration signal. This specifically includes the following steps:

[0112] Step 511: Associate the initial transmission time of the first priority signal with the corresponding target storage node information, and associate the initial transmission time of the second priority signal with the corresponding target storage node information to form the initial signal sequence.

[0113] In this embodiment, the initial transmission times of all first-priority signals are first extracted, and each first-priority signal is bound to its corresponding target storage node information. Then, the initial transmission times of all second-priority signals are extracted, and each second-priority signal is bound to its corresponding target storage node information. Next, this bound information is categorized and organized according to the target storage node to form an initial signal sequence that includes the initial associations of all signals.

[0114] Step 512: Based on the first interval duration in the control parameters, adjust the first transmission interval between adjacent first priority signals under the same target storage node in the initial sequence of signals to obtain the first transmission time.

[0115] In this step, "same target storage node" refers to the same storage node that multiple migration signals in the initial signal sequence all point to, and which is the planned data receiving node. The first transmission interval refers to the time interval between two adjacent first-priority signals under the same target storage node in the initial signal sequence.

[0116] In this embodiment, all first priority signals are first selected from the initial signal sequence and grouped according to the target storage node to obtain first priority signal groups under the same target storage node. Then, the initial transmission time of adjacent first priority signals in each group is extracted, and the initial first transmission interval is calculated. The first interval duration in the control parameters is retrieved, and the first transmission interval of adjacent first priority signals in each group is adjusted to the first interval duration. The first transmission time of the first signal is taken as the initial transmission time, and the first transmission time of subsequent signals is the first transmission time of the previous signal plus the first interval duration. Finally, the first transmission time of all first priority signals in each group is obtained.

[0117] Step 513: Based on the second interval duration in the control parameters, adjust the second transmission interval between adjacent second priority signals under the same target storage node in the initial signal sequence to obtain the second transmission time.

[0118] In this step, the second transmission interval refers to the time interval between two adjacent second priority signals under the same target storage node in the initial sequence of signals.

[0119] In this embodiment, all second-priority signals are first selected from the initial signal sequence and grouped according to the target storage node to obtain second-priority signal groups under the same target storage node. The initial transmission time of adjacent second-priority signals within each group is extracted, and the initial second transmission interval is calculated. Then, the second interval duration in the control parameters is retrieved, and the second transmission interval of adjacent second-priority signals within each group is adjusted to the second interval duration. The second transmission time of the first signal is taken as the initial transmission time, and the second transmission time of subsequent signals is the second transmission time of the previous signal plus the second interval duration, finally obtaining the second transmission times of all second-priority signals within each group.

[0120] Step 514: Based on the transmission offset value in the control parameters, the first transmission time and the second transmission time corresponding to the same target storage node are offset and corrected to form the corrected first transmission time and the corrected second transmission time.

[0121] In this step, the corrected first transmission time refers to the first transmission time of the first priority signal under the same target storage node after correction by the transmission offset value.

[0122] In this embodiment, the transmission offset value corresponding to each target storage node in the control parameters is retrieved in groups according to the target storage nodes. Then, for the first priority signal under the same target storage node, the corrected first transmission time is obtained by adding the transmission offset value of the target node to the first transmission time. Next, for the second priority signal under the same target storage node, the corrected second transmission time is obtained by adding the transmission offset value of the target node to the second transmission time. This correction operation ensures that the transmission times of the first and second priority signals received by the same target storage node do not overlap, avoiding node load conflicts.

[0123] Step 515: Associate the first priority signal with the corrected first transmission time, and associate the second priority signal with the corrected second transmission time to form the target migration signal.

[0124] In this embodiment, each first priority signal needs to be associated with its corresponding modified first transmission time, and each second priority signal needs to be associated with its corresponding modified second transmission time. Simultaneously, all associated signals are sorted according to the chronological order of their modified transmission times; and the sorted signals are integrated into a complete set of association relationships to obtain the target migration signal.

[0125] The embodiments of this application realize fine-grained control of the migration signal transmission timing, ensuring that the signals received by the same target storage node do not overlap or conflict, avoiding node overload caused by concentrated signal transmission, and ensuring the stability and orderliness of the data migration process.

[0126] Figure 3 is a schematic diagram of a specific implementation of a hot and cold data tiering system based on heterogeneous storage media provided in this application. Referring to Figure 3, the system may include: an acquisition module 21, used to acquire the storage data of each storage node in the heterogeneous storage environment and the operating parameters of each storage node during operation. The storage nodes include high-performance storage nodes using high-speed storage media and low-performance storage nodes using low-speed storage media, and are deployed based on heterogeneous storage media; and an analysis module 22, used to perform caching processing on each storage data using a static caching mechanism to obtain cached data, monitor and analyze the operating parameters, and obtain the load status of each storage node. The load status includes overload status, normal status, and idle status. The system comprises: a determination module 23, which determines attribute partitioning rules and storage matching rules based on the load status of each storage node and the access frequency information of cached data; an association module 24, which performs association matching on the cached data and load status of each storage node based on the attribute partitioning rules and storage matching rules to generate a migration signal; a control module 25, which uses pulse control technology to control the migration signal to obtain a target migration signal; and a migration module 26, which migrates hot target data in the cached data that meets the hot data attributes to high-performance storage nodes and cold target data in the cached data that meets the cold data attributes to low-performance storage nodes according to the target migration signal, so as to realize hot and cold data stratification based on heterogeneous storage media.

[0127] This application provides an embodiment of a hot and cold data tiering system based on heterogeneous storage media to implement the aforementioned hot and cold data tiering method based on heterogeneous storage media. Therefore, the specific implementation of a hot and cold data tiering system based on heterogeneous storage media can be found in the embodiment section of the hot and cold data tiering method based on heterogeneous storage media mentioned above. The specific implementation can be referred to the description of the corresponding embodiments, which will not be repeated here.

[0128] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of a cold and hot data stratification method based on heterogeneous storage media as described above.

[0129] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described methods for cold and hot data stratification based on heterogeneous storage media.

[0130] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0131] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the embodiments of the above-described method for cold and hot data stratification based on heterogeneous storage media.

[0132] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0133] The above provides a detailed description of a method and system for hot and cold data tiering based on heterogeneous storage media provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for tiered cold and hot data storage based on heterogeneous storage media, characterized in that, include: The system acquires storage data from each storage node in a heterogeneous storage environment, as well as the operating parameters of each storage node during operation. The storage nodes include high-performance storage nodes using high-speed storage media and low-performance storage nodes using low-speed storage media, and are deployed based on heterogeneous storage media. The system uses a static caching mechanism to cache the storage data to obtain cached data, and monitors and analyzes the operating parameters to obtain the load status of each storage node. The load status includes overload status, normal status, and idle status. Based on the load status of each storage node and the access frequency information of cached data, attribute partitioning rules and storage matching rules are determined; based on the attribute partitioning rules and the storage matching rules, the cached data and load status of each storage node are correlated and matched to generate migration signals. The migration signal is modulated using pulse modulation technology to obtain the target migration signal; According to the target migration signal, hot target data in the cached data that meets the hot data attributes are migrated to high-performance storage nodes, and cold target data in the cached data that meets the cold data attributes are migrated to low-performance storage nodes, so as to realize hot and cold data stratification based on heterogeneous storage media.

2. The method according to claim 1, characterized in that, A static caching mechanism is used to cache the stored data to obtain cached data. The operating parameters are monitored and analyzed to obtain the load status of each storage node. This includes: collecting data access records from each storage node to extract access frequency information for each stored data, and calculating the access popularity value of each stored data based on a preset time period; allocating cache space to each stored data based on the maximum cache capacity of the static caching mechanism and the access popularity value to obtain a cache allocation result; storing the stored data in the storage area of ​​the static caching mechanism based on the cache allocation result to obtain cached data; setting multiple statistical periods based on the operating parameter characteristics of each storage node to calculate the parameter statistics of each operating parameter within each statistical period; setting a load judgment threshold based on the parameter statistics and a preset strategy, and determining the load status of each storage node according to the load judgment threshold.

3. The method according to claim 1, characterized in that, Based on the load status of each storage node and the access frequency information of cached data, attribute partitioning rules and storage matching rules are determined, including: generating access frequency information of cached data based on the data access records and corresponding access popularity values ​​of each storage node; extracting the number of accesses within a preset unit time from the access frequency information; arranging the access counts to obtain a ranking result; and using the target access count located at a preset quantile position in the ranking result as the access reference value; setting the data capacity that storage nodes in an overloaded state can bear to zero; calculating the remaining allocable resources based on the storage resource utilization rate, maximum storage capacity, and stored data volume of storage nodes in a normal state; using the remaining allocable resources as the data capacity that can bear; and using the difference between the maximum storage capacity and the stored data volume of storage nodes in an idle state as the data capacity that can bear; and determining attribute partitioning rules based on the access reference value, and determining storage matching rules in combination with the data capacity that each storage node can bear and the performance characteristics of heterogeneous storage media.

4. The method according to claim 1, characterized in that, Based on the attribute partitioning rules and the storage matching rules, the cached data and load status of each storage node are correlated and matched to generate a migration signal. This includes: determining the attributes of the cached data of each storage node according to the attribute partitioning rules, designating cached data with hot data attributes as hot target data, and cached data with cold data attributes as cold target data; integrating high-performance storage nodes with normal and idle load statuses into target high-performance nodes, and integrating low-performance storage nodes with normal and idle load statuses into target low-performance nodes, based on the data capacity of each storage node; associating and matching the hot target data with the target high-performance nodes to obtain a first association relationship, and associating and matching the cold target data with the target low-performance nodes to obtain a second association relationship; and generating a migration signal based on the first and second association relationships, combined with a preset format.

5. The method according to claim 4, characterized in that, Based on the first and second association relationships and combined with a preset format, a migration signal is generated, including: generating a first migration instruction based on the first identification information of each hot target data in the first association relationship and the load status of the corresponding high-performance node in the first association relationship; generating a second migration instruction based on the second identification information of each cold target data in the second association relationship and the load status of the corresponding low-performance node in the second association relationship; filling the first identification information of each hot target data, the corresponding first source storage node information, the first target storage node information, and the first migration instruction into the corresponding fields in the preset format to generate a first migration signal for each hot target data; filling the second identification information of each cold target data, the corresponding second source storage node information, the second target storage node information, and the second migration instruction into the corresponding fields in the preset format to generate a second migration signal for each cold target data; and removing missing information fields from the first migration signal of each hot target data and the second migration signal of each cold target data to generate a migration signal.

6. The method according to claim 1, characterized in that, The migration signal is modulated using pulse modulation technology to obtain a target migration signal, including: marking the migration signal of the target storage node with an idle load state as a first priority signal, and marking the migration signal of the target storage node with a normal load state as a second priority signal; setting modulation parameters using pulse modulation technology according to the carrying requirements of each target storage node, the modulation parameters including a first interval duration of the first priority signal, a second interval duration of the second priority signal, and a transmission offset value of each target storage node; and adjusting the initial transmission time of the first priority signal and the second priority signal based on the modulation parameters to obtain the target migration signal.

7. The method according to claim 6, characterized in that, Based on the control parameters, the initial transmission times of the first priority signal and the second priority signal are controlled to obtain a target migration signal, including: associating the initial transmission time of the first priority signal with the corresponding target storage node information, and associating the initial transmission time of the second priority signal with the corresponding target storage node information to form an initial signal sequence; adjusting the first transmission interval between adjacent first priority signals under the same target storage node in the initial signal sequence based on the first interval duration in the control parameters to obtain a first transmission time; adjusting the second transmission interval between adjacent second priority signals under the same target storage node in the initial signal sequence based on the second interval duration in the control parameters to obtain a second transmission time; offsetting and correcting the first and second transmission times corresponding to the same target storage node based on the transmission offset value in the control parameters to form a corrected first transmission time and a corrected second transmission time; associating the first priority signal with the corrected first transmission time, and associating the second priority signal with the corrected second transmission time to form a target migration signal.

8. A cold and hot data tiering system based on heterogeneous storage media, characterized in that, include: The acquisition module is used to acquire the storage data of each storage node in the heterogeneous storage environment and the operating parameters of each storage node during operation. The storage nodes include high-performance storage nodes using high-speed storage media and low-performance storage nodes using low-speed storage media, and are deployed based on heterogeneous storage media. The analysis module is used to cache the stored data using a static caching mechanism to obtain cached data, monitor and analyze the operating parameters, and obtain the load status of each storage node, including overload status, normal status, and idle status. The determination module is used to determine attribute partitioning rules and storage matching rules based on the load status of each storage node and the access frequency information of cached data; the association module is used to perform association matching on the cached data and load status of each storage node based on the attribute partitioning rules and the storage matching rules, so as to generate a migration signal. The control module is used to control the migration signal using pulse control technology to obtain the target migration signal; The migration module is used to migrate hot target data that meets the hot data attributes in the cached data to high-performance storage nodes according to the target migration signal, and to migrate cold target data that meets the cold data attributes in the cached data to low-performance storage nodes, so as to realize hot and cold data stratification based on heterogeneous storage media.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of a hot and cold data tiering method based on heterogeneous storage media as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables a method for stratifying hot and cold data based on heterogeneous storage media as described in any one of claims 1 to 7.

Images