A transparent archival data migration and rollback method across storage media

Abstract

The present application relates to the technical field of transparent archiving data management, and more particularly to a cross-storage medium transparent archiving data migration and rollback method, which comprises the following steps: calculating an interleaving pollution deviation for evaluating the pollution degree of read-write requests to a high-performance cache layer; calculating a cache pollution index according to the interleaving pollution deviation and the current available cache capacity of a target archiving block area, the actual cumulative amount of additional writing and the safe writing baseline of the underlying storage medium; when the cache pollution index exceeds a preset threshold, identifying the physical separation boundary between a high-heat additional writing sub-block and a pure read-only cold flow sub-block within the target archiving block area, migrating the high-heat additional writing sub-block to the high-performance cache layer, and maintaining the pure read-only cold flow sub-block in the low-speed archiving medium layer, so as to realize the physical isolation and differentiated transparent migration of cold and hot data in the same archiving block area, and effectively improve the efficiency of data migration and rollback.

Description

Technical Field

[0001] This invention relates to the field of transparent archived data management technology. More specifically, this invention relates to a method for transparent archived data migration and backtracking across storage media. Background Technology

[0002] With the rapid development of cloud computing and big data technologies, tiered storage of massive archived data has become a core means of balancing storage costs and read / write performance. Existing tiered storage systems generally adopt an automatic tiered storage architecture, which automatically migrates frequently accessed data to a cache layer built with high-performance solid-state storage media, and pushes infrequently accessed archived data down to a low-speed, high-capacity hard disk drive or object storage layer, in order to achieve a balance between hardware costs and business response speed.

[0003] In the prior art, Chinese patent application CN1760875A discloses a data migration method in an automatic tiered storage system. Its core features include: monitoring the absolute I / O access frequency per unit time of each data block at a fixed physical block granularity; when the cumulative access count of a data block exceeds a preset threshold, migrating the entire data block from the low-speed storage layer to the high-speed cache layer; and migrating the entire data block back when it has not been accessed within a certain period. This method uses periodic software scanning threads to determine the hot / cold status and schedule data movement, achieving automated flow of storage resources at the macro level.

[0004] However, when dealing with scenarios involving high-frequency append writes at the end of extremely large archive files and concurrent large-span sequential read-only cold streams, the aforementioned solutions, with their fixed physical block granularity monitoring and migration mechanisms, treat all data within the same physical block area as a unified thermodynamic whole, ignoring the spatiotemporal interleaving characteristics of multiple service flows accessing concurrently within the same physical hardware stripe. The end of extremely large archive files typically experiences continuous high-frequency append write requests, while the file header or middle section may concurrently carry large-span sequential read-only backtracking flows. These two flows may fall into the same monitored block area in physical storage space, but semantically belong to the high-hot-write domain and the pure-cold-read domain, respectively. If full caching or block relocation is performed solely based on the absolute IO frequency of the macro-block area, the cold-read flow will be migrated to the high-performance cache layer, directly causing severe cache pollution, crowding out the truly needed effective hot data space, leading to a significant decrease in the cache hit rate of the high-performance storage medium, while also increasing the frequency of invalid erases and writes, and shortening the hardware lifespan.

[0005] Therefore, how to build a refined control and routing mechanism that can detect and evaluate read-write interleaving pollution within the same hardware stripe in real time, and achieve physical isolation and differentiated transparent migration of high-frequency append writes at the end of ultra-large archive files and large-span sequential read-only cold flows, is a technical problem that urgently needs to be solved in the field of data storage and management technology. Summary of the Invention

[0006] To address the aforementioned technical challenges of constructing a refined management and routing mechanism capable of real-time detection and assessment of read / write interleaving contamination within the same hardware stripe, and achieving physical isolation and differentiated transparent migration of high-frequency append writes to the end of ultra-large archive files and large-span sequential read-only cold flows, this invention proposes a transparent archive data migration and backtracking method across storage media. This method includes the following steps:

[0007] S1. Construct a spatiotemporal IO interleaving feature tracking model to collect the operation type, logical address offset, and arrival timestamp of each read / write request within the target archive block in real time, thereby obtaining the IO request sequence of the target archive block on the time axis. S2. Calculate the alternation frequency of read / write requests on the time axis and the span distance in the logical address space within the target archive block based on the IO request sequence, and then calculate the interleaving pollution deviation degree used to assess the degree of pollution of the high-performance cache layer by read / write requests. S3. Calculate the cache pollution index based on the interleaving pollution deviation degree, the currently available cache capacity of the target archive block, the actual cumulative append write volume, and the safe write baseline of the underlying storage medium. S4. When the cache pollution index exceeds a preset threshold, identify the physical separation boundary between high-hot append write sub-blocks and pure read-only cold flow sub-blocks within the target archive block, route and migrate the high-hot append write sub-blocks to the high-performance cache layer, and maintain the pure read-only cold flow sub-blocks in the low-speed archive medium layer, thereby achieving physical isolation and differentiated transparent migration of hot and cold data within the same archive block.

[0008] This invention provides a transparent archival data migration and backtracking method across storage media. It constructs a spatiotemporal I / O interleaving feature tracking model from the underlying hardware access logic. When a dynamic cache pollution risk index based on multi-dimensional constraint calculation is triggered, sub-block-level splitting is actively initiated. High-hot append write sub-blocks are precisely routed to the high-performance cache layer for aggregation and reorganization along the density mutation boundary. At the same time, pure read-only cold flows are forcibly isolated in the low-speed media layer. While eliminating the cache hit rate drop caused by cold reads and subsequent layer upgrades, it effectively avoids the invalid frequent erases and writes caused by disordered entangled reads and writes to the high-performance solid-state storage media, significantly extending the hardware fatigue life and achieving dual optimization of storage system response acceleration and low-cost hardware utilization.

[0009] According to the present invention, a transparent archival data migration and backtracking method across storage media is provided, wherein the target archival block area refers to a fixed logical address range in a low-speed archival medium that requires hot / cold status determination and migration decision; the operation type, logical address offset and arrival timestamp of the read / write request are intercepted and recorded in real time by a front-end interception agent deployed on the storage volume manager or block device layer.

[0010] According to the present invention, a transparent archiving data migration and backtracking method across storage media is provided. The calculation of the interleaving pollution deviation degree, which is used to evaluate the degree of pollution of the high-performance cache layer by read and write requests, includes: setting a dynamically sliding time window on the IO request sequence, counting the total number of alternations between read and write requests in the current time window, and the absolute value of the maximum logical address span between read and write alternation requests; multiplying the total number of alternations between read and write requests by the absolute value of the maximum logical address span, and then dividing by the window duration of the current time window to obtain the interleaving pollution deviation degree.

[0011] This invention directly multiplies the total number of alternations representing the frequency of context switching in the time dimension with the absolute value of the maximum logical address span representing the addressing and paging distance in the spatial dimension, and performs a sliding time window averaging process. This objectively characterizes the spatiotemporal interweaving that causes the underlying flash memory mapping table oscillation and channel switching overhead at the abstract logic level, providing a basis for judgment in subsequent interception of disruptive service flows.

[0012] According to the present invention, a method for transparent archived data migration and backtracking across storage media is provided. The calculation of the cache pollution index includes: multiplying the normalized value of the interleaving pollution deviation by the interleaving pollution risk amplification weight, and dividing by the normalized value of the cache capacity to obtain a first risk term; dividing the difference between the actual appended write accumulation and the safe write baseline by the safe write baseline, and multiplying by the wear risk amplification weight to obtain a second risk term; and adding the first risk term and the second risk term to obtain the cache pollution index.

[0013] This invention mathematically couples the interleaving contamination deviation, the normalized value of the available cache capacity allocated by the media, and the actual amount of additional write overruns with the dual risk amplification weights. This enables a unified assessment of the cache exhaustion risk caused by logical addressing interleaving and the physical lifetime decay risk caused by overruns. In turn, it builds a double-insurance defense barrier that balances high-speed performance and media lifetime from the source of data flow, accurately blocking unnecessary high-frequency invalid migrations.

[0014] According to the present invention, a transparent archiving data migration and backtracking method across storage media is provided. The calculation of the cache pollution index further includes correcting the weighting of the interleaving pollution risk amplification: during the system operation monitoring cycle, the actual physical cache hit rate of the high-performance cache layer and the actual media erase / write wear rate of the underlying layer are collected; based on the difference between the actual physical cache hit rate and the preset target cache hit rate, and the difference between the actual media erase / write wear rate of the underlying layer and the target secure erase / write wear rate, the weighting of the interleaving pollution risk amplification is corrected in a reverse closed loop.

[0015] This invention establishes an end-to-end negative feedback closed-loop adjustment mechanism for physical indicators by continuously collecting the actual physical cache hit rate and the actual wear rate of the underlying media. When extreme changes occur in the business type or the equipment enters the mid-to-late stage, it gives the evaluation engine the ability to autonomously and reversely fine-tune the risk weights according to the actual decay state of the hardware. This effectively prevents excessive interception or unchecked pollution caused by rigid algorithms and ensures the robustness of the entire evaluation system over a very long life cycle.

[0016] According to the present invention, a transparent archiving data migration and backtracking method across storage media is provided, wherein the reverse closed-loop correction of the interleaving contamination risk amplification weight includes: reducing the interleaving contamination risk amplification weight when the actual physical cache hit rate is lower than the target cache hit rate; and increasing the interleaving contamination risk amplification weight when the underlying actual media erase / write wear rate is higher than the target secure erase / write wear rate.

[0017] According to the present invention, a transparent archive data migration and backtracking method across storage media is provided. The step of identifying the physical separation boundary between a high-hot append write subblock and a pure read-only cold flow subblock within the target archive block area includes: scanning the logical address space of the target archive block area through a sliding time window, calculating the relative rate of change of the absolute difference between the write request frequency and the read request frequency within adjacent time windows; if the relative rate of change is greater than a set proportion, it is determined that there is a density mutation point, and the position with the largest relative rate of change is taken as the physical separation boundary, and the target archive block area is divided into two subblocks along the physical separation boundary; wherein, the subblock in which write requests are absolutely dominant is marked as a high-hot append write subblock, and the other subblock in which read requests are absolutely dominant is marked as a pure read-only cold flow subblock.

[0018] This invention scans the relative change rate of the absolute difference between hot and cold frequencies in adjacent regions using a sliding time window. Without the need for manual intervention or pre-set prior knowledge, it automatically locates the boundary of density cliff-like abrupt changes. It rigidly separates the macroscopic block, which was originally a unified whole, into completely mutually exclusive high-hot write domains and pure cold read domains. This physically blocks the erosion of scarce high-speed solid-state cache space by long-tail cold flow data, ensuring a leap in the hit rate of the cache layer.

[0019] According to the present invention, a transparent archive data migration and backtracking method across storage media is provided, wherein the high-hot append write subblock routing is migrated to the high-performance cache layer, and before that, the method includes: writing to the write-ahead log and forcibly flushing to disk; after the system restarts, the incomplete transactions are rolled back or replayed by scanning the write-ahead log.

[0020] According to the present invention, a transparent archive data migration and backtracking method across storage media is provided, wherein the high-hot append write subblock routing migration to the high-performance cache layer includes: intercepting micro-block write requests of the migrated high-hot append write subblock in the memory aggregation buffer pool of the high-performance cache layer, and sorting them in the memory aggregation buffer pool according to the arrival timestamp order; in response to the total amount of data accumulated in the memory aggregation buffer pool reaching an integer multiple of the underlying physical flash page size, concatenating the corresponding set of micro-blocks into a continuous large block and sequentially flushing it to the continuous physical block region of the high-performance cache layer, and establishing a redirection mapping relationship from the original logical address to the newly allocated physical block address in the global metadata mapping table.

[0021] According to the present invention, a transparent archival data migration and backtracking method across storage media is provided. The method involves migrating high-hot append write subblocks to a high-performance cache layer, followed by data backtracking: maintaining hot monitoring of high-hot append write subblocks residing in the high-performance cache layer; when no append write requests are detected in multiple consecutive monitoring periods and the frequency of read requests is lower than the cold flow backtracking threshold, the high-hot append write subblocks are written back to the low-speed archival media layer in the original logical address order, and the corresponding redirection mapping relationship in the global metadata mapping table is deleted or invalidated.

[0022] The present invention has the following beneficial effects: Based on the above technical solution, this invention provides a transparent archival data migration and backtracking method across storage media. By constructing a spatiotemporal IO interleaving feature tracking model, the archive block area, which is regarded as a unified thermodynamic whole under traditional fixed block granularity monitoring, is deconstructed into a set of multi-service flows heterogeneously entangled in the time axis and logical address space. A dynamic sliding time window is used to capture the coupling strength between the frequency of alternating read and write requests and the maximum logical address span in real time. This transforms the high-frequency append writes at the end of ultra-large archive files and the large-span read-only cold flow falling into the same block area into a measurable interleaving pollution deviation. On this basis, the interleaving pollution deviation is fused with cache capacity constraints and media-safe write baselines to form a cache pollution index, establishing a dynamic survival risk boundary that directly maps the underlying flash memory addressing and paging pressure and physical erase / write fatigue threat. Based on this risk boundary, sub-block-level splitting is triggered, precisely tracking along the cold and hot density abrupt change points. This method accurately identifies the physical separation boundary between hot append write subblocks and pure read-only cold flow subblocks, changing the thermodynamic binding of cold read flow to hot write flow under the whole block migration mechanism. It routes hot append write subblocks to the high-performance cache layer and reassembles them into continuous physical blocks sequentially through the memory aggregation buffer pool according to their arrival time. This eliminates the addressing oscillation of the flash memory mapping table and the invalid wear of physical erase blocks caused by random micro-block writes. At the same time, it rigidly locks pure read-only cold flow subblocks to the low-speed archive media layer, preventing their associated layer promotion from crowding out cache space and causing invalid pollution. Thus, it achieves physical isolation and differentiated transparent migration of hot and cold data within the block area. While ensuring low-latency response of hot data, it also constrains the cache hit fidelity and physical erase / write lifetime of the high-performance media to a safe baseline, effectively solving the problems of cache pollution and sudden drop in media lifetime caused by cold data layer promotion in the existing technology. Attached Figure Description

[0023] Figure 1 A flowchart illustrating the steps of a transparent archival data migration and backtracking method across storage media provided in this embodiment of the invention; Figure 2 This is a schematic diagram of a sliding window partitioning of logical address space provided in an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0025] This invention provides a transparent method for archiving data migration and backtracking across storage media, applicable to scenarios with multiple concurrent service flows, enabling fine-grained isolation, routing migration, and hardware adaptation optimization of massive amounts of archived data between high-performance cache media and low-speed archive media.

[0026] For example, the system involved in this embodiment of the invention is deployed between a distributed storage controller and the underlying physical storage medium, and includes at least an IO feature acquisition module, a spatiotemporal IO interleaving feature tracking model, a dynamic cache pollution risk assessment module, a sub-block level splitting and reorganization engine, a high-performance cache layer space allocation manager, a memory aggregation buffer pool, a global metadata mapping table, and a multi-dimensional physical negative feedback closed-loop correction module. The above modules interact with each other through the internal control bus or message queue of the storage system. Their system layout and connection relationships are conventional designs in the art and will not be described in detail here.

[0027] Please see Figure 1 , Figure 1 A flowchart illustrating the steps of a transparent archived data migration and backtracking method across storage media provided in this embodiment of the invention includes the following steps: S1: Obtain the IO request sequence of the target archive block area on the timeline.

[0028] For example, a spatiotemporal I / O interleaving feature tracking model can be constructed to collect the operation type, logical address offset, and arrival timestamp of each read / write request in the target archive block area in real time, so as to obtain the I / O request sequence of the target archive block area on the time axis.

[0029] In this embodiment of the invention, the target archive block area refers to a fixed logical address range in the low-speed archive medium that requires hot / cold status determination and migration decisions. Its size is pre-defined by the volume management strategy of the storage system. The IO request sequence is intercepted and recorded in real time by a front-end interception agent deployed at the underlying storage volume manager or block device layer. Specifically, it includes the operation type, logical address offset, and arrival timestamp of each read / write (Input Output, or IO) request.

[0030] It should be noted that, in order to distinguish the overlapping phenomenon of high-frequency append writes at the end of large-scale archive files and discrete sequential read-only cold flows occurring in the same time and space, this embodiment of the invention introduces spatiotemporal interleaving feature tracking, which can accurately analyze the intensity of entanglement of read and write requests in physical hardware space and time axis, and provide multi-dimensional physical data support with underlying hardware correlation for subsequent routing allocation.

[0031] This invention embodiment sets a dynamically sliding time window on the IO request sequence. Specifically, a fixed-length subsequence of IO requests can be continuously extracted in time as the current analysis window. The time window slides forward with a fixed step size over time, thereby continuously updating the data samples used for feature calculation. In this invention embodiment, the window duration of the dynamic sliding time window mechanism can be denoted as... Its value can be pre-configured by the system administrator based on the business load characteristics of the target archive block area, and for example, it is an integer value between 30 seconds and 300 seconds.

[0032] Optionally, the window duration can also be set automatically by the system. Specifically, it is the average I / O request arrival interval of the target archive block area in the most recent historical statistical period. For example, 24 hours; ,in The amplification factor is set to 1000 to 10000 for example, and a lower limit constraint and an upper limit constraint are imposed on the window duration. For example, the window duration is limited to not less than 10 seconds and not more than 600 seconds.

[0033] Thus, by capturing the original read and write log sequences, this embodiment of the invention can effectively establish a spatiotemporal tracing model of concurrent IO characteristics, thereby providing an accurate data foundation for subsequent assessment of the pollution risk of high-performance cache layers.

[0034] S2: Determine the degree of deviation of interwoven pollution.

[0035] For example, the frequency of alternation of read and write requests in the target archive block area on the time axis and the span distance in the logical address space can be calculated based on the IO request sequence, and then the interleaving pollution deviation degree used to evaluate the degree of pollution of the high-performance cache layer by read and write requests can be calculated.

[0036] Furthermore, for the IO request sequence intercepted and sorted by arrival timestamp within the current sliding time window, the total number of read-write alternating requests and the absolute value of the maximum logical address span between read-write alternating requests within each time window are extracted to identify key basic variables with cache crowding tendencies.

[0037] In this embodiment of the invention, when obtaining the total number of alternations between read and write requests, the request sequence sorted by arrival timestamp within the current time window can be traversed. The operation types of adjacent requests are compared sequentially. If the operation type of the i-th request is different from that of the (i+1)-th request, i.e., a combination of adjacent read and write requests or adjacent write and read requests occurs, then a read-write alternation is determined, and the total number of read-write alternation requests is incremented by one. Here, i represents the arrival order of the requests. It should be noted that for concurrent requests with identical arrival timestamps, a secondary sort is performed according to their logical address offsets from smallest to largest to determine their sequential position in the sequence and thus their adjacency. Optionally, the order can also be determined according to the actual dequeue order of the requests in the physical queue, which can be set according to actual needs.

[0038] In this embodiment of the invention, when obtaining the absolute value of the maximum logical address span between read-write alternation requests, for each read-write alternation event included in the read-write alternation request, the logical address offsets of the two adjacent requests involved in this alternation are obtained, and the absolute value of the difference between the two is calculated and recorded as the address span of this alternation event; among all alternation events within the current time window, the maximum address span is selected as the absolute value of the maximum logical address span. The logical address offset can be uniformly calculated based on the standard 512-byte sector address or the 4KB aligned address, and its address unit is determined during system initialization and remains globally consistent.

[0039] Furthermore, the interleaving pollution deviation is calculated to evaluate the degree of pollution of the high-performance cache layer by read and write requests. This includes: counting the total number of alternations between read and write requests within the current time window, and the absolute value of the maximum logical address span between alternating read and write requests; multiplying the total number of alternations between read and write requests by the absolute value of the maximum logical address span, and then dividing by the window duration of the current time window to obtain the interleaving pollution deviation.

[0040] In the process of constructing the interleaved pollution deviation, the hardware essence of cache pollution is the hardware addressing failure caused by the high spatial separation and high temporal density of read and write requests.

[0041] Therefore, by directly multiplying the total number of alternations between read and write requests (representing the frequency of context switching in the time dimension) with the absolute value of the maximum logical address span (representing the distance between cache pages in the spatial dimension), and then normalizing the result by dividing the denominator by the window duration, the true spatiotemporal oscillation rate of the storage medium can be objectively calculated. This value directly maps the frequent switching overhead of the internal bus channels of the storage medium and the addressing and paging pressure of the flash memory mapping table at the physical level.

[0042] S3: Calculate the cache pollution index.

[0043] For example, the cache pollution index can be calculated based on the interleaving pollution deviation, the currently available cache capacity of the target archive block area, the actual cumulative append writes, and the safe write baseline of the underlying storage medium.

[0044] It is important to note that when underlying archived data blocks encounter complex interleaved business flows, relying solely on the interleaving contamination deviation calculated at the front end cannot objectively reflect the real physical write fatigue threat it poses to the upper-layer high-performance media. Conventional frameworks cannot combine logical-level read-write interleaving with the actual usable capacity or safe wear life of the underlying physical media, and static threshold assessments mask the potential risk of media bandwidth oscillations. Therefore, it is necessary to construct a risk assessment system directly related to the physical state of the hardware, mathematically integrating interleaving characteristics with cache capacity limitations and write life limits, thereby accurately quantifying mixed business flows at the source of data flow and preventing unnecessary physical wear of high-performance media.

[0045] Based on this, embodiments of the present invention can combine interleaving contamination deviation, available cache capacity allocated by the media, and actual additional write throughput to construct a cache contamination risk assessment logic oriented towards the dual constraints of the underlying hardware, accurately mapping the abstract interleaving and entanglement characteristics into a hardware survival risk scalar.

[0046] In this embodiment of the invention, the hardware basic parameters of the target archive block area include at least the cache capacity currently available to the target archive block area from the high-performance cache layer, the secure write baseline of the underlying high-performance storage medium, and the actual cumulative amount of append writes within the current time window.

[0047] Specifically, the cache capacity can be obtained in real time by querying the storage controller management interface of the high-performance cache layer. For example, if the high-performance cache layer uses page-based management, the cache capacity is the number of currently free cache pages multiplied by the physical capacity of a single page, where the single page capacity is 4KB in this example. If a unified cache pool shared architecture is used, the cache capacity is the available capacity value allocated from the global free cache capacity according to the quota ratio of the target archive block area.

[0048] Specifically, the actual cumulative append write volume is the total amount of data written by append write requests actually received by the target archive block within the current monitoring period or the current sliding time window. Its unit is data volume, typically MB or GB. Append write requests can be identified by determining whether the logical address offset of the write request is greater than the maximum logical address offset of the data currently stored in the target archive block; alternatively, it can be identified by the append write flag provided by the file system layer.

[0049] Specifically, the secure write baseline can be determined based on the physical lifespan specifications of the underlying high-performance storage medium. In this embodiment of the invention, the system pre-enters the nominal total write bytes or nominal maximum program / erase cycles of the solid-state drive (SSD) or flash memory medium used, and calculates the permissible cumulative write volume for the current cycle based on the service life and the system-set safe usage ratio coefficient. For example, the secure write baseline... for Where TBW is the manufacturer's claimed total bytes written. For safety, the ratio is typically set between 0.7 and 0.9. The total service days of the system design. The number of days in the monitoring period; optionally, for a running system, the write baseline can be directly set to the maximum cumulative write volume threshold allowed within the current monitoring period, for example, 0.5 to 1.0 times the nominal daily safe write volume of the media.

[0050] For example, in an embodiment of the present invention, calculating the cache pollution index includes: multiplying the normalized value of the interleaving pollution deviation by the interleaving pollution risk amplification weight, and dividing by the normalized value of the cache capacity to obtain a first risk term; dividing the difference between the actual appended write accumulation and the safe write baseline by the safe write baseline, and multiplying by the wear risk amplification weight to obtain a second risk term; and adding the first risk term and the second risk term to obtain the cache pollution index.

[0051] The cache contamination index is used to assess the degree of contamination threat posed by the target archive block area to high-performance media.

[0052] It should be noted that, to ensure the additive nature of the two risk contribution components mentioned above, in this embodiment of the invention, both the interleaving contamination deviation and the available cache capacity are normalized to a dimensionless value before being substituted into the formula. Specifically, the interleaving contamination deviation is divided by the maximum interleaving contamination deviation reference value that occurred in the target archive block within a historical statistical period, such as the last 7 days; after obtaining the normalized value of the interleaving contamination deviation, the available cache capacity is divided by the maximum cache capacity quota allocated to the target archive block in the high-performance cache layer to obtain the normalized value of the cache capacity. Thus, the first term, which physically represents the interleaving contamination load risk score to be borne per unit of available cache, is a dimensionless quantity.

[0053] In the second item, the excess cumulative wear ratio is obtained by dividing the difference between the actual additional write accumulation and the safe write baseline by the safe write baseline. This ratio physically assesses the extent to which the actual cumulative write accumulation exceeds the safe baseline within the current cycle and is a dimensionless quantity. (Interleaved contamination risk amplification weight) Weighting of wear and tear risk The initial value is preset by the system according to the business type and satisfies the normalization constraint. For example, for read-write hybrid services, The initial value is 0.6. The initial value is set to 0.4; for read-intensive archiving operations, The initial value is 0.4. The initial value is 0.6; for write-intensive log operations, The initial value is 0.7. The initial value is 0.3.

[0054] In the above acquisition method, the survival risks faced by high-performance hardware stem from two independent physical dimensions: cache page exhaustion and erase block wear. The first part divides the interleaving pollution deviation by the available cache capacity, physically representing the cache exhaustion rate per unit capacity; the second part subtracts the hardware safe write baseline from the actual write volume and normalizes it, accurately extracting the excessive write amplification burden physically. By linearly superimposing these two parts through weighting, the risk of logical addressing failure and the risk of physical media degradation are unified, forming a closed-loop hardware health threat indicator.

[0055] Thus, by calculating the cache pollution index, this embodiment of the invention can assess the degree of pollution of the high-performance cache layer by the disordered data flow, thereby providing an accurate data basis for the subsequent launch of the sub-block level split and reorganization engine.

[0056] In the calculation of the cache pollution index, the embodiments of the present invention can also correct the weight parameters of the model based on the multidimensional physical negative feedback closed loop.

[0057] It should be noted that the workload of large-scale distributed storage systems fluctuates dramatically throughout their lifecycle. The statically configured adjustment parameters in the model may experience physical degradation after long-term operation, leading to a significant decrease in the hit rate of pollution identification or excessive triggering of interceptions. Therefore, an end-to-end dynamic feedback calibration link can be introduced. This leverages the underlying physical telemetry metrics of the actual hardware operation to reverse-correct the amplified weights in the pre-calculation formula, ensuring that the method retains long-term physical robustness even under extreme business flow mutation scenarios.

[0058] In this embodiment of the invention, a weight parameter iterative update can be performed once at the end of each preset system monitoring cycle. The system monitoring cycle is typically 5 minutes, 15 minutes, or 30 minutes, and is pre-configured by the system administrator according to the frequency of business load fluctuations. Optionally, the monitoring cycle can also be adaptively set, for example, to 1 / 4 to 1 / 2 of the historical business peak interval cycle, with a minimum value of 1 minute and a maximum value of 2 hours.

[0059] For example, in this embodiment of the invention, calculating the cache contamination index further includes correcting the weighting of the interleaving contamination risk amplification: during the system operation monitoring cycle, the actual physical cache hit rate of the high-performance cache layer and the actual media erase / write wear rate of the underlying layer are collected; based on the difference between the actual physical cache hit rate and the preset target cache hit rate, and the difference between the actual media erase / write wear rate of the underlying layer and the target safe erase / write wear rate, the weighting of the interleaving contamination risk amplification is corrected in reverse closed loop.

[0060] The preset target cache hit rate can be determined according to the service level agreement requirements of the upper-layer business. For example, for online transaction processing business, the target cache hit rate is 95% to 99%; for batch archiving query business, the target cache hit rate is 80% to 90%.

[0061] The target safe write wear rate can be determined based on the manufacturer's specifications of the underlying high-performance storage medium. For example, the average daily wear rate corresponding to 70% of the nominal total write bytes is taken.

[0062] Specifically, at the end of each system monitoring cycle, the true physical cache hit rate can be obtained by statistically analyzing the proportion of data requests that hit the physical cache among the read requests from upper-layer services to the high-performance cache layer during that monitoring cycle. The number of read requests that hit the cache is recorded as the total number of read requests. The actual wear rate of the underlying media is obtained by reading the standard SMART attribute of the underlying SSD. For example, SMART attribute 241 or attribute 177 is read, and combined with the physical page size and service life of the media, it is converted into the equivalent physical erase block wear ratio or cumulative wear index in the current cycle. Optionally, if the storage controller maintains an independent physical erase block erase count counter, the statistical value of this counter is directly used as the actual wear rate of the underlying media.

[0063] For example, in this embodiment of the invention, the reverse closed-loop correction of the interleaving contamination risk amplification weight includes: reducing the interleaving contamination risk amplification weight when the actual physical cache hit rate is lower than the target cache hit rate; and increasing the interleaving contamination risk amplification weight when the underlying actual media erase / write wear rate is higher than the target secure erase / write wear rate.

[0064] Furthermore, the closed-loop iterative correction relationship for the weighting of intertwined pollution risk amplification is defined as follows: ; in, The weighting of intertwined pollution risk has been amplified in this round of iteration and update; The pollution amplification weight is applied to the area before the previous update. Target cache hit rate; This represents the actual physical cache hit rate. Optimize the step size for the learning rate; This represents the actual wear rate of the underlying media during erasure and writing. The target is the write wear rate for safe erasure.

[0065] The learning rate optimization step size is typically set to a fixed value, such as 0.01, 0.05, or 0.1, and is pre-configured by the system based on the desired closed-loop response speed. Alternatively, the learning rate optimization step size can also adopt an adaptive adjustment strategy. Specifically, the weight update direction in the last three system monitoring cycles is recorded. If the updates are continuously in the same direction, the current step size is maintained. If directional oscillation occurs, the step size is halved until it drops to a preset minimum value, for example, 0.001, in order to suppress oscillations and accelerate convergence.

[0066] The aforementioned physical control logic is that when the actual cache hit rate is lower than the target value, This indicates that the current contamination interception intensity may be excessive, causing some valid hot data that should be cached to be incorrectly intercepted in the low-speed archiving layer. Therefore, it is necessary to reduce the contamination amplification weight, decrease the sensitivity of subsequent trigger separation, and allow more data to be promoted to improve the hit rate; when the actual erase and write wear rate exceeds the safe value, This indicates that the current write pressure is too high, and the high-performance media is facing the threat of excessive wear. Therefore, it is necessary to increase the contamination amplification weight, improve the sensitivity of subsequent trigger separation, strengthen the fine-grained interception and aggregation control of high-heat write microblocks, and reduce the direct wear of the media caused by disordered random writing.

[0067] Thus, the two deviation factors drive the weights to adjust in opposite directions, forming a stable negative feedback control loop.

[0068] Furthermore, to prevent the weight parameters from diverging or oscillating during the iteration process, embodiments of the present invention can also impose explicit value range constraints on the weights for amplifying the risk of interleaved contamination. Specifically, a lower limit and an upper limit for the weights are set; for example, the lower limit is set as follows: The upper limit of weight is After each iteration, a limiting process is performed.

[0069] For example, when the absolute change in the weight of the intertwined pollution risk amplification is small in three consecutive system monitoring cycles, it can be determined that the closed loop has entered a steady state, and the weight update is suspended until a major business load change is detected and the iteration is reactivated.

[0070] Thus, this embodiment of the invention adjusts the weight of interleaved pollution risk amplification through a multi-dimensional physical negative feedback closed-loop correction mechanism, so that the weight of interleaved pollution risk amplification can adapt to changes in the system, thereby enabling accurate assessment of the risk level of the target archive block area.

[0071] S4: Enables physical isolation and transparent migration of hot and cold data.

[0072] For example, when the cache pollution index exceeds a preset threshold, the physical separation boundary between hot append write subblocks and pure read-only cold flow subblocks can be identified within the target archive block area. The hot append write subblocks can be routed and migrated to the high-performance cache layer, while the pure read-only cold flow subblocks can be maintained in the low-speed archive media layer, so as to achieve physical isolation and differentiated transparent migration of hot and cold data within the same archive block area.

[0073] It should be noted that traditional block-level migration techniques, when determining that a logical block is in an active, hot state, migrate its entire data to the high-performance cache layer all at once. This cascading migration directly leads to a large area of ​​effective high-speed cache being squeezed out and invalidated by archive cold read streams. The embodiments of this invention can change the conservative routing logic of whole-block migration. When the dynamic risk level exceeds the system's preset security boundary, a sub-block-level physical isolation strategy with extremely fine granularity is proactively adopted. This ensures that only the tiny data slices corresponding to high-frequency append writes are accelerated to the next layer, while large-span sequential read-only cold streams are forcibly retained within the low-speed, high-capacity archive media layer. This achieves a physical balance between high-performance storage media lifetime protection and overall read / write response speed.

[0074] In this embodiment of the invention, the system pre-sets a system contamination interception threshold. This threshold is configured by the system administrator based on hardware redundancy and business tolerance, and is typically a constant between 0.5 and 0.8; alternatively, the threshold can also be automatically set by the system based on the statistical quantiles of the historical risk index distribution, for example, taking the 85th percentile value of the cache contamination index within the last 30 days.

[0075] The cache pollution index is rigorously compared with a threshold. If the cache pollution index exceeds the threshold, the underlying sub-block level split and reorganization engine is immediately triggered. The sub-block level split and reorganization engine is a functional module deployed in the logical volume management layer of the storage system, used to perform physical separation and reorganization scheduling of hot and cold data within the target archive block area.

[0076] For example, in an embodiment of the present invention, identifying the physical separation boundary between a hot append write subblock and a pure read-only cold flow subblock within the target archive block area includes: scanning the logical address space of the target archive block area through a sliding time window, calculating the relative rate of change of the absolute difference between the write request frequency and the read request frequency within adjacent time windows; if the relative rate of change is greater than a set ratio, it is determined that there is a density abrupt change point, the position with the largest relative rate of change is taken as the physical separation boundary, and the target archive block area is divided into two subblocks along the physical separation boundary to obtain a hot append write subblock and a pure read-only cold flow subblock.

[0077] Specifically, the sub-block-level splitting and reassembly engine first determines the sliding address time window parameters. The sliding address time window refers to a scanning window covering a continuous sub-interval in the logical address space of the target archive block region. Its window size is typically set to a fixed value, such as 1024 logical blocks, 4096 logical blocks, or 1 / 64 to 1 / 8 of the total logical address range of the target archive block region. The sliding step size of the sliding address time window is typically half the window size to ensure overlapping coverage between adjacent windows and avoid missing boundary abrupt changes. It should be noted that both the window size and step size can be adaptively adjusted by the system according to the total capacity of the target archive block region, but a minimum window constraint must be met. For example, the minimum window is limited to no less than 128 logical blocks to prevent over-subdivision.

[0078] Furthermore, the sub-block-level splitting and reorganization engine starts from the starting logical address of the target archive block and linearly scans the entire logical address space with a sliding step size as the increment. For each sub-interval covered by a time window, the frequency of write requests and the frequency of read requests within the current time window are counted, and the absolute value of the difference between the two is calculated and recorded as the absolute difference between the hot and cold frequencies of that window.

[0079] In this embodiment of the invention, the sub-block-level splitting and reassembly engine performs a single binary split on the target archive block area. Specifically, the engine searches for a unique optimal split boundary within the entire logical address space. This boundary divides the target archive block area into two mutually exclusive sub-blocks: one side is a high-hot append-write sub-block dominated by write requests, and the other side is a pure read-only cold-flow sub-block dominated by read requests. The two sub-blocks are subject to differentiated physical routing scheduling, achieving transparent isolation between hot and cold data.

[0080] Furthermore, the sub-block-level split-recombination engine identifies density abrupt change points by comparing the absolute difference in hot and cold frequencies between adjacent time windows. In this embodiment of the invention, the quantification criterion for a density cliff-like abrupt change is defined as the relative rate of change of the absolute difference in hot and cold frequencies between two adjacent sliding address time windows exceeding a set proportion. .

[0081] Specifically, calculate the relative rate of change. ,in For the first The absolute difference in the frequency of hot and cold events within a time window. For the first The absolute difference in the frequency of hot and cold events within a time window. It is the absolute value symbol. This is a function that maximizes the value. If... Then determine the first Window and the There is a sudden drop in density near the boundary of the window. The set scale is taken as a constant between 0.5 and 0.9, which means that the absolute difference in frequency between adjacent time windows is required to be more than 150% to 300%.

[0082] It should be noted that multiple mutation points satisfying the above conditions may occur during the scanning process. In this embodiment of the invention, the sub-block level split and reorganization engine records the locations of all mutation points and their corresponding relative change rates, and selects the mutation point with the largest relative change rate as the optimal separation boundary, i.e., the physical separation boundary; if multiple identical maximum relative change rate values ​​exist, the mutation point closest to the geometric center of the logical address space is selected as the boundary to balance the sub-block sizes on both sides. If no mutation point satisfying the conditions exists within the entire logical address space, the sub-block level split and reorganization engine abandons this sub-block level separation, maintains the integrity of the target archive block area, and waits for re-evaluation in the next monitoring cycle.

[0083] Furthermore, after determining the optimal separation boundary, the boundary can be smoothed by incorporating a cohesive offset correction to prevent frequent splits. Specifically, a minimum subblock granularity constraint is set, for example, 4MB, 8MB, or 16MB. If any subblock defined by the optimal separation boundary is smaller than the minimum subblock granularity constraint, the boundary is shifted to the nearest neighbor until both subblocks on both sides satisfy the minimum granularity constraint. If the shift results in another subblock being smaller than the minimum subblock granularity constraint, the separation is canceled. The cohesive offset correction is the address offset involved in the above shift operation, and its absolute value does not exceed twice the number of logical blocks corresponding to the minimum subblock granularity constraint.

[0084] Furthermore, after determining the physical separation boundary, the continuity binding of hot and cold data can be rigidly severed at the logical volume mapping layer, generating a high-hot append-write sub-block sequence and a pure read-only cold-flow sub-block sequence. Specifically, sub-blocks on one side of the separation boundary where write requests dominate are marked as high-hot append-write sub-blocks, with an example write request ratio higher than 80%, while sub-blocks on the other side where read requests dominate are marked as pure read-only cold-flow sub-blocks, with an example read request ratio higher than 80%.

[0085] Please see Figure 2 As shown, Figure 2 This is a schematic diagram of a sliding window partitioning of logical address space provided in an embodiment of the present invention.

[0086] The diagram shows a physical separation boundary line in the middle, defining the critical point between hot and cold flow data. This indicates that the system, based on the calculated interleaving contamination deviation and cache contamination index, has identified the physical distribution characteristics of heterogeneous sub-blocks within the block region without disrupting the continuity of the upper-level logical volume.

[0087] The left side corresponds to the high-performance media area, showcasing precisely separated high-hot append-write sub-blocks. These sub-blocks are redirected and migrated to a higher-throughput, lower-latency high-performance cache layer. Continuing along the timeline, subsequent append-write operations in this layer can be executed independently on the high-bandwidth medium, eliminating the bottleneck of asynchronous underlying hardware responses in traditional scheduling. The right side corresponds to the low-speed media area, showcasing pure read-only cold stream sub-blocks. After boundary identification, these sub-blocks remain in the original low-speed archive media layer. In subsequent time series, this layer only responds to pure read-only cold data streams, avoiding cache pollution caused by frequent, sporadic append-write operations.

[0088] Furthermore, the sub-block-level splitting and reorganization engine issues differentiated physical routing instructions to the underlying storage driver layer through an internal control protocol. Each physical routing instruction includes at least a sub-block identifier, a source logical start address, a source logical end address, a destination storage layer identifier, an operation type, and an instruction effective timestamp. The destination storage layer identifier includes either a high-performance cache layer or a low-speed archive layer, and the operation type includes either a migration to a higher layer or maintaining the current location. Upon receiving the physical routing instruction, the storage driver layer parses the instruction content and performs a migration scheduling to the high-performance cache layer for high-hot append-write sub-block sequences. For pure read-only cold-flow sub-block sequences, it performs an isolation maintenance operation to maintain them in the low-speed archive media layer. This ensures that the space partitioning operation occurs precisely within the physical absolute isolation zone between hot and cold flows, preventing secondary interleaving pollution after data fragmentation and layer upgrading from the source of the physical layout.

[0089] After achieving subblock separation based on the above steps, the high-hot append write subblock sequence that has been separated and migrated to the high-performance cache layer can be received, and space layout reorganization based on log aggregation structure can be performed and disked within the space allocation manager of the high-performance cache layer.

[0090] It should be noted that while the preliminary steps have achieved fine-grained extraction and separation of sub-block-level data, directly writing massive amounts of highly fragmented, hot data to high-performance solid-state storage media concurrently will inevitably lead to severe random write amplification penalties and a surge in physical erase block wear on the underlying media. Traditional direct address mapping cannot handle highly discrete hot micro-block streams. Therefore, a physical space aggregation mechanism must be embedded in the high-performance cache receiver to forcibly transform logically discontinuous but temporally intensive fragmented write requests into sequential large-block physical append writes, thereby fully unleashing the underlying concurrent throughput potential of the high-performance storage media.

[0091] For example, in this embodiment of the invention, the route migration of high-hot append write subblocks to the high-performance cache layer includes: intercepting micro-block write requests of the migrated high-hot append write subblocks in the memory aggregation buffer pool of the high-performance cache layer, and sorting them in the memory aggregation buffer pool according to the arrival timestamp order; in response to the total amount of data accumulated in the memory aggregation buffer pool reaching an integer multiple of the underlying physical flash page size, concatenating the micro-block set corresponding to the data amount into a continuous large block and sequentially flushing it to the continuous physical block region of the high-performance cache layer, and establishing a redirection mapping relationship from the original logical address to the newly allocated physical block address in the global metadata mapping table.

[0092] In this embodiment of the invention, the space allocation manager of the high-performance cache layer opens up an independent memory aggregation buffer pool in the controller memory. The memory aggregation buffer pool is a memory area used to temporarily cache micro-block write requests in the high-hot append write sub-block sequence that is being routed and upgraded. It has a preset capacity limit, typically 64MB, 128MB or 256MB. Its specific capacity can be dynamically allocated according to the total controller memory resources of the high-performance cache layer and the number of concurrent migration tasks.

[0093] Furthermore, the space allocation manager actively intercepts all micro-block write requests in the high-hot append write sub-block sequence routed to the media layer. Simultaneously, it abandons the rigid logical address alignment constraints of the original file system and employs a time-arrival-first sorting strategy to reassemble micro-blocks in the memory aggregation buffer pool. In this embodiment, the time-arrival-first sorting strategy refers to a strict first-in-first-out queue organization method, queuing micro-block write requests according to the timestamps of their arrival in the memory aggregation buffer pool. It should be noted that for subsequent read requests, the system queries the global metadata mapping table to redirect the original logical address to the newly allocated physical address, thereby achieving complete transparency to upper-layer services. Optionally, if there are append write requests with explicit data dependencies, such as consecutive appends to the same file handle, the relative order within this dependency sequence remains unchanged.

[0094] Furthermore, the space allocation manager monitors the total backlog of data in the memory aggregation buffer pool in real time. The standard aligned physical size of the underlying physical flash page is determined as follows: During system initialization, the physical sector size or optimal write unit size is read by sending a standard data structure to the SSD used by the high-performance cache layer, and this read value is used as the underlying physical flash page size; if the media returns multiple optional values, the least common multiple or the recommended value is selected as the underlying physical flash page size, with common values ​​being 4KB, 8KB, or 16KB. It should be noted that if the system uses media with various page sizes, the underlying physical flash page size is set independently according to the actual page size used by the current target high-performance cache media.

[0095] If the amount of hot data accumulated in the memory aggregation buffer pool reaches or exceeds an integer multiple of the underlying physical flash page size, the system immediately triggers a high-speed flush operation to disk, typically reaching 1, 2, or 4 times the size. The specific multiple can be set according to the buffer pool capacity and write concurrency. Specifically, a set of micro-blocks whose accumulated data volume reaches an integer multiple of the underlying physical flash page size is sequentially retrieved from the head of the memory aggregation buffer pool. A contiguous physical free block area is requested from the high-performance cache medium, and the retrieved micro-block data is concatenated into a continuous large block in chronological order and written sequentially to this area.

[0096] Furthermore, during the disk write process, the system simultaneously establishes redirection mapping relationships in the global metadata mapping table. In this embodiment of the invention, the global metadata mapping table refers to a data structure that maintains the mapping relationship between the original logical address and the newly allocated contiguous physical block address. It is implemented in key-value pair form and is typically organized based on a hash table or B+ tree index. Each entry contains at least the following fields: original logical start address, original logical length, newly allocated contiguous physical start address, data status flag, creation timestamp, and most recent update timestamp. After the disk write is completed, the system atomically writes the mapping relationship between the original logical address and the newly allocated contiguous physical block address of each microblock in the high-hot microblock set into the global metadata mapping table.

[0097] Furthermore, the memory aggregation buffer pool is configured with backpressure and flow control mechanisms. When the amount of backlogged data in the memory aggregation buffer pool reaches its capacity limit, the system triggers a forced synchronous flush, suspending the reception of new micro-block write requests until the flush is completed and the buffer pool occupancy rate drops below the safe threshold. Optionally, the system can also dynamically expand the capacity of the memory aggregation buffer pool or increase the number of concurrent flush threads to alleviate backpressure.

[0098] It should be noted that the embodiments of the present invention also provide a transparent data backtracking mechanism corresponding to the layer-up migration, which is used to migrate the sub-block sequence with decreasing heat in the high-performance cache layer back to the low-speed archive media layer.

[0099] For example, in this embodiment of the invention, the hot append write subblock is routed to the high-performance cache layer, and then data backtracking is also included: maintaining hot monitoring of the hot append write subblock residing in the high-performance cache layer; when no append write request is detected in multiple consecutive monitoring periods and the frequency of read requests is lower than the cold flow backtracking threshold, the hot append write subblock is written back to the low-speed archive media layer in the original logical address order, and the corresponding redirection mapping relationship in the global metadata mapping table is deleted or invalidated.

[0100] Specifically, the system maintains a heat monitoring timer for each hot append-write sub-block sequence residing in the high-performance cache layer. If no append-write requests are detected for a sub-block sequence within N consecutive system monitoring periods, and the frequency of read requests for that sub-block sequence within those periods is lower than a preset cold flow backtracking threshold (for example, less than one read request per hour on average), then the heat of that sub-block sequence is determined to have decayed, triggering a backtracking migration. Here, N is the number of monitoring periods, typically ranging from 3 to 6.

[0101] Furthermore, during the rollback migration execution, the system merges and reorganizes the sub-block sequence to be rolled back according to the original logical address order, and writes it back to the low-speed, high-capacity archive media sequentially through the write interface of the low-speed archive media layer, releasing the contiguous physical block area occupied by the high-performance cache layer. Simultaneously, it atomically deletes the corresponding redirection mapping entries in the global metadata mapping table, or marks them as invalid, and restores the direct mapping relationship from the original logical address to the low-speed archive layer's physical address. Thus, the upper-layer business's data access path is transparently switched back to the archive layer.

[0102] Optionally, when the overall physical utilization of the high-performance cache layer exceeds a preset capacity alarm threshold, the system initiates a batch backtracking process, typically set to 85% or 90%. Specifically, according to the order of the most recently accessed timestamps of each sub-block sequence from oldest to newest, the system selects several sub-block sequences with the lowest access frequency to perform backtracking first, until the cache layer utilization drops to a safe level.

[0103] By implementing transparent archived data migration and backtracking based on the above steps, the problem of asynchronous data scheduling decisions and underlying storage hardware physical responses in traditional algorithms can be effectively reduced.

[0104] In this embodiment of the invention, the process of migrating the hot append write subblock route to the high-performance cache layer in the above steps also includes: writing a write-ahead log and forcibly flushing it to disk; after the system restarts, by scanning the write-ahead log, performing rollback or replay on incomplete transactions, so as to ensure crash consistency during subblock-level migration, route switching and layout reorganization.

[0105] Specifically, for the flushing operation of the memory aggregation buffer pool, a two-phase commit protocol is adopted to ensure the consistency of data and metadata. The first phase is the preparation phase, in which aggregated hot data is written to a pre-allocated temporary physical free area in the high-performance cache layer, and the mapping relationship between the physical address and the original logical address of the temporary area is written to the write-ahead log. The second phase is the commit phase, in which, after confirming that the data in the temporary area has been successfully written, the redirection mapping relationship in the global metadata mapping table is atomically updated, officially pointing the original logical address to the temporary area, and then the corresponding transaction in the write-ahead log is marked as committed. If the system crashes at any stage, after restarting, the write-ahead log is scanned to perform rollback or replay of incomplete transactions, ensuring that the logical volume mapping layer is always in a consistent state.

[0106] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for transparent archiving data migration and backtracking across storage media, characterized in that, include: S1, construct a spatiotemporal IO interleaving feature tracking model, collect the operation type, logical address offset and arrival timestamp of each read and write request in the target archive block area in real time, so as to obtain the IO request sequence of the target archive block area on the time axis; S2, calculate the frequency of alternation of read and write requests in the target archive block area on the time axis and the span distance in the logical address space based on the IO request sequence, and then calculate the interleaving pollution deviation degree used to evaluate the degree of pollution of the high-performance cache layer by read and write requests; S3, calculate the cache pollution index based on the interleaving pollution deviation, the currently available cache capacity of the target archive block area, the actual cumulative amount of append writes, and the safe write baseline of the underlying storage medium; S4. When the cache pollution index exceeds the preset threshold, the physical separation boundary between the hot append write sub-block and the pure read-only cold flow sub-block is identified within the target archive block area. The hot append write sub-block is routed and migrated to the high-performance cache layer, while the pure read-only cold flow sub-block is maintained in the low-speed archive media layer, so as to achieve physical isolation and differentiated transparent migration of hot and cold data within the same archive block area.

2. The method for transparent archiving data migration and backtracking across storage media according to claim 1, characterized in that, The target archive block area refers to the fixed logical address range in the low-speed archive medium that requires hot / cold status determination and migration decision-making; the operation type, logical address offset, and arrival timestamp of the read / write request are intercepted and recorded in real time by the front-end interception agent deployed on the storage volume manager or block device layer.

3. The method for transparent archiving data migration and backtracking across storage media according to claim 1, characterized in that, The calculation is used to evaluate the interleaving pollution deviation of read / write requests on the degree of pollution of the high-performance cache layer, including: Set a dynamically sliding time window on the IO request sequence, count the total number of times read and write requests alternate within the current time window, and the absolute value of the maximum logical address span between read and write alternating requests; multiply the total number of times read and write requests alternate by the absolute value of the maximum logical address span, and then divide by the window duration of the current time window to obtain the interleaving pollution deviation.

4. The method for transparent archiving data migration and backtracking across storage media according to claim 1, characterized in that, The calculation of the cache pollution index includes: The first risk term is obtained by multiplying the normalized value of the interleaving contamination deviation by the interleaving contamination risk amplification weight and dividing it by the normalized value of the cache capacity. The second risk term is obtained by dividing the difference between the actual additional write accumulation and the safe write baseline by the safe write baseline and multiplying it by the wear risk amplification weight. The first and second risk terms are added together to obtain the cache contamination index.

5. The method for transparent archiving data migration and backtracking across storage media according to claim 4, characterized in that, The calculation of the cache pollution index also includes correcting the weighting of the interleaved pollution risk amplification: During the system operation monitoring cycle, the actual physical cache hit rate of the high-performance cache layer and the actual erase and write wear rate of the underlying media are collected. Based on the difference between the actual physical cache hit rate and the preset target cache hit rate, and the difference between the actual media erase / write wear rate and the target safe erase / write wear rate, the weight of the interleaving contamination risk amplification is corrected in reverse closed loop.

6. The method for transparent archiving data migration and backtracking across storage media according to claim 5, characterized in that, The reverse closed-loop correction of the weighting for the amplification of intertwined pollution risk includes: When the actual physical cache hit rate is lower than the target cache hit rate, the weight of interleaving contamination risk amplification is reduced; when the actual wear rate of the underlying media is higher than the target safe wear rate, the weight of interleaving contamination risk amplification is increased.

7. The method for transparent archiving data migration and backtracking across storage media according to claim 1, characterized in that, The process of identifying the physical separation boundary between hot append-write subblocks and pure read-only cold-flow subblocks within the target archive block area includes: By scanning the logical address space of the target archive block area through a sliding time window, the relative rate of change of the absolute difference between the write request frequency and the read request frequency within adjacent time windows is calculated. If the relative rate of change is greater than a set ratio, it is determined that there is a density mutation point. The position with the largest relative rate of change is used as the physical separation boundary, and the target archive block area is divided into two sub-blocks along the physical separation boundary. The sub-block in which write requests are absolutely dominant is marked as a high-hot append write sub-block, and the other sub-block in which read requests are absolutely dominant is marked as a pure read-only cold flow sub-block.

8. The method for transparent archiving data migration and backtracking across storage media according to claim 1, characterized in that, The process of migrating high-hot append-write subblock routes to a high-performance cache layer previously included: writing to the write-ahead log and forcing a disk flush; and after a system restart, performing rollback or replay of incomplete transactions by scanning the write-ahead log.

9. The method for transparent archiving data migration and backtracking across storage media according to claim 1, characterized in that, The migration of high-hot append-write subblock routes to a high-performance caching layer includes: In the memory aggregation buffer pool of the high-performance cache layer, micro-block write requests of the migrated hot append write sub-blocks are intercepted and sorted in the memory aggregation buffer pool according to the arrival timestamp order; in response to the total amount of data backed up in the memory aggregation buffer pool reaching an integer multiple of the underlying physical flash page size, the micro-block set corresponding to the data amount is concatenated into a continuous large block and then sequentially flushed to the continuous physical block area of ​​the high-performance cache layer, and a redirection mapping relationship from the original logical address to the newly allocated physical block address is established in the global metadata mapping table.

10. A method for transparent archiving data migration and backtracking across storage media according to claim 9, characterized in that, The process of migrating high-hot append-write subblock routes to a high-performance caching layer also includes data backtracking: Maintain heat monitoring for high-hot append write subblocks residing in the high-performance cache layer; When no additional write requests are detected in multiple consecutive monitoring periods and the frequency of read requests is lower than the cold flow backtrace threshold, the high-hot append write subblocks are written back to the low-speed archive media layer in the original logical address order, and the corresponding redirection mapping relationship in the global metadata mapping table is deleted or invalidated.

Images