A data migration method, system and device

Abstract

The application discloses a data migration method, system and device in the computer technology field. In the application, a target dynamic random storage area is used as an extension of a host dynamic random storage area, and a two-level cache structure is built for the host. In addition, data can be migrated from a storage disk to the target dynamic random storage area and from the target dynamic random storage area to the host dynamic random storage area according to data access heat, so that the migration of data in different storage media can be flexibly and finely realized in the granularity of a storage block size, and the hit rate of the host near-end storage is improved as much as possible. The adaptive adjustment of the migration threshold value can also improve the memory hit rate of the migrated data and the host access performance and read-write efficiency, and reduce the extension memory access delay.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data transfer method, system and device. Background Technology

[0002] Currently, when host memory is insufficient, it can be expanded. However, the access latency of expanded host memory is much higher than that of host memory, which affects the efficiency of host data reading and writing. Therefore, how to improve host read and write efficiency is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a data transfer method, system, device, medium and program product to improve host read and write efficiency.

[0004] In a first aspect, this application provides a data migration method applied to a target device, the target device including a target dynamic random access memory (DRAM) and a storage disk, comprising: using the target DRAM and the storage disk as scheduling objects; dividing the storage space of the scheduling objects into multiple storage blocks according to a preset partitioning strategy; determining the first access frequency of each storage block using a first popularity statistics method; calculating the current migration threshold of the migration destination based on the free capacity of the migration destination, the migration frequency of the migration destination, and the historical migration threshold of the migration destination; wherein, the target device is connected to a host, the host including a host DRAM; the target DRAM is an extension of the host DRAM; when the target DRAM is used as the scheduling object, the migration destination is the host DRAM; when the storage disk is used as the scheduling object, the migration destination is the target DRAM; if the first access frequency of any storage block is not less than the current migration threshold, then the data stored in that storage block is migrated to the migration destination.

[0005] Secondly, this application provides a data migration system, including: a host and a target device connected to the host; the target device includes: a first scheduler, a second scheduler, a target dynamic random access memory (DRAM), and a storage disk; the host includes: a host DRAM; the target DRAM is an extension of the host DRAM; the first scheduler is used to: divide the storage space of the target DRAM into multiple storage blocks according to a preset partitioning strategy; determine the first access frequency of each storage block using a first popularity statistics method; and calculate the host DRAM based on the free capacity of the host DRAM, the migration frequency of the host DRAM, and the historical migration threshold of the host DRAM. The current migration threshold of the storage area; if the first access heat of any storage block is not less than the current migration threshold, then the data stored in that storage block is moved to the host dynamic random access memory; the second scheduler is used to: divide the storage space of the storage disk into multiple storage blocks according to a preset partitioning strategy; determine the first access heat of each storage block using the first heat statistics method; calculate the current migration threshold of the target dynamic random access memory based on the free capacity of the target dynamic random access memory, the migration frequency of the target dynamic random access memory, and the historical migration threshold of the target dynamic random access memory; if the first access heat of any storage block is not less than the current migration threshold, then the data stored in that storage block is moved to the target dynamic random access memory.

[0006] Thirdly, this application provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the aforementioned data transfer method.

[0007] Fourthly, this application provides a non-volatile storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned disclosed data transfer method.

[0008] Fifthly, this application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned disclosed data transfer method.

[0009] As can be seen from the above scheme, this application provides a data migration method applied to a target device, which includes a target dynamic random access memory (DRAM) and a storage disk. The method includes: using the target DRAM and storage disk as scheduling objects; dividing the storage space of the scheduling objects into multiple storage blocks according to a preset partitioning strategy; determining the first access frequency of each storage block using a first-degree-of-access statistical method; calculating the current migration threshold of the migration destination based on the free capacity of the migration destination, the migration frequency of the migration destination, and the historical migration threshold of the migration destination; wherein, the target device is connected to a host, and the host includes a host DRAM; the target DRAM is an extension of the host DRAM; when the target DRAM is used as the scheduling object, the migration destination is the host DRAM; when the storage disk is used as the scheduling object, the migration destination is the target DRAM; if the first access frequency of any storage block is not less than the current migration threshold, the data stored in that storage block is migrated to the migration destination.

[0010] As can be seen, the beneficial effects of this application are as follows: the target dynamic random access memory (DRAM) serves as an extension of the host DRAM, constructing a two-level cache structure for the host; and it can move data from the storage disk to the target DRAM and from the target DRAM to the host DRAM based on data access frequency. This allows for flexible and fine-grained data migration between different storage media at the storage block size level, maximizing the hit rate of the host's near-end storage; the current migration threshold of the migration destination is calculated based on the free capacity of the migration destination, the migration frequency of the migration destination, and the historical migration threshold of the migration destination, enabling adaptive adjustment of the migration threshold, improving the memory hit rate of migrated data, enhancing host access performance and read / write efficiency, reducing extended memory access latency, and optimizing memory access performance.

[0011] Correspondingly, the data transfer system, equipment, media, and program product provided in this application also have the above-mentioned technical effects. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0013] Figure 1 This is a flowchart of a data transfer method disclosed in this application;

[0014] Figure 2 This is a schematic diagram of a data transfer system disclosed in this application;

[0015] Figure 3 This is a schematic diagram of another data transfer system disclosed in this application;

[0016] Figure 4 This is a schematic diagram of the structure of a page popularity monitoring and hot page identification module disclosed in this application;

[0017] Figure 5 This is a schematic diagram of an access popularity calculation disclosed in this application;

[0018] Figure 6 This is a schematic diagram of a cache controller structure disclosed in this application;

[0019] Figure 7 This is a schematic diagram of a hot and cold page transfer process disclosed in this application;

[0020] Figure 8 A server architecture diagram provided in this application;

[0021] Figure 9 A terminal structure diagram provided for this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0023] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0024] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] Currently, when host memory is insufficient, it can be expanded. However, the access latency of expanded host memory is much higher than that of host memory, which affects the efficiency of host data read and write. To address this, this application provides a data migration scheme that can flexibly and finely migrate data between different storage media at the storage block size level, maximizing the hit rate of the host's near-end storage. The adaptive adjustment of the migration threshold can improve the memory hit rate of migrated data, enhance host access performance and read / write efficiency, reduce the access latency of expanded memory, and optimize memory access performance.

[0026] See Figure 1 As shown in the figure, this application discloses a data transfer method applied to a target device, the target device including a target dynamic random access memory and a storage disk, including:

[0027] S101. The target dynamic random access memory area and storage disk are used as scheduling objects.

[0028] In this embodiment, when the target dynamic random access memory (DRAM) is used as the scheduling object, the destination for data transfer is the host DRAM. That is, the data stored in each storage block obtained by partitioning the target DRAM needs to be transferred to the host DRAM using the method described in this embodiment. This allows the data to be moved from extended memory (i.e., the target DRAM) to host memory (i.e., the host DRAM), improving the memory hit rate of host read / write operations. When the storage disk is used as the scheduling object, the destination for data transfer is the target DRAM. That is, the data stored in each storage block obtained by partitioning the storage disk needs to be transferred to the target DRAM using the method described in this embodiment. This allows the data to be moved from the underlying storage (i.e., the storage disk, such as an SSD) to host extended memory (i.e., the storage disk's cache), improving the memory hit rate of host read / write operations. The target DRAM and the host DRAM can be uniformly addressed by the host. The target device can be implemented based on FPGA, etc.

[0029] S102. Divide the storage space of the scheduled object into multiple storage blocks according to the preset partitioning strategy.

[0030] The partitioning strategy for the target dynamic random access memory (DRAM) and the storage disk can vary depending on the storage medium. For example, if the target DRAM is DRAM and the storage disk is SSD (Solid State Drive), then different preset partitioning strategies can be set for DRAM and SSD based on the properties of the DRAM and SSD storage chips. In one implementation, the storage space of the scheduling object is divided into multiple storage blocks according to the preset partitioning strategy. This includes: when the target DRAM is the scheduling object, the preset partitioning strategy is a byte partitioning strategy, which divides the storage space of the target DRAM into multiple storage blocks in bytes, achieving byte-level movement and addressing; when the storage disk is the scheduling object, the preset partitioning strategy is a minimum read / write size partitioning strategy, which divides the storage space of the storage disk into multiple storage blocks in units of minimum read / write size (e.g., 4K), achieving minimum read / write size movement and addressing.

[0031] Of course, the partitioning strategies for the target dynamic random access memory (DRAM) and the storage disk can also be the same. In one implementation, the storage space of the scheduling object is divided into multiple storage blocks according to a preset partitioning strategy, including: dividing the storage space of the target DRAM and the storage space of the storage disk into multiple storage blocks respectively using the same preset partitioning strategy.

[0032] S103. Determine the first access popularity of each storage block using the first popularity statistics method.

[0033] To achieve high-precision, low-overhead heatscanning and avoid repeated relocations of the same storage block in the same direction, one implementation determines the first access heatscan of each storage block using a first heatscanning method. This includes: performing hash operations on the addresses of each storage block; determining the corresponding position markers of the storage block in multiple hash tables based on the hash operation results; determining corresponding counter values ​​for each position marker corresponding to the same storage block; selecting the minimum value from the counter values ​​corresponding to the same storage block, incrementing the minimum value by one to obtain a target value, and using the target value as the first access heatscan of that storage block. To facilitate subsequent iterations, for the same storage block, if any counter value other than the target value is less than the target value, the counter value is modified to the target value; otherwise, the counter value remains unchanged. In one implementation, determining corresponding counter values ​​for each position marker corresponding to the same storage block includes: if this is the first calculation for the corresponding storage block, setting the counter values ​​for each position marker corresponding to the corresponding storage block to zero; if this is not the first calculation for the corresponding storage block, retaining the current counter values ​​for each position marker corresponding to the corresponding storage block.

[0034] For example: using hash functions h1(.), h2(.), ..., h d (.) Perform hash operations on the addresses of storage block i to obtain d hash values: h1(i), h2(i), ..., h d (i) represents the corresponding position marker in each of the d hash tables; each position marker corresponds to a counter value. When storage block i is first counted for its first access popularity, the counter value corresponding to each position marker is set to 0. Then, when storage block i is first counted for its first access popularity, the corresponding counter values ​​Ti are... h1(i) Ti h2(i) Ti hd(i) Can be recorded as: 0 h1(i) 1 h2(i) ... X hd(i) Among these counter values, the minimum value, 0, is selected. This minimum value is then incremented by one to obtain the target value, 1. This 1 is used as the first access priority for this storage block. To enable calculations in the next loop, counter values ​​less than 1 (excluding 1) are changed to 1, while counter values ​​not less than 1 (excluding 1) remain unchanged. Therefore, the counter values ​​corresponding to storage block i are changed from 0 to 1. h1(i) 1 h2(i) ... X hd(i) Updated to: 1 h1(i) 0 h2(i) ... X hd(i) When calculating the first access frequency of storage block i for the second time, the corresponding counter values ​​marked at each location corresponding to storage block i are: 1 h1(i) 0 h2(i) ... X hd(i) Based on this, the initial visit popularity is determined. It can be seen that the values ​​of each counter are updated in a timely manner according to the popularity statistics.

[0035] To avoid repeated relocation of the same storage block in the same direction, this embodiment assigns an indicator to each counter value, with 1 representing a hot indicator and 0 representing a cold indicator. In one implementation, after setting the target value as the first access heat of the storage block, instead of immediately determining the heat, it checks whether the counter values ​​corresponding to the storage block have been marked with hot indicators. If the counter values ​​corresponding to the storage block have been marked with hot indicators, it means that the storage block was previously identified as a hot block that needs to be relocated, meaning that the relocation has already been completed. Therefore, to avoid repeated relocation, the step of checking whether the first access heat of the corresponding storage block is not less than the current relocation threshold is skipped, and the process ends directly. If the counter values ​​corresponding to the storage block are not marked with a hot indicator, it means that the storage block was not identified as a hot block that needed to be moved in the previous step, that is, the storage block was not moved in the previous step. In this step, the first access hotness of the corresponding storage block is checked to see if it is not less than the current migration threshold to determine whether the storage block needs to be moved. If the first access hotness of the storage block is not less than the current migration threshold, the storage block can be determined as a hot block, and the data in it needs to be moved to the corresponding upper-level cache. In this case, the counter values ​​corresponding to the storage block are marked with a hot indicator. If it is less than the threshold, the storage block can be determined as a cold block, and the data in it does not need to be moved to the corresponding upper-level cache. In this case, the counter values ​​corresponding to the storage block are marked with a cold indicator, and the process ends.

[0036] After marking each counter value corresponding to the storage block with a hot indicator, a relocation step can be performed, such as moving the data stored in the storage block to the relocation destination. The relocation process can be implemented based on DMA (Data Memory Access).

[0037] S104. Calculate the current relocation threshold of the destination based on the free capacity of the destination, the relocation frequency of the destination, and the historical relocation threshold of the destination; wherein, the target device is connected to a host, and the host includes a host dynamic random access memory (DRAM); the target DRAM is an extension of the host DRAM; when the target DRAM is used as the scheduling object, the relocation destination is the host DRAM; when the storage disk is used as the scheduling object, the relocation destination is the target DRAM.

[0038] To achieve adaptive adjustment and updating of the migration threshold, this embodiment, in one implementation, calculates the current migration threshold of the migration destination based on the free capacity of the migration destination, the migration frequency of the migration destination, and the historical migration threshold of the migration destination. This includes: calculating the threshold parameter Y of the current migration threshold according to a first formula; calculating the difference Q between 1 and Y; and taking the Q-th quantile of the first access frequency of each storage block as the current migration threshold. Wherein, the first formula is: Y = Y t-1 ×[(1+α)a / (1+β) b ]; α is the idle capacity, β is the shift-in frequency, a is the first coefficient corresponding to the idle capacity, and b is the second coefficient corresponding to the shift-in frequency; Y t-1 The threshold parameter for the historical shift threshold is Y, initially... t-1 This is a set value. This embodiment uses the first formula to achieve real-time updating and adjustment of the shift threshold, enabling more precise data shifting and improving memory hit rate. For example, consider five storage blocks: p1, p2, p3, p4, and p5, with first access heat values ​​of 2, 3, 4, 6, and 7 respectively. After sorting the first access heat values ​​from smallest to largest, if Y = 0.2, then the difference between 1 and Y, Q = 5 × (1 - 0.2) = 4. Therefore, the 4th quantile after the sorting, 6, is taken as the current shift threshold.

[0039] S105. If the first access heat of any storage block is not less than the current migration threshold, then the data stored in that storage block is moved to the migration destination.

[0040] This embodiment uses a first-level heat statistics method to move data from the storage disk to the target dynamic random access memory (DRAM), and from the target DRAM to the host DRAM. The target DRAM, as an extension of the host DRAM, constructs a two-level cache structure for the host. Furthermore, it can move data from the storage disk to the target DRAM and from the target DRAM to the host DRAM based on data access frequency. This allows for flexible and fine-grained data movement between different storage media at the storage block size level, maximizing the hit rate of near-end storage on the host. The current migration threshold of the migration destination is calculated based on the free capacity of the migration destination, the migration frequency of the destination, and the historical migration threshold of the destination. This allows for adaptive adjustment of the migration threshold, improving the memory hit rate of migrated data, enhancing host access performance and read / write efficiency, reducing extended memory access latency, and optimizing memory access performance.

[0041] To ensure timely write-to-disk transfer of cold data within the target dynamic random access memory (DRAM), thereby improving DRAM utilization and saving storage space, a second access frequency (BRF) method can be used to quickly perform BRF statistics on the data within the DRAM. Specifically, the BRF method calculates the second access frequency of each data block within the DRAM. The computational complexity of the BRF method is lower than that of the first BRF method. If the BRF of any data block is lower than a preset threshold and the data block has not been modified, it is deleted from the DRAM. If the BRF of any data block is lower than the preset threshold and the data block has been modified, it is moved from the DRAM to the disk. A mapping table can be constructed within the DRAM, recording the address mapping between each data block and the disk, the BRF, and a dirty field. If the dirty bit field corresponding to any data block is the first target value, then the data block is confirmed to have not been modified; if the dirty bit field corresponding to any data block is the second target value, then the data block is confirmed to have been modified.

[0042] In one implementation, the second access popularity of each data block in the target dynamic random access memory (DRAM) is calculated using a second popularity statistics method, including: if the access count of any data block increases by one, the second access popularity of that data block increases by one. To update the preset shift threshold, the value of the preset shift threshold can be updated based on the remaining capacity of the target DRAM, the current shift threshold, and the preset shift threshold.

[0043] The following describes a data transfer system provided by an embodiment of this application. The data transfer system described below can be referred to in conjunction with other embodiments described herein.

[0044] See Figure 2 As shown in the figure, this application discloses a data migration system, including: a host and a target device connected to the host; the target device includes: a first scheduler, a second scheduler, a target dynamic random access memory (DRAM) and a storage disk; the host includes: a host DRAM; the target DRAM is an extension of the host DRAM.

[0045] The host and target device connect using a cache coherence protocol. This protocol can be CXL (ComputeExpress Link), a high-speed interface protocol that optimizes the interaction between compute, storage, and communication resources within a data center. CXL actually consists of three sub-protocols: CXL.io, CXL.cache, and CXL.mem. CXL.io handles initialization, linking, device identification and enumeration, and register access, providing a non-consistent load / store interface for the device. CXL.cache accesses the cache, defining the interaction between the processor and the device, allowing connected CXL devices to use request and response methods to efficiently cache processor memory with extremely low latency. CXL.mem accesses memory, providing the processor with access to the device's attached memory using load and store commands, where the processor acts as the master and the CXL device as the slave, supporting both volatile and persistent memory architectures. These protocols are dynamically multiplexed together, enabling data transfer at 32GT / s through the standard PCIe 5.0 physical layer.

[0046] The first scheduler is used to: divide the storage space of the target dynamic random access memory into multiple storage blocks according to a preset partitioning strategy; determine the first access heat of each storage block using a first heat statistics method; calculate the current migration threshold of the host dynamic random access memory based on the free capacity of the host dynamic random access memory, the migration frequency of the host dynamic random access memory, and the historical migration threshold of the host dynamic random access memory; if the first access heat of any storage block is not less than the current migration threshold, then move the data stored in that storage block to the host dynamic random access memory.

[0047] The second scheduler is used to: divide the storage space of the storage disk into multiple storage blocks according to a preset partitioning strategy; determine the first access heat of each storage block using a first heat statistics method; calculate the current migration threshold of the target dynamic random access memory (DRAM) based on the free capacity of the target DRAM, the migration frequency of the target DRAM, and the historical migration threshold of the target DRAM; and if the first access heat of any storage block is not less than the current migration threshold, then move the data stored in that storage block to the target DRAM.

[0048] In one embodiment, the target device further includes: a cache controller; a first scheduler and a second scheduler invoke the cache controller to perform data migration using direct memory access technology. In one embodiment, the target device further includes: a cache consistency controller; a host is connected to the cache consistency controller, the cache consistency controller is connected to the cache controller, and the cache controller is connected to the first scheduler, the second scheduler, the target dynamic random access memory area, and the storage disk.

[0049] In this embodiment, the data migration from the storage disk to the target dynamic random access memory (DRAM) and the data migration from the target DRAM to the host DRAM are implemented using two schedulers. The two schedulers do not interfere with each other, which can realize data migration under the dual-cache structure and improve the hit rate.

[0050] Please see Figure 3 The target device is implemented using FPGA programmable hardware. This device uses high-performance DIMM (Dual-Inline-Memory-Modules) interface DRAM media as a cache for NVMe SSDs (i.e., the target dynamic random access memory area). A two-level hot-slot dynamic data scheduling strategy is designed to effectively improve the access performance of the CXL memory expansion device, enabling the device to possess both high performance and large capacity characteristics. Specifically, the target device supports two types of interfaces: DIMM interface and NVMe interface, and accordingly, DRAM and flash SSD media are selected for combination.

[0051] Figure 3 The CXL controller (Cache Coherence Controller) supports the CXL.io and CXL.mem protocols and mainly consists of a physical layer, a data link layer, and a transaction layer. The Flexbus physical layer is responsible for physical information exchange, interface initialization, and maintenance. The data link layer is responsible for reliable data transmission services, including flow control, credit management, link status, and transaction management. The transaction layer handles transactions associated with each protocol, performing packet encapsulation and parsing. When a host sends a CXL request, the CXL controller parses it, and the cache controller reads data from the DRAM cache or NVMe SSD, returning read / write responses and results. The host can also access and configure the two page heat monitoring and hot page identification modules via the CXL.io protocol. It should be noted that the data granularity monitored by the two page heat monitoring and hot page identification modules A and B (corresponding to the first and second schedulers) can be flexibly set. The data granularity does not necessarily have to be a physical storage page; it can also be other artificially defined granularities, such as the storage block granularity described in the previous embodiments.

[0052] The page popularity monitoring and hot page identification module can perform functions such as address access frequency statistics, dynamic threshold calculation, and hot page determination. This model monitors access frequency as page popularity, and dynamically calculates popularity thresholds by combining migration frequency and DRAM free capacity. Based on the threshold, it determines whether a page is hot or cold, updates the popularity indicator, identifies newly added hot pages, and performs data migration for them. This module effectively prevents duplicate migrations, avoids resource waste, and improves data migration efficiency and system performance.

[0053] The cache controller is the core module of the device controller, primarily responsible for quickly responding to memory access requests submitted by the CXL controller. Its external interface supports two memory media: DDR DRAM with a DIMM interface and SSDs with an NVMe interface. This model allows multiple DRAM subsystems to provide ample cache space for the SSD, improving the memory access performance of the CXL SSD. Internally, this module implements access scheduling and hot / cold page migration for both the DRAM and SSD subsystems.

[0054] The CXL SSD consists of an NVMe SSD subsystem and a DRAM cache subsystem. The former includes the NVMe IP Core, an SSD read controller based on PCIe Hard IP, while the latter includes the DDR4 DIMMs and their corresponding controller. The NVMe SSD subsystem serves as a high-capacity, persistent storage medium, providing non-volatile data retention capabilities and is the final destination address for data storage in the CXL SSD device. The DRAM cache subsystem acts as a high-speed cache for the SSD, temporarily storing hot data and frequently accessed pages to reduce latency from direct access to the NVMe SSD and improve overall system responsiveness and performance.

[0055] The internal structure of the single-page popularity monitoring and hot page identification module can be found in [reference needed]. Figure 4 Since the frequency of host access requests differs from the frequency of page popularity monitoring, page addresses are first placed in a FIFO buffer for cross-clock domain data transmission. The address access frequency index and frequency value analysis module can use methods such as hardware counters to statistically analyze page access frequencies. Simultaneously, the host can access the analysis results and configure its registers, such as enabling registers and adjusting threshold parameters. The dynamic popularity threshold calculation module dynamically adjusts the popularity threshold based on the address access frequency distribution and the current migration frequency. The new hot page judgment module combines the frequency analysis results and the dynamic threshold to determine the addition of a hot page, preventing duplicate hot page judgments that would waste device resources. Then, the address of the newly added hot page is placed in the hot page buffer for hot page migration or data prefetching.

[0056] The following is an example of how an address access frequency index and frequency value analysis module calculates the first access popularity. For example... Figure 5 As shown, d hash tables are created, and each hash table has w hash buckets (corresponding to...). Figure 5The hash table consists of various boxes. Each hash bucket contains a counter value and a 1-bit hotness indicator (i.e., hot bit), with the hotness indicator taking the value 0 or 1. At the start of the analysis, all counter values ​​T and hotness indicators are set to 0. For the access address sequence P={p1,p2,…,pn} arriving within one analysis cycle, this represents all addresses at a monitoring granularity, such as all addresses of a physical page or all addresses of a memory block. Then, a set of hash functions h1(.), h2(.),…,hd(.) are used to hash this address sequence, thereby determining a unique hash bucket in each hash table. The IDi in this hash bucket is filled with the values ​​of h1(.), h2(.),…,hd(.). For example, if ID1=h1(.) in hash table 1, then T1(ID1)=T1h1(.), where h1(.) represents the position of the hash bucket in hash table 1, which is the tag value of T1, and the 1 in T1 is the tag of hash table 1. Specifically, using the formula IDi = hi(p), i = 1, ..., d, d position markers can be calculated, thereby determining d counter values ​​T1(ID1), T2(ID2), ..., Td(IDd). Using the formula c(p) = min(T... i (ID i Let i = 1, ..., d. Select the minimum value Tmin from T1(ID1), T2(ID2), ..., Td(IDd), and increment the counter of the minimum value by 1 to obtain c. c is the access frequency of this monitoring granularity, which is also the first access popularity. Subsequently, use the formula Ti(IDi) = max(Ti(IDi), c), i = 1, ..., d, to update the counter values. That is, if the value is less than c, set all of them to c; otherwise, leave them unchanged.

[0057] To avoid duplicate data being sent to the hot page buffer, before moving data, the hot page determination module first checks the hot page indicators in the d hash buckets corresponding to the address. If all indicators are 1, it indicates that the page has already been recorded as a hot page, and regardless of whether it is determined to be a hot page or a cold page this time, it does not need to be moved again. If any hash bucket indicator is 0, and c is greater than or equal to the heat threshold θ (i.e., the current migration threshold), it indicates that this is a newly detected hot page. The indicator is then set to 1, and the page address p is sent to the hot page buffer. Specifically, if c is greater than or equal to the heat threshold θ, it is a hot page, and the indicator is 1; if c is less than θ, it is a cold page, and the indicator is 0. This can be expressed by the formula: .

[0058] The current threshold for hot page migration is set as follows: The page popularity monitoring module can obtain the distribution of page access frequency. The (1-Y) quantile is taken as the threshold θ for hot page determination, that is, the percentage of data access frequency Y is higher than the threshold θ. Page popularity monitoring and hot page identification modules A and B can take different Y values ​​and corresponding θ values ​​respectively. The Y value can be dynamically adjusted according to the DRAM cache free capacity ratio α and the current data migration frequency β (the number of migrations per unit time). The larger the DRAM memory free capacity ratio α is, the larger the Y value is; the higher the current data migration frequency β is, the smaller the Y value is, that is: Y=Y0×[(1+α)] a / (1+β) b []. Where a and b are adjustable parameters. After the Y value is dynamically determined, the popularity threshold θ is obtained from the access frequency distribution. This can avoid the ping-pong phenomenon caused by excessively frequent page migrations or migrations of data with too low popularity, and control the data migration overhead. Y0 is the threshold obtained from the previous calculation.

[0059] Please see Figure 6 The cache controller is the core module in the device controller. It is mainly responsible for quickly responding to memory access requests submitted by the CXL controller and implementing dynamic data scheduling between the two types of memory.

[0060] The cache controller may include an access control module. The DRAM cache contains a cache mapping table, which mainly includes three items: the mapping relationship between cache pages and SSD physical addresses (the corresponding device physical address Addr field, 46 bits), page access frequency (Freq field, 17 bits, i.e., the second highest access frequency), and a dirty bit (Dirty bit, 1 bit). The access control module's mapping table retrieval engine indexes the table based on the mapping relationship to determine if the DRAM cache has a hit: if a hit, it reads or writes cache lines in 64-byte units and maintains information such as page frequency and status; if a miss occurs and there is remaining space in the DRAM cache, a new cache line is allocated; if the DRAM is full, the existing cache line is evicted to the NVMe SSD, and a new mapping relationship is established.

[0061] Regarding the dirty bit in the cache mapping table, if it's a read operation, the dirty bit doesn't need to be changed; if it's a write operation, the dirty bit is changed to 1. When the DRAM cache is fully occupied and new data needs to be written, if the dirty bit of the evicted cold data is 0, it means the NVMe SSD contains clean metadata, and the DRAM cache line can be discarded directly; if the dirty bit of the evicted cold data is 1, it means the cache line has been modified and needs to be written back to the NVMe SSD before the new data is written. Therefore, the dirty bit can be used to maintain data consistency between the DRAM cache and the NVMe SSD.

[0062] The cache controller may also include a DRAM cold data monitoring module. For each read / write operation on a DRAM cache line, the Freq field of the corresponding page entry in the cache mapping table needs to be incremented by 1. Since DRAM caches are generally less than 16GB, and the frequency counter cache overhead for each 4KB page is 2B, this counter method can easily achieve hot data statistics with a memory footprint of no more than 8MB. The Freq field can be used to analyze the data access frequency of the current analysis period. Data accessed below the threshold θc is considered cold data. If the dirty flag is 0, it is discarded directly; if it is 1, the cold / hot data scheduling module writes it back to the NVMe SSD, and then resets the dirty flag to 0. Similarly, the cold data judgment threshold (i.e., the preset downward threshold) can be set using the dynamically configured method mentioned earlier.

[0063] The cache controller may also include a cold / hot page migration mechanism. The dynamic migration mechanism between DRAM and NVMe SSD is as follows: when the cache controller detects that a page migration condition has been triggered, it sends a migration request to the DMA engine. The DMA engine is responsible for migrating cold pages from DRAM to the low-speed NVMe SSD to release DRAM cache resources; and migrating hot pages from the NVMe SSD back to the high-speed DRAM to improve cache hit rate.

[0064] Please see Figure 7 The cold page migration process includes: The cache controller detects that the number of cold pages meets the migration conditions and initiates a cold page migration request to the DMA engine, including its register configuration information, such as the source address being DRAM and the destination address being NVMe SSD. The DMA engine notifies the DRAM controller to read the cold page data. The DRAM controller reads the cold page and returns the data. The DMA engine then writes the data to the NVMe SSD through the NVMe controller. The DMA engine is notified upon confirmation of the write completion. The DMA engine sends an interrupt notification to the cache controller to indicate that the migration is complete. The cache controller updates the cache mapping table, clears dirty bits, and releases cache line resources.

[0065] The hot page migration process includes: The cache controller detects that the number of hot pages meets the migration conditions and initiates a hot page migration request to the DMA engine, including its register configuration information, such as the source address being the NVMe SSD and the destination address being DRAM. The DMA engine notifies the NVMe controller to read the hot page data. The NVMe controller reads the hot page and returns the data. The DMA engine writes the data to the DRAM cache through the DRAM controller. The DMA engine is notified upon confirmation that the write is complete. The DMA engine interrupts to notify the cache controller that the migration is complete.

[0066] It should be noted that CXL technology enables SSD storage devices to expand host memory in a low-cost, high-capacity manner. In this embodiment, the target device uses DDR DRAM memory as the cache for NVMe SSDs. The cache controller is designed to support fast read and write of data when the DRAM cache is hit, and supports the migration of hot pages from the device to the host DRAM memory and the hot and cold page scheduling between the NVMe SSD and the DRAM cache within the device. The main advantages are: (1) Memory capacity expansion is achieved based on CXL SSD media, with strong expansion capability and low cost. At the same time, the hot and cold page scheduling between the DRAM cache and NVMe SSD within the device and between the device and the host DRAM memory improves the device access performance. (2) The device measures the hardware-level monitoring of page access frequency, and perceives the hotness of memory access with high precision and low overhead. (3) The cache controller is designed to support DDR DRAM memory as the cache for SSDs, realize byte-level addressing of CXL SSD devices, improve the bandwidth utilization efficiency of the CXL bus, and significantly reduce the access latency of CXL SSD devices when the cache is hit. (4) Page heat monitoring module A monitors, identifies, and extracts the heat of pages that are missed in the DRAM cache and access the local address of the NVMe SSD. The hotter pages are prefetched into the DRAM cache through the cache controller to improve the cache hit rate. (5) Page heat monitoring module B monitors, identifies, and extracts the heat of the CXL SSD access address. It supports the CPU to access and control it, realizes the migration of hot pages to the host DRAM, optimizes resource utilization, reduces the latency of application accessing memory, and reduces the access pressure of CXL SSD. (6) The page heat threshold is dynamically set and is dynamically related to the overall page heat, the current page migration frequency, and the free capacity of the DRAM cache, so as to avoid the increase in migration overhead caused by too frequent migration.

[0067] This achieves: 1) A two-level hot page scheduling architecture, where hot and cold page scheduling between the device's DRAM cache and NVMe SSD improves device access performance, and hot pages between the device and the host are moved to the host DRAM, reducing system memory access latency and optimizing memory access performance. 2) A hardware-level page heat monitoring and hot page identification module, which achieves high-precision, low-overhead frequency statistics and avoids repeated hot page relocation based on hot bit judgment. 3) An adaptive page heat threshold adjustment method and implementation, which dynamically calculates the threshold based on DRAM idle rate and relocation frequency, and uses the (1-Y) quantile method to determine the hot page standard. 4) An intelligent cache management mechanism, supporting a byte-addressable cache controller architecture design, integrating a mapping table for address mapping, cache access frequency counting, and dirty bit management, and a DMA engine-coordinated hot and cold page relocation process.

[0068] This embodiment can be widely applied to edge computing devices and AI inference servers, especially in data-intensive scenarios with high latency requirements, to improve inference speed and responsiveness. Further improvements to the hot page identification module can be made based on machine learning or deep learning technologies to achieve more accurate access pattern prediction and dynamic threshold adjustment, thereby making hot page migration more intelligent.

[0069] The following describes an electronic device provided by an embodiment of this application. The electronic device described below can be referred to in conjunction with other embodiments described herein. The electronic device in this embodiment can be any device or functional module mentioned in the foregoing embodiments.

[0070] This application discloses an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the method disclosed in any of the above embodiments.

[0071] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: using the target dynamic random access memory (DRAM) and the storage disk as scheduling objects; dividing the storage space of the scheduling objects into multiple storage blocks according to a preset partitioning strategy; determining the first access frequency of each storage block using a first popularity statistics method; calculating the current migration threshold of the migration destination based on the free capacity of the migration destination, the migration frequency of the migration destination, and the historical migration threshold of the migration destination; wherein, the target device is connected to a host, and the host includes a host DRAM; the target DRAM is an extension of the host DRAM; when the target DRAM is used as the scheduling object, the migration destination is the host DRAM; when the storage disk is used as the scheduling object, the migration destination is the target DRAM; if the first access frequency of any storage block is not less than the current migration threshold, then the data stored in that storage block is migrated to the migration destination.

[0072] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: when the target dynamic random access memory is used as the scheduling object, the preset partitioning strategy is the byte partitioning strategy, and the storage space of the target dynamic random access memory is divided into multiple storage blocks in units of bytes according to the byte partitioning strategy; when the storage disk is used as the scheduling object, the preset partitioning strategy is the minimum read / write size partitioning strategy, and the storage space of the storage disk is divided into multiple storage blocks in units of minimum read / write size according to the minimum read / write size partitioning strategy.

[0073] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: using the same preset partitioning strategy, the storage space of the target dynamic random access memory area and the storage space of the storage disk are divided into multiple storage blocks respectively.

[0074] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: perform hash operation on the address of each storage block; determine the position mark corresponding to the corresponding storage block in multiple hash tables according to the hash operation result; determine the corresponding counter value for each position mark corresponding to the same storage block; select the minimum value among the counter values ​​corresponding to the same storage block, add one to the minimum value to obtain the target value, and use the target value as the first access heat of the storage block.

[0075] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: for the same storage block, if the counter value other than the target value is less than the target value, then the counter value is modified to the target value; if the counter value other than the target value is not less than the target value, then the counter value is kept unchanged.

[0076] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: detecting whether each counter value corresponding to the storage block has been marked with a hot indicator; if each counter value corresponding to the storage block has been marked with a hot indicator, then abandoning the step of detecting whether the first access heat of the corresponding storage block is not less than the current shift threshold, and the process ends; if each counter value corresponding to the storage block has not been marked with a hot indicator, then detecting whether the first access heat of the corresponding storage block is not less than the current shift threshold; if not less, then marking each counter value corresponding to the storage block with a hot indicator; if less, then marking each counter value corresponding to the storage block with a cold indicator, and the process ends.

[0077] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: if this is the first calculation of the corresponding storage block, then the counter value corresponding to each position mark of the corresponding storage block is set to zero; if this is not the first calculation of the corresponding storage block, then the current counter value corresponding to each position mark of the corresponding storage block is retained.

[0078] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: calculate the second access popularity of each data block in the target dynamic random access memory using a second popularity statistics method; the computational complexity of the second popularity statistics method is lower than that of the first popularity statistics method; if the second access popularity of any data block is lower than a preset downward shift threshold and the data block has not been modified, then the data block is deleted from the target dynamic random access memory; if the second access popularity of any data block is lower than a preset downward shift threshold and the data block has been modified, then the data block is moved from the target dynamic random access memory to the storage disk.

[0079] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: if the dirty bit field corresponding to any data block is a first target value, then it is confirmed that the data block has not been modified; if the dirty bit field corresponding to any data block is a second target value, then it is confirmed that the data block has been modified.

[0080] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: if the access count of any data block increases by one, then the second access popularity of the data block increases by one.

[0081] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: construct a mapping table in the target dynamic random access memory area, and record the address mapping relationship between each data block and the storage disk, the second access hotness, and the dirty bit field in the mapping table.

[0082] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: update the value of the preset shift threshold according to the remaining capacity of the target dynamic random access memory area, the current shift threshold, and the preset shift threshold.

[0083] Furthermore, embodiments of this application also provide an electronic device. The aforementioned electronic device can be, for example,... Figure 8 The server shown can also be as follows: Figure 9 The terminal shown. Figure 8 and Figure 9 These are all diagrams illustrating the structure of an electronic device according to an exemplary embodiment. The content in the diagrams should not be considered as any limitation on the scope of this application.

[0084] Figure 8 This is a schematic diagram of a server structure provided in an embodiment of this application. The server may specifically include: at least one processor, at least one memory, a power supply, a communication interface, an input / output interface, and a communication bus. The memory stores a computer program, which is loaded and executed by the processor to implement the relevant steps in the data transfer process disclosed in any of the foregoing embodiments.

[0085] In this embodiment, the power supply is used to provide operating voltage for each hardware device on the server; the communication interface can create a data transmission channel between the server and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0086] In addition, the memory, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored on it include operating system, computer programs and data, etc., and the storage method can be temporary storage or permanent storage.

[0087] The operating system manages and controls the various hardware devices and computer programs on the server to enable the processor to perform operations and processes on the data in the memory. It can be Windows Server, Netware, Unix, Linux, etc. In addition to computer programs capable of performing the data transfer methods disclosed in any of the foregoing embodiments, the computer programs may further include computer programs capable of performing other specific tasks. The data may include application update information and application developer information.

[0088] Figure 9 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application. The terminal may include, but is not limited to, a smartphone, tablet computer, laptop computer, or desktop computer.

[0089] Typically, the terminal in this embodiment includes a processor and a memory.

[0090] The processor may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor can be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor may also include a main processor and coprocessors. The main processor, also known as the CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which handles computational operations related to machine learning.

[0091] The memory may include one or more computer non-volatile storage media, which may be non-transitory. The memory may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory is used to store at least the following computer programs, which, after being loaded and executed by the processor, are capable of implementing the relevant steps in the data transfer method executed by the terminal side as disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory may also include operating systems and data, and the storage method may be temporary or permanent storage. The operating system may include Windows, Unix, Linux, etc. The data may include, but is not limited to, application update information.

[0092] In some embodiments, the terminal may further include a display screen, an input / output interface, a communication interface, a sensor, a power supply, and a communication bus.

[0093] Those skilled in the art will understand that Figure 9 The structure shown does not constitute a limitation on the terminal and may include more or fewer components than illustrated.

[0094] The following describes a non-volatile storage medium provided in an embodiment of this application. The non-volatile storage medium described below can be referred to in conjunction with other embodiments described herein.

[0095] A non-volatile storage medium is provided for storing a computer program, wherein the computer program, when executed by a processor, implements the data transfer method disclosed in the foregoing embodiments. The non-volatile storage medium is a computer-readable non-volatile storage medium, which, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored thereon include an operating system, computer programs, and data, and the storage method can be temporary storage or permanent storage.

[0096] The following describes a computer program product provided by an embodiment of this application. The computer program product described below can be referred to in conjunction with other embodiments described herein.

[0097] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned disclosed data transfer method.

[0098] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the steps in any of the above embodiments.

[0099] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0100] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of non-volatile storage medium known in the art.

[0101] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only intended to help understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data migration method, characterized in that, Applied to a target device, the target device including a target dynamic random access memory and a storage disk, comprising: The target dynamic random access memory area and the storage disk are used as scheduling objects; The storage space of the scheduling object is divided into multiple storage blocks according to a preset partitioning strategy; The first access frequency of each storage block is determined using the first-frequency statistics method; The current relocation threshold of the destination is calculated based on the available capacity of the destination, the relocation frequency of the destination, and the historical relocation threshold of the destination; wherein, the target device is connected to a host, and the host includes a host dynamic random access memory (DRAM); the target DRAM is an extension of the host DRAM; when the target DRAM is used as the scheduling object, the relocation destination is the host DRAM; when the storage disk is used as the scheduling object, the relocation destination is the target DRAM. If the first access frequency of any storage block is not less than the current migration threshold, then the data stored in that storage block is moved to the migration destination. Specifically, the current relocation threshold of the relocation destination is calculated based on the available capacity of the relocation destination, the relocation frequency of the relocation destination, and the historical relocation threshold of the relocation destination, including: The threshold parameter Y of the current shift threshold is calculated according to the first formula; Calculate the difference Q between 1 and Y; The Q quantile of the first access heat of each storage block is taken as the current shift threshold. The first formula is: Y = Y t-1 ×[(1+α) a / (1+β) b ]; α is the idle capacity, β is the shift-in frequency, a is the first coefficient corresponding to the idle capacity, and b is the second coefficient corresponding to the shift-in frequency; Y t-1 The threshold parameter for the historical shift threshold is Y, initially... t-1 This is the set value.

2. The method according to claim 1, characterized in that, The storage space of the scheduling object is divided into multiple storage blocks according to a preset partitioning strategy, including: When the target dynamic random access memory is used as the scheduling object, the preset partitioning strategy is a byte partitioning strategy, and the storage space of the target dynamic random access memory is divided into multiple storage blocks in units of bytes according to the byte partitioning strategy. When the storage disk is used as the scheduling object, the preset partitioning strategy is the minimum read / write volume partitioning strategy. According to the minimum read / write volume partitioning strategy, the storage space of the storage disk is divided into multiple storage blocks in units of minimum read / write volume.

3. The method according to claim 1, characterized in that, The storage space of the scheduling object is divided into multiple storage blocks according to a preset partitioning strategy, including: Using the same preset partitioning strategy, the storage space of the target dynamic random access storage area and the storage space of the storage disk are divided into multiple storage blocks.

4. The method according to claim 1, characterized in that, The first access frequency of each storage block is determined using a first-frequency statistics method, including: Perform hash operations on the addresses of each storage block; The position markers of the corresponding storage blocks in multiple hash tables are determined based on the hash operation results; Determine the corresponding counter value for each location marker corresponding to the same memory block; Select the minimum value from the counter values ​​corresponding to the same storage block, add one to the minimum value to obtain the target value, and use the target value as the first access heat of the storage block.

5. The method according to claim 4, characterized in that, Also includes: For the same storage block, if any counter value other than the target value is less than the target value, then the counter value is modified to the target value; if any counter value other than the target value is not less than the target value, then the counter value remains unchanged.

6. The method according to claim 4, characterized in that, After setting the target value as the first access frequency of the storage block, the following is also included: Check whether the counter values ​​corresponding to the storage block have been marked with hot indicators; If the counter values ​​corresponding to the storage block have been marked with a hot indicator, then the step of detecting whether the first access hotness of the corresponding storage block is not less than the current shift threshold is abandoned, and the process ends. If the counter values ​​corresponding to the storage block are not marked with a hot indicator, then it is checked whether the first access hotness of the corresponding storage block is not less than the current shift threshold; if it is not less than, then a hot indicator is marked for each counter value corresponding to the storage block; if it is less than, then a cold indicator is marked for each counter value corresponding to the storage block, and the process ends.

7. The method according to claim 6, characterized in that, After marking each counter value corresponding to the storage block with a hot indicator, the step of moving the data stored in the storage block to the moving destination is performed.

8. The method according to claim 5, characterized in that, Determine the corresponding counter value for each location marker corresponding to the same memory block, including: If this is the first calculation for the corresponding storage block, then the counter value corresponding to each location marker of the corresponding storage block is set to zero; If this is not the first calculation for the corresponding storage block, then the current counter value corresponding to each location marker of the corresponding storage block is retained.

9. The method according to any one of claims 1 to 8, characterized in that, Also includes: The second access frequency of each data block in the target dynamic random access area is calculated using a second access frequency statistical method. The computational complexity of the second popularity statistics method is lower than that of the first popularity statistics method; If the second access frequency of any data block is lower than the preset shift threshold and the data block has not been modified, then the data block is deleted from the target dynamic random access area. If the second access frequency of any data block is lower than the preset shift threshold and the data block is modified, then the data block is moved from the target dynamic random access area to the storage disk.

10. The method according to claim 9, characterized in that, Also includes: If the dirty bit field corresponding to any data block is the first target value, then it is confirmed that the data block has not been modified. If the dirty bit field corresponding to any data block is the second target value, then the data block has been confirmed to have been modified.

11. The method according to claim 9, characterized in that, The second access frequency of each data block within the target dynamic random access memory is calculated using a second access frequency statistical method, including: If the access count of any data block increases by one, then the second access popularity of that data block increases by one.

12. The method according to claim 9, characterized in that, Also includes: A mapping table is constructed in the target dynamic random access memory area. The mapping table records the address mapping relationship between each data block and the storage disk, the second access hotness, and the dirty bit field.

13. A data migration system, characterized in that, include: The host and the target device connected to the host; The target device includes: a first scheduler, a second scheduler, a target dynamic random access memory area, and a storage disk; The host includes: a host dynamic random access memory (DRAM); the target DRAM is an extension of the host DRAM. The first scheduler is configured to: divide the storage space of the target dynamic random access memory (DRAM) into multiple storage blocks according to a preset partitioning strategy; determine the first access frequency of each storage block using a first popularity statistics method; calculate the current migration threshold of the host DRAM based on the free capacity of the host DRAM, the migration frequency of the host DRAM, and the historical migration threshold of the host DRAM; and if the first access frequency of any storage block is not less than the current migration threshold, then move the data stored in that storage block to the host DRAM. The second scheduler is used to: divide the storage space of the storage disk into multiple storage blocks according to a preset partitioning strategy; determine the first access heat of each storage block using a first heat statistics method; calculate the current migration threshold of the target dynamic random access memory (DRAM) based on the free capacity of the target DRAM, the migration frequency of the target DRAM, and the historical migration threshold of the target DRAM; and if the first access heat of any storage block is not less than the current migration threshold, then move the data stored in that storage block to the target DRAM. Specifically, the threshold parameter Y of the current shift threshold is calculated according to the first formula; the difference Q between 1 and Y is calculated; and the Q quantile of the first access frequency of each storage block is taken as the current shift threshold; wherein, the first formula is: Y = Y t-1 ×[(1+α) a / (1+β) b ]; α is the idle capacity, β is the shift-in frequency, a is the first coefficient corresponding to the idle capacity, and b is the second coefficient corresponding to the shift-in frequency; Y t-1 The threshold parameter for the historical shift threshold is Y, initially... t-1 This is the set value.

14. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the method as claimed in any one of claims 1 to 12.

Images