Data migration method, device, apparatus, and storage medium

Abstract

A data migration method, device, equipment and storage medium are disclosed, and relate to the technical field of computers. The method is applied to a chip connected with a first storage medium and a second storage medium, the first storage medium and the second storage medium have different I / O performances, and each N address segment in the first storage medium corresponds to each N address segment in the second storage medium. The method comprises the following steps: after data in a first address segment indication position in the first storage medium is migrated to a second address segment indication position in the second storage medium, updating M-bit migration identification of the first address segment. The M-bit migration identification is used for recording the migration state of the content stored in the N address segments in the first storage medium and the second storage medium.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to data migration methods, apparatus, devices, and storage media. Background Technology

[0002] Memory is one of the key resources supporting various applications and services. Whether it's database queries, big data analysis, or machine learning training, all tasks require a significant amount of memory resources.

[0003] Currently, to facilitate memory and storage management, different types of memory are hierarchically managed based on factors such as access speed, capacity, and cost. This allows data to be stored in different tiers according to its access frequency or importance, achieving tiered memory storage. For example, memory closer to the central processing unit (CPU) (also known as high-speed memory) is more expensive but has faster read / write speeds, while memory farther from the CPU is cheaper but has slower read / write speeds. Frequently accessed data (also known as hot data) is stored in high-speed memory, while relatively infrequently accessed data (also known as cold data) is moved out of high-speed memory and stored in lower-energy-consumption and more economical storage media. In practical applications, the hotness or coldness of data can change dynamically. Data migration can be used to adjust the data's storage location, and metadata records the latest storage address of the data so that subsequent software can accurately access the data based on the address recorded in the metadata.

[0004] However, when there is a large amount of data to migrate, the metadata needs to record a large number of addresses, which results in the storage of metadata occupying a large amount of storage space and affecting memory access speed. Summary of the Invention

[0005] This application provides a data migration method, apparatus, device, and storage medium that reduces the storage space occupied by migration identifiers while improving memory access speed.

[0006] To achieve the above objectives, this application adopts the following technical solution:

[0007] Firstly, a data migration method is provided, applied to a chip connected to a first storage medium and a second storage medium. The first and second storage media have different input / output (I / O) capabilities. Every N address segments in the first storage medium correspond to every N address segments in the second storage medium, where N is greater than or equal to 1. After migrating data from the location indicated by the first address segment in the first storage medium to the location indicated by the second address segment in the second storage medium, the M-bit migration flag of the first address segment is updated. The second address segment is one of the N address segments to which the first address segment belongs, and the M-bit migration flag records the migration status of the content stored in the N address segments between the first and second storage media.

[0008] The solution provided in this application maps N address segments in two storage media to each other, ensuring that data migration only occurs within the corresponding address segments. Simultaneously, a migration identifier records the migration status of the stored content across different storage media, replacing the approach of directly recording physical addresses. Since data migration occurs within N address segments, the M-bit migration identifier only needs to indicate at most N relative positions, provided that the configured N is less than the bit width of the physical address. Therefore, the storage space occupied by the information indicating the data storage location (i.e., the migration identifier) ​​after data migration is significantly less than that of the current approach of recording physical addresses, thus reducing the storage space required for storing the migration identifier. Furthermore, because the size of the migration identifier is smaller, it can be read faster when accessing memory, thereby improving memory access speed.

[0009] One possible implementation is that the value of M can be... Round up, or M is N. An M-bit migration flag only needs to indicate at most N relative positions, where M is... Rounding up ensures that M is as small as possible while indicating N relative positions, thus minimizing the number of bits in the migration flag and consequently reducing the storage space occupied by the migration flag. When N is 1, the migration flag requires only one bit. In cases where multiple address segments are mapped, adjusting the value of N controls the size of the migration flag's bit width M. As long as M is smaller than the physical address's bit width, the occupied storage space can be reduced. By choosing the value of N, the storage space occupied by the migration flag can be directly controlled. The smaller the value of N, the smaller the storage space occupied by the migration flag, and the less impact on the memory access speed of the storage medium.

[0010] Another possible implementation is that the third address segment in the first storage medium corresponds to the fourth address segment in the second storage medium. When the migration flag of the third address segment is a first value, the first value indicates that the initial storage content of the first address segment has been migrated to the fourth address segment. When the migration flag of the third address segment is a second value, the second value indicates that the initial storage content of the first address segment has not been migrated. The migration flag is only used to indicate whether the migration status has been changed, and there is no need to record the address after the migration. Therefore, it can be implemented with simple characters or very few bits, which greatly reduces the storage space occupied by storing the migration flag.

[0011] Another possible implementation, the data migration method provided in this application, further includes: receiving a data processing request, the data processing request including a destination address, the destination address being located in the fifth address segment; and executing the data processing request on the data in the storage location indicated by the migration identifier in the fifth address segment. By using the migration identifier, the data in the storage location corresponding to the destination address to be processed is determined, thereby processing the data. Because the migration identifier provided in this application has a small bit width, the access speed of the data processing process can be effectively improved, resulting in higher data processing efficiency.

[0012] Another possible implementation is to store the migration identifier in the error correction code (ECC). When reading the migration identifier, it can be directly read from the ECC of the fifth address segment. By storing the migration identifier in the ECC, since the ECC and the data are in the same location, the data can be read directly after reading the migration identifier without additional lookup or access operations. This simplifies the reading process and improves efficiency.

[0013] Another possible implementation is to store the migration flag in a free space within the ECC. By reusing the free space in the ECC, it maintains compatibility with the current data structure, does not increase the need for additional storage space, and avoids the space occupation caused by reallocating storage space to store the migration flag.

[0014] Another possible implementation involves configuring a level 3 cache (L3 cache) in the chip to store migration identifiers. When reading migration identifiers, the identifier in the fifth address segment can be directly retrieved from the L3 cache. Since the migration identifier occupies relatively little storage space, a dedicated L3 cache can be configured to store it. The L3 cache offers faster read speeds, avoiding the slow read speeds that occur when storing migration identifiers in memory.

[0015] Another possible implementation involves, while migrating the data from the location indicated by the first address segment in the first storage medium to the location indicated by the second address segment in the second storage medium, also migrating the data from the location indicated by the second address segment in the second storage medium to the location indicated by the first address segment in the first storage medium. This achieves data exchange between different storage media, and the data exchange status can be identified using only a single migration identifier, avoiding the problem of large storage space occupation caused by recording two physical addresses.

[0016] Another possible implementation, the data migration method provided in this application, further includes: if the migration identifier for the fifth address segment does not exist in the L3 cache, reading the migration identifier for the fifth address segment from the third storage medium; wherein, the third storage medium is a storage medium in a layer of the computing device. By configuring an L3 cache specifically for storing migration identifiers, the L3 cache read speed is faster. If there is no migration identifier in the L3 cache, it is read directly from the third storage medium, ensuring read speed while avoiding the situation where the migration identifier cannot be read.

[0017] Another possible implementation is that data processing requests include: data write requests or data read requests.

[0018] Secondly, a data migration apparatus is provided, comprising: a migration module and an update module. Wherein:

[0019] The migration module is used to migrate data in the first address segment of the first storage medium to the location indicated by the second address segment in the second storage medium.

[0020] The update module is used to update the M-bit migration flag of the first address segment.

[0021] The second address segment is one of the N address segments to which the first address segment belongs, and the M-bit migration flag is used to record the migration status of the contents stored in the N address segments between the first and second storage media.

[0022] One possible implementation is that M can take the value log2. N+1 Round up, or M equals N.

[0023] Another possible implementation is that the third address segment in the first storage medium corresponds to the fourth address segment in the second storage medium. When the migration flag of the third address segment is a first value, the first value indicates that the initial storage content of the first address segment has been migrated to the fourth address segment. When the migration flag of the third address segment is a second value, the second value indicates that the initial storage content of the first address segment has not been migrated.

[0024] In another possible implementation, the data migration apparatus provided in this application further includes a data processing module. The data processing module is used to receive a data processing request, which includes a destination address located in the fifth address segment. Based on a migration identifier in the fifth address segment, the data processing request is executed on the data in the storage location indicated by the migration identifier in the fifth address segment.

[0025] Another possible implementation is that the migration identifier can be stored in the ECC. The aforementioned data processing module is also used to directly read the migration identifier of the fifth address segment from the ECC of the fifth address segment when reading the migration identifier.

[0026] Another possible implementation is to store the migration flag in a free space in the ECC.

[0027] Another possible implementation is that the migration identifier can also be stored in the L3 cache. The aforementioned data processing module is also used to directly read the migration identifier of the fifth address segment from the L3 cache when reading the migration identifier.

[0028] In another possible implementation, the migration module is further configured to migrate, while migrating the data in the location indicated by the first address segment in the first storage medium to the location indicated by the second address segment in the second storage medium, simultaneously migrate the data in the location indicated by the second address segment in the second storage medium to the location indicated by the first address segment in the first storage medium.

[0029] The data migration apparatus provided in the second aspect is used to execute the data migration method provided in the first aspect or any possible implementation of the first aspect. The technical effects corresponding to any implementation of the second aspect can be referred to the technical effects corresponding to any implementation of the first aspect, and will not be repeated here.

[0030] Thirdly, a computing device is provided, comprising: a processor and a memory, wherein the memory stores at least one computer program, the at least one computer program being loaded and executed by the processor to implement the data migration method as described in the first aspect or any possible implementation thereof.

[0031] Fourthly, a computer-readable storage medium is provided, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor to implement the data migration method as described in the first aspect or any implementation thereof.

[0032] Fifthly, a computer program product is provided, comprising a computer program or instructions, which, when executed by a processor, implement the data migration method described in the first aspect or any implementation thereof.

[0033] In a sixth aspect, a chip is provided, comprising at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions and send instructions to the at least one processor, wherein when the at least one processor executes the instructions, the at least one processor executes to implement the data migration method as described in the first aspect or any implementation thereof.

[0034] In a seventh aspect, a computer system is provided, the computer system including a chip, a first storage medium and a second storage medium connected to the chip; when the chip executes instructions, it executes a data migration method as described in the first aspect or any implementation thereof based on the first storage medium and the second storage medium.

[0035] The solutions provided in aspects three through seven above are used to implement the methods provided in aspect one above, and their specific implementations will not be described in detail here. The technical effects corresponding to any implementation of the solutions provided in aspects three through seven above can be found in the technical effects corresponding to any implementation of aspect one above, and will not be described in detail here.

[0036] It should be noted that any of the possible implementations of any of the above aspects can be combined, provided that the solutions do not contradict each other. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of a data migration scenario provided by an exemplary embodiment;

[0038] Figure 2 This is a schematic diagram of the chip architecture provided in an exemplary embodiment of this application;

[0039] Figure 3 This is a deployment diagram of a data migration method provided in an exemplary embodiment of this application;

[0040] Figure 4 This is a schematic diagram of a data acquisition unit at different locations provided in an exemplary embodiment of this application;

[0041] Figure 5 This is a flowchart of a data migration method provided in an exemplary embodiment of this application;

[0042] Figure 6 This is a schematic diagram of a data migration scenario provided by an exemplary embodiment of this application;

[0043] Figure 7 This is a flowchart of another data migration method provided in an exemplary embodiment of this application;

[0044] Figure 8 This is a schematic diagram illustrating data processing of data in an address segment provided by an exemplary embodiment of this application;

[0045] Figure 9 This is a schematic diagram illustrating another method of data processing in an address segment, provided by an exemplary embodiment of this application.

[0046] Figure 10 This is a schematic diagram of a data migration method architecture provided in an exemplary embodiment of this application;

[0047] Figure 11 This is another schematic diagram of the architecture of the data migration method provided in an exemplary embodiment of this application;

[0048] Figure 12 This is a schematic diagram of an ECC bit read / write migration identifier provided in an exemplary embodiment of this application;

[0049] Figure 13 This is another architectural schematic diagram of a data migration method provided in an exemplary embodiment of this application;

[0050] Figure 14 This is a flowchart illustrating an exemplary embodiment of the present application of a method for executing a data write request based on ECC functionality;

[0051] Figure 15 This is a flowchart illustrating an exemplary embodiment of the present application of a method for executing a data read request based on ECC functionality;

[0052] Figure 16 This is a flowchart illustrating the execution of a data processing request based on a three-level cache unit storage migration identifier function, provided in an exemplary embodiment of this application.

[0053] Figure 17 This is a flowchart illustrating another method for executing a data processing request based on the storage migration identifier function of a three-level cache unit, provided in an exemplary embodiment of this application;

[0054] Figure 18 This is a schematic diagram of the structure of a data migration apparatus provided in an exemplary embodiment of this application;

[0055] Figure 19 This is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application. Detailed Implementation

[0056] In the embodiments of this application, in order to clearly describe the technical solutions of the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different. The technical features described by "first" and "second" have no sequential or size order.

[0057] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner to facilitate understanding.

[0058] In the embodiments of this application, at least one can also be described as one or more, and multiple can be two, three, four or more, and this application does not impose any restrictions.

[0059] Furthermore, the network architecture and scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0060] To facilitate understanding, the following explanations are provided for several terms used in this application.

[0061] Cold data: Data that is not frequently accessed, such as backup and archive data, and business and operational log data. Cold data has lower storage performance requirements and generally requires large-capacity storage media.

[0062] Hot data: Data that is accessed frequently, such as data that users frequently query. Hot data places high demands on storage performance.

[0063] Data migration refers to the process of moving data from one storage medium to another. For example, by swapping cold data in fast storage medium (dynamic random-access memory, DRAM) (hereinafter referred to as fast media or L1) with hot data in slow storage medium (storage class memory, SCM) (hereinafter referred to as slow media or L2), so that as much hot data as possible is stored in L1, this swapping process can be called data migration.

[0064] It should be noted that all information (including but not limited to device information, personal information of the target), data (including but not limited to data used for analysis, stored data, and displayed data), and signals involved in this application have been authorized by the target or by all parties in full, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the address ranges and data processing requests involved in this application were obtained with full authorization.

[0065] With the rapid development of big data and cloud computing technologies, the demand for data storage is growing exponentially. Storage media, as a crucial component for data preservation and processing, is undeniably important. Take memory as an example: during software operation, not all programs frequently access memory. In some business scenarios, truly "hot" data may only account for 10%-20% or even as low as 5%-10% of total memory. That is, most memory may simply occupy available space but not be accessed. Secondly, servers are initially configured for maximum capacity business scenarios, but they only run at peak times for a small portion of the time. In some business tests, up to 40% of idle memory may remain unused. These factors lead to excessively low actual memory utilization, resulting in resource waste. Furthermore, memory costs account for up to 40% of the overall server cost.

[0066] Based on this, the industry has proposed the concept of tiered memory storage. Data frequently accessed by the CPU is placed on storage media closer to the CPU. This type of media is characterized by high cost but high speed, meeting the application's bandwidth and latency requirements. Data less frequently accessed by the CPU is placed on media farther from the CPU. This type of media is low-cost, and even with relatively poor bandwidth and latency parameters, it can still meet the application's space requirements. In the CPU, the Core uses virtual addresses, which are translated into physical addresses by the Memory Management Unit (MMU). The translation from virtual to physical addresses requires consulting page tables, with common page table sizes being 4K / 64K / 2M. For the operating system, each program that needs to use memory must request pages from the operating system; that is, page access is related to the actual software's memory access. Therefore, memory page tables can be used as units to identify and migrate data between hot and cold pages. How to quickly label all pages as hot or cold and migrate them is a crucial function of tiered memory storage, and this function determines the performance of tiered storage.

[0067] Traditional hot / cold data identification and data migration are implemented at the software level. The software migration approach involves determining cold data on the fast medium (hereinafter referred to as L1) and hot data on the slow medium (hereinafter referred to as L2), then swapping L2 hot data with L1 cold data to maximize the storage of hot data in L1. This swapping process is called data migration. In the software-level data migration process, the migration between cold and hot data is achieved by changing the mapping between virtual addresses (VA) and physical addresses (PA).

[0068] Figure 1 This is a schematic diagram of a data migration scenario. Figure 1 (a) illustrates the mapping relationship between virtual and physical addresses at the software level before data migration. For example... Figure 1 As shown in (a), virtual address VA0 corresponds to physical address PA0. Physical address PA0 points to the slow media SCM space where data D0 is stored. Virtual address VA101 corresponds to physical address PA101, and physical address PA101 points to the fast media DRAM space where data SD1 is stored. When the address segment containing data D0 is frequently accessed and is identified as hot data, a relatively cold data SD1 is found in the fast media DRAM. Data D0 in the slow media SCM and data SD1 in the fast media DRAM are swapped, and the data is migrated. After this data migration, because the location of the stored data has changed, the mapping relationship between the virtual address and the physical address needs to be changed. Figure 1 (b) illustrates the mapping relationship between virtual and physical addresses after data migration at the software level. For example... Figure 1 As shown in (b), virtual address VA0 corresponds to physical address PA101. Physical address PA101 points to the fast DRAM space, which stores data D0. Virtual address VA101 corresponds to physical address PA0, which points to the slow SCM space, which stores data SD1. This process shows that virtual address VA0 still corresponds to data D0, and virtual address VA101 still corresponds to data SD1, but the corresponding storage medium has changed to fast medium.

[0069] Currently, software-based cold and hot data migration achieves 90%-95% performance. However, due to software overhead and other factors, a 5% gap remains. This 5% performance gap can be improved by deploying migration at the hardware level. Implementing migration at the hardware level requires additional maintenance of mapping relationships and recording the physical address (metadata address, MA) of the data after migration to ensure accurate data access later.

[0070] Figure 1(c) illustrates the mapping relationship between the metadata management address and the physical address before data migration at the hardware level. For example... Figure 1 As shown in (c), data D0 is stored in the slow medium SCM space pointed to by physical address PA0, and data SD1 is stored in the fast medium DRAM space pointed to by physical address PA101. In the metadata, the metadata management address corresponding to physical address PA0 is MA0 (essentially physical address 0), indicating that data D0 pointed to by PA0 is stored in the medium at address 0. The metadata management address corresponding to physical address PA101 is MA101, indicating that data SD1 pointed to by PA101 is stored in the medium at address 101. After swapping D0 and SD1 in the slow medium SCM, since the data location has changed, the physical address of the migrated data after migration needs to be recorded in the metadata. Figure 1 (d) in the diagram illustrates the mapping relationship between the metadata management address and the physical address after data migration at the hardware level. For example... Figure 1 As shown in (d) in the metadata, the metadata management address corresponding to physical address PA0 is MA101, indicating that the data D0 pointed to by PA0 is stored in the medium at address 101. The metadata management address corresponding to physical address PA101 is MA0, indicating that the data SD1 pointed to by PA101 is stored in the medium at address 0. This process shows that physical address PA0 still corresponds to data D0, and physical address PA101 still corresponds to data SD1, but the storage medium has changed to a fast medium. The target data can still be accurately accessed through the physical address.

[0071] However, in the above method, after migration, the migrated address needs to be recorded in the metadata. When there is a lot of migration data, a large number of addresses need to be recorded in the metadata, which results in the storage of metadata occupying a large amount of storage space and affecting memory access speed.

[0072] The storage space occupied by metadata can be calculated as: Metadata memory consumption = ENTRY_Num * Addr_Width. Here, ENTRY_Num represents the number of physical addresses to be managed, and Addr_Width represents the width of the physical address. Therefore, to reduce the memory consumption of metadata, we can either reduce the number of addresses to be managed (ENTRY_Num) or reduce the width of the physical address (Addr_Width). However, reducing ENTRY_Num means reducing the overall memory size, which is not desirable; therefore, we can only reduce Addr_Width. Taking a 32-bit physical address width and 128GB of memory as an example, there are approximately 32 million 4K pages, each requiring a 20-bit physical address after migration, requiring approximately 80MB of storage space to store metadata; there are approximately 2.1 billion 64B pages, each requiring a 26-bit physical address after migration, requiring approximately 6GB of storage space to store metadata. This demonstrates that metadata management occupies a very large amount of storage space. On the one hand, metadata requires a large amount of memory space to store it. On the other hand, every time memory is accessed, the metadata needs to be read again, which not only consumes a lot of memory access bandwidth, but also seriously affects the speed of normal memory access.

[0073] Based on this, this application provides a data migration method. By mapping N address segments in a first and second storage medium with different I / O performance to each other, after migrating data from the location indicated by the first address segment in the first storage medium to the location indicated by the second address segment in the second storage medium, the M-bit migration flag of the first address segment is updated. This M-bit migration flag records the migration status of the content stored in the N address segments in the first and second storage media. By mapping the N address segments in the two storage media, data migration only occurs within the corresponding address segments. Simultaneously, recording the migration status of the stored content in different storage media using the M-bit migration flag replaces the scheme of directly recording the physical address. The size of the migration flag is much smaller than the size of the currently recorded physical address, thereby reducing the storage space required to store the migration flag. Furthermore, because the data volume of the migration flag is smaller, it can be read faster when accessing memory, thus improving memory access speed.

[0074] The solutions provided by the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0075] Figure 2 This is a schematic diagram of a chip architecture provided in an embodiment of this application. The data migration method provided in this embodiment can be applied to this chip. See also... Figure 2The chip 100 includes a processor 110 and at least one interface circuit 130, and is connected to multiple storage media 120. The data migration method provided in this application embodiment can be applied to the chip 100.

[0076] The plurality of storage media 120 includes a first storage medium for storing hot data, such as a solid-state drive (SSD) or compute express link (CXL) memory capable of providing high input / output operations per second (IOPS) and low latency; the plurality of storage media 120 also includes a second storage medium for storing cold data, such as object storage capable of providing large-capacity, low-cost storage services. In some embodiments, the first storage medium is also referred to as high-performance storage or hot storage, and the second storage medium is also referred to as cold storage.

[0077] It is understandable that the above description of the first and second storage media as different types of storage media is not a specific limitation on the first and second storage media. That is, the first and second storage media can be the same type of storage media or different types of storage media. For example, both the first and second storage media can be SSDs or CXL memory, but the IO performance of the first storage media is relatively higher than that of the second storage media.

[0078] In some embodiments, the data in the plurality of storage media 120 can be stored in the form of a dataset. The data structure of the dataset is determined based on the data storage architecture of the storage space. For example, if the storage space is a relational database, the dataset stored in the storage space can be a relational data table; or, if the storage space is an object storage, the dataset stored in the storage space can be objects. This application embodiment does not limit the data structure of the dataset. In other embodiments, the granularity of the dataset can be determined according to business requirements. For example, the dataset can be a data table or a table partition. This application embodiment does not limit the granularity of the dataset.

[0079] Optionally, the storage medium 120 may be a hard disk drive (HDD), SSD, random access memory (RAM), CXL memory, controller, central processing unit (CPU), neural processing unit (NPU), but is not limited thereto, and the embodiments of this application do not specifically limit it. CXL memory is memory expanded by cxl expander or other volatile memory.

[0080] At least one interface circuit 130 is used to perform transceiver functions, and the interface circuit 130 is responsible for receiving signals or data from external devices or networks. These signals can be electrical signals, optical signals, or other forms of signals. The interface circuit 130 is also responsible for sending data processed internally by the chip to the processor 110, external devices, or networks. The processor 110 is used to manage the datasets stored in the plurality of storage media 120, thereby providing data migration services. In some embodiments, the processor 110 can obtain the access status of data in the plurality of storage media 120 in real time, thereby performing data migration for data in the plurality of storage media 120. In some embodiments, the processor 110 can read and write data in the plurality of storage media 120 in response to read and write operations on data in any of the storage media 120. This application embodiment does not limit the additional functions that the processor 110 can implement.

[0081] In some embodiments, the data migration method provided in this application can be implemented in the chip 100 as a computing instance. This computing instance can run in the processor 110 as a virtual machine, container, or process; that is, the data migration method provided in this application can be configured as a logical functional module in at least one processor 110.

[0082] For ease of understanding, this application provides a deployment diagram of a data migration method. See also Figure 3 , Figure 3 Figure (a) is a schematic diagram of a data acquisition unit provided in an embodiment of this application. This data acquisition unit is used to collect and analyze statistical results, so as to... Figure 2 The processor 110 in the chip 100 determines whether the data in the multiple storage media 120 is cold data or hot data, and thus decides whether to migrate the data. The data acquisition unit includes an information acquisition subunit 301, a conversion subunit 302, and a storage subunit 303.

[0083] The information acquisition subunit 301 can determine the data block corresponding to the storage address (i.e., memory access information) based on the storage address of the accessed data, and then the conversion subunit 302 obtains the access count corresponding to the data block from the storage subunit 303, increments the access count by one, and stores the incremented access count in the storage subunit 303.

[0084] Optionally, the data acquisition unit further includes a compression subunit 304. Before sending the access statistics results to the processor 305, the compression subunit 304 can compress the access statistics results and then send the compressed access statistics results to the processor 305. The access statistics results can be access statistics results for one or more data blocks. The address range of the data blocks can be used to distinguish the access statistics results for different data blocks.

[0085] For example, the storage sub-unit 303 may be random access memory (RAM).

[0086] In some embodiments, after obtaining access statistics, the process of processing cold data and / or hot data in the storage medium includes: moving hot data from the first storage medium to the second storage medium, or moving cold data from the second storage medium to the first storage medium, wherein the performance of the second storage medium is better than that of the first storage medium.

[0087] For example, please refer to Figure 3 Figure (b) in the middle, Figure 3 Figure (b) is a schematic diagram of processing cold and hot data according to this application. The storage media include: storage medium 1, storage medium 2, and storage medium 3. Storage medium 1 has better performance than storage medium 2, and storage medium 2 has better performance than storage medium 3. The processor can determine the cold data in storage medium 1, the cold and hot data in storage medium 2, and the hot data in storage medium 3. The processor can then migrate the cold data from storage medium 1 to storage medium 2, migrate the hot data from storage medium 2 to storage medium 1, migrate the cold data from storage medium 2 to storage medium 3, and migrate the hot data from storage medium 3 to storage medium 2.

[0088] Optionally, since storage medium 1 is the highest-performing medium, it is only necessary to identify the least accessed data within it and migrate this data to storage medium 2. Storage medium 2 should be a medium with moderate performance and price. In this case, the most frequently accessed data from storage medium 2 should be migrated to storage medium 1, and the least accessed data from storage medium 2 should be migrated to storage medium 3. Similarly, for storage medium 3, it is only necessary to identify the most frequently accessed data within it and migrate it to storage medium 2.

[0089] It is understood that this application can be applied to chip or computer systems connected to memory composed of hierarchical storage media. By accessing statistical results through a data acquisition unit, cold and hot data can be migrated, fully leveraging the characteristics of different storage media to achieve optimal performance and cost for the entire chip or computer system. This approach is suitable for high-performance, high-capacity storage system applications in future big data and in-memory computing systems. Deployment is possible in fields such as artificial intelligence, big data, and cloud computing, and it is applicable to various hierarchical storage media scenarios.

[0090] Those skilled in the art should understand that the processors and data acquisition units described above are merely examples. Other existing or future processors and data acquisition units that are applicable to the embodiments of this application should also be included within the scope of protection of the embodiments of this application, and are hereby incorporated by reference.

[0091] It should be noted that the application scenarios and implementation environments described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0092] The above embodiments have provided a detailed description of the structure and function of the data acquisition unit. The following will briefly describe the implementation location of the data acquisition unit within the chip, which can be... Figure 2 Chip 100 in the middle.

[0093] like Figure 4 The diagram shows data acquisition units at different locations. Figure 4 As shown in Figure (a), the data acquisition unit can be configured within the media module (i.e., the storage media module), meaning the processor and the data acquisition unit are independent of each other. In other words, the data acquisition unit's actions to collect access statistics results are all performed within the media module. After acquiring the access statistics results, the cached access statistics results are sent back via in-band or out-of-band methods. Since the access statistics results are not stored within the processor, the operation of sending back access statistics results can be optimized by compressing the results. After obtaining the compressed access statistics results, the application analyzes the results through the driver unit, identifies cold and hot data, and then performs data migration through the migration engine.

[0094] like Figure 4As shown in Figure (b), the data acquisition unit can be configured within the hardware module, meaning the processor and the data acquisition unit are independent of each other. In other words, the data acquisition unit's actions to collect access statistics results are all executed within the hardware module. After acquiring the access statistics results, the cached access statistics results are sent back via in-band or out-of-band methods. Since the access statistics results are not stored within the processor, the operation of sending back access statistics results can be optimized by compressing the results. After obtaining the compressed access statistics results, the application analyzes the results through the driver unit, identifies cold and hot data, and then performs data migration through the migration engine.

[0095] like Figure 4 As shown in Figure (c), the data acquisition unit can be configured within the processor; that is, the processor and the data acquisition unit are integrated together. In other words, the actions of the data acquisition unit in acquiring access statistics results are all executed within the processor. After acquiring the access statistics results, the cached access statistics results are sent back via in-band or out-of-band methods. Since the access statistics results are stored within the processor, there is no need for compression optimization when sending them back. After obtaining the access statistics results, the application analyzes the results through the driver unit, identifies cold and hot data, and then performs data migration through the migration engine.

[0096] As mentioned above Figure 3 or Figure 4 Once the data migration is determined in the content, the data migration method provided in this application can be executed to complete the migration process. Figure 5 This is a flowchart illustrating a data migration method provided in an embodiment of this application. The method is applied to a chip connected to a first storage medium and a second storage medium, the first and second storage media having different I / O performance. For example, the chip can be... Figure 2 Chip 100 in the middle.

[0097] Furthermore, every N address segments in the first storage medium correspond to every N address segments in the second storage medium. N is greater than or equal to 1. Data within the corresponding N address segments is migrated. It should be understood that data migration only occurs within the corresponding N address segments.

[0098] For example, when N is 1, the address segments in the first storage medium are mapped one-to-one with the address segments in the second storage medium; this is called a "one-to-one mapping" scenario. When N is 2, the two address segments in the first storage medium are mapped to the two address segments in the second storage medium; this is called a "two-to-two mapping" scenario. Similarly, when N is 3, the three address segments in the first storage medium are mapped to the three address segments in the second storage medium; this is called a "three-to-three mapping" scenario. Of course, the value of N can be configured according to actual needs.

[0099] Optionally, there may be one mapping relationship or multiple mapping relationships between the two storage media. For example, there may be both "one-to-one mapping" and "three-to-three mapping".

[0100] like Figure 5 As shown, the data migration method may include:

[0101] Step 502: The chip migrates the data in the first address segment of the first storage medium to the location indicated by the second address segment in the second storage medium.

[0102] In this context, an address segment refers to a contiguous range of addresses used for addressing and accessing data within a storage medium. For example, in a hard disk drive, an address segment might refer to a contiguous range of disk sectors. In memory management, an address segment might refer to the memory space of a process or a fixed-size memory region allocated by the operating system.

[0103] In some embodiments, the first storage medium and the second storage medium connected to the chip are storage media belonging to different levels in hierarchical storage management (HSM).

[0104] Specifically, when data at the location indicated by the first address segment in the first storage medium is determined to be unsuitable for storage in the first storage medium, it needs to be migrated out, and the process of step 502 is executed. For example, cold data determined in a fast medium, or hot data determined in a slow medium, can be understood as data at the location indicated by the first address segment in the first storage medium described in this application.

[0105] The second address segment is one of the N address segments to which the first address segment belongs, and one of the corresponding N address segments in the second storage medium.

[0106] In one possible implementation, N is 1, and the first storage medium and the second storage medium are in a "one-to-one mapping" scenario. In this case, the second address segment is the address segment corresponding to the first address segment.

[0107] In another possible implementation, N is not 1, and the first storage medium and the second storage medium are in a scenario other than a "one-to-one mapping" scenario. Before executing step 502, it is necessary to determine the second address segment from the second storage medium.

[0108] For example, first determine the N address segments (address segment set A) to which the first address segment belongs, and the corresponding N address segments (address segment set B) in the second storage medium. Then, select an address segment from address segment set B that has not undergone data migration and meets the conditions as the second address segment.

[0109] The above conditions are used to indicate the characteristics of the data removed from the second storage medium, or to indicate that data migration to other storage media is supported. The content of these conditions can be configured according to actual needs. Several implementations of these conditions are illustrated below, but do not constitute specific limitations.

[0110] For example, the above condition could be that the space indicated by the address segment is empty, that is, the space indicated by the address segment has not yet stored data.

[0111] For example, if the I / O performance of the first storage medium is higher than that of the second storage medium, the above condition can be used to indicate the highest data usage. If the I / O performance of the first storage medium is lower than that of the second storage medium, the above condition can be used to indicate the lowest data usage.

[0112] For example, if the I / O performance of the first storage medium is higher than that of the second storage medium, the above condition can be that the stored data has the highest popularity and is more popular than the data in the first address range. If the I / O performance of the first storage medium is lower than that of the second storage medium, the above condition can be that the stored data has the lowest popularity and is less popular than the data in the first address range.

[0113] In another possible implementation, N is not 1. Before executing step 502, the second address segment needs to be determined from the second storage medium. It's understood that the second address segment determined in the first round of data migration and the second round of data migration may or may not be the same address segment. For example, in a "two-to-two mapping" scenario between the first and second storage media, during the first round of data migration, the stored content in the first address segment of the first storage medium can be migrated to either the first or second address segment of the second storage medium. For instance, the stored content in the first address segment of the first storage medium can be migrated to the first address segment of the second storage medium. Before the second round of data migration, the data migrated in the first round needs to be reset, that is, the migrated storage data needs to be restored to its initial address segment. Then, the process of determining the second address segment in the second storage medium is performed, which is the same as the process of determining the second address segment in the first round of data migration, and will not be elaborated here.

[0114] In some embodiments, the migration of data from a first address segment in a first storage medium to a second address segment in a second storage medium may include at least one of the following, but is not limited thereto, and the embodiments of this application do not specifically limit the following:

[0115] Scenario 1: The location indicated by the second address segment is not empty, and the data in the first and second address segments are swapped.

[0116] For example, while performing step 502, the chip also migrates the data at the location indicated by the second address segment in the second storage medium to the location indicated by the first address segment in the first storage medium. For instance, if the first address segment stores first data and the second address segment stores second data, the first data is migrated to the second address segment, and simultaneously, the second data is migrated to the first address segment.

[0117] Scenario 2: The location indicated by the second address segment is empty. The data in the first address segment is migrated to the second address segment. That is, the first address segment stores the first data, and the second address segment is empty. Therefore, the first data can be directly migrated to the second address segment.

[0118] Step 504: The chip updates the M-bit migration flag of the first address segment.

[0119] The M-bit migration flag is used to record the migration status of the contents stored in the N address segments between the first and second storage media. The contents stored in the address segments should be understood as the initial stored contents, that is, the initial data pointed to by the physical address.

[0120] For example, migration status describes whether data stored in an address range has been migrated. Alternatively, migration status describes whether data in an address range has been migrated and where it has been migrated.

[0121] In some embodiments, the description of the migration identifier is associated with a mapping scenario between address segments in the first storage medium and address segments in the second storage medium, as shown in Table 1.

[0122] Table 1

[0123] Mapping Scene Migration status One-to-one mapping Migration NN mapping (N is not 1) Migration required + Migration location

[0124] As shown in Table 1, when the address segments in the first storage medium and the second storage medium are in a one-to-one mapping, since the mapping relationship between the address segments is unique, the migration flag only needs to indicate whether a migration has occurred. When the address segments in the first storage medium and the second storage medium are in an "NN mapping" and N is not 1, since the mapping relationship between the address segments can be selected, the migration flag needs to indicate not only whether a migration has occurred, but also the specific migration location.

[0125] Furthermore, M is smaller than the bit width of the physical address of the storage medium. Migration identifiers differ from physical addresses; for example, migration identifiers can be a small number of numbers, a small number of letters, or other characters used for identification.

[0126] Specifically, the M-bit migration flag needs to indicate N+1 possible migrations (including no migration). Therefore, the number of bits in M ​​is determined by N, and a reasonable value for N needs to be configured so that M is less than the bit width of the physical address of the storage medium. The M-bit migration flag can be M bits.

[0127] In one possible implementation, the M-bit migration flag is used to indicate N+1 possible migrations, and M is configured as... Round up to represent N+1 cases. However, when N is 1, M can only be 1.

[0128] For example, the relationship between the number of bits in M ​​and the mapping scenario can be shown in Table 2.

[0129] Table 2

[0130]

[0131] As shown in Table 2, when the address segments in the first and second storage media are in a one-to-one mapping, only a 1-bit migration identifier is needed because the mapping relationship between the address segments is unique. When the address segments in the first and second storage media are in a two-to-two mapping, since the mapping relationship between the address segments can have two choices, a 2-bit migration identifier can be used to identify the specific migration location. For example, 00 indicates no migration, 10 indicates that the data has been migrated to the first address segment of the two corresponding address segments, and 01 indicates that the data has been migrated to the second address segment of the two address segments.

[0132] In one possible implementation, M is equal to N, and the i-th bit of the migration identifier indicates whether the data has been migrated to the i-th address among the N address segments. For example, in a 3x3 mapping scenario, the migration identifier is 3 bits, where a value of 0 for the i-th bit indicates that the data has not been migrated to the i-th address segment, and a value of 1 for the i-th bit indicates that the data has not been migrated to the i-th address segment. For example, 100 could indicate that the data has been migrated to the first address segment among the N address segments in the second storage medium, 010 could indicate that the data has been migrated to the second address segment among the N address segments in the second storage medium, and 001 could indicate that the data has been migrated to the third address segment among the N address segments in the second storage medium.

[0133] For example, the third address segment in the first storage medium corresponds to the fourth address segment in the second storage medium. When the migration flag of the third address segment is a first value, the first value indicates that the initial storage content of the first address segment has been migrated to the fourth address segment; when the migration flag of the third address segment is a second value, the second value indicates that the initial storage content of the first address segment has not been migrated.

[0134] Furthermore, the two address ranges that are exchanging data share a single migration identifier.

[0135] For example, Figure 6 This diagram illustrates a data migration scenario. Figure 6 As shown in Figure (a), Figure 6 In Figure (a), the address segments of the first and second storage media have a one-to-one mapping relationship. The address segment number in the first storage media is the first address segment number, and the address segment number in the second storage media is the second address segment number. That is, address segment a, where data D0 in the first storage media is located, corresponds to address segment A, where data SD0 in the second storage media is located. Address segment b, where data D1 in the first storage media is located, corresponds to address segment B, where data SD1 in the second storage media is located. Other address segments are similar and will not be listed here. When data D0 in the first storage media is exchanged with data SD0 in the second storage media, the migration flags of address segments a and A are updated to 1, indicating that a migration has occurred, while the migration flags of address segments that have not been migrated are 0.

[0136] like Figure 6 As shown in Figure (b), Figure 6In Figure (b), the address segments of the first and second storage media have a two-to-two mapping relationship. That is, the address segment where data D0 and D1 in the first storage medium are located corresponds to the address segment where data SD0 and SD1 in the second storage medium are located. Before data migration, data D0 in the first storage medium can choose its migration location. Data D0 can be migrated to address segment A where SD0 is located, or it can be migrated to address segment B where SD1 is located. For example, if data D0 is migrated to address segment B where data SD1 is located, the corresponding migration identifier is "10", and if data D1 is migrated to address segment A where data SD0 is located, the corresponding migration identifier is "01". When data D0 and data D1 in the first storage medium are exchanged with data SD0 and data SD1 in the second storage medium, address segment a and address segment A become 01, and address segment b and address segment B become 10.

[0137] In summary, the solution provided in this embodiment proposes a data migration method that maps N address segments in two storage media to each other, ensuring that data migration only occurs within the corresponding address segments. Simultaneously, a migration identifier records the migration status of the stored content across different storage media, replacing the method of directly recording the physical address. The size of the migration identifier is much smaller than the size of the currently recorded physical address, thereby reducing the storage space required to store the migration identifier. Furthermore, because the data volume of the migration identifier is smaller, it can be read faster when accessing memory, thus improving memory access speed.

[0138] Furthermore, after the data is migrated and the updated M-bit migration identifier is obtained, the migration identifier can be stored in different locations.

[0139] In some embodiments, the storage location of the migration identifier may include at least one of the following locations, but is not limited thereto, and the embodiments of this application do not specifically limit this.

[0140] Location 1: Store the migration identifier in the ECC corresponding to the data in each address segment. Optionally, when storing the migration identifier in the ECC, it is stored in a free space within the ECC. For example, when storing the migration identifier in the ECC corresponding to the data in each address segment of the first storage medium, it can be stored in the ECC corresponding to the data in each address segment of the second storage medium. For instance, if both the first and second storage media are high-speed media, the migration identifier can be preferentially stored in the ECC corresponding to the data in the address segment of the first storage medium. When reading the migration identifier, it can be directly read from the ECC of the first storage medium, which has a faster read speed.

[0141] Location 2: Configure a separate L3 cache unit within the CPU to store migration identifiers. When reading migration identifiers, only the L3 cache needs to be accessed. Optionally, the L3 cache can store a portion or all of the migration identifiers.

[0142] When all migration identifiers are stored in the L3 cache, reading migration identifiers only requires reading from the L3 cache. When only some migration identifiers are stored in the L3 cache, all migration identifiers are stored in either the first or second storage medium. When reading migration identifiers, the L3 cache is read first. If the migration identifier for the address range does not exist in the L3 cache, it can be read from either the first or second storage medium.

[0143] For example, when reading the migration identifier of the fifth address segment, it can first be read from the L3 cache. If the migration identifier of the fifth address segment does not exist in the L3 cache, it can then be read from the third storage medium, which is a storage medium in one level of the computing device. Optionally, the third storage medium can be the same as the first and second storage media, or it can be a different storage medium.

[0144] Location 3: Store the migration identifier in the first storage medium or the second storage medium. When reading the migration identifier, it can be read only from the first storage medium or the second storage medium.

[0145] Furthermore, after the data migration, when the chip receives a data request for data stored in the address range, it can locate the data's new storage location based on the updated migration identifier and perform data processing on the data according to the data request. For example... Figure 7 As shown, the data migration method provided in this application embodiment further includes:

[0146] Step 506: The chip receives a data processing request; for the data in the storage location indicated by the migration identifier of the fifth address segment, the data processing request is executed.

[0147] For example, a data processing request includes a data write request or a data read request. A data write request refers to modifying data stored in an address segment, or writing data into an empty address segment. A data read request refers to reading data stored in an address segment.

[0148] Specifically, for example, if the data processing request is to read the data stored in the fifth address segment of the first storage medium, then the steps for executing the data processing request are as follows: determine the location of the fifth address segment of the first storage medium and read the migration flag corresponding to the fifth address segment. If the migration flag corresponding to the fifth address segment indicates "not migrated", directly read the data stored in the fifth address segment; if the migration flag corresponding to the fifth address segment indicates "migrated", directly read the data stored in the address segment corresponding to the fifth address segment in the second storage medium.

[0149] like Figure 8 The diagram shown illustrates the data processing of data within an address segment, as follows: Figure 8 As shown in Figure (a), the first storage medium includes 100 address segments, and the second storage medium includes 100 address segments. The address segments in the first storage medium are mapped one-to-one with those in the second storage medium; that is, data stored in the address segment corresponding to physical address PA0 can only be migrated to the address segment corresponding to physical address PA100, and data stored in the address segment corresponding to physical address PA1 can only be migrated to the address segment corresponding to physical address PA101. Initially, neither the first nor the second storage medium stores any data in its address segments; that is, the data stored in both address segments is 0.

[0150] During the data writing phase, such as Figure 8 As shown in Figure (b), the CPU initiates a data write request for the address segments corresponding to physical addresses PA0 and PA1 in the first storage medium. It reads that the migration flags for both physical addresses PA0 and PA1 are 0, meaning the data in these address segments has not been migrated. Therefore, it writes the data "5" to the address segment corresponding to physical address PA0 and the data "9" to the address segment corresponding to physical address PA1. Figure 8 As shown in Figure (c), the CPU initiates a data write request for the address segments corresponding to physical addresses PA100 and PA101 in the second storage medium. The migration flags of both physical address PA100 and PA101 are 0, meaning the data in these address segments has not been migrated. Therefore, data "6" is written to the address segment corresponding to physical address PA100, and data "8" is written to the address segment corresponding to physical address PA101. After swapping the data "6" stored in the address segment corresponding to physical address PA0 with the data "5" stored in the address segment corresponding to physical address PA100, the migration flag of the address segment corresponding to physical address PA0 is changed from 0 to 1.

[0151] During the data reading phase, such as Figure 8 As shown in Figure (d), the CPU initiates a data read request for the address segment corresponding to physical address PA0 in the first storage medium. When the migration flag of the address segment corresponding to physical address PA0 is 1, that is, the data in the address segment corresponding to physical address PA0 has been migrated, then according to the one-to-one mapping relationship, the CPU directly reads the data from the address segment corresponding to physical address PA0, which corresponds to physical address PA100, and obtains the data "5". That is, when reading data from the address segment corresponding to physical address PA0, the data read is still the data "5". The content of the data read is unchanged, but the storage medium to which the data "5" belongs has changed.

[0152] like Figure 9 The diagram illustrates data processing within an address range. It assumes that migration identifiers are stored in a L3 cache unit, and that only a portion of the migration identifiers are stored in the L3 cache unit. For example... Figure 9 As shown in Figure (a), the first storage medium includes 100 address segments, and the second storage medium includes 100 address segments. The address segments in the first storage medium are mapped one-to-one with the address segments in the second storage medium. Initially, neither the address segments in the first nor the second storage medium store any data, and the migration flags stored in the L3 cache units are all 0.

[0153] During the data writing phase, such as Figure 9 As shown in Figure (b), the CPU initiates a data write request for the address segment corresponding to physical address PA100 in the second storage medium. The migration flag for the address segment corresponding to physical address PA100 reads from the L3 cache is 0, meaning the data in the address segment corresponding to physical address PA100 has not been migrated. Therefore, the data "8" is written to the address segment corresponding to physical address PA100. Figure 9 As shown in Figure (c), the CPU initiates a data write request for the address segments corresponding to physical addresses PA0 and PA1 in the first storage medium. It reads from the L3 cache that the migration flags for both physical addresses PA0 and PA1 are 0, meaning the data in these address segments has not been migrated. Therefore, it writes data "5" to the address segment corresponding to physical address PA0 and data "9" to the address segment corresponding to physical address PA1. After swapping the data "6" stored in the address segment corresponding to physical address PA0 with the data "5" stored in the address segment corresponding to physical address PA100, the migration flag for the address segment corresponding to physical address PA0 is changed from 0 to 1.

[0154] During the data reading phase, such as Figure 9As shown in Figure (d), the CPU initiates a data read request for the address segment corresponding to physical address PA0 in the first storage medium. It reads the migration flag of the address segment corresponding to physical address PA0 from the L3 cache unit. If the migration flag of the address segment corresponding to physical address PA0 is 1, that is, the data in the address segment corresponding to physical address PA0 has been migrated, then according to the one-to-one mapping relationship, it directly reads the data from the address segment corresponding to physical address PA100, which is the address segment corresponding to physical address PA0, and obtains the data "5".

[0155] In summary, the solution provided in this embodiment proposes a data migration method that uses migration identifiers to determine the data in the storage location corresponding to the destination address to be processed, thereby processing the data. The location can be determined by a simple migration identifier, which is simpler, more convenient, and has higher data processing efficiency than querying the location through the physical address.

[0156] The above content introduces the solution provided in this application. The following section uses a Reduced Instruction Set Computer (RISC) platform as an example to describe the overall architecture of the data migration method.

[0157] In one possible implementation, such as Figure 10 The data migration method architecture diagram shown is as follows: Figure 10 As shown, every N address segments in the first storage medium 1006 correspond to every N address segments in the second storage medium 1007. A storage space is allocated in the first storage medium 1006 for storing migration identifiers. The migration module 1004 is used to migrate the data stored in the address segments of the first storage medium 1006 and the second storage medium 1007. The migration module 1004 also includes a migration identifier management unit, which is used to update or manage the migration identifiers stored in the first storage medium 1006. The processor 1001 is connected to the migration module 1004 via the memory bus 1002 and the peripheral bus 1003, respectively. The first storage medium 1006 and the second storage medium 1007 are connected to the migration module 1004 via the memory management component 1005, respectively. The processor 1001 can process the data corresponding to the address segments in the first storage medium 1006 and the second storage medium 1007 through the memory bus 1002, and the processor 1001 can configure or modify the parameters of the migration module 1004 and the migration identifier management unit through the peripheral bus 1003.

[0158] When processing data in the first address segment of the first storage medium 1006, the processor 1001 sends a data processing request to the migration module 1004 via the peripheral bus 1003. The migration identifier management unit in the migration module 1004 reads the migration identifier stored in the first storage medium 1006. After reading the migration identifier of the first address segment from the first storage medium 1006, it determines whether the data in the first address segment has been migrated based on the migration identifier. If the data in the first address segment has not been migrated, data processing is performed directly on the data in the first address segment of the first storage medium 1006; if the data in the first address segment has been migrated, data processing is performed on the data in the second address segment of the second storage medium 1007 corresponding to the first address segment.

[0159] In one possible implementation, such as Figure 11 Another architectural diagram of the data migration method shown is as follows: Figure 11 As shown, every N address segments in the first storage medium 1106 correspond to every N address segments in the second storage medium 1107, and migration identifiers are stored in the ECC of the address segments. The migration module 1104 is used to migrate the data stored in the address segments of the first storage medium 1106 and the second storage medium 1107. The migration module 1104 also includes a migration identifier management unit, which is used to update or manage the migration identifiers stored in the first storage medium 1106. The processor 1101 is connected to the migration module 1104 via the memory bus 1102 and the peripheral bus 1103, respectively. The first storage medium 1106 and the second storage medium 1107 are connected to the migration module 1104 via the memory management component 1105, respectively. The processor 1101 can process the data corresponding to the address segments in the first storage medium 1106 and the second storage medium 1107 through the memory bus 1102. The processor 1101 can configure or modify the parameters of the migration module 1104 and the migration identifier management unit through the peripheral bus 1103.

[0160] When processing data in the first address segment of the first storage medium 1106, the processor 1101 initiates a data processing request to the migration module 1104 via the peripheral bus 1103. The migration identifier management unit in the migration module 1104 reads the migration identifier stored in the ECC of the address segment. After reading the migration identifier of the first address segment from the ECC, it determines whether the data in the first address segment has been migrated based on the migration identifier. If the data in the first address segment has not been migrated, data processing is performed directly on the data in the first address segment of the first storage medium 1106; if the data in the first address segment has been migrated, data processing is performed on the data in the second address segment of the second storage medium 1107 corresponding to the first address segment.

[0161] For example, such as Figure 12 The diagram illustrates the read / write migration flag in the ECC bit. The migration flag is stored in the ECC of the data in the first storage medium. Address segments in the first storage medium are mapped one-to-one with address segments in the second storage medium. Initially, neither the first nor the second storage medium stores any data in its address segments; that is, the data stored in both address segments is 0, and correspondingly, the ECC of the data in the first storage medium is also 0.

[0162] like Figure 12 As shown in (a), during the data write phase, the CPU initiates a data write request for the address segments corresponding to physical addresses PA0 and PA2 in the first storage medium. From the ECC records of the data in these address segments, the CPU reads that the migration flags for both physical addresses PA0 and PA2 are 0, meaning the data in these address segments has not been migrated. Therefore, the CPU writes the data "8" to the address segment corresponding to physical address PA0 and the data "5" to the address segment corresponding to physical address PA2. The CPU performs ECC calculation on the data "8" in the address segment corresponding to physical address PA0, obtaining an ECC value of f0. Similarly, the CPU performs ECC calculation on the data "5" in the address segment corresponding to physical address PA2, obtaining an ECC value of f100.

[0163] like Figure 12 As shown in Figure (b), during the data write phase, the CPU initiates a data write request for the address segments corresponding to physical addresses PA10 and PA12 in the second storage medium. First, it reads the ECC of the data in the address segments corresponding to physical addresses PA0 and PA2 in the first storage medium and finds that the migration flags of the address segments corresponding to physical addresses PA0 and PA2 are both 0. Then, it writes the data "9" into the address segment corresponding to physical address PA10 and the data "6" into the address segment corresponding to physical address PA12. The ECC value for the data "9" in the address segment corresponding to physical address PA10 is calculated as s0. Similarly, the ECC value for the data "6" in the address segment corresponding to physical address PA12 is calculated as s100.

[0164] like Figure 12 As shown in Figure (c), after swapping the data "5" stored in the address segment corresponding to physical address PA2 with the data "6" stored in the address segment corresponding to physical address PA12, the migration flag of the address segment corresponding to physical address PA2 is changed from 0 to 1.

[0165] like Figure 12 As shown in Figure (d), the CPU initiates a data read request for the address segments corresponding to physical addresses PA10 and PA12 in the second storage medium. First, it reads the migration flags of the address segments corresponding to physical addresses PA0 and PA2 in the first storage medium from the ECC. It can be seen that the data in the address segment corresponding to physical address PA0 in the first storage medium has not been migrated, while the data in the address segment corresponding to physical address PA2 has been migrated. Therefore, the data in the address segment corresponding to physical address PA10 in the second storage medium is read directly, resulting in the data "9". However, when reading the data in the address segment corresponding to physical address PA12 in the second storage medium, since the data has been migrated, the data in the address segment corresponding to physical address PA2 in the first storage medium needs to be read, resulting in the data "6".

[0166] In one possible implementation, such as Figure 13 Another architectural diagram of the data migration method shown is as follows: Figure 13 As shown, every N address segments in the first storage medium 1307 correspond to every N address segments in the second storage medium 1308, and migration identifiers are stored by configuring a three-level cache unit 1305. The migration module 1304 is used to migrate the data stored in the address segments of the first storage medium 1307 and the second storage medium 1308. The migration module 1304 also includes a migration identifier management unit, which is used to update or manage the migration identifiers stored in the three-level cache unit 1305 and the first storage medium 1307. The processor 1301 is connected to the migration module 1304 via the memory bus 1302 and the peripheral bus 1303, respectively. The first storage medium 1307 and the second storage medium 1308 are connected to the migration module 1304 via the memory management component 1306, respectively. The processor 1301 can process the data corresponding to the address segments in the first storage medium 1307 and the second storage medium 1308 through the memory bus 1302. The processor 1301 can configure or modify the parameters of the migration module 1304 and the migration identifier management unit through the peripheral bus 1303.

[0167] When processing data in the first address segment of the first storage medium 1307, the processor 1301 sends a data processing request to the migration module 1304 via the peripheral bus 1303. The migration flag management unit in the migration module 1304 reads the migration flag stored in the L3 cache unit 1305. If the migration flag is not found in the L3 cache unit 1305, it reads the migration flag stored in the first storage medium 1307. After reading the migration flag of the first address segment from the L3 cache unit 1305 or the first storage medium 1307, it determines whether the data in the first address segment has been migrated based on the migration flag. If the data in the first address segment has not been migrated, data processing is performed directly on the data in the first address segment of the first storage medium 1307; if the data in the first address segment has been migrated, data processing is performed on the data in the second address segment corresponding to the first address segment in the second storage medium 1308.

[0168] like Figure 14 The flowchart shown illustrates a method for executing a data write request based on ECC functionality. This method includes:

[0169] Step 1401: The processor sends a data write request carrying the first address segment.

[0170] For example, the processor sends a write request to the memory manager for data in the first address segment. The memory manager is part of the storage media system and is responsible for handling data transfer between the processor and the storage media.

[0171] Step 1402: Calculate the ECC value based on the data to be written.

[0172] For example, after receiving a data write request from the processor, the memory manager calculates the ECC value corresponding to the first address segment based on the data to be written and the ECC algorithm.

[0173] Step 1403: Read the data from the first address segment.

[0174] For example, before performing a data write operation on the first address segment, the memory manager will first read the data currently stored in the first address segment.

[0175] Step 1404: Determine whether the current migration identifier needs to be updated based on the migration identifier in the ECC.

[0176] For example, after reading the data currently stored in the first address segment, it is determined whether the migration identifier corresponding to the first address segment needs to be updated based on the migration identifier in the read data and the currently stored data. If the storage location of the data stored in the first address segment has changed, the migration identifier is updated; if the storage location of the data stored in the first address segment has not changed, the migration identifier does not need to be updated.

[0177] Step 1405: Store the data to be written, the migration identifier, and the ECC value to the storage medium simultaneously.

[0178] For example, the memory manager writes the new data to be written, the migration identifier, and the ECC value corresponding to the data to be written as a whole to the first address segment in the storage medium at the same time.

[0179] Step 1406: Store the data to be written, migration identifier, and ECC value in a separate storage medium.

[0180] For example, after obtaining the data to be written, the migration identifier, and the ECC value corresponding to the data to be written, the data to be written, the migration identifier, and the ECC value corresponding to the data to be written are written to different locations. For example, the data to be written is written to the data bits in the first address segment, the ECC value is written to the ECC bits in the first address segment, and the migration identifier is written to the free bits in the ECC bits.

[0181] like Figure 15 The flowchart shown illustrates a method for executing a data read request based on ECC functionality. This method includes:

[0182] Step 1501: The processor reads the data and ECC from the first address segment in the first storage medium.

[0183] For example, when executing a data read request, the processor reads data from a first address segment and ECC data from the first storage medium.

[0184] Step 1502: Read the migration identifier from the ECC.

[0185] For example, since a migration identifier is stored in the ECC free space, the migration identifier corresponding to the first address segment is read from the ECC data after the ECC data is obtained.

[0186] Step 1503: Determine whether to migrate.

[0187] For example, based on the read migration identifier, it is determined whether the data stored in the first address segment has been migrated. If the data stored in the first address segment has been migrated, step 1504 is executed; if the data stored in the first address segment has not been migrated, step 1505 is executed.

[0188] Step 1504: Read data from the second address segment in the second storage medium.

[0189] For example, if the data stored in the first address segment has been migrated, the data is read from the second address segment in the second storage medium. The second address segment in the second storage medium corresponds to the first address segment in the first storage medium.

[0190] Step 1505: Read data from the first address segment in the first storage medium.

[0191] For example, if the data stored in the first address segment has not been migrated, the data is read directly from the first address segment in the first storage medium.

[0192] Step 1506: Calculate the ECC value based on the read data.

[0193] For example, the ECC algorithm is used to perform ECC calculation on the read data to obtain the new ECC value corresponding to the data.

[0194] Step 1507: Determine if they are consistent.

[0195] For example, determine whether the new ECC value calculated based on the read data is consistent with the original read ECC value. If the two ECC values ​​are consistent, proceed to step 1508. If the two ECC values ​​are inconsistent, proceed to step 1509.

[0196] Step 1508: Return the read data.

[0197] For example, if the new ECC value is consistent with the original ECC value, it indicates that no errors occurred during data transmission, and the data can be directly sent back to the processor to complete the data read request.

[0198] Step 1509: Perform ECC error correction.

[0199] For example, if the new ECC value is inconsistent with the original ECC value, this indicates that the data may have been corrupted in the storage medium. First, an ECC algorithm can be used to correct the error. If the error can be corrected, the corrected data will be re-encoded with an ECC value and sent back to the processor. If the error cannot be corrected, an error handling mechanism may be triggered, such as notifying the processor that a data error has occurred.

[0200] like Figure 16 The flowchart shown illustrates the execution of data processing requests based on the three-level cache unit storage migration identifier function. The method includes:

[0201] When the processor needs to execute a data processing request to the first address segment in the second storage medium 1603, it sends a data processing request to the migration identifier management unit 1601. Since the migration identifier is stored in the L3 cache unit 1605 and the first storage medium 1604, the migration identifier management unit 1601 first initiates a migration identifier read request to the L3 cache unit 1605 to obtain the migration identifier corresponding to the first address segment. If the L3 cache unit 1605 has a cache hit (i.e., the L3 cache unit 1605 contains the migration identifier), the migration identifier can be returned directly. If the L3 cache unit 1605 has a cache miss (i.e., the L3 cache unit 1605 does not contain the migration identifier), then a migration identifier read request is initiated to the first storage medium 1604 to read the migration identifier.

[0202] Upon receiving a read migration identifier, the migration identifier management unit 1601 sends a data processing request to the data migration management unit 1602 according to the location indicated by the migration identifier. The data migration management unit 1602 then executes the data processing request at the corresponding location indicated by the migration identifier.

[0203] like Figure 17 The flowchart shown illustrates the execution of data processing requests based on the three-level cache unit storage migration identifier function. The method includes:

[0204] Step 1701: Begin.

[0205] Step 1702: The processor initiates a data processing request for the first address segment.

[0206] For example, when the processor needs to perform data processing on a first address segment in the first storage medium, the processor sends a data processing request.

[0207] Step 1703: Query the migration identifier corresponding to the first address segment in the Level 3 cache unit.

[0208] For example, upon receiving a data processing request, since the migration identifier is stored in the L3 cache and the first storage medium, the processor first initiates a migration identifier read request to the L3 cache to obtain the migration identifier corresponding to the first address segment.

[0209] Step 1704: Check if the L3 cache unit is hit.

[0210] For example, if the L3 cache unit is hit, that is, the L3 cache unit includes a migration identifier, then step 1706 is executed; if the L3 cache unit is not hit, then step 1705 is executed.

[0211] Step 1705: Read the migration identifier from the first storage medium.

[0212] For example, if the Level 3 cache unit does not include a migration identifier, then a migration identifier read request is initiated to the first storage medium 1604, and the migration identifier is read.

[0213] Step 1706: Determine whether the data has been migrated.

[0214] For example, after reading the migration identifier, it is determined whether the data in the first address segment has been migrated based on the migration identifier.

[0215] Step 1707: Update the data processing address.

[0216] For example, if data in the first address segment is migrated based on the migration identifier, the data processing address is updated according to the location indicated by the migration identifier. That is, data processing is performed on the data at the location indicated by the migration identifier.

[0217] Step 1708: Perform data processing on the data in the first address segment.

[0218] For example, based on the migration identifier, if the data in the first address segment has not been migrated, then the data in the first address segment is directly processed.

[0219] Step 1709: End.

[0220] The foregoing mainly describes the solution provided in this application. Accordingly, this application also provides a data migration apparatus for implementing the above-described method embodiments.

[0221] In some embodiments, the data migration apparatus includes hardware structures and / or software modules corresponding to the execution of each function in order to achieve the above-described functions. Those skilled in the art will readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0222] This application embodiment can divide the data migration device into functional modules according to the above method embodiment. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0223] In some embodiments, this application provides a data migration apparatus for implementing the functions of the data migration apparatus in the above-described data migration method embodiments. For example... Figure 18 The diagram shows the structure of the data migration device. The data migration device may include a migration module 1801, an update module 1802, and a data processing module 1803.

[0224] Among them, the migration module 1801 is used to execute Figure 5 The illustrated method includes step 502. The update module 1802 is used to perform this operation. Figure 5 The illustrated method includes step 504. The data processing module 1803 is used to execute... Figure 7 The illustrated method includes step 506.

[0225] like Figure 19 As shown, the computer device provided in this application embodiment may include a processor 1901, a bus 1902, a communication interface 1903, and a memory 1904. The processor 1901, the memory 1904, and the communication interface 1903 communicate with each other via the bus 1902. It should be understood that this application does not limit the number of processors and memories in the computer device.

[0226] The 1902 bus can be a PCI bus, an Extended Industry Standard Architecture (EISA) bus, or a UB bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 19 The bus 1902 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 1902 may include a path for transmitting information between various components of a computer device (e.g., memory 1904, processor 1901, communication interface 1903).

[0227] Processor 1901 may include any one or more processors such as CPU, graphics processing unit (GPU), microprocessor (MP), or digital signal processor (DSP).

[0228] The memory 1904 may include volatile memory, such as random access memory (RAM). The processor 1901 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0229] The communication interface 1903 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between computer devices and other devices or communication networks.

[0230] The memory 1904 stores executable program code, which the processor 1901 executes to implement the functions of the data migration device or the CPU core in the aforementioned method embodiments. That is, the memory 1904 stores information for executing the aforementioned data migration method.

[0231] In another aspect, a computer-readable storage medium is provided, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor to implement the data migration method provided in the above-described method embodiments.

[0232] In another aspect, a computer program product is provided, which includes a computer program or instructions that, when executed by a processor, implement the data migration method described above.

[0233] In another aspect, a chip is provided, including at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions and send instructions to the at least one processor, and when the at least one processor executes the instructions, the at least one processor performs to implement the data migration method as described above.

[0234] In another aspect, a computer system is provided, comprising a chip, a first storage medium and a second storage medium connected to the chip; when the chip executes instructions, it executes the data migration method described above based on the first storage medium and the second storage medium.

[0235] The method steps in this embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a computer device. Of course, the processor and storage medium can also exist as discrete components in the computer device.

[0236] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are performed entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid-state drive (SSD). The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data migration method, characterized in that, The method is applied to a chip connected to a first storage medium and a second storage medium. The first storage medium and the second storage medium have different I / O performance. Every N address segments in the first storage medium correspond to every N address segments in the second storage medium, where N is greater than or equal to 1. The method includes: The data at the location indicated by the first address segment in the first storage medium is migrated to the location indicated by the second address segment in the second storage medium; the second address segment is one of the N address segments to which the first address segment belongs in the second storage medium. Update the M-bit migration flag of the first address segment; the M-bit migration flag is used to record the migration status of the contents stored in the N address segments in the first storage medium and the second storage medium.

2. The method according to claim 1, characterized in that, The M is Round up, or, M is N.

3. The method according to claim 1 or 2, characterized in that, The third address segment in the first storage medium corresponds to the fourth address segment in the second storage medium; when the migration flag of the third address segment is a first value, the first value indicates that the initial storage content of the first address segment has been migrated to the fourth address segment; when the migration flag of the third address segment is a second value, the second value indicates that the initial storage content of the first address segment has not been migrated.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Receive a data processing request, the data processing request including a destination address, the destination address being located in the fifth address segment; The data processing request is executed for the data in the storage location indicated by the migration identifier of the fifth address segment.

5. The method according to claim 4, characterized in that, The migration identifier is stored in the checksum bit (ECC); the method further includes: In the ECC of the fifth address segment, the migration identifier of the fifth address segment is read.

6. The method according to claim 5, characterized in that, The migration identifier is stored in a free space in the ECC.

7. The method according to claim 4, characterized in that, The chip is configured with an L3 cache for storing the migration identifier; the method further includes: In the L3 cache, read the migration identifier of the fifth address segment.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: While migrating the data in the first address segment of the first storage medium to the second address segment of the second storage medium, the data in the second address segment of the second storage medium to the first address segment of the first storage medium is also migrated.

9. A data migration device, characterized in that, The device includes: The migration module is used to migrate data from the location indicated by the first address segment in the first storage medium to the location indicated by the second address segment in the second storage medium; the second address segment is one of the N address segments to which the first address segment belongs in the second storage medium. An update module is used to update the M-bit migration identifier of the first address segment; the M-bit migration identifier is used to record the migration status of the contents stored in the N address segments in the first storage medium and the second storage medium.

10. A computing device, characterized in that, The computing device includes a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the data migration method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the data migration method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, The computer program product includes a computer program or instructions that, when executed by a processor, implement the data migration method as described in any one of claims 1 to 8.

13. A chip, characterized in that, The chip includes at least one processor and at least one interface circuit, the at least one interface circuit being used to perform transceiver functions and send instructions to the at least one processor, and when the at least one processor executes the instructions, the at least one processor performs to implement the data migration method as described in any one of claims 1 to 8.

14. A computer system, characterized in that, The computer system includes a chip, a first storage medium and a second storage medium connected to the chip; when the chip executes instructions, it performs the data migration method as described in any one of claims 1 to 8 based on the first storage medium and the second storage medium.

Images