Data processing method, readable medium, storage system and program product

Abstract

The invention relates to the technical field of data storage, and discloses a data processing method, a readable medium, a storage system and a program product. In the method, in the process of migrating to-be-migrated data from a first storage space of a first storage medium to a second storage space of a second storage medium, if a first access request for the first storage space or the second storage space is detected, the to-be-migrated data is migrated based on an access condition of the first storage space or the second storage space. And determining a corresponding access strategy. Wherein the access condition can be determined based on the data migration state of the access storage space corresponding to the first access request and the access type of the first access request. Thus, in the data migration process, the first access request determines the corresponding access strategy based on the data migration state of the access storage space corresponding to the first access request for access, the waiting time of the first access request can be shortened, and then the execution efficiency of the process corresponding to the first access request can be improved.

Description

Technical Field

[0001] The present application relates to the field of data storage technology, and in particular to a data processing method, a readable medium, a storage system, and a program product. Background Art

[0002] In data tiered storage technology, the data type corresponding to the data in the storage system, such as hot data or cold data, can be determined based on parameters such as the access frequency of the data in the storage system. Furthermore, hot data can be stored in the storage space corresponding to the high-performance tier of the storage system, while cold data can be stored in the storage space corresponding to the low-performance tier of the storage system. It is understood that the higher the performance tier, the smaller the corresponding storage space size and the faster the access speed. The high-performance tier can be, for example, main memory, and the low-performance tier can be, for example, extended memory.

[0003] To improve access efficiency, the storage system needs to migrate cold data with low access frequency in the storage space corresponding to the high-performance tier to the storage space corresponding to the low-performance tier, and migrate hot data with high access frequency in the storage space corresponding to the low-performance tier to the storage space corresponding to the high-performance tier.

[0004] Currently, data migration is typically accomplished by executing load and store instructions to read data from one storage space and write data to another, or by using direct memory access (DMA) to read data from one storage space and write data to another. However, during the data migration process within a storage system, if access to that storage space is restricted, data in that storage space cannot be read, and new data cannot be written to that storage space. Summary of the Invention

[0005] The purpose of this application is to provide a data processing method, a readable medium, a storage system and a program product.

[0006] A first aspect of the present application provides a data processing method, which is applied to a hierarchical storage system, the hierarchical storage system including a first storage medium and a second storage medium with different performance levels; and the method includes: receiving a data migration request, wherein the data migration request is used to request that the data to be migrated in the first storage space of the first storage medium be migrated to the second storage space of the second storage medium; during the data migration process, receiving a first access request; based on the data migration status of the access storage space corresponding to the first access request and the access type of the first access request, determining an access policy corresponding to the first access request, wherein the access storage space is the first storage space or the second storage space, and the access policy includes an allow access policy.

[0007] In an embodiment of the present application, the first storage medium may be an extended memory, the second storage medium may be a main memory, and the data migration request may be a migration request.

[0008] It can be understood that during the data migration process, an access policy corresponding to the first access request is determined based on the data migration status of the access storage space corresponding to the first access request and the access type of the first access request. Therefore, the first access request does not need to wait for the migration of all the to-be-migrated data in the to-be-migrated pages in the access storage space (hereinafter referred to as the to-be-migrated data set) to be completed before accessing the access storage space. Instead, the access policy is determined based on the data migration status of the access storage space corresponding to the first access request to perform the access. This can shorten the waiting time of the first access request and thereby improve the execution efficiency of the process corresponding to the first access request.

[0009] In a possible implementation of the first aspect above, the allowed access strategy includes at least one of the following: a direct access strategy and a waiting access strategy; wherein the direct access strategy includes: reading data in the access storage space, or writing the data corresponding to the first access request into the access storage space; the waiting access strategy includes: using the data corresponding to the first access request as the data to be written into the access storage space, and, corresponding to the data migration status of the access storage space being completed migration, writing the data to be written into the access storage space.

[0010] In one possible implementation of the first aspect above, the access strategy also includes: a transfer access strategy; and the transfer access strategy includes at least one of the following: reading the data to be written in the access storage space, wherein the data to be written includes the data to be migrated; writing the data corresponding to the first access request into the storage space corresponding to the data to be written in the access storage space, wherein the storage space corresponding to the data to be written includes the first storage space.

[0011] In an embodiment of the present application, the waiting access strategy includes: using the data corresponding to the first access request as the data to be written to the access storage space, and when the data migration status of the access storage space changes from unfinished migration to completed migration, writing the data to be written to the access storage space.

[0012] In one possible implementation of the first aspect above, an access policy corresponding to the first access request is determined based on the data migration status of the access storage space corresponding to the first access request and the access type of the first access request, including: when the data migration status corresponding to the access storage space is completed migration, the access policy is determined to be a direct access policy.

[0013] In a possible implementation of the first aspect above, an access policy corresponding to the first access request is determined based on the data migration status of the access storage space corresponding to the first access request and the access type of the first access request, and the access policy is also determined by at least one of the following methods: corresponding to the access storage space being the first storage space, the access policy is determined to be a direct access policy or a transfer access policy; corresponding to the access storage space being the second storage space, the access policy is determined to be a transfer access policy, and the storage space corresponding to the data to be written is the first storage space.

[0014] In a possible implementation of the first aspect above, an access policy corresponding to the first access request is determined based on the data migration status of the access storage space corresponding to the first access request and the access type of the first access request, and the access policy is also determined by at least one of the following methods: corresponding to the access storage space being the first storage space, the access policy is determined to be a waiting access policy; corresponding to the access storage space being the second storage space, the access policy is determined to be a transfer access policy, and the data to be written is data to be migrated.

[0015] In a possible implementation of the first aspect above, the data migration request includes at least one of the following parameters: address information of the first storage space, address information of the second storage space, and exchange indication information, wherein the exchange indication information is used to indicate whether the data in the first storage space is exchanged and stored with the data in the second storage space.

[0016] In one possible implementation of the first aspect above, the data to be migrated in the first storage space is migrated to the second storage space by the following method: reading the data to be migrated in the first storage space, or storing first data with address information corresponding to the first storage space in the third storage space corresponding to the third storage medium, reading the first data, wherein the performance level of the third storage medium is higher than that of the first storage medium and the second storage medium; and writing the data to be migrated or the first data into the second storage space, or storing second data with address information corresponding to the second storage space in the fourth storage space corresponding to the fourth storage medium, clearing the second data, and writing the data to be migrated or the first data into the second storage space, wherein the performance level of the fourth storage medium is higher than that of the first storage medium and the second storage medium.

[0017] In an embodiment of the present application, the data to be migrated in the first storage space is also migrated to the second storage space by the following method: reading the data to be migrated in the first storage space, or, storing the first data corresponding to the address information of the first storage space in the third storage space corresponding to the third storage medium, reading the first data, wherein the third storage medium is a storage medium with the highest performance level among the storage media storing the first data corresponding to the address information of the first storage space higher than the first storage medium and the second storage medium; and, writing the data to be migrated or the first data to the second storage space, or, storing the second data corresponding to the address information of the second storage space in multiple fourth storage media, clearing the second data stored in multiple fourth storage media, and writing the data to be migrated or the first data to the second storage space, wherein the performance levels of multiple fourth storage media are higher than the first storage medium and the second storage medium.

[0018] A second aspect of the present application provides a readable medium having instructions stored thereon. When the instructions are executed in a storage system, the storage system is caused to execute any one of the methods of the first aspect.

[0019] A third aspect of the present application provides a storage system, comprising: a memory for storing instructions executed by one or more processors of the storage system, and a processor for executing any one of the methods of the first aspect above.

[0020] In a possible implementation of the third aspect, the processor further includes a migrator configured to execute any one of the methods of the first aspect.

[0021] In an embodiment of the present application, the data processing method is implemented by a migrator, which has global access capabilities, that is, it can access caches at all levels, main memory, and extended memory.

[0022] It can be understood that when data needs to be migrated, the migration request is sent to the migrator, and the data migration is implemented by the migrator. The processor core is not needed to perform data migration. The processor core can perform other operations during the data migration process. In this way, the processor time can be reduced and the processor performance can be improved.

[0023] A fourth aspect of the present application provides a program product, which includes instructions. When the instructions are executed in a storage system, the storage system implements any one of the methods of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 According to an embodiment of the present application, a schematic diagram of a pyramid structure of a memory is shown;

[0026] Figure 2 According to an embodiment of the present application, a structural diagram of a hierarchical storage system is shown;

[0027] Figure 3 According to an embodiment of the present application, a schematic diagram of address mapping is shown;

[0028] Figure 4a According to an embodiment of the present application, a schematic diagram of a process for implementing data migration through load instructions and store instructions is shown;

[0029] Figure 4b According to an embodiment of the present application, a schematic diagram of implementing data migration through load instructions and store instructions is shown;

[0030] Figure 4c According to an embodiment of the present application, a schematic diagram of data migration overhead is shown;

[0031] Figure 4d According to an embodiment of the present application, another schematic diagram of data migration overhead is shown;

[0032] Figure 5a According to an embodiment of the present application, a schematic diagram of a process for implementing data migration through DMA is shown;

[0033] Figure 5b According to an embodiment of the present application, a schematic diagram of implementing data migration through DMA is shown;

[0034] Figure 6 According to an embodiment of the present application, a schematic diagram of a data migration process is shown;

[0035] Figure 7 According to an embodiment of the present application, a structural diagram of a storage system is shown;

[0036] Figure 8a According to an embodiment of the present application, a schematic diagram of a process for implementing data migration through a migrator is shown;

[0037] Figure 8b According to an embodiment of the present application, a schematic diagram of implementing data migration through a migrator is shown.

[0038] Figure 9 According to an embodiment of the present application, a flow chart of a data processing method is shown;

[0039] Figure 10 According to an embodiment of the present application, a schematic structural diagram of a migration request is shown. DETAILED DESCRIPTION

[0040] Illustrative embodiments of the present application include, but are not limited to, a data processing method, a readable medium, a storage system, and a program product.

[0041] In order to make the objectives, technical solutions and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and comprehensively described below with reference to the accompanying drawings.

[0042] Before introducing the technical solutions involved in the embodiments of the present application, some of the terms included in the embodiments of the present application are explained.

[0043] (1) Hierarchical storage system

[0044] In computers, a multi-level memory architecture is usually adopted. Figure 1 According to an embodiment of the present application, a schematic diagram of a pyramid structure of a memory is shown.

[0045] like Figure 1As shown in Figure 1, the computer's memory pyramid structure includes registers, cache, main memory, compute express link memory (CXL memory), solid state disks (SSDs), and hard disk drives (HDDs). Registers and cache are integrated into the processor, and the cache can be multi-level, such as L1 cache, L2 cache, and L3 cache.

[0046] It can be understood that in the memory pyramid structure, the higher the performance level, the smaller the corresponding storage space, the closer it is to the processor, and the faster the access speed. Conversely, the lower the performance level, the larger the corresponding storage space, the farther it is from the processor, and the slower the access speed.

[0047] For example, Figure 2 According to an embodiment of the present application, a structural diagram of a hierarchical storage system is shown.

[0048] like Figure 2 As shown, the storage system 100 may include a processor 110, a main memory 120, an extended memory 130, a solid state drive 140, a mechanical hard disk 150, and the like.

[0049] The processor 110 may include one or more processing units, for example, a central processing unit (CPU), a graphics processing unit (GPU), a neural-network processing unit (NPU), an application processor (AP), a modem processor, an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), and / or a baseband processor. The different processing units may be independent devices or integrated into one or more processors.

[0050] like Figure 2As shown, processor 110 may include n processor cores: processor core 1, processor core 2, ..., processor core n. Each processor core has a corresponding level 1 cache and a corresponding level 2 cache. The n processor cores share a level 4 cache, which may be referred to as a last level cache (LLC). It will be appreciated that each processor core also has at least one corresponding register (not shown) for storing data and / or instructions.

[0051] The cache is a high-speed cache memory, which may include static random access memory (SRAM), and is used to store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the instruction or data again, it can directly call it from the cache. This avoids repeated accesses, reduces processor 110's waiting time, and thus improves system efficiency.

[0052] Main memory 120 may include dynamic random access memory (DRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, DDR for short), etc. Main memory 120 is used to store currently executing computer programs and data to be processed by processor 110. It should be understood that main memory 120 can be directly accessed and randomly accessed by processor 110 and is a temporary storage space, not a permanent storage medium.

[0053] Extended memory 130 is connected to processor 110 via a CXL serial interface. Processor 110 can access extended memory 130 by executing load and store instructions, or by DMA. Extended memory 130 may include DRAM, low-power double-data-rate synchronous dynamic random access memory (LPDDR), and non-volatile memory (NVM). Extended memory 130 may also include new types of persistent memory (PMEM), such as phase change memory (PCM) and 3D XPoint.

[0054] When accessing data, the processor core first checks the L1 cache for the data it needs to access. If the data doesn't exist in the L1 cache, the processor core then checks the L2 cache for the data it needs to access. If the data doesn't exist in the L2 cache, the processor core then checks the L3 cache for the data it needs to access, and so on. If the processor core finds the data it needs to access in a cache at a lower performance level, it copies the data to a cache at a higher performance level to make it more quickly available the next time it's accessed. For example, if the processor core finds data A in the L2 cache, it copies data A to the L1 cache. If the data is not found in the cache, the processor core reads the data from main memory 120 and copies it to the appropriate cache level for faster access the next time it's accessed. For example, if the processor core finds data B in main memory 120, it copies data B to the L3 cache, or to both the L2 and L3 caches, or to the L1, L2, and L3 caches. It is understandable that if the required data is not found in the main memory 120 , the required data may be further searched in the extended memory 130 .

[0055] Among them, the smallest cache unit in the cache is called a cache line, and the size of the cache line can be determined based on the size of the corresponding cache. For example, the size of the cache is 64 bytes, that is, a maximum of 64 bytes of data can be stored in the cache. The size of the corresponding cache line can be 8 bytes, that is, the cache is evenly divided into 8 cache lines of 8 bytes in size. The size of the corresponding cache line can also be 4 bytes, that is, the cache is evenly divided into 16 cache lines of 4 bytes in size. In actual applications, the size of a cache line is usually between 4 bytes and 128 bytes, for example, 4 bytes, 8 bytes, 16 bytes, 64 bytes, 128 bytes, etc.

[0056] It is understood that the minimum unit of data transfer between the cache and main memory 120 is a cache line. For example, the size of a cache line is 8 bytes. Even if the data to be accessed is 1 byte, the processor core will read 8 bytes of data from main memory 120 and write it to the cache.

[0057] (2) Virtual address (VA) and physical address (PA)

[0058] During the execution of a computer program, all addresses used by the computer program are virtual addresses, and the memory management unit of the processor 110 allocates available physical storage space (storage space corresponding to the physical address) to each computer program.

[0059] The memory management unit maps the virtual addresses used in computer programs to specific physical addresses, usually in units of pages. When performing memory management, the memory management unit divides the physical storage space into pages, where the pages obtained by dividing the physical storage space can be called physical pages. Similarly, the virtual storage space of the computer program (the storage space corresponding to the virtual address) is also divided into pages, where the pages obtained by dividing the virtual storage space can be called virtual pages. The pages obtained by dividing the virtual storage space and the physical storage space are both numbered sequentially to obtain page numbers. Address mapping is to map a virtual page to a physical page.

[0060] Taking the page size (or page size) as 4 kilobytes (KB) as an example, in the address mapping process, if Figure 3 As shown, both the virtual address and the physical address are 64 bits (bits for short), and accordingly, the page occupies 4KB of space. The lower 12 bits of the virtual address and the physical address (i.e., the [11:0] field) remain unchanged. In other words, the lower 12 bits of the virtual address and the physical address are the same, and are both used to represent the offset within the page. In the virtual address, it is called the virtual page offset (VA offset), and in the physical address, it is called the physical page offset (PA offset). The upper portion of the virtual address and the physical address (i.e., the [63:12] field) is called the virtual page number (VPN) in the virtual address and the physical frame number (PFN) in the physical address. It can be understood that during the address mapping process, the VA offset of the physical address and the PA offset of the virtual address are always the same.

[0061] I understand. Figure 3 The address mapping process is introduced only by taking the page size of 4KB as an example. The page size can also be 8KB, 16KB, 64KB, etc. This application does not limit the page size.

[0062] (3) Page table (or translation table)

[0063] During the execution of a computer program, an electronic device typically establishes a complete mapping relationship from virtual addresses to physical addresses for each computer program and stores it in a page table data structure. A page table can include multiple page table entries, each of which contains corresponding VPN information and PFN information. The page table can be a single-level or multi-level structure. It is understood that each process corresponds to a page table.

[0064] (4) Translation lookaside buffer (TLB)

[0065] Typically, the page table is stored in memory (main memory 120 or extended memory 130). The conversion of virtual addresses to physical addresses requires multiple memory accesses, significantly reducing access performance. Therefore, a fast translation buffer (also known as a translation lookaside buffer, translation lookaside cache, or fast table) is typically used on the processor 110 to store partial page table entries (PTEs) of the page table. It can be understood that the TLB can be considered a cache for the page table, storing a portion of the complete page table.

[0066] (5) Hot data and cold data

[0067] Hot data refers to data that is frequently accessed and critical to business and applications. This data usually needs to be accessed and processed quickly and efficiently, so it needs to be stored in high-performance, low-latency storage media, such as Figure 2 Main memory 120 is shown.

[0068] Cold data refers to data that is accessed less frequently and is not very important to business and applications. This data needs to be stored for a long time, but does not require frequent access and processing. Therefore, it can be stored in lower-cost and higher-capacity storage media, such as Figure 2 Expansion memory 130 is shown.

[0069] Therefore, the hotness of data can be determined based on the access frequency of the data. When the hotness of the data exceeds the hotness threshold, the data type is determined to be hot data. Otherwise, the data type is determined to be cold data.

[0070] It's understandable that the hotness or coldness of data can also be related to the data's access time window. That is, data popularity can be time-sensitive. For example, data types corresponding to a certain time interval are considered hot, while data types corresponding to data outside that time interval are considered cold. Therefore, data hotness and coldness can also be determined based on the data's access time window, further determining the data type.

[0071] It can be understood that the above-mentioned methods of determining data types are only some examples listed in the embodiments of the present application and do not represent all examples. Other methods of determining data types obtained based on the methods of determining data types mentioned in the embodiments of the present application are also within the protection scope of the embodiments of the present application and are not specifically limited in the embodiments of the present application.

[0072] As mentioned above, in order to improve access efficiency, the storage system needs to migrate cold data with low access frequency in the storage space corresponding to the high-performance tier to the storage space corresponding to the low-performance tier, and migrate hot data with high access frequency in the storage space corresponding to the low-performance tier to the storage space corresponding to the high-performance tier.

[0073] In some embodiments, data migration is typically implemented via load instructions and store instructions.

[0074] For example, taking the data type of the data to be migrated as hot data, Figure 4a According to an embodiment of the present application, a flow chart of implementing data migration through load instructions and store instructions is shown. It can be understood that Figure 4a The execution subject of each step of the process shown is the storage system 100. Figure 4a The execution entities of each step will not be described repeatedly in the steps of the process shown. Figure 4a As shown, the process includes but is not limited to the following steps:

[0075] S401: Clearing a page table corresponding to data to be migrated and refreshing a TLB corresponding to the page table, wherein the data to be migrated is stored in a first storage medium.

[0076] In some embodiments, the storage system 100 clears the page table corresponding to the data to be migrated and refreshes the TLB. That is, the mapping relationship between the virtual address of the data to be migrated stored in the storage system 100 and the physical address in the first storage medium is cancelled. The first storage medium may be Figure 2 Expansion memory 130 is shown.

[0077] S402: Apply for a new page for the data to be migrated in the second storage medium.

[0078] In some embodiments, the storage system 100 determines a storage space for storing the data to be migrated from the second storage medium so as to store the data to be migrated in the storage space. The second storage medium may be Figure 2 The main memory 120 is shown. It can be understood that the performance level corresponding to the second storage medium in the storage system 100 is higher than the performance level corresponding to the first storage medium in the storage system 100.

[0079] In some embodiments, since the data stored in the main memory 120 / extended memory 130 is usually in pages, the storage system 100 applies for new pages in the second storage medium for the data to be migrated.

[0080] S403: Migrate the data to be migrated from the page of the first storage medium to the page of the second storage medium through a load instruction and a store instruction.

[0081] In some embodiments, after the storage system 100 determines the storage space for storing the data to be migrated from the second storage medium, the data to be migrated is read from the first storage medium to the register through the data bus through a load instruction, and then the data to be migrated in the register is written into the corresponding storage space in the second storage medium through a storage instruction.

[0082] It is understood that the load instruction and the store instruction are executed by the processor core. Figure 4b The diagram shown is a schematic diagram of data migration achieved through load instructions and store instructions.

[0083] Specific as Figure 4b As shown, when the first storage medium is Figure 2 The extended memory 130 shown, the second storage medium is Figure 2 When the main memory 120 is shown, the processor core loads the data to be migrated from the extended memory 130 , and then stores the loaded data to be migrated into the main memory 120 .

[0084] S404: Modify the page table information in the page table corresponding to the data to be migrated.

[0085] In some embodiments, after the data to be migrated is migrated from the first storage medium to the second storage medium, the storage system 100 modifies the page table information in the page table corresponding to the data to be migrated in the second storage medium, that is, the mapping relationship between the virtual address of the data to be migrated and the physical address in the second storage medium is stored in the page table.

[0086] Since the process of implementing data migration through load instructions and store instructions requires the processor core to control the migration of the data to be migrated, a large amount of processor time is wasted, which may in turn reduce processor performance. In addition, to avoid access errors, during the data migration process, the storage system 100 usually locks the page corresponding to the data to be migrated in the first storage medium, that is, sets the storage space corresponding to the data to be migrated in the first storage medium to an inaccessible state. As a result, during the data migration process, access requests to the storage space corresponding to the data to be migrated need to wait until all the data to be migrated in the page corresponding to the data to be migrated has been migrated before the corresponding storage space corresponding to the data to be migrated can be accessed, which wastes a lot of time.

[0087] For example, Figure 4c According to an embodiment of the present application, a schematic diagram of data migration overhead of a 4KB page is shown.

[0088] like Figure 4cAs shown, in the process of upgrading the data of a 4KB page, that is, migrating the data to be migrated from a low-performance storage medium to a high-performance storage medium, the stage of applying for a new page consumes 0.378 microseconds (us), the stage of canceling the mapping relationship consumes 6.258us, the stage of data migration (that is, reading and writing data) consumes 3.949us, the stage of locking the page consumes 0.524us, and the stage of returning to the old page consumes 1.122us. In the process of downgrading the data of a 4KB page, that is, migrating the data to be migrated from a high-performance storage medium to a low-performance storage medium, the stage of applying for a new page consumes 0.511us, the stage of canceling the mapping consumes 6.776us, the stage of data migration consumes 3.723us, the stage of locking the page consumes 0.566us, and the stage of returning to the old page consumes 0.972us.

[0089] For example, Figure 4d According to an embodiment of the present application, a schematic diagram of data migration overhead of a 2MB page is shown.

[0090] like Figure 4d As shown in the figure, during the upgrade migration of 2MB page data, the unmapping phase took 226us and the data migration phase took 1426us. During the downgrade migration of 2MB page data, the unmapping phase took 236us and the data migration phase took 1739us.

[0091] Depend on Figure 4c and Figure 4d As shown in the diagram, regardless of whether it is an upgrade migration or a downgrade migration, the data migration phase for 4KB pages takes about 30% of the total time, and the data migration phase for 2MB pages takes about 70% of the total time, which takes up a lot of CPU time. Moreover, the larger the size of the data to be migrated, the more CPU time the data migration phase takes up.

[0092] Therefore, in other embodiments, data migration may be implemented through a DMA method.

[0093] For example, still taking the case where the data type of the data to be migrated is hot data, Figure 5a According to the embodiment of the present application, a flow chart of implementing data migration by DMA is shown. It can be understood that Figure 5a The execution subject of each step of the process shown is the storage system 100. Figure 5a The execution entities of each step will not be described repeatedly in the steps of the process shown. Figure 5a As shown, the process includes but is not limited to the following steps:

[0094] S501: Clearing a page table corresponding to data to be migrated and refreshing a TLB corresponding to the page table, wherein the data to be migrated is stored in a first storage medium.

[0095] S502: Apply for a new page for the data to be migrated in the second storage medium.

[0096] Specifically, S501 and S502 can refer to the above Figure 4a The relevant descriptions of S401 and S402 in the present application will not be repeated here.

[0097] S503: Migrate the data to be migrated from the page of the first storage medium to the page of the second storage medium by DMA.

[0098] In some embodiments, after the storage system 100 determines the storage space for storing the data to be migrated from the second storage medium, the data to be migrated is read from the first storage medium through the data bus by DMA and written into the corresponding storage space in the second storage medium.

[0099] It can be understood that the reading and writing of the data to be migrated is performed by a direct memory access controller (DMAC).

[0100] Specifically, reading the data to be migrated from the first storage medium via the data bus and writing it to the corresponding storage space in the second storage medium via DMA can include: the processor core sending a control signal to the DMAC to start the DMAC and transfer control of the data bus to the DMAC. The DMAC reads the data to be migrated from the first storage medium via the data bus and writes it to the second storage medium. After all the data to be migrated has been written to the second storage medium, the DMAC sends an interrupt request to the processor core to notify the processor core that migration of the data to be migrated is complete and returns control of the data bus to the processor core.

[0101] For example, refer to Figure 5b The schematic diagram of implementing data migration through DMA is shown.

[0102] like Figure 5b As shown, the processor core sends a control signal to the DMAC, and the DMAC reads the data to be migrated from the extended memory 130 and writes the data to be migrated to the main memory 120. After all the data to be processed is written, the DMAC sends an interrupt request to the processor core to notify the processor core that the migration of the data to be migrated is complete.

[0103] S504: Modify the page table information in the page table corresponding to the data to be migrated.

[0104] Specifically, S504 can refer to the above Figure 4a The relevant description of S404 in this application will not be repeated here.

[0105] It is understood that implementing data migration through DMA eliminates the need for the processor core to execute data migration. The processor core can perform other operations during the data migration process, reducing processor time and thereby improving processor performance. However, to avoid access errors, during the data migration process, the storage system 100 will still lock the pages corresponding to the data to be migrated in the first storage medium and the second storage medium.

[0106] That is, during the data migration process, the storage space corresponding to the data to be migrated cannot be accessed. If another process needs to access the storage space, it needs to wait until the data migration is completed. For example, when another process accesses the first storage space of the first storage medium, it needs to wait until all the data to be migrated in the pages to be migrated in the first storage medium are migrated before accessing the storage space of the first storage medium. This will result in a longer waiting time for the access instruction of the other process, which may affect the execution efficiency of the other process.

[0107] In view of this, an embodiment of the present application proposes a data processing method. In this method, during the process of migrating data to be migrated from a first storage space of a first storage medium to a second storage space of a second storage medium, if a first access request to the first storage space or the second storage space is detected, a corresponding access policy is determined based on the access conditions of the first storage space or the second storage space. The access conditions can be determined based on the data migration status of the access storage space (the first storage space or the second storage space) corresponding to the first access request and the access type of the first access request.

[0108] Thus, during the data migration process, a first access request does not need to wait for all the data to be migrated (hereinafter referred to as the to-be-migrated data set) in the pages to be migrated in the access storage space to complete migration before accessing the access storage space. Instead, the corresponding access policy is determined based on the data migration status of the access storage space corresponding to the first access request to perform the access. This can shorten the waiting time for the first access request and further improve the execution efficiency of the process corresponding to the first access request.

[0109] It can be understood that the access type of the first access request includes a read request and a write request, and the data migration status includes completed migration and incomplete migration.

[0110] In some embodiments, the access policy includes: a permission access policy and a transfer access policy, wherein the permission access policy includes a direct access policy and a wait access policy.

[0111] Specifically, the direct access strategy includes: reading the data in the access storage space, or writing the data corresponding to the first access request into the access storage space. The waiting access strategy includes: using the data corresponding to the first access request as the data to be written into the access storage space, and when the data migration status of the access storage space changes from unfinished migration to completed migration, writing the data to be written into the access storage space. The transfer access strategy includes: reading the data to be written into the access storage space, wherein the data to be written includes the data to be migrated; or writing the data corresponding to the first access request into the storage space corresponding to the data to be written into the access storage space, wherein the storage space corresponding to the data to be written includes the first storage space.

[0112] In some embodiments, if the migration status of the data to be migrated corresponding to the first storage space or the second storage space (i.e., the access storage space) is migration completed, the corresponding access policy is determined to be a direct access policy, that is, directly reading the data in the access storage space, or directly writing the data corresponding to the write request to the access storage space.

[0113] In other embodiments, if the migration status of the to-be-migrated data corresponding to the first storage space or the second storage space is incomplete migration, the corresponding access policy is determined based on the access type of the first access request.

[0114] Specifically, the access storage space corresponds to the first storage space:

[0115] If the first access request is a read request, it is determined whether there is data to be written in the first storage space. If so, the data to be written is read, that is, the access policy is a transfer access policy; if not, the data in the first storage space is directly read, that is, the access policy is a direct access policy.

[0116] If the first access request is a write request, the data corresponding to the write request is used as the data to be written to the first storage space, and the data to be written is written to the first storage space after the data to be migrated corresponding to the first storage space is migrated. That is, the access strategy is a waiting access strategy.

[0117] Corresponding to accessing the storage space as the second storage space:

[0118] If the first access request is a read request, the data in the first storage space corresponding to the second storage space in the first storage medium is read, that is, the data to be migrated to be written to the second storage space is read. In other words, the access policy is a transfer access policy, and the data to be written is data to be migrated.

[0119] If the first access request is a write request, the data corresponding to the write request is written to the first storage space corresponding to the second storage space in the first storage medium. That is, the data to be written to the second storage space is modified to the data corresponding to the write request, so that after the migration is complete, the data stored in the second storage space is the data corresponding to the write request. In other words, the access policy is a transfer access policy, and the storage space corresponding to the data to be written is the first storage space.

[0120] For example, Figure 6 According to an embodiment of the present application, a schematic diagram of a data migration process is shown.

[0121] like Figure 6 As shown, the data stored in storage space 61 of main memory 120 is a set of data to be migrated. The set of data to be migrated includes: data 01 to be migrated stored in storage space 611, data 02 to be migrated stored in storage space 612, data 03 to be migrated stored in storage space 613, data 04 to be migrated stored in storage space 614, and data 05 to be migrated stored in storage space 615. The set of data to be migrated is migrated to storage space 62 of extended memory 130. Storage space 621 corresponds to storage space 611 and is used to store data 01 to be migrated; storage space 622 corresponds to storage space 612 and is used to store data 02 to be migrated, and so on. Furthermore, data 01 to be migrated and data 02 to be migrated have already been migrated. That is, data 01 to be migrated has been written to storage space 621, and data 02 to be migrated has been written to storage space 622.

[0122] Corresponding to Figure 6 In the migration scenario shown, if a read request is received for storage space 621, data 01 to be migrated in storage space 621 is directly read. If a read request is received for storage space 624, data 04 to be migrated in storage space 614 corresponding to storage space 624 is read, that is, the data to be written to storage space 624 is read.

[0123] If a write request (writing data 06) is received for storage space 622, data 06 is directly written to storage space 622. If a write request (writing data 06) is received for storage space 625, data 06 is written to storage space 615 corresponding to storage space 625, that is, the data to be written to storage space 625 is modified to data 06.

[0124] If a read request is received for storage space 611, the data to be migrated 01 in storage space 611 is directly read. If a read request is received for storage space 614, the data to be migrated 04 in storage space 614 is directly read, or the data to be written to storage space 614 is read.

[0125] If a write request (writing data 07) is received for storage space 612, data 07 is directly written to storage space 612. If a write request (writing data 07) is received for storage space 615, data 05 to be migrated in storage space 615 is waited for to be migrated to storage space 625 before data 07 is written to storage space 615.

[0126] It can be understood that in the data processing method provided in the embodiment of the present application, during the data migration process, the pages corresponding to the data to be migrated will not be locked. That is to say, during the data migration process, the storage space corresponding to the data to be migrated can be accessed based on the corresponding access strategy. If another process needs to access the storage space corresponding to the data to be migrated, the access request of the other process does not need to wait until the entire set of data to be migrated is migrated before accessing the storage space corresponding to the data to be migrated. This can shorten the waiting time of the access request of the other process, thereby improving the execution efficiency of the other process.

[0127] Furthermore, the data processing method provided in the embodiments of the present application implements the aforementioned data migration by adding a migrator. This migrator has global access capabilities, meaning it can access all levels of cache, main memory, and extended memory. When data migration is required, a migration request is sent to the migrator, which then performs the data migration. This eliminates the need for the processor core to perform the data migration, allowing the processor core to perform other operations during the data migration process. This reduces processor time and improves processor performance.

[0128] In order to better understand the technical solutions of the embodiments of the present application, some technical solutions of the present application are introduced in detail below.

[0129] For example, Figure 7 According to an embodiment of the present application, a structural diagram of a storage system 200 is shown.

[0130] like Figure 7 The storage system 200 may include a processor 210, a main memory 220, an extended memory 230, a solid-state drive 240, and a mechanical hard disk 250. The processor 210 may include n processor cores: processor core 1, processor core 2, ..., processor core n. Each processor core corresponds to a first-level cache and a second-level cache, and the n processor cores share an N-level cache (LLC). The processor 210 also includes a migrator 211 for completing the data processing method of the embodiment of the present application based on a migration request.

[0131] Specifically, the relevant description of the structure of the storage system 200 can refer to the above Figure 2 The description of the storage system 100 in FIG. 1 is not repeated in this application.

[0132] For example, still taking the case where the data type of the data to be migrated is hot data, Figure 8a According to an embodiment of the present application, a flow chart of implementing data migration through a migrator is shown. It can be understood that Figure 8a The execution subject of each step of the process shown is the storage system 200. Figure 8a The execution entities of each step will not be described repeatedly in the steps of the process shown. Figure 8a As shown, the process includes but is not limited to the following steps:

[0133] S801: Clearing a page table corresponding to data to be migrated and refreshing a TLB corresponding to the page table, wherein the data to be migrated is stored in a first storage medium.

[0134] S802: Apply for a new page in the second storage medium for the data to be migrated.

[0135] Specifically, S801 and S802 can refer to the above Figure 4a The relevant descriptions of S401 and S402 in the present application will not be repeated here.

[0136] S803: Migrating the data to be migrated from the page of the first storage medium to the page of the second storage medium through the migrator.

[0137] In some embodiments, after the storage system 200 determines the storage space for storing the data to be migrated from the second storage medium, it sends a migration request to the migrator, which reads the data to be migrated from the first storage medium and writes it to the corresponding storage space in the second storage medium.

[0138] Specifically, reading the data to be migrated from the first storage medium and writing it into the corresponding storage space in the second storage medium through the migrator may include: the migrator receives a migration request, reads the data to be migrated from the first storage medium according to the migration request, and writes it into the second storage medium.

[0139] For example, refer to Figure 8b The diagram shows how to implement data migration through the migrator.

[0140] like Figure 8b As shown, the migrator executes a consistent read instruction (such as readunique) from Figure 7 The extended memory 230 (as an example of the first storage medium) shown in FIG. 1 reads the data to be migrated and writes the data to be migrated into the memory 230 by executing a consistent write instruction (eg, writeunique). Figure 7 Main memory 220 is shown (as an example of a second storage medium).

[0141] It is understood that during the execution of the consistent read instruction, it is first determined whether there is a storage space (as an example of the third storage space) in the level 1 to N cache (as an example of the third storage medium) with the same address information as the storage space corresponding to the data to be migrated in the extended memory 230. If so, the migrator reads the updated data (as an example of the first data) in the corresponding storage space of the cache with the highest performance level in the level 1 to N cache, where the first data may refer to data that has been written to the cache and has not yet been synchronized to the main memory 220 or the extended memory 230; if not, the migrator directly reads the data to be migrated from the corresponding storage space in the extended memory 230.

[0142] During the execution of the consistency write instruction, it is first determined whether there is a storage space (as an example of the fourth storage space) in the 1 to N level cache (as an example of the fourth storage medium) with the same address information as the storage space corresponding to the data to be migrated in the main memory 220. If so, the migrator clears the data (as an example of the second data) in the corresponding storage space of each level of cache that has the same address information as the storage space corresponding to the consistency write instruction, where the second data refers to the data stored in the storage space with the same address information as the storage space corresponding to the consistency write instruction in the cache, and then writes the read data to be migrated or the updated data into the corresponding storage space in the main memory 220; if not, the migrator directly writes the read data to be migrated or the updated data into the corresponding storage space in the main memory 220.

[0143] S804: Modify the page table information in the page table corresponding to the data to be migrated.

[0144] Specifically, S804 can refer to the above Figure 8a The relevant description of S804 in this application will not be repeated here.

[0145] In order to better understand the technical solutions of the embodiments of the present application, Figure 9 The flowchart shown provides a detailed introduction to the data processing method of the present application.

[0146] For example, Figure 9 According to the embodiment of the present application, a flow chart of a data processing method is shown. It can be understood that Figure 9 The execution subject of each step of the process shown is the storage system 200. Figure 9 The execution entities of each step will not be described repeatedly in the steps of the process shown. Figure 9 As shown, the process includes but is not limited to the following steps:

[0147] S901: Receive a migration request.

[0148] In some embodiments, the migration request may include: the first address (src) of the storage space corresponding to the first storage medium (source storage medium), the first address (dst) of the storage space corresponding to the second storage medium (destination storage medium), the index information (index) of the storage space corresponding to the first storage space in the first storage medium, and the swap indication information (swap).

[0149] Among them, the performance levels corresponding to the first storage medium and the second storage medium are different, the index information is used to indicate the location information of the data to be processed in the set of data to be processed, and the exchange indication information is used to indicate whether the data stored in the first storage space and the data stored in the second storage space are exchanged for storage. For example, if the exchange indication information is 1, it means that the data stored in the first storage space and the data stored in the second storage space need to be exchanged for storage during the migration process. If the exchange indication information is 0, it means that the data stored in the first storage space and the data stored in the second storage space do not need to be exchanged for storage during the migration process, and only the data in the first storage space needs to be migrated to the second storage space for storage.

[0150] For example, in conjunction with reference Figure 7 If the first storage medium is the extended memory 230 and the second storage medium is the main memory 220, and the swap indication information is 0, it means that the hot data in the extended memory 230 needs to be migrated to the main memory 220. If the first storage medium is the main memory 220 and the second storage medium is the extended memory 230, and the swap indication information is 0, it means that the cold data in the main memory 220 needs to be migrated to the extended memory 230. If the first storage medium is the extended memory 230 and the second storage medium is the main memory 220, and the swap indication information is 1, it means that the hot data in the extended memory 230 needs to be swapped with the cold data in the main memory 220 for storage.

[0151] In some embodiments, the exchange indication information can be determined based on the size of the unallocated storage space in the second storage medium. For example, if the size of the unallocated storage space in the second storage medium is less than or equal to the first threshold, the exchange indication information is determined to be 1, that is, exchange storage is required. If the size of the unallocated storage space in the second storage medium is greater than the first threshold, the exchange indication information is determined to be 0 or 1, that is, exchange storage can be performed, or the corresponding unallocated storage space can be allocated to the data set to be migrated in the second storage medium (without exchange storage). This application does not make specific restrictions on this.

[0152] It can be understood that the storage space corresponding to the first storage medium is the storage space corresponding to the data set to be migrated, and its size can be, for example, 1KB, 2MB, etc. The size of the storage space corresponding to the second storage medium is the same as the size of the storage space corresponding to the first storage medium. The first storage space and the second storage space are sub-storage spaces of the corresponding storage space in the first storage medium and the corresponding storage space in the second storage medium, respectively. The sizes of the first storage space and the second storage space are the smallest storage units of data in the storage system 200, such as cache lines. The second storage space corresponds to the first storage space, and the index information of the storage space corresponding to the second storage space in the second storage medium is the same as the index information of the storage space corresponding to the first storage space in the first storage medium.

[0153] It can be understood that the address information of the first storage space can be determined based on the first address and index information of the storage space corresponding to the first storage medium, and the address information of the second storage space can be determined based on the first address and index information of the storage space corresponding to the second storage medium.

[0154] For example, refer to Figure 10 The structure diagram of the migration request is shown in FIG. Figure 10 As shown, the migration request includes src, dst, index and swap corresponding to multiple data to be migrated.

[0155] For example, if the size of the storage space corresponding to the data set to be migrated is 4KB and the size of the storage space corresponding to the data to be migrated is 64 bytes, the data set to be migrated includes 64 data to be migrated, that is, the index range in the migration request is [0:63].

[0156] S902: Perform data migration according to the migration request.

[0157] In some embodiments, the storage system 200 migrates the data to be migrated in the data set to be migrated in sequence based on the migration request.

[0158] Specifically, the storage system 200 can complete the migration of the data to be migrated by executing a consistent read instruction and a consistent write instruction.

[0159] It can be understood that if the first storage medium is the extended memory 230 and the second storage medium is the main memory 220, then during the execution of the consistency read instruction, it is first determined whether there is a storage space in the cache with the same address information as the storage space corresponding to the data to be migrated in the extended memory 230. If so, the migrator reads the updated data in the corresponding storage space in the cache; if not, the migrator directly reads the data to be migrated in the corresponding storage space in the extended memory 230. During the execution of the consistency write instruction, it is first determined whether there is a storage space in the cache with the same address information as the storage space corresponding to the data to be migrated in the main memory 220. If so, the migrator clears the data in the corresponding storage space in the cache, and then writes the read data to be migrated or the updated data into the storage space corresponding to the main memory 220; if not, the migrator directly writes the read data to be migrated or the updated data into the storage space corresponding to the main memory 220.

[0160] The following takes the example of a data set to be migrated with a storage space size of 4KB, a storage space size of 64 bytes, a consistent read instruction of readunique, and a consistent write instruction of writeunique to describe the migration process of the data set to be migrated in detail.

[0161] If the first storage medium is a slow medium and the second storage medium is a fast medium, that is, the performance level of the first storage medium is lower than the performance level of the second storage medium, and the exchange indication information is 0, the data process includes the following steps:

[0162] S9021: The migrator issues a consistent read request (src readunique request) for the slow media address and waits for data to be returned. In other words, the migrator waits to read updated data from the third storage medium or data to be migrated from the first storage medium. The third storage medium has a higher performance level than the fast media.

[0163] S9022: After the src readunique request returns data, the migrator writes the returned data to the flash medium using a writeunique request.

[0164] S9023: Increment the index information by 1, and continue executing S9021 until all the data in the to-be-migrated data set is migrated to the fast medium.

[0165] If the first storage medium is a fast medium and the second storage medium is a slow medium, that is, the performance level of the first storage medium is higher than the performance level of the second storage medium, and the exchange indication information is 0, the data process includes the following steps:

[0166] S9024: The migrator issues a consistent read request (src readunique request) for the fast media address and waits for data to be returned from the request. In other words, the migrator waits to read updated data from the third storage medium or data to be migrated from the first storage medium. The third storage medium has a higher performance level than the fast media.

[0167] S9025: After the src readunique request returns data, the migrator writes the returned data to the slow medium through a writeunique request.

[0168] S9026: Increment the index information by 1, and continue executing S9024 until all the data in the to-be-migrated data set is migrated to the slow medium.

[0169] If the first storage medium is a slow medium and the second storage medium is a fast medium, that is, the performance level of the first storage medium is lower than the performance level of the second storage medium, and the exchange indication information is 1, the data process includes the following steps:

[0170] S9027: The migrator issues a consistent read request src readunique1 for the slow media address.

[0171] S9028: The migrator issues a consistent read request src readunique1' for the fast media address.

[0172] S9029: Wait for both the src readunique1 request and the src readunique1' request to return data. In other words, the process waits for reading the first updated data in the third storage medium or the first data to be migrated in the first storage medium, and for reading the second updated data in the third storage medium or the second data to be migrated in the second storage medium. The performance level of the third storage medium is higher than that of the fast medium.

[0173] S9030: The migrator writes the data returned by the src readunique1 request to the fast media through the writeunique2 request.

[0174] S9031: The migrator writes the data returned by the src readunique1' request to the slow medium through the 'writeunique2' request.

[0175] S9032: Increment the index information by 1, and continue to execute S9027. All data in the data set to be migrated in the slow medium and the data set to be migrated in the fast medium are exchanged and stored.

[0176] S903: Receive a first access request and determine an access policy corresponding to the first access request.

[0177] In some embodiments, the storage system 200 determines the access type of the first access request based on the data migration status of the access storage space corresponding to the first access request and the access type of the first access request.

[0178] The access storage space corresponding to the first access request is the first storage space or the second storage space. The access type of the first access request includes a read request and a write request, and the data migration status includes completed migration and incomplete migration.

[0179] In some embodiments, if the migration status of the data to be migrated corresponding to the first storage space or the second storage space is completed, the data in the first storage space or the second storage space is directly read, or the data corresponding to the write request is directly written to the first storage space or the second storage space.

[0180] In other embodiments, if the migration status of the to-be-migrated data corresponding to the first storage space or the second storage space is incomplete migration, a corresponding access policy is determined based on the access type of the first access request.

[0181] For example, the access storage space is the first storage space:

[0182] If the first access request is a read request, it is determined whether there is data to be written in the first storage space. If so, the data to be written is read; if not, the data in the first storage space is directly read.

[0183] If the first access request is a write request, the data corresponding to the write request is used as the data to be written in the first storage space, and the data to be written is written into the first storage space after the data to be migrated corresponding to the first storage space is migrated.

[0184] For another example, corresponding to accessing the second storage space:

[0185] If the first access request is a read request, data in the first storage space corresponding to the second storage space in the first storage medium is read, that is, the data to be migrated to be written into the second storage space is read.

[0186] If the first access request is a write request, the data corresponding to the write request is written to the first storage space corresponding to the second storage space in the first storage medium, that is, the data to be written to the second storage space is modified to the data corresponding to the write request, so that after the migration is completed, the data stored in the second storage space is the data corresponding to the write request.

[0187] In other embodiments, if the swap indication information in the migration request is 1, that is, the data in the first storage space needs to be swapped with the data in the second storage space, the access policy for the access request to the first storage space and the second storage space is determined by the following method:

[0188] If the migration status of the to-be-migrated data corresponding to the access storage space is migration completed, the data in the access storage space is directly read, or the data corresponding to the write request is directly written to the access storage space.

[0189] If the migration status of the data to be migrated corresponding to the access storage space is incomplete migration, and the first access request is a read request, the data in the access storage space is read after the migration of the data to be migrated corresponding to the access storage space is completed.

[0190] If the migration status of the data to be migrated corresponding to the access storage space is incomplete migration, and the first access request is a write request, the data corresponding to the write request will be used as the data to be written to the access storage space. After the data to be migrated corresponding to the access storage space is completed, the data to be written will be written to the access storage space.

[0191] For example, if the starting address of the storage space corresponding to the first storage medium in the migration request is 0×2000, and the starting address of the storage space corresponding to the second storage medium is 0×4000, the data migration process has proceeded to the storage space with index information 5, that is, the storage space with address information 0×2140 and 0×4140.

[0192] The first access request corresponds to a read request:

[0193] If the address information of the access space corresponding to the first access request is 0×4040, and the data migration status of the access space is migration completed, the data in the storage space with the address information 0×4040 is directly read.

[0194] If the address information of the access space corresponding to the first access request is 0×4200 and the data migration status of the access space is incomplete migration, then the data in the storage space with the address information of 0×2200 is read.

[0195] If the address information of the access space corresponding to the first access request is 0×2040, and the data migration status of the access space is migration completed, the data in the storage space with the address information 0×2040 is directly read.

[0196] If the address information of the access space corresponding to the first access request is 0×2200 and the data migration status of the access space is incomplete migration, then determine whether there is a blocking write operation (i.e., data to be written). If so, read the data corresponding to the blocking write operation; if not, directly read the data in the storage space with the address information 0×2200.

[0197] The first access request corresponds to a write request:

[0198] If the address information of the access space corresponding to the first access request is 0×4040, and the data migration status of the access space is migration completed, the data corresponding to the first access request is directly written into the storage space with the address information 0×4040.

[0199] If the address information of the access space corresponding to the first access request is 0×4200 and the data migration status of the access space is incomplete migration, the data corresponding to the first access request is directly written into the storage space with the address information of 0×2200.

[0200] If the address information of the access space corresponding to the first access request is 0×2040 and the data migration status of the access space is migration completed, the storage space with the address information 0×2040 is directly read to write the data corresponding to the first access request.

[0201] If the address information of the access space corresponding to the first access request is 0×2200, and the data migration status of the access space is incomplete migration, a blocking write operation is performed (that is, the data corresponding to the first access request is written as the data to be written) until the index information reaches 8, that is, the data migration of the access space is completed and then the data corresponding to the first access request is written into the storage space with the address information of 0×2200.

[0202] S904: Access data based on the determined access policy.

[0203] In some embodiments, the storage system 200 performs a read or data write operation on the access space corresponding to the first access request or the storage space to which data is to be written corresponding to the access space based on the determined access policy.

[0204] S905: Data migration is completed.

[0205] In some embodiments, the storage system 200 determines that data migration is completed when all data in the to-be-migrated data set has completed migration or storage exchange.

[0206] It is understood that in other embodiments, according to actual needs, the above Figure 9The steps shown can be combined, deleted or replaced with other steps that are conducive to achieving the purpose of this application. For example, the above steps S903 and S904 can be combined into one step. This application does not impose any restrictions on this.

[0207] In summary, the data processing method provided by the embodiment of the present application is such that, during the data migration process, the pages corresponding to the data to be migrated will not be locked. In other words, during the data migration process, the storage space corresponding to the data to be migrated can be accessed based on the corresponding access policy. If another process needs to access the storage space corresponding to the data to be migrated, there is no need to wait for the data migration to be completed before accessing the storage space. Furthermore, the data processing method provided by the embodiment of the present application implements the above-mentioned data migration by adding a migrator. When data needs to be migrated, a migration request is sent to the migrator, which implements the data migration. The processor core does not need to execute the data migration. The processor core can execute other operations during the data migration process, which can reduce processor time and thus improve processor performance.

[0208] Embodiments of the present application also provide a storage system comprising a memory and a processor. The memory is configured to store instructions executed by one or more processors of the storage system. The processor is configured to implement the data processing method provided in the embodiments of the present application. In some embodiments, the processor further comprises a migrator for implementing the data processing method provided in the embodiments of the present application.

[0209] An embodiment of the present application also provides a program product, which includes instructions. When the instructions are executed in a storage system, the storage system implements the data processing method provided by the embodiment of the present application.

[0210] The various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0211] Program code can be applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0212] Program code can be implemented with high-level programming languages or object-oriented programming languages to communicate with the processing system. Where necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0213] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including but not limited to a floppy disk, an optical disk, an optical disk, a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic card or an optical card, a flash memory, or a tangible machine-readable memory for transmitting information (e.g., a carrier wave, an infrared signal, a digital signal, etc.) using the Internet in an electrical, optical, acoustic, or other form of propagation signal. Accordingly, machine-readable media includes any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).

[0214] In the accompanying drawings, some structural or method features may be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order may not be required. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of a structural or method feature in a particular figure does not imply that such feature is required in all embodiments, and in some embodiments, such features may not be included or may be combined with other features.

[0215] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems raised by this application. In addition, in order to highlight the innovative part of this application, the above-mentioned device embodiments of this application do not introduce units / modules that are not closely related to solving the technical problems raised by this application. This does not mean that other units / modules do not exist in the above-mentioned device embodiments.

[0216] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "including a" does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0217] Although the present application has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the application.

Claims

1. A data processing method, characterized in that: The method is applied to a hierarchical storage system, the hierarchical storage system comprising a first storage medium and a second storage medium having different performance levels; and The method comprises: receiving a data migration request, wherein the data migration request is used to request migration of data to be migrated in the first storage space of the first storage medium to the second storage space of the second storage medium; During the data migration process, a first access request is received; Based on the data migration status of the access storage space corresponding to the first access request and the access type of the first access request, an access policy corresponding to the first access request is determined, wherein the access storage space is the first storage space or the second storage space, and the access policy includes an allow access policy.

2. The method according to claim 1, characterized in that The access permission policy includes at least one of the following: direct access policy, waiting access policy; wherein, The direct access strategy includes: reading data in the access storage space, or writing data corresponding to the first access request into the access storage space; The waiting access strategy includes: using the data corresponding to the first access request as the data to be written to the access storage space, and when the data migration status of the access storage space is migration completed, writing the data to be written to the access storage space.

3. The method according to claim 2, characterized in that The access policy also includes: a transfer access policy; and The transfer access strategy includes at least one of the following: Reading the data to be written in the accessed storage space, wherein the data to be written includes the data to be migrated; The data corresponding to the first access request is written into the storage space corresponding to the data to be written in the access storage space, wherein the storage space corresponding to the data to be written includes the first storage space.

4. The method according to claim 3, characterized in that The determining, based on the data migration status of the access storage space corresponding to the first access request and the access type of the first access request, an access policy corresponding to the first access request includes: The data migration status corresponding to the access storage space is migration completed, and the access policy is determined to be a direct access policy.

5. The method according to claim 3, characterized in that The determining, based on the data migration status of the access storage space corresponding to the first access request and the access type of the first access request, an access policy corresponding to the first access request further includes: The data migration status corresponding to the access storage space is incomplete migration, and the access type of the first access request is a read request. The access policy is determined by at least one of the following methods: Corresponding to the access storage space being the first storage space, determining that the access policy is a direct access policy or a transfer access policy; Corresponding to the access storage space being the second storage space, the access policy is determined to be a transfer access policy, and the storage space corresponding to the data to be written is the first storage space.

6. The method according to claim 3, characterized in that The determining, based on the data migration status of the access storage space corresponding to the first access request and the access type of the first access request, an access policy corresponding to the first access request further includes: The data migration status corresponding to the access storage space is incomplete migration, and the access type of the first access request is a write request, and the access policy is determined by at least one of the following methods: Corresponding to the access storage space being the first storage space, determining the access strategy to be a waiting access strategy; Corresponding to the access storage space being the second storage space, the access policy is determined to be a transfer access policy, and the data to be written is the data to be migrated.

7. The method according to claim 1, characterized in that The data migration request includes at least one parameter from the following: The address information of the first storage space, the address information of the second storage space, and exchange indication information, wherein the exchange indication information is used to indicate whether to exchange and store data in the first storage space with data in the second storage space.

8. The method according to claim 1, migrating the data to be migrated in the first storage space to the second storage space by: Read the data to be migrated in the first storage space, or The first data corresponding to the address information of the first storage space is stored in the third storage space corresponding to the third storage medium, and the first data is read, wherein: The performance level of the third storage medium is higher than that of the first storage medium and the second storage medium; and writing the data to be migrated or the first data into the second storage space, or, The fourth storage space corresponding to the fourth storage medium stores second data corresponding to the address information of the second storage space, clears the second data, and writes the data to be migrated or the first data into the second storage space, wherein the performance level of the fourth storage medium is higher than that of the first storage medium and the second storage medium.

9. A readable medium, characterized in that The readable medium stores instructions, which, when executed in a storage system, enable the storage system to execute the method according to any one of claims 1 to 8.

10. A storage system, characterized in that: include: a memory for storing instructions to be executed by one or more processors of the storage system, and, A processor, configured to execute the method according to any one of claims 1 to 8.

11. The storage system according to claim 10, wherein: The processor further comprises a migrator for executing the method according to any one of claims 1 to 8.

12. A program product, characterized in that The program product includes instructions, which, when executed in a storage system, enable the storage system to implement the method according to any one of claims 1 to 8.

Images