Data migration method, device and equipment and computer readable storage medium
By assigning numbers to physical blocks in SSD storage devices and establishing block mapping tables, low-overhead data migration is achieved, solving the problem of data migration consuming a large amount of resources in the prior art, and improving the response speed and stability of the storage device.
Patent Information
- Application Number
- CN202510235967.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-28
- Publication Date
- 2025-04-18
AI Technical Summary
Existing SSD data migration methods require a large amount of I/O resources, caches and controller computing power during data migration, resulting in the impact of the response speed and stability of storage devices, especially in resource-intensive storage devices.
By assigning numbers to the physical addresses of each physical block in the storage device, a block mapping table is established, the data in the source physical block is migrated to the target physical block, and the target block is checked after completion. If successful, the addresses of the source and target blocks are exchanged in the block mapping table.
It realizes data migration with extremely low computing and cache overhead, improves data persistence efficiency, and reduces dependence on I/O resources, cache and controller computing capabilities.
Smart Images
Figure CN119847449A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of data storage, and in particular to a data migration method, device, equipment and computer-readable storage medium. Background Art
[0002] Solid-state drives (SSDs) are the mainstream technology for modern data storage. They are widely used in various scenarios such as personal computers, enterprise servers, and mobile devices due to their advantages such as high-speed reading and writing, low power consumption, and vibration resistance. SSDs use flash memory (NAND Flash) as storage media, and data is persisted by controlling the charge state of storage cells through voltage.
[0003] Data persistence refers to the ability of SSD to maintain stored data without loss. With the advancement of NAND flash memory technology, especially the application of high-density storage technologies such as multi-level cell (MLC), triple-level cell (TLC) and quad-level cell (QLC), the challenges faced by data persistence are increasing. Although these high-density storage technologies provide larger storage capacity and higher performance, they also lead to a decrease in data retention capabilities, especially when data is not updated for a long time, or in a high temperature environment and frequent read and write operations, the integrity and reliability of the data are threatened. Especially in large-scale data centers and enterprise-level storage, the requirements for data persistence are becoming more and more stringent.
[0004] Currently, SSDs use a data migration strategy to solve the data persistence problem. A commonly used data migration method is to periodically read data from an aging storage unit into a controller cache, and then rewrite the data into a new storage unit.
[0005] This data migration method can extend the data storage time to a certain extent, but in actual operation, the data involved in the NAND Flash read and write operations during the data migration process need to be directly controlled by the controller; in addition, after the data migration is completed, the correspondence between the logical address and the NAND Flash physical address has changed, and it is necessary to further update the logical block address (LBA) mapping table after the data migration, which requires a large amount of I / O resources, cache, and controller computing power. Especially in some storage devices with limited resources, this will greatly affect the response speed and stability of the storage device. Summary of the invention
[0006] To solve at least one of the above technical problems, the present application provides a data migration method, device, equipment and computer-readable storage medium.
[0007] In a first aspect, an embodiment of the present application provides a data migration method, the data migration method comprising:
[0008] Assign numbers to the physical addresses of each physical block in the storage device to obtain a block mapping table;
[0009] Migrating data in a source physical block to a target physical block, where the target physical block is a physical block that does not store data;
[0010] After the data migration is completed, the data in the target physical block is verified;
[0011] If the verification is successful, the positions of the physical address of the source physical block and the physical address of the target physical block in the block mapping table are exchanged.
[0012] In combination with the first aspect, in one implementation, the earlier the physical block is arranged in the storage device, the smaller its number is.
[0013] In combination with the first aspect, in one implementation, the source physical block is a physical block storing cold data.
[0014] In combination with the first aspect, in one implementation, the storage medium of the storage device is a flash memory, and migrating the data in the source physical block to the target physical block includes:
[0015] The data in the source physical block is migrated to the target physical block through the Copy Back function of the flash memory.
[0016] In combination with the first aspect, in one implementation, verifying the data in the target physical block includes:
[0017] Sending a read instruction to the target physical block, wherein the read instruction is a data bypass mode, so as to read the data in the target physical block to the error correction unit of the controller, and detecting whether the read data contains invalid data based on the error correction unit;
[0018] If there is invalid data, it is determined that the verification has failed;
[0019] If there is no invalid data, the verification is determined to be successful.
[0020] In combination with the first aspect, in one implementation, after verifying the data in the target physical block, the method further includes:
[0021] If the verification fails, the data in the target physical block is deleted, and the process returns to migrate the data in the source physical block to the target physical block.
[0022] In combination with the first aspect, in one implementation, after exchanging the positions of the physical address of the source physical block and the physical address of the target physical block in the block mapping table if the verification is successful, the method further includes:
[0023] Delete the data in the source physical block.
[0024] In a second aspect, an embodiment of the present application provides a data migration device, the data migration device comprising:
[0025] Establish a module for assigning numbers to the physical addresses of each physical block in the storage device to obtain a block mapping table;
[0026] A migration module, used to migrate data in a source physical block to a target physical block, where the target physical block is a physical block that does not store data;
[0027] A verification module is used to verify the data in the target physical block after the data migration is completed;
[0028] The update module is used for exchanging the positions of the physical address of the source physical block and the physical address of the target physical block in the block mapping table if the verification is successful.
[0029] In a third aspect, an embodiment of the present application provides a data migration device, comprising a processor, a memory, and a data migration program stored in the memory and executable by the processor, wherein when the data migration program is executed by the processor, the steps of the data migration method described in the first aspect are implemented.
[0030] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a data migration program is stored, wherein when the data migration program is executed by a processor, the steps of the data migration method described in the first aspect are implemented.
[0031] The beneficial effects brought by the technical solution provided in the embodiments of the present application include:
[0032] In the embodiment of the present application, the physical address of each physical block in the storage device is assigned a number to obtain a block mapping table; the data in the source physical block is migrated to the target physical block, and the target physical block is a physical block that does not store data; after the data migration is completed, the data in the target physical block is verified; if the verification is successful, the position of the physical address of the source physical block and the physical address of the target physical block in the block mapping table is exchanged. Through the embodiment of the present application, by establishing a specific block mapping table, only the block mapping table is updated after the data migration, and the amount of updated data is small, thereby completing the data migration with extremely low calculation and cache overhead. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 This is a flow chart of an embodiment of a data migration method of the present application;
[0034] Figure 2 This is a schematic diagram of a data migration scenario in accordance with an embodiment of the data migration method of the present application;
[0035] Figure 3 This is a functional module diagram of an embodiment of a data migration device of the present application;
[0036] Figure 4 This is a schematic diagram of the hardware structure of the data migration device involved in the embodiment of the present application. DETAILED DESCRIPTION
[0037] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0038] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.
[0039] In a first aspect, an embodiment of the present application provides a data migration method.
[0040] In one embodiment, referring to Figure 1 , Figure 1 This is a flow chart of an embodiment of the data migration method of this application. Figure 1 As shown, the data migration methods include:
[0041] Step S10, assigning numbers to the physical addresses of each physical block in the storage device to obtain a block mapping table;
[0042] In this embodiment, it is assumed that the storage device includes N physical blocks, which are respectively denoted as physical block 0, physical block 1, ..., physical block N-1 ; Physical block 0 ~ physical block N-1 The physical addresses are recorded as A0, A1, ..., A N-1 The numbering rule is set based on actual needs, for example, a number is randomly assigned to the physical address of each physical block, and the numbers corresponding to the physical addresses of different physical blocks are different.
[0043] Furthermore, in one embodiment, the earlier the physical block is arranged in the storage device, the smaller its number is.
[0044] In this embodiment, it is assumed that the physical arrangement order of the physical blocks in the storage device is: physical block 0, physical block 1, ..., physical block N-1 ; then the number 0 can be assigned to A0, the number 1 can be assigned to A1, the number 2 can be assigned to A2, and so on. Referring to Table 1, Table 1 is a schematic diagram of a block mapping table in an embodiment.
[0045]
[0046]
[0047] Table 1
[0048] Step S20, migrating data in the source physical block to a target physical block, wherein the target physical block is a physical block that does not store data;
[0049] In this embodiment, the source physical block is a physical block storing source data to be migrated, and the source physical block can be specified based on manual operation. When there are multiple physical blocks without storing data in the storage device, one can be randomly selected as the target physical block.
[0050] Furthermore, in one embodiment, the source physical block is a physical block storing cold data.
[0051] In this embodiment, if the physical block i If the data stored in the physical block is not updated for a long time (exceeding the preset time) and is valid data, then i The data stored in is cold data, so the physical block can be determined i is the source physical block.
[0052] Further, in one embodiment, the storage medium of the storage device is a flash memory, and step S20 includes:
[0053] The data in the source physical block is migrated to the target physical block through the Copy Back function of the flash memory.
[0054] In this embodiment, after determining the target physical block when there is a source physical block, the data in the source physical block can be migrated to the target physical block. Wherein, the storage medium of the storage device is flash memory NAND Flash, on this basis, the data in the source physical block can be migrated to the target physical block through the Copy Back function of NAND Flash. Wherein, the Copy Back function of NAND Flash is an operation method for optimizing data management and improving storage performance. In NAND Flash memory, data is usually programmed and read in units of pages, while erase operations are performed in units of blocks. Copy Back technology allows data to be copied directly from one page to another without first reading it into the main memory (such as DRAM) and then writing it back to the new location. This process reduces unnecessary read and write operations, saving time and energy consumption.
[0055] In this embodiment, the Copy Back method is reasonably used to move cold data, reducing the intervention of the controller and the use of cache. The block mapping table is used, which can quickly find the physical address after the move without interfering with the original LBA mapping table. It can be seen that this method can complete data migration with extremely low computing and cache overhead, thereby effectively improving the efficiency of data persistence.
[0056] Step S30, after the data migration is completed, verify the data in the target physical block;
[0057] In this embodiment, after the data is migrated from the source physical block to the target physical block, it is necessary to further verify whether this migration action is wrong. For example, the data in the source physical block is recorded as D, and the data in the target physical block after the data migration is completed is recorded as D'. This verification can be to compare whether D and D' are consistent. If they are consistent, the verification is successful; if they are inconsistent, the verification fails. Among them, whether D and D' are consistent can be determined by comparing whether the hash values of D and D' are equal.
[0058] Further, in one embodiment, verifying the data in the target physical block includes:
[0059] Sending a read instruction to the target physical block, wherein the read instruction is a data bypass mode, so as to read the data in the target physical block to the error correction unit of the controller, and detecting whether the read data contains invalid data based on the error correction unit;
[0060] If there is invalid data, it is determined that the verification has failed;
[0061] If there is no invalid data, the verification is determined to be successful.
[0062] In this embodiment, after the data is migrated from the source physical block to the target physical block, the controller (which can serve as the executor of the data migration method) sends a read instruction to the target physical block. The read instruction is a data bypass mode, that is, the data is read from the target physical block to the error correction unit of the controller, but not read into the SSD cache. The error correction unit then detects whether the read data contains invalid data; if invalid data exists, it is determined that the verification has failed; if invalid data does not exist, it is determined that the verification has succeeded.
[0063] In this embodiment, the migration action is verified and no additional cache is occupied.
[0064] Step S40: if the verification is successful, the positions of the physical address of the source physical block and the physical address of the target physical block in the block mapping table are exchanged.
[0065] In this embodiment, refer to Figure 2 , Figure 2 This is a schematic diagram of a data migration scenario in accordance with an embodiment of the data migration method of this application. Figure 2 As shown, physical block 2 is the source physical block, and physical block N-2 is the target physical block. After the cold data in physical block 2 is successfully migrated (meaning the migration is completed and the verification is successful) to physical block N-2, the physical address of physical block 2 and the physical address of physical block N-2 are exchanged in the block mapping table. The updated block mapping table is shown in Table 2:
[0066] serial number Physical Address 0 <![CDATA[A0]]> 1 <![CDATA[A1]]> 2 <![CDATA[A N-2 ]]> ...... ...... N-2 <![CDATA[A2]]> N-1 <![CDATA[A N-1 ]]>
[0067] Table 2
[0068] In the embodiment of the present application, the physical address of each physical block in the storage device is assigned a number to obtain a block mapping table; the data in the source physical block is migrated to the target physical block, and the target physical block is a physical block that does not store data; after the data migration is completed, the data in the target physical block is verified; if the verification is successful, the position of the physical address of the source physical block and the physical address of the target physical block in the block mapping table is exchanged. Through the embodiment of the present application, by establishing a specific block mapping table, only the block mapping table is updated after the data migration, and the amount of updated data is small, thereby completing the data migration with extremely low calculation and cache overhead.
[0069] Furthermore, in one embodiment, after step S30, the method further includes:
[0070] If the verification fails, the data in the target physical block is deleted, and the process returns to migrate the data in the source physical block to the target physical block.
[0071] In this embodiment, if the verification fails, that is, after the data migration is completed, the data in the target physical block is erroneous, the data migration action needs to be performed again. First, the data in the target physical block is deleted, and then the step S20 is returned to be executed until the migration is completed and the verification is successful.
[0072] Furthermore, in one embodiment, after step S40, the method further includes:
[0073] Delete the data in the source physical block.
[0074] In this embodiment, after the data in the source physical block has been migrated to the target physical block and verified successfully, the data in the source physical block does not need to be retained any more and is deleted, thereby saving storage device space.
[0075] In a second aspect, an embodiment of the present application also provides a data migration device.
[0076] In one embodiment, referring to Figure 3 , Figure 3 This is a functional module diagram of an embodiment of the data migration device of the present application. Figure 3 As shown, the data migration device includes:
[0077] Establishing module 10, for assigning numbers to the physical addresses of each physical block in the storage device to obtain a block mapping table;
[0078] A migration module 20, configured to migrate data in a source physical block to a target physical block, wherein the target physical block is a physical block that does not store data;
[0079] A verification module 30 is used to verify the data in the target physical block after the data migration is completed;
[0080] The update module 40 is used to exchange the positions of the physical address of the source physical block and the physical address of the target physical block in the block mapping table if the verification is successful.
[0081] Furthermore, in one embodiment, the earlier the physical block is arranged in the storage device, the smaller its number is.
[0082] Furthermore, in one embodiment, the source physical block is a physical block storing cold data.
[0083] Furthermore, in one embodiment, the storage medium of the storage device is a flash memory, and the migration module 20 is specifically used for:
[0084] The data in the source physical block is migrated to the target physical block through the Copy Back function of the flash memory.
[0085] Furthermore, in one embodiment, the verification module 30 is used to:
[0086] Sending a read instruction to the target physical block, wherein the read instruction is a data bypass mode, so as to read the data in the target physical block to the error correction unit of the controller, and detecting whether the read data contains invalid data based on the error correction unit;
[0087] If there is invalid data, it is determined that the verification has failed;
[0088] If there is no invalid data, the verification is determined to be successful.
[0089] Furthermore, in one embodiment, the data migration device further includes a re-execution module, which is used to:
[0090] If the verification fails, the data in the target physical block is deleted, and the migration module 20 is notified to perform migration of the data in the source physical block to the target physical block.
[0091] Furthermore, in one embodiment, the data migration device further includes a deletion module, which is used to:
[0092] Delete the data in the source physical block.
[0093] Among them, the functional implementation of each module in the above-mentioned data migration device corresponds to each step in the above-mentioned data migration method embodiment, and its functions and implementation processes are no longer repeated here.
[0094] In a third aspect, an embodiment of the present application provides a data migration device, which may be a personal computer (PC), a laptop computer, a server, or other device having a data processing function.
[0095] Reference Figure 4 , Figure 4 Schematic diagram of the hardware structure of the data migration device involved in the embodiment of the present application. In the embodiment of the present application, the data migration device may include a processor, a memory, a communication interface and a communication bus.
[0096] The communication bus may be of any type and is used to interconnect the processor, the memory, and the communication interface.
[0097] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces, which are used to interconnect devices within the data migration device, and interfaces used to interconnect the data migration device with other devices (such as other computing devices or user devices). The physical interface can be an Ethernet interface, a fiber optic interface, an ATM interface, etc.; the user device can be a display, a keyboard, etc.
[0098] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.
[0099] The processor may be a general-purpose processor, and the general-purpose processor may call the data migration program stored in the memory and execute the data migration method provided in the embodiment of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the data migration program is called may refer to the various embodiments of the data migration method of the present application, which will not be repeated here.
[0100] Those skilled in the art will understand that Figure 4 The hardware structure shown in the figure does not constitute a limitation on the present application, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0101] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium.
[0102] A data migration program is stored on the computer-readable storage medium of the present application, wherein when the data migration program is executed by a processor, the steps of the data migration method described above are implemented.
[0103] Among them, the method implemented when the data migration program is executed can refer to the various embodiments of the data migration method of the present application, and will not be repeated here.
[0104] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.
[0105] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit "first", "second" and "third" to different types.
[0106] In the description of the embodiments of the present application, "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a specific way.
[0107] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; the “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.
[0108] In some processes described in the embodiments of the present application, multiple operations or steps that appear in a specific order are included, but it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or in parallel, and the sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.
[0109] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.
[0110] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A data migration method, characterized in that: The data migration method comprises: Assign numbers to the physical addresses of each physical block in the storage device to obtain a block mapping table; Migrating data in a source physical block to a target physical block, where the target physical block is a physical block that does not store data; After the data migration is completed, the data in the target physical block is verified; If the verification is successful, the positions of the physical address of the source physical block and the physical address of the target physical block in the block mapping table are exchanged.
2. The data migration method according to claim 1, characterized in that: The earlier the physical block is in the physical arrangement order of the storage device, the smaller its number is.
3. The data migration method according to claim 1, characterized in that: The source physical block is a physical block in which the stored data is cold data.
4. The data migration method according to claim 1, characterized in that: The storage medium of the storage device is a flash memory, and the migrating of data in the source physical block to the target physical block includes: The data in the source physical block is migrated to the target physical block through the Copy Back function of the flash memory.
5. The data migration method according to claim 1, characterized in that: Verifying the data in the target physical block includes: Sending a read instruction to the target physical block, wherein the read instruction is a data bypass mode, so as to read the data in the target physical block to the error correction unit of the controller, and detecting whether the read data contains invalid data based on the error correction unit; If there is invalid data, it is determined that the verification has failed; If there is no invalid data, the verification is determined to be successful.
6. The data migration method according to claim 1, characterized in that: After verifying the data in the target physical block, it also includes: If the verification fails, the data in the target physical block is deleted, and the process returns to migrate the data in the source physical block to the target physical block.
7. The data migration method according to claim 1, characterized in that: After exchanging the positions of the physical address of the source physical block and the physical address of the target physical block in the block mapping table if the verification is successful, the method further includes: Delete the data in the source physical block.
8. A data migration device, characterized in that: The data migration device comprises: Establish a module for assigning numbers to the physical addresses of each physical block in the storage device to obtain a block mapping table; A migration module, used to migrate data in a source physical block to a target physical block, where the target physical block is a physical block that does not store data; A verification module is used to verify the data in the target physical block after the data migration is completed; The update module is used for exchanging the positions of the physical address of the source physical block and the physical address of the target physical block in the block mapping table if the verification is successful.
9. A data migration device, characterized in that: The data migration device includes a processor, a memory, and a data migration program stored in the memory and executable by the processor, wherein when the data migration program is executed by the processor, the steps of the data migration method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a data migration program, wherein when the data migration program is executed by a processor, the steps of the data migration method according to any one of claims 1 to 7 are implemented.