Method, device, storage medium and electronic equipment for processing data migration

Abstract

Embodiments of the present application provide a method and device for processing data migration, a storage medium and an electronic device, wherein the method comprises: obtaining a threshold value for triggering data migration; performing a full scan on a solid state disk to determine all valid super blocks in the solid state disk; comparing the remaining data retention time corresponding to the super blocks with the threshold value, and determining the super blocks with the remaining data retention time less than the threshold value as target super blocks; and triggering data migration of the target super blocks. Through the present application, data pile-up migration is avoided, data congestion is avoided, normal use of the SSD is ensured, and technical problems such as data migration strategy in the related art easily leading to data congestion, affecting the use performance of the SSD, and response timeout are solved.

Description

Technical Field

[0001] This application relates to the field of data processing, and more specifically, to a method and apparatus. Background Technology

[0002] In related technologies, the typical processing strategy in solid-state drives (SSDs) is to scan the data retention time of all superblocks and directly trigger a forced migration strategy for those that meet the migration conditions. However, if the data on the SSD all reaches the conditions to trigger migration within a short period of time, it will cause a concentrated data migration, thus forming a data migration storm. This can easily cause data congestion, severely affect the performance of the SSD, and even cause timeout issues. Summary of the Invention

[0003] This application provides a method, apparatus, storage medium, and electronic device for processing data migration, to at least solve the technical problems in the related art, such as data migration strategies easily leading to data blockage, affecting the performance of SSDs, and response timeouts.

[0004] According to one embodiment of this application, a method for processing data migration is provided, comprising: obtaining a pre-set threshold value for triggering data migration, wherein the threshold value is less than the data retention time, the data retention time being the difference between the first moment corresponding to the data being stored in the first page of the solid-state drive and the target moment; performing a full scan of the solid-state drive to identify all valid superblocks in the solid-state drive; comparing the remaining data retention time corresponding to the superblock with the threshold value, and identifying the superblock whose remaining data retention time is less than the threshold value as the target superblock; and triggering data migration of the target superblock.

[0005] In one exemplary embodiment, a full disk scan of the solid-state drive includes: determining a target period as the ratio of a threshold value to the number of all valid superblocks, and performing a full disk scan of the solid-state drive at target period intervals.

[0006] In an exemplary embodiment, after triggering a data migration to a target superblock, the method further includes: obtaining the number of targets corresponding to the triggered target superblock, comparing the number of targets with a predetermined value, and determining whether to update the target period based on the comparison result.

[0007] In an exemplary embodiment, determining whether to update the target period based on the comparison result includes: keeping the target period unchanged if the number of targets is less than a predetermined value; and updating the target period if the number of targets is greater than the predetermined value, wherein the updated target period is a first preset multiple of the original target period, and the first preset multiple is greater than 1.

[0008] In an exemplary embodiment, the method further includes: if it is determined that the target period has been updated, then performing a second update on the updated target period, wherein the target period after the second update is a second preset multiple of the original target period, and the second preset multiple is greater than a first preset multiple.

[0009] In one exemplary embodiment, the method further includes: after updating the target period, detecting whether the number of target superblocks to be forced data transfer is less than a predetermined value; if the number of target superblocks is less than the predetermined value, restoring the target period to its original value.

[0010] In one exemplary embodiment, the threshold value is the product of the data retention time and a preset ratio, wherein the preset ratio is less than 1.

[0011] In an exemplary embodiment, the method further includes: collecting superblocks in the target superblock whose data retention time meets preset requirements at preset intervals, wherein meeting the preset requirements includes: the data retention time corresponding to each target superblock is sorted from largest to smallest and is located in the top N positions of the sorting result; determining the maximum data retention time among the collected superblocks whose data retention time meets the preset requirements, and if the maximum data retention time is greater than a threshold value, then directly triggering data migration.

[0012] According to another embodiment of this application, an apparatus for processing data migration is also provided, comprising: an acquisition module, configured to acquire a pre-set threshold value for triggering data migration, wherein the threshold value is less than a data retention time, and the data retention time is the difference between a first moment corresponding to the storage of the first page in the solid-state drive and a target moment; a scanning module, configured to perform a full scan of the solid-state drive to determine all valid superblocks in the solid-state drive; a comparison module, configured to compare the remaining data retention time corresponding to the superblock with the threshold value, and determine the superblock whose remaining data retention time is less than the threshold value as the target superblock; and a triggering module, configured to trigger data migration of the target superblock.

[0013] According to another embodiment of this application, a computer-readable storage medium is also provided, in which a computer program is stored, wherein the computer program, when executed by a processor, implements any method for processing data transfer.

[0014] According to another embodiment of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any method for processing data transfer.

[0015] In the embodiments of this application, by setting a threshold value for triggering data migration, i.e. the time for triggering data migration, data migration is triggered in advance when the data retention time of the superblock reaches the trigger time. By avoiding data cluster migration, data blockage is avoided, the normal use of SSD is guaranteed, and the technical problems such as data migration strategies in related technologies that easily lead to data blockage, affect the performance of SSD, and cause response timeouts are solved. Attached Figure Description

[0016] Figure 1 This is a flowchart of a data transfer method according to an embodiment of this application;

[0017] Figure 2 This is a schematic diagram of the data transfer process in one embodiment of this application;

[0018] Figure 3 This is a schematic diagram of a data transfer processing apparatus according to another embodiment of this application;

[0019] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0020] The embodiments of this application will be described in detail below with reference to the accompanying drawings and examples.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0022] To facilitate a better understanding of the embodiments of this application by those skilled in the art, the technical terms or some nouns that may be involved in this application are explained as follows:

[0023] 1. The superblock is the beginning of a hard disk partition—the first byte is byte 0, and the data starts from byte 1024 and continues. Since the minimum block size is 1024 bytes, the superblock could be in block 1 (where the block size is exactly 1024 bytes) or block 0. The data in the superblock is essentially the control information for the file volume; it can also be considered the volume resource table, storing most of the information about the file volume. For example, it contains the size of each block in the hard disk partition, the total number of block groups on the partition, and the number of inodes in each block group.

[0024] 2. SSD Cells, Pages, and Blocks. There are two important components in an SSD: cells and the controller. We'll discuss the controller later; let's first talk about the storage units. Today's mainstream SSDs use NAND flash memory (although newer SSDs use 3D NAND) to store data. Each cell can store 1 bit (SLC), 2 bits (MLC), 3 bits (TLC), or even 4 bits (QLC) of data. The more bits a cell stores, the higher its density and the lower its manufacturing cost, but the lower its durability (or lifespan, number of write cycles). The smallest unit for reading or writing in an SSD is not a cell, but a page composed of a group of cells, typically 4KB in size. An important characteristic of SSDs is that once a cell has been written to, it cannot be overwritten, unlike magnetic hard drives.

[0025] 3. NAND flash memory is a type of flash memory that uses a non-linear macrocell model internally, providing a cost-effective solution for realizing large-capacity solid-state memory. NAND flash memory has advantages such as large capacity and fast rewrite speed, making it suitable for storing large amounts of data. Therefore, it has been increasingly widely used in the industry, such as in embedded products including digital cameras, MP3 player memory cards, and compact USB flash drives. NAND flash data is stored in memory cells in bits. Generally, one cell can only store one bit. These cells are connected in units of 8 or 16 to form a bit line, forming what is called a byte (x8) / word (x16), which is the bit width of a NAND device.

[0026] With the development and widespread application of technologies such as the Internet, cloud computing, and the Internet of Things, massive amounts of data are constantly generated in human life that require processing and storage. The rapid development of information technology places higher demands on the performance of storage systems. Solid-state drives (SSDs) are widely used due to their fast read / write speeds and low power consumption. However, due to the inherent physical properties of NAND flash memory, after user data is written to NAND, charge overflow can occur within the cells over time. Therefore, data retention time is crucial; after this period, valid data must be moved and reclaimed, otherwise, there is a high probability of error correction failure.

[0027] Currently, the common strategy in SSDs is to scan the data retention time of all superblocks and directly trigger a forced migration strategy for those that meet the migration conditions. However, if the data on the SSD all reaches the conditions that trigger migration within a short period of time, it will cause a concentrated data migration, thus forming a data migration storm. This will seriously affect performance, affect normal use, and may even cause SSD timeout issues.

[0028] To address the aforementioned technical problems, this embodiment provides a method for processing data migration on a solid-state drive. Figure 1 This is a flowchart of a data transfer method according to an embodiment of this application, such as... Figure 1 As shown, the process includes the following steps:

[0029] S102, obtain the pre-set threshold value for triggering data migration, wherein the threshold value is less than the data retention time, and the data retention time is the difference between the first moment when the data is stored in the first page of the solid-state drive and the target moment;

[0030] Optionally, the preset threshold for triggering data migration is the product of the data retention period and a preset ratio, where the preset ratio is less than 1. For example, if the data retention period is 30 days, then the preset threshold for triggering data migration is 30. 10% = 3 days.

[0031] It should be noted that in order to improve write efficiency, SSDs generally use RAID striping. Therefore, when a migration is triggered, all block data in the entire stripe needs to be migrated. When the proportion of valid data in the superblock is very high, the amount of data to be migrated is relatively large.

[0032] Considering that the actual data retention time varies across different granularities, a pre-set time can be used as a trigger condition. This reserved time can effectively mitigate the impact of concentrated data migration. Through the technical solution in step S102, a pre-set threshold for triggering data migration can be obtained. When the remaining data retention time is less than the threshold, data migration is triggered a certain time in advance, reducing the frequency and amount of concentrated data migration.

[0033] S104 performs a full scan of the SSD to identify all valid superblocks on the SSD.

[0034] Optionally, the aforementioned solid-state drives include different types such as SATA interface type, PCIe interface type, and M.2 interface type. Among them, SATA interface solid-state drives are slightly slower than the other two types in terms of transfer speed, but have larger capacity and relatively lower price; PCIe interface solid-state drives can achieve faster read and write speeds and have higher working efficiency, making them suitable for high-performance computing; while M.2 interface type is a new type of solid-state drive introduced in recent years, which is small in size and highly portable.

[0035] S106, compare the remaining data retention time of the superblock with the threshold value, and determine the superblock whose remaining data retention time is less than the threshold value as the target superblock;

[0036] The technical solution in step S106 allows for the selection of target superblocks whose remaining retention time is less than a pre-set threshold for triggering data migration, and then data migration processing is performed on these target superblocks. It should be noted that superblocks whose remaining retention time is less than the threshold can include superblocks whose remaining retention time is equal to the threshold. That is, the phrase "remaining retention time less than the threshold" includes superblocks whose remaining retention time is less than or equal to the threshold.

[0037] For example, if the preset threshold for triggering data migration is 30 days, and the remaining effective duration (i.e., remaining retention duration) of the data retention time for superblocks A, B, and C is 35 days, 30 days, and 28 days respectively, then superblocks B and C can be determined as the target superblocks (or simply target blocks) to be migrated.

[0038] S108 triggers a data migration to the target superblock.

[0039] Taking the preset threshold for triggering data migration as 30 days, and the remaining effective duration (i.e., remaining retention duration) of the data retention time for superblocks A, B, and C as 35 days, 30 days, and 28 days respectively, it can be determined that superblocks B and C are the target superblocks to be migrated, and data migration of target superblocks B and C can be triggered.

[0040] In the above-described data migration method, a threshold value is set to trigger data migration, i.e., the time for triggering data migration. When the data retention time of the superblock reaches the trigger time, data migration is triggered in advance. This is achieved by obtaining the pre-set threshold value, where the threshold value is less than the data retention time, which is the difference between the first moment when the data is stored in the first page of the SSD and the target moment. A full scan of the SSD is performed to identify all valid superblocks. The remaining data retention time of the superblock is compared with the threshold value, and the superblock with a remaining data retention time less than the threshold value is identified as the target superblock. Data migration to the target superblock is then triggered, avoiding data clustering and data congestion, ensuring the normal use of the SSD, and solving the technical problems of data migration strategies in related technologies that easily lead to data congestion, affecting SSD performance, and causing response timeouts.

[0041] In an optional embodiment of this application, before performing a full scan of the solid-state drive (SSD), the scan cycle can be determined as follows: The target cycle is determined by the ratio of a threshold value to the number of all valid superblocks, and a full scan of the SSD is performed at intervals equal to the target cycle. For example, if the threshold value is 30 days and the number of all valid superblocks is 300, then the target cycle can be determined as 30 / 300 = 0.1 days. Then, a full scan of the SSD is performed every 0.1 days (i.e., every 2.4 hours).

[0042] As another alternative implementation method, the above-mentioned full-disk scan of the solid-state drive can be achieved in the following way:

[0043] 1. Open a command line in the terminal that contains the SSD and log in to the system with root administrator privileges.

[0044] 2. Use the following command to check the file system: sudo fsck -c / dev / sdb. In this example, " / dev / sdb" is the solid-state drive code name. You need to replace it with the actual device code.

[0045] 3. After executing this command, the program will scan the entire disk and look for any superblock errors or corruption. This process may take a considerable amount of time to complete. However, any errors found will be logged in the log file.

[0046] 4. Alternatively, for a faster and more comprehensive method to scan and test superblocks, the "badblocks" Linux tool can be used to perform manual expansion by comparing the logical location (LBA) address region of the running data store written by the controller with the tag bit array to verify its validity.

[0047] 5. Execute the following command: `sudo badblocks -vsnw / dev / sdb 1000 > badblock_result.txt`, where `-v` indicates verbose output, `-s` displays a progress bar for each iteration, `-N` pre-searches 500M-1G, `-n` specifies skipped regions, `-i` sets the tolerance number to allow inconsistency, `-w` runs the read revision algorithm to force uniqueness, and `-R` repeats the intelligent run, saving the scan results in a text file named "badblock_result.txt".

[0048] 6. Finally, analyze the logs and badblocks detection data to determine which superblocks are valid if they exceed the specified safety percentage. If the problem is limited to a specific area, you can narrow down the test scope using commands—for example, `sudofdisk / dev / sdb` will prune a controllable block to capture specific abnormal operation types.

[0049] Understandably, even though the technical solution of this application sets a threshold value for triggering data migration to ensure that the target blocks that need to be migrated are migrated a certain amount of time in advance, there may still be a large number of superblocks that need to be migrated in the advance time, that is, the number of target blocks is large. In this case, even if an advance time is set, data blockage may still occur. To avoid this situation, in an exemplary embodiment, after triggering the data migration of the target superblock, the number of targets corresponding to the triggered target superblock can be obtained, the number of targets can be compared with a predetermined value, and it can be determined whether to update the target period based on the comparison result. Understandably, the shorter the scanning period, the more target blocks that need to be processed may be scanned. By updating the target period, the number of target blocks scanned can be adjusted appropriately, further avoiding the impact on the performance of SSD during data migration.

[0050] Specifically, the determination of whether to update the target period based on the comparison results includes: if the number of targets is less than a predetermined value, then the target period remains unchanged; if the number of targets is greater than a predetermined value, then the target period is updated, wherein the updated target period is a first preset multiple of the original target period, and the first preset multiple is greater than 1.

[0051] For example, if the predetermined value is 25 and the target cycle is 2.4 hours, then when the target number is 20, the 2.4-hour scan cycle remains unchanged. When the target number is 30, the target cycle can be adjusted according to a first preset multiplier. Assuming the first preset multiplier is 1.5, the updated target cycle, i.e., the scan cycle in the next round, will be 2.4 hours. 1.5 = 3.6h. It's understandable that the shorter the scanning cycle, the more target blocks that need processing will be scanned. The above technical solution can lengthen the target cycle used in the next scan, thereby appropriately adjusting the number of target blocks scanned.

[0052] Since a single update may not achieve the desired effect, and the amount of data to be moved from the SSD remains substantial, as an optional implementation, if it is determined that the target period has already been updated, a second update is performed on the updated target period. The target period after the second update is a second preset multiple of the original target period, where the second preset multiple is greater than the first preset multiple. For example, if the second preset multiple is 2, then the target period after the second update is 2.4. 2 = 4.8h.

[0053] It is worth noting that the above target period can be adjusted multiple times according to the actual situation. For example, the target period Third preset multiple, target period Fourth preset multiple, target period The fifth preset multiple, wherein the third preset multiple is greater than the second preset multiple, and the third preset multiple, the fourth preset multiple, and the fifth preset multiple increase sequentially. For example, the third preset multiple, the fourth preset multiple, and the fifth preset multiple can be 2.5, 3, and 3.5, respectively. It should be noted that the above multiples are illustrative examples and do not constitute a limitation on the numerical range of each preset multiple in this application. For example, the first preset multiple, the second preset multiple, the third preset multiple, the fourth preset multiple, and the fifth preset multiple can also be other values ​​such as 2.1, 2.2, 2.3, 2.4, and 2.5, respectively.

[0054] To ensure efficient data transfer and avoid resource idleness, in one exemplary embodiment, after updating the target cycle, it is detected whether the number of target blocks to be forcibly transferred is less than a predetermined value; if the number of target blocks is less than the predetermined value, the target cycle is restored to its original value. It is understood that when the number of target blocks is less than the predetermined value, the amount of data to be transferred by the SSD is reduced because the number of target blocks to be moved is smaller, which may lead to idle processing resources. Therefore, the previously increased scan cycle can be restored to the previously unadjusted target cycle, or even a smaller target cycle.

[0055] To ensure that the data does not exceed the maximum data retention time threshold, in an exemplary embodiment, superblocks within the target superblock whose data retention time meets preset requirements can be collected at preset intervals. Meeting the preset requirements includes: the data retention time corresponding to each target superblock is sorted from largest to smallest and is located in the top N positions of the sorting result; the maximum data retention time among the collected superblocks that meet the preset requirements is determined, and if the maximum data retention time exceeds the threshold, data migration is directly triggered. The value of N can be 4.

[0056] Figure 2 This is a schematic diagram of the data transfer process in one embodiment of this application, which is illustrated below. Figure 2 This embodiment will be described in detail, and the specific implementation steps are as follows:

[0057] 1. Predefine the time PRE_MOVE_DELAY for triggering data migration (i.e., the pre-set threshold for triggering data migration), which can be 10% of the upper limit of data retention time.

[0058] 2. Pre-set the maximum number of forced data migrations that have been triggered but not yet completed within the SSD disk to Movedata_Limit.

[0059] 3. Define a timer T0 with a detection period of PRE_MOVE_DELAY, and scan all valid superblocks on the entire disk at equal time intervals T1.

[0060] Note: T1 = T0 / SUPER_BLK_NUM (i.e., the target period).

[0061] 4. When the data hold time is detected to enter the final PRE_MOVE_DELAY interval, a request message for forced data migration of the superblock is sent to update the number of forced data migrations that have been triggered but not yet completed.

[0062] 5. A threshold value for triggering data migration is set to ensure that target blocks requiring data migration are migrated a certain amount of time in advance. However, within this advance time, there may be a large number of superblocks that need data migration, meaning there are many target blocks. This could potentially cause data congestion. To avoid this, after triggering data migration of the target superblock, the number of targets corresponding to the triggered target superblock can be obtained and compared with a predetermined value. Based on the comparison result, it is determined whether to update the target cycle. It is understandable that the shorter the scanning cycle, the more target blocks that need to be processed will be scanned. By updating the target cycle, the number of target blocks scanned can be appropriately adjusted to further avoid impacting the performance of the SSD during data migration. Therefore, it is also necessary to determine whether the number of triggered data migrations is less than the predetermined value.

[0063] 1) If so, then keep the detection superblock period unchanged (i.e., if the number of targets is less than the predetermined value, then keep the target period unchanged).

[0064] 2) If not, adjust the triggering condition, that is, extend the detection period of the next round to 1.5 times T1 (that is, if the number of targets is greater than the predetermined value, the target period will be updated, where the updated target period is the first preset multiple of the original target period, where the first preset multiple is greater than 1).

[0065] 3) If the detection period has been adjusted, the detection period needs to be extended by 2 times T1. (It should be noted that although an update has been performed, the expected effect may not be achieved and the amount of data to be moved from the SSD is still large. Therefore, as an optional implementation method, if it is determined that the target period has been updated, the updated target period is updated a second time. The target period after the second update is the second preset multiple of the original target period, where the second preset multiple is greater than the first preset multiple.)

[0066] 6. Monitor the completion status of forced data migration, that is, when the number of superblocks to be forced data migration is detected to decrease to a preset threshold, restore the initial detection period T1. (That is, to avoid resource idleness, in an exemplary embodiment, after updating the target period, it is detected whether the number of target blocks to be forced data migration is less than a predetermined value; if the number of target blocks is less than the predetermined value, the target period is restored to the original value).

[0067] 7. Considering the dynamically adjusted detection cycle, some superblocks may fall outside the upper limit of the data retention time supported by the SSD. Based on the search acceleration unit, the four superblocks with the largest time RT values ​​in the superblock info are retrieved:

[0068] Set timer T2, which means that every hour, scan the four blocks with the largest RT. If the largest one exceeds the data retention threshold, a forced migration will be triggered directly. (That is, to ensure that the data does not exceed the maximum data retention time threshold, superblocks in the target superblock that meet the preset requirements for data retention time can be collected at preset intervals. Meeting the preset requirements includes: the data retention time of each target superblock is sorted from largest to smallest and is located in the top N positions of the sorting result; the maximum data retention time among the collected superblocks that meet the preset requirements is determined, and if the maximum data retention time is greater than the threshold value, a data migration will be triggered directly.)

[0069] Figure 3 This is an apparatus for processing data transfer according to another embodiment of this application, such as... Figure 3 As shown, the device includes:

[0070] The acquisition module 30 is used to acquire a pre-set threshold value for triggering data migration, wherein the threshold value is less than the data retention time, and the data retention time is the difference between the first moment when the data is stored in the first page of the solid-state drive and the target moment.

[0071] Scan module 32 is used to perform a full scan of the solid-state drive and identify all valid superblocks in the solid-state drive;

[0072] The comparison module 34 is used to compare the remaining data retention time corresponding to the superblock with the threshold value, and to determine the superblock whose remaining data retention time is less than the threshold value as the target superblock;

[0073] Trigger module 36 is used to trigger the data migration of the target superblock.

[0074] In this data migration processing device, the acquisition module 30 is used to acquire a pre-set threshold value for triggering data migration, wherein the threshold value is less than the data retention time, and the data retention time is the difference between the first moment corresponding to the data being stored in the first page of the solid-state drive and the target moment; the scanning module 32 is used to perform a full scan of the solid-state drive to identify all valid superblocks in the solid-state drive; the comparison module 34 is used to compare the remaining data retention time corresponding to the superblock with the threshold value, and to identify the superblock whose remaining data retention time is less than the threshold value as the target superblock; the triggering module 36 is used to trigger the data migration of the target superblock, avoiding data cluster migration, avoiding data blockage, ensuring the normal use of the SSD, and solving the technical problems of data migration strategies in related technologies that easily lead to data blockage, affecting the performance of the SSD, and response timeouts.

[0075] In some embodiments of this application, the scanning module includes a determining unit, configured to determine a target period as the ratio of a threshold value to the number of all valid superblocks, and to perform a full disk scan of the solid-state drive at target period intervals. For example, if the threshold value is 30 days and the number of all valid superblocks is 300, then the target period can be determined to be 30 / 300 = 0.1 days. Then, a full disk scan of the solid-state drive is performed every 0.1 days (i.e., every 2.4 hours).

[0076] As an optional embodiment, the above-mentioned device further includes: an update module, used to obtain the number of targets corresponding to the triggered target superblock after triggering data migration of the target superblock, compare the number of targets with a predetermined value, and determine whether to update the target cycle based on the comparison result. It is understood that the shorter the scanning cycle, the more target blocks that need to be processed may be scanned. By updating the target cycle, the number of target blocks scanned can be appropriately adjusted, further avoiding the impact on the performance of SSD during data migration.

[0077] Optionally, the update module includes: a holding unit, used to keep the target period unchanged when the number of targets is less than a predetermined value; and a first update unit, used to update the target period when the number of targets is greater than a predetermined value, wherein the updated target period is a first preset multiple of the original target period, wherein the first preset multiple is greater than 1.

[0078] For example, if the predetermined value is 25 and the target cycle is 2.4 hours, then when the target number is 20, the 2.4-hour scan cycle remains unchanged. When the target number is 30, the target cycle can be adjusted according to a first preset multiplier. Assuming the first preset multiplier is 1.5, the updated target cycle, i.e., the scan cycle in the next round, will be 2.4 hours. 1.5 = 3.6h. It's understandable that the shorter the scanning cycle, the more target blocks that need processing will be scanned. The above technical solution can extend the target cycle used in the next scan, allowing for an appropriate adjustment of the number of target blocks scanned.

[0079] Since a single update may not achieve the desired effect, and the amount of data to be moved from the SSD remains substantial, as an optional embodiment, the above apparatus further includes: a second update unit, configured to perform a second update on the updated target period if it is determined that the target period has already been updated. The target period after the second update is a second preset multiple of the original target period, wherein the second preset multiple is greater than a first preset multiple. For example, if the second preset multiple is 2, then the target period after the second update is 2.4. 2 = 4.8h.

[0080] It is worth noting that the above target period can be adjusted multiple times according to the actual situation. For example, the target period Third preset multiple, target period Fourth preset multiple, target period The fourth preset multiple, wherein the third preset multiple is greater than the second preset multiple, and the third, fourth, and fifth preset multiples increase sequentially. For example, the third, fourth, and fifth preset multiples can be 2.5, 3, and 3.5, respectively. It should be noted that the above multiples are illustrative examples and do not constitute a limitation on the numerical range of each preset multiple in this application. For example, the first, second, third, fourth, and fifth preset multiples can also be 2.1, 2.2, 2.3, 2.4, 2.5, and other values, respectively.

[0081] As an optional embodiment, the above apparatus further includes: a detection unit, configured to detect whether the number of target blocks to be forced data transfer is less than a predetermined value after updating the target period; and a recovery unit, configured to restore the target period to its original value if the number of target blocks is less than the predetermined value.

[0082] It should be noted that the threshold value is the product of the data retention time and the preset ratio, where the preset ratio is less than 1.

[0083] In some embodiments of this application, the above-mentioned device further includes: a data acquisition module, used to acquire superblocks in the target superblock whose data retention time meets preset requirements at preset intervals, wherein meeting the preset requirements includes: the data retention time corresponding to each target superblock is sorted from largest to smallest and is located in the top N positions of the sorting result; and a data transfer module, used to determine the maximum data retention time among the acquired superblocks whose data retention time meets the preset requirements, and if the maximum data retention time is greater than a threshold value, then data transfer is directly triggered.

[0084] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0085] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0086] According to another embodiment of this application, a computer-readable storage medium is also provided, in which a computer program is stored, wherein the computer program, when executed by a processor, implements any method for processing data transfer.

[0087] Specifically, the aforementioned storage medium is used to store program instructions for the following functions, which implement the following functions: obtaining a pre-set threshold value for triggering data migration, wherein the threshold value is less than the data retention time, and the data retention time is the difference between the first moment corresponding to the data being stored in the first page of the solid-state drive and the target moment; performing a full scan of the solid-state drive to identify all valid superblocks in the solid-state drive; comparing the remaining data retention time corresponding to the superblock with the threshold value, and identifying the superblock whose remaining data retention time is less than the threshold value as the target superblock; and triggering data migration of the target superblock.

[0088] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0089] According to another embodiment of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any method for processing data transfer.

[0090] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0091] Figure 4 A schematic block diagram of an example electronic device 400 that can be used to implement embodiments of this application is shown.

[0092] like Figure 4As shown, device 400 includes a computing unit 401, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 402 or a computer program loaded from storage unit 408 into random access memory (RAM) 403. RAM 403 may also store various programs and data required for the operation of device 400. The computing unit 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0093] Multiple components in device 400 are connected to I / O interface 405, including: input unit 406, such as keyboard, mouse, etc.; output unit 407, such as various types of monitors, speakers, etc.; storage unit 408, such as disk, optical disk, etc.; and communication unit 409, such as network card, modem, wireless transceiver, etc. Communication unit 409 allows device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0094] The computing unit 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as methods for processing data transfer. For example, in some embodiments, the methods for processing data transfer may be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the methods for processing data transfer described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform methods for processing data transfer by any other suitable means (e.g., by means of firmware).

[0095] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0096] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0097] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0098] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0099] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0100] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for handling data migration, characterized in that, include: Obtain a pre-set threshold value for triggering data migration, wherein the threshold value is less than the data retention time, and the data retention time is the difference between the first moment corresponding to the data being stored in the first page of the solid-state drive and the target moment; Perform a full scan of the solid-state drive to identify all valid superblocks within it; Compare the remaining data retention time corresponding to the superblock with the threshold value, and determine the superblock whose remaining data retention time is less than the threshold value as the target superblock; Trigger data migration of the target superblock; The process of performing a full scan of the solid-state drive includes: determining the ratio of the threshold value to the number of all valid superblocks as the target period, and performing a full scan of the solid-state drive at intervals of the target period. After triggering the data migration of the target superblock, the method further includes: obtaining the number of targets corresponding to the triggered target superblock, comparing the number of targets with a predetermined value, and determining whether to update the target period based on the comparison result.

2. The method according to claim 1, characterized in that, Determining whether to update the target period based on the comparison results includes: If the number of targets is less than the predetermined value, the target period remains unchanged; If the number of targets is greater than the predetermined value, the target period is updated, wherein the updated target period is a first preset multiple of the original target period, and the first preset multiple is greater than 1.

3. The method according to claim 2, characterized in that, The method further includes: If it is determined that the target period has been updated, then the updated target period is updated a second time, wherein the target period after the second update is a second preset multiple of the original target period, and the second preset multiple is greater than the first preset multiple.

4. The method according to claim 1, characterized in that, The method further includes: After updating the target cycle, it is checked whether the number of target superblocks to be forced data transfer is less than the predetermined value; If the number of target superblocks is less than the predetermined value, the target period will be restored to its original value.

5. The method according to claim 1, characterized in that, The threshold value is the product of the data retention time and a preset ratio, wherein the preset ratio is less than 1.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: The superblocks in the target superblocks whose data retention time meets the preset requirements are collected at preset intervals. The preset requirements include: the data retention time of each target superblock is sorted from largest to smallest and is located in the top N positions of the sorting result. The maximum data retention time among the data retention times corresponding to the superblocks that meet the preset requirements is determined. If the maximum data retention time is greater than the threshold value, data migration is directly triggered.

7. An apparatus for processing data transfer, characterized in that, include: The acquisition module is used to acquire a pre-set threshold value for triggering data migration, wherein the threshold value is less than the data retention time, and the data retention time is the difference between the first moment corresponding to the data being stored in the first page of the solid-state drive and the target moment. The scanning module is used to perform a full scan of the solid-state drive (SSD) to identify all valid superblocks in the SSD. The comparison module is used to compare the remaining data retention time corresponding to the superblock with the threshold value, and determine the superblock whose remaining data retention time is less than the threshold value as the target superblock; The triggering module is used to trigger the data migration of the target superblock; The process of performing a full scan of the solid-state drive includes: determining the ratio of the threshold value to the number of all valid superblocks as the target period, and performing a full scan of the solid-state drive at intervals of the target period. The process of triggering a data migration of the target superblock includes: obtaining the number of targets corresponding to the triggered target superblock, comparing the number of targets with a predetermined value, and determining whether to update the target period based on the comparison result.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the data transfer method as described in any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the data transfer method as described in any one of claims 1 to 6.

Images