A preprocessing method and storage device under a solid state disk hybrid test scene
By employing a preprocessing method of partitioning and random sampling estimation in a mixed testing scenario for solid-state drives, the problems of long preprocessing time and resource waste in large-capacity solid-state drives are solved, achieving the effects of rapid entry into steady state and efficient resource saving.
Patent Information
- Application Number
- CN202511256771.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-03
- Publication Date
- 2025-11-11
- Estimated Expiration
- 2045-09-03
AI Technical Summary
Existing technologies for preprocessing large-capacity solid-state drives are time-consuming and require too many NAND flash memory erases, resulting in resource waste and making it difficult to quickly reach a steady state for performance and functional testing.
A preprocessing method for mixed testing scenarios is adopted. By obtaining the proportion of valid user data across the entire disk, partitioning and performing random sampling estimation, and combining layer-by-layer partitioning and disk filling strategies, the disk filling strategy is dynamically adjusted to reduce preprocessing time and the number of erasures.
It significantly reduces preprocessing time and NAND flash memory erase cycles, improves resource utilization efficiency, is suitable for performance testing of large-capacity storage disks, adapts to SSDs under different usage conditions, and saves at least 40% of time and hardware wear.
Smart Images

Figure CN120743204B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of storage technology, and in particular to a preprocessing method and storage device for a hybrid testing scenario of solid-state drives. Background Technology
[0002] The storage industry has experienced rapid development and continuous technological innovation in recent years, with solid-state drives (SSDs) now widely used in enterprise servers, large data centers, and other fields. With the advent of the AI (Artificial Intelligence) era, AI applications such as deep learning and big data analytics require the processing of massive amounts of data, and the parameters of large AI models are becoming increasingly large, significantly increasing the performance, especially capacity, demands on storage devices. Currently, mainstream storage devices with capacities of 4TB, 8TB, and 16TB are insufficient to meet the ever-growing storage needs. The development of flash memory technologies such as TLC (Triple-Level Cell, storing 3 bits of data per cell) and QLC (Quad-Level Cell, storing 4 bits of data per cell) provides technological support for large-capacity SSDs. As SSD prices gradually decrease, the actual demand for large-capacity and even ultra-large-capacity SSDs such as 64TB and 128TB will continue to grow, and they will be widely used in more fields in the future.
[0003] Since steady state represents the optimal performance state of a solid-state drive (SSD) during normal use, it is necessary to preprocess the drive to reach or near a steady state before performance and functional tests. A basic requirement for steady state is that the entire drive capacity must be written at least once. However, the large capacity of a large-capacity SSD, combined with over-provisioning (OP), results in a very large amount of writing required for each preprocessing operation. Currently, the common preprocessing method is to first perform a full erase before performing a full write operation. This method is not only time-consuming but also excessively consumes the erase / programmable (P / E) cycles of NAND flash memory (i.e., SSD durability) due to its limited lifespan (e.g., 3000-10000 P / E cycles for MLC, 500-3000 for TLC, and 100-1000 for QLC), leading to resource waste.
[0004] How to preprocess large-capacity hard drives before performance and functional testing to enable them to quickly reach a steady state, while reducing the time consumed in the preprocessing process and the wear and tear from NAND flash memory erase cycles, and avoiding resource waste, has become an urgent technical problem to be solved. Summary of the Invention
[0005] To address the aforementioned technical problems, this invention provides a preprocessing method and storage device for mixed testing scenarios of solid-state drives, thereby solving the problems of large write volumes and long processing times caused by full disk writing in existing preprocessing methods.
[0006] In a first aspect, embodiments of the present invention provide a preprocessing method for a mixed testing scenario of a solid-state drive, the method comprising:
[0007] Obtain the percentage of valid user data across the entire system. If the percentage of valid user data across the entire system meets a first preset condition, then initiate a full-system preprocessing operation.
[0008] If the percentage of valid user data in the entire system meets the second preset condition, then a controllable preprocessing operation is initiated.
[0009] The controllable preprocessing operation includes:
[0010] The entire disk will be partitioned. Based on the number of partitions and the estimated percentage of valid user data in the entire disk, it will be determined whether to continue to the next level of partitioning until the estimated percentage of valid user data in the entire disk meets the third preset condition.
[0011] Scan each partition, fill the disk for partitions that meet the fourth preset condition, and determine whether to re-partition the entire disk based on the percentage of valid user data after filling and the set preprocessing ratio, until the set preprocessing ratio is reached.
[0012] In some embodiments of this disclosure, the second preset condition specifically includes:
[0013] The overall usage value is zero; or
[0014] The value of the overall usage is not zero, and the proportion of valid user data in the entire system is greater than the first preset threshold but less than the preprocessing ratio.
[0015] In some embodiments of this disclosure, the entire disk is partitioned, and the decision to proceed to the next level of partitioning is determined based on the number of partitions and the estimated percentage of valid user data across the entire disk, until the estimated percentage of valid user data across the entire disk meets a third preset condition, specifically including:
[0016] The entire disk's logical block address is divided into multiple first-level partitions, and each first-level partition is sequentially numbered.
[0017] Each primary partition is further subdivided into multiple secondary partitions, and each secondary partition is sequentially numbered.
[0018] Randomly sample the first-level partition and / or the second-level partition, and estimate the percentage of valid user data in the whole disk based on the percentage of valid user data in each randomly sampled first-level partition and / or second-level partition;
[0019] If the estimated percentage of valid user data across the entire disk meets the third preset condition, the partitioning is completed; if not, it is determined whether the number of secondary partitions has reached the maximum value. If not, it is returned and secondary partitioning is performed again; if so, the next level of partitioning continues until the estimated percentage of valid user data across the entire disk meets the third preset condition.
[0020] In some embodiments of this disclosure, random sampling is performed on the primary partition and / or secondary partition, and the proportion of valid user data in the entire disk is estimated based on the proportion of valid user data in each randomly sampled primary partition and / or secondary partition. Specifically, this includes:
[0021] Randomly select N primary partitions, calculate the percentage of valid user data in each selected primary partition, and then estimate the percentage of valid user data across the entire disk; or,
[0022] In each primary partition, N secondary partitions are randomly selected. The percentage of valid user data in each randomly selected secondary partition is calculated, and the percentage of valid user data in each randomly selected primary partition is estimated. Finally, the percentage of valid user data in the entire disk is estimated.
[0023] In some embodiments of this disclosure, the process of partitioning the entire disk specifically includes:
[0024] Divide the entire disk's logical block address into two primary partitions;
[0025] Then divide each of the two primary partitions into a different number of secondary partitions.
[0026] In some embodiments of this disclosure, scanning each partition specifically includes:
[0027] Each partition in each level of each first-level partition is indexed sequentially, with the index of the first partition at the lowest level of each first-level partition being 0.
[0028] Parallel operations are performed on the first-level partitions. For each first-level partition, the index is judged sequentially. The partition number corresponding to the current index that satisfies the fourth preset condition with zero user data is recorded in the test log.
[0029] In some embodiments of this disclosure, scanning each partition specifically includes:
[0030] Each partition at each level under each first-level partition is indexed sequentially, with the index of the first partition at the lowest level of the first first-level partition being 0.
[0031] The system will make sequential judgments according to the index order, and record the partition number corresponding to the current index that satisfies the fourth preset condition (zero percentage of user data) in the test log.
[0032] In some embodiments of this disclosure, the disk filling method includes sequential disk filling, parallel sequential disk filling, or merged partition sequential disk filling.
[0033] In some embodiments of this disclosure, the parallel sequential filling specifically includes:
[0034] The records in the acquired test logs are classified by their numbers according to the first-level partition. Within each category, the records are sorted in ascending order of their numbers. The disks are then filled in parallel sequentially at each level within each category, following the order of the sorting.
[0035] The sequential disk filling for merging partitions specifically includes:
[0036] Identify the partitions at each level that have an adjacent relationship based on the numbers recorded in the obtained test logs. Merge the partitions at each level that have an adjacent relationship into a new partition. The new partition uses the number of any partition before the merger. Arrange the numbers in ascending order. Fill the disk one by one for each partition at each level in the order of arrangement. The adjacent partitions to be merged contain consecutive free blocks.
[0037] Secondly, embodiments of this disclosure also provide a storage device, including: a memory and a processor, the processor including multiple cores, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the preprocessing method described above.
[0038] This disclosure provides a preprocessing method and storage device for mixed testing scenarios of solid-state drives. The technical solutions provided by the embodiments of this disclosure bring at least the following beneficial effects:
[0039] (1) Wide range of applications: This solution is suitable for performance and functional testing of large-capacity storage disks, and the larger the capacity, the more significant the benefits.
[0040] (2) High integration flexibility: This solution can be integrated into other automated testing processes, dynamically adjust the disk filling strategy according to the proportion of effective data, adapt to SSDs under different usage conditions, and achieve different proportions of disk filling effect through parameter configuration.
[0041] (3) Strong adaptability to scenarios: This scheme uses random sampling to estimate the effective LBA of the entire disk, which is suitable for situations where the state of the storage disk is completely unknown.
[0042] (4) High efficiency and resource saving: Taking a 128T SSD as an example, in a scenario with 50% effective data, each round of preprocessing can save at least 40% of the time and one P / E erase, reducing test time and hardware wear.
[0043] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description
[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0045] Figure 1 A schematic flowchart of a preprocessing method provided in an embodiment of this disclosure;
[0046] Figure 2 This is a schematic diagram of the overall process of a preprocessing method provided in an embodiment of the present disclosure;
[0047] Figure 3 A schematic diagram illustrating the process of performing full disk partitioning using the layer-by-layer partitioning method provided in this embodiment of the disclosure;
[0048] Figure 4 A flowchart illustrating one embodiment of scanning and filling provided in this disclosure;
[0049] Figure 5 A flowchart illustrating another implementation of scanning and filling provided in this disclosure;
[0050] Figure 6 A schematic flowchart of another preprocessing method provided in an embodiment of this disclosure;
[0051] Figure 7 This is a schematic diagram of the structure of a storage device provided in an embodiment of this disclosure. Detailed Implementation
[0052] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this disclosure.
[0053] Before describing the technical solution in this disclosure in detail, the technical terms used in the technical solution will be explained first:
[0054] SSD (Solid State Drive): A data storage device based on flash memory technology. It stores data through integrated circuits, replacing the mechanical rotating disks and read / write heads of traditional hard drives (HDDs). With its superior performance and reliability, SSD has gradually replaced HDD as the mainstream storage device, especially in scenarios where speed and stability are critical.
[0055] NCAP (Total Disk Capacity): This refers to the total physical storage capacity of a storage device (such as a hard drive or solid-state drive), usually measured in bytes (Byte), gigabytes (GB), or terabytes (TB). This parameter is the most basic indicator for measuring the storage capacity of a storage device, representing the theoretical maximum limit of data that the device can store. Users can use this indicator to understand how much data the device can hold.
[0056] NUSE (Disk Usage Status): This parameter describes the specific status of used and remaining available storage space on a storage device. It provides a clear picture of how well the storage device is being used. Through NUSE, users can understand the storage device's space usage and decide whether to clean up data, delete unnecessary files, or expand storage capacity.
[0057] LBA (Logical Block Address): In a storage system, LBA is a logical number for a storage unit, used to abstract the physical storage location. For example, an SSD divides the storage space into multiple logical blocks (typically 4KB or larger), and each block corresponds to a unique LBA.
[0058] Random pattern (random data format): This refers to the data written to the SSD being a randomly generated sequence of bytes, rather than fixed values (such as all 0s, all 1s, or continuously increasing data). For example, the pattern might be an irregular combination of bytes like 0xAB, 0x3C, or 0xF7. Random data patterns are used to simulate diverse data types in real-world scenarios, testing the SSD's performance, reliability, or error correction capabilities under different data characteristics.
[0059] Sequential Write: Data is written by accessing memory cells sequentially in ascending (or descending) order of LBAs, for example, writing from LBA1000 to LBA2000 without skipping any LBAs. The difference between sequential write and random write is that sequential writes access LBAs continuously, while random writes access LBAs discretely (e.g., writing to different LBAs in a skipping manner).
[0060] Test Log: The Test Log is a file that records all operations, parameters, and results during the testing of a storage device (SSD) for subsequent analysis, debugging, or verification.
[0061] Figure 1 This is a flowchart illustrating a preprocessing method for a hybrid testing scenario of a solid-state drive provided in this disclosure embodiment, as shown below. Figure 1 As shown, it includes the following steps:
[0062] S101. Obtain the percentage of valid user data across the entire system. If the percentage of valid user data across the entire system meets the first preset condition, then start the full system preprocessing operation.
[0063] S102. In response to the fact that the percentage of valid user data in the entire disk meets the second preset condition, a controllable preprocessing operation is initiated.
[0064] The controllable preprocessing operation includes:
[0065] The entire disk will be partitioned. Based on the number of partitions and the estimated percentage of valid user data on the entire disk, it will be determined whether to continue to the next level of partitioning until the third preset condition is met.
[0066] Scan each partition, fill the disk for partitions that meet the fourth preset condition, and determine whether to re-partition the entire disk based on the percentage of valid user data after filling and the set preprocessing ratio, until the set preprocessing ratio is reached.
[0067] Figure 2 This is a schematic diagram of the overall process of a preprocessing method for a hybrid testing scenario of a solid-state drive provided in this embodiment of the disclosure. The method generally includes the following steps:
[0068] S201. Calculate the percentage of valid user data across the entire SSD and determine whether to initiate full disk preprocessing or controlled preprocessing.
[0069] S202. In the controllable preprocessing operation, the entire disk is partitioned layer by layer, and the distribution model of valid LBAs (Logical Block Addresses) and holes (invalid LBAs) in the entire disk is statistically analyzed.
[0070] S203. Based on the estimated distribution model, determine the partitions to be filled and select an appropriate filling method for preprocessing (filling).
[0071] S204. Recheck whether the percentage of valid user data on the entire SSD meets the requirements (set preprocessing ratio). If it does not meet the requirements, repeat the controlled preprocessing operation until it meets the requirements.
[0072] The following sections respectively address... Figure 1 The steps described above will be explained in detail.
[0073] Before step S101 above, the method further includes setting a preprocessing ratio by running a script to input the preprocessing ratio R and start the preprocessing operation. The preprocessing ratio R is set according to the needs of the test.
[0074] In one implementation, the percentage of valid user data across the entire disk is obtained by acquiring and calculating the values of the total disk capacity and the disk usage.
[0075] The percentage of valid user data across the entire platform is calculated using the following formula:
[0076] Percentage of valid user data across the entire platform = NUSE / NCAP.
[0077] Using NVMe command-line tools, such as the nvme-cli tool, executing commands like `nvmeid -ns / dev / nvme0n1` and `nvmesmart -log / dev / nvme0n1` (support varies depending on the SSD and firmware), the returned data includes NUSE and NCAP related fields. These can be directly read and parsed to obtain the NCAP and NUSE parameters. In the data obtained above, total disk capacity (NCAP) and total disk usage (NUSE) are attribute parameters in the NVMe (Non-Volatile Memory Express) protocol used to describe SSD capacity. NCAP is a storage capacity-related parameter for non-volatile memory, used to characterize SSD storage capacity and available space measurement attributes, such as the total logical capacity and physical capacity of the corresponding SSD. NUSE is the range of used non-volatile memory space units, used to statistically analyze the space occupied by effective user data and related functions or parameters of used LBAs in the SSD, assisting in determining the space usage status of the storage disk.
[0078] In one implementation, the percentage of all valid user data is estimated by random sampling.
[0079] By partitioning the entire disk and randomly sampling from these partitions using a random sampling method, third-party tools such as Fio (Flexible I / O Tester) or the DD (Data Duplicator) command are used to read each sampled partition to determine the percentage of valid user data in that partition, and based on this, the percentage of valid user data across the entire disk is estimated. Fio is an open-source, cross-platform I / O performance testing tool designed specifically for storage devices (such as SSDs, HDDs, NVMe, RAID, etc.). It can flexibly simulate read and write loads under various real-world application scenarios and is one of the core tools for SSD performance testing, firmware verification, and storage system tuning. The DD command is a powerful low-level data copying and transformation tool capable of writing a certain amount of data. For example, after partitioning the entire disk, N partitions are randomly selected, and the percentage of valid user data p in each selected partition is calculated. i Based on this, estimate the percentage of valid user data Y, where Y = (p1 + p2 + p3 + ... + p N ) / N.
[0080] The default logical block address (LBA) size for the entire SSD is 4096 bytes, or 4KB aligned, which matches the default allocation unit size of most file systems. This configuration not only reduces internal management overhead but also significantly improves SSD performance. Therefore, this disclosure also uses 4KB as the smallest basic unit for LBA implementation, counting the number of valid logical block addresses (LBAs) for user data across the entire SSD. The overall SSD usage needs to be further determined based on the NUSE value for overall disk usage.
[0081] In step S101 above, the first preset condition specifically includes:
[0082] The value of the overall usage is not zero, and the percentage of valid user data is less than or equal to the first preset threshold.
[0083] In step S101 above, the second preset condition specifically includes:
[0084] The overall usage value is zero; or
[0085] The value of the overall usage is not zero, and the proportion of valid user data in the entire system is greater than the first preset threshold but less than the preprocessing ratio.
[0086] If the SSD does not support the NUSE function for full disk usage (NUSE value is zero), it is considered that the current state of the SSD storage disk has not achieved the desired effect of preprocessing, and then the controlled preprocessing operation is started directly.
[0087] If the SSD supports the NUSE (Non-zero) feature for total disk usage, the percentage of valid user data on the entire disk can be calculated by the ratio of NUSE to NCAP (Non-capacity Capacity Capacity). Based on the relationship between the percentage of valid user data, the preprocessing ratio, and the first preset threshold, the following scenarios can be further defined:
[0088] (1) When the proportion of valid user data in the whole disk is less than the first preset threshold, considering that the amount of preprocessed data is not much different from the actual capacity of the whole disk (equivalent to an empty disk), the whole disk preprocessing operation is started and the whole disk sequential filling is performed once.
[0089] (2) When the percentage of valid user data on the entire disk is between the first preset threshold and the set preprocessing ratio R, it indicates that the state of the entire SSD disk has not achieved the desired effect of preprocessing. In this case, a controllable preprocessing operation is started, i.e., non-full disk preprocessing.
[0090] (3) When the percentage of valid user data on the entire disk is greater than or equal to the set preprocessing ratio R, it means that the state of the entire SSD has reached the expected effect of preprocessing and no preprocessing operation is required.
[0091] The aforementioned first preset threshold can be adjusted according to the actual application scenario, for example, 30%. This disclosure uses a block size (BS) of 128K, a queue depth (QD) of 512, and a single thread to perform sequential disk filling operations. The data filled can be selected with a random pattern to achieve the stability of the SSD storage disk.
[0092] In the controlled preprocessing operation, the entire disk needs to be partitioned. This disclosure uses a hierarchical partitioning method for full disk partitioning, such as... Figure 3 As shown. The above steps will partition the entire disk. Based on the number of partitions and the estimated percentage of valid user data on the entire disk, it will determine whether to continue to the next level of partitioning until the third preset condition is met, specifically including:
[0093] S301. Divide the entire disk's logical block address into multiple first-level partitions and number each first-level partition sequentially.
[0094] Because SSDs have high concurrency capabilities and support multi-threaded access, this disclosure employs a maximum of n concurrent tasks to maximize SSD performance and save preprocessing time. In the aforementioned S2011, there are n primary partitions, each managed independently. To ensure balanced load across data streams, the same queue depth is used, and each primary partition is sequentially numbered A0, A1, ... A n-1Since the maximum number of concurrent connections that an SSD can support is limited, the value of n is in the range of 1 to 32.
[0095] In SSD storage, a valid Logical Block Address (LBA) refers to the address of a logical block currently storing valid data. This data is actively written by the user or system and has not yet been deleted or overwritten. An invalid Logical Block Address (LBA) refers to the address of a logical block that previously stored data, but which has been deleted (e.g., by performing a delete operation) or overwritten (e.g., by writing new data to the same LBA). In this case, the data in the LBA is invalid and is no longer considered valid by the operating system. Statistical analysis of the distribution of valid and invalid LBAs in an SSD is a core method for optimizing storage performance, managing hardware lifespan, and diagnosing faults. By quantifying the spatial distribution and dynamic changes of valid / invalid LBAs, SSDs can achieve more precise management from the hardware level to upper-level applications. Even after dividing the entire SSD's LBAs into multiple primary partitions, the capacity of each primary partition is still large. To improve the efficiency and accuracy of valid or invalid LBA statistics, it is usually necessary to further refine all primary partitions.
[0096] S302. Subdivide each primary partition into multiple secondary partitions and number each secondary partition sequentially.
[0097] In one implementation, each primary partition is subdivided into the same number of m secondary partitions, numbered A0B0, A0B1...A0B m-1 A1B0, A1B1...A1B m-1 ;...;A n-1 B0, A n-1 B1...A n-1 B m-1 For example, the entire SSD can be divided into 32 primary partitions, numbered sequentially as A0, A1...A... 31 Each primary partition is further divided into m secondary partitions, which are numbered sequentially as A0B0, A0B1...A0B m-1 A1B0, A1B1...A1B m-1 ;...;A 31 B0, A 31 B1...A 31 B m-1 .
[0098] Wherein, the larger the value of m, the finer the granularity of the logical block address (LBA) level partitioning, and the higher the percentage of the disk full after preprocessing. The maximum value of n*m is the total number of logical block addresses (LBAs) of the entire SSD disk in 4KB units. Meanwhile, since more partitions also lead to longer preprocessing execution time, this disclosure uses partitioning with m as an integer multiple of n. The initial value of m can be appropriately large, thereby reducing the number of partition refinements.
[0099] In another implementation, each primary partition is subdivided into a different number of secondary partitions, which are sequentially numbered A0B0, A0B1...A0B k-1 A1B0, A1B1...A1B k-1 ;...;A n-1 B0, A n-1 B1...A n-1 B k-1 For example, the entire SSD can be divided into two primary partitions, with the first half designated A0 and the second half A1; primary partition A1 can then be divided into m secondary partitions, numbered sequentially as A1B0, A1B1...A1B... m-1 Divide the primary partition A0 into k = 1000*m secondary partitions, and number them sequentially as A0B0, A0B1...A0B k-1 In this implementation, considering that the use of large-capacity SSD storage disks is not uniform, with the first 50% of the disk capacity being used more and the second 50% of the disk capacity being more likely to be empty, when partitioning, the primary partition (i.e., the second half of the disk) with a higher probability of being empty is subdivided into a relatively small number of secondary partitions.
[0100] S303. Randomly sample the primary partition and / or secondary partition, and estimate the percentage of valid user data in the entire disk based on the percentage of valid user data in each randomly sampled primary partition and / or secondary partition.
[0101] In step S303 above, random sampling is performed on the partitions according to the random sampling method. The sampled partitions are partitions at each level. Third-party tools such as Fio or the DD command are used to read each sampled partition to determine the proportion of effective user data in that partition, and based on this, the proportion of effective user data in the entire disk is further estimated. The following example uses two levels of partitioning for the entire disk, where the sampled partitions are either first-level or second-level partitions.
[0102] In one implementation, N primary partitions are randomly selected, and the effective LBA percentage p of each selected primary partition is calculated. i Then estimate the percentage of valid user data Y across the entire dataset, for example, Y = (p1 + p2 + p3 + ... + p N ) / N.
[0103] In another implementation, when there are m primary partitions, N secondary partitions are randomly selected from each primary partition. First, the percentage of valid user data in each randomly selected secondary partition is calculated, and the percentage of valid user data in each randomly selected primary partition is estimated to be A. x p=(A x p1+A x p2+...+A x p N ) / N, to obtain the first-level partition A x Estimate the percentage of effective user data; then estimate the total percentage of effective user data Y, Y = {(A0p1 + ... + A0p...} N ) + (A1p1 + ... + A1p N )+...+(A m-1 p1+...+A m-1 p N )} / (m*N), where m is the number of first-level partitions.
[0104] S304. If the estimated percentage of valid user data across the entire disk meets the third preset condition, then the partitioning is completed; if not, then determine whether the number of secondary partitions has reached the maximum value. If not, return and re-partition the secondary partitions; if so, continue to the next level of partitioning until the estimated percentage of valid user data across the entire disk based on the partitioning meets the third preset condition.
[0105] The third preset condition is that the standard deviation between the estimated percentage of total effective user data and the obtained percentage of total effective user data is less than or equal to a preset threshold.
[0106] In one implementation, a preset threshold is set to ±2% of the percentage of valid user data acquired across the entire disk. If a third preset condition is met, i.e., the standard deviation is within ±2% of the percentage of valid user data acquired across the entire disk, it indicates that the partitions (number of layers and number of partitions) accurately reflect the distribution of valid user data on the SSD, and the partitioning method is appropriate. The number of secondary partitions within each primary partition is denoted as M, meaning the entire SSD is divided into N*M partitions.
[0107] If the third preset condition is not met, i.e., the standard deviation exceeds ±2% of the calculated effective user data percentage of the SSD, then the current partitioning (number of levels and / or number of partitions) is inappropriate and requires adjustment of the partitioning method. This adjustment is carried out in a progressive manner. Since the number of first-level partitions, n, is related to the maximum concurrency supported by the SSD, the number of second-level partitions, m, is adjusted first. This adjustment includes increasing or decreasing m, and the adjusted m remains an integer multiple of n. Because the maximum value of n*m is the total number of Logical Block Addresses (LBAs) of the entire SSD in 4KB units, there is an upper limit to the adjustment of the number of second-level partitions, m. If the adjusted number of second-level partitions, m, has reached its maximum value, and the estimated effective user data percentage still does not meet the third preset condition, then the next level of partitioning needs to be performed, i.e., a third-level partitioning. Specific partitioning methods and sequential numbering methods are detailed in the relevant content on second-level partitioning and will not be described in detail here.
[0108] It should be noted that when continuing to the next level of partitioning, it is necessary to return to step S302, adjust the number of second-level partitions m, and redo the second-level partitioning and the next level of partitioning. The number of partitions i in the next level also needs to satisfy the maximum value of n*m*i, which is the total number of logical block addresses (LBAs) of the entire SSD in 4KB units, and so on.
[0109] The entire disk is partitioned using the layer-by-layer partitioning method described above until the standard deviation between the estimated percentage of valid user data based on the partitions and the actual percentage of valid user data is within a preset threshold, thus satisfying the third preset condition. At this point, the entire disk is partitioned. Next, based on this partitioning, each partition will be scanned sequentially to identify the non-full disk partitions that need to be populated.
[0110] In one implementation, such as Figure 4 As shown, the above steps involve scanning each partition, filling the disk with partitions that meet the fourth preset condition, and determining whether to re-partition the entire disk based on the percentage of valid user data on the disk after filling and the set preprocessing ratio, until the set preprocessing ratio is reached. Specifically, this includes:
[0111] S401. Perform sequential indexing of each partition at each level in each first-level partition, where the index of the first partition at the lowest level in each first-level partition is 0.
[0112] Taking a two-level partitioning of the entire SSD as an example, the entire SSD is divided into N primary partitions and M secondary partitions, with the partitions numbered sequentially as A0B0, A0B1...A0B... M-1 A1B0, A1B1... A1BM-1 ...;A N-1 B0, A N-1 B1... A N-1 B M-1 For each second-level partition within each first-level partition, an index is sequentially created, with the indexes for each partition being A0B0 (index 0), A0B1 (index 1), ... A0B... M-1 (Index M-1); A1B0 (Index 0), A1B1 (Index 1)... A1B M-1 (Index M-1)...; A N-1 B0 (index 0), A N-1 B1 (Index 1)... A N-1 B M-1 (Index M-1).
[0113] S402. Perform parallel operations according to the first-level partitions, and simultaneously judge each first-level partition in the order of the index; determine whether the index is less than J-1. If so, read the user data of the partition corresponding to the current index; count the number of zeros in the user data of the corresponding partition, and calculate the proportion of zero user data. Record the number of the partition corresponding to the current index whose proportion of zero user data meets the fourth preset condition in the test log. Increment the current index, return, and repeat this step, where J is the number of lower-level partitions contained in the first-level partition; if not, end the scanning of each partition.
[0114] A user data value of zero indicates that the current logical block address LBA previously stored data, but that data has been deleted or overwritten and is currently invalid. Calculating the percentage of logical block address LBAs with zero user data allows us to understand the current usage of logical block addresses LBAs in the secondary partition.
[0115] The fourth preset condition is that the proportion of user data that is zero is greater than or equal to the preprocessing proportion.
[0116] S403. Fill the disk with the partition corresponding to the number recorded in the test log, compare the percentage of valid user data on the disk after filling with the set preprocessing ratio. If it is greater than or equal to the set preprocessing ratio, end the controllable preprocessing operation; if it is less than the set preprocessing ratio, return and redo the secondary partitioning, and continue to execute the subsequent operations.
[0117] In one implementation, such as Figure 5 As shown, the above steps involve scanning each partition, filling the disk with partitions that meet the fourth preset condition, and determining whether to re-partition the entire disk based on the percentage of valid user data on the disk after filling and the set preprocessing ratio, until the set preprocessing ratio is reached. Specifically, this includes:
[0118] S501. Perform sequential indexing of each partition at each level under each first-level partition, where the index of the first partition at the lowest level in the first first-level partition is 0.
[0119] Taking a two-level partitioning of the entire SSD as an example, the entire SSD is divided into N primary partitions and M secondary partitions, with the partitions numbered sequentially as A0B0, A0B1...A0B... M-1 A1B0, A1B1... A1B M-1 ...;A N-1 B0, A N-1 B1... A N-1 B M-1 A unified index is set for each partition level, with the indexes for each partition being A0B0 (index 0), A0B1 (index 1), ... A0B... M-1 (Index M-1); A1B0 (Index M), A1B1 (Index M+1)... A1B M-1 (Index 2M)...; A N-1 B0(index(N-1)*M), A N-1 B1(index(N-1)*M+1)... A N-1 B M-1 (Index N*M-1).
[0120] S502. Perform sequential judgments according to the index order; determine whether the index is less than J-1. If so, read the user data of the partition corresponding to the current index; count the number of zeros in the user data of the corresponding partition and calculate the proportion of zero user data. Record the number of the partition corresponding to the current index whose proportion of zero user data satisfies the fourth preset condition in the test log. Increment the current index, return, and repeat this step. J is the number of partitions contained in the entire disk. If not, end the scan of each partition.
[0121] A user data value of zero indicates that the current logical block address LBA previously stored data, but that data has been deleted or overwritten and is currently invalid. Calculating the percentage of user data values with zero data allows us to understand the current usage of logical block address LBAs in the secondary partition.
[0122] S503. Fill the disk with the partition corresponding to the number recorded in the test log, compare the percentage of valid user data on the disk after filling with the set preprocessing ratio. If it is greater than or equal to the set preprocessing ratio, end the controllable preprocessing operation; if it is less than the set preprocessing ratio, return and redo the secondary partitioning, and continue to execute the subsequent operations.
[0123] In the above steps, the disk filling operation is performed using specialized tools for the storage device, such as Fio (Flexible I / OTester).
[0124] The disk filling methods include sequential disk filling, parallel sequential disk filling, or merged partition sequential disk filling. These are described below:
[0125] (1) Fill the plate one by one in sequence
[0126] The sequential filling of the tray specifically includes:
[0127] The records in the acquired test logs are arranged in ascending order of their numbers. A block size of 128KB and a single queue depth of 256 are used to sequentially fill each partition at each level according to the arrangement order.
[0128] The above ascending order is based on a two-level partitioning of the entire disk, where the first-level partitions are partitioned from smallest to largest, followed by the second-level partitions. For example, if the entire SSD is divided into N first-level partitions and M second-level partitions, with the ascending order of partitions being A0B0, A0B1...A0B... M-1 A1B0, A1B1...A1B M-1 ...;A N-1 B0, A N-1 B1...A N-1 B M-1 .
[0129] The sequential filling method has the following characteristics:
[0130] High throughput and stability: The 128KB block size, combined with a high queue depth of 256, is suitable for continuous large data stream transmission, maximizing the sequential read and write bandwidth of the SSD and reducing fragmentation loss in data transmission.
[0131] Compatibility and standardization: Parameter settings (such as BS=128KB) conform to mainstream storage testing specifications (such as enterprise-level SSD performance testing), and the results can be benchmarked against industry standards, which facilitates performance verification and troubleshooting.
[0132] Sequential operation reliability: sequential disk filling avoids the scheduling overhead caused by multi-task concurrency, making it suitable for scenarios with high data consistency requirements (such as disk initialization and firmware verification) and reducing the risk of operational errors.
[0133] (2) Parallel sequential filling
[0134] The parallel sequential disk filling specifically includes:
[0135] The records in the acquired test logs are classified by number according to the first-level partition. Within each category, the records are sorted in ascending order of their numbers. The block size is set to 4KB and the single queue depth is set to 64. The records are then filled in parallel sequentially at each level of each category according to the sorting order.
[0136] The parallel sequential filling method has the following characteristics:
[0137] Parallel efficiency improvement: By utilizing the multi-channel parallel architecture inside the SSD (such as multiple CE channels of NAND flash memory), multiple partitions can be processed in parallel, which can fully release the hardware concurrency capabilities and shorten the overall disk filling time, especially suitable for SSDs with multi-core controllers.
[0138] Fine-grained performance testing: The 4KB block parameters closely resemble everyday user applications (such as operating system file read and write), which can accurately test the parallel processing capabilities of SSDs in fragmented data scenarios and evaluate the benchmark performance of random access.
[0139] Resource utilization optimization: The queue depth of 64 balances concurrent requests and controller load, avoiding scheduling latency under high queue depth, making it suitable for testing the stability of SSDs under moderate concurrency pressure.
[0140] (3) Merge partitions sequentially and fill in the disk.
[0141] The sequential disk filling for merging partitions specifically includes:
[0142] Based on the numbers recorded in the obtained test logs, identify each partition at each level that has an adjacent relationship. Merge each partition at each level that has an adjacent relationship into a new partition. The new partition uses the number of any partition before the merger. Arrange the numbers in ascending order. Use parameters with a block size of 128KB and a single queue depth of 512. Fill each partition at each level sequentially according to the arrangement order. The adjacent partitions to be merged contain consecutive free blocks.
[0143] The sequential disk filling method for merging partitions has the following characteristics:
[0144] Optimization of contiguous address space: Merging adjacent partitions can reduce logical address fragmentation, form contiguous large blocks, reduce the query overhead of the SSD internal address mapping table (FTL), and improve data writing efficiency.
[0145] Ultra-high throughput scenario adaptation: 128KB block size with a high queue depth of 512 can simulate enterprise-level big data streams (such as cloud computing storage and data center backup), giving full play to the performance limits of SSDs when continuously reading and writing ultra-large files.
[0146] Reduce metadata management burden: Reducing the number of partitions simplifies the management of partition tables by SSD firmware. Especially in high-density storage scenarios, it can reduce performance degradation caused by frequent partitioning operations and extend the lifespan of SSDs.
[0147] In the steps described above, three methods are used to optimize SSD performance in terms of stability, parallel performance, and sequential read / write efficiency through different parameter combinations and partitioning strategies. The appropriate method can be flexibly selected based on actual needs (such as performance benchmarking, scenario simulation, and lifespan verification).
[0148] In the above steps, the percentage of valid user data on the entire disk after filling is checked, calculated by obtaining NCAP and NUSE parameters; or a specialized tool (such as third-party tools Fio or DD) is used to scan the entire disk to statistically determine the percentage of valid user data. This preprocessing method effectively avoids performing a full disk erase and write operation every time. For large-capacity SSDs, the benefits of reduced preprocessing time and fewer P / E erase cycles are significant.
[0149] Figure 6 This is a flowchart illustrating another preprocessing method provided in an embodiment of the present disclosure, which differs from... Figure 1 The method shown in this process involves re-partitioning the entire disk at the first level when the percentage of valid LBAs is lower than the test requirements. Specifically, this includes:
[0150] S601. Run the script, input the preprocessing ratio R, and start the preprocessing operation; obtain the values of NCAP and NUSE using the nvme cli tool;
[0151] S602. Determine if NUSE is enabled: If NUSE is not enabled, execute S603; if NUSE is enabled and NUSE / NCAP is greater than or equal to the set ratio R, terminate the script execution and do not enter the preprocessing operation; if NUSE is enabled and NUSE / NCAP is less than or equal to 30%, perform a full disk write in sequence and terminate the script execution; if NUSE is enabled and NUSE / NCAP is between 30% and R, execute S603.
[0152] S603. Divide the entire disk's LBA space into several regions according to certain principles and number them A0, A1, ... An. Then, further divide each LBA partition into several regions and number them, for example, A0B0, A0B1... A0B... m-1 A1B0, A1B1... An B m The number of partitions and the method for determining the optimal solution are based on random sampling.
[0153] S604. There are a total of N*M LBA partitions on the entire disk. Select LBA partition A0B0 as the starting point and mark the index as 0.
[0154] S605. Determine if the index is less than the total number of LBA partitions, then execute S606; if the index is not less than the total number of LBA partitions, then execute S611.
[0155] S606: Read the user data in the LBA space with index=0, and then execute S607;
[0156] S607 counts the number of LBAs with zero user data in S606, calculates the percentage of LBAs with zero user data as A, and then executes S608.
[0157] S608. Determine if the proportion of LBAs with zero user data is greater than or equal to R, then proceed to step 9; if the proportion of LBAs with zero user data is less than R, then proceed to step 10.
[0158] S609. Record the LBA space number into the test log, then execute S610;
[0159] S610, index + 1, then execute S605~S610;
[0160] S611. Write the LBA space recorded in the test log in a random pattern order, and then execute S612.
[0161] S612. Recheck the percentage of valid LBAs across the entire disk. If the percentage of valid LBAs across the entire disk meets the requirements, end the script execution. If the percentage of valid LBAs across the entire disk is lower than the test requirements, continue to execute the preprocessing process, executing S603~S612 until the percentage of valid LBAs across the entire disk meets the requirements, then end the script execution.
[0162] The above preprocessing method will be further explained through three application examples below. For ease of explanation, all three application examples use a two-level partitioning of the entire disk.
[0163] Example 1:
[0164] This embodiment mainly focuses on the preprocessing of a 64T SSD, with a preprocessing ratio R set to 95%.
[0165] Run the script and input the preprocessing ratio R=95%; according to the product white paper, the SSD storage disk supports the NUSE function; obtain the NCAP and NUSE values through the nvme cli tool, and the effective LBA ratio of the SSD storage disk is X=NUSE / NCAP=75%.
[0166] Divide the entire disk into 8 equal regions, numbered A0, A1, ... A7 as primary partitions. Determine the optimal partition using random sampling. Further divide each primary partition into m secondary partitions, numbered A0B0, A0B1, ... A0B1. m-1 A1B0, A1B1.. A7B m-1 Using random sampling, N first-level partitions are randomly selected and scanned. The percentage of effective LBAs (p) for each first-level partition is then calculated. i Based on the effective LBA percentage of the N extracted primary partitions, estimate the total effective LBA percentage Y = (p1 + p2 + p3 + ... + p N ) / N. Compare the estimated effective LBA percentage Y with the actual effective LBA percentage X of the SSD storage disk in S101, and calculate the standard deviation. If the standard deviation exceeds ±2%, repeat S102; if the standard deviation is within ±2%, the partition is appropriate, and continue execution.
[0167] The disk is divided into 8*M LBA partitions, which are then sequentially numbered as A0B0, A0B1... A0B M-1 A1B0, A1B1... A7B M-1 Parallel operations are performed based on the first-level partitions. Taking partition A0 as an example, the LBA partition A0B0 is selected as the starting point, and the index is marked as 0. This process is repeated until A0B0 is reached. M-1 Marked as (M-1). Execute synchronously for each first-level partition.
[0168] The process terminates if the index is not less than M-1; otherwise, it continues. It reads user data from the LBA space at index=0. It counts the number of LBAs with zero values among the read user data and calculates the percentage (P) of LBAs with zero values. If the percentage (P) of zero values is greater than or equal to R, it records the LBA space number in the test log; it increments the index by 1 and returns to continue the process until the index equals M-1.
[0169] Write the LBA space recorded in the test log in a random pattern order. Recheck the percentage of valid LBAs across the entire disk. If the percentage of valid LBAs meets the requirements, terminate the script. If the percentage of valid LBAs is lower than the test requirements, continue the preprocessing process until the percentage of valid LBAs meets the requirements, then terminate the script.
[0170] Based on this embodiment, the SSD storage capacity is 64TB (61.44TB), with 75% of the data already valid. The expected target is a 95% fill rate. Assuming that LBA holes (i.e., invalid LBAs) are uniformly distributed, the amount of data to be filled is approximately 12.288TB to 49.152TB, which is 20% to 80% of the disk capacity. In this scenario, given the known percentage of valid data already in the SSD, the amount of data to be written can be estimated based on the difference between the current fill rate and the target fill rate. This avoids the problem of blind writing during traditional fill processes. By dynamically and adaptively adjusting the fill strategy, write amplification can be reduced, allowing for reasonable control of the write amplification rate (WAF), avoiding unnecessary data writing, and extending the lifespan of the SSD.
[0171] Example 2:
[0172] This embodiment mainly focuses on the preprocessing of a 128T SSD, with a preprocessing ratio R set to 98%.
[0173] Run the script and input the preprocessing ratio R=98%; if it is known that the SSD storage disk does not support the NUSE function; use the nvmecli tool to obtain the values of NCAP and NUSE, and find that the effective LBA ratio of the SSD storage disk is X=NUSE / NCAP=1, so the effective LBA ratio of the SSD storage disk cannot be determined.
[0174] Divide the entire board into 32 equal areas, and number them sequentially as A0, A1, ... A 31 As a primary partition, the optimal partition is determined using random sampling. Each primary partition is then further divided into m secondary partitions, numbered A0B0, A0B1...A0B1. m-1 A1B0, A1B 1. A 31 B m-1 Using random sampling, N partitions are randomly selected from each primary partition and scanned. The percentage of valid LBAs (Alternative Base Access Providers) for each partition is then calculated. x p i Based on the effective LBA percentage of N partitions extracted from each primary partition, estimate the total effective LBA percentage Y = (A0p1 + A0p2 + ... + A0p...). n +A1p1+ ... +A1pn + ... +A 31 p n ) / (32 * N), and estimate the effective LBA percentage for each first-level partition as y x =(A x p1+ A x p2+ ...+A x p n ) / N. Compare the estimated effective LBA percentage Y with the actual effective LBA percentage X of the SSD storage disk in S101, and calculate the standard deviation. If the standard deviation exceeds ±2%, repeat S102; if the standard deviation is within ±2%, the partition is appropriate, and continue execution.
[0175] The total number of LBA partitions is determined to be 32*M, and they are renumbered sequentially as index=0, index=1, ..., index=(32*M-1).
[0176] The process terminates when the index is not less than (32*M-1); otherwise, it continues. It reads user data from the LBA space at index=0. It counts the number of LBAs with zero values among the read user data and calculates the percentage (P) of LBAs with zero values. If the percentage (P) of zero values is greater than or equal to R, it records the LBA space number in the test log; it increments the index by 1 and returns to continue the process until the index equals 32*M-1.
[0177] Write the LBA space recorded in the test log in a random pattern order. Recheck the percentage of valid LBAs across the entire disk. If the percentage of valid LBAs meets the requirements, terminate the script. If the percentage of valid LBAs is lower than the test requirements, continue the preprocessing process until the percentage of valid LBAs meets the requirements, then terminate the script.
[0178] Based on this embodiment, the SSD storage capacity is 128TB (122.88TB), and the percentage of valid data on the entire disk is unknown. The expected target is a disk fill rate of R=98%. According to the disk effective ratio Y estimated by S10205c, the amount of data to be filled is approximately (RY) / (1-Y) to (RY) / (1-Y) of the disk capacity. By estimating the percentage of valid data Y, the disk fill strategy is dynamically adjusted to adapt to SSDs under different usage conditions, improving the versatility and flexibility of the method. It can support large-capacity SSDs (such as 64TB, 128TB and above) and has good scalability and adaptability.
[0179] Example 3:
[0180] This embodiment mainly focuses on the preprocessing of a 128T SSD, with a preprocessing ratio R set to 90%.
[0181] Run the script and input the preprocessing ratio R=90%; if it is known that the SSD storage disk does not support the NUSE function; use the nvmecli tool to obtain the values of NCAP and NUSE, and find that the effective LBA ratio of the SSD storage disk is X=NUSE / NCAP=1, so the effective LBA ratio of the SSD storage disk cannot be determined.
[0182] Considering that the use of large-capacity SSD storage disks is not completely random, the entire disk is divided into two equal areas: the first half, A0, and the second half, A1, as primary partitions. The optimal partition is determined using random sampling. Partition A1 is then further divided into m areas and numbered, for example, A1B0, A1B1... A1B... m-1 Divide partition A0 into k = 1000*m regions and number them, for example, A0B0, A0B1... A0B k-1 Using random sampling, N partitions are randomly selected from each primary partition and scanned. The effective LBA percentage (Axpi) of each partition is calculated. Based on the effective LBA percentages of the N partitions selected from each primary partition, the overall effective LBA percentage (Y) is estimated as follows: Y = (A0p1 + A0p2 + ... + A0p...) n + A1p1+ ... + A1p n ) / (2 * N), and estimate the effective LBA percentage for each first-level partition as y. x =(A x p1+ A x p2+ ... + A x p n ) / N. Compare the estimated effective LBA percentage Y with the actual effective LBA percentage X of the SSD storage disk, and calculate the standard deviation. If the standard deviation exceeds ±2%, repeat the above steps; if the standard deviation is within ±2%, the partition is appropriate, and continue.
[0183] The total number of LBA partitions is determined to be 2*M, and they are renumbered sequentially as index=0, index=1, ..., index=(2*M-1).
[0184] The process terminates when the index is not less than (2*M-1); otherwise, it continues. It reads user data from the LBA space at index=0. It counts the number of LBAs with zero values among the read user data and calculates the percentage (P) of LBAs with zero values. If the percentage (P) of zero values is greater than or equal to R, it records the LBA space number in the testlog; it increments the index by 1 and returns to continue the process until the index equals 2*M-1.
[0185] Write the LBA space recorded in the test log in a random pattern order. Recheck the percentage of valid LBAs across the entire disk. If the percentage of valid LBAs meets the requirements, terminate the script. If the percentage of valid LBAs is lower than the test requirements, continue the preprocessing process until the percentage of valid LBAs meets the requirements, then terminate the script.
[0186] Based on this embodiment, the SSD storage capacity is 128TB (122.88TB), and the percentage of valid data on the entire disk is unknown. The expected target is a disk fill rate of R=90%, with most of the valid data located in the first 50% of the disk capacity. According to the estimated valid data ratio Y in this embodiment, the amount of data to be filled is approximately (RY)~(RY) / (1-Y) of the disk capacity. Since the LBA region is non-uniformly partitioned, the efficiency is greatly improved. Furthermore, the LBA regions that need to be written are merged. Through the integration and optimization of the write regions, although the proportion of written data is the same as in Example 2, the execution efficiency after merging is approximately 20% higher than in Example 2. For cases where valid data is concentrated in specific LBA regions, by merging consecutive free blocks or optimizing the write order, the disk fill efficiency is significantly improved, and the performance loss caused by random writes is reduced.
[0187] In summary, this disclosure provides a simplified preprocessing scheme suitable for large-capacity storage disks. This scheme is applicable not only to performance and functional testing, but also yields greater benefits as the storage disk capacity increases. Furthermore, this method can be integrated into other automated testing processes, and through flexible parameter configuration, it can achieve different disk-filling ratios according to testing needs. Its core lies in estimating the effective LBA of the entire disk through random sampling. As a basic application method, it is particularly suitable for scenarios where the storage disk status is completely unknown. Based on the sequential write bandwidth estimation of enterprise-grade SSDs on the market, it would take at least 4 hours to fill a 128TB SSD in an empty state. If calculated based on 50% effective data on the disk capacity, each round of preprocessing can save at least 40% of the time and one P / E erase.
[0188] It should be understood that, although Figures 1 to 6The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figures 1 to 6 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.
[0189] It is understood that the same / similar parts between the various embodiments of the methods described above in this specification can be referred to each other. Each embodiment focuses on the differences from other embodiments, and relevant parts can be referred to the description of other method embodiments.
[0190] According to embodiments of this disclosure, a storage device is also provided.
[0191] Figure 7 This is a schematic diagram of the structure of a storage device 700 provided in an embodiment of this disclosure, as shown below. Figure 7 As shown, the storage device may include a processor 701 and a memory 702. The memory 702 stores a computer program 703. When the computer program 703 is executed by the processor 701, it can implement the method provided in any of the above embodiments. The execution mode and beneficial effects are similar and will not be described again here.
[0192] Of course, for the sake of simplicity, Figure 7 Only some of the components of the storage device relevant to the present invention are shown, omitting components such as buses, input / output interfaces, input devices, and output devices. In addition, the storage device may include any other suitable components depending on the specific application.
[0193] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.
[0194] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.
Claims
1. A preprocessing method for a mixed testing scenario of solid-state drives, characterized in that, The method includes: Obtain the percentage of valid user data across the entire system. If the percentage of valid user data across the entire system meets a first preset condition, then initiate a full-system preprocessing operation. If the percentage of valid user data in the entire system meets the second preset condition, then a controllable preprocessing operation is initiated. The controllable preprocessing operation includes: The entire disk will be partitioned. Based on the number of partitions and the estimated percentage of valid user data in the entire disk, it will be determined whether to continue to the next level of partitioning until the estimated percentage of valid user data in the entire disk meets the third preset condition. Scan each partition, fill the disk for partitions that meet the fourth preset condition, and determine whether to re-partition the entire disk based on the percentage of valid user data after filling and the set preprocessing ratio, until the set preprocessing ratio is reached. The process involves partitioning the entire disk. Based on the number of partitions and the estimated percentage of valid user data across the disk, a decision is made whether to proceed to the next level of partitioning until the estimated percentage of valid user data meets a third preset condition. This condition includes: The entire disk's logical block address is divided into multiple first-level partitions, and each first-level partition is sequentially numbered. Each primary partition is further subdivided into multiple secondary partitions, and each secondary partition is sequentially numbered. Randomly sample the first-level partition and / or the second-level partition, and estimate the percentage of valid user data in the whole disk based on the percentage of valid user data in each randomly sampled first-level partition and / or second-level partition; If the estimated percentage of valid user data across the entire disk meets the third preset condition, the partitioning is completed; if not, it is determined whether the number of secondary partitions has reached the maximum value. If not, it is returned and secondary partitioning is performed again; if so, the next level of partitioning continues until the estimated percentage of valid user data across the entire disk meets the third preset condition.
2. The method according to claim 1, characterized in that, The second preset condition specifically includes: The overall usage value is zero; or The value of the overall usage is not zero, and the proportion of valid user data in the entire system is greater than the first preset threshold but less than the preprocessing ratio.
3. The method according to claim 1, characterized in that, Random sampling is performed on the primary and / or secondary partitions. Based on the percentage of valid user data in each randomly sampled primary and / or secondary partition, the percentage of valid user data across the entire disk is estimated. Specifically, this includes: Randomly select N primary partitions, calculate the percentage of valid user data in each selected primary partition, and then estimate the percentage of valid user data across the entire disk; or, In each primary partition, N secondary partitions are randomly selected. The percentage of valid user data in each randomly selected secondary partition is calculated, and the percentage of valid user data in each randomly selected primary partition is estimated. Finally, the percentage of valid user data in the entire disk is estimated.
4. The method according to claim 1, characterized in that, The process of partitioning the entire disk specifically includes: Divide the entire disk's logical block address into two primary partitions; Then divide each of the two primary partitions into a different number of secondary partitions.
5. The method according to claim 3 or 4, characterized in that, Scan each partition, specifically including: Each partition in each level of each first-level partition is indexed sequentially, with the index of the first partition at the lowest level of each first-level partition being 0. Parallel operations are performed on the first-level partitions. For each first-level partition, the index is judged sequentially. The partition number corresponding to the current index that satisfies the fourth preset condition with zero user data is recorded in the test log.
6. The method according to claim 3 or 4, characterized in that, Scan each partition, specifically including: Each partition at each level under each first-level partition is indexed sequentially, with the index of the first partition at the lowest level of the first first-level partition being 0. The system will make sequential judgments according to the index order, and record the partition number corresponding to the current index that satisfies the fourth preset condition (zero percentage of user data) in the test log.
7. The method according to claim 1, characterized in that, The disk filling methods include sequential disk filling, parallel sequential disk filling, or merged partition sequential disk filling.
8. The method according to claim 7, characterized in that, The parallel sequential disk filling specifically includes: The records in the acquired test logs are classified by their numbers according to the first-level partition. Within each category, the records are sorted in ascending order of their numbers. The disks are then filled in parallel sequentially at each level within each category, following the order of the sorting. The sequential disk filling for merging partitions specifically includes: Identify the partitions at each level that have an adjacent relationship based on the numbers recorded in the obtained test logs. Merge the partitions at each level that have an adjacent relationship into a new partition. The new partition uses the number of any partition before the merger. Arrange the numbers in ascending order. Fill the disk one by one for each partition at each level in the order of arrangement. The adjacent partitions to be merged contain consecutive free blocks.
9. A storage device, characterized in that, include: A memory and a processor, the processor including multiple cores, wherein the memory stores a computer program that, when executed by the processor, implements the preprocessing method as described in any one of claims 1 to 8.
Citation Information
Patent Citations
Hard disk test method, hard disk fault early warning method and hard disk fault early warning device
CN120407307A
Testing method, device and equipment for solid state disk and medium
CN120412693A
