Solid state drive based hot and cold data identification test method and apparatus

Abstract

The application relates to a solid state disk-based hot and cold data identification test method and device, computer equipment and a storage medium, wherein the method comprises the following steps: performing safe erasing on a to-be-tested solid state disk to make the initial state of the to-be-tested solid state disk consistent; performing FIO sequential writing on the whole disk, recording the performance per second as a first array, and stopping the test when the recorded performance in a continuous period of time is less than half of the performance per second; sequentially changing the size of blocks and repeatedly performing the above-mentioned erasing and writing operations on the whole disk, recording the corresponding performance per second as a corresponding array; and judging whether there is hot and cold data identification according to whether the recorded performances under different block sizes are consistent. The application uses FIO to perform writing of different data sizes and multiple read-write accesses on the tested disk, confirms whether the tested solid state disk passes the data block size and data amount as the hot and cold data judgment condition, and can identify whether the to-be-tested solid state disk has the hot and cold data judgment condition.

Description

Technical Field

[0001] This invention relates to the field of solid-state drive (SSD) testing technology, and in particular to a method, apparatus, computer device, and storage medium for identifying hot and cold data based on SSDs. Background Technology

[0002] Solid-state drives (SSDs) offer advantages such as fast access speeds, low power consumption, and good shock resistance. In recent years, with the development of flash memory technology and the decrease in flash memory media prices, SSDs have been widely used in consumer electronics, enterprise servers, and data centers. Flash memory media includes SLC (Single Level Cache, 1 bit per cell, hereinafter referred to as SLC), which is fast and has a long lifespan (approximately 100,000 write / erase cycles); TLC (Trinary-Level Cell, 3 bits per cell, hereinafter referred to as TLC); and QLC (Quad-level cells, 4 bits per cell, hereinafter referred to as QLC). Currently, mainstream SSDs mainly use TLC media. Its write performance bandwidth is less than one-third that of SLC and its lifespan is short, but its advantages are low price and large capacity, so it has gradually become the mainstream SSD. To enable TLC-based SSDs to provide SLC-level performance during testing and use, and to meet the high bandwidth requirements of specific scenarios, various companies employ a strategy of simulating an SLC layer on top of the TLC layer to create an SLC cache. When the host stops writing data to the SSD, the SSD begins moving data from the SLC cache to the TLC cache, thus restoring write performance to the level of simulated SLC. However, since flash memory can only perform off-site updates, when data recently written to SLC is moved to TLC, invalid data pages are created in the original physical block. Garbage collection of the physical block is required to reuse these invalid data pages. Repeatedly writing large amounts of cold data to SLC and then moving it to TLC can cause approximately 2x write amplification, impacting the flash memory's lifespan and long-term performance. Therefore, cold data can be identified and written directly to TLC, avoiding the need for migration from SLC to TLC, thereby improving flash memory performance and lifespan.

[0003] Currently, mainstream methods for separating hot and cold data in flash memory include: using write request data size and data volume as thresholds for hot and cold data, which can distinguish some data with low system overhead; and determining hot and cold data by recording the access frequency of logical pages, which requires a large amount of space to record the access frequency of each logical page, resulting in significant space overhead. Therefore, most current methods for verifying SSD performance test fixed-range read / write performance or full-disk steady-state performance, but they cannot identify the conditions for determining whether hot or cold data exists. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer equipment, and storage medium for identifying and testing hot and cold data based on solid-state drives (SSDs) to address the aforementioned technical problems.

[0005] A method for identifying hot and cold data based on solid-state drives, the method comprising:

[0006] Perform a secure erase on the solid-state drive under test to bring it back to its initial state.

[0007] Write the entire disk sequentially using FIO, and record the performance per second as the first array. Stop the test when the recorded performance is less than half of the performance in the first second for a continuous period of time.

[0008] The block size is changed sequentially, and the above erase and write operations are repeated. The corresponding performance per second is recorded as the corresponding array.

[0009] The presence of hot and cold data can be determined by whether the performance recorded is consistent across different block sizes.

[0010] In one embodiment, the step of sequentially writing to the entire disk using FIO, recording the performance per second as the first array, and terminating the test when the recorded performance is less than half of the performance in the first second for a continuous period of time further includes:

[0011] The entire disk is written sequentially using FIO, with a queue depth of 8192 / block size, 1 thread, and a block size of 1024KB. The performance per second is recorded in the first array. The test is stopped when the performance is less than half of the performance in the first second for 5 consecutive seconds.

[0012] In one embodiment, the step of sequentially changing the block size and repeatedly performing the erase and write-to-disk operations, and recording the corresponding performance per second as a corresponding array, further includes:

[0013] The block size is changed sequentially to 512KB, 256KB, 128K, 64K, 32K, and 16KB. The erase and write operations are repeated, and the performance per second is recorded for the corresponding second, third, fourth, fifth, sixth, and seventh arrays.

[0014] In one embodiment, the step of determining whether there is hot or cold data identification based on whether the performance recorded at different block sizes is consistent further includes:

[0015] The test results for the solid-state drive under test are determined by using the write request data size and data volume as the criteria for judging hot and cold data thresholds, or by the absence of such criteria.

[0016] A solid-state drive-based cold and hot data identification and testing device, the device comprising:

[0017] A secure erase module is used to securely erase the solid-state drive under test to ensure that the initial state of the solid-state drive under test is consistent.

[0018] The performance recording module is used to sequentially write the entire disk using FIO, and record the performance per second as the first array. The test is stopped when the recorded performance is less than half of the performance in the first second for a continuous period of time.

[0019] The repeated testing module is used to change the size of the block sequentially and repeatedly perform the above-mentioned erasure and write-to-full-disk operations, and record the corresponding performance per second as a corresponding array.

[0020] The result judgment module is used to determine whether there is hot or cold data based on whether the performance recorded at different block sizes is consistent.

[0021] In one embodiment, the performance recording module is further configured to:

[0022] The entire disk is written sequentially using FIO, with a queue depth of 8192 / block size, 1 thread, and a block size of 1024KB. The performance per second is recorded in the first array. The test is stopped when the performance is less than half of the performance in the first second for 5 consecutive seconds.

[0023] In one embodiment, the repeatability module is further configured to:

[0024] The block size is changed sequentially to 512KB, 256KB, 128K, 64K, 32K, and 16KB. The erase and write operations are repeated, and the performance per second is recorded for the corresponding second, third, fourth, fifth, sixth, and seventh arrays.

[0025] In one embodiment, the result determination module is further configured to:

[0026] The test results for the solid-state drive under test are determined by using the write request data size and data volume as the criteria for judging hot and cold data thresholds, or by the absence of such criteria.

[0027] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described above.

[0028] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above methods.

[0029] The aforementioned method, apparatus, computer equipment, and storage medium for identifying hot and cold data on solid-state drives (SSDs) involve: 1) Securely erasing the SSD under test to ensure a consistent initial state; 2) Sequentially writing the entire disk using FIO (Fulfilled Instructions for Propagation), recording the performance per second as the first array; 3) Stopping the test when the recorded performance is less than half of the first second's performance for a continuous period; 4) Repeating the erasing and writing operations while sequentially changing the block size, recording the corresponding performance per second as the corresponding array; 5) Determining whether hot and cold data are identified based on the consistency of the recorded performance across different block sizes. This invention utilizes FIO to write different data sizes and perform multiple read / write accesses on the tested disk to confirm whether the tested SSD uses data block size and data volume as criteria for hot and cold data identification, thus identifying the conditions for determining whether the tested SSD has hot and cold data. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating a solid-state drive-based method for identifying hot and cold data in one embodiment.

[0031] Figure 2 This is a flowchart illustrating a solid-state drive-based method for identifying hot and cold data in another embodiment.

[0032] Figure 3 This is a flowchart illustrating a solid-state drive-based method for identifying hot and cold data in another embodiment.

[0033] Figure 4 This is a schematic diagram illustrating the test results of the product under test in one embodiment with a block size of 1024KB;

[0034] Figure 5 This is a schematic diagram illustrating the test results of the product under test in one embodiment with a block size of 32KB;

[0035] Figure 6 This is a structural block diagram of a solid-state drive-based cold and hot data identification and testing device in one embodiment;

[0036] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0038] Currently, most methods for verifying SSD performance involve testing read / write performance within a fixed range or steady-state performance across the entire drive, but these methods cannot identify whether there are hot or cold data conditions.

[0039] Based on this, the present invention proposes a hot and cold data identification test method based on solid-state drives, which aims to identify the judgment conditions for whether the solid-state drive under test has hot and cold data.

[0040] In one embodiment, such as Figure 1 As shown, a method for identifying hot and cold data based on solid-state drives is provided. This method includes:

[0041] Step 102: Perform a secure erase on the solid-state drive under test to ensure that the solid-state drive under test is in the same initial state as the solid-state drive under test.

[0042] Step 104: Use FIO to write the entire disk sequentially and record the performance per second as the first array. Stop the test when the recorded performance is less than half of the performance in the first second for a continuous period of time.

[0043] Step 106: Change the block size sequentially and repeat the above erase and write-to-disk operations, recording the corresponding performance per second as the corresponding array;

[0044] Step 108: Determine whether there is hot or cold data based on whether the performance recorded at different block sizes is consistent.

[0045] In this embodiment, a hot and cold data identification test method based on solid-state drives is proposed. This method designs a test method for whether solid-state drives have hot and cold data judgment conditions. It uses FIO to write different data sizes and perform multiple read and write accesses on the disk under test to confirm whether the solid-state drive under test uses data block size and data volume as hot and cold data judgment conditions.

[0046] Specifically, first, the slave disk is safely erased using the NVME format command to ensure the consistency of the disk's initial state.

[0047] Next, FIO is used to write the entire disk sequentially, and the performance per second is recorded as the first array. The test is stopped when the recorded performance is less than half of the performance in the first second for a continuous period of time.

[0048] Then, the block size is changed sequentially, and the erase and full-disk write operations are repeated, recording the corresponding performance per second in an array. Finally, the consistency of performance across different block sizes is used to determine whether there is hot or cold data.

[0049] In one embodiment, the step of determining whether there is hot or cold data identification based on whether the performance recorded at different block sizes is consistent further includes: the test result of the solid-state drive under test is either using the write request data size and data volume as the hot or cold data threshold judgment standard or the judgment standard does not exist.

[0050] Specifically, FIO was used to write different data sizes to the tested disk and perform multiple read accesses followed by writes to confirm whether the tested SSD met the criteria for determining hot and cold data. The test results for the tested disk included the following two cases: 1) the write request data size and data volume were used as the criteria for determining hot and cold data, and 2) the criteria described in 1 did not exist.

[0051] In the above embodiment, a secure erase is performed on the SSD under test to ensure a consistent initial state. The entire disk is sequentially written using FIO, and the performance per second is recorded as the first array. The test is stopped when the recorded performance is less than half of the first second's performance for a continuous period. The block size is changed sequentially, and the erase and full-disk write operations are repeated, with the corresponding performance per second recorded as the corresponding array. The consistency of the recorded performance across different block sizes determines whether hot or cold data is present. This solution utilizes FIO to write different data sizes and perform multiple read / write accesses on the disk under test to confirm whether the SSD under test uses block size and data volume as criteria for hot / cold data identification, thus identifying whether the SSD under test has hot or cold data.

[0052] In one embodiment, such as Figure 2 As shown, a method for identifying hot and cold data based on solid-state drives is provided. This method also includes:

[0053] Step 202: Use FIO to write the entire disk sequentially, with a queue depth of 8192 / block size, 1 thread, and a block size of 1024KB. Record the performance per second as the first array. Stop the test when the performance is less than half of the performance in the first second for 5 consecutive seconds.

[0054] Step 204: Change the block size to 512KB, 256KB, 128K, 64K, 32K, and 16KB in sequence, and repeat the above erase and write full disk operations. Record the performance per second for the corresponding second array, third array, fourth array, fifth array, sixth array, and seventh array each time.

[0055] In one embodiment, a method for identifying hot and cold data based on solid-state drives is provided, and the test environment is as follows:

[0056] Hardware Requirements: The computer under test is an ASUS Z690. The specific model is not mandatory. This embodiment preferably uses the ASUS Z690. The solid-state drive (SSD) under test is a TLC SSD that supports simulated SLC caching. The testing method described herein is applicable to various multi-layer storage unit SSDs with simulated SLC. For ease of description, the most mainstream TLC SSD is used as the test and description object.

[0057] Software Requirements: The operating system is Linux Red Hat; the testing tools are FIO (an open-source testing tool mainly used for stress testing and performance verification of hard drives) and Iostat (an open-source tool for monitoring the I / O load of system devices); the test scripts are self-written batch scripts. There are no strict requirements for the testing OS and tools, but Red Hat, FIO, and Iostat are preferred in this embodiment.

[0058] Specifically, refer to Figure 3 The diagram shown illustrates a test method for identifying hot and cold data based on solid-state drives. The test process includes:

[0059] Use the NVME format command to perform a secure erase on the slave disk to ensure the consistency of the disk's initial state.

[0060] The entire disk is written sequentially using FIO, with a queue depth of 8192 / block size, 1 thread, and a block size of 1024KB. The performance per second is recorded using Iostat as an array A, where the performance in the first second is A1. Another process determines whether the test will stop if the performance is less than (performance in the first second) / 2 for 5 consecutive seconds.

[0061] Change the block size to 512KB, 256KB, 128K, 64K, 32K, and 16KB, and repeat steps 1 to 2. Record the performance per second as arrays B, C, D, E, and F each time.

[0062] The presence of hot and cold data can be determined by whether the performance is consistent across different block sizes.

[0063] Here are two examples of test results for different products:

[0064] Product Y: Reference Figure 4 and Figure 5 The test results diagram shows that when writing large data blocks of 64KB to 1024KB consecutively, the data is identified as cold data, and data beyond 3GB is directly written to the TLC to avoid migration. Conversely, smaller data blocks (less than 64KB) are identified as hot data and are written continuously until the SLC is exhausted before being written to the TLC. Product Z: For blocks of 16KB to 1024KB, neither hot nor cold data is identified; the SLC is exhausted before writing to the TLC.

[0065] In this embodiment, it is possible to confirm whether the solid-state drive uses data block size and data volume as criteria for determining hot and cold data.

[0066] It should be understood that, although Figures 1-5The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figures 1-5 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0067] In one embodiment, such as Figure 6 As shown, a solid-state drive-based hot and cold data identification and testing device 600 is provided, the device comprising:

[0068] A secure erase module 601 is used to securely erase the solid-state drive under test so that the initial state of the solid-state drive under test is consistent.

[0069] The performance recording module 602 is used to sequentially write the entire disk using FIO, and record the performance per second as the first array. When the recorded performance is less than half of the performance in the first second for a continuous period of time, the test is stopped.

[0070] The repeat test module 603 is used to change the size of the block sequentially and repeatedly perform the above-mentioned erasure and write-to-full-disk operations, and record the corresponding performance per second as a corresponding array.

[0071] Result judgment module 604 is used to determine whether there is hot or cold data based on whether the performance recorded at different block sizes is consistent.

[0072] In one embodiment, the performance recording module 602 is further configured to:

[0073] The entire disk is written sequentially using FIO, with a queue depth of 8192 / block size, 1 thread, and a block size of 1024KB. The performance per second is recorded in the first array. The test is stopped when the performance is less than half of the performance in the first second for 5 consecutive seconds.

[0074] In one embodiment, the repeat testing module 603 is further configured to:

[0075] The block size is changed sequentially to 512KB, 256KB, 128K, 64K, 32K, and 16KB. The erase and write operations are repeated, and the performance per second is recorded for the corresponding second, third, fourth, fifth, sixth, and seventh arrays.

[0076] In one embodiment, the result determination module 604 is further configured to:

[0077] The test results for the solid-state drive under test are determined by using the write request data size and data volume as the criteria for judging hot and cold data thresholds, or by the absence of such criteria.

[0078] For specific limitations on the solid-state drive-based hot and cold data identification test device, please refer to the limitations on the solid-state drive-based hot and cold data identification test method mentioned above, which will not be repeated here.

[0079] In one embodiment, a computer device is provided, the internal structure of which can be shown as follows: Figure 7 As shown, the computer device includes a processor, memory, and a network interface connected via a device bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores operating devices, computer programs, and a database. The internal memory provides an environment for the operation of the operating devices and computer programs stored in the non-volatile storage medium. The network interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a solid-state drive-based method for identifying hot and cold data during testing.

[0080] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0081] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in the various method embodiments described above.

[0082] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the various method embodiments described above.

[0083] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0084] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0085] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for identifying hot and cold data based on a solid-state drive, the method comprising: Perform a secure erase on the solid-state drive under test to bring it back to its initial state. The entire disk is written sequentially using FIO, with a queue depth of 8192 / block size, 1 thread, and a block size of 1024KB. The performance per second is recorded in the first array. The test is stopped when the performance is less than half of the performance in the first second for 5 consecutive seconds. The block size is changed sequentially, and the above erase and write operations are repeated. The corresponding performance per second is recorded as the corresponding array. The presence of hot and cold data can be determined by whether the performance recorded is consistent across different block sizes.

2. The method for identifying hot and cold data based on solid-state drives according to claim 1, characterized in that, The step of sequentially changing the block size and repeatedly performing the erase and write-to-disk operations, and recording the corresponding performance per second as a corresponding array, also includes: The block size is changed sequentially to 512KB, 256KB, 128K, 64K, 32K, and 16KB. The erase and write operations are repeated, and the performance per second is recorded for the corresponding second, third, fourth, fifth, sixth, and seventh arrays.

3. The method for identifying hot and cold data based on a solid-state drive according to any one of claims 1-2, characterized in that, The step of determining whether there is hot or cold data identification based on whether the performance recorded at different block sizes is consistent also includes: The test results for the solid-state drive under test are determined by using the write request data size and data volume as the criteria for judging hot and cold data thresholds, or by the absence of such criteria.

4. A cold / hot data identification and testing device based on a solid-state drive, characterized in that, The device includes: A secure erase module is used to securely erase the solid-state drive under test to ensure that the initial state of the solid-state drive under test is consistent. The performance recording module is used to sequentially write the entire disk using FIO, with a queue depth of 8192 / block size, 1 thread, and a block size of 1024KB. The performance per second is recorded as the first array. The test is terminated when the performance is less than half of the performance in the first second for 5 consecutive seconds. The repeated testing module is used to change the size of the block sequentially and repeatedly perform the above-mentioned erasure and write-to-full-disk operations, and record the corresponding performance per second as a corresponding array. The result judgment module is used to determine whether there is hot or cold data based on whether the performance recorded at different block sizes is consistent.

5. The solid-state drive-based cold and hot data identification and testing device according to claim 4, characterized in that, The repeat testing module is also used for: The block size is changed sequentially to 512KB, 256KB, 128K, 64K, 32K, and 16KB. The erase and write operations are repeated, and the performance per second is recorded for the corresponding second, third, fourth, fifth, sixth, and seventh arrays.

6. The cold and hot data identification and testing device based on solid-state drive according to any one of claims 4-5, characterized in that, The result judgment module is also used for: The test results for the solid-state drive under test are determined by using the write request data size and data volume as the criteria for judging hot and cold data thresholds, or by the absence of such criteria.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 3.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 3.

Images