Hard disk performance test method, system and device
By conducting first and secondary performance tests on cached hard disks, and combining alternating write operations, the performance indicators of slowing inside and slowing outside are determined, which solves the problem of how to quickly and accurately test the performance of different types of cached hard disks, and achieves a comprehensive coverage of hard disk performance.
Patent Information
- Application Number
- CN202411942953.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-26
- Publication Date
- 2025-04-25
AI Technical Summary
Before deploying a distributed storage system, how to quickly and accurately test the performance indicators of different types of cached hard disks through a general method, especially slowing down internal and external performance.
A hard disk performance testing method is adopted, including performing the first performance test on the target hard disk, obtaining the continuous write time, performing continuous write and random write operations alternately, performing secondary performance tests, and determining multiple performance indicators based on the test data and log data.
Through two rounds of performance testing, different types of cached hard disks can be covered, and the internal and external performance can be accurately tested, solving the problem of how to quickly and accurately test hard disk performance.
Smart Images
Figure CN119883758A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of storage systems, and in particular to a hard disk performance testing method, system and device. Background Art
[0002] In a distributed storage system, cache hard disks provide users with cache space for reading and writing data on distributed storage gateways. This cache mechanism can significantly reduce the latency of users accessing file sharing data, thereby improving the system's response speed and overall performance. Therefore, before deploying a distributed storage system, it is particularly important to quickly and accurately test the performance indicators of cache hard disks.
[0003] However, different types of cache hard drives have different characteristics. On the one hand, some solid-state drives (SSDs) are equipped with a single-level cell (SLC) cache. When the amount of data written is small, the data is mainly stored in the SLC cache. At this time, the test results in the cache-in performance; when the amount of data written is large and fills up the SLC cache, the data is stored in other areas outside the SLC cache. At this time, the test results in the cache-out performance. On the other hand, some solid-state drives and hard disk drives (HDDs) are not equipped with SLC caches. The test results are equivalent to the cache-out performance after the SLC cache is full.
[0004] For distributed storage systems, both cache internal performance and cache external performance are important indicators of cache hard disk performance. Therefore, how to use a universal method to cover different types of cache hard disks and quickly and accurately test their various performance indicators is an urgent problem to be solved in this application. Summary of the invention
[0005] The present application provides a hard disk performance testing method, system and device to solve the problem of how to cover different types of cache hard disks through a universal method and quickly and accurately test their various performance indicators.
[0006] A first aspect of the present application provides a hard disk performance test method, which is applied to a hard disk performance test device of a distributed storage system, wherein the distributed storage system further includes multiple cache hard disks, and the target hard disk is any one of the multiple cache hard disks; for the target hard disk, the method includes:
[0007] Performing a first performance test on the target hard disk to obtain first test data; wherein the amount of data written in the first performance test is less than the predicted buffer capacity of the target hard disk;
[0008] Get the continuous writing time of the target hard disk;
[0009] Alternately performing continuous write operations and random write operations on the target hard disk, and recording log data of the instantaneous write speed, until the log data indicates that the instantaneous write speed meets the preset speed requirement, or the duration of the alternating write operation reaches the continuous write time, then determining that the amount of data written by the alternating write operation is greater than the predicted buffer content capacity;
[0010] Performing a secondary performance test on the target hard disk to obtain second test data;
[0011] A plurality of performance indicators of the target hard disk are obtained according to the first test data, the second test data and the log data.
[0012] In a possible design, obtaining the sustained writing time of the target hard disk includes:
[0013] Get the hard disk capacity and hard disk type of the target hard disk;
[0014] According to the hard disk capacity and hard disk type, the expected writing time is obtained; the expected writing time refers to the time expected to fill up the hard disk capacity;
[0015] The shorter of the expected writing time and the preset upper limit time of the test is determined as the continuous writing time.
[0016] In a possible design, the hard disk type is a first type or a second type; wherein the first type is used to indicate that the target hard disk includes a mechanical hard disk, and the second type is used to indicate that the target hard disk does not include a mechanical hard disk;
[0017] When the hard disk type is the second type, the expected write time is obtained according to the hard disk capacity and hard disk type, including:
[0018] Correcting the hard disk capacity according to a preset proportionality factor; wherein the hard disk capacity after correction is smaller than the hard disk capacity before correction;
[0019] Based on the corrected hard disk capacity, the expected write time is obtained.
[0020] In one possible design, the upper limit time of the test corresponding to the first type of target hard disk is shorter than the upper limit time of the test corresponding to the second type of target hard disk.
[0021] In a possible design, obtaining the hard disk capacity and hard disk type of the target hard disk includes:
[0022] Fields for obtaining the attribute information of the target hard disk;
[0023] Parse the fields of the attribute information to obtain the hard disk capacity and hard disk type.
[0024] In a possible design, sequential write operations and random write operations are alternately performed on a target hard disk, including:
[0025] Within a first preset time, a continuous write operation and a random write operation are alternately performed on the target hard disk for the first time; wherein the amount of data alternately written for the first time is less than the predicted buffer content amount;
[0026] The continuous write operation and the random write operation are alternately performed on the target hard disk for a second time.
[0027] In a possible design, the first test data includes: random read and write latency, continuous read bandwidth, and random read IOPS within a second preset time; the second test data includes: random read and write latency, continuous read bandwidth, and random read IOPS within a third preset time;
[0028] According to the first test data, the second test data and the log data, multiple performance indicators of the target hard disk are obtained, including:
[0029] Determine the actual cache capacity of the target hard disk based on the log data;
[0030] When the actual cache capacity is greater than zero, the cache internal performance of the target hard disk is obtained according to the first test data and the log data, and the cache external performance of the target hard disk is obtained according to the second test data.
[0031] In a possible design, after obtaining multiple performance indicators of the target hard disk according to the first test data, the second test data and the log data, the method further includes:
[0032] An untested cache hard disk is determined as a new target hard disk to test multiple performance indicators of the new target hard disk.
[0033] A second aspect of the present application provides a hard disk performance testing system, the system comprising:
[0034] A plurality of cache hard disks, and a hard disk performance testing device respectively connected to each cache hard disk for communication, wherein the hard disk performance testing device is used to implement the hard disk performance testing method as described in any one of the first aspects.
[0035] A third aspect of the present application provides a hard disk performance test device, which is located in a hard disk performance test system, the hard disk performance test system also includes multiple cache hard disks, and the target hard disk is any one of the multiple cache hard disks; for the target hard disk, the device includes:
[0036] The first test module is used to perform a first performance test on the target hard disk to obtain first test data; wherein the amount of data written in the first performance test is less than the predicted buffer capacity of the target hard disk;
[0037] A time acquisition module is used to obtain the continuous writing time of the target hard disk;
[0038] A continuous writing module is used to alternately perform continuous writing operations and random writing operations on the target hard disk and record log data of the instantaneous writing speed until the log data indicates that the instantaneous writing speed meets the preset speed requirement, or the duration of the alternating writing operation reaches the continuous writing time, then it is determined that the amount of data written by the alternating writing operation is greater than the predicted buffer content capacity;
[0039] A secondary test module, used for performing a secondary performance test on the target hard disk to obtain second test data;
[0040] The performance determination module is used to obtain multiple performance indicators of the target hard disk according to the first test data, the second test data and the log data.
[0041] A fourth aspect of the present application provides an electronic device, comprising: a memory, and a memory communicatively connected to a processor;
[0042] Memory stores computer-executable instructions;
[0043] When the processor executes the computer-executable instructions stored in the memory, it is used to implement any hard disk performance testing method of the first aspect.
[0044] The fifth aspect of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement any hard disk performance testing method of the first aspect.
[0045] The sixth aspect of the present application provides a computer program product, including a computer program, which, when executed by a processor, is used to implement any hard disk performance testing method of the first aspect.
[0046] The present application provides a hard disk performance testing method, system and device. For a target hard disk of a distributed storage system, the method includes: performing an initial performance test on the target hard disk to obtain first test data; obtaining the continuous write time of the target hard disk; performing continuous write operations and random write operations on the target hard disk alternately, and recording log data of the instantaneous write speed until the log data indicates that the instantaneous write speed meets a preset speed requirement, or the duration of the alternating write operation reaches the continuous write time; performing a secondary performance test on the target hard disk to obtain second test data; and obtaining multiple performance indicators of the target hard disk based on the first test data, the second test data and the log data. The following technical effects are achieved: through two rounds of performance tests, the performance of hard disks equipped with SLC cache can be tested, and the performance of hard disks not equipped with SLC cache can also be tested, which solves the problem of how to cover different types of cache hard disks through a universal method; the in-cache performance of the target hard disk is determined through the first performance test, the out-cache performance of the target hard disk is determined through the second performance test, and the in-cache and out-cache writing speed of the target hard disk is determined through alternating write operations, which solves the problem of how to accurately test various performance indicators of the target hard disk; through alternating write operations, it is ensured that the SLC cache of the target hard disk is filled, and at the same time, the speed requirement and duration of the alternating write operation are limited, which solves the problem of how to quickly test various performance indicators of the target hard disk. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0048] Figure 1 Schematic diagram of the process of the hard disk performance testing method provided in the embodiment of the present application Figure 1 ;
[0049] Figure 2 Schematic diagram of the process of the hard disk performance testing method provided in the embodiment of the present application Figure 2 ;
[0050] Figure 3 Schematic diagram of the process of the hard disk performance testing method provided in the embodiment of the present application Figure 3 ;
[0051] Figure 4 A schematic diagram of the structure of a hard disk performance test system provided in an embodiment of the present application;
[0052] Figure 5 A schematic diagram of the structure of a hard disk performance testing device provided in an embodiment of the present application;
[0053] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
[0054] Reference numerals:
[0055] 410-cache hard disk; 420-hard disk performance test device;
[0056] 510 - first test module; 520 - time acquisition module; 530 - continuous writing module; 540 - second test module; 550 - performance determination module;
[0057] 610 - processor; 620 - memory; 630 - communication component; 640 - bus. DETAILED DESCRIPTION
[0058] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0059] In the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference. It should be noted that in the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way. In the present application, "at least one" refers to one or more, and "more" refers to two or more.
[0060] It should be noted that the "at..." in this application can be the instant when a certain situation occurs, or it can be a period of time after a certain situation occurs, and this application does not make specific limitations on this. In addition, the hard disk performance testing method provided in this application is only an example, and the hard disk performance testing method can also include more or less content. The user information (including but not limited to user device information and user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data, etc.) involved in one or more embodiments of this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0061] In order to clearly describe the technical solution of this application, some terms and technologies involved in this application are briefly introduced below:
[0062] Distributed Storage System (DSS): refers to a data storage architecture that stores data in multiple physical nodes in a distributed manner and connects them through a network to provide high availability and scalability.
[0063] Solid State Drive (SSD): refers to a hard disk made of a solid-state electronic storage chip array, which has the characteristics of fast reading and writing speed, shock resistance and drop resistance.
[0064] Single-Level Cell (SLC) cache: refers to a technology in SSDs where each cell stores only 1 bit of data. It has higher stability and faster read and write speeds and is often used as a cache.
[0065] Hard Disk Drive (HDD): refers to a traditional hard disk that uses magnetic disks to store data and has a slower read and write speed than an SSD.
[0066] Performance testing of cache hard disks: This refers to measuring the hard disk's performance indicators such as read and write speeds and response time under different conditions through a series of test methods.
[0067] Sequential Write Operation (SWO): refers to the operation of writing data to the hard disk in a continuous manner.
[0068] Random Write Operation (RWO): refers to the operation of writing data to the hard disk in a random manner.
[0069] Solid State Hybrid Drive (SSHD): A hybrid hard drive that combines the features of SSD and HDD. It has the read and write speed of SSD and the storage capacity of HDD.
[0070] SATA SSD: refers to a solid-state drive that uses the Serial Advanced Technology Attachment (SATA) interface to connect.
[0071] NVMe SSD: refers to a solid-state drive that uses the Non-Volatile Memory Express (NVMe) protocol, which has higher read and write speeds and lower latency.
[0072] Cache capacity: refers to the storage capacity of the SLC cache.
[0073] Random Read Latency (RRL): refers to the time interval from the issuance of a request to the start of data transmission when performing a random read operation.
[0074] Random Write Latency (RWL): refers to the time interval from the issuance of a request to the start of data writing when performing a random write operation.
[0075] Sequential Read Bandwidth (SRB): refers to the amount of data that can be transmitted per second when performing continuous read operations.
[0076] Random read Input / Output Operations Per Second (IOPS): refers to the number of random read input / output operations that can be processed per second.
[0077] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The present application will be described below in conjunction with the accompanying drawings.
[0078] In order to clearly understand the technical solution of the present application, the solution of the prior art is first introduced in detail.
[0079] In a distributed storage system, cache hard disks provide users with cache space for reading and writing data on distributed storage gateways. This cache mechanism can significantly reduce the latency of users accessing file sharing data, thereby improving the system's response speed and overall performance.
[0080] However, different types of cache hard drives have different characteristics. Among them, the type of hard drive is a key factor. There are significant differences in performance between solid-state drives and mechanical hard drives. Even hard drives of the same type may exhibit different performance characteristics due to different technologies. On the one hand, some solid-state drives are equipped with SLC cache, which can significantly improve performance when the amount of data written is small. When the amount of data written is small, the data is mainly stored in the faster SLC cache. At this time, the test results are in-cache performance; when the amount of data written is large, so that the cache capacity is filled, the data is stored in other slower areas outside the SLC cache. At this time, the test results are out-of-cache performance. On the other hand, some solid-state drives and mechanical hard drives (Hard Disk Drive, HDD) are not equipped with SLC cache, and the test results are equivalent to out-of-cache performance after the cache capacity is full.
[0081] Before deploying a distributed storage system, it is particularly important to quickly and accurately test the performance indicators of the cache hard disk. This is because only by understanding the actual performance of the hard disk can we choose the appropriate hard disk type according to actual needs and optimize the system configuration to ensure high performance and stability of the system.
[0082] For distributed storage systems, both cache internal performance and cache external performance are important indicators of cache hard disk performance. Because different types of hard disks have different characteristics and testing requirements, how to use a universal method to cover different types of cache hard disks and quickly and accurately test their various performance indicators is an urgent problem to be solved in this application.
[0083] Therefore, in response to the above technical issues, it was found in the research that in order to solve the problem, the cache hard disk can be tested twice. The first test ensures that the written data is stored in the possible SLC cache, and the second test ensures that the written data is stored in other areas outside the possible SLC cache, so as to cover different types of hard disks and improve the versatility of the test method; at the same time, the total time of the test process is controlled to quickly complete the test of the hard disk performance indicators.
[0084] Based on the above creative discovery, the technical solution of the present application is proposed. The embodiments of the present application are introduced below in conjunction with the drawings of the specification.
[0085] Figure 1 Schematic diagram of the process of the hard disk performance testing method provided in the embodiment of the present application Figure 1 .like Figure 1As shown, in the embodiment of the present application, the execution subject may be a hard disk performance test device of a distributed storage system, the device may be located in an electronic device, and the device may be a data processing server. The distributed storage system also includes multiple cache hard disks, and the target hard disk is any one of the multiple cache hard disks; for the target hard disk, the hard disk performance test method provided in the embodiment of the present application includes the following steps:
[0086] S101, performing a first performance test on a target hard disk to obtain first test data; wherein the amount of data written in the first performance test is smaller than the predicted buffer capacity of the target hard disk.
[0087] Specifically, the test process can be divided into two rounds for multiple performance indicators. This is because if the target hard drive is equipped with an SLC cache, to test the target hard drive's out-of-cache performance, the hard drive capacity needs to be filled, and it takes a certain amount of time for the target hard drive to release cache migration data. Therefore, in order to quickly obtain multiple performance indicators of the target hard drive, the target hard drive is subjected to two rounds of performance tests. When the target hard drive is equipped with an SLC cache, the first performance test obtains the target hard drive's in-cache performance, and the second performance test obtains the target hard drive's out-of-cache performance; when the target hard drive is not equipped with an SLC cache, it can also be regarded as two rounds of ordinary performance tests, obtaining two sets of test data under different conditions.
[0088] For the first performance test, the amount of data written should be less than the predicted cache capacity of the target hard disk. This is to ensure that the SLC cache of the target hard disk can fully function in the first performance test, so as to more accurately evaluate its performance. The predicted cache capacity refers to the predicted capacity of the SLC cache, which can be predicted based on the capacity of the SLC cache of other cache hard disks produced or installed during the same period.
[0089] S102: Obtain the continuous writing time of the target hard disk.
[0090] Specifically, before performing subsequent alternating write operations, it is necessary to obtain the target hard disk's continuous write time, in order to ensure that the cache capacity is fully written. In the secondary performance test, the target hard disk's out-of-cache performance is tested. The continuous write time can be calculated and set based on the cache capacity, total hard disk capacity, and write speed of other cache hard disks produced or installed during the same period.
[0091] This time can be estimated or set based on the specifications, model and actual application scenario of the hard disk.
[0092] It should be noted that the execution order between S101 and S102 is not limited. Figure 1 As shown, S101 is executed first and then S102. Alternatively, S102 may be executed first and then S101. Alternatively, S101 and S102 may be executed simultaneously.
[0093] S103. Alternately perform continuous write operations and random write operations on the target hard disk, and record log data of the instantaneous write speed until the log data indicates that the instantaneous write speed meets the preset speed requirement, or the duration of the alternating write operation reaches the continuous write time, then it is determined that the amount of data written by the alternating write operation is greater than the predicted buffer capacity.
[0094] Specifically, the purpose of alternating write operations is to simulate different write modes that the target hard disk may face in actual applications, so as to more comprehensively determine its performance; the recorded log data should record the instantaneous write speed of each write operation in detail for subsequent analysis.
[0095] The alternating write operation will continue until the log data indicates that the instantaneous write speed meets the preset speed requirement, or the duration of the alternating write operation reaches the continuous write time, then the amount of data written by the alternating write operation, specifically the amount of data written by the alternating write operation and the first performance test, is greater than the predicted buffer capacity, and the target hard disk is filled. The preset speed requirement can be that the instantaneous write speed drops and reaches a certain stable threshold; or that the instantaneous write speed fluctuates.
[0096] S104: Perform a secondary performance test on the target hard disk to obtain second test data.
[0097] Specifically, after the previous alternating write operation, if the target hard disk is equipped with an SLC cache, then the cache is also fully written, and a secondary performance test is performed on the target hard disk. At this time, the result of the continued test is the cache performance of the target hard disk, or the test data of the latter of the two different situations.
[0098] S105 . Obtain multiple performance indicators of the target hard disk according to the first test data, the second test data and the log data.
[0099] Specifically, after completing two rounds of performance testing, it is necessary to analyze the target hard disk's out-of-cache write speed. When the target hard disk's cache is full, performance will suddenly drop significantly. For example, based on the log data of the instantaneous write speed, when the instantaneous write speed increases continuously or is higher than the average of subsequent speeds, it means that the critical point of the cache being full has been reached. The average value of the speed before the critical point can be calculated as a reference value for the in-cache write speed; the average value of the speed after the critical point can be calculated as a reference value for the out-of-cache write speed.
[0100] The target hard disk is any one of the multiple cache hard disks of the distributed storage system. The performance test method of other cache hard disks of the distributed storage system is the same as the performance test method of the target hard disk, and this embodiment will not be repeated. The hard disk performance test method can test the performance of the target hard disk in the distributed storage system. These performance indicators provide an important reference for the deployment and optimization of the distributed storage system, and help ensure the high performance and stability of the distributed storage system.
[0101] A hard disk performance testing method provided in an embodiment of the present application, for a target hard disk of a distributed storage system, the method includes: performing an initial performance test on the target hard disk to obtain first test data; obtaining the continuous write time of the target hard disk; alternately performing continuous write operations and random write operations on the target hard disk, and recording log data of the instantaneous write speed, until the log data indicates that the instantaneous write speed meets a preset speed requirement, or the duration of the alternating write operation reaches the continuous write time; performing a secondary performance test on the target hard disk to obtain second test data; and obtaining multiple performance indicators of the target hard disk based on the first test data, the second test data and the log data. The following technical effects are achieved: through two rounds of performance tests, the performance of hard disks equipped with SLC cache can be tested, and the performance of hard disks not equipped with SLC cache can also be tested, which solves the problem of how to cover different types of cache hard disks through a universal method; the in-cache performance of the target hard disk is determined through the first performance test, the out-cache performance of the target hard disk is determined through the second performance test, and the in-cache and out-cache writing speeds of the target hard disk are determined through alternating write operations, which solves the problem of how to accurately test various performance indicators of the target hard disk; through alternating write operations, it is ensured that the cache content of the target hard disk is filled, and at the same time, the speed requirements and duration of the alternating write operations are limited, which solves the problem of how to quickly test various performance indicators of the target hard disk.
[0102] Figure 2 Schematic diagram of the process of the hard disk performance testing method provided in the embodiment of the present application Figure 2 ,like Figure 2 As shown, the hard disk performance testing method provided in the embodiment of the present application is Figure 1 Based on the hard disk performance test method provided in the embodiment, the first test data includes: random read and write latency, continuous read bandwidth and random read IOPS within the second preset time; the second test data includes: random read and write latency, continuous read bandwidth and random read IOPS within the third preset time, then the hard disk performance test method provided in the embodiment includes the following steps.
[0103] S201, performing an initial performance test on a target hard disk to obtain first test data.
[0104] After executing S201, the hard disk capacity and hard disk type of the target hard disk are obtained, that is, the following S202 and S203 are executed.
[0105] S202: Obtain fields of attribute information of a target hard disk.
[0106] S203: Parse the fields of the attribute information to obtain the hard disk capacity and hard disk type.
[0107] Specifically, the attribute information of the target hard disk includes the hard disk capacity and the hard disk type, and the hard disk type is the first type or the second type; wherein the first type is used to indicate that the target hard disk includes a mechanical hard disk, for example, it can be an HDD or SSHD, etc.; the second type is used to indicate that the target hard disk does not include a mechanical hard disk, for example, it can be a SATA SSD or NVMe SSD, etc.
[0108] For example, in a Linux system, you can determine whether the target hard disk is an SSD or SSHD based on the rotation attribute field of the target hard disk (rota value, used to indicate whether it includes rotating parts); you can also determine whether the target hard disk is a SATA SSD or NVMe SSD based on whether the device name includes fields such as NVMe, SATA, or SSD; you can also get the hard disk type of the target hard disk through related commands or capacity attribute fields.
[0109] S204: Obtain expected writing time according to the hard disk capacity and hard disk type.
[0110] Among them, the expected write time refers to the expected time to fill up the hard disk capacity; the expected write time can be determined by technicians based on cache hard disks of similar capacity and the same type; it can also be calculated based on the hard disk capacity and hard disk type, specifically, the average write speed of the target hard disk is predicted based on the hard disk type and historical experience, and the expected write time is calculated based on the hard disk capacity and the average write speed.
[0111] In a possible design, when the hard disk type is the second type, S204 includes:
[0112] S2041. Correct the hard disk capacity according to a preset proportionality factor; wherein the corrected hard disk capacity is smaller than the hard disk capacity before correction.
[0113] S2042. Obtain expected writing time according to the corrected hard disk capacity.
[0114] Specifically, there are significant differences in the changing patterns of the instantaneous write speeds of mechanical hard disks and solid-state hard disks when they are full. When a mechanical hard disk writes data, as the amount of data increases, the distance the head needs to move will also increase, which will cause the instantaneous write speed to gradually decrease; especially when writing to the inner circle of the hard disk, due to the increase in track density, the head addressing time becomes longer, and the write speed will be further reduced. When writing data to a solid-state hard disk, the instantaneous write speed usually does not drop significantly with the increase in data volume, because there are no mechanical moving parts inside, and there are no restrictions on head addressing and turntable speed; although when the solid-state hard disk is close to being full, its write speed may be slightly affected by internal management mechanisms such as garbage collection and wear leveling, but these effects are usually much smaller than the speed drop caused by the head addressing and turntable speed restrictions of the mechanical hard disk.
[0115] At the same time, for SATA SSD and NVMe SSD, especially for the target hard disk with large enough hard disk capacity, this test may not cover the test of buffer-out performance due to the limitation of continuous writing time; and considering that when the hard disk capacity is large enough, the importance of buffer-out performance will also decrease, therefore, when the target hard disk is of the second type, the write hard disk capacity is limited to a certain range. For example, if the proportional coefficient is set to 80%, the hard disk capacity after correction is 80% of the hard disk capacity before correction.
[0116] The technical effect of the embodiment of the present application is that the hard disk capacity of the second type of target hard disk is corrected by the proportional coefficient, thereby reducing the test time of the solid state hard disk.
[0117] In a possible design, when the hard disk type is the first type, the preset speed requirement in S103 specifically refers to the instantaneous writing speed continuously decreasing. At this time, it can be considered that the hard disk capacity has been fully written.
[0118] The technical effect of the embodiment of the present application is that, by determining whether to stop the alternating write operation of the first type of target hard disk through the changing trend of the instantaneous write speed, the test time of the mechanical hard disk is reduced.
[0119] S205: Determine the shorter time between the expected writing time and the preset upper limit time of the test as the continuous writing time.
[0120] In one possible design, the upper limit time of the test corresponding to the first type of target hard disk is shorter than the upper limit time of the test corresponding to the second type of target hard disk.
[0121] Specifically, the cache capacity of SATA SSD is usually small, and the SLC cache strategies of NVMe SSD are various and complex, and some even have several different stages. However, usually we only need to pay attention to the performance outside the cache, so in order to ensure that the cache capacity of the NVMe SSD is filled, the test upper limit time corresponding to the second type of target hard disk needs to be set longer.
[0122] S206: Within a first preset time, alternately perform a continuous write operation and a random write operation on the target hard disk for the first time.
[0123] Specifically, the amount of data written in the first alternating write is less than the predicted amount of buffer content, so the data written in the first alternating write is written into the possible SLC buffer. When the target hard disk is equipped with an SLC buffer, the buffer write speed can be determined based on the log data corresponding to the first alternating write.
[0124] S207, alternately performing a continuous write operation and a random write operation on the target hard disk for a second time.
[0125] Specifically, the first part of the data written twice is written into the SLC cache that may be equipped. After the cache is full, the data written twice is written outside the cache. When the target hard disk is equipped with an SLC cache, the speed of writing outside the cache can be determined based on the log data corresponding to the second alternating write.
[0126] S208. Record log data of the instantaneous write speed until the log data indicates that the instantaneous write speed meets the preset speed requirement, or the duration of the alternating write operation reaches the continuous write time, then determine that the amount of data written by the alternating write operation is greater than the predicted buffer capacity.
[0127] S209: Perform a secondary performance test on the target hard disk to obtain second test data.
[0128] S210: Determine the actual buffer capacity of the target hard disk according to the log data.
[0129] Specifically, after completing two rounds of performance testing, the log data is traversed to determine the critical point of the cache capacity, thereby determining whether the target hard disk is equipped with an SLC cache and the actual cache capacity.
[0130] S211. When the actual cache capacity is greater than zero, obtain the cache performance of the target hard disk according to the first test data and the log data, and obtain the cache performance of the target hard disk according to the second test data.
[0131] Specifically, when the actual cache capacity is greater than zero, it means that the target hard disk is equipped with an SLC cache, and the in-cache performance and out-cache performance of the target hard disk can be determined. When the actual cache capacity is equal to zero, it means that the target hard disk is not equipped with an SLC cache, and multiple performance indicators of the target hard disk under different conditions are obtained based on the first test data, the log data, and the second test data.
[0132] Figure 3 Schematic diagram of the process of the hard disk performance testing method provided in the embodiment of the present application Figure 3 ,like Figure 3 FIG. 4 shows a specific example of a hard disk performance test method, including:
[0133] S301, obtaining hard disk capacity and hard disk type.
[0134] S302, random read and write latency, continuous read bandwidth and random read IOPS, each for one minute. The above content, i.e. the first performance test of the target hard disk, includes the following steps: first, use a smaller concurrency to test the random read latency and random write latency of the target hard disk, each for one minute; then, test the continuous read bandwidth for one minute; finally, increase the concurrency to test the random read IOPS. In these four tests, the amount of data written is small. If the target hard disk is equipped with SLC cache, the performance within the cache is tested at this time.
[0135] S303, for the first time, the continuous write operation and the random write operation are performed alternately, each for thirty seconds, for a total of two minutes. The above content is the first alternating write of the target hard disk. Considering that the cache capacity of the SSHD equipped with SLC cache is usually 8GB or 16GB, and the cache capacity of the SSD equipped with SLC cache is usually from tens of GB to hundreds of GB. Therefore, the amount of data written in each write operation is 2GB, but the writing time is limited to within 30 seconds. The amount of data written in the first alternating write is less than 8GB, and the total time is 2 minutes.
[0136] S304, alternately perform continuous write operation and random write operation twice, each for one minute, until the stop action is satisfied. The above content is the secondary alternating write of the target hard disk. The continuous write operation and the random write operation are alternately performed for 1 minute each, and the test process is dynamically adjusted according to the hard disk type and hard disk capacity. The log data of the instantaneous write speed is recorded in S303 and S304, which is convenient for the subsequent analysis of the write speed inside and outside the buffer and the dynamic adjustment of the test process.
[0137] The second alternating write is enough to cover the target hard disk's cache capacity. Assuming the target hard disk is an SSHD, its cache continuous write speed is several hundred MB / s, assuming it is 300MB / s, 87GB of data can be written in 5 minutes, which is enough to cover its SLC cache; assuming the target hard disk is an NVME SSD, its cache continuous write speed can reach thousands of MB / s, assuming it is 2GB / s, then more than 1TB of data can be written in 10 minutes, which is also enough to cover its SLC cache.
[0138] The stop action means that the log data indicates that the instantaneous write speed meets the preset speed requirement, or the duration of the alternating write operation reaches the continuous write time, where the continuous write time for HDD and SSHD can be 10 minutes, and the continuous write time for SATASSD and NVMe SSD can be 20 minutes.
[0139] S305, random read and write latency, continuous read bandwidth and random read IOPS, each for one minute. The above content, namely the secondary performance test of the target hard disk, is similar to the first performance test and will not be described in detail in this embodiment.
[0140] S306: Determine the internal cache performance and external cache performance of the target hard disk.
[0141] In one possible design, after S211, the method further includes: determining an untested cache hard disk as a new target hard disk to test multiple performance indicators of the new target hard disk.
[0142] Specifically, the hard disk performance test device tests each cache hard disk until all cache hard disks have been tested, or the cache hard disk test is terminated. The hard disk performance test device can test one cache hard disk at a time, or can test multiple cache hard disks at the same time, which is not limited in this embodiment.
[0143] Figure 4 This is a schematic diagram of the structure of the hard disk performance test system provided in the embodiment of the present application. Figure 4 As shown, an embodiment of the present application also provides a hard disk performance testing system, which includes: multiple cache hard disks 410, and a hard disk performance testing device 420 respectively connected to each cache hard disk 410 for communication, and the hard disk performance testing device 420 is used to implement the hard disk performance testing method of the above embodiment.
[0144] The hard disk performance testing system provided in the embodiment of the present application can be executed Figure 1 The technical solution of the method embodiment shown in the figure has the same implementation principle and technical effect as Figure 1 The method embodiments shown are similar and will not be described in detail in the embodiments of the present application.
[0145] Figure 5A schematic diagram of the structure of a hard disk performance testing device provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, in the embodiment of the present application, the hard disk performance test device can be located in an electronic device. The hard disk performance test device includes:
[0146] The first test module 510 is used to perform a first performance test on the target hard disk to obtain first test data; wherein the amount of data written in the first performance test is less than the predicted buffer capacity of the target hard disk;
[0147] The time acquisition module 520 is used to acquire the continuous writing time of the target hard disk;
[0148] The continuous writing module 530 is used to alternately perform continuous writing operations and random writing operations on the target hard disk, and record log data of the instantaneous writing speed until the log data indicates that the instantaneous writing speed meets the preset speed requirement, or the duration of the alternating writing operation reaches the continuous writing time, then it is determined that the amount of data written by the alternating writing operation is greater than the predicted buffer content capacity;
[0149] A secondary testing module 540 is used to perform a secondary performance test on the target hard disk to obtain second test data;
[0150] The performance determination module 550 is used to obtain multiple performance indicators of the target hard disk according to the first test data, the second test data and the log data.
[0151] The hard disk performance testing device provided in the embodiment of the present application can be executed Figure 1 The technical solution of the method embodiment shown in the figure has the same implementation principle and technical effect as Figure 1 The method embodiments shown are similar and will not be described in detail in the embodiments of the present application.
[0152] At the same time, the hard disk performance testing device provided in the embodiment of the present application is further refined based on the hard disk performance testing device provided in the embodiment of the previous application.
[0153] In one possible design, the time acquisition module 520 includes:
[0154] The attribute acquisition module is used to obtain the hard disk capacity and hard disk type of the target hard disk;
[0155] The time calculation module is used to obtain the expected writing time according to the hard disk capacity and hard disk type; wherein the expected writing time refers to the time expected to fill up the hard disk capacity;
[0156] The time updating module is used to determine the shorter time between the expected writing time and the preset test upper limit time as the continuous writing time.
[0157] In a possible design, the hard disk type is a first type or a second type; wherein the first type is used to indicate that the target hard disk includes a mechanical hard disk, and the second type is used to indicate that the target hard disk does not include a mechanical hard disk;
[0158] When the hard disk type is the second type, the time calculation module includes:
[0159] The capacity correction module is used to correct the hard disk capacity according to a preset proportionality coefficient; wherein the hard disk capacity after correction is smaller than the hard disk capacity before correction;
[0160] The time correction module is used to obtain the expected writing time according to the corrected hard disk capacity.
[0161] In one possible design, the upper limit time of the test corresponding to the first type of target hard disk is shorter than the upper limit time of the test corresponding to the second type of target hard disk.
[0162] In a possible design, the attribute acquisition module includes:
[0163] A field acquisition module is used to acquire fields of attribute information of a target hard disk;
[0164] The field parsing module is used to parse the fields of attribute information to obtain the hard disk capacity and hard disk type.
[0165] In a possible design, the continuous writing module 530 includes:
[0166] The first writing module is used to alternately perform a continuous writing operation and a random writing operation on the target hard disk for the first time within a first preset time; wherein the amount of data alternately written for the first time is less than the predicted buffer content amount;
[0167] The secondary writing module is used to perform a secondary alternating continuous writing operation and a random writing operation on the target hard disk.
[0168] In a possible design, the first test data includes: random read and write latency, continuous read bandwidth, and random read IOPS within a second preset time; the second test data includes: random read and write latency, continuous read bandwidth, and random read IOPS within a third preset time;
[0169] The performance determination module 550 includes:
[0170] A capacity determination module, used to determine the actual buffer capacity of the target hard disk according to the log data;
[0171] The performance calculation module is used to obtain the in-cache performance of the target hard disk according to the first test data and the log data when the actual in-cache capacity is greater than zero, and to obtain the out-cache performance of the target hard disk according to the second test data.
[0172] In a possible design, the hard disk performance testing device further includes:
[0173] The hard disk replacement module is used to determine an untested cache hard disk as a new target hard disk to test multiple performance indicators of the new target hard disk.
[0174] The hard disk performance testing device provided in the embodiment of the present application can perform Figures 1 to 3 The technical solution of the method embodiment shown in the figure has the same implementation principle and technical effect as Figures 1 to 3 The method embodiments shown are similar and will not be described in detail in the embodiments of the present application.
[0175] The present application also provides an electronic device, Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 6 As shown, the electronic device includes: the electronic device includes: at least one processor 610 and a memory 620. The electronic device also includes a communication component 630. The processor 610, the memory 620 and the communication component 630 are connected via a bus 640.
[0176] In the specific implementation process, at least one processor 610 executes the computer execution instructions stored in the memory 620, so that the at least one processor 610 is used to implement the hard disk performance testing method of the above embodiment.
[0177] The specific implementation process of the processor 610 can be found in the above-mentioned method embodiment, and its implementation principle and technical effect are similar, so the embodiments of the present application will not be repeated here.
[0178] In the above embodiment, it should be understood that the processor 610 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the application can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.
[0179] The memory 620 may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk storage.
[0180] The bus 640 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 640 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus 640 in the drawings of the present application is not limited to only one bus or one type of bus.
[0181] The above functions implemented by the electronic device and the main control device introduce the scheme provided by the embodiment of the present application. It is understandable that in order to implement the above functions, the electronic device or the main control device includes a hardware structure and / or software module corresponding to each function. In combination with the units and algorithm steps of each example described in the embodiment disclosed in the embodiment of the present application, the embodiment of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present application.
[0182] The embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, they are used to implement the hard disk performance test method of the above embodiment. In the specific implementation of the above hard disk performance test method, each module can be implemented as a processor.
[0183] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.
[0184] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in an electronic device or a main control device.
[0185] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the hard disk performance testing method of the above embodiment.
[0186] The computer program is stored in a readable storage medium. At least one processor can read the computer program from the readable storage medium. At least one processor executes the computer program to execute the solution provided in any of the above embodiments.
[0187] A person skilled in the art can understand that all or part of the steps of implementing the above-mentioned application embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiment are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk, etc., various media that can store program codes.
[0188] So far, the technical solution of the present application has been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments, and the above embodiments are only used to illustrate the technical solution of the present application rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A hard disk performance testing method, characterized in that: The method is applied to a hard disk performance test device of a distributed storage system, wherein the distributed storage system further comprises a plurality of cache hard disks, and the target hard disk is any one of the plurality of cache hard disks; For the target hard disk, the method includes: Performing a first performance test on the target hard disk to obtain first test data; wherein the amount of data written in the first performance test is less than the predicted buffer capacity of the target hard disk; Obtaining the continuous writing time of the target hard disk; Alternately performing continuous write operations and random write operations on the target hard disk, and recording log data of the instantaneous write speed, until the log data indicates that the instantaneous write speed meets the preset speed requirement, or the duration of the alternating write operation reaches the continuous write time, then determining that the amount of data written by the alternating write operation is greater than the predicted buffer capacity; Performing a secondary performance test on the target hard disk to obtain second test data; A plurality of performance indicators of the target hard disk are obtained according to the first test data, the second test data and the log data.
2. The method according to claim 1, characterized in that The obtaining of the continuous writing time of the target hard disk includes: Obtaining the hard disk capacity and hard disk type of the target hard disk; According to the hard disk capacity and the hard disk type, an expected writing time is obtained; wherein the expected writing time refers to the time expected to fill up the hard disk capacity; The shorter of the expected writing time and the preset upper limit time of the test is determined as the continuous writing time.
3. The method according to claim 2, characterized in that The hard disk type is a first type or a second type; wherein the first type is used to indicate that the target hard disk includes a mechanical hard disk, and the second type is used to indicate that the target hard disk does not include a mechanical hard disk; When the hard disk type is the second type, obtaining the expected write time according to the hard disk capacity and the hard disk type includes: According to a preset proportionality factor, the hard disk capacity is corrected; wherein the hard disk capacity after correction is smaller than the hard disk capacity before correction; The expected writing time is obtained according to the corrected hard disk capacity.
4. The method according to claim 3, characterized in that The upper limit time of the test corresponding to the first type of target hard disk is shorter than the upper limit time of the test corresponding to the second type of target hard disk.
5. The method according to claim 2, characterized in that: The obtaining of the hard disk capacity and hard disk type of the target hard disk includes: A field for obtaining attribute information of the target hard disk; The fields of the attribute information are parsed to obtain the hard disk capacity and the hard disk type.
6. The method according to claim 2, characterized in that The alternately performing a continuous write operation and a random write operation on the target hard disk includes: Within a first preset time, performing a continuous write operation and a random write operation alternately on the target hard disk for the first time; wherein the amount of data alternately written for the first time is less than the predicted buffer content amount; A continuous write operation and a random write operation are alternately performed twice on the target hard disk.
7. The method according to claim 6, characterized in that The first test data includes: random read and write latency, continuous read bandwidth and random read IOPS within a second preset time; the second test data includes: random read and write latency, continuous read bandwidth and random read IOPS within a third preset time; The step of obtaining a plurality of performance indicators of the target hard disk according to the first test data, the second test data and the log data includes: Determining the actual buffer capacity of the target hard disk according to the log data; When the actual cache capacity is greater than zero, the cache performance of the target hard disk is obtained according to the first test data and the log data, and the cache performance of the target hard disk is obtained according to the second test data.
8. The method according to claim 1, characterized in that After obtaining a plurality of performance indicators of the target hard disk according to the first test data, the second test data and the log data, the method further includes: An untested cache hard disk is determined as a new target hard disk to test multiple performance indicators of the new target hard disk.
9. A hard disk performance testing system, characterized in that: The system comprises: A plurality of cache hard disks, and a hard disk performance testing device respectively connected to each of the cache hard disks for communication, wherein the hard disk performance testing device is used to implement the hard disk performance testing method as described in any one of claims 1 to 8.
10. A hard disk performance testing device, characterized in that: The device is located in a hard disk performance test system, the hard disk performance test system further includes a plurality of cache hard disks, and the target hard disk is any one of the plurality of cache hard disks; For the target hard disk, the device includes: A first test module, used for performing a first performance test on the target hard disk to obtain first test data; wherein the amount of data written in the first performance test is less than the predicted buffer capacity of the target hard disk; A time acquisition module, used to acquire the continuous writing time of the target hard disk; a continuous writing module, configured to alternately perform continuous writing operations and random writing operations on the target hard disk, and record log data of instantaneous writing speed, until the log data indicates that the instantaneous writing speed meets the preset speed requirement, or the duration of the alternating writing operation reaches the continuous writing time, then determining that the amount of data written by the alternating writing operation is greater than the predicted buffer capacity; A secondary testing module, used for performing a secondary performance test on the target hard disk to obtain second test data; The performance determination module is used to obtain multiple performance indicators of the target hard disk according to the first test data, the second test data and the log data.