Storage equipment testing method and device and electronic equipment

Abstract

The invention provides a storage equipment testing method and device and electronic equipment, and can be applied to the technical field of computers. The method comprises the steps that an initial test script is executed, target configuration parameters and target test strategies of the storage device are obtained, the initial test script comprises a diagnosis script and a test strategy matching script, the target configuration parameters are obtained by executing the diagnosis script, and the test strategy matching script is used for selecting the target test strategies based on the target configuration parameters; according to the target configuration parameters and the expected use duration of the equipment, a target test period is determined, and the target test period is a test period obtained after an initial test period in the initial test script is compressed; and obtaining a test evaluation report by using test execution information obtained by testing the storage device based on the target test period and the target test strategy.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically to a storage device testing method, apparatus, and electronic device. Background Art

[0002] Stress testing of storage devices can evaluate the performance and stability of storage devices under high loads or extreme conditions. By simulating the high-intensity loads in actual usage scenarios, stress testing can help understand the performance of storage devices under extreme conditions, thereby ensuring their reliability and performance in actual applications.

[0003] Continuous read and write scripts can be used for long-term verification. However, this method requires sequentially executing stress tests such as mixed read and write operations on the file system and raw disk, multi-mode sequential writes, file operations, and compression. This results in uninterrupted testing, long testing times, and low test efficiency. Summary of the Invention

[0004] In view of the above problems, the present application provides a storage device testing method, apparatus and electronic device.

[0005] According to the first aspect of the present application, a storage device testing method is provided, including: executing an initial test script to obtain target configuration parameters and a target test strategy of the storage device, wherein the initial test script includes a diagnostic script and a test strategy matching script, the target configuration parameters are obtained by executing the diagnostic script, and the test strategy matching script is used to select a target test strategy based on the target configuration parameters; determining a target test cycle based on the target configuration parameters and the expected usage time of the device, wherein the target test cycle is a test cycle after compressing the initial test cycle in the initial test script; and obtaining a test evaluation report using test execution information obtained by testing the storage device based on the target test cycle and the target test strategy.

[0006] The second aspect of the present application provides a storage device testing device, including: an execution module, used to execute an initial test script to obtain target configuration parameters and a target test strategy of the storage device, wherein the initial test script includes a diagnostic script and a test strategy matching script, the target configuration parameters are obtained by executing the diagnostic script, and the test strategy matching script is used to select a target test strategy based on the target configuration parameters; a determination module, used to determine a target test cycle based on the target configuration parameters and the expected usage time of the device, wherein the target test cycle is a test cycle after compressing the initial test cycle in the initial test script; an evaluation module, used to obtain a test evaluation report using test execution information obtained by testing the storage device based on the target test cycle and the target test strategy.

[0007] The third aspect of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0008] According to the storage device testing method, apparatus, and electronic device provided by the present application, since the diagnostic script and the test strategy matching script are embedded in the initial test script, it is possible to determine the target configuration parameters of the storage device at the beginning of the test, and then compress the initial test cycle according to the target configuration parameters and the expected usage time of the device to obtain the target test cycle, thereby shortening the test cycle. In addition, the target configuration parameters reflect the performance characteristics of the storage device, and the target test strategy is efficiently matched according to the performance characteristics of the storage device, thereby eliminating redundant test items and matching targeted test items, optimizing the test process, and improving test efficiency. Without modifying the original logic of the initial test script and without relying on external hardware, the test cycle and target test strategy can be dynamically adjusted based on the target configuration parameters, thereby shortening the test cycle and improving efficiency at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above contents and other objects, features and advantages of the present application will become more apparent through the following description of the embodiments of the present application with reference to the accompanying drawings, in which:

[0010] Figure 1 A diagram showing an application scenario of a storage device testing method, apparatus, and electronic device according to an embodiment of the present application is shown;

[0011] Figure 2 A flowchart of a storage device testing method according to an embodiment of the present application is shown;

[0012] Figure 3 A flowchart of determining a target test cycle and a target test strategy according to an embodiment of the present application is shown;

[0013] Figure 4 A schematic diagram of deploying monitoring nodes according to an embodiment of the present application is shown;

[0014] Figure 5 shows a structural block diagram of a storage device testing apparatus according to an embodiment of the present application; and

[0015] Figure 6 A block diagram of an electronic device suitable for implementing a storage device testing method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0016] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present application. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present application. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present application.

[0017] The terms used herein are only for describing specific embodiments and are not intended to limit the present application. The terms "comprise," "include," etc. used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0018] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0019] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0020] The current testing methods for storage devices in the industry generally have the following limitations.

[0021] Conflict between test cycle and efficiency: The company relies heavily on long-term continuous load testing (e.g., continuous reading and writing for more than 15 days), lacks a dynamic adjustment mechanism, and cannot optimize the test process based on device characteristics.

[0022] Single evaluation dimension: A single indicator is often used as the passing standard, ignoring comprehensive health assessments such as performance degradation rate, signal stability, and firmware anomalies.

[0023] Insufficient environmental adaptability: Some solutions are designed only for specific hardware topologies and lack cross-interface and cross-device compatibility. The same test tool needs to repeatedly develop adaptation logic for two storage devices, increasing maintenance costs.

[0024] Lack of abnormal recovery capabilities: There is no integrated breakpoint resumption mechanism. After the test is interrupted, the entire test must be retried, resulting in a waste of time and resources. For example, if a power outage causes data loss, it is impossible to recover from the intermediate state.

[0025] Therefore, it is urgent to build an efficient, recoverable, and full-dimensional storage device testing method without modifying existing test scripts and relying on external hardware.

[0026] In view of this, an embodiment of the present application provides a storage device testing method, including: executing an initial test script to obtain target configuration parameters and a target test strategy of the storage device, wherein the initial test script includes a diagnostic script and a test strategy matching script, the target configuration parameters are obtained by executing the diagnostic script, and the test strategy matching script is used to select a target test strategy based on the target configuration parameters; determining a target test cycle based on the target configuration parameters and the expected usage time of the device, wherein the target test cycle is a test cycle after compressing the initial test cycle in the initial test script; and obtaining a test evaluation report using the test execution information obtained by testing the storage device based on the target test cycle and the target test strategy.

[0027] Figure 1 The following diagram shows an application scenario of a storage device testing method, apparatus, and electronic device according to an embodiment of the present application.

[0028] like Figure 1 As shown, an application scenario 100 of the storage device testing method, apparatus, and electronic device according to an embodiment of the present application includes a storage device 101, a test server 102, and a network 103. Network 103 is a medium for providing a communication link between the first storage device 101 and the test server 102. Network 103 can include various connection types, such as wired or wireless communication links or fiber optic cables. Test server 102 can be a server providing testing services. Storage device 101 can be a backend physical storage medium, such as a hard disk drive, solid-state drive, storage tape, or other hardware device.

[0029] It should be understood that Figure 1 The number of storage devices in the example is only illustrative. Any number of storage devices and test servers may be provided as required.

[0030] The following will be based on Figure 1 The scene described by Figures 2 to 4 The storage device testing method of the disclosed embodiment is described in detail.

[0031] Figure 2 A flowchart of a storage device testing method according to an embodiment of the present application is shown.

[0032] like Figure 2 As shown, the storage device testing method 200 of this embodiment includes operations S210 to S230.

[0033] In operation S210 , an initial test script is executed to obtain target configuration parameters and a target test policy of the storage device.

[0034] Storage devices are hardware devices that store data and programs. They can store user data, computer operating systems, applications, and various configuration information. Examples include Serial ATA Solid-State Drives (SATA SSDs), Quad-Level Cell Solid-State Drives (QLCSSDs), and Non-Volatile Memory Express Solid-State Drives (NVME SSDs).

[0035] Long-term stress testing of storage devices involves mounting the storage devices on a test server and evaluating the performance of the storage devices by simulating scenarios such as high concurrent access and large data volume write / read operations in actual use.

[0036] The initial test script is a pre-written original test script that encapsulates environmental configuration parameters, test parameters, etc., and then simulates user operations or system load to evaluate the performance and stability of the storage device under high load conditions.

[0037] The diagnostic script is first embedded in the preset position of the initial test script, and then the test strategy matching script is embedded. The initial test script includes a header portion and a body portion. The header portion is the environment configuration portion, and the body portion is the test portion. The preset position can be the location where the body portion begins.

[0038] For example, after the test server completes executing the header portion of the initial test script, it begins executing the diagnostic script to obtain the target configuration parameters. For example, the preset position may be a position corresponding to a preset execution duration. The preset execution duration is 2 hours. Two hours after the test server begins executing the initial test script, it begins executing the diagnostic script to obtain the target configuration parameters.

[0039] The target configuration parameters are key characteristic parameters of the storage device, such as interface type, flash memory manufacturing process, firmware version, etc.

[0040] After obtaining the target configuration parameters, a test strategy matching script is executed. The test strategy matching script is used to select a target test strategy based on the target configuration parameters.

[0041] Select a matching target test strategy based on the target configuration parameters of the storage device. For example, based on the target configuration parameters, it is determined that the storage device itself has extremely low durability and is not designed for high-intensity read and write scenarios to verify low durability (such as repeated erase and write). Therefore, select a target test strategy that skips the low-endurance verification test items.

[0042] In operation S220 , a target test period is determined according to the target configuration parameters and the expected usage time of the device.

[0043] The expected device usage time is the planned usage time of the storage device, which can be obtained based on the user-defined accelerated test target or the user usage time constraint in the compliance.

[0044] A time compression algorithm can be used to process target configuration parameters and the expected usage time of the device to determine a compression ratio. The initial test period can then be compressed based on the time compression ratio to obtain a target test period.

[0045] The target test cycle is a test cycle obtained by compressing the initial test cycle in the initial test script.

[0046] For example, the initial test cycle in the initial test script is 15 days. After compressing the initial test cycle, the target test cycle is determined to be 3 days.

[0047] In operation S230 , a test evaluation report is obtained by using test execution information obtained by testing the storage device based on the target test cycle and the target test strategy.

[0048] After determining the target test cycle and target test strategy, the test server continues to execute the main part that was not executed in the initial test script to test the storage device and obtain test execution information.

[0049] Test execution information is the performance information, function information, and stability information of the storage device that is recorded and generated during the test process.

[0050] A multi-dimensional performance evaluation model can be constructed, and the test execution information can be processed using the multi-dimensional performance evaluation model to obtain a test evaluation report.

[0051] By embedding diagnostic scripts and test strategy matching scripts in the initial test script, the target configuration parameters of the storage device are determined at the beginning of the test. The initial test cycle is then compressed to the target test cycle based on the target configuration parameters and the expected usage time of the device, shortening the test cycle. In addition, the target configuration parameters reflect the performance characteristics of the storage device. The target test strategy is efficiently matched according to the performance characteristics of the storage device, thereby eliminating redundant test items and matching targeted test items, optimizing the test process and improving test efficiency. Without modifying the original logic of the initial test script and without relying on external hardware, the test cycle and target test strategy can be dynamically adjusted based on the target configuration parameters, solving the technical problem of being unable to simultaneously shorten the test cycle and improve efficiency.

[0052] The target configuration parameters include at least one of the following: device type parameters, device capacity parameters and read-write performance parameters; wherein, determining the target test period based on the target configuration parameters and the expected usage time of the device includes: determining the theoretical usage time of the device based on at least one of the device type parameters, device capacity parameters and read-write performance parameters; determining the target test period based on the theoretical usage time of the device and the expected usage time of the device.

[0053] The theoretical usage time of the device may be determined based on one or more of a device type parameter, a device capacity parameter, and a read / write performance parameter.

[0054] The nominal cycle of the storage device is determined based on the device type parameters. The nominal cycle is the program / erase cycle (Program / Erase Cycles), which is used to measure the endurance of flash memory.

[0055] The theoretical device lifespan can be determined based on the device's nominal cycle, device capacity, and read / write performance. For example, the read / write performance parameter could be the daily write volume of the storage device, and the theoretical device lifespan is the theoretically usable lifespan of the storage device. For example, if the storage device's nominal cycle is 300 cycles, the device capacity is 1TB, and the read / write performance parameter is 0.1 cycles, the theoretical device lifespan = 300 × 1 / 0.1 = 3000 hours.

[0056] Based on the theoretical usage time and expected usage time of the equipment, the initial test cycle is compressed to obtain the target test cycle.

[0057] Determining the target test period based on the theoretical usage time of the device and the expected usage time of the device includes: obtaining a time compression factor based on the ratio between the theoretical usage time of the device and the expected usage time of the device; and obtaining a target test period based on the initial test period and the time compression factor.

[0058] For example, if the theoretical usage time of a device is 5 days and the expected usage time of the device is 1 day, the time compression factor is 5.

[0059] The time compression factor is the ratio by which the test cycle is shortened. The target test cycle can be calculated based on the ratio of the initial test cycle to the time compression factor. For example, if the initial test cycle is 15 days and the time compression factor is 5, the target test cycle is 3 days.

[0060] The theoretical usage time of the device is determined based on the target configuration parameters of the storage device. The time compression factor is determined based on the theoretical usage time and the expected usage time of the device. The initial test cycle is compressed based on the time compression factor, and the test cycle is flexibly and dynamically compressed according to the device characteristics.

[0061] Selecting a target test strategy based on target configuration parameters includes: when the target configuration parameters meet the first performance, determining the target test strategy as not requiring durability testing; when the target configuration parameters meet the second performance, determining the target test strategy as adding multi-queue testing and durability testing, wherein the second performance is better than the first performance.

[0062] The second performance and the first performance can both represent read and write performance, and the second performance is better than the first performance.

[0063] The target configuration parameters include the quantity parameters, device type parameters, read and write performance parameters, etc. of the storage devices. Based on one or more of the quantity parameters, device type parameters, and read and write performance parameters, the performance conditions met by the storage device can be determined, and then the scenario type applicable to the storage device can be determined, and the target test strategy can be matched.

[0064] If the target configuration parameters meet the first performance requirement, the storage device is suitable for low-intensity read and write scenarios. If the target configuration parameters meet the second performance requirement, the storage device is suitable for high-intensity read and write scenarios or high-concurrency load scenarios.

[0065] For example, if the quantity parameter of the storage device is less than the preset quantity, the target configuration parameter is judged to meet the first performance; if the device type parameter is NVME SSD, and NVME SSD is a scale-level device, the target configuration parameter is judged to meet the second performance; if the read and write performance parameters of the storage device are less than the preset read and write values, the target configuration parameter is judged to meet the first performance.

[0066] When the target configuration parameters meet the first performance, testing the low endurance of the storage device (such as repeated erasure) is meaningless, which will increase the redundant testing time and require skipping the low endurance verification items. Therefore, the target test strategy is determined to be no endurance testing required.

[0067] When the target configuration parameters meet the second performance requirement, the storage device supports high concurrent loads. Multi-queue depth testing can verify its stability and performance degradation under high pressure. Therefore, the target test strategy is to add multi-queue testing and endurance testing.

[0068] By setting different performance conditions and determining the performance conditions met by the target configuration parameters of the storage device, the target test strategy can be efficiently matched, eliminating redundant test items and matching them with targeted test items. This optimizes the test process, improves test efficiency, and resolves the technical problem in related technologies where long-term continuous load testing cannot optimize the test process based on device characteristics. For example, storage devices suitable for low-intensity read and write scenarios still need to perform the same durability test items as storage devices suitable for high-intensity read and write scenarios, resulting in a large number of redundant operations.

[0069] Figure 3 A flowchart of determining a target test cycle and a target test strategy according to an embodiment of the present application is shown.

[0070] like Figure 3 As shown, the steps of this embodiment include step S310 to step S360.

[0071] Step S310: execute the initial test script to obtain the device type parameter of the storage device.

[0072] Step S320 , determine whether the device type parameter is QLC SSD. If so, jump to step S330 ; otherwise, jump to step S340 .

[0073] Step S330, skip the durability test and jump to step S360.

[0074] Step S340, determine whether the device type parameter is NVME SSD, if so, jump to step S350, otherwise jump to step S360.

[0075] Step S350, add multi-queue test, jump to step S360.

[0076] Step S360: Testing the storage device based on the target test cycle and the target test strategy.

[0077] The test execution information obtained by testing the storage device based on the target test cycle and the target test strategy is used to obtain a test evaluation report, including: using the evaluation model to process the test execution information to obtain the evaluation result; obtaining the test level according to the level evaluation rules and the evaluation result; and obtaining the test evaluation report according to the coverage loss and the test level.

[0078] An evaluation model is constructed based on multi-dimensional quantitative evaluation indicators. The evaluation model is used to process test execution information to obtain evaluation results. The evaluation results represent the effectiveness of the test process.

[0079] The grade assessment rules are grading standards. For example, if the assessment result is ≥90, the test grade is A+, and A+ represents excellent; if the assessment result is between [70,89], the test grade is B, and B represents the need for retesting; if the assessment result is <70, the test grade is C, and C represents unqualified.

[0080] Coverage loss is the proportion of hardware functions, features, or code that is not covered during testing.

[0081] The coverage loss can be calculated and verified based on the root Monte Carlo model. When the coverage loss is less than a preset threshold, it means that the integrity and reliability verification of the storage device based on the target test cycle and target test strategy has passed.

[0082] The coverage loss and test level are summarized to obtain a test evaluation report.

[0083] The test execution information includes the read and write performance at the start of the test and the read and write performance at the end of the test, the bit error rate, and the number of firmware exceptions; wherein, the test execution information is processed using the evaluation model to obtain the evaluation results including: obtaining the performance decay rate based on the read and write performance at the start of the test, the read and write performance at the end of the test, and the target test cycle; obtaining the firmware robustness evaluation value based on the number of firmware exceptions and the target test cycle; and obtaining the evaluation result based on the performance decay rate, the firmware robustness evaluation value, and the bit error rate.

[0084] Insert a lightweight monitoring tool into the initial test script without modifying the original logic of the initial test script, and collect read and write performance every 10 minutes.

[0085] Capture bit error rate through the physical layer debug interface.

[0086] In one embodiment, the performance degradation rate PAR is as shown in formula (1):

[0087] (1);

[0088] in, Characterize the read and write performance at the beginning of the test, It represents the read and write performance at the beginning of the test, and T represents the target test period.

[0089] In one embodiment, the firmware robustness evaluation value FRI is as shown in formula (2):

[0090] (2);

[0091] Where A represents the number of firmware exceptions, and T represents the target test period.

[0092] The bit error rate is processed to obtain a signal integrity score.

[0093] In one embodiment, the signal integrity score SIS is as shown in formula (3):

[0094] (3);

[0095] Here, M represents the bit error rate.

[0096] In one embodiment, the evaluation result F is shown in formula (4):

[0097] (4);

[0098] When the performance degradation rate, the firmware robustness evaluation value, and the signal integrity score each meet a threshold, an evaluation result is obtained based on the performance degradation rate, the firmware robustness evaluation value, and the signal integrity score.

[0099] For example, the performance degradation rate is ≤3%, the firmware robustness evaluation value is ≤0.5, and the signal integrity score is ≥80.

[0100] By defining multi-dimensional indicators such as performance degradation rate, firmware robustness evaluation value and signal integrity score, an evaluation model is constructed to conduct a comprehensive health assessment from dimensions such as performance degradation degree, signal stability and firmware anomalies. This improves the scientificity, objectivity and accuracy of the evaluation results, and solves the technical problem of low evaluation accuracy caused by using a single indicator (such as no log errors or total number of erase and write times) as the passing standard in related technologies.

[0101] The test execution information also includes the number of failures; wherein the coverage loss is determined based on the following operations: determining the target detection rate based on the number of failures and the number of test simulations; determining the coverage loss based on the target detection rate and the initial detection rate, wherein the initial detection rate is determined based on the initial test cycle and the number of test simulations.

[0102] The number of failures is the number of times the storage device triggers a failure during the test.

[0103] Based on 1000 test simulation times, 10,000 sets of test parameters are randomly generated to simulate different storage device states and load conditions. The target test cycle and target test strategy are determined when each set of test parameters is run. The storage device is tested based on the target test cycle, target test strategy and test parameters. Whether a fault is triggered during this test process is recorded to obtain the number of faults.

[0104] The target detection rate is determined based on the ratio between the number of failures obtained based on the target test cycle and the number of test simulations.

[0105] When each set of test parameters is run, the test cycle is not compressed. The storage device is tested based on the initial test cycle and test parameters. Whether a fault is triggered during this test process is recorded to obtain the number of faults.

[0106] The initial detection rate is determined based on the ratio between the number of failures obtained based on the initial test cycle and the number of test simulations.

[0107] In one embodiment, the coverage loss W is expressed as formula (5):

[0108] (5);

[0109] Among them, W1 represents the target detection rate, and W2 represents the initial detection rate.

[0110] A coverage loss of less than 5% indicates that the integrity and reliability verification of the storage device has passed based on the target test cycle and target test strategy.

[0111] The storage device testing method also includes: in the event that the execution operation of the initial test script is interrupted, in response to receiving a restart request from the test server, obtaining test execution information corresponding to the target collection time for restart, wherein the test execution information includes test progress and test execution parameters; and continuing the interrupted execution operation in the test environment rebuilt using the test progress and test execution parameters.

[0112] The initial test script runs under the memory operating system, which is installed on the disk. The test execution information during the test cannot be uploaded to the memory. After the test is completed, the test execution information is uploaded to the memory. If there is an unexpected power outage or system crash during the test, the test progress will be reset to zero. At that time, it will be impossible to recover from the breakpoint and a full retry will be required, resulting in serious waste of resources.

[0113] Deploy an independent monitoring node (such as another host) outside the test server. The monitoring node is isolated from the RAM operating system through the intranet. The monitoring node stores the test execution information of each collection time in the network file system based on the collection time interval parameters. The monitoring node mounts the network file system shared storage to persist the test status. The monitoring node performs periodic data backup. After the test is interrupted, there is no need to retry the whole test, and the test can be resumed at the breakpoint, reducing time cost and resource waste.

[0114] In the event that the execution of the initial test script is interrupted, the monitoring node determines the restart time in response to receiving the restart request from the test server, and then automatically parses the storage information in the network file system, takes the collection time closest to the restart time as the target collection time, and obtains the test execution information corresponding to the target collection time.

[0115] Test execution information includes test progress and test execution parameters. The test environment is rebuilt using the test progress and test execution parameters (e.g., mounting storage devices and restoring process parameters). The interrupted execution operation is resumed in the rebuilt test environment.

[0116] For example, if the restart time is 12:00 and the target collection time is determined to be 11:00, the test environment is rebuilt based on the test progress and test execution parameters corresponding to 11:00, and 11:00 is used as the checkpoint. The remaining test progress is continued from the test progress corresponding to 11:00. Therefore, the maximum progress loss after the interruption is 1 hour.

[0117] It is compatible with the initial test script without modifying the memory operating system configuration, and the test status is persisted through the external monitoring service of the test server. It can be restored to the collection time closest to the restart time, reducing the loss of test progress, solving the technical problem of complete loss of test progress and data after the memory operating system environment is interrupted, and reducing resource waste.

[0118] Figure 4 A schematic diagram of deploying monitoring nodes according to an embodiment of the present application is shown.

[0119] like Figure 4 As shown, the initial test script runs under the memory operating system, and an independent monitoring node (host) is deployed outside the test server. The monitoring node and the memory operating system are isolated through the intranet. The monitoring node mounts the network file system shared storage. The monitoring node executes scheduled tasks, periodically pulls the test execution information of each at different collection times and stores it in the network file system. The network file system persists the test status.

[0120] During the test of the Communication Management Unit (CMU) slot insertion interface, the monitoring node was configured first, and then the initial test script was run: the device type parameter of the storage device was determined to be a Triple Level Cell Serial Advanced Technology Solid State Drive (TLC SATA SSD). The standard test strategy was to enable high endurance testing, and the initial test period of 5 days was compressed to a target test period of 30 hours. Data Collection and Evaluation: PAR: =85k, =82k, PAR=97.22; Bit Error Rate=0.02%, SIS=99.8; Number of Firmware Abnormalities=0, FRI=0; Evaluation Result: 0.4×97.22 + 0.3×99.8 + 0.3×100 = 98.828; Test Grade is A+, and a test evaluation report is output.

[0121] During the test of the motherboard directly connected to the NVME SSD, a power outage was simulated and the system continued from the most recent checkpoint (the target acquisition time was the third day of the test). The serial link status was captured by a monitoring tool, and an error in the data storage location mapping of the managed storage device was detected, resulting in a test level of B. The test evaluation report was output, recommending a review of the firmware version and restarting the test after an update.

[0122] Based on the above storage device testing method, the present application also provides a storage device testing device. Figure 5 The device is described in detail.

[0123] Figure 5 A structural block diagram of a storage device testing apparatus according to an embodiment of the present application is shown.

[0124] like Figure 5 As shown, the storage device testing apparatus 500 of this embodiment includes an execution module 510 , a determination module 520 , and an evaluation module 530 .

[0125] Execution module 510 is configured to execute an initial test script to obtain target configuration parameters and a target test policy for the storage device. The initial test script includes a diagnostic script and a test policy matching script. The target configuration parameters are obtained by executing the diagnostic script, and the test policy matching script is configured to select a target test policy based on the target configuration parameters. In one embodiment, execution module 510 may be configured to perform operation S210 described above, and will not be further described herein.

[0126] Determination module 520 is configured to determine a target test cycle based on the target configuration parameters and the expected device usage duration, where the target test cycle is a test cycle obtained by compressing the initial test cycle in the initial test script. In one embodiment, determination module 520 can be configured to perform operation S220 described above, which will not be further described here.

[0127] The evaluation module 530 is configured to obtain a test evaluation report using the test execution information obtained by testing the storage device based on the target test cycle and the target test strategy. In one embodiment, the evaluation module 530 may be configured to perform the operation S230 described above, which will not be described in detail here.

[0128] The determination module 520 includes a first determination submodule and a second determination submodule.

[0129] The first determining submodule is configured to determine a theoretical usage time of the device according to at least one of a device type parameter, a device capacity parameter, and a read / write performance parameter.

[0130] The second determination submodule is used to determine a target test period according to a theoretical usage time of the device and an expected usage time of the device.

[0131] The second determining submodule includes a first determining unit and a second determining unit.

[0132] The first determining unit is configured to obtain a time compression factor according to a ratio between a theoretical usage time of the device and an expected usage time of the device.

[0133] The second determining unit is configured to obtain a target test period according to the initial test period and the time compression factor.

[0134] The execution module 510 includes a first execution submodule and a second execution submodule.

[0135] The first execution submodule is configured to determine that the target test strategy does not require durability testing when the target configuration parameters meet the first performance.

[0136] The second execution submodule is used to determine the target test strategy to add multi-queue testing and endurance testing when the target configuration parameters meet the second performance, wherein the second performance is better than the first performance.

[0137] The evaluation module 530 includes a first evaluation submodule, a second evaluation submodule, and a third evaluation submodule.

[0138] The first evaluation submodule is used to process the test execution information using the evaluation model to obtain an evaluation result.

[0139] The second evaluation submodule is used to obtain the test grade according to the grade evaluation rules and evaluation results.

[0140] The third evaluation submodule is used to obtain a test evaluation report based on the coverage loss and the test level.

[0141] The first evaluation submodule includes a first evaluation unit, a second evaluation unit, and a third evaluation unit.

[0142] The first evaluation unit is configured to obtain a performance degradation rate according to the read and write performance at the beginning of the test, the read and write performance at the end of the test, and a target test cycle.

[0143] The second evaluation unit is used to obtain a firmware robustness evaluation value according to the number of firmware anomalies and a target test cycle.

[0144] The third evaluation unit is used to obtain an evaluation result according to the performance degradation rate, the firmware robustness evaluation value and the bit error rate.

[0145] The storage device testing apparatus 500 further includes a response module and a reconstruction module.

[0146] The response module is configured to obtain test execution information corresponding to a target collection time for restarting in response to receiving a restart request from the test server when the execution operation of the initial test script is interrupted, wherein the test execution information includes a test progress and test execution parameters.

[0147] The reconstruction module is used to continue the interrupted execution operation in the test environment rebuilt by using the test progress and test execution parameters.

[0148] Any multiple modules among the execution module 510, determination module 520, and evaluation module 530 can be combined into a single module, or any one of them can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in a single module. At least one of the execution module 510, determination module 520, and evaluation module 530 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented in hardware or firmware through any other reasonable means of circuit integration or packaging, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or any suitable combination of any of these. Alternatively, at least one of the execution module 510, determination module 520, and evaluation module 530 can be at least partially implemented as a computer program module that, when executed, can perform the corresponding functionality.

[0149] Figure 6 A block diagram of an electronic device suitable for implementing a storage device testing method according to an embodiment of the present application is shown.

[0150] like Figure 6As shown, an electronic device 600 according to an embodiment of the present application includes a processor 601, which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 602 or a program loaded from a storage unit 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present application.

[0151] Various programs and data required for the operation of the electronic device 600 are stored in the RAM 603. The processor 601, ROM 602, and RAM 603 are connected to each other via a bus 604. The processor 601 performs various operations of the method flow according to the embodiment of the present application by executing the programs in the ROM 602 and / or RAM 603. It should be noted that the programs may also be stored in one or more memories other than the ROM 602 and RAM 603. The processor 601 may also perform various operations of the method flow according to the embodiment of the present application by executing the programs stored in one or more memories.

[0152] Electronic device 600 may also include an input / output (I / O) interface 605, which is also connected to bus 604. Electronic device 600 may also include one or more of the following components connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 608 including a hard disk; and a communication section 609 including a network interface card such as a LAN card or modem. Communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. Removable media 611, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 610 as needed, so that computer programs read from the removable media can be installed into storage section 608 as needed.

[0153] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the storage device testing method according to the embodiments of this application is implemented.

[0154] The computer-readable storage medium may be a non-volatile computer-readable storage medium, and may include, but is not limited to, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above, and / or one or more memories other than ROM 602 and RAM 603.

[0155] The embodiments of the present application also include a computer program product, which includes a computer program containing program code for executing the method shown in the flowchart. When the computer program product is executed in a computer system, the program code is used to enable the computer system to implement the storage device testing method provided in the embodiments of the present application.

[0156] The computer program executes the above functions defined in the system / device of the embodiment of the present application when the computer program is executed by the processor 601. The above-described systems, devices, modules, units, etc. can be implemented by computer program modules.

[0157] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0158] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609 and / or installed from a removable medium 611. When the computer program is executed by the processor 601, the above-described functions defined in the system of the embodiment of the present application are performed. The systems, devices, apparatuses, modules, units, etc. described above can be implemented by computer program modules.

[0159] The program code for executing the computer program provided by the embodiments of the present application can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, Python, "C" language, or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0160] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0161] Those skilled in the art will appreciate that the features described in the various embodiments of this application may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in this application. In particular, the features described in the various embodiments of this application may be combined and / or coupled in various ways without departing from the spirit and teachings of this application. All such combinations and / or couplings fall within the scope of this application.

[0162] The embodiments of the present application have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present application. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present application, those skilled in the art may make various substitutions and modifications, and these substitutions and modifications should all fall within the scope of the present application.

Claims

1. A storage device testing method, characterized in that: The method comprises: Executing an initial test script to obtain target configuration parameters and a target test policy of the storage device, wherein the initial test script includes a diagnostic script and a test policy matching script, the target configuration parameters are obtained by executing the diagnostic script, and the test policy matching script is used to select a target test policy based on the target configuration parameters; Determining a target test cycle based on the target configuration parameters and the expected usage time of the device, wherein the target test cycle is a test cycle obtained by compressing the initial test cycle in the initial test script; A test evaluation report is obtained by using test execution information obtained by testing the storage device based on the target test cycle and the target test strategy.

2. The method according to claim 1, characterized in that The target configuration parameters include at least one of the following: a device type parameter, a device capacity parameter, and a read / write performance parameter; The step of determining a target test period based on the target configuration parameters and the expected usage time of the device includes: determining a theoretical usage time of the device according to at least one of the device type parameter, the device capacity parameter, and the read / write performance parameter; A target test period is determined based on the theoretical usage time of the device and the expected usage time of the device.

3. The method according to claim 2, characterized in that Determining the target test period according to the theoretical usage time of the device and the expected usage time of the device includes: Obtaining a time compression factor according to a ratio between a theoretical usage time of the device and an expected usage time of the device; The target test period is obtained according to the initial test period and the time compression factor.

4. The method according to claim 1, wherein The selecting a target test strategy based on the target configuration parameters includes: When the target configuration parameters meet the first performance, determining the target test strategy as not requiring durability testing; When the target configuration parameters meet the second performance, the target test strategy is determined to be to add a multi-queue test and an endurance test, wherein the second performance is better than the first performance.

5. The method according to claim 1, wherein The obtaining of a test evaluation report by using the test execution information obtained by testing the storage device based on the target test cycle and the target test strategy includes: Processing the test execution information using an evaluation model to obtain an evaluation result; Obtaining a test grade according to the grade assessment rules and the assessment results; A test evaluation report is obtained based on the coverage loss and the test level.

6. The method according to claim 5, characterized in that The test execution information includes the read and write performance at the beginning of the test and the read and write performance at the end of the test, the bit error rate, and the number of firmware anomalies; The step of processing the test execution information using the evaluation model to obtain an evaluation result includes: Obtaining a performance decay rate according to the read and write performance at the start of the test, the read and write performance at the end of the test, and the target test cycle; Obtaining a firmware robustness evaluation value according to the number of firmware exceptions and the target test cycle; The evaluation result is obtained according to the performance degradation rate, the firmware robustness evaluation value and the bit error rate.

7. The method according to claim 6, characterized in that The test execution information also includes the number of failures; The coverage loss is determined based on the following operations: Determining a target detection rate based on the number of failures and the number of test simulations; The coverage loss is determined according to the target detection rate and an initial detection rate, wherein the initial detection rate is determined based on an initial test cycle and the number of test simulations.

8. The method according to claim 1, characterized in that The method further comprises: In the event that the execution of the initial test script is interrupted, In response to receiving the test server restart request, obtaining the test execution information corresponding to the target collection time for restart, wherein the test execution information includes a test progress and a test execution parameter; The interrupted execution operation is continued in the test environment rebuilt using the test progress and the test execution parameters.

9. A storage device testing device, characterized in that: The device comprises: an execution module, configured to execute an initial test script to obtain target configuration parameters and a target test policy of the storage device, wherein the initial test script includes a diagnostic script and a test policy matching script, the target configuration parameters are obtained by executing the diagnostic script, and the test policy matching script is used to select a target test policy based on the target configuration parameters; a determination module, configured to determine a target test cycle according to the target configuration parameters and the expected usage time of the device, wherein the target test cycle is a test cycle obtained by compressing the initial test cycle in the initial test script; An evaluation module is configured to obtain a test evaluation report by utilizing test execution information obtained by testing the storage device based on the target test cycle and the target test strategy.

10. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

Images