Verification method and device for robustness and data consistency under FW hot upgrade and medium

Abstract

The invention relates to the field of storage, in particular to a method and device for verifying robustness and data consistency under FW hot upgrade and a medium. The method comprises the following steps: firstly, ensuring that the state of an SSD to be tested is normal, recording SMART information before testing, carrying out full-disk low-lattice operation on the SSD to be tested, creating two physical partitions on the SSD to be tested, writing a test file into a first partition, calculating and storing a Hash check code of the test file, and issuing high-pressure random hybrid read-write to a created second partition by using an FIO tool. And performing K times of FW hot upgrade and hot degradation operation, and performing power-down operation on the test machine in the upgrade and degradation process. And after all upgrading and downgrading tests are completed, calculating the hash check code of the test file again. And comparing the SMART Log information and the Hash check code before and after the test to obtain a test result. According to the method, the firmware upgrading pressure and abnormal interference in a real complex scene are simulated, and the robustness and data consistency of the SSD to be tested during firmware upgrading are comprehensively evaluated.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of storage, especially the field of design and verification of solid state disk, in particular to a method and device for verifying robustness and data consistency under FW hot upgrade and a medium. BACKGROUND

[0002] The main purpose of the robustness and data consistency test of the FW (firmware, a program running on the solid state disk) of the solid state disk under the hot upgrade is to ensure that the solid state disk can still maintain the atomicity of the disk upgrade operation and the consistency of the data when facing unexpected situations (such as forced system cold restart) during the firmware upgrade process, and ensure the stability and reliability of the system. The current test method often only considers single read-write load pressure or simple power-off situation, and does not consider the combination verification scene under different read-write load pressure, which cannot guarantee that the solid state disk system can run normally and provide reliable performance after upgrading. SUMMARY

[0003] In view of the defects of the prior art, the present application provides a method and device for verifying the robustness and data consistency under the FW hot upgrade and a medium, which verifies that the solid state disk can effectively handle various abnormal situations during the upgrading process while maintaining the integrity and consistency of the data, thereby ensuring that the system can run normally and provide reliable performance after upgrading.

[0004] In order to solve the technical problem, the present application verifies the robustness and data consistency of the NVMe SSD in the process of firmware hot upgrade / degradation by simulating the firmware upgrade pressure and abnormal interference in real complex scenarios. By superimposing high-load read-write operations (simulating actual user scenarios) and random power-off (simulating unexpected abnormalities) during firmware upgrade, the state and data integrity of the SSD are checked after repeated operations. The specific method is as follows: first, ensure that the state of the to-be-tested SSD is normal, and record the SMART Log and other log information before testing. Perform full-disk low-grade on the to-be-tested sample, create two physical partitions on the to-be-tested sample, and each partition occupies 50% of the space size of the to-be-tested sample. Write a test file on the first partition, and the capacity of the test file is set by simulating the scenario of storing a large amount of data in the to-be-tested SSD. Calculate and save the hash check code of the test file, which is used to verify the integrity of the test file. Use the FIO tool to issue a large pressure random mixed read-write to the second partition (the partition without writing files) created. During the FIO operation, perform K times of FW hot upgrade and hot degradation operation, and perform power-off operation on the test machine during the upgrade and degradation process. After all the upgrade and degradation tests are completed, calculate the hash check code of the file created in the first partition again. Compare the SMART Log information and the hash check code before and after the test, and obtain the test result according to the comparison result.

[0005] The purpose of creating two partitions is twofold: The first partition stores test files and calculates their hash checksums, simulating important user-stored data to verify the consistency of static data during firmware upgrades (unaffected by the upgrade process). The second partition applies high-pressure random read / write operations via FIO to simulate dynamic user operations (such as system operation and file transfer), testing the SSD's stability under concurrent read / write loads during firmware upgrades. The two partitions respectively cover static data storage and dynamic read / write data scenarios, ensuring that the test not only verifies the security of existing data but also verifies the processing capabilities for real-time read / write operations during the upgrade process (such as cache management and data commit mechanisms).

[0006] The purpose of the hash checksum: The hash checksum is a unique hash value of the file content. If the hash checksums before and after the test are consistent, it means that the content of the test file in the first partition has not been tampered with or damaged by firmware upgrades, power outages, or other operations, proving that the SSD can guarantee the consistency of static data under abnormal scenarios. If the hash checksums are inconsistent, it indicates that data loss, write errors, or incorrect cache refresh may have occurred during the upgrade process, indirectly reflecting a defect in the firmware's logic for handling upgrades and data protection.

[0007] SMART Log information reflects key parameters of hardware health status, data integrity risks, and the impact of firmware upgrade anomalies, such as Critical Warning, media error, Number of Error Log Entries, Uncorrectable Sector Count, Firmware Version, and Power Cycle Count. By comparing SMART Log information before and after the test, it is possible to determine whether the SSD under test experienced any anomalies during the test.

[0008] The beneficial effects of this invention are: Real-world user simulation: Simulates a sudden power outage during firmware upgrades to verify the data consistency and recovery capabilities of the SSD under such abnormal conditions.

[0009] Verify concurrent read / write: Simulate concurrent read / write operations during firmware upgrades to evaluate the robustness and data consistency of the SSD under high load conditions.

[0010] It helps in optimization and improvement: Test results can provide important references for the robustness of SSD firmware upgrades and downgrades. By analyzing the test data, targeted optimization strategies can be developed.

[0011] In summary, this invention can comprehensively evaluate the robustness of upgraded firmware and data consistency of storage devices, providing strong support for the reliable operation of the devices. Attached Figure Description

[0012] Figure 1 This is a flowchart of the method described in Example 1; Figure 2 This is a schematic diagram of the device described in Example 2. Detailed Implementation

[0013] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0014] Example 1 This invention discloses a method for verifying robustness and data consistency during hot upgrades of firewalls, such as... Figure 1 As shown, this method includes the following steps: S01. Insert the SSD under test into the test machine. The test machine is powered on and the SSD under test is in a normal initial state. In this embodiment, ensuring that the SSD under test is in a normal initial state means that the SSD under test can be recognized normally.

[0015] S02. Before performing the SSD upgrade / downgrade test, record the SMART Log information of the SSD under test. This information will serve as a reference and basis for comparison after the test is completed. Then, perform a full low-level format on the SSD under test.

[0016] In this embodiment, the SMART Log can reflect key parameters of hardware health status, data integrity risks, and the impact of abnormal firmware upgrades, such as Critical Warning, media error, Number of Error Log Entries, Uncorrectable Sector Count, Firmware Version, and Power Cycle Count.

[0017] S03. Create two physical partitions on the SSD under test. Write a test file to the first partition. The capacity of the test file is set by simulating a scenario where the SSD under test stores a large amount of data. Calculate and save the hash checksum of the test file. The hash checksum is used to verify the integrity of the test file.

[0018] In this embodiment, the size of both the first and second partitions accounts for 50% of the capacity of the SSD under test. After creating the first partition, an EXT4 file system is created on the first partition and mounted to a specified directory. Then, a 200GB test file is written with an MD5 hash checksum. The EXT4 file system is chosen in this embodiment as a preferred option based on scenario adaptation, rather than a mandatory requirement. EXT4 is chosen because it better simulates mainstream storage scenarios in a Linux environment (such as servers, embedded devices, and desktop Linux systems), highly aligning with the realistic testing objectives. EXT4 has high stability and maturity, reducing the interference of file system defects on test results. Furthermore, EXT4 supports large files and large partitions and has good compatibility with FIO tools. The 200GB file is chosen to simulate large file storage scenarios and test the SSD's performance when processing large amounts of data; other file sizes could also be used. MD5 checksum was chosen because it has high computational efficiency, making it suitable for large file scenarios; it has a low collision probability, meeting the accuracy requirements of testing; and it has good cross-platform compatibility, natively supporting MD5 checksum on commonly used Linux systems (such as Ubuntu and CentOS) as well as Windows systems.

[0019] S04. Use the FIO tool to issue high-pressure random mixed read and write to the created second partition. The random mixed read and write under IO pressure saturation state achieved by the combination of test parameters is called high-pressure random mixed read and write. K FW hot upgrade and hot degradation operations are performed during the operation of the FIO tool. During the FW hot upgrade and hot degradation operations, the test machine is randomly powered off.

[0020] In this embodiment, the test parameter combination for high-pressure random mixed read / write is: BS=4K, QD=128, job=1, libaio, rwmixread=50. BS is an abbreviation for Block Size, meaning that the data block size for each read / write operation is 4KB (4 kilobytes). QD is an abbreviation for Queue Depth, referring to the number of incomplete IO requests that the SSD controller can handle simultaneously, which is 128. The higher the queue depth, the better it tests the SSD's concurrent processing capabilities, closely approximating the IO pressure under real high-load scenarios. job represents the number of FIO test tasks, here indicating the start of one test process. Combined with the queue depth (QD=128), a single task can generate dense IO requests, avoiding CPU bottleneck interference caused by multiple tasks. libaio indicates the use of the Linux system's asynchronous IO engine. In this mode, the test process does not need to wait for the previous IO request to complete before sending a new request, maximizing the continuity and density of IO requests, simulating asynchronous read / write scenarios in real applications. `rwmixread` represents the percentage of read operations in the total I / O in mixed read / write mode. `rw` indicates mixed read / write mode, and `mixread=50` means read operations account for 50% and write operations account for 50% of the total I / O. This parameter is used to simulate real-world scenarios of concurrent read / write operations (such as loading data while saving files), comprehensively testing the firmware's ability to schedule read / write requests.

[0021] The value of K is set based on one or more of the following: A. Determined by referring to industry standards or specifications; B. Determined based on the product's expected lifespan and usage scenarios; C. Determined based on testing time and resource limitations; D. Determined based on usage risks. In this embodiment, K=100. In other embodiments, the value of K is adjusted according to actual needs.

[0022] S05. If the new firmware has already been submitted at the moment of power failure, the new firmware will take effect after power is restored; otherwise, the old firmware will take effect.

[0023] S06. Repeat steps S04 and S05, repeating 100 times.

[0024] S07. After all FW hot upgrade and hot downgrade operations are completed, recalculate the hash checksum of the test files in the first partition, and compare the SMART Log information and hash checksum before and after the FW hot upgrade and hot downgrade operations.

[0025] S08. If the comparison reveals that the SMART Log contains error messages exceeding the fault tolerance range, inconsistent hash checksums, or the SSD under test hangs or drops during the entire test, the test fails; otherwise, the test passes.

[0026] In this embodiment, error messages exceeding the fault tolerance range include critical warnings and media errors. Critical warnings are the most severe warning type in the SSD's SMART log, comprehensively reflecting urgent faults that may directly threaten data security. They typically include multiple sub-items (such as power failure, media error, temperature exceeding limits, controller failure, etc.), and triggering any of these sub-items will cause the warning to appear. Media errors refer to unrecoverable errors (such as physical bad blocks, data verification failures, erase / write failures, etc.) that occur in the NAND flash memory chips (storage media) of the SSD under test during read / write operations. They are directly related to the reliability of data storage and are a core indicator of data integrity risk.

[0027] The method described in this embodiment can comprehensively evaluate the robustness and data consistency of upgraded firmware of storage devices, providing strong support for the reliable operation of the devices. Firmware robustness refers to "whether the firmware can maintain functional stability and avoid crashes or permanent failures under extreme conditions (such as high load, sudden power outages)." The solution is mainly verified through the following design: 1. Extreme scenario superposition test: Step S04 clearly states that the "three high-pressure operations" will be executed simultaneously. This superposition scenario of "high load IO + frequent firmware changes + sudden power failure" directly tests the firmware's anti-interference ability. If the firmware does not have problems such as "hanging, disk loss, or inability to be recognized" during the process, it means that its robustness meets the standard.

[0028] 2. Full monitoring of hardware status: Steps S02 and S07 focus on checking some indicators by “comparing the SMART logs before and after the test”. If the above indicators are normal, it proves that the firmware has not caused permanent damage to the hardware under extreme operation and the underlying logic is robust.

[0029] 3. Verification of the atomicity of the upgrade: Step S05 explicitly requires that "if the new firmware has been submitted at the moment of power failure, the new firmware must take effect after power-on; otherwise, the old version will take effect." This rule directly verifies the atomicity of the firmware upgrade, disallowing intermediate states such as partial upgrades or incomplete versions. It ensures that the firmware maintains a stable and normally functioning state regardless of whether the upgrade process is successful or fails, and is a core indicator of robustness.

[0030] Data consistency testing demonstrates: 1. Static data integrity verification: Step S03: Write a 200G file to the first partition and save the MD5 checksum (MD5 is the unique hash value of the file content; any minor modification will cause the checksum to change). Step S07: After all tests are completed, "recalculate the MD5 value of the file and compare it." If they match, it proves that the static data was not damaged, tampered with, or lost during the "high load + upgrade + power outage" process, and data consistency is guaranteed.

[0031] 2. Stability verification of dynamic read and write: When applying high pressure to the second partition for random mixed read and write in step S04, the firmware needs to handle real-time IO requests and upgrade logic at the same time. If no read / write timeout or data write error occurs during the test (which can be verified by FIO log or SSD error log), it means that the dynamic data processing process has not suffered from consistency vulnerabilities due to the upgrade.

[0032] 3. Verification of hardware-level data protection: The parameters in the SMART log directly reflect the integrity of the data at the storage medium level. Step S07 shows no new additions to these parameters, proving that the firmware's underlying protection of the data (such as ECC verification and bad block remapping) is effective during the upgrade process, and no physical data corruption has occurred.

[0033] Example 2 This disclosure provides a robustness and data consistency verification device 300 for FW hot upgrades, such as... Figure 2 As shown, the device includes a processor 304 and a memory 301. Optionally, the device may further include a communication interface 302 and a bus 303. The processor 304, communication interface 302, and memory 301 can communicate with each other via the bus 303. The communication interface 302 can be used for information transmission. The processor 304 can invoke logical instructions in the memory 301 to execute the robustness and data consistency verification method for FW hot upgrades described in the above embodiment.

[0034] Furthermore, the logic instructions in the aforementioned memory 301 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0035] The memory 301, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 304 executes functional applications and data processing by running the program instructions / modules stored in the memory 301, thereby implementing the robustness and data consistency verification method for FW hot upgrades in the above embodiments.

[0036] The memory 301 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 301 may include high-speed random access memory and may also include non-volatile memory.

[0037] Example 3 This disclosure provides a computer-readable storage medium storing computer-executable instructions configured to perform the aforementioned verification method for robustness and data consistency under FW hot upgrade.

[0038] The aforementioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0039] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, including: a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media capable of storing program code; it can also be a transient storage medium.

[0040] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for descriptive purposes only and is not intended to limit the scope of protection. As used in the description herein, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used herein means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0041] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0042] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection shown or discussed between each other may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

Claims

1. A method for verifying robustness and data consistency under FW hot upgrade, characterized in that: Includes the following steps: S01. Insert the SSD under test into the test machine, record the SMART Log information of the SSD under test, and then perform a full low-level format operation on the SSD under test. S02. Create two physical partitions on the SSD under test. Write a test file to the first partition. The capacity of the test file is set by simulating the scenario of storing a large amount of data on the SSD under test. Calculate and save the hash check code of the test file. The hash check code is used to verify the integrity of the test file. S03. Use the FIO tool to issue high-pressure random mixed read and write to the created second partition. The random mixed read and write under IO pressure saturation state achieved by the combination of test parameters is called high-pressure random mixed read and write. K FW hot upgrade and hot degradation operations are performed during the operation of the FIO tool. During the FW hot upgrade and hot degradation operations, the test machine is randomly powered off. S04. If the new firmware has already been submitted at the moment of power failure, the new firmware will take effect after power is restored; otherwise, the old firmware will take effect. S05. Repeat steps S03 and S04; S06. After all FW hot upgrade and hot downgrade operations are completed, recalculate the hash checksum of the test files in the first partition, and compare the SMART Log information and hash checksum before and after the FW hot upgrade and hot downgrade operations. S07. If the comparison reveals that the SMART Log contains error messages exceeding the fault tolerance range, the hash checksum is inconsistent, or the SSD under test hangs or drops during the entire test, the test fails; otherwise, the test passes.

2. The verification method for robustness and data consistency under FW hot upgrade according to claim 1, characterized in that: The test parameter combination for high-pressure random mixed read and write is: BS=4K, QD=128, job=1, libaio, rwmixread=50. BS represents the data block size of each read and write operation, QD represents the queue depth, job represents the number of FIO test tasks, libaio represents the use of the asynchronous IO engine of the Linux system, and rwmixread represents the proportion of read operations in the total IO volume under mixed read and write mode.

3. The verification method for robustness and data consistency under FW hot upgrade according to claim 1, characterized in that: Error messages exceeding the tolerance range include critical warnings and media errors.

4. The verification method for robustness and data consistency under FW hot upgrade according to claim 1, characterized in that: K is set based on one or more of the following criteria: A. Determined by referring to industry standards or specifications; B. Determined based on the product's expected lifespan and usage scenarios; C. Determined based on testing time and resource limitations; D. Determined based on usage risks.

5. The verification method for robustness and data consistency under FW hot upgrade according to claim 1, characterized in that: In step S02, after creating the first partition, an EXT4 file system is created on the first partition, and the file system is mounted to a specified directory, and then a test file is written.

6. The verification method for robustness and data consistency under FW hot upgrade according to claim 1, characterized in that: The hash checksum is an MD5 checksum.

7. The verification method for robustness and data consistency under FW hot upgrade according to claim 1, characterized in that: Before starting the test, the SSD to be tested must be in a normal initial state.

8. The verification method for robustness and data consistency under FW hot upgrade according to claim 1, characterized in that: The first and second partitions each account for 50% of the capacity of the SSD being tested.

9. A verification device for robustness and data consistency during hot upgrades of a firmware, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to, when running the program instructions, execute the robustness and data consistency verification method for FW hot upgrade as described in any one of claims 1 to 8.

10. A storage medium storing program instructions, characterized in that, When the program instructions are executed, they perform the robustness and data consistency verification method for FW hot upgrade as described in any one of claims 1 to 8.

Images