An SSD data consistency test method, device, equipment and storage medium

Abstract

This application relates to the technical field of SSD data testing, and more particularly to an SSD data consistency testing method, apparatus, device, and storage medium. It includes: generating a multi-dimensional test dataset and dividing it into multiple independent data shards; determining node weights based on the real-time load status of each node, and generating corresponding routing paths for each data shard based on the weights; performing verification in response to the current data shard arriving at the current node, encapsulating the verification result and device status information into verification metadata, and forwarding it to the next hop node along the corresponding routing path until full path verification is completed; comparing the checksums of the same data shard on different nodes based on all verification metadata to identify data consistency; if an anomaly is found, locating the first faulty node, determining the cause of the fault based on the verification metadata of the faulty node, and generating a fault report. This application improves the reliability of SSD data consistency testing through a distributed collaborative verification mechanism.

Description

Technical Field

[0001] This application relates to the technical field of SSD data testing, and in particular to an SSD data consistency testing method, apparatus, device, and storage medium. Background Technology

[0002] Solid-state drives (SSDs) are core devices for modern data storage, and their performance, reliability, and data consistency are crucial in consumer electronics, enterprise data centers, and other fields. With the exponential growth of data volumes and the accelerated iteration of SSD technology (such as upgrades in flash memory chip types and the widespread adoption of the NVMe protocol), traditional manual testing methods can no longer meet the needs of efficient verification in complex scenarios. Especially in scenarios involving multi-disk collaboration, extreme concurrent access, and abnormal power outages, ensuring data consistency faces severe challenges, necessitating the construction of automated and intelligent testing systems to improve testing efficiency and coverage accuracy.

[0003] Current mainstream technical solutions mostly rely on host-driven data assembly and verification logic, monitoring data consistency through asynchronous retransmission and machine learning models. While this approach is efficient in isolated single-disk tests, it is limited by host-side verification mechanisms and struggles to cover the complex edge scenarios throughout the SSD's entire lifecycle. For example, when test data is written to edge areas such as reserved blocks and metadata areas, the host only performs simple checksums and comparisons, failing to capture the silent repair logic of the firmware layer after an abnormal power outage, leading to a "pseudo-consistency" illusion.

[0004] It can be seen that the current technical solutions cannot achieve in-depth detection of end-to-end data consistency in complex scenarios, which limits the effectiveness of testing SSD data. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this application provides an SSD data consistency testing method, apparatus, device, and storage medium. By using a distributed collaborative verification mechanism, it overcomes the limitations of retransmission verification testing, thereby improving the reliability of SSD data consistency testing.

[0006] The technical solution adopted by this application to solve its technical problem is:

[0007] Firstly, this application provides an SSD data consistency testing method, the method comprising:

[0008] A multi-dimensional test dataset is generated based on configuration file parameters, and the test dataset is divided into multiple independent data fragments.

[0009] The node weights are determined based on the real-time load status of each node in the ring network. Corresponding routing paths are generated for each data fragment based on the weights, and all data fragments are stored in the initial node of the corresponding routing path.

[0010] When the current data fragment arrives at the current node, a verification is performed. The verification result and device status information are encapsulated into verification metadata and forwarded to the next hop node along the corresponding routing path with the current data fragment until the full path verification is completed.

[0011] Based on all verification metadata, the checksums of the same data shard on different nodes are compared to identify whether there are any data consistency anomalies.

[0012] If a data consistency anomaly is found, the first faulty node is located, and the cause of the fault is determined based on the verification metadata of the faulty node, and a fault report is generated.

[0013] Optionally, the step of generating a multi-dimensional test dataset based on configuration file parameters includes:

[0014] Parse the preset configuration file to obtain the configuration file parameters, and generate standard test data, boundary feature data, and pressure load data based on the configuration file parameters;

[0015] The multi-dimensional test dataset is obtained by integrating the standard test data, boundary feature data, and stress load data.

[0016] Optionally, the step of dividing the test dataset into multiple independent data shards includes:

[0017] Based on the configuration file parameters, the sharding mode and error correction coding type are determined, and the multi-dimensional test dataset is dynamically sharded according to the sharding mode to obtain multiple initial shards;

[0018] Based on the error correction coding type, generate verification information corresponding to each initial fragment, and encapsulate each initial fragment and its corresponding verification information into a data fragment.

[0019] Optionally, the step of performing verification in response to the arrival of the current data fragment at the current node, and encapsulating the verification result and device status information into verification metadata, includes:

[0020] The pre-stored original checksum is compared with the hash value of the current data shard. The validity of the checksum is determined based on the comparison result, and the timestamp status is determined based on the timestamp of the current data shard and the last processing time of the local record of the current node.

[0021] Based on the redundancy factor set in the sharding mode, the current data shard is split into data units and redundant check code units. An error correction operation is performed based on the data units and redundant check code units to obtain an error correction result. The error correction result, checksum validity and timestamp status are integrated to form the check result.

[0022] Collect the device status information of the current node, and encapsulate the device status information and the verification result to form the verification metadata.

[0023] Optionally, the step of comparing the checksums of the same data shard on different nodes based on all verification metadata to identify whether there are data consistency anomalies includes:

[0024] Based on the data fragment identifier corresponding to each verification metadata, the data fragments are classified to generate a verification data sequence corresponding to each data fragment.

[0025] Extract the checksum validity of each node in all the verification data sequences. If the checksum validity of any node in any of the verification data sequences is abnormal, the corresponding verification data sequence is determined to be an abnormal sequence with data consistency anomalies.

[0026] Optionally, the step of locating the first faulty node if a data consistency anomaly exists, determining the cause of the fault based on the verification metadata of the faulty node, and generating a fault report includes:

[0027] Fault scoring initialization is performed for each of the aforementioned nodes;

[0028] Based on the timestamp status corresponding to the abnormal sequence, locate the first fault node in the abnormal sequence, accumulate the fault score of the fault node once, and determine the corresponding fault type based on the error correction result of the fault node.

[0029] The fault scores of each node in all the verification data sequences are summarized, the node with the highest fault score is identified as the fault source node, and the corresponding confidence level is calculated based on the score ratio.

[0030] The fault report is generated by integrating the device status information of the fault source node, the fault source node identifier, the fault type, and the confidence level.

[0031] Optionally, the step of generating corresponding routing paths for each data shard based on the weights includes:

[0032] Get the preset full set of nodes, and randomly generate the path hop count corresponding to the current data shard within the preset hop count range;

[0033] Based on the weights and the path hop count, a corresponding number of nodes are extracted from the full set of nodes without repetition, and the extraction order is used to form the routing path for the current data shard.

[0034] Secondly, this application provides an SSD data consistency testing device, comprising:

[0035] The test data sharding module is used to generate a multi-dimensional test dataset based on configuration file parameters and divide the test dataset into multiple independent data shards.

[0036] The routing path generation module is used to determine the node weight based on the real-time load status of each node in the ring network, generate corresponding routing paths for each data fragment based on the weight, and store all data fragments to the initial node of the corresponding routing path.

[0037] The node forwarding verification module is used to perform verification when the current data fragment arrives at the current node. It encapsulates the verification result and device status information into verification metadata, and forwards it to the next hop node along the corresponding routing path with the current data fragment until the full path verification is completed.

[0038] The data anomaly detection module is used to compare the checksums of the same data shard on different nodes based on all verification metadata in order to identify whether there are any data consistency anomalies.

[0039] The fault report generation module is used to locate the first fault node if there is a data consistency anomaly, determine the cause of the fault based on the verification metadata of the fault node, and generate a fault report.

[0040] Thirdly, this application provides an electronic device, comprising:

[0041] One or more processors;

[0042] One or more memory units;

[0043] And one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions that, when executed by the one or more processors, cause the electronic device to perform the methods described above.

[0044] Fourthly, this application provides a computer-readable storage medium storing a program or instructions that, when executed, implement the above-described method.

[0045] The beneficial effects of this application are as follows: Generating a multi-dimensional test dataset through configuration files and dividing it into multiple independent data shards helps to accurately capture data consistency issues of SSDs in different application scenarios. Determining node weights based on the real-time load status of each node in the ring network and allocating storage paths for data shards according to these weights effectively balances the load pressure among nodes, improving system stability and response speed. Verification is performed when a data shard arrives at the current node, and the verification result, along with device status information, is encapsulated into verification metadata. This process ensures real-time monitoring and feedback of data during transmission. The verification metadata is forwarded among nodes along with the data shards, and finally, by comparing the verification results of different nodes, a comprehensive identification of data consistency anomalies is achieved. If an anomaly is detected, the first faulty node can be quickly located, and the specific cause of the fault can be determined based on the verification metadata of the faulty node, generating a detailed fault report.

[0046] This distributed collaborative verification mechanism breaks through the limitations of traditional single-host-dominated testing methods, achieving in-depth detection of end-to-end data consistency, significantly improving the effectiveness and accuracy of SSD data testing, and ensuring the reliability and integrity of data storage. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating the SSD data consistency testing method provided in an embodiment of this application;

[0048] Figure 2 This is a schematic diagram of the virtual structure of the SSD data consistency testing device provided in this application;

[0049] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0050] The present application will be further described below with reference to the accompanying drawings and embodiments.

[0051] The following will clearly and completely describe the concept, specific structure, and resulting technical effects of this application in conjunction with embodiments and accompanying drawings, so as to fully understand the purpose, features, and effects of this application. Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are all within the scope of protection of this application. Furthermore, all connections / linkages involved in the patent do not simply refer to direct contact between components, but rather to the ability to form a better connection structure by adding or reducing connecting accessories according to specific implementation conditions. The various technical features in this application can be combined interactively without contradicting each other.

[0052] Reference Figure 1, Figure 1 This is a flowchart illustrating the SSD data consistency testing method provided in this application, which involves multiple steps provided by this application. Each step is described in detail below:

[0053] In step S1, a multi-dimensional test dataset is generated based on the configuration file parameters, and the test dataset is divided into multiple independent data fragments.

[0054] Among them, the configuration file parameters refer to the test data generation rules set in the configuration file in advance, including basic parameters such as total data volume, data type (such as binary / text), storage format, boundary coverage ratio, as well as configurations such as very small / extreme large fragments, address boundaries, and firmware special processing logic;

[0055] Among them, the multi-dimensional test dataset refers to a data set generated by integrating different test requirements, covering standard data (such as all 0 / all 1 / random numbers), boundary data (such as LBA=0, reserved block address), stress load data (such as high-concurrency read and write requests), etc., to simulate real business scenarios and extreme conditions;

[0056] Independent data sharding refers to splitting a complete dataset into multiple non-overlapping subsets according to rules. Each shard contains complete test data features, and there is no dependency between data shards. They can be used independently for different test scenarios or executed in parallel.

[0057] Specifically, by reading parameters from the configuration file, a dataset covering multiple test dimensions is automatically generated, such as containing normal business data, boundary address data, and abnormal simulation data simultaneously. Subsequently, the dataset is divided into multiple independent shards according to a sharding strategy (such as fixed-size, variable-size, or overlapping sharding), ensuring that each data shard represents the overall data characteristics while also being able to be tested independently. By adopting a sharding design, test tasks are parallelized, improving efficiency, and multi-dimensional data covers different test scenarios (such as functional verification, stress testing, and fault injection). It is worth noting that this application targets the internal components of the SSD, not its transmission link. This step is completed in the control node (global scheduling center) within the SSD. The global scheduling center is responsible for generating test data, formulating sharding strategies, allocating routing paths, and analyzing consistency verification results.

[0058] Fault injection is a technique that assesses system reliability by actively introducing specific fault scenarios. After system startup, it first reads the configuration file to obtain parameters such as fault type, probability of occurrence, and duration, and initializes the monitoring module to identify injectable nodes in the data processing flow. During data transmission or storage, the system randomly selects data fragments based on probability to perform corrupting operations, such as flipping specific locations or deleting checksums, and then continues the normal process to observe the system's ability to detect and repair abnormal data. During transmission, the system may delay the transmission of specific fragments, directly discard fragments, or shuffle the transmission order to simulate network packet loss or out-of-order delivery, testing the system's fault tolerance mechanism for transmission failures. For storage device firmware, the system injects state interference by tampering with register values ​​or clearing cached data to test the firmware's recovery capabilities and its impact on data consistency. Throughout the process, the system continuously records key indicators such as fault detection time and repair success rate. Finally, by analyzing this data, the system's stability and fault tolerance under different fault scenarios are evaluated, providing a basis for design optimization.

[0059] More specifically, in this embodiment of the application, the step of generating a multi-dimensional test dataset based on configuration file parameters includes:

[0060] The preset configuration file is parsed to obtain the configuration file parameters, and standard test data, boundary feature data, and pressure load data are generated based on the configuration file parameters.

[0061] Standard test data refers to standardized data that conforms to typical business scenarios, such as all-zero data, all-one data, random number data, or sequential / random pattern data, used to verify basic read and write functions. The system determines whether standard test data needs to be generated. If the determination is yes, the system enters the standard data generation process, including all-zero data generation, all-one data generation, random number data generation, sequential / random pattern data, and custom data template processing.

[0062] Specifically:

[0063] The all-zero data generation is achieved by initializing a data container based on the preset total data volume and data storage format, and filling all storage units in the container with 0 according to the data storage rules to generate all-zero test data.

[0064] The generation of all-1 data is also based on the preset total amount of data and storage format. The data container is initialized and all storage units in the container are filled with 1 to complete the generation of all-1 test data.

[0065] Random number generation involves calling the system's random number generation function to generate a corresponding number of random numbers based on the total data volume requirement. These random numbers are then written sequentially into a data container according to the data storage format to obtain random number test data.

[0066] Sequential / random mode data generation first determines the mode type. If it is sequential mode, starting from the initial value, a continuous sequence of values ​​is generated by incrementing sequentially according to the total data requirement, and then written into the data container according to the storage format. If it is random mode, the random number generation function is called multiple times within the specified data range to generate a non-repeating random sequence of values, which is then written into the data container to form test data.

[0067] Custom data template processing refers to the process where, when a custom data template is required, the system parses the user-uploaded custom data template file and extracts key information such as data structure, data content examples, and data generation rules. Based on this information and a preset total data volume, a data filling algorithm is used to expand or transform the template data according to the rules, generating simulated data that conforms to real business scenarios. For example, for a template simulating database logs, simulated log data with time-series characteristics and business operation logic is generated based on timestamp generation rules and operation type distribution; for a template simulating video streams, continuous simulated video data frames are generated according to video frame format, frame rate, and other requirements.

[0068] Among them, boundary feature data refers to data generated for the edge regions of the SSD address space (such as LBA=0, reserved block address, maximum LBA-1) or firmware special processing logic (such as TR IM command simulation), which is used to trigger boundary condition tests. Specifically, when the boundary data generator is started, the system automatically loads the configuration file and obtains the core parameters for boundary data generation, including the capacity information of the target storage device, address range limits, firmware special processing rule document path, etc. At the same time, it initializes the basic environment for data generation, such as allocating data storage buffers and establishing a simulated communication channel with the storage device.

[0069] Next, based on the configuration parameters, it is determined whether to generate extremely small or extremely large data shards. Extremely small shards represent the data granularity boundary, testing the storage device's ability to process the smallest data unit (such as metadata encapsulation and protocol compatibility). Extremely large shards represent the data capacity boundary, verifying the storage device's limits in processing large blocks of data (such as contiguous address allocation and space allocation algorithm stability). Both are boundary data in the data sharding dimension, covering extreme cases of storage protocols for data scale. If extremely small data shards need to be generated, the data size is forcibly set to 1 byte. According to the storage device's data storage protocol, a 1-byte data unit containing necessary metadata is constructed to complete the generation of extremely small data shards. If extremely large data shards need to be generated, a data container of the corresponding size is created based on the obtained maximum capacity of a single SSD block. The container is filled with preset default data (such as all zeros) until a capacity close to the size of a single SSD block is reached, forming extremely large data shards.

[0070] Furthermore, the address range parameters of the storage device are parsed, and data generation is performed for LBA (Logical Block Address) boundary cases. When generating address boundary data with LBA=0, a metadata tag identifying LBA=0 is added to the data header, and the corresponding data content is generated in accordance with the data format specifications of the storage device. For data with LBA=maximum value-1, the maximum LBA value of the storage device is first calculated, and then this value is subtracted by 1 as the address identifier, constructing metadata and corresponding data content containing this address identifier. For the generation of reserved block address data, metadata and simulated data with specific reserved block address identifiers are generated according to the reserved block address rules of the storage device.

[0071] The data generation process that triggers special firmware processing logic involves the system reading the firmware's special processing rule document to determine the data characteristics that trigger the special logic (such as TRIM simulated data consisting entirely of 0xFF). Taking the generation of TRIM simulated data consisting entirely of 0xFF as an example, a data container matching the storage device's data storage format is created. All storage cells within the container are filled with 0xFF, and a metadata tag identifying the data as TRIM simulated data is added to the data header, thus completing the generation of this type of data. The TRIM simulated data consisting entirely of 0xFF represents the data content boundary and is used to trigger specific operations in the firmware (such as TRIM command recognition and garbage collection mechanisms). By constructing a data pattern that conforms to the firmware rules, the firmware's recognition and response logic for special data is verified.

[0072] Finally, after all boundary data has been generated, the system initiates a data verification program to check the data's format integrity, address accuracy, and compliance with firmware-specific processing rules. If the verification passes, the generated boundary data is stored in the designated test data repository; if the verification fails, error information is recorded, and the system returns to the parameter configuration stage, awaiting user correction before regeneration. The verification stage represents the quality boundary of the boundary data to ensure the validity of test cases.

[0073] The stress load data refers to load data simulating high concurrency and abnormal scenarios (such as random power outages and delayed transmissions) to verify the performance and consistency of SSDs under extreme conditions. The generation process begins with setting up the load environment. The system first allocates computing resources according to the configuration file, creates a multi-threaded runtime environment, and initializes network communication and storage simulation interfaces, while loading test parameters (such as concurrent request range, access mode type, IOPS target value, and bandwidth stress rules). Subsequently, the system starts multi-threaded parallel generation of read and write requests. Each thread randomly selects a storage address to initiate a read operation according to a preset strategy, or generates a write instruction containing random data pointing to the target address, thereby simulating high-concurrency business load.

[0074] To recreate complex real-world scenarios, the system dynamically switches between random and sequential access modes. Specifically, in random mode, threads generate discrete address sequences; in sequential mode, they generate continuous address sequences according to an increasing or decreasing pattern. The two modes are mixed and alternated based on time or request ratio. Simultaneously, the system monitors IOPS and bandwidth usage in real time, dynamically matching the target load value by increasing or decreasing the number of threads, adjusting request frequency, or packet size, ensuring the load remains within a preset range.

[0075] Ultimately, all read and write requests and their execution status are logged into log files, and key metrics (such as IOPS, bandwidth utilization, and response time) are presented in real time through a visualization interface. After the test, the complete results are stored for analyzing system performance and data consistency, forming a structured dataset covering multi-dimensional stress scenarios, providing a quantitative basis for performance optimization and capacity planning.

[0076] Specifically, the system first reads the rules in the configuration file to obtain the configuration file parameters, and then determines the total amount of data, type, and coverage of boundary / stress scenarios. Next, it generates standard data (to verify basic functions), boundary feature data (to trigger address edge or firmware logic tests), and stress load data (to simulate abnormal scenarios) to ensure that the test covers both normal and extreme conditions.

[0077] It is worth noting that both standard test data and custom-generated data undergo a format validation module after generation to check if the data format meets preset requirements. If the validation passes, the generated test data is stored in the specified storage location for subsequent data consistency testing. If the validation fails, a corresponding error message is returned, and the data generation process is terminated, waiting for the user to reconfigure or correct the data template before resuming the generation operation.

[0078] Furthermore, the standard test data, boundary feature data, and stress load data are integrated to obtain the multi-dimensional test dataset.

[0079] Specifically, the three types of data are integrated into a multi-dimensional test dataset, and data independence and parallel testing are achieved through sharding strategies (such as fixed size or dynamic allocation), thereby comprehensively evaluating the data consistency of SSDs in complex scenarios.

[0080] More specifically, assuming the configuration file sets the total data volume to 50GB, the data type to binary, the boundary coverage ratio to 15%, and requires simulating power outage and high-concurrency scenarios, firstly, 30GB of random binary data can be generated as standard test data for regular read / write verification; then, 7.5GB of data can be generated as boundary feature data, of which 5GB is LBA=0 address data and 2.5GB is reserved block simulation data; subsequently, 12.5GB of data can be generated as stress load data, including power outage scenario fragmentation (such as forced interruption of transmission) and high-concurrency log data.

[0081] Furthermore, the step of dividing the test dataset into multiple independent data fragments includes:

[0082] Based on the configuration file parameters, the sharding mode and error correction coding type are determined, and the multi-dimensional test dataset is dynamically sharded according to the sharding mode to obtain multiple initial shards.

[0083] Among them, the sharding mode refers to the rules for splitting the dataset into independent shards, including fixed size (such as 10GB per shard), variable size (dynamically adjusted according to the importance of the data), or overlapping sharding (adjacent shards have some data duplicated);

[0084] Among them, error correction coding type refers to the coding algorithm used for data verification and recovery, which is used to detect and correct errors in transmission or storage.

[0085] Further, according to the error correction coding type, verification information corresponding to each initial segment is generated, and each initial segment and its corresponding verification information are paired and encapsulated into a data segment.

[0086] Specifically, the multi-dimensional test dataset is structured using configuration file parameters (such as sharding mode and error correction coding type). First, the dataset is dynamically split into multiple initial shards according to the sharding mode, ensuring that each shard conforms to preset rules (such as fixed size or covering specific data types). Next, for each initial shard, verification information (such as redundant check codes) is generated based on the error correction coding type, and the shard and verification information are bound and encapsulated into independent data shards, thereby ensuring data integrity through verification information.

[0087] More specifically, assume that the three types of data are sharded into fixed-size chunks (e.g., 5GB each), and mixed to form 10 independent initial chunks. Chunk 3 may contain power-down simulation data and LBA=0 address data, used to test anomaly recovery capabilities; chunk 7 contains high-concurrency logs and standard random data, used to verify multi-threaded performance. The final integrated dataset comprehensively covers functional, boundary, and stress testing requirements.

[0088] More specifically, the machine learning optimizer periodically collects system runtime data, including sharding processing time, data consistency verification results, and failure occurrences. Based on this historical data, it trains a failure prediction model, analyzes the impact of different sharding parameters (such as redundancy and offset) on system performance and data consistency, calculates the optimal combination of sharding parameters using reinforcement learning algorithms, and feeds adjustment suggestions back to the sharding strategy engine. Subsequently, the sharding strategy engine receives parameter adjustment suggestions from the machine learning optimizer and dynamically modifies the sharding parameters. For example, when a certain type of failure is predicted to have a high probability of occurrence, redundancy is increased to improve data recovery capabilities; based on changes in data access patterns, sharding offsets are adjusted to optimize data storage and access efficiency. After adjustment, the sharding operation is re-executed using the new parameters to ensure that the sharding strategy is always in an optimal state.

[0089] When machine learning is used to assist testing, system performance and reliability can be continuously improved through machine learning optimizers. Its operation is based on data collection and preprocessing. By connecting to the interfaces of various testing modules, it collects multi-dimensional data in real time, including sharding parameters, routing paths, fault injection records, and data consistency verification results. This data undergoes preprocessing operations such as cleaning, denoising, and completion to form a standardized training dataset. The predictive model built based on historical fault data is the core of optimization. For example, when using the random forest algorithm, classification accuracy is improved by dividing the training and test sets and performing cross-validation parameter tuning. For fault scenarios with significant temporal characteristics, LSTM networks are used to capture long-term data dependencies, ultimately forming a forward-looking fault prediction capability.

[0090] At the policy optimization level, the optimizer abstracts the sharding policy and routing decision into an action space for reinforcement learning. It constructs a reward function using key metrics such as system throughput and data consistency retention rate, and trains the agent using algorithms such as Q-learning or deep deterministic policy gradients, enabling it to autonomously explore the optimal policy combination under complex system conditions. This dynamic learning mechanism allows the optimizer to continuously evolve based on environmental feedback, achieving adaptive adjustment of policy parameters.

[0091] For the testing process, the optimizer combines the predictive model with the output of quantified test case risk levels. Based on the output of the fault prediction model, it analyzes the probability and severity of different test cases causing system failures. Test cases with a high probability of causing serious failures are marked as high-risk test cases, and priority queues are built according to risk levels. During testing, high-risk test cases are executed first to discover potential system defects earlier. Finally, the system periodically analyzes the model output and runtime data, providing reports to system administrators and developers. Based on actual system operation, the machine learning model and optimization strategies are continuously adjusted to ensure that the optimizer can always provide accurate and effective recommendations.

[0092] In step S2, the node weights are determined based on the real-time load status of each node in the ring network. Based on the weights, corresponding routing paths are generated for each data fragment, and all data fragments are stored in the initial node of the corresponding routing path.

[0093] Among them, the ring network is a closed ring topology structure formed by connecting multiple SSD test nodes through a high-speed data channel, and data can flow bidirectionally within the ring;

[0094] Among them, real-time load status refers to the current workload of the node, including dynamic indicators such as CPU utilization, memory usage, and flash read / write speed, which reflect the node's ability to process data;

[0095] The routing path refers to the path through which data fragments are transmitted in a ring network. It consists of a series of nodes in sequence, and the generation of the path needs to take into account the node weights to achieve load balancing.

[0096] Specifically, by dynamically sensing the real-time load status of each node in the ring network, the node weights are calculated to quantify their processing capabilities, thereby generating an optimal routing path for each data shard. Nodes with high weights (low load) are preferentially selected as initial nodes, ensuring that data shards are transmitted starting from nodes with stronger processing capabilities, avoiding high-load nodes becoming bottlenecks. After all shards are independently stored in the initial nodes of their corresponding paths, subsequent transmissions will proceed according to the preset paths, thereby achieving load balancing and improving testing efficiency and data consistency assurance capabilities.

[0097] More specifically, an SSD node health trend prediction model can be built by extracting relevant data for each SSD node from a historical database, including flash write counts, erase counts, bad block counts, temperature variations, and performance metrics. Then, time series analysis algorithms (such as ARMIMA and LSTM) are used to establish the SSD health prediction model, analyzing the changing trends of flash wear rate and controller failure rate. Based on the model's prediction results, early warnings can be issued for SSD nodes that may fail, providing a basis for maintenance planning, load assessment, and weight adjustment.

[0098] More specifically, in this embodiment of the application, the step of generating corresponding routing paths for each data shard based on the weights includes:

[0099] Get the preset full set of nodes, and randomly generate the path hop count corresponding to the current data shard within the preset hop count range.

[0100] The full set of nodes refers to the set of all available node resources in the system, including nodes with different performance, load or geographical location; the path hop count refers to the number of nodes that a data fragment needs to pass through in the transmission path. For example, a hop count of 3 means that the fragment needs to pass through 3 nodes.

[0101] Specifically, a hop count value (e.g., 3) is randomly generated from a preset hop count range (e.g., minimum 2, maximum 5) and used as the path hop count for the current data fragment. The larger the hop count, the longer the path, which increases the redundancy transmission capability of the data fragment.

[0102] Furthermore, based on the weight and the path hop count, a corresponding number of nodes are extracted from the full set of nodes without repetition, and the extraction order is used to form the routing path for the current data shard.

[0103] Specifically, based on node weights, a number of nodes, equal to the hop count, are randomly selected from the entire node set without repetition. Nodes with higher loads have lower weights, and nodes with lower weights have a lower probability of being selected. After each selection, the selected nodes are removed from the candidate pool, and the weight ratios of the remaining nodes are recalculated. For example, if the original weights are [0.5, 0.3, 0.1, 0.1], after removing the first node, the remaining weights are normalized to [0.3 / 0.5, 0.1 / 0.5, 0.1 / 0.5]. Subsequently, the nodes are arranged into routing paths according to the selection order, ensuring that the corresponding data shards pass through these nodes in sequence, achieving load balancing and randomness in verification, and avoiding testing vulnerabilities.

[0104] More specifically, where permissible, at least one alternative routing path can be generated for each data fragment, and network performance data, including actual transmission latency, throughput, and packet loss rate, can be collected in real time. This data is compared with expected performance metrics; if actual performance falls short of expectations, the reasons are analyzed and routing strategies are adjusted. For example, when the packet loss rate of a certain path suddenly increases, the system automatically switches to an alternative path; and based on network traffic trends, path selection strategies are adjusted in advance to ensure the network always operates efficiently and stably.

[0105] In step S3, when the current data fragment arrives at the current node, a verification is performed. The verification result and device status information are encapsulated into verification metadata and forwarded to the next hop node along the corresponding routing path with the current data fragment until the full path verification is completed.

[0106] Among them, device status information refers to the hardware and firmware status data of the current node, including flash memory write count, temperature, voltage, controller register status, etc.

[0107] Among them, verification metadata refers to structured data that combines and encapsulates verification results with device status information, and is used to record the processing status of data fragments at the current node.

[0108] Specifically, when a data fragment arrives at the current node, the system triggers a verification process to check the integrity of the fragment data, including checksums and ECC error correction. Simultaneously, it collects device status information from the current node (such as flash memory health and controller temperature), encapsulates both into verification metadata, and appends it to the data fragment. Subsequently, the data fragment is forwarded to the next-hop node along a preset routing path, repeating the verification, status collection, and metadata encapsulation process until full-path verification is completed. In this case, the data fragment is transmitted completely along the preset path in the ring network, with each node recording its own status and verification results. This ultimately forms a set of verification metadata containing the full-path status, which is used for subsequent analysis of data consistency and location of potential faults.

[0109] More specifically, the step of performing verification in response to the arrival of the current data fragment at the current node, and encapsulating the verification result and device status information into verification metadata includes:

[0110] The pre-stored original checksum is compared with the hash value of the current data shard. The validity of the checksum is determined based on the comparison result, and the timestamp status is determined based on the timestamp of the current data shard and the last processing time of the local record of the current node.

[0111] The original checksum refers to the correct data check value (such as CRC32 or SHA hash) pre-stored in the system; the timestamp status refers to the comparison between the timestamp of the current data fragment and the last processing time recorded locally on the node, used to determine whether the fragment arrives within a reasonable time range, such as to prevent replay attacks or delay processing.

[0112] Specifically, by comparing the pre-stored original checksum with the hash value of the current data shard, it verifies whether the data has been tampered with or corrupted. At the same time, it determines whether the shard arrives within a reasonable time window based on the timestamp, in order to avoid delay or replay attacks.

[0113] Furthermore, based on the redundancy factor set in the sharding mode, the current data shard is split into data units and redundant check code units. An error correction operation is performed based on the data units and redundant check code units to obtain an error correction result. The error correction result, checksum validity, and timestamp status are then integrated to form the verification result.

[0114] The timestamp status refers to the comparison between the timestamp of the current data shard and the last processing time of the node's local record. It is used to determine whether the shard arrives within a reasonable time range, such as to prevent replay attacks or delay processing.

[0115] Among them, the data unit refers to the original data part in the current data segment after removing redundant check codes, which is used for actual business logic verification; the redundant check code unit refers to the error correction code data generated based on the redundancy factor, which is used for error location and repair when the data unit is faulty;

[0116] Among them, the error correction result refers to the state of the data unit after error correction through the redundancy check unit, including three cases: no error, correctable error, or uncorrectable error.

[0117] Specifically, based on the redundancy factor in the sharding mode, the shards are split into data units and redundant check code units. The check codes are used to correct errors in the data units, generating error correction results (such as error location and repair status). The error correction results are then combined with the checksum validity and timestamp status to form a complete check result.

[0118] Furthermore, the device status information of the current node is collected, and the device status information and the verification result are encapsulated to form the verification metadata.

[0119] Specifically, the device status information of the current node (such as flash memory lifespan, temperature, etc.) is collected and encapsulated together with the verification results into verification metadata, which is then forwarded to the next-hop node along with the data fragment. The verification metadata refers to structured data that combines and encapsulates the verification results (including error correction results, checksum validity) with the device status information, and is used to record the processing status of the data fragment at the current node.

[0120] More specifically, the system reads the pre-stored original checksum, calculates the hash value of the currently received data fragment, and compares the two to determine that the checksum is valid if they match. At the same time, it checks whether the difference between the timestamp of the current data fragment (e.g., 16:00:00) and the local last processing time (e.g., 15:59:50) is within the allowable range (e.g., ±1 minute). If it is within the allowable range, the timestamp status is determined to be normal. For timeout nodes (data transmission time exceeds a preset threshold) and sudden delay nodes (delay suddenly increases significantly), the system records its node identifier, the time of the anomaly, the delay duration, and other information as the timestamp status.

[0121] Assuming a redundancy factor of 20%, the data allocation is divided into data units (8GB) and redundancy check code units (1.6GB) based on the redundancy factor of 20%. An error correction algorithm (such as Reed-Solomon decoding) is performed on the data units. A single bit error is found at the 3GB position. After repairing it with the check code, an error correction result (which can correct the error) is generated. Then, the checksum validity (valid), timestamp status (normal), and error correction result (repaired) are integrated to form the verification result.

[0122] In step S4, based on all verification metadata, the checksums of the same data shard on different nodes are compared to identify whether there are any data consistency anomalies.

[0123] Specifically, by aggregating full-path verification metadata, the checksums of the same data shard across all nodes in the ring network are compared to identify whether data inconsistencies are caused by transmission errors, storage failures, or node anomalies. In particular, the checksums generated by each node for the same data shard are extracted and checked for complete consistency. If discrepancies exist, they are marked as potential anomalies.

[0124] More specifically, in this embodiment of the application, the step of comparing the checksums of the same data shard on different nodes based on all verification metadata to identify whether there are any data consistency anomalies includes:

[0125] The data fragments are categorized according to the data fragment identifiers corresponding to each verification metadata, and a verification data sequence corresponding to each data fragment is generated.

[0126] Specifically, all verification metadata is categorized based on data fragment identifiers, and the verification metadata of the same fragment at different nodes is combined into a verification data sequence according to the transmission order. Each sequence completely records the processing status of the fragment throughout the entire path. The data fragment identifier is a unique number or tag used to identify data fragments and distinguish the verification metadata of different data fragments. The verification data sequence refers to the sequence of verification metadata generated by the same data fragment at all nodes, arranged according to the transmission path order, reflecting the processing status of the data fragment at each node along the entire path.

[0127] Furthermore, the checksum validity of each node in all the verification data sequences is extracted. If the checksum validity of any node in any verification data sequence is abnormal, the corresponding verification data sequence is determined to be an abnormal sequence with data consistency anomalies.

[0128] Specifically, the checksum validity status of each node is extracted from each verification data sequence. If the checksum validity of any node in a sequence is "invalid" (e.g., data corruption or verification failure), the sequence is determined to be an abnormal sequence, indicating that the corresponding data fragment has an inconsistency problem during transmission or storage.

[0129] In step S5, if there is a data consistency anomaly, the first faulty node is located, and the cause of the fault is determined based on the verification metadata of the faulty node, and a fault report is generated.

[0130] Specifically, the entire path of the data shard is traversed to verify the data sequence and find the first node whose checksum validity is "invalid". For example, if the sequence includes nodes (Node A, Node B, Node C) in sequence, and the shard is normal at Node A, normal at Node B, and abnormal at Node C, then Node C is the first faulty node.

[0131] Subsequently, the verification metadata of the node is extracted, and its verification results (such as error correction failure, checksum mismatch) and device status information (such as flash memory high temperature, voltage fluctuation) are combined to comprehensively determine the cause of the failure. For example, if the device status shows that the flash memory temperature is too high and the error correction result is "unrepairable error", it may be caused by storage media failure.

[0132] Finally, the fault cause (fault type), node location (node ​​identifier), timestamp status, and device status information are encapsulated into a structured report for maintenance personnel to process. For example, the output fault report may indicate "NodeC flash memory temperature exceeds the limit, causing data shard_001 storage error. It is recommended to check the heat dissipation system immediately."

[0133] More specifically, in this embodiment of the application, the step of locating the first faulty node if a data consistency anomaly exists, determining the cause of the fault based on the verification metadata of the faulty node, and generating a fault report includes:

[0134] For each of the nodes, a fault score is initialized.

[0135] Specifically, a fault score rating table is constructed, a profile is created for each node, and an initial fault score (usually 0) is set for all nodes for quantitative assessment of the probability of subsequent cumulative faults.

[0136] Furthermore, based on the timestamp status corresponding to the abnormal sequence, the first fault node in the abnormal sequence is located, the fault score of the fault node is accumulated once, and the corresponding fault type is determined according to the error correction result of the fault node.

[0137] Specifically, for each abnormal sequence, based on the timestamp status of the abnormal sequence, that is, the time order in which the data shards arrive at each node, the first node whose checksum validity is invalid is identified and marked as a potential fault node (i.e., a fault node). It is worth noting that a fault node does not necessarily mean that it is the source of the fault, but is only a candidate node for fault location.

[0138] Subsequently, a fault score is assigned based on the error correction result of the faulty node (e.g., "correctable error" or "uncorrectable error") and the device status (e.g., flash memory overheating, abnormal voltage). That is, the score of the node in the fault score table is increased by 1. In addition, the fault type is further distinguished based on the error correction result of the faulty node. Specifically, if the error correction fails, it is initially judged as "flash memory damage"; if the error correction is successful, it is judged as "controller error".

[0139] Furthermore, the fault scores of each node in all the verification data sequences are aggregated, the node with the highest fault score is identified as the fault source node, and the corresponding confidence level is calculated based on the score ratio.

[0140] Specifically, after completing the analysis of the verification data sequences corresponding to all data shards, the confidence level is calculated based on the fault score table. Specifically, the maximum value of the fault score is first obtained. If the maximum value is 0, it indicates no fault, and "No fault detected" is returned with a confidence level of 1 (confidence level is between 0 and 1; the higher the confidence level, the more reliable the system). If a fault score exists, the node with the highest score is selected as the fault source node, and the confidence level is calculated based on the proportion of the fault source node's score to the total score of all nodes.

[0141] Furthermore, the device status information of the fault source node, the fault source node identifier, the fault type, and the confidence level are integrated to form the fault report.

[0142] Specifically, the device status information extracted from the fault source node is called to obtain the detailed internal status of the fault source node. The fault type obtained in the previous step is verified a second time based on the device status information. Finally, a fault report including the fault source node identifier, fault type and corresponding confidence level is obtained to achieve fault location.

[0143] More specifically, through data collection and integration, multi-source data, including consistency verification results, fault location information, and trend prediction data, are aggregated from testing modules and monitoring systems, and cleaned to ensure accuracy. Based on this, the fault details processing and classification stage systematically organizes the identifiers of fault source nodes (such as node ID and module location) and their equipment status information, combining factors such as error type and occurrence time. Simultaneously, based on trend prediction models or statistical analysis results, a confidence level is assigned to each fault, quantifying its probability of occurrence. Finally, in the multi-dimensional report generation stage, this integrated information, along with basic indicator analysis and optimization suggestions, is structured and arranged. The fault characteristics, impact scope, and risk level are detailed in charts and text, forming a comprehensive report that includes equipment status, node location, fault type, and confidence level assessment. This report supports export in multiple formats to meet different scenario requirements.

[0144] Reference Figure 2 , Figure 2This is a virtual structural diagram of the SSD data consistency testing device provided in this application. A second aspect of this application provides an SSD data consistency testing device, comprising:

[0145] The test data sharding module 100 is used to generate a multi-dimensional test dataset based on configuration file parameters and divide the test dataset into multiple independent data shards.

[0146] The routing path generation module 200 is used to determine the node weight based on the real-time load status of each node in the ring network, generate corresponding routing paths for each data fragment based on the weight, and store all data fragments to the initial node of the corresponding routing path respectively.

[0147] The node forwarding verification module 300 is used to perform verification when the current data fragment arrives at the current node, encapsulate the verification result and device status information into verification metadata, and forward it to the next hop node along the corresponding routing path with the current data fragment until the full path verification is completed.

[0148] The data anomaly identification module 400 is used to compare the checksums of the same data shard on different nodes based on all verification metadata in order to identify whether there are any data consistency anomalies.

[0149] The fault report generation module 500 is used to locate the first fault node if there is a data consistency anomaly, determine the cause of the fault based on the verification metadata of the fault node, and generate a fault report.

[0150] The SSD data consistency testing device described in this application embodiment can execute the SSD data consistency testing method provided in the above embodiments. The SSD data consistency testing device has the corresponding functional steps and beneficial effects of the SSD data consistency testing method described in the above embodiments. For details, please refer to the embodiments of the above SSD data consistency testing method. The embodiments of this application will not be repeated here.

[0151] This application also provides an electronic device, please refer to... Figure 3 , Figure 3This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include a processor and a memory, which can be connected via a bus or other means. The processor may be a Central Processing Unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof. The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the SSD data consistency testing method in the embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the SSD data consistency testing method in the above method embodiments.

[0152] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. One or more modules are stored in the memory and, when executed by the processor, perform the SSD data consistency test method as described in the above method embodiments. Specific details of the above electronic device can be understood by referring to the corresponding descriptions and effects in the above method embodiments, and will not be repeated here. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it may include the processes of the embodiments of the above methods. The storage medium may be read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium may also include combinations of the above types of memory.

[0153] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification. It should be noted that the foregoing embodiments are illustrative and not restrictive of this application, and that alternative embodiments may be devised by those skilled in the art without departing from the scope of the appended claims.

Claims

1. A method for testing SSD data consistency, characterized in that, The method includes: A multi-dimensional test dataset is generated based on configuration file parameters, and the test dataset is divided into multiple independent data fragments. The node weights are determined based on the real-time load status of each node in the ring network. Corresponding routing paths are generated for each data fragment based on the weights, and all data fragments are stored in the initial node of the corresponding routing path. When the current data fragment arrives at the current node, a verification is performed. The verification result and device status information are encapsulated into verification metadata and forwarded to the next hop node along the corresponding routing path with the current data fragment until the full path verification is completed. Based on all verification metadata, the checksums of the same data shard on different nodes are compared to identify whether there are any data consistency anomalies. If a data consistency anomaly is found, the first faulty node is located, and the cause of the fault is determined based on the verification metadata of the faulty node, and a fault report is generated.

2. The SSD data consistency testing method according to claim 1, characterized in that, The steps for generating a multi-dimensional test dataset based on configuration file parameters include: Parse the preset configuration file to obtain the configuration file parameters, and generate standard test data, boundary feature data, and pressure load data based on the configuration file parameters; The multi-dimensional test dataset is obtained by integrating the standard test data, boundary feature data, and stress load data.

3. The SSD data consistency testing method according to claim 1, characterized in that, The step of dividing the test dataset into multiple independent data fragments includes: Based on the configuration file parameters, the sharding mode and error correction coding type are determined, and the multi-dimensional test dataset is dynamically sharded according to the sharding mode to obtain multiple initial shards; Based on the error correction coding type, generate verification information corresponding to each initial fragment, and encapsulate each initial fragment and its corresponding verification information into a data fragment.

4. The SSD data consistency testing method according to claim 3, characterized in that, The step of performing verification when the current data fragment arrives at the current node, and encapsulating the verification result and device status information into verification metadata includes: The pre-stored original checksum is compared with the hash value of the current data shard. The validity of the checksum is determined based on the comparison result, and the timestamp status is determined based on the timestamp of the current data shard and the last processing time of the local record of the current node. Based on the redundancy factor set in the sharding mode, the current data shard is split into data units and redundant check code units. An error correction operation is performed based on the data units and redundant check code units to obtain an error correction result. The error correction result, checksum validity and timestamp status are integrated to form the check result. Collect the device status information of the current node, and encapsulate the device status information and the verification result to form the verification metadata.

5. The SSD data consistency testing method according to claim 4, characterized in that, The step of comparing the checksums of the same data shard on different nodes based on all verification metadata to identify whether there are data consistency anomalies includes: Based on the data fragment identifier corresponding to each verification metadata, the data fragments are classified to generate a verification data sequence corresponding to each data fragment. Extract the checksum validity of each node in all the verification data sequences. If the checksum validity of any node in any of the verification data sequences is abnormal, the corresponding verification data sequence is determined to be an abnormal sequence with data consistency anomalies.

6. The SSD data consistency testing method according to claim 5, characterized in that, The steps of locating the first faulty node if a data consistency anomaly exists, determining the cause of the fault based on the verification metadata of the faulty node, and generating a fault report include: Fault scoring initialization is performed for each of the aforementioned nodes; Based on the timestamp status corresponding to the abnormal sequence, locate the first fault node in the abnormal sequence, accumulate the fault score of the fault node once, and determine the corresponding fault type based on the error correction result of the fault node. The fault scores of each node in all the verification data sequences are summarized, the node with the highest fault score is identified as the fault source node, and the corresponding confidence level is calculated based on the score ratio. The fault report is generated by integrating the device status information of the fault source node, the fault source node identifier, the fault type, and the confidence level.

7. The SSD data consistency testing method according to claim 1, characterized in that, The step of generating corresponding routing paths for each data shard based on the weights includes: Get the preset full set of nodes, and randomly generate the path hop count corresponding to the current data shard within the preset hop count range; Based on the weights and the path hop count, a corresponding number of nodes are extracted from the full set of nodes without repetition, and the extraction order is used to form the routing path for the current data shard.

8. An SSD data consistency testing device, characterized in that, include: The test data sharding module is used to generate a multi-dimensional test dataset based on configuration file parameters and divide the test dataset into multiple independent data shards. The routing path generation module is used to determine the node weight based on the real-time load status of each node in the ring network, generate corresponding routing paths for each data fragment based on the weight, and store all data fragments to the initial node of the corresponding routing path. The node forwarding verification module is used to perform verification when the current data fragment arrives at the current node. It encapsulates the verification result and device status information into verification metadata, and forwards it to the next hop node along the corresponding routing path with the current data fragment until the full path verification is completed. The data anomaly detection module is used to compare the checksums of the same data shard on different nodes based on all verification metadata in order to identify whether there are any data consistency anomalies. The fault report generation module is used to locate the first fault node if there is a data consistency anomaly, determine the cause of the fault based on the verification metadata of the fault node, and generate a fault report.

9. An electronic device, characterized in that, include: One or more processors; One or more memory units; And one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, the one or more computer programs including instructions that, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a program or instructions that, when executed, implement the method as described in any one of claims 1 to 7.

Images