A data storage integrity verification method and system for solid state memory

Abstract

The application relates to the technical field of data storage, and discloses a data storage integrity verification method and system for a solid-state memory. The method comprises the following steps: collecting read-write instructions, programming and erasing cycle times, and charge retention times; performing voltage drift analysis based on the read-write instructions to obtain drift grouping data, and counting flip frequency distribution; performing storage block degradation analysis by combining the flip frequency distribution, the programming and erasing cycle times, and the charge retention times, to obtain a high-risk area; extracting a risk voltage drift amount, counting read-write interference amounts, and combining the risk voltage drift amount and the read-write interference amounts into an error feature vector; identifying a hard error type through a support vector machine model; performing reliability determination based on the hard error type and the flip frequency distribution, generating a check trigger signal, and positioning an accurate bit flip address; combining a data importance degree library to evaluate a repair priority sequence; performing data repair, periodically checking a repaired successful area, and obtaining an integrity verification report. The method improves the reliability of a storage system.

Description

Technical Field

[0001] This invention relates to the field of data storage technology, and in particular to a data storage integrity verification method and system for solid-state storage. Background Technology

[0002] Currently, in the field of information storage, data integrity verification is crucial for ensuring data security and reliability. With the surge in data volume and the widespread application of non-volatile storage such as solid-state drives (SSDs), ensuring that data is not tampered with or damaged during storage and transmission has become an urgent need. However, the physical characteristics of non-volatile storage make it susceptible to errors such as bit flips after long-term use, posing a challenge to accurately locating high-risk areas and implementing efficient verification without impacting performance. This necessitates technologies capable of dynamically monitoring the storage environment, promptly identifying risks, and strengthening reliability checks to maintain data integrity.

[0003] In one existing technology, the system first traverses the physical addresses of all storage units according to a preset time period, reading the raw data block by block. Next, a pre-set error correction code decoder verifies the read data, identifying and correcting correctable bit errors. For errors discovered during verification, the system records their physical location and accumulates the error count. When the error count of a storage block exceeds a statically set global threshold, the system marks the block as "suspicious" or "bad." Subsequently, a data migration process is triggered, moving the data in the marked block to a spare block and updating the mapping relationship in the address mapping table. Traditional methods rely on periodic global scanning and static threshold judgment, which suffers from problems such as lag, inability to dynamically assess unit health, and potential interference with system performance during the inspection process.

[0004] Existing technologies suffer from low reliability in storage systems. Summary of the Invention

[0005] This invention provides a data storage integrity verification method and system for solid-state storage to improve the reliability of storage systems.

[0006] Firstly, in order to solve the above-mentioned technical problems, the present invention provides a data storage integrity verification method for solid-state storage, comprising: Collect read / write commands and program / erase cycle counts from the solid-state storage controller, and the charge retention time of the solid-state storage. According to the read / write instructions, voltage drift is clustered and grouped to obtain drift group data, and the flip frequency is statistically analyzed based on the drift group data to obtain the flip frequency distribution; Based on the flip frequency distribution, the number of programmable erase cycles, and the charge retention time, a memory block degradation analysis is performed to obtain core risk blocks, and risk boundaries are determined based on the core risk blocks to obtain high-risk areas. Based on the flip frequency distribution, the drift grouping data, and the high-risk area, the risk voltage drift is extracted to obtain the risk voltage drift amount. The cumulative number of interferences caused by adjacent block operations is counted as the read / write interference amount, which is combined with the risk voltage drift amount to obtain the error feature vector. Based on the error feature vector, the hard error type is identified by a pre-built support vector machine model; Based on the hard error type and the flip frequency distribution, the error evolution trend is analyzed and the reliability is determined to obtain the inspection trigger signal; Based on the inspection trigger signal, perform precise bit flip location to obtain the accurate bit flip address, and combine it with the pre-stored data importance library to perform repair priority evaluation and obtain a priority sequence; Based on the priority sequence and the accurate bit flip address, data repair is performed to obtain the readback level state, and the state is updated based on the readback level state to obtain the repaired area. Periodic data integrity checks are performed on the successfully repaired areas to obtain integrity verification reports.

[0007] Secondly, the present invention provides a data storage integrity verification system for solid-state storage, comprising: The data acquisition module is used to acquire read and write commands and program erase cycle counts of the solid-state storage controller, as well as the charge retention time of the solid-state storage. The bit flip frequency analysis module is used to perform voltage drift clustering and grouping according to the read and write instructions to obtain drift group data, and to count the flip frequency according to the drift group data to obtain the flip frequency distribution. The degradation risk analysis module is used to perform memory block degradation analysis based on the flip frequency distribution, the number of programming erase cycles, and the charge retention time to obtain core risk blocks, and to determine risk boundaries based on the core risk blocks to obtain high-risk areas. The error feature construction module is used to extract the risk voltage drift amount based on the flip frequency distribution, the drift grouping data and the high-risk area, obtain the risk voltage drift amount, and count the cumulative number of interferences caused by adjacent block operations as the read-write interference amount, and combine it with the risk voltage drift amount to obtain the error feature vector; The hard error identification module is used to identify the hard error type based on the error feature vector using a pre-built support vector machine model, and obtain the hard error type. The inspection trigger module is used to analyze the error evolution trend and make a reliability determination based on the hard error type and the flip frequency distribution, and obtain an inspection trigger signal; The priority evaluation module is used to perform precise bit flip location based on the inspection trigger signal, obtain the accurate bit flip address, and combine it with the pre-stored data importance library to perform repair priority evaluation and obtain a priority sequence. The repair module is used to perform data repair according to the priority sequence and the accurate bit flip address, obtain the readback level state, and update the state according to the readback level state to obtain the repair success area; The integrity verification module is used to periodically verify the data integrity of the successfully repaired area and obtain an integrity verification report.

[0008] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention collects read and write instructions, the number of programming erase cycles and charge holding time, combines voltage drift analysis and DBSCAN clustering to group drift patterns, and statistically analyzes the flip frequency distribution, thereby realizing dynamic monitoring and trend extraction of the state of the storage cell. This overcomes the lag problem of traditional periodic global scanning, thus enabling earlier and more accurate identification of potential degradation areas and improving the preventive maintenance capability of the storage system.

[0009] (2) This invention adopts a support vector machine model, dynamically adjusts the classification boundary through the RBF kernel function, and combines historical error feature vectors and time series trend parameters to identify hard error types, thereby achieving high-precision classification and early warning of complex error patterns, improving the accuracy and adaptability of error identification, and providing a reliable basis for targeted repair.

[0010] (3) This invention calculates the recurrence probability of bit flipping based on Bayesian network, determines the repair priority sequence in combination with data importance library, and uses LDPC error correction and write mask technology to perform data repair. After repair, periodic integrity verification is performed through SHA-256 hash algorithm to realize intelligent scheduling and long-term verification of data repair, thereby improving the reliability of data storage and the overall lifespan of the system. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the data storage integrity verification method for solid-state storage provided in the first embodiment of the present invention; Figure 2 This is a schematic diagram of the data storage integrity verification system for solid-state storage provided in the second embodiment of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] Reference Figure 1 The first embodiment of the present invention provides a data storage integrity verification method for solid-state storage, comprising the following steps: S11, collects read / write commands and program / erase cycle counts from the solid-state storage controller, and the charge retention time of the solid-state storage; S12, according to the read / write instruction, perform voltage drift clustering and grouping to obtain drift group data, and statistically analyze the flip frequency based on the drift group data to obtain the flip frequency distribution; S13, based on the flip frequency distribution, the number of programming erase cycles and the charge retention time, perform memory block degradation analysis to obtain core risk blocks, and determine risk boundaries based on the core risk blocks to obtain high-risk areas; S14. Based on the flip frequency distribution, the drift grouping data, and the high-risk area, the risk voltage drift is extracted to obtain the risk voltage drift. The cumulative number of interferences caused by adjacent block operations is counted as the read / write interference. The risk voltage drift is combined with the risk voltage drift to obtain the error feature vector. S15, Based on the error feature vector, perform hard error type identification through a pre-built support vector machine model to obtain the hard error type; S16, Based on the hard error type and the flip frequency distribution, analyze the error evolution trend and make a reliability determination to obtain the check trigger signal; S17. Based on the inspection trigger signal, perform bit flip precise positioning to obtain the accurate bit flip address, and combine it with the pre-stored data importance library to perform repair priority evaluation and obtain a priority sequence. S18, perform data repair according to the priority sequence and the accurate bit flip address to obtain the readback level state, and update the state according to the readback level state to obtain the repaired area; S19, perform periodic data integrity checks on the successfully repaired areas to obtain an integrity verification report.

[0014] In step S11, the read / write commands and programming / erase cycle counts of the solid-state storage controller, as well as the charge retention time of the solid-state storage, are collected.

[0015] Specifically, the system collects read / write commands and program / erase cycle counts from the solid-state storage (SSD) controller, as well as the charge retention time of the SSD. Read / write command data originates from the command queue generated in real-time by the SSD controller during operation. Parsing this queue yields the physical address and operation type corresponding to each read or write operation. Program / erase cycle count data comes from the wear leveling management module integrated within the SSD controller. This module records and maintains the total number of erase operations accumulated for each physical storage block since its activation, stored using the physical block address as an index. Acquiring charge retention time data requires a specific testing process. This process involves applying a preset retention voltage to a selected storage block while the SSD is idle and maintaining it for a fixed retention period (e.g., 3 hours). After the retention period, the actual voltage of the storage cells is read. By comparing the actual voltage with the theoretical standard voltage during writing, the proportion of storage cells with voltage drift exceeding a preset drift threshold is calculated out of the total number of cells in the block. This proportion is used as a quantitative indicator of the charge retention capability of the storage block, i.e., the charge retention time.

[0016] It should be noted that the charge retention time characterizes the ability of a storage cell to retain its stored charge under the absence of external electrical stress interference. Its decay is a major cause of data silencing errors. In one specific embodiment, a target storage block can be selected during a period when the solid-state memory is idle; a holding voltage is applied to it, typically lower than or equal to the nominal read reference voltage of the flash memory chip, and maintained for a predetermined stress time, such as typical values ​​recommended by JEDEC standards or chip manufacturer reliability models, like 1 hour, 3 hours, or 24 hours; after the stress ends, the threshold voltage of each cell within the block is immediately read and compared with the theoretical voltage value recorded before the stress was applied. Compare and statistically analyze the number of memory cells whose absolute voltage drift exceeds a preset silent error threshold, for example, 50mV, and calculate the percentage of this number relative to the total number of cells in the block. The lower this percentage, the stronger the charge retention capability of the block. To facilitate subsequent unified processing, this percentage can be normalized or its reciprocal can be taken to obtain a value that is positively correlated with the retention capability and has uniform dimensions, which can be used as the charge retention time characteristic value for subsequent analysis. Those skilled in the art will understand that the specific values ​​of the above test voltage, stress time, and drift threshold can be adaptively adjusted according to the specific NAND flash memory chip model, process node, and reliability specifications used.

[0017] This step provides the most basic and essential multi-dimensional characteristic data for subsequent storage status analysis and risk assessment. Read / write instruction sequences reveal system access patterns and the logical location of potential bit-flip events. The number of program erase cycles directly reflects the cumulative physical wear and tear on storage cells due to frequent use, and is a key parameter for assessing block lifetime. Charge retention time characterizes the storage cell's inherent ability to maintain data stability in an inactive state; its decay is one of the main causes of silent data errors. These three types of data form the comprehensive monitoring foundation for storage cell dynamic operational stress, historical wear load, and static retention performance, enabling subsequent analysis to correlate and analyze the degradation behavior of storage cells from multiple levels of operation, lifetime, and physical characteristics.

[0018] In step S12, voltage drift clustering is performed according to the read / write instruction to obtain drift group data, and the switching frequency is statistically analyzed based on the drift group data to obtain the switching frequency distribution, including: The read / write instructions are parsed to obtain the physical address and theoretical voltage; Obtain the storage cell voltage corresponding to the physical address, and mark the physical address whose storage cell voltage is not equal to the theoretical voltage as a bit-flipped address; Based on the bit-flip address, the difference between the storage cell voltage and the theoretical voltage is calculated as the voltage drift amount, and drift timing data including the bit-flip address, voltage drift amount and timestamp is output. Based on the drift time series data, the data is segmented into time series segments through a preset sliding window, and the drift pattern is grouped using the DBSCAN clustering algorithm to obtain drift group data; Based on the drift grouping data, the frequency of bit flipping is statistically analyzed to obtain the flipping frequency distribution.

[0019] Specifically, each read / write instruction is first parsed to extract the physical address of the memory cell pointed to by the instruction and the theoretical standard voltage value expected to be written or read in this operation. Then, the actual voltage measurement value of the memory cell corresponding to that physical address is read through the flash memory interface. The actual voltage measurement value of each physical address is compared with the theoretical standard voltage value. When the difference between the two voltage values ​​is not zero, it is determined that a bit flip has occurred at that physical address, and it is marked as a bit-flipped address. For each marked bit-flipped address, the algebraic difference between its actual voltage measurement value and the theoretical standard voltage value is calculated; this difference is defined as the voltage drift. The system records a time stamp containing the bit-flipped address, the calculated voltage drift, and the time of the read / write operation, forming a drift timing data record.

[0020] After continuous drift time-series data forms a data stream, it is segmented using a pre-defined sliding window of fixed length, resulting in a series of data segments arranged chronologically. For all data points within each segment (each data point consists of voltage drift and a timestamp), the DBSCAN clustering algorithm is applied to group them by drift pattern. This algorithm includes pre-defined parameters: neighborhood radius and minimum sample size. The neighborhood radius defines the distance threshold for determining whether data points are densely clustered, and the minimum sample size defines the minimum number of data points required to form a cluster. These two parameters are determined based on statistical analysis of historical drift time-series data.

[0021] The specific steps are as follows: Collect historical drift time-series data generated during normal system operation; calculate the Euclidean distance between all pairs of data points; and select the 5th percentile of these distance values ​​as the initial reference value for the neighborhood radius. The minimum sample size is set by multiplying the average number of data points within the historical segments of the sliding window by a fixed percentage (e.g., 10%). The neighborhood radius and minimum sample size determined by the above method are used as preset parameters input into the DBSCAN clustering algorithm. During algorithm execution, for each drift time-series data point within a sliding window segment, within a circular area centered on each point and with the neighborhood radius as the radius, if the number of data points contained within this area reaches or exceeds the minimum sample size, the points within this area are marked as core points. Core points and their neighboring points are then grouped into the same cluster group through density connectivity. Finally, drift grouping data is output, where each group represents a voltage drift pattern that repeatedly occurs within a specific time period.

[0022] It is worth noting that the key to grouping drift time series data by drift pattern using the DBSCAN algorithm lies in reasonably setting the neighborhood radius and the minimum number of samples. In a preferred embodiment, the parameters can be determined as follows: First, for the time series data after sliding window segmentation, calculate the distance from each data point to its k-th nearest neighbor, where the initial value of k can be set as the dimension of the data point. In this example, the voltage drift is 1-dimensional, and the timestamp can be regarded as another dimension after standardization, so k can be initially set to 2 or 4. Then, sort the nearest neighbor distance values ​​of all points in ascending order and plot the curve. This refers to the k-distance graph; the curve usually shows a clear inflection point, and the nearest neighbor distance value corresponding to the inflection point can be used as a reasonable estimate of the neighborhood radius; the minimum number of samples can usually be set to the data dimension plus 1, that is, the initial value is 3, or it can be fine-tuned according to an empirical proportion of the average number of data points in the sliding window, such as 5%~10%; those skilled in the art know that DBSCAN is quite sensitive to parameters. In actual deployment, it can be trained offline using historical normal data, and the initial parameters can be fine-tuned by observing the clustering effect, such as the cluster separation degree and the proportion of noise points, in order to obtain a stable grouping effect.

[0023] After obtaining the drift packet data, the system statistically analyzes the bit flip frequency distribution based on the packet results. Using physical addresses as the basic unit, the system retrieves the number of times each physical address appears in all drift packet data, divides this number by the total observation time span, and calculates the bit flip frequency per unit time for that physical address. By traversing all the appearing physical addresses, a flip frequency distribution covering the entire monitored memory space is obtained. This distribution records the frequency of bit flips occurring at different physical locations.

[0024] This step transforms raw, discrete bit-flip events into quantitative information with pattern characteristics and statistical regularities. By calculating and recording voltage drift, the bit-flip phenomenon is moved from qualitative judgment to quantitative characterization. Using a time-series sliding window and DBSCAN clustering to group the drift data, it is possible to identify temporally clustered bit-flip patterns caused by the same physical degradation mechanisms (such as accelerated wear of specific memory blocks or interference between adjacent cells). This goes beyond simple counting and reveals the correlation between errors. Finally, the statistically obtained flip frequency distribution provides a view of memory cell vulnerability from a spatial perspective.

[0025] In step S13, based on the flip frequency distribution, the number of programmable erase cycles, and the charge retention time, a memory block degradation analysis is performed to obtain core risk blocks. Then, based on these core risk blocks, risk boundaries are determined to identify high-risk regions, including: The flip frequency distribution, the number of programmed erase cycles, and the charge retention time are used as the degradation feature dataset; Based on the degradation feature dataset, the degradation score is obtained by scoring the storage physical blocks using the random forest algorithm. When the degradation score does not exceed the preset degradation threshold, the storage physical block is marked as a normal storage physical block. When the degradation score exceeds the preset degradation threshold, the storage physical block is marked as an abnormal storage physical block, and physically adjacent abnormal storage physical blocks are aggregated and output as core risk blocks. Based on the core risk block, the gradient change rate of the degradation score of the storage physical page at the edge of each abnormal storage physical block is calculated to obtain the boundary gradient; When the boundary gradient does not exceed a preset gradient mutation threshold, the corresponding storage physical page is marked as a normal boundary. When the boundary gradient exceeds the preset gradient mutation threshold, the corresponding storage physical page is marked as a risk boundary. The risk boundaries are combined into a continuous address range and output as a high-risk region.

[0026] Specifically, for each physical block of solid-state memory, the system extracts three corresponding feature values. The first feature value is the average bit flip frequency of all cells in the physical block, obtained from the flip frequency distribution. The second feature value is the current program-erase cycle count of the physical block, directly read from the wear leveling management module. The third feature value is the quantized value of the charge retention time of the physical block, obtained from the charge retention test. These three values ​​are combined into a three-dimensional vector, and the set of vectors for all physical blocks constitutes the degradation feature dataset.

[0027] The system uses a pre-built random forest model to process the degradation feature dataset to calculate the degradation score for each physical storage block. The random forest model is built based on the historical degradation feature dataset and the corresponding final failure state labels of the storage blocks. The historical degradation feature dataset is derived from each record collected during the system's long-term operation. Each record corresponds to a physical storage block and contains three feature values: the average bit flip frequency of all cells in the block, the current number of program erase cycles for the block, and the quantized charge retention time of the block. The final failure state label of the storage block is a binary value, determined through independent, highly reliable offline detection to determine whether each block has experienced physical failure. Key parameters of the model include the number of decision trees, the maximum depth of each tree, and the number of features considered when splitting a node. The number of decision trees is preset to one value (e.g., one hundred trees), the maximum depth is preset to another value (e.g., ten levels), and the number of features considered when splitting a node is preset to the number of features randomly selected from all three features (e.g., two). The specific values ​​of these preset parameters are determined by a grid search combined with cross-validation. That is, a set of candidate values ​​for each parameter is set in advance. For each combination of parameters, a temporary model is trained using a part of the historical degradation feature dataset, and its prediction accuracy is verified in another part of the dataset. Finally, the parameter combination with the highest average validation accuracy is selected as the final parameters of the model.

[0028] During model training, a bootstrap sampling method is used to extract a subset of samples with replacement from the historical dataset to construct each decision tree. The construction process for each decision tree is as follows: starting from the root node, the Gini impurity of the sample data within the node is calculated. Then, the node is bisected using three features: flip frequency, number of programmed erase cycles, and charge retention time, along with all possible splitting thresholds. The feature that maximizes the reduction in Gini impurity and its corresponding splitting threshold are selected as the splitting rule for that node, dividing the sample data into two child nodes. This process is recursively repeated until the number of samples within a node falls below a preset lower limit or reaches a preset maximum depth. This node then becomes a leaf node, and the proportion of failed samples within it is recorded. During model application, the feature vector of the current storage physical block—a three-dimensional vector composed of the block's average bit flip frequency, number of programmed erase cycles, and charge retention time—is input into each decision tree. Each tree starts from the root node and, according to the splitting rules stored in the nodes, traverses along the direction where the corresponding feature value in the feature vector is satisfied to a leaf node. The proportion of the failure state of the historical samples in that leaf node is used as the tree's preliminary degradation judgment value for that block. Finally, the arithmetic mean of the preliminary degradation judgment values ​​output by all decision trees is taken to obtain the degradation score of the storage physical block. This score is a continuous value, and the larger the value, the more severe the degradation of the storage physical block.

[0029] A preset degradation threshold is used to classify storage physical blocks as normal or abnormal based on their degradation scores. This threshold is determined by collecting the final degradation scores and corresponding failure states (failed or normal) of all storage physical blocks of the same model before the end of their lifecycle. The distribution of historical degradation scores for all normal storage physical blocks is analyzed, and the score value corresponding to the 95th percentile of this distribution is selected as the degradation threshold. Selecting the 95th percentile ensures that the threshold is near the upper limit of normal sample scores, thus effectively identifying abnormal blocks with scores higher than the vast majority of normal blocks. In application, the system compares the calculated degradation score of each storage physical block with the degradation threshold. If the score is not greater than the degradation threshold, the storage physical block is marked as normal; if the score is greater than the degradation threshold, it is marked as abnormal.

[0030] After marking is complete, the system scans the physical address space, aggregating all contiguous abnormal storage physical blocks into a single unit, which is then output as a core risk block. Each core risk block represents a spatially contiguous and concentrated abnormal storage region.

[0031] To define the impact range of core risk blocks, the system performs risk boundary determination. For each anomalous physical storage block within the core risk block, the system calculates the degree of drastic change in degradation scores between adjacent physical storage pages at its physical edge. The degradation score of the physical storage page at the edge of the anomalous block is taken, and its degradation score is compared with that of the adjacent physical storage page located outside the core risk block. The difference between the two scores is calculated, and this difference is divided by the address interval between the two physical pages (usually the address span of one page). The quotient obtained is the boundary gradient. This value reflects the average rate of change in degradation scores per unit address distance when crossing the risk block boundary.

[0032] A preset gradient mutation threshold is used to determine whether a boundary gradient signifies a rapid spread of risk. This threshold is determined by collecting boundary gradient data around all identified core risk blocks over a relatively long historical period, calculating the distribution of these boundary gradient values, and selecting the 90th percentile of this distribution as the gradient mutation threshold. This threshold represents the high level of boundary gradients observed in the past. In application, the system compares each calculated boundary gradient with the gradient mutation threshold. If the boundary gradient is not greater than the gradient mutation threshold, the corresponding physical storage page is marked as a normal boundary; if the boundary gradient is greater than the gradient mutation threshold, it is marked as a risk boundary. Finally, all physical storage pages marked as risk boundaries are merged according to the continuity of their physical addresses to form one or more consecutive address ranges, which are output as high-risk regions.

[0033] This step transforms the multi-dimensional monitoring data of the storage unit into a quantitative health score. Based on the score, it identifies degraded blocks that are spatially clustered. By analyzing the gradient changes of the score at the edge of the block, it accurately delineates the boundaries of high-risk areas, thus providing a clear spatial location basis for subsequent error feature extraction and repair.

[0034] In step S14, based on the flip frequency distribution, the drift grouping data, and the high-risk area, the risk voltage drift is extracted to obtain the risk voltage drift amount. The cumulative number of interferences caused by adjacent block operations is counted as the read / write interference amount, and combined with the risk voltage drift amount to obtain an error feature vector, including: Based on the flip frequency distribution and the high-risk area, extract the flip frequency distribution corresponding to the high-risk area to obtain the risk flip frequency; When the risk reversal frequency does not exceed a preset reversal frequency threshold, the high-risk area is marked as a healthy area; When the risk flip frequency exceeds a preset flip frequency threshold, the voltage drift amount corresponding to the high-risk area is extracted based on the drift grouping data to obtain the risk voltage drift amount. The cumulative number of times that adjacent storage physical blocks perform read and write operations within a preset time period, causing voltage drift in the target storage physical block, is counted as the read / write interference amount. The risk voltage drift and the read / write interference are combined into an error feature vector using the same physical address.

[0035] Specifically, firstly, based on the continuous address range defined by the high-risk region, the bit flip frequency values ​​corresponding to all memory cells within this range are retrieved from the flip frequency distribution. The arithmetic mean of these frequency values ​​is calculated to obtain the risk flip frequency of the high-risk region. A preset flip frequency threshold is used to determine whether the high-risk region is in an electrically active state requiring further analysis. This threshold is determined by collecting historical flip frequency data of all memory regions marked as "healthy" during long-term system operation. The distribution of flip frequency values ​​in these healthy regions is analyzed, and the frequency value corresponding to the 99th percentile of this distribution is selected as the flip frequency threshold. The system compares the calculated risk flip frequency with the flip frequency threshold. If the risk flip frequency is not greater than the flip frequency threshold, the system determines that the high-risk region has insignificant electrical activity at the current moment, marks it as a healthy region, and does not perform subsequent feature extraction in this step. If the risk flip frequency is greater than the flip frequency threshold, the region is determined to be an electrically active risk region, and its features need to be further extracted.

[0036] For regions identified as electrically active risk areas, the system filters drift packet data based on their address range. Drift packet data contains the physical addresses of all bit-flipping events in the historical record, the corresponding voltage drift amounts, and timestamps. The system extracts all data records whose physical addresses fall within the risk area and obtains the corresponding voltage drift value set from these records. The system calculates the average absolute value of the voltage drift values ​​in this set as an indicator of the typical voltage shift intensity in the region, and outputs this as the risk voltage drift amount.

[0037] The calculation of read / write interference is performed on each physical storage block within the target risk area. The system sets a fixed statistical time period (e.g., the most recent 24 hours). Within this time period, all read / write commands issued by the storage controller are monitored. For each physical storage block within the target risk area, the system identifies other physical storage blocks directly adjacent to its physical address, forming a set of adjacent blocks for that target block, based on the physical structure definition of the flash memory. The system continuously parses each read / write command; when the target physical address in the command belongs to the set of adjacent blocks for the target block, a potential interference event for that target block is determined, and the count is accumulated. The cumulative number of potential interference events caused by all adjacent block read / write operations within this time period is defined as the read / write interference amount for that target physical storage block.

[0038] After completing the above calculations, the system constructs an error feature vector using storage physical pages as the basic unit. For each storage physical page within the risk region, its corresponding error feature vector is a two-dimensional vector. The first dimension of the vector is the risk voltage drift calculated from the storage physical block containing the page. The second dimension of the vector is the read / write interference calculated from the storage physical block containing the page. By combining the risk voltage drift and read / write interference according to their common physical address, the system generates an error feature vector for each physical page within the risk region.

[0039] This step transforms identified high-risk areas into quantifiable error features. By setting thresholds to filter inactive areas, processing efficiency is improved. The extracted risk voltage drift reflects the degree of degradation within the memory cell, while read / write interference measures the impact of external operations. The combination of these two elements constitutes an error feature vector, providing multi-dimensional data for accurate error type identification.

[0040] In step S15, based on the error feature vector, hard error type identification is performed using a pre-built support vector machine model to obtain the hard error type.

[0041] In one implementation, the support vector machine model construction process includes: Obtain historical error feature vectors and corresponding historical hard error types; An initial vector machine model is constructed using the support vector machine algorithm, and the classification boundary is initialized using the RBF kernel function, with a penalty coefficient set. Based on the historical error feature vector, the curvature adjustment parameter of the classification boundary is calculated using the kernel density estimation method, and the curvature of the classification boundary is dynamically adjusted through the RBF kernel function; Based on the historical hard error types, the penalty coefficient is optimized using a grid search method to determine the error tolerance threshold; Cross-validate the adjusted classification boundaries and calculate the set of classification accuracy and error distribution after boundary adjustment; When the error distribution set exceeds the error tolerance threshold, the gradient descent method is used to re-optimize the curvature parameter of the classification boundary, and the classification boundary is iteratively updated until the error distribution set does not exceed the error tolerance threshold. By using RBF kernel function mapping, the optimized classification boundary is matched with the hard error type in a multi-dimensional space to determine the hyperplane equation corresponding to each error feature vector; Based on the time series characteristics of the historical hard error types, the sliding window method is used to extract dynamic trend parameters. The dynamic trend parameters are then correlated with the hyperplane equation to construct the final support vector machine model.

[0042] Specifically, the first step is to acquire historical training data, which comes from cases continuously collected and validated during the system's long-term operation. Historical training data includes historical error feature vectors and their corresponding historical hard error type labels. Each historical error feature vector consists of two values: risk voltage drift and read / write interference. The historical hard error type labels are classification results determined by diagnosing the memory cells that generated the error feature using independent, highly reliable offline physical detection and circuit analysis methods, such as categorizing them into "hold characteristic failure," "programming interference," and "read interference."

[0043] In constructing the initial model, a Support Vector Machine (SVM) algorithm was employed, with radial basis functions (RBFs) used as the kernel function to initialize the classification boundary of the data in the high-dimensional feature space. A penalty coefficient was also set to control the model's tolerance for classification errors, with its initial value set to a standard intermediate value. Based on the acquired historical error feature vector dataset, a kernel density estimation algorithm was used to calculate the curvature adjustment parameter required for the classification boundary. The kernel density estimation algorithm estimates the shape of the entire data distribution by calculating the density distribution of each historical data point in the feature space, thereby deriving the curvature parameter that the classification boundary should have in local regions. Using this parameter, the curvature of the initial classification boundary in the feature space was dynamically adjusted via the radial basis function to better match the actual shape of the historical data distribution.

[0044] Next, the penalty coefficient is optimized using a grid search method based on historical hard error type labels. The grid search method generates a series of candidate penalty coefficient values ​​within a pre-defined numerical range with a fixed step size. For each candidate value, a temporary model is trained using historical data, and its classification accuracy is evaluated on a reserved validation dataset. The candidate value that maximizes the classification accuracy is selected as the optimized penalty coefficient. The error tolerance threshold is determined during this process, and its value is set as the classification error rate produced by the model on the validation set under the optimal penalty coefficient.

[0045] Cross-validation is used to evaluate the classification boundary after curvature adjustment and penalty coefficient optimization. The historical dataset is randomly divided into several non-overlapping subsets. One subset is used as the test set in turn, and the rest as the training set, for multiple training and testing rounds. The average classification accuracy of the model after boundary adjustment across all these testing rounds is calculated, and features of all misclassified cases are collected to form an error distribution set. The system compares the range of the error distribution set with an error tolerance threshold. If the range of the error distribution set exceeds the error tolerance threshold, gradient descent is used to re-optimize the curvature parameters of the classification boundary. Gradient descent calculates the gradient of the current model's loss function with respect to the curvature parameters on the training set, and then updates the curvature parameter values ​​in the opposite direction of the gradient with a fixed step size to gradually reduce the loss function. This process is iterated until the range of the new model's error distribution set no longer exceeds the error tolerance threshold.

[0046] After the classification boundary stabilizes, the optimized nonlinear classification boundary is matched with different historical hard error types in a high-dimensional space through the mapping relationship of radial basis functions, determining the decision hyperplane equation for each error type. The parameters of the hyperplane equation are directly solved during the training process of the support vector machine. Finally, combined with the time series features of historical hard error type labels, the sliding window method is used to extract dynamic trend parameters. The sliding window method slides a window of fixed length on the time axis, and the rate of change of the frequency of each type of hard error within each window is statistically analyzed, which serves as the dynamic trend parameter. These dynamic parameters, representing the error evolution trend, are correlated with the previously determined hyperplane equation, representing the static classification boundary, and the constant term parameters in the hyperplane equation are corrected so that the model can consider the recent evolution trend of error types during classification, thus constructing the final support vector machine model.

[0047] The significance of this step is that by using an optimized support vector machine model, it can achieve high-precision classification and dynamic trend prediction of erroneous features, thereby providing early warning and providing a core basis for subsequent reliability assessment and repair decisions.

[0048] In step S16, based on the hard error type and the flip-flop frequency distribution, the error evolution trend is analyzed and a reliability determination is made to obtain the check trigger signal, including: Based on the hard error type, the physical address is indexed to obtain the precise error address; Based on the precise address of the error and the distribution of the bit flip frequency, the error growth rate is obtained by analyzing the evolution trend of the bit flip frequency through the calculation of the first derivative of the time series. When the error growth rate does not exceed a preset error growth rate threshold, the exact address of the error is determined to be in a reliable qualified state. When the error growth rate exceeds a preset error growth rate threshold, the precise address of the error is determined to be in a reliability-to-check state, and a check trigger signal is generated.

[0049] Specifically, based on the physical address information carried in the hard error type data, the index is used to locate the precise memory cell where the specific type of hard error occurred, thus obtaining the precise address of the error.

[0050] After obtaining the precise address of the error, the system extracts the historical observation data corresponding to that address from the bit flip frequency distribution. The bit flip frequency distribution not only includes the current unit-time flip frequency value for each address but also records the historical sequence of that frequency value at fixed time intervals. The system constructs a discrete time series with time as the horizontal axis and the bit flip frequency value corresponding to the precise address of the error as the vertical axis. To analyze the evolution trend of the bit flip frequency at that address, the system calculates the first derivative of the time series. For two adjacent time points in the series, the frequency value of the later time point is subtracted from the frequency value of the earlier time point, and then divided by the time interval between the two time points to obtain the instantaneous rate of change within that time interval. By traversing all adjacent points of the entire time series, a series of instantaneous rates of change are calculated, and then the arithmetic mean of these instantaneous rates of change is taken. The final average value is defined as the error growth rate of the precise address of the error.

[0051] A preset error growth rate threshold is used to determine whether the reliability status of an address has reached a critical point requiring immediate inspection. This threshold is determined by collecting historical error growth rate data for all memory cell addresses that have ultimately been verified as long-term stable and reliable during the initial phase of system deployment or within a known healthy operating cycle. The distribution of error growth rate values ​​for these reliable cells is analyzed, and the rate value corresponding to the 95th percentile of this distribution is selected as the error growth rate threshold.

[0052] The system compares the calculated error growth rate of the precise address with a preset error growth rate threshold. If the calculated error growth rate is not greater than the threshold, the system determines that the reliability of the precise address is acceptable. This means that although a specific type of hard error has occurred at the address, the rate of deterioration of its error frequency is still within a controllable and normal range, and currently does not pose an imminent risk of propagation. If the calculated error growth rate is greater than the threshold, the reliability of the precise address is determined to be pending inspection. This indicates that the error frequency of the address is accelerating, posing a risk of rapid error propagation or impending data corruption. For addresses determined to be in a pending inspection state, the system generates an inspection trigger signal. This signal is a logical instruction containing the precise address of the error, the corresponding hard error type, and a high error growth rate indicator, used to trigger subsequent precise diagnosis and repair processes.

[0053] This step advances the assessment of storage unit reliability from a static determination of the existence of errors to a dynamic analysis of error evolution trends, thereby enabling effective maintenance before errors lead to actual data loss and improving the overall preventive maintenance capabilities and reliability of the storage system.

[0054] In step S17, based on the inspection trigger signal, precise bit flip location is performed to obtain the accurate bit flip address. Combined with a pre-stored data importance database, a repair priority assessment is performed to obtain a priority sequence, including: Based on the inspection trigger signal, the storage physical block corresponding to the exact address of the error is marked as the area to be inspected; Based on the area to be inspected, the integrity of the same physical storage block is verified by multiple reads through adjusting the read reference voltage, and the accurate bit flip position is determined by the ECC decoder to obtain the accurate bit flip address. Based on the accurate bit-flipped address, the recurrent flipped posterior probability is calculated using a pre-constructed Bayesian network model to obtain the recurrence probability; Based on the recurrence probability and combined with the pre-stored data importance database, the priority sequence for bit flip repair is determined by the weighted product method, thus obtaining the priority sequence.

[0055] Specifically, the check trigger signal contains the precise address of the error transmitted from step S16 and its associated hard error type. The system first determines the storage physical block to which the error belongs based on the precise address of the error, and marks the entire storage physical block as the area to be checked.

[0056] For the area to be inspected, the system generates a series of different read reference voltage values ​​with fixed steps within a preset voltage adjustment range. A complete data read operation is performed on the same physical storage block using each reference voltage value, resulting in multiple sets of potentially slightly different raw data. This data is then fed into the hardware-integrated ECC decoder. The ECC decoder decodes and verifies each set of read data based on a preset error correction code algorithm (such as BCH code or LDPC code). By comparing the decoding results of the read data under different reference voltages and combining the checksum calculation of the error correction code, the ECC decoder can identify and correct correctable errors, and ultimately determine the exact physical location of the bit error causing the read difference, outputting the accurate bit flip address.

[0057] It is worth noting that the aforementioned adjustment of the read reference voltage refers to generating a series of different read voltage levels within a reasonable offset range centered on the nominal read voltage, for example, ±100mV, with a certain voltage step size, such as 10mV or 25mV, under the support of the features provided by the NAND flash memory interface controller. A full-page read is then performed on the target physical block using each voltage level to obtain multiple sets of raw data. By comparing the differences in read data under different voltages and the error correction information from the ECC decoder, the actual threshold voltage distribution of the memory cell can be determined more accurately, thereby assisting in determining the exact electrical cause and location of bit flips. The specific voltage adjustment range and step size can be set according to the specifications for read voltage adjustment function in the feature table of the NAND flash memory chip used.

[0058] After obtaining the accurate bit-flip address, the system calculates the probability of a recurrent bit-flip at that address using a pre-built Bayesian network model. This Bayesian network model is constructed based on historical data, which consists of all bit-flip events recorded during the system's long-term operation. Each record contains the following features: hard error type, error growth rate before the error occurred, risk voltage drift, read / write interference, degradation score of the associated storage physical block, and a crucial posterior label—whether the address experienced a bit-flip again within a fixed observation period after the initial repair. The Bayesian network is structured as a directed acyclic graph with six nodes: degradation score, risk voltage drift, read / write interference, hard error type, error growth rate, and recurrence label. The degradation score serves as the parent node, pointing to three child nodes: risk voltage drift, read / write interference, and hard error type. This indicates that the overall degradation state of the memory block simultaneously affects its electrical parameters and error type. The risk voltage drift and read / write interference serve as intermediate nodes, both pointing to the error growth rate node. This indicates that electrical interference jointly determines the dynamic evolution trend of the error. The error growth rate node and the hard error type node both point to the final recurrence label node, indicating that the dynamic trend and static type of the error jointly determine the probability of recurrence.

[0059] It is worth noting that the labeled historical data required for the support vector machine model and the Bayesian network model can be accumulated and constructed in the following ways: During the R&D verification or early deployment phase of solid-state memory, high-precision semiconductor parameter testers can be used to test memory cells with different known physical defects, such as tunnel oxide layer breakdown and charge trap accumulation, to simulate their voltage drift and bit flip behavior under different pressures, PEC, hold times, and read / write interference, and record the corresponding feature vectors and manually labeled hard error types, such as hold failure and program interference, to form an initial training set; During the operation of mass-produced products, offline deep diagnostic tools, such as diagnostic firmware based on error pattern analysis, can be used to perform post-analysis of occasional high-frequency error cells, and their characteristics and diagnostic conclusions can be used as new samples to continuously expand and optimize the model; The recurrence label can be obtained by intensively monitoring the corresponding address for a period of time after the repair is implemented and recording whether bit flips occur again.

[0060] Since degradation scores, error growth rates, risk voltage drift, and read / write interference are continuous variables, they need to be discretized to construct a probability table. Discretization uses equal-frequency binning, which sorts the historical data values ​​of each continuous variable and divides them into several equal intervals (e.g., five intervals), ensuring that each interval contains approximately the same number of data records. Hard error types and recurrence labels are discrete variables and do not require processing. For each node, the specific combinations of values ​​for all its parent nodes in the historical dataset are traversed. For each combination of parent node values, the proportion of times the node takes each discrete value under that condition relative to the total number of records under that condition is calculated; this proportion is the conditional probability. This statistical process is performed on all nodes and all possible combinations of parent node values, ultimately yielding a complete set of conditional probability tables, which constitute the parameter set of the Bayesian network.

[0061] When applying the model, the hard error type, error growth rate, risk voltage drift, read / write interference, and degradation score of the physical storage block corresponding to the current accurate bit flip address are input into the network as observational evidence. Before input, this continuous variable evidence needs to be mapped to corresponding discrete interval labels according to the same discretization interval rules used when constructing the model. The network uses a probability propagation algorithm (such as a connection tree algorithm) to infer from the stored conditional probability table to calculate the posterior probability that the recurrence label node is "yes" under the specific set of input evidence. This probability value is the recurrence probability.

[0062] The pre-stored data importance database is an indexed database using logical block addresses or file identifiers. Each record contains a data object identifier and its corresponding importance weight value. This weight value is determined according to a predefined strategy during system initialization or data writing. For example, areas storing core operating system data are assigned higher weights, while user cache data areas are assigned lower weights. The specific settings of importance weight values ​​are tiered based on business logic or management strategies, such as being divided into four levels: critical, important, normal, and minor, with a fixed weight value assigned to each level.

[0063] It is worth noting that the data importance library is used to weigh the value of the data itself when evaluating repair priorities; its construction and maintenance can be achieved in various ways and work in conjunction with the host system or file management module; in one embodiment, file attributes can be obtained from the operating system or file system interface, for example, assigning higher weights to data blocks marked as system files, read-only files, or from specific protected directories; assigning lower weights to user temporary files or cached data; by monitoring the access history of logical blocks, frequently read or recently written hot data is assigned higher weights to ensure the reliability of critical data for system performance; applications can be supported in marking the importance of specific data ranges through API interfaces; the importance weight can be quantified into several discrete levels, such as high, medium, and low, corresponding to numerical weights such as 1.0, 0.5, and 0.2, and the mapping relationship between the data and the logical address is recorded by the controller or driver layer in a specific table in non-volatile memory when the data is written, forming the data importance library; this library can be updated accordingly as data is written, moved, and deleted.

[0064] The system determines the repair priority sequence using a weighted product method based on the recurrence probability and a data importance database. First, based on the accurate bit-flip address, the system maps its corresponding importance weight in the data importance database to that address. Then, it calculates the priority score for that bit-flip address, which is the product of the recurrence probability and the importance weight. This process iterates through all accurate bit-flip addresses that need to be processed, calculating their respective priority scores. Finally, all bit-flip addresses to be repaired are sorted in descending order of priority score to generate a priority sequence. A higher score indicates a higher likelihood of recurrence and a more important impact on the data, thus warranting a higher repair priority.

[0065] This step transforms the warning signals generated in the preceding steps into specific and executable repair scheduling instructions, providing a clear and quantifiable execution sequence for subsequent repair actions, thereby optimizing system maintenance efficiency and data protection effectiveness.

[0066] In step S18, data repair is performed according to the priority sequence and the accurate bit flip address to obtain the readback level state, and the state is updated according to the readback level state to obtain the repair success area, including: Based on the priority sequence, the original binary data of the storage physical page to be repaired is extracted to obtain the data stream to be repaired; Based on the data stream to be repaired, error correction codewords are generated by a hardware LDPC encoder, and a programming prohibition bitmap is generated by a NAND flash interface controller as a write mask. According to the write mask, the error correction codeword is overwritten to the accurate bit flip address, and the level state of the accurate bit flip address after overwriting is read back to obtain the readback level state; The readback level state is compared with the error correction codeword. If all bits are consistent, a repair success flag is generated. If there are inconsistencies, the codeword is overwritten again and a write consistency check is performed. If all bits are consistent, a repair success flag is generated. If there are inconsistencies, a repair failure flag is generated. Based on the successful repair flag and the failed repair flag, update the availability status and physical address of the storage physical block to obtain the successfully repaired area.

[0067] Specifically, the system processes each bit-flipped address to be repaired sequentially according to the priority sequence. For the currently processed address, the system first reads all the original binary data of the physical page containing that address through the storage controller, forming the data stream to be repaired, and then sends it to the hardware-integrated LDPC encoder. The LDPC encoder operates based on a pre-designed and hardware-embedded sparse parity-check matrix. This sparse parity-check matrix is ​​a binary matrix whose number of rows corresponds to the length of the parity bits, and the number of columns corresponds to the sum of the length of the data stream to be repaired and the length of the parity bits. The vast majority of elements in the matrix are zero, and the number and position of non-zero elements are designed according to a predetermined graph structure (e.g., a quasi-cyclic structure) to ensure encoding performance and facilitate hardware implementation. The encoding process involves treating the data stream to be repaired as a row vector, performing a matrix multiplication operation on the sparse parity-check matrix in the binary field to generate a parity bit vector. Subsequently, the original data stream to be repaired and the calculated parity bit vector are sequentially concatenated to form a complete error-correcting codeword. Simultaneously, the NAND flash interface controller generates a programming prohibition bitmap as a write mask based on the current state of the target storage physical block. The rule for generating this write mask is that the controller reads the programming status of all memory cells within the physical block, marks the bits corresponding to cells that have been marked as unusable or in a state of extreme wear as write-prohibited in the mask, and marks the remaining bits as write-enabled. This rule aims to avoid invalid write operations to known bad cells and reduce additional stress on vulnerable cells.

[0068] The system writes the calculated error correction codewords into the flash memory medium based on the generated write mask. The write operation is applied to the target memory cell in the form of a programming pulse sequence. Specifically, for bits marked as write-allowed in the write mask, the binary value in the corresponding error correction codeword is converted into a specific voltage level and programmed into the memory cell corresponding to the accurate bit flip address; for bits marked as write-prohibited in the mask, the voltage level of the corresponding cell remains unchanged during this write operation. After completing this overwrite write operation, the system immediately performs a read operation on the same accurate bit flip address to obtain its current voltage level measurement value, which is recorded as the readback voltage level state.

[0069] The system compares the readback level state bit by bit with the error correction codeword prepared for writing. The comparison rule requires that the two binary sequences be completely identical at every bit. This is a deterministic requirement based on the physical characteristics of the storage unit, as reliable storage requires that the written and read data must match precisely. If all corresponding bit values ​​are consistent, the system generates a successful repair flag for that address. If any bit values ​​are inconsistent, the system automatically initiates a retry process, i.e., it performs the overwrite operation again with the same error correction codeword and write mask, and then performs readback and comparison again. If all bits are consistent after the retry, a successful repair flag is generated again; if inconsistent bits still exist after the retry, a failed repair flag is generated.

[0070] The system updates its internally maintained storage block status mapping table based on the generated repair success and failure flags. For storage blocks whose addresses with all accurate bit flips have obtained the repair success flag, the system updates their flag in the status mapping table to an available state. Simultaneously, the range of this series of successfully repaired, physically address-contiguous storage blocks is recorded and output to obtain the repair success region. For blocks containing repair failure flags, their status is updated to bad block and they are added to the bad block management list.

[0071] The significance of this step lies in transforming the diagnostic conclusions and repair priority decisions generated in the preceding steps into actual physical operations on the storage medium, completing the closed loop from analysis to execution, and clearly identifying the successfully repaired areas. This provides a target range for subsequent periodic verification and accurate real-time information for wear leveling and capacity management of the storage system.

[0072] In step S19, periodic data integrity checks are performed on the successfully repaired areas to obtain an integrity verification report, including: Based on the successfully repaired region, a hash value is calculated for each storage physical page using the SHA-256 hash algorithm, which serves as the integrity sequence. According to a preset period, the hash value of the successfully repaired area is periodically calculated using the SHA-256 hash algorithm based on the storage physical page, as a verification sequence; Based on the integrity sequence and the test sequence, a position-by-position comparison is performed to generate a difference feature matrix; Based on the difference feature matrix, data integrity analysis is performed by calculating the Hamming distance to obtain an integrity verification report.

[0073] Specifically, the system first performs a complete data read of each physical page within the defined contiguous physical address range of the successfully repaired region. The raw binary data of each physical page is then fed into the SHA-256 hash algorithm calculation module. This module processes the data according to the standard SHA-256 algorithm procedure. The calculation process includes padding the input data to a length that is a multiple of 512 bits, dividing the padded message into contiguous 512-bit data blocks, performing multiple rounds of logical operations and compression functions on each data block, and finally iteratively generating a fixed-length 256-bit binary hash value. The 256-bit hash values ​​calculated for each physical page within the successfully repaired region are arranged in order of their physical addresses to form an integrity sequence. This sequence is generated and stored immediately after the repair operation is completed, serving as the benchmark for subsequent periodic verification.

[0074] The system periodically verifies successfully repaired regions based on a preset cycle length. This preset cycle is determined using historical data statistics. Specifically, it collects data on the time intervals at which data integrity failures first occurred in regions following repair operations over a relatively long historical period. The distribution of these failure intervals is analyzed, and the time interval corresponding to the minimum value of this distribution is selected. This time interval is then multiplied by a safety factor less than 1 (e.g., 0.5), and the product is used as the preset cycle value. This method ensures that the verification frequency is higher than the earliest historical interval at which problems occurred, thus enabling earlier detection of potential data degradation. At the end of each cycle, the system reads the data from each physical page within the successfully repaired region again in the exact same manner as generating the integrity sequence, and calculates its hash value using the same SHA-256 algorithm to obtain the verification sequence.

[0075] The system compares the verification sequence bit-by-bit with the previously stored integrity sequence. The comparison operation is performed on a per-page basis, XORing the two 256-bit hash values ​​corresponding to that page. If the two hash values ​​are identical, the XOR result is a 256-bit binary string of all zeros; if any differences exist, the bits with a value of 1 in the XOR result indicate the location of the difference. The XOR results of all physical pages are arranged in page address order to form a two-dimensional difference feature matrix. Rows in the matrix correspond to different physical pages, and columns correspond to each bit of the hash value.

[0076] The system performs data integrity analysis based on a difference feature matrix by calculating the Hamming distance. For each row of the difference feature matrix (i.e., a physical page), the total number of bits with a value of 1 in its 256-bit column is counted; this total number represents the Hamming distance between the two hash values ​​of that page. The Hamming distance ranges from 0 to 256; a larger value indicates a greater difference between the current data state of the page and the baseline state when repair is complete. The system then summarizes the Hamming distance calculation results for all physical pages.

[0077] A preset Hamming distance threshold is used to determine whether the integrity of a single physical storage page is acceptable. This threshold is determined based on statistics of background noise during fault-free system operation. Specifically, within multiple preset verification cycles during normal system operation, Hamming distance data (ideally 0, but possibly very small non-zero values ​​due to slight read interference or circuit noise) generated by all healthy physical storage pages not marked for repair during periodic self-verification is collected. The distribution of these non-zero Hamming distance values ​​is analyzed, and the distance value corresponding to the 99th percentile of this distribution is selected as the threshold. This threshold represents the extreme noise level that may occur in the absence of substantial errors. When generating an integrity verification report, the system lists the addresses of all physical storage pages with Hamming distances greater than this threshold and their specific distance values, calculates the average and maximum Hamming distances for the entire successfully repaired area, and finally organizes this information into a structured integrity verification report.

[0078] This step establishes a long-term monitoring mechanism based on cryptographic hash functions for the areas that have been repaired, providing a basis for decision-making on assessing the durability of the repair effect and determining whether a new round of maintenance processes needs to be initiated, thereby achieving a closed-loop and long-term guarantee of data integrity status.

[0079] In summary, this invention discloses a data storage integrity verification method for solid-state storage. Through hardware and software collaboration, a closed-loop integrity assurance system is constructed, from low-level physical signal monitoring to mid-level feature analysis and intelligent diagnosis, and then to high-level repair scheduling and verification.

[0080] Reference Figure 2The second embodiment of the present invention provides a data storage integrity verification system for solid-state storage, comprising: The data acquisition module is used to acquire read and write commands and program erase cycle counts of the solid-state storage controller, as well as the charge retention time of the solid-state storage. The bit flip frequency analysis module is used to perform voltage drift clustering and grouping according to the read and write instructions to obtain drift group data, and to count the flip frequency according to the drift group data to obtain the flip frequency distribution. The degradation risk analysis module is used to perform memory block degradation analysis based on the flip frequency distribution, the number of programming erase cycles, and the charge retention time to obtain core risk blocks, and to determine risk boundaries based on the core risk blocks to obtain high-risk areas. The error feature construction module is used to extract the risk voltage drift amount based on the flip frequency distribution, the drift grouping data and the high-risk area, obtain the risk voltage drift amount, and count the cumulative number of interferences caused by adjacent block operations as the read-write interference amount, and combine it with the risk voltage drift amount to obtain the error feature vector; The hard error identification module is used to identify the hard error type based on the error feature vector using a pre-built support vector machine model, and obtain the hard error type. The inspection trigger module is used to analyze the error evolution trend and make a reliability determination based on the hard error type and the flip frequency distribution, and obtain an inspection trigger signal; The priority evaluation module is used to perform precise bit flip location based on the inspection trigger signal, obtain the accurate bit flip address, and combine it with the pre-stored data importance library to perform repair priority evaluation and obtain a priority sequence. The repair module is used to perform data repair according to the priority sequence and the accurate bit flip address, obtain the readback level state, and update the state according to the readback level state to obtain the repair success area; The integrity verification module is used to periodically verify the data integrity of the successfully repaired area and obtain an integrity verification report. It should be noted that the data storage integrity verification system for solid-state storage provided in this embodiment of the invention executes all the process steps of the data storage integrity verification method for solid-state storage described in the above embodiment. The working principles and beneficial effects of both correspond one-to-one, and therefore will not be elaborated further.

[0081] It should be noted that the system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the system embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

Claims

1. A method for verifying the integrity of data storage in solid-state storage, characterized in that, include: Collect read / write commands and program / erase cycle counts from the solid-state storage controller, and the charge retention time of the solid-state storage. According to the read / write instructions, voltage drift is clustered and grouped to obtain drift group data, and the flip frequency is statistically analyzed based on the drift group data to obtain the flip frequency distribution; Based on the flip frequency distribution, the number of programmable erase cycles, and the charge retention time, a memory block degradation analysis is performed to obtain core risk blocks, and risk boundaries are determined based on the core risk blocks to obtain high-risk areas. Based on the flip frequency distribution, the drift grouping data, and the high-risk area, the risk voltage drift is extracted to obtain the risk voltage drift amount. The cumulative number of interferences caused by adjacent block operations is counted as the read / write interference amount, which is combined with the risk voltage drift amount to obtain the error feature vector. Based on the error feature vector, the hard error type is identified by a pre-built support vector machine model; Based on the hard error type and the flip frequency distribution, the error evolution trend is analyzed and the reliability is determined to obtain the inspection trigger signal; Based on the inspection trigger signal, perform precise bit flip location to obtain the accurate bit flip address, and combine it with the pre-stored data importance library to perform repair priority evaluation and obtain a priority sequence; Based on the priority sequence and the accurate bit flip address, data repair is performed to obtain the readback level state, and the state is updated based on the readback level state to obtain the repaired area. Periodic data integrity checks are performed on the successfully repaired areas to obtain integrity verification reports.

2. The data storage integrity verification method for solid-state memory according to claim 1, characterized in that, The step of performing voltage drift clustering and grouping according to the read / write instructions to obtain drift group data, and statistically analyzing the flip frequency based on the drift group data to obtain the flip frequency distribution, includes: The read / write instructions are parsed to obtain the physical address and theoretical voltage; Obtain the storage cell voltage corresponding to the physical address, and mark the physical address whose storage cell voltage is not equal to the theoretical voltage as a bit-flipped address; Based on the bit-flip address, the difference between the storage cell voltage and the theoretical voltage is calculated as the voltage drift amount, and drift timing data including the bit-flip address, voltage drift amount and timestamp is output. Based on the drift time series data, the data is segmented into time series segments through a preset sliding window, and the drift pattern is grouped using the DBSCAN clustering algorithm to obtain drift group data; Based on the drift grouping data, the frequency of bit flipping is statistically analyzed to obtain the flipping frequency distribution.

3. The data storage integrity verification method for solid-state memory according to claim 2, characterized in that, The process involves performing memory block degradation analysis based on the flip frequency distribution, the number of programmable erase cycles, and the charge retention time to identify core risk blocks. Furthermore, risk boundary determination is performed based on these core risk blocks to identify high-risk regions, including: The flip frequency distribution, the number of programmed erase cycles, and the charge retention time are used as the degradation feature dataset; Based on the degradation feature dataset, the degradation score is obtained by scoring the storage physical blocks using the random forest algorithm. When the degradation score does not exceed the preset degradation threshold, the storage physical block is marked as a normal storage physical block. When the degradation score exceeds the preset degradation threshold, the storage physical block is marked as an abnormal storage physical block, and physically adjacent abnormal storage physical blocks are aggregated and output as core risk blocks. Based on the core risk block, the gradient change rate of the degradation score of the storage physical page at the edge of each abnormal storage physical block is calculated to obtain the boundary gradient; When the boundary gradient does not exceed a preset gradient mutation threshold, the corresponding storage physical page is marked as a normal boundary. When the boundary gradient exceeds the preset gradient mutation threshold, the corresponding storage physical page is marked as a risk boundary. The risk boundaries are combined into a continuous address range and output as a high-risk region.

4. The data storage integrity verification method for solid-state memory according to claim 3, characterized in that, The risk voltage drift is extracted based on the flip frequency distribution, the drift grouping data, and the high-risk area. The cumulative number of interferences caused by adjacent block operations is counted as the read / write interference. This is combined with the risk voltage drift to obtain an error feature vector, including: Based on the flip frequency distribution and the high-risk area, extract the flip frequency distribution corresponding to the high-risk area to obtain the risk flip frequency; When the risk reversal frequency does not exceed a preset reversal frequency threshold, the high-risk area is marked as a healthy area; When the risk flip frequency exceeds a preset flip frequency threshold, the voltage drift amount corresponding to the high-risk area is extracted based on the drift grouping data to obtain the risk voltage drift amount. The cumulative number of times that adjacent storage physical blocks perform read and write operations within a preset time period, causing voltage drift in the target storage physical block, is counted as the read / write interference amount. The risk voltage drift and the read / write interference are combined into an error feature vector using the same physical address.

5. The data storage integrity verification method for solid-state memory according to claim 1, characterized in that, The support vector machine model construction process includes: Obtain historical error feature vectors and corresponding historical hard error types; An initial vector machine model is constructed using the support vector machine algorithm, and the classification boundary is initialized using the RBF kernel function, with a penalty coefficient set. Based on the historical error feature vector, the curvature adjustment parameter of the classification boundary is calculated using the kernel density estimation method, and the curvature of the classification boundary is dynamically adjusted through the RBF kernel function; Based on the historical hard error types, the penalty coefficient is optimized using a grid search method to determine the error tolerance threshold; Cross-validate the adjusted classification boundaries and calculate the set of classification accuracy and error distribution after boundary adjustment; When the error distribution set exceeds the error tolerance threshold, the gradient descent method is used to re-optimize the curvature parameter of the classification boundary, and the classification boundary is iteratively updated until the error distribution set does not exceed the error tolerance threshold. By using RBF kernel function mapping, the optimized classification boundary is matched with the hard error type in a multi-dimensional space to determine the hyperplane equation corresponding to each error feature vector; Based on the time series characteristics of the historical hard error types, the sliding window method is used to extract dynamic trend parameters. The dynamic trend parameters are then correlated with the hyperplane equation to construct the final support vector machine model.

6. The data storage integrity verification method for solid-state memory according to claim 3, characterized in that, The step of analyzing the error evolution trend and making a reliability determination based on the hard error type and the flip-flop frequency distribution to obtain the check trigger signal includes: Based on the hard error type, the physical address is indexed to obtain the precise error address; Based on the precise address of the error and the distribution of the bit flip frequency, the error growth rate is obtained by analyzing the evolution trend of the bit flip frequency through the calculation of the first derivative of the time series. When the error growth rate does not exceed a preset error growth rate threshold, the exact address of the error is determined to be in a reliable qualified state. When the error growth rate exceeds a preset error growth rate threshold, the precise address of the error is determined to be in a reliability-to-check state, and a check trigger signal is generated.

7. The data storage integrity verification method for solid-state memory according to claim 6, characterized in that, The step involves performing precise bit-flip location based on the inspection trigger signal to obtain the accurate bit-flip address. This address is then combined with a pre-stored data importance database to perform a repair priority assessment, resulting in a priority sequence, including: Based on the inspection trigger signal, the storage physical block corresponding to the exact address of the error is marked as the area to be inspected; Based on the area to be inspected, the integrity of the same physical storage block is verified by multiple reads through adjusting the read reference voltage, and the accurate bit flip position is determined by the ECC decoder to obtain the accurate bit flip address. Based on the accurate bit-flipped address, the recurrent flipped posterior probability is calculated using a pre-constructed Bayesian network model to obtain the recurrence probability; Based on the recurrence probability and combined with the pre-stored data importance database, the priority sequence for bit flip repair is determined by the weighted product method, thus obtaining the priority sequence.

8. The data storage integrity verification method for solid-state memory according to claim 7, characterized in that, The process involves performing data repair based on the priority sequence and the accurate bit flip address to obtain a readback level state, and updating the state based on the readback level state to obtain a successfully repaired region, including: Based on the priority sequence, the original binary data of the storage physical page to be repaired is extracted to obtain the data stream to be repaired; Based on the data stream to be repaired, error correction codewords are generated by a hardware LDPC encoder, and a programming prohibition bitmap is generated by a NAND flash interface controller as a write mask. According to the write mask, the error correction codeword is overwritten to the accurate bit flip address, and the level state of the accurate bit flip address after overwriting is read back to obtain the readback level state; The readback level state is compared with the error correction codeword. If all bits are consistent, a repair success flag is generated. If there are inconsistencies, the codeword is overwritten again and a write consistency check is performed. If all bits are consistent, a repair success flag is generated. If there are inconsistencies, a repair failure flag is generated. Based on the successful repair flag and the failed repair flag, update the availability status and physical address of the storage physical block to obtain the successfully repaired area.

9. The data storage integrity verification method for solid-state memory according to claim 3, characterized in that, The periodic data integrity verification of the successfully repaired area, resulting in an integrity verification report, includes: Based on the successfully repaired region, a hash value is calculated for each storage physical page using the SHA-256 hash algorithm, which serves as the integrity sequence. According to a preset period, the hash value of the successfully repaired area is periodically calculated using the SHA-256 hash algorithm based on the storage physical page, as a verification sequence; Based on the integrity sequence and the test sequence, a position-by-position comparison is performed to generate a difference feature matrix; Based on the difference feature matrix, data integrity analysis is performed by calculating the Hamming distance to obtain an integrity verification report.

10. A data storage integrity verification system for solid-state storage, characterized in that, include: The data acquisition module is used to acquire read and write commands and program erase cycle counts of the solid-state storage controller, as well as the charge retention time of the solid-state storage. The bit flip frequency analysis module is used to perform voltage drift clustering and grouping according to the read and write instructions to obtain drift group data, and to count the flip frequency according to the drift group data to obtain the flip frequency distribution. The degradation risk analysis module is used to perform memory block degradation analysis based on the flip frequency distribution, the number of programming erase cycles, and the charge retention time to obtain core risk blocks, and to determine risk boundaries based on the core risk blocks to obtain high-risk areas. The error feature construction module is used to extract the risk voltage drift amount based on the flip frequency distribution, the drift grouping data and the high-risk area, obtain the risk voltage drift amount, and count the cumulative number of interferences caused by adjacent block operations as the read-write interference amount, and combine it with the risk voltage drift amount to obtain the error feature vector; The hard error identification module is used to identify the hard error type based on the error feature vector using a pre-built support vector machine model, and obtain the hard error type. The inspection trigger module is used to analyze the error evolution trend and make a reliability determination based on the hard error type and the flip frequency distribution, and obtain an inspection trigger signal; The priority evaluation module is used to perform precise bit flip location based on the inspection trigger signal, obtain the accurate bit flip address, and combine it with the pre-stored data importance library to perform repair priority evaluation and obtain a priority sequence. The repair module is used to perform data repair according to the priority sequence and the accurate bit flip address, obtain the readback level state, and update the state according to the readback level state to obtain the repair success area; The integrity verification module is used to periodically verify the data integrity of the successfully repaired area and obtain an integrity verification report.

Images