Enterprise-level solid state disk data distributed storage management method

Abstract

This application relates to the field of data processing technology and discloses an enterprise-level distributed storage management method for solid-state drives (SSDs). The method includes: collecting physical characteristic data such as the P / E cycle of the SSD through an intelligent controller; establishing a lifetime prediction model using a Bayesian network algorithm; obtaining a state transition probability matrix through Markov chain-based health state classification; implementing wear-aware write load scheduling based on this matrix; and finally achieving SSD wear-leveled storage management through distributed data redistribution. This application solves the problem of enterprise-level SSD arrays lacking intelligent wear-aware scheduling and distributed load balancing management based on lifetime prediction.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an enterprise-level distributed storage management method for solid-state drives. Background Technology

[0002] In current technologies, enterprise-level solid-state drive (SSD) data storage management primarily relies on traditional wear leveling algorithms and basic SMART monitoring technology. Typical SSD controllers manage write operations through simple polling or static load balancing, while utilizing the NVMe SMART command set to obtain basic physical parameters such as P / E cycle counts, temperature, and bad block information. These traditional methods can provide basic wear leveling functionality in single-disk or small-scale array environments and determine SSD health status using fixed thresholds.

[0003] However, existing technologies have significant shortcomings: First, traditional SMART monitoring can only provide instantaneous status data and lacks the ability to accurately predict the lifespan of SSDs, making it impossible to perform forward scheduling based on future health trends; second, most existing wear leveling algorithms use static weight allocation and fail to dynamically adjust the write strategy according to the actual health status of each SSD unit; third, traditional methods lack a correlation mechanism with the health status of SSDs when processing hot, warm, and cold data, which may result in high-load data being allocated to SSD units with poor health status.

[0004] Based on the above analysis, it can be inferred that the core problem with existing technologies lies in the lack of a complete intelligent SSD lifespan management system encompassing monitoring, prediction, scheduling, and management. This deficiency leads to a series of progressive technical challenges: how to establish an accurate lifespan prediction model based on multi-dimensional physical characteristics; how to transform the prediction results into an operational health status classification system; how to achieve wear-aware intelligent scheduling based on health status; and how to achieve globally optimized data redistribution management in a distributed environment. Solving these problems requires the organic integration of probability statistics, machine learning, and distributed scheduling algorithms to form a multi-layered, adaptive SSD management framework. Summary of the Invention

[0005] This application provides an enterprise-level distributed storage management method for solid-state drive data, which addresses the lack of intelligent wear-aware scheduling and distributed load balancing management based on lifetime prediction in enterprise-level SSD arrays.

[0006] This application provides an enterprise-level solid-state drive (SSD) data distributed storage management method, the method comprising: The P / E cycle status acquisition and processing of enterprise-level solid-state drive arrays is performed by an intelligent wear-sensing controller to obtain SSD physical characteristic monitoring data. Based on the SSD physical characteristic monitoring data, a lifetime prediction model is obtained by performing lifetime prediction modeling through a Bayesian network algorithm. The SSD lifetime prediction distribution model is subjected to Markov chain health classification to obtain the SSD state transition probability matrix. Based on the SSD state transition probability matrix, wear-aware scheduling processing is performed on the distributed storage write load to obtain an enterprise-level SSD write allocation strategy. Based on the enterprise-level SSD write allocation strategy, the stored data is distributed and redistributed to obtain a solid-state drive wear leveling storage solution.

[0007] The technical solution provided in this application utilizes SSD physical characteristic monitoring data obtained through P / E cycle state acquisition and processing of enterprise-level solid-state drive arrays by an intelligent wear-sensing controller. This allows for real-time monitoring of the wear status and performance degradation trend of each SSD unit. Compared to the traditional single SMART monitoring method, this solution significantly improves the accuracy and comprehensiveness of state perception through comprehensive monitoring of multi-dimensional physical characteristics. Based on this precise monitoring data, Bayesian network algorithms for lifetime prediction modeling effectively integrate multiple uncertainties. The established SSD lifetime prediction distribution model can not only predict the remaining usage time but also quantify the reliability of the prediction, providing a probabilistic scientific basis for decision-making. By processing the lifetime prediction distribution model through Markov chain health classification to obtain the SSD state transition probability matrix, the continuous lifetime prediction results are successfully transformed into discrete health state classifications. This allows complex probabilistic prediction results to be directly applied to actual scheduling decisions, while the state transition characteristics of the Markov chain can dynamically track the evolution of the SSD health status. Wear-aware scheduling of distributed storage write loads based on a state transition probability matrix enables a shift from a passive response to a proactive prevention management model. Enterprise-grade SSD write allocation strategies dynamically adjust write weights based on the health status and expected lifespan of each SSD unit, preventing SSDs in poor health from bearing excessive loads and thus extending the overall array lifespan. The resulting SSD wear-leveling storage solution, achieved through distributed management and redistribution, not only achieves static load balancing but, more importantly, establishes a dynamic and predictive wear-leveling mechanism, maximizing the overall lifespan of the SSD array while ensuring system performance.

[0008] In enterprise-level storage applications, the probabilistic reasoning characteristics of Bayesian network algorithms enable systems to make optimal decisions under uncertain environments. Their ability to handle multivariate dependencies is particularly suitable for complex scenarios like SSD lifetime prediction, which involves the coupling of multiple factors such as temperature, P / E cycles, and failure rates. The algorithm's incremental learning characteristics also continuously optimize prediction accuracy with the accumulation of historical data. The memoryless nature of Markov chain algorithms simplifies the computational complexity of state transitions, making real-time health status classification possible. Their state transition probability matrix provides a mathematical basis for wear-aware scheduling, and their computational efficiency advantage is even more pronounced when dealing with large-scale SSD arrays. Distributed management redistribution algorithms achieve globally optimal wear leveling through incremental migration and network topology optimization, ensuring data consistency and system availability. Their timing control mechanism avoids the impact of data migration on normal business operations, demonstrating the algorithm design's dual consideration of performance and reliability in practical applications. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram of one embodiment of the enterprise-level solid-state drive data distributed storage management method in this application. Detailed Implementation

[0011] This application provides an enterprise-level distributed storage management method for solid-state drive data. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0012] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 One embodiment of the enterprise-level solid-state drive data distributed storage management method in this application includes: Step S101: The P / E cycle status acquisition and processing of the enterprise-level solid-state drive array is performed through the intelligent wear sensing controller to obtain SSD physical characteristic monitoring data; Step S102: Based on the SSD physical characteristic monitoring data, perform lifetime prediction modeling using Bayesian network algorithm to obtain the SSD lifetime prediction distribution model. Step S103: Perform Markov chain health classification processing on the SSD lifetime prediction distribution model to obtain the SSD state transition probability matrix. Step S104: Perform wear-aware scheduling on the distributed storage write load based on the SSD state transition probability matrix to obtain an enterprise-level SSD write allocation strategy. Step S105: Perform distributed management and redistribution of storage data according to the enterprise-level SSD write allocation strategy to obtain a solid-state drive wear leveling storage solution.

[0013] It is understood that the executing entity of this application can be an enterprise-level solid-state drive data distributed storage management system, or it can be a terminal or a server; the specific implementation is not limited here. This application's embodiment uses a server as an example for illustration.

[0014] Specifically, a smart wear-sensing controller is used to collect the P / E cycle status of the enterprise-grade solid-state drive array. This controller is specifically designed to monitor changes in the physical characteristics of NAND flash memory. The P / E cycle, or program erase cycle, represents the number of write and erase cycles each storage cell in the SSD can withstand, a key indicator of SSD lifespan. The data acquisition process first uses the NVMe SMART command set to read the remaining P / E cycle count for each SSD cell. SMART stands for Self-Monitoring Analysis Reporting Technology, which can obtain real-time health status parameters of the SSD. Simultaneously, temperature sensors collect the operating temperature of each SSD cell, as temperature changes directly affect the reliability of NAND flash memory and the P / E cycle consumption rate. Next, the temperature data is correlated with the bad block remapping frequency. Bad block remapping refers to the process of automatically transferring data to a spare block when a NAND flash memory block fails; an increased remapping frequency indicates that the SSD is experiencing more physical damage. The real-time calculation of the write amplification factor is obtained by comparing the actual amount of data written with the amount of data requested by the user. This factor reflects the efficiency of the SSD's internal garbage collection and wear leveling algorithms; the closer the value is to 1, the higher the write efficiency. The collected data is timestamped and structured to form SSD physical characteristic monitoring data.

[0015] SSD lifetime prediction modeling is based on a Bayesian network algorithm for in-depth analysis of collected physical characteristic data. A Bayesian network is a probabilistic graphical model capable of handling complex dependencies and uncertainties between variables. The modeling process first defines the P / E cycle counter data as node variables in the Bayesian network, with each node representing a different P / E cycle usage range. The connections between nodes represent the causal relationship between P / E cycle consumption and other factors. Temperature-related node modeling establishes a mathematical correlation between SSD operating temperature and lifetime degradation; accelerated electron migration in NAND flash memory at high temperatures leads to a shorter P / E cycle life. NAND flash memory block failure statistics are processed using conditional probability calculations. Conditional probability describes the likelihood of an event occurring under certain conditions, such as the probability of flash memory block failure under specific temperature and P / E cycle usage conditions. The maximum likelihood estimation algorithm training process adjusts network parameters using historical SSD operating data to maximize the likelihood between the model's predictions and actual observations. The variational inference algorithm approximates complex posterior probability distributions, transforming precise inference into an optimization problem, ultimately generating a lifetime prediction probability distribution for each SSD.

[0016] Markov chain health classification transforms continuous lifespan prediction data into discrete health states. A Markov chain is a stochastic process whose future state depends only on the current state and is independent of historical states. A five-level classification system is used for health states: excellent (0-20% P / E cycle utilization), good (20-40%), normal (40-60%), warning (60-80%), and critical (80-100%). State transition frequency statistics calculate the number of transitions between states by analyzing historical SSD state evolution data, and then normalize the results using maximum likelihood estimation to obtain the transition probabilities. Each element of the transition probability matrix represents the probability that the SSD will transition from one healthy state to another within a unit of time. Critical states are marked as absorbing states, indicating that once the SSD enters this state, it will not recover to a better state.

[0017] Wear-aware scheduling employs a Hidden Markov Model (HMM) to infer the SSD's state in real time. The HMM assumes the true health state is hidden and can only be inferred from observable performance metrics. The state inference process uses the Viterbi algorithm to calculate the most probable health state sequence. This algorithm finds the hidden state sequence that maximizes the probability of the observed sequence through dynamic programming. Data stratification and classification divides stored data into hot, warm, and cold data based on access frequency. Hot data, accessed frequently, requires high-performance SSDs, while cold data, accessed less frequently, is suitable for allocation to SSDs in a warning state to fully utilize their remaining lifespan. Write weights are dynamically calculated to determine the write allocation ratio based on the health state and data type of each SSD, with SSDs in better health receiving higher write weights. Dynamic programming optimization minimizes overall wear unevenness while meeting performance requirements, solving for the optimal write strategy by constructing a state transition equation.

[0018] Distributed management and redistribution processes intelligently reallocate data across the SSD array. A data-SSD health status mapping table establishes a relationship between stored data and target SSDs, ensuring that critical data is prioritized for allocation to SSDs in good health. Progressive migration path planning avoids the impact of large-scale data movement on storage performance; the migration process is conducted in stages, and the amount of data migrated in a single step is strictly controlled. Network topology shortest path calculation selects the optimal data transmission path by analyzing the physical connections between SSDs, reducing network bandwidth consumption and transmission latency. System load monitoring identifies low-load periods for data migration operations, avoiding resource contention with foreground services. P / E cycle distribution uniformity assessment quantifies wear leveling effectiveness by calculating the standard deviation of P / E cycle utilization among each SSD; a smaller standard deviation indicates a more uniform wear distribution.

[0019] In one specific embodiment, the process of performing step S101 may specifically include the following steps: The remaining P / E cycle count is read and processed for each SSD unit in the enterprise-grade solid-state drive array using the NVMe SMART command set to obtain the P / E cycle counter data; Based on the temperature sensor, the operating temperature of the SSD unit corresponding to the P / E cycle counter data is collected and processed to obtain the SSD temperature status data. By correlating and monitoring SSD temperature status data with bad block remapping frequency, NAND flash memory block failure rate statistics are obtained. Based on the NAND flash memory block failure rate statistics, the write amplification factor is calculated in real time to obtain the SSD write performance degradation index. Based on the SSD write performance degradation index, timestamp marking and data structured storage processing are performed to obtain SSD physical characteristic monitoring data.

[0020] Specifically, the NVMe SMART command set for reading remaining P / E cycles uses a smart wear-aware controller to send specific SMART commands to enterprise-grade SSDs to acquire data. NVMe is a next-generation non-volatile memory host controller interface standard, and SMART stands for Self-Monitoring Analysis and Reporting Technology, specifically designed to obtain health status information of storage devices. The read process first establishes a communication connection between the controller and each storage unit in the SSD array. Then, it sends a 0xB0 command code to query P / E cycle-related parameters. The data returned by this command includes key information such as total P / E cycle capacity, used P / E cycles, and remaining P / E cycles. Data reading uses a polling method to sequentially access each SSD unit in the array, with a read frequency set to once every ten minutes to ensure data real-time performance without significantly impacting SSD performance. The returned raw data undergoes protocol parsing and format conversion to form standardized P / E cycle counter data. This data includes fields such as SSD identifier, timestamp, total cycle capacity, used cycles, and remaining cycles. The data is stored in a JSON structure for easy subsequent processing and analysis. P / E cycle counter data reflects the current wear status and remaining lifespan of each SSD cell, laying the data foundation for subsequent lifespan prediction and load distribution decisions.

[0021] The temperature sensor's operating temperature acquisition and processing establishes a correspondence between temperature monitoring and storage units based on the SSD identifier in the P / E cycle counter data. Each SSD unit's built-in temperature sensor monitors the NAND flash memory chip's operating temperature in real time. The temperature acquisition process obtains the Celsius temperature value by reading the SSD's temperature register. The temperature sensor's accuracy is typically ±1 degree Celsius, and the acquisition frequency is synchronized with the P / E cycle data to ensure data consistency. The impact of SSD workload on temperature must be considered during temperature data acquisition. High-intensity read / write operations can cause chip temperature to rise, which in turn accelerates electron migration in the NAND flash memory, affecting the P / E cycle consumption rate. The acquired temperature data is associated and bound with the corresponding P / E cycle counter data to form SSD temperature status data containing complete information such as SSD identifier, P / E cycle status, operating temperature, and acquisition time. The temperature status data is stored in a time-series format, facilitating the analysis of temperature change trends and the long-term impact of temperature on SSD performance. A temperature anomaly detection mechanism is also established to trigger an alert immediately when the temperature of an SSD exceeds a safe threshold.

[0022] The bad block remapping frequency correlation monitoring process performs deep correlation analysis between SSD temperature status data and NAND flash memory bad block management information. Bad block remapping is the process by which SSD firmware automatically replaces faulty storage blocks with spare blocks, and changes in the remapping frequency directly reflect the physical health of the NAND flash memory. Correlation monitoring obtains the frequency and time distribution of remapping events by reading the SSD's bad block management table, and then matches and analyzes it with temperature status data by timestamp. Increased temperature usually accelerates the aging process of NAND flash memory, leading to an increase in bad blocks; therefore, there is a positive correlation between temperature data and bad block remapping frequency. The correlation analysis uses a sliding window algorithm to calculate the bad block remapping frequency within a specific time window, with the window size set to 24 hours to capture changing patterns within the daily working cycle. Frequency calculation is done by dividing the total number of remapping events within the statistical time window by the time length to obtain the remapping frequency per unit time. This frequency data is then correlated with the average temperature and P / E cycle consumption for the corresponding time period. NAND flash memory block failure rate statistics are calculated by analyzing historical bad block remapping data. The failure rate reflects the probability of NAND flash memory blocks failing under specific usage conditions, and this data has become an important indicator for evaluating SSD reliability and predicting lifespan.

[0023] The real-time calculation of write amplification factor is based on NAND flash memory block failure rate statistics to analyze the efficiency of the SSD's internal data management mechanism. Write amplification refers to the phenomenon where the amount of data actually written by the SSD exceeds the amount of data requested by the user, mainly caused by internal operations such as garbage collection, wear leveling, and bad block remapping. The calculation process monitors the amount of data written by the host and the amount of data actually written by the NAND flash memory; the ratio of the two is the write amplification factor. Failure rate statistics affect the calculation of the write amplification factor because an increase in bad blocks leads to more frequent remapping operations and garbage collection processes, thus increasing additional write operations. Real-time calculation uses a moving average algorithm to smooth short-term fluctuations, and the calculation window is set to one hour to reflect the actual working state of the SSD. The calculation of the write amplification factor also needs to consider the impact of different types of data on SSD performance. Sequential writes and random writes produce different write amplification effects; random small block writes usually produce a higher write amplification factor. The SSD write performance degradation index is calculated by comprehensively analyzing factors such as write amplification factor, temperature changes, and bad block growth. This index quantifies the degree of degradation of SSD write performance relative to its initial state; a higher value indicates more severe performance degradation.

[0024] Timestamp marking and structured data storage integrate SSD write performance degradation metrics with all aforementioned monitoring data to form a complete SSD physical characteristic monitoring dataset. Timestamp marking uses Coordinated Universal Time (UTC) to ensure time consistency across different SSDs, achieving millisecond-level precision for high-precision analysis. Structured data storage employs a hierarchical data model: the top layer is the SSD array identifier, the next layer is the individual SSD identifier, and the bottom layer contains specific monitoring parameters and time-series data. The storage structure includes a metadata layer recording basic SSD information such as model, capacity, and manufacturer; a parameter layer recording dynamic monitoring information such as P / E cycle data, temperature data, bad block data, and write amplification data; and a correlation layer recording the dependencies and calculation logic between different parameters. Data compression and indexing mechanisms ensure efficient storage and query performance for large amounts of monitoring data. A time-sharding strategy archives historical data monthly to optimize storage space usage. Data integrity verification ensures the accuracy of stored data through checksum algorithms, and regular data backup and recovery tests verify data reliability.

[0025] In one specific embodiment, the process of performing step S102 may specifically include the following steps: The P / E cyclic counter data in the SSD physical characteristic monitoring data is processed by Bayesian network node variable definition to obtain the P / E cyclic node probability distribution. Based on the probability distribution of P / E cycle nodes, temperature-affected node modeling is performed on the SSD temperature status data to obtain the temperature-lifetime correlation probability matrix. Based on the temperature-lifetime correlation probability matrix, conditional probability calculations are performed on the failure rate statistics of NAND flash memory blocks to obtain the conditional probability distribution of failure rate. The failure rate conditional probability distribution and the SSD write performance degradation index are trained using the maximum likelihood estimation algorithm to obtain the Bayesian network parameter set. Based on the Bayesian network parameter set, a variational inference algorithm is used for prediction to obtain the SSD lifetime prediction distribution model.

[0026] Specifically, the P / E cycle counter data Bayesian network node variable definition process transforms continuous P / E cycle utilization data into a discrete probability distribution. A Bayesian network is a directed acyclic graph model where nodes represent random variables and edges represent conditional dependencies between variables. The node variable definition process first discretizes the P / E cycle utilization rate into different intervals, establishing five main nodes corresponding to utilization rate intervals of 0-20%, 20-40%, 40-60%, 60-80%, and 80-100%. The probability distribution of each node is calculated by statistically analyzing the distribution frequency of historical P / E cycle data in each interval. Frequency statistics use a sliding window method to analyze SSD operation data from the past six months, with a window size set to 1000 data points to ensure the stability of the statistical results. The probability calculation process divides the number of data points in each interval by the total number of data points to obtain the prior probability for that interval. The sum of the probabilities of all intervals equals 1 to ensure the integrity of the probability distribution. The probability distribution of P / E cyclic nodes reflects the distribution pattern of SSDs at different wear stages. This distribution forms the basis for calculating the conditional probability of other nodes in the Bayesian network and is also one of the core input parameters of the lifetime prediction model.

[0027] The temperature-affected node modeling process analyzes the impact mechanism of temperature changes on SSD lifespan based on the probability distribution of P / E cycle nodes. The temperature-affected nodes describe the causal relationship between operating temperature and P / E cycle consumption rate. The modeling process categorizes SSD temperature state data according to temperature range, establishing three temperature nodes: low temperature, normal temperature, and high temperature, corresponding to operating environments below 30℃, 30-40℃, and above 40℃, respectively. The correlation analysis between temperature and P / E cycle consumption establishes a conditional probability table by calculating the average consumption rate of P / E cycles under different temperature conditions. High-temperature environments accelerate electron migration in NAND flash memory, leading to a shorter P / E cycle lifespan, while low-temperature environments maintain relatively stable flash memory performance but reduce write speed. The conditional probability calculation employs Bayes' theorem, calculating the probability distribution of each temperature node state given a P / E cycle node state. This calculation process requires traversing all possible combinations of P / E cycles and temperature states. The temperature-lifetime correlation probability matrix is ​​constructed by combining all states of the P / E cycle node and the temperature node as matrix elements. The value of each element represents the conditional probability of SSD lifetime under a specific P / E cycle state and temperature condition. The rows of the matrix represent the P / E cycle state and the columns represent the temperature state. The matrix elements are determined by combining historical data statistics and expert knowledge.

[0028] The conditional probability calculation of NAND flash memory block failure rate statistics, based on the temperature-lifetime correlation probability matrix, further analyzes the impact of physical failures of flash memory blocks on SSD lifespan. Conditional probability calculation is a core step in Bayesian network inference, used to quantify the strength of dependencies between variables. The calculation process first establishes a failure rate node, which describes the probability of a NAND flash memory block failing per unit time. The failure rate node forms a complex dependency with the P / E cycle node and the temperature node. The conditional probability calculation uses a chain rule to decompose the complex joint probability into a product of conditional probabilities, calculating the probability distribution of flash memory block failure rate under given P / E cycle states and temperature conditions. The calculation of the failure rate conditional probability distribution needs to consider multiple causes of flash memory block failure, including physical mechanisms such as electromigration, tunnel oxide degradation, and thermal stress. Each mechanism has different weights under different P / E cycle stages and temperature conditions. The construction of the conditional probability table is completed through statistical analysis of a large amount of historical failure data, with data sources including laboratory accelerated aging tests, field operation failure records, and reliability data provided by manufacturers. The failure rate conditional probability distribution represents the probability of physical failure of an SSD under specific operating conditions, and this distribution directly affects the accuracy and reliability of lifespan prediction.

[0029] The maximum likelihood estimation (MLE) algorithm training process combines the failure rate conditional probability distribution with the SSD write performance degradation index for Bayesian network parameter optimization. MLE is a parameter estimation method that determines model parameters by finding the parameter values ​​that maximize the probability of observed data occurrences. During training, the write performance degradation index is added as a new observed variable to the Bayesian network, establishing causal connections with P / E cycle nodes, temperature nodes, and failure rate nodes. Parameter estimation employs the expectation-maximization (EM) algorithm to iteratively optimize the parameter values ​​of all conditional probability tables in the network. Each iteration includes two phases: an expectation step and a maximization step. The expectation step calculates the posterior probability distribution of latent variables under the current parameters, while the maximization step updates the parameters so that the log-likelihood function of the data reaches a local maximum. The training data includes a large number of historical SSD operation records, each containing complete information such as P / E cycle status, operating temperature, failure rate, and write performance degradation. A convergence criterion needs to be set during algorithm training: training stops when the parameter change between two consecutive iterations is less than a preset threshold or the increment of the log-likelihood function is less than a threshold. The parameter set of a Bayesian network contains complete information such as the conditional probability table of all nodes, network structure parameters, and hyperparameters. These parameters completely determine the probabilistic reasoning ability of the Bayesian network.

[0030] The variational inference algorithm predicts the probabilistic lifetime of SSDs based on a Bayesian network parameter set. Variational inference is an approximate inference method that approximates a complex posterior probability distribution by optimizing the variational distribution. The prediction process inputs the current observed state of the SSD as evidence into the Bayesian network, including parameters such as the current P / E cycle rate, operating temperature, and failure rate. Variational inference uses the mean-field approximation assumption that all latent variables are independent, decomposing the complex joint posterior distribution into a product of independent variational distributions. The algorithm iteratively optimizes the variational parameters to minimize the KL divergence between the variational distribution and the true posterior distribution; the KL divergence is an indicator of the difference between the two probability distributions. Each iteration updates the variational parameters of each latent variable, and the update formula is calculated based on the gradient of the variational lower bound. The convergence of variational inference is judged by monitoring the change in the variational lower bound; the algorithm is considered convergent when the increment of the lower bound is less than a threshold for multiple consecutive iterations. The output of the SSD lifetime prediction distribution model is a probability density function, describing the probability distribution of the remaining usage time of the SSD. This distribution includes statistical characteristics such as mean, variance, and confidence interval.

[0031] In one specific embodiment, the process of performing variational inference algorithm prediction based on the Bayesian network parameter set can specifically include the following steps: The Bayesian network parameter set is input into the variational inference algorithm to approximate the latent variable posterior distribution, thus obtaining the initial parameters of the variational distribution. Based on the initial parameters of the variational distribution, the mean field approximate iteration process is performed on the P / E cycle node and the temperature-affected node to obtain the converged value of the node variational parameter; The lower bound of evidence is optimized based on the convergence value of the nodal variational parameters to obtain the optimal solution of the model parameters. The optimal solution of the model parameters is verified by Bayesian inference with historical failure data of SSDs to obtain the lifetime prediction probability distribution. The remaining usage time distribution of the current SSD is calculated based on the lifetime prediction probability distribution to obtain the SSD lifetime prediction distribution model.

[0032] Specifically, the approximation of the posterior distribution of latent variables in the Bayesian network parameter set transforms the complex Bayesian inference problem into an optimization problem through variational inference algorithms. Variational inference is an approximate inference method that approximates a complex posterior probability distribution by finding a simpler distribution. Latent variables refer to the potential states in the Bayesian network that cannot be directly observed but affect the observed variables. In SSD lifetime prediction, these include the internal physical state of NAND flash memory, future temperature trends, and potential failure modes. The posterior distribution approximation first assumes that the latent variables are independent and uses a mean-field variational method to decompose the joint posterior distribution into a product of independent distributions. The initial parameters of the variational distribution are determined through random initialization or heuristic methods based on prior knowledge. The initial parameters include the mean and variance parameters of the variational distribution of each latent variable. The parameter initialization process needs to ensure the numerical stability and convergence of the variational distribution to avoid numerical computation problems such as gradient vanishing or exploding. The variational distribution form is usually chosen to be a family of distributions conjugate to the prior distribution, such as a Gaussian or gamma distribution, to simplify the computational complexity of variational parameter updates. The quality of the initial parameters directly affects the convergence speed and final accuracy of variational inference. Good initialization can significantly reduce the number of iterations and improve prediction accuracy.

[0033] The mean-field approximation iterative processing for P / E cycle nodes and temperature-affected nodes employs a coordinate ascent algorithm to optimize the variational parameters of each node sequentially. The mean-field approximation assumes that the variational distributions of all latent variables are independent; while this simplifies computation, it may overlook the correlations between variables. The iterative process alternately fixes the variational parameters of other nodes, optimizing the parameters of the current node individually to maximize the variational lower bound. Updating the variational parameters of P / E cycle nodes requires considering the conditional dependencies between the node and temperature and failure rate nodes; the update formula is calculated based on the partial derivatives of the variational lower bound with respect to the parameters. The parameter update process for temperature-affected nodes is similar, but it requires additional consideration of the temporal correlation and seasonality of temperature changes. The iterative algorithm uses a damped update strategy to prevent parameter oscillations, and the update step size is dynamically adjusted through line search or adaptive methods. Convergence criteria include multiple indicators such as the parameter change between adjacent iterations, the increment of the variational lower bound, and the norm of the gradient. The convergent values ​​of the node variational parameters represent the optimal approximate distribution of each latent variable under the current observational evidence. These parameters completely determine the shape and location of the variational distribution, forming the basis for subsequent inference and prediction.

[0034] The lower bound of evidence optimization further optimizes the global parameters of the Bayesian network based on the convergence values ​​of the node variational parameters. The lower bound of evidence is a core concept in variational inference; it is a lower bound estimate of the log-marginal likelihood. The lower bound of evidence consists of two parts: the expected log-likelihood term and the KL divergence term. The former measures the goodness of fit of the variational distribution to the observed data, while the latter measures the difference between the variational distribution and the prior distribution. The optimization calculation uses the gradient ascent method to maximize the lower bound of evidence. Gradient calculation requires calculating the partial derivatives of all network parameters, including conditional probability table parameters and network structure parameters. The parameter update process uses the Adam optimizer or other adaptive learning rate methods, with learning rate scheduling strategies including exponential decay and cosine annealing. During optimization, the model's performance on the validation set needs to be evaluated periodically to prevent overfitting. The optimal solution for the model parameters corresponds to the local or global maximum point of the lower bound of evidence; this solution contains the optimal values ​​of all learnable parameters of the Bayesian network. Obtaining the optimal solution of the parameters marks the completion of the variational inference training process, and the model possesses the ability to make inferences and predictions on new observed data.

[0035] SSD historical failure data Bayesian inference validation processes apply the optimal solution of model parameters to historical data to validate the model's predictive accuracy and generalization ability. Bayesian inference validation is a crucial step in model evaluation, ensuring the model's reliability in practical applications. The validation process divides historical failure data into training and test sets. The training set is used for model parameter optimization, while the test set is used to independently validate model performance. The Bayesian inference process uses SSD state data from the test set as observational evidence input to the optimized Bayesian network to calculate the lifetime prediction probability distribution for each SSD. Validation metrics include prediction accuracy, mean squared error, log-likelihood, and coverage of the prediction interval, which evaluate the model's predictive performance from different perspectives. The quality of the lifetime prediction probability distribution is evaluated by comparing it with the actual failure time; an ideal prediction distribution should include the actual failure time within a high-probability region. The validation process also includes sensitivity analysis to assess the model's sensitivity and stability to changes in input parameters. Cross-validation is used to evaluate the model's performance consistency across different data subsets, ensuring the model's generalization ability. The validation results provide a scientific basis for the model's practical deployment and application.

[0036] The remaining usage time distribution calculation process calculates the specific remaining usage time probability distribution for each SSD based on the lifetime prediction probability distribution and the current SSD status information. This calculation is the process of applying a general lifetime prediction model to a specific SSD instance. The calculation process first acquires the real-time status data of the current SSD, including observed variables such as P / E cycle rate, operating temperature, failure rate, and write performance. This status data is input into a trained Bayesian network for inference calculations, and the network outputs the survival probability of the SSD at various future time points. The remaining usage time distribution is calculated using the derivative of the survival probability function, which describes the probability density of SSD failure at different time points. The distribution calculation needs to consider the impact of factors such as the SSD's current health status, historical usage patterns, and expected workload on its future lifespan. The calculation results include statistical characteristics such as expected remaining lifespan, variance, confidence interval, and quantiles, providing quantitative basis for storage management decisions. The SSD lifetime prediction distribution model, as the final output, contains complete lifetime probability distribution information for each SSD, and the model can be dynamically updated to reflect real-time changes in the SSD's status.

[0037] In one specific embodiment, the process of executing step S103 may specifically include the following steps: The remaining usage time data in the SSD lifetime prediction distribution model is discretized and divided into health status categories to obtain five levels of SSD health status: excellent, good, normal, warning, and critical. Based on the SSD health status classification, the P / E cycle utilization rate is mapped to a state interval to obtain the P / E cycle utilization rate range corresponding to each health status. Based on the P / E cycle usage range, the historical SSD state evolution data is statistically processed to obtain a state transition count matrix by performing state transition frequency statistical processing. The state transition count matrix is ​​normalized using the maximum likelihood estimation algorithm to obtain the transition probability values ​​between each state. The SSD state transition probability matrix is ​​obtained by constructing a matrix based on the transition probability values ​​between each state and by absorbing state labeling.

[0038] Specifically, the discretization of remaining usage time data for health status classification transforms the continuous SSD lifetime prediction distribution into discrete health level classifications. Discretization is the process of converting continuous variables into categorical variables, facilitating subsequent state transition analysis and decision-making. The classification process first analyzes the probability distribution of remaining usage time for each SSD in the lifetime prediction distribution model, extracting the expected remaining lifetime as the primary basis for classification. A five-level system is used for health status classification: excellent status corresponds to SSDs with a remaining lifetime exceeding 80% of the expected total lifetime; good status corresponds to 60%-80%; normal status corresponds to 40%-60%; warning status corresponds to 20%-40%; and critical status corresponds to SSDs with less than 20% remaining lifetime. The classification thresholds are determined based on statistical analysis of historical lifetime data from a large number of enterprise-level SSDs, combined with manufacturer technical specifications and actual operating experience. The classification process also considers the uncertainty of lifetime prediction, adjusting the classification boundaries by analyzing the variance and confidence interval of the prediction distribution to ensure the stability and reliability of the classification results. The output of the health status classification is a discrete health level label for each SSD, reflecting the current overall health level and future risk level of the SSD. The classification results serve as the state space for Markov chain modeling.

[0039] The P / E cycle utilization status interval mapping process establishes a correspondence between health status classifications and P / E cycle physical parameters. Status interval mapping is the process of associating abstract health levels with specific physical measurement parameters. The mapping process first statistically analyzes the P / E cycle utilization distribution of SSDs under various health states, identifying the P / E cycle utilization range corresponding to different health levels. A good state typically corresponds to a P / E cycle utilization range of 0-20%, indicating that the SSD is still in its early stages of use with minimal NAND flash wear. A good state corresponds to a utilization range of 20%-40%, indicating stable SSD performance but with the beginning of slight wear signs. A normal state corresponds to a range of 40%-60%, indicating that the SSD has entered the mid-stage of use and wear leveling needs to be monitored. A warning state corresponds to a range of 60%-80%, indicating that the SSD is approaching the accelerated wear phase and write load needs to be reduced. A critical state corresponds to a range of 80%-100%, indicating that the SSD is nearing the end of its lifespan and data migration should be performed as soon as possible. The mapping relationship is established by determining the boundary values ​​of each status interval through regression analysis and clustering algorithms, considering the impact of different SSD models and operating environments on the mapping relationship. Determining the P / E cycle utilization range provides the physical basis for the Markov chain state definition, giving state transitions a clear physical meaning. Historical SSD state evolution data state transition frequency statistics are processed by analyzing a large amount of historical SSD operating data to calculate the transition frequency between each health state. State transition frequency statistics are a core step in Markov chain modeling, used to quantify the transition patterns between states. Statistical processing first collects long-term operating records of enterprise-level SSD arrays, including information such as the P / E cycle utilization and health status of each SSD at different time points. Data preprocessing includes missing value imputation, outlier detection, and time series alignment to ensure data quality and consistency. Frequency statistics use a sliding window method to analyze SSD state transition patterns, with the window size set to one month to capture short-term state changes while avoiding noise interference. Transition frequency calculation involves counting the number of SSDs that transition from one healthy state to another within a given time interval, calculating all possible state transition combinations. The state transition count matrix is ​​constructed by organizing the transition frequencies according to the initial state and the target state. The rows of the matrix represent the initial state, the columns represent the target state, and the matrix elements represent the observation count of the corresponding transition. The counting matrix contains 25 elements, corresponding to 5×5 state transition combinations, where the diagonal elements represent the frequency of state retention and the off-diagonal elements represent the frequency of state transitions.

[0040] The maximum likelihood estimation (MLA) algorithm's normalization process transforms the state transition count matrix into a probabilistic transition matrix. MLA is a parameter estimation method that estimates model parameters by finding the parameter values ​​that maximize the probability of occurrence of observed data. The normalization process first sums each row of the count matrix to obtain the total number of transitions from each initial state. The transition probabilities are calculated by dividing each matrix element by the sum of its corresponding row, ensuring that the sum of each row's elements equals 1, satisfying the basic requirements of probability distribution. MLA possesses asymptotic unbiasedness and consistency in Markov chain parameter estimation, providing optimal parameter estimates when the sample size is sufficiently large. The algorithm also includes smoothing techniques, such as Laplace smoothing or Good-Turing smoothing, to prevent the influence of zero-probability problems on subsequent inference. The smoothing process adds a small constant to each element of the count matrix and then re-normalizes, ensuring that all transition probabilities are positive. The calculated transition probability values ​​reflect the likelihood of transitions between different health states in the SSD; larger values ​​indicate a higher probability of transition. The transition probability values ​​between states form a complete description of transition probabilities, providing core parameters for the Markov chain model.

[0041] The matrix construction and absorption state labeling process organizes the transition probability values ​​into a standard Markov chain transition matrix. Matrix construction is the process of organizing the probability data according to the mathematical requirements of a Markov chain. The matrix construction process arranges the transition probability values ​​between states into a 5×5 matrix, where the rows and columns correspond to the health state numbers or labels, respectively. The arrangement of matrix elements follows the standard conventions of Markov chains, ensuring the randomness of the matrix, i.e., each row element is non-negative and sums to 1. Absorption state labeling is an important concept in Markov chain theory, representing a special state that cannot transition to other states once entered. In SSD health state transitions, critical states are usually labeled as absorption states because once SSD reaches a critical health level, it will not automatically recover to a better state. Absorption state labeling is achieved by modifying the transition matrix, setting the row corresponding to the critical state to have only diagonal elements of 1 and other elements of 0. The labeling process also includes the identification of instantaneous states, i.e., non-absorption states that may transition to absorption states; these states correspond to various intermediate health levels of SSD. The SSD state transition probability matrix, as the final output, fully describes the transition patterns of SSD between different health states.

[0042] In one specific embodiment, the process of executing step S104 may specifically include the following steps: The SSD state transition probability matrix and the current write load request are processed by Hidden Markov Model state inference to obtain the current health state sequence of each SSD unit. Based on the current health status sequence of the SSD unit, the written data is processed by hot, warm and cold data stratification and classification to obtain differentiated data access mode labels; Based on the differentiated data access mode tags, the write weight of each health status SSD is dynamically calculated and processed to obtain the wear-aware weight allocation matrix. The wear perception weight allocation matrix and the write amplification control threshold are dynamically optimized to obtain the optimal write path selection result. Based on the optimal write path selection results, SSD load balancing adjustments and scheduling instruction generation are performed to obtain an enterprise-level SSD write allocation strategy.

[0043] Specifically, the SSD state transition probability matrix Hidden Markov Model (HMM) state inference processing infers the true health status of each SSD unit by analyzing the characteristic patterns of the current write load requests. The HMM assumes that the true health status of the SSD is hidden and cannot be directly observed, but can only be inferred through observable performance metrics. The state inference processing first collects characteristic data of the current write load requests, including observable metrics such as write latency, IOPS changes, queue depth, and error rate. The inference algorithm uses the Viterbi algorithm to calculate the most probable health state sequence. This algorithm uses dynamic programming to find the hidden state path that maximizes the probability of the observed sequence. The calculation process requires combining three core parameters: the state transition probability matrix, the observation probability matrix, and the initial state probability distribution. The state transition probability matrix describes the probability of the SSD transitioning between different health states, and the observation probability matrix describes the probability of observing a certain performance behavior in a specific health state. The algorithm iterates by maintaining a dynamic programming table, recording the maximum probability path to each state at each time step, and obtains the optimal state sequence through backtracking. The current health state sequence of the SSD unit contains the most probable health state label for each SSD at the current time step, reflecting the overall health distribution and individual differences of the SSD array. The accuracy of inference results depends on the quality of the observation data and the precision of the model parameters. Regularly calibrating and updating the model parameters ensures the reliability of the inference results.

[0044] Data tiering and classification are implemented based on the current health status sequence of SSD units to differentiate the management of stored data. Data tiering is a technique that categorizes data according to its access frequency and importance. The tiered classification process first analyzes data access patterns, including read / write frequency, access time distribution, and data size. Hot data is defined as data with high access frequency and strict performance requirements, such as database indexes, cached files, and real-time computing data. This type of data should be allocated to SSDs in excellent health to ensure optimal performance. Warm data is defined as data with moderate access frequency and moderate performance requirements, such as application files, user documents, and log files. It is suitable for allocation to SSDs in good or normal health. Cold data is defined as data with low access frequency, mainly used for archiving, such as backup files, historical records, and archived documents. It is suitable for allocation to SSDs with health warnings or even criticality to fully utilize remaining capacity. The classification algorithm combines multiple factors, including data access history statistics, business priority marking, and data lifecycle management strategies, for comprehensive judgment. Differentiated data access pattern labels assign corresponding identifiers to each type of data. The labels contain attribute information such as data type, priority, performance requirements, and reliability requirements. The tag information is correlated with the SSD health status sequence to form the basis for data allocation decisions.

[0045] The dynamic write weight calculation process assigns different write weights to SSDs in different health states based on differentiated data access mode tags. The write weight reflects the relative proportion and priority of the write load undertaken by each SSD. The dynamic calculation process first establishes a mapping relationship between health state and write capacity. SSDs in excellent health have the highest write weight and can handle the heaviest write tasks; those in good health have the next highest, those in normal health have moderate weights, those in warning health have low weights, and those in critical health have the lowest weights or even stop writing altogether. The weight calculation also considers the matching degree of data types. Hot data has the highest matching degree with SSDs in excellent health and receives a higher weight, while cold data has a higher matching degree with SSDs in warning health. The calculation algorithm uses a multi-factor weighted model to comprehensively evaluate factors such as SSD health state, data type, current load, and historical performance. The dynamic weight adjustment mechanism updates the weight allocation in real time according to changes in SSD status and load. When the health state of an SSD declines, its write weight is automatically reduced; when the load increases, the weight distribution of each SSD is rebalanced. The wear-aware weighting matrix is ​​constructed by organizing the write weights of all SSDs in a matrix format. Rows represent different data types, columns represent different SSD units, and matrix elements are the corresponding write weight values. The weight matrix exhibits sparsity; most elements being zero indicates that certain data types do not match a specific SSD, while the values ​​of non-zero elements reflect the matching priority and weight.

[0046] Dynamic programming optimization combines a wear-aware weight allocation matrix with a write amplification control threshold for global optimization. Dynamic programming is an algorithmic approach for solving multi-stage decision-making problems by breaking down complex problems into subproblems and solving them step by step. The goal of optimization is to maximize overall storage performance and SSD lifetime balance while satisfying the write amplification control threshold constraint. The write amplification control threshold is defined as the maximum allowable write amplification factor for each SSD unit; exceeding this threshold accelerates SSD wear and performance degradation. The optimization algorithm constructs a state transition equation, with state variables including the current write load, health status, and remaining capacity of each SSD, and decision variables being the data allocation strategy and write scheduling order. The algorithm recursively calculates and finds the optimal decision sequence for the objective function, which comprehensively considers multiple dimensions such as performance metrics, lifetime metrics, and load balancing metrics. The optimization process uses a heuristic search algorithm to accelerate the solution and employs pruning strategies to eliminate obviously suboptimal solutions. The optimal write path selection result includes detailed information such as the target SSD allocation, write timing arrangement, and load balancing strategy for each data block. The selection result ensures that overall wear unevenness and write amplification effects are minimized while meeting performance requirements.

[0047] SSD load balancing adjustment and scheduling instruction generation are based on the optimal write path selection results to formulate specific execution strategies. Load balancing adjustment achieves relative load balance among SSDs by redistributing write tasks. The adjustment process first analyzes the unevenness of the current load distribution, calculates the load deviation and variance of each SSD, and identifies SSD units with excessively high or low loads. The load redistribution algorithm uses an iterative adjustment method, gradually transferring some write tasks from overloaded SSDs to lightly loaded SSDs until the preset balance target is reached. Scheduling instruction generation includes various types such as data migration instructions, write redirection instructions, and load limiting instructions, each containing specific execution parameters and timing arrangements. The instruction generation process considers the priority and dependencies of operations to ensure that the adjustment process does not affect normal storage services. The enterprise-level SSD write allocation strategy, as the final output, includes complete write scheduling rules, load balancing mechanisms, and exception handling strategies. The strategy has adaptive characteristics, dynamically adjusting the allocation rules according to changes in SSD status and load.

[0048] In one specific embodiment, the process of executing step S105 may specifically include the following steps: The hot, warm and cold data tiering information written into the allocation strategy of enterprise-grade SSDs is used to match the target SSDs for storage, resulting in a data-SSD health status correspondence table. Based on the data-SSD health status correspondence table, a progressive migration path planning process is performed on the current stored data to obtain the data migration execution sequence between SSDs; The network topology shortest path is calculated based on the data migration execution sequence between SSDs to obtain an optimized data transmission path scheme. The optimized data transmission path scheme and system load monitoring data are processed for migration timing control to obtain a migration scheduling plan for low-load periods. Based on the migration scheduling plan during low-load periods, the uniformity of P / E cycle distribution is evaluated in real time to obtain a wear-balanced storage scheme for solid-state drives.

[0049] Specifically, the hot, warm, and cold data tiering information storage target SSD matching process establishes the optimal correspondence between data and SSD health status by analyzing data classification information in enterprise-level SSD write allocation strategies. Storage target matching is an intelligent pairing process based on data characteristics and SSD health status. The matching process first extracts key attributes from the hot, warm, and cold data tiering information, including multi-dimensional features such as data access frequency, performance requirements, reliability requirements, and data importance level. Hot data has high access frequency and strict performance requirements, requiring matching with SSDs in excellent or good health status to ensure optimal response speed. Warm data has moderate access frequency and moderate performance requirements, suitable for matching with SSDs in normal health status to achieve a balance between performance and lifespan. Cold data has low access frequency and relaxed performance requirements, suitable for matching with SSDs in health warning or even critical status to fully utilize remaining storage capacity. The matching algorithm adopts a multi-criteria decision-making method, comprehensively considering factors such as the compatibility of data characteristics and SSD health status, current load distribution, and historical performance to calculate the matching degree score. The data-SSD health status mapping table establishes a mapping relationship between each data block and its most suitable SSD health status category. The table records complete information such as data identifier, data type, target health status, matching score, and alternative solutions. The mapping table also includes a dynamic adjustment mechanism that automatically updates the matching relationship when the SSD health status changes, ensuring the real-time effectiveness of the data allocation strategy.

[0050] The current incremental data migration path planning process is based on a data-SSD health status mapping table to formulate a data redistribution execution plan. Incremental migration refers to a strategy of migrating data in stages and small batches to avoid drastic impacts on storage performance. The path planning process first analyzes the difference between the current data distribution and the ideal distribution to identify the data blocks that need to be migrated and the target locations. The planning algorithm considers the priority of data migration, prioritizing data with higher importance to migrate to SSDs with better matching, and prioritizing data with high access frequency to receive migration resources. The migration path design needs to avoid circular dependencies and resource conflicts, and a topology sorting algorithm is used to determine a reasonable migration order. The incremental strategy decomposes large-scale migration tasks into multiple small batches, controlling the data volume of each batch within a reasonable range, and setting appropriate time intervals between batches to allow SSD performance recovery. The SSD data migration execution sequence records detailed information for each migration operation, including the source SSD, target SSD, migrated data blocks, execution order, and dependencies. The execution sequence is represented by a directed acyclic graph structure, where nodes represent migration operations and edges represent dependencies between operations, ensuring logical consistency and operational safety during the migration process.

[0051] The SSD data migration execution sequence network topology shortest path calculation process optimizes data transmission paths by analyzing the physical connections of the storage network. The network topology describes the connection structure and communication paths between SSD devices. The shortest path calculation uses Dijkstra's algorithm or Floyd's algorithm to find the optimal transmission path from the source SSD to the target SSD. Path optimization aims to minimize transmission latency, reduce network bandwidth usage, and avoid network bottlenecks. The calculation process needs to consider dynamic changes in the network topology, including real-time information such as link status, bandwidth utilization, and device load. The path selection algorithm also needs to consider the concurrency of data migration, avoiding path conflicts and resource contention when multiple migration tasks are executed simultaneously. The optimized data transmission path scheme includes detailed information such as the specific transmission path, estimated transmission time, bandwidth requirements, and alternative paths for each migration task. The path scheme has fault tolerance capabilities, automatically switching to an alternative path to continue the migration task when the primary transmission path fails. The selection of transmission paths also considers load balancing to avoid unbalanced situations where some network links are overloaded while others are idle.

[0052] The optimized data transmission path migration timing control process selects the optimal migration execution time window by analyzing system load monitoring data. Migration timing control is a key technology to ensure that data migration does not affect normal business operations. The control process first collects system load monitoring data, including multi-dimensional performance indicators such as CPU utilization, memory usage, network bandwidth utilization, and SSDIOPS load. Load analysis uses time series analysis methods to identify the periodic patterns and fluctuations of system load, determining the time distribution of low-load periods. The criteria for identifying low-load periods include system load below a preset threshold, low business access volume, and sufficient network bandwidth. The timing control algorithm, combined with a load prediction model, predicts future load change trends and selects a low-load window of sufficient duration to execute the migration task. The low-load period migration scheduling plan needs to consider the time requirements and resource consumption of the migration task to ensure that the predetermined migration goals are completed within the time window. The scheduling plan also includes an emergency handling mechanism, which can suspend or delay migration operations when the load suddenly increases, prioritizing the service quality of front-end businesses. The plan execution has flexible adjustment capabilities, dynamically adjusting the timing of subsequent migration tasks based on actual execution conditions.

[0053] The real-time evaluation and processing of P / E cycle distribution uniformity during low-load migration scheduling assesses the wear leveling effect by monitoring the P / E cycle usage of each SSD during data migration. P / E cycle distribution uniformity is a key indicator for measuring the wear balance of the SSD array. The real-time evaluation process continuously monitors the P / E cycle utilization changes of each SSD, calculating statistical indicators such as the standard deviation, coefficient of variation, and Gini coefficient to quantify the distribution uniformity. The evaluation algorithm uses a sliding window method to analyze the dynamic trend of P / E cycle distribution, with the window size set to an integer multiple of the migration cycle to capture the complete change process. Uniformity evaluation also needs to consider the initial state differences of SSDs, eliminating the influence of different initial P / E cycle numbers through normalization. Evaluation results include the current uniformity level, improvement trend, and expected time to reach the target. An adjustment mechanism is triggered when distribution unevenness worsens during the evaluation process. The solid-state drive wear leveling storage solution, as the final output, includes a complete data distribution strategy, migration execution plan, load balancing mechanism, and performance monitoring scheme. The storage solution has adaptive adjustment capabilities, dynamically optimizing data distribution and migration strategies based on changes in SSD health status and load patterns.

[0054] In one specific embodiment, the process of performing migration timing control processing on the optimized data transmission path scheme and system load monitoring data can specifically include the following steps: The optimized data transmission path scheme is input into the system load monitoring module for real-time IOPS performance index acquisition and processing to obtain the current load status data of the SSD array. Based on the current load status data of the SSD array, the historical load change patterns are periodically analyzed to obtain statistical results of system load fluctuation patterns. Based on the statistical results of system load fluctuation patterns, low load time windows are identified and predicted to obtain a list of candidate periods for optimal migration. The optimal migration timing candidate time period list is matched and optimized with the data migration execution sequence to obtain the migration task time allocation matrix; Based on the migration task time allocation matrix, the data volume limit for each migration and the execution priority sorting are performed to obtain the migration scheduling plan for low-load periods.

[0055] Specifically, the optimized data transmission path scheme system load monitoring module collects and processes real-time IOPS performance metrics. This is achieved by inputting transmission path information into the load monitoring module to trigger targeted performance data collection. IOPS, short for Input / Output Operations per Second, represents the storage device's ability to handle read and write requests. The collection process first determines the SSD units and network nodes to be monitored based on the transmission path scheme, establishing a dynamic configuration of monitoring points. The real-time collection process obtains the current IOPS value by reading the performance counters of the SSD controller, while simultaneously collecting relevant performance metrics such as read / write latency, queue depth, and error rate. The collection frequency is set to once per second to ensure the capture of rapid load changes and instantaneous peaks. The data collection process also includes system-level performance metrics such as network bandwidth utilization, CPU utilization, and memory usage, forming a multi-dimensional description of the load status. The generation of the current load status data for the SSD array employs data fusion technology, synchronizing and correlating performance metrics from different monitoring points according to timestamps. The load status data structure includes fields such as timestamp, SSD identifier, IOPS value, latency metric, and utilization metric. The data format uses a time-series structure to facilitate subsequent trend analysis and pattern recognition. Data quality control is implemented during the data collection process, including preprocessing operations such as outlier detection, missing value supplementation, and noise filtering to ensure the accuracy and integrity of the data.

[0056] This paper analyzes the historical load change patterns of SSD arrays using time series analysis to uncover the inherent patterns in load changes. Periodic analysis is a statistical method for identifying recurring time patterns in the data. The analysis first constructs a time series of historical load data, covering at least a full month of operation to ensure the capture of daily, weekly, and other levels of change patterns. Periodic identification uses a Fast Fourier Transform (FFT) algorithm to convert the time-domain load signal into a frequency-domain representation, identifying the main periodic components by analyzing spectral characteristics. The algorithm process includes data preprocessing, trend removal, periodic detection, and significance testing. Trend removal uses moving averages or multinomial fitting methods to eliminate the influence of long-term trends. Periodic detection identifies the main periods by finding peaks in the spectrum, and significance testing verifies whether the detected periods are statistically significant. The extraction of fluctuation patterns includes not only periodic components but also analysis of random and seasonal components. The statistical results of system load fluctuation patterns include parameters such as frequency, amplitude, and phase of various periodic components, as well as statistical characteristics of load changes such as mean, variance, and peak distribution. The statistical results are represented using a parametric model, which facilitates subsequent load forecasting and time window planning.

[0057] The system load fluctuation pattern statistical results, low-load time window identification and prediction processing, are based on historical patterns to predict future low-load periods. A low-load time window refers to a period where the system load remains at a consistently low level, suitable for background tasks such as data migration. The identification process first defines low-load criteria, including a specific percentage of IOPS below average, CPU utilization below a threshold, and sufficient network bandwidth utilization. The prediction algorithm uses a time series forecasting model, combining historical periodic patterns and trend information to predict future load levels. Prediction models include ARIMA, exponential smoothing, and machine learning regression models, with cross-validation selecting the model with the highest prediction accuracy. The window identification process scans the predicted load curve to find consecutive low-load periods, with the window length meeting the minimum time requirements of the data migration task. The identification algorithm also considers the uncertainty of load fluctuations, setting a safety margin to ensure the load does not exceed the expected range during the window. The generation of the optimal migration timing candidate period list prioritizes the identified low-load windows based on factors such as window duration, load level, available bandwidth, and historical reliability. The candidate time period list contains detailed information such as the start time, end time, average load level, and available resource capacity for each window. The list has the ability to be dynamically updated to adjust the prediction results based on the latest load monitoring data.

[0058] The optimal migration timing candidate time slot list, data migration execution sequence, and time matching optimization process involve optimally pairing migration tasks with available time windows. Time matching optimization is a combinatorial optimization problem that seeks task allocation schemes under time constraints. The matching process first analyzes the time requirements of each migration task, including expected execution time, resource consumption, and dependencies. The optimization algorithm uses a bin packing problem approach, treating migration tasks as items and time windows as containers, to find the bin packing scheme that maximizes resource utilization. The algorithm considers the dependencies between tasks, ensuring that subsequent tasks can only begin execution after the preceding tasks are completed. The matching process also needs to consider the interruptibility of tasks; some migration tasks can be executed in segments while others must be completed continuously. Optimization objectives include maximizing time window utilization, minimizing task latency, and balancing resource usage. The migration task time allocation matrix is ​​constructed in two-dimensional form, with rows representing migration tasks and columns representing time windows. Matrix elements indicate whether a task is assigned to a corresponding window. Matrix constraints include each task can only be assigned to one window, window capacity cannot be exceeded, and task dependencies must be satisfied. The allocation matrix can be solved using heuristic or exact algorithms, with the appropriate solution method chosen based on the problem size and time requirements.

[0059] The migration task time allocation matrix, single-migration data volume limit, and execution priority ranking ensure a smooth migration process by refining task execution parameters. The single-migration data volume limit controls the size of the data blocks processed in each migration operation to avoid excessive impact on storage performance. The limit processing first determines a reasonable upper limit for the data volume based on the SSD's performance characteristics and current load status, considering factors such as SSD write bandwidth, IOPS capacity, cache size, and controller processing capacity. The data volume limit setting adopts an adaptive strategy, dynamically adjusting the limit parameters based on real-time performance monitoring results. When a performance degradation is detected, the data volume is reduced; when performance is sufficient, the data volume is appropriately increased. Execution priority ranking is based on a comprehensive evaluation across multiple dimensions, including data importance, business impact, migration urgency, and resource requirements. The ranking algorithm uses a multi-criteria decision-making method, assigning weights to each factor and calculating a comprehensive priority score. Higher-priority tasks receive priority execution opportunities and resource allocation; tasks with the same priority are processed according to a first-come, first-served principle. The migration scheduling plan for low-load periods is the final output, containing complete information such as a detailed task execution schedule, resource allocation scheme, priority ranking, and exception handling strategies. The scheduling plan has an execution monitoring and feedback adjustment mechanism, which dynamically optimizes the scheduling of subsequent tasks based on actual execution and performance.

[0060] In one specific embodiment, the process of performing time-matching optimization processing on the optimal migration timing candidate time period list and the data migration execution sequence can specifically include the following steps: The duration of the time window in the optimal migration timing candidate time period list is matched with the migration task time requirements to obtain a set of executable migration tasks. Based on the set of executable migration tasks, the migration complexity of the data migration execution sequence between SSDs is evaluated and calculated to obtain the expected execution time data of each migration task; Based on the expected execution duration data, the candidate time periods for the optimal migration timing are allocated using a task binning algorithm to obtain a time period-task mapping table. The time period-task mapping table is used to perform conflict detection and adjustment with the current load status data of the SSD array to obtain a conflict-free time allocation scheme; The migration task time allocation matrix is ​​obtained by matrix organization and task dependency labeling based on the conflict-free time allocation scheme.

[0061] Specifically, the time window duration migration task time requirement matching process filters feasible task combinations by comparing the available duration of each time window in the optimal migration timing candidate period list with the time requirement of the migration task to be executed. Migration task time requirement matching is a crucial step to ensure that tasks can be completed within a given time window. The matching process first extracts the duration information of each time window in the candidate period list, including parameters such as window start time, end time, and net available duration. Time requirement assessment needs to consider multiple components for each migration task, including basic execution time, data transfer time, verification and confirmation time, and buffer time. The basic execution time is calculated based on the amount of migration data and SSD write speed; data transfer time considers the impact of network bandwidth and transmission path; and verification and confirmation time includes the time spent on data integrity verification and consistency checks. The matching algorithm uses a capacity-constrained screening mechanism, only tasks with time requirements less than or equal to the window duration are included in the candidate set. The screening process also considers the divisibility of tasks; some large migration tasks can be decomposed into multiple sub-tasks to be executed in different time windows. The executable migration task set contains all tasks filtered through time matching, and each task in the set is labeled with its corresponding available time window and execution constraints. The collection is organized using a multi-level classification structure, and is managed hierarchically according to dimensions such as task priority, time requirements, and resource consumption.

[0062] The migration complexity assessment of the executable migration task set calculates the execution time by analyzing the complexity of each task in the data migration sequence between SSDs. Migration complexity assessment is a process of quantitatively predicting migration time by comprehensively considering multiple influencing factors. The assessment calculation first establishes a complexity assessment model, which includes multiple dimensions such as data volume factor, transmission distance factor, SSD health status factor, network topology factor, and concurrency conflict factor. The data volume factor calculates the basic time consumption based on the total size of the data to be migrated. The transmission distance factor considers the impact of the physical distance and network hops between the source and target SSDs on transmission time. The SSD health status factor reflects the impact of the current performance level of the source and target SSDs on migration speed; SSDs with poor health experience decreased read / write speeds, leading to longer migration times. The network topology factor analyzes the complexity of the transmission path and bottlenecks; complex paths and bandwidth-constrained links increase transmission time. The concurrency conflict factor assesses resource contention when multiple migration tasks are executed simultaneously; high concurrency will correspondingly extend the execution time of each task. The complexity calculation uses a weighted scoring method to quantify the influence of each factor and calculate a comprehensive complexity index. The estimated execution time is obtained by multiplying the complexity index by the baseline execution time, which is determined based on statistics from historical migration tasks. The execution time data also includes an uncertainty estimate, expressed as a confidence interval representing the possible range of fluctuations in the predicted time.

[0063] The task binning algorithm allocates migration tasks according to time constraints, placing them into candidate optimal migration time slots. Binning is a classic algorithm for optimizing utilization by placing items in a limited-capacity container. The allocation process treats migration tasks as items to be packed, time windows as containers, the estimated execution duration of a task as the volume of an item, and the duration of a time window as the capacity of the container. The binning algorithm uses a first-fit decreasing algorithm, first sorting migration tasks in descending order of execution duration, and then finding the first time window that can accommodate each task. During the algorithm, the remaining capacity information of each time window is maintained, and the remaining capacity of that window is reduced accordingly when a task is allocated to it. Allocation optimization considers the priority weight of tasks, with higher-priority tasks receiving better time window allocations. The binning process also needs to handle dependencies between tasks, ensuring that the execution window of a preceding task is earlier than that of a subsequent task. The time slot-task mapping table records the specific task list allocated to each time slot, containing information such as time slot identifier, allocated task list, remaining capacity, and utilization rate. The mapping table uses a hash table structure for fast lookup and update operations. The quality assessment of the allocation results is quantified by calculating indicators such as overall time window utilization and task allocation success rate.

[0064] The SSD array's current load status data conflict detection and adjustment process, based on the time-slot-task mapping table, identifies and resolves resource conflicts in task allocation by analyzing real-time load conditions. Conflict detection is a security mechanism to ensure that migration tasks do not severely interfere with existing business loads. The detection and processing first establishes criteria for judging resource conflicts, including SSDIOPS utilization exceeding a safety threshold, excessive network bandwidth usage, and peak CPU utilization exceeding limits. The conflict detection algorithm traverses each task allocation in the mapping table, checking the expected load level of the corresponding SSD and network resources when executing the task. Load prediction is calculated based on current load status data and task resource consumption models, considering the cumulative effect and mutual influence of loads. When a conflict is detected, an adjustment mechanism is triggered, with adjustment strategies including task reallocation, time window adjustment, and execution parameter optimization. Task reallocation moves conflicting tasks to other time windows with lighter loads, and time window adjustment optimizes resource allocation by subdividing or merging windows. Execution parameter optimization includes measures such as reducing migration concurrency, reducing the amount of data transmitted in a single transmission, and extending task intervals. The adjustment process uses an iterative optimization method, repeatedly executing conflict detection and adjustment operations until all conflicts are resolved. The conflict-free time allocation scheme, as the result of the adjustment, ensures that all migration tasks can be executed safely without affecting normal business operations. The allocation scheme includes complete information such as detailed execution schedules, resource usage plans, and performance monitoring requirements.

[0065] The conflict-free time allocation scheme employs matrix organization and task dependency labeling to convert the final task allocation result into a standardized matrix representation. Matrix organization is a method of structurally representing complex time allocation relationships using mathematical matrices. The organization process first establishes a two-dimensional matrix structure. Rows represent migration tasks, columns represent time windows, and matrix elements indicate whether a task is allocated to its corresponding time window. Matrix elements use binary encoding: 1 indicates the task is allocated to that time window, and 0 indicates it is not allocated. Extended information in the matrix includes attributes such as the specific time period of task execution, resource usage, and execution priority. Task dependency labeling is achieved by adding dependency constraints to the matrix. Dependencies are represented using an adjacency matrix of a directed graph, where edges represent the sequential dependencies between tasks. The labeling process verifies whether the matrix allocation result satisfies all dependency constraints, ensuring that the execution time of preceding tasks is earlier than that of subsequent tasks. Dependency checks use a topological sorting algorithm to verify the logical correctness of the task execution sequence. The matrix also includes labels for exception handling and rollback mechanisms, enabling rapid identification of affected subsequent tasks when a task fails. The migration task time allocation matrix, as the final output, provides a complete task scheduling scheme and execution constraint information. The matrix data structure supports efficient query and update operations, facilitating real-time scheduling and dynamic adjustment during the migration process.

[0066] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An enterprise-level distributed storage management method for solid-state drive data, characterized in that, The method includes: The P / E cycle status acquisition and processing of enterprise-level solid-state drive arrays is performed by an intelligent wear-sensing controller to obtain SSD physical characteristic monitoring data. Based on the SSD physical characteristic monitoring data, a lifetime prediction model is obtained by performing lifetime prediction modeling through a Bayesian network algorithm. The SSD lifetime prediction distribution model is subjected to Markov chain health classification to obtain the SSD state transition probability matrix. Based on the SSD state transition probability matrix, wear-aware scheduling processing is performed on the distributed storage write load to obtain an enterprise-level SSD write allocation strategy. Based on the enterprise-level SSD write allocation strategy, the storage data is distributed and redistributed to obtain a solid-state drive wear leveling storage solution.

2. The enterprise-level solid-state drive data distributed storage management method according to claim 1, characterized in that, The process of acquiring and processing the P / E cycle status of the enterprise-level solid-state drive array through an intelligent wear-sensing controller to obtain SSD physical characteristic monitoring data includes: The remaining P / E cycle count is read and processed for each SSD unit in the enterprise-grade solid-state drive array using the NVMe SMART command set to obtain the P / E cycle counter data; Based on the temperature sensor, the operating temperature of the SSD unit corresponding to the P / E cycle counter data is acquired and processed to obtain the SSD temperature status data. The SSD temperature status data is correlated with the bad block remapping frequency to obtain NAND flash memory block failure rate statistics. Based on the NAND flash memory block failure rate statistics, the write amplification factor is calculated in real time to obtain the SSD write performance degradation index. Based on the SSD write performance degradation index, timestamp marking and data structured storage processing are performed to obtain SSD physical characteristic monitoring data.

3. The enterprise-level solid-state drive data distributed storage management method according to claim 1, characterized in that, The process of performing lifetime prediction modeling based on the SSD physical characteristic monitoring data using a Bayesian network algorithm to obtain an SSD lifetime prediction distribution model includes: The P / E cyclic counter data in the SSD physical characteristic monitoring data is processed by Bayesian network node variable definition to obtain the P / E cyclic node probability distribution. Based on the probability distribution of the P / E cycle nodes, the SSD temperature status data is modeled to obtain the temperature-lifetime correlation probability matrix by performing temperature-influence node modeling. Based on the temperature-lifetime correlation probability matrix, conditional probability calculations are performed on the NAND flash memory block failure rate statistics to obtain the failure rate conditional probability distribution. The failure rate conditional probability distribution and the SSD write performance degradation index are trained using the maximum likelihood estimation algorithm to obtain the Bayesian network parameter set. Based on the Bayesian network parameter set, a variational inference algorithm is used for prediction to obtain the SSD lifetime prediction distribution model.

4. The enterprise-level solid-state drive data distributed storage management method according to claim 3, characterized in that, The step of performing variational inference algorithm prediction based on the Bayesian network parameter set to obtain the SSD lifetime prediction distribution model includes: The Bayesian network parameter set is input into the variational inference algorithm to perform latent variable posterior distribution approximation processing to obtain the initial parameters of the variational distribution. Based on the initial parameters of the variational distribution, the mean field approximate iteration process is performed on the P / E cycle node and the temperature-affected node to obtain the converged value of the node variational parameter; Based on the convergence value of the node variational parameters, the evidence lower bound optimization calculation is performed to obtain the optimal solution of the model parameters. The optimal solution of the model parameters is verified by Bayesian inference with historical SSD failure data to obtain the lifetime prediction probability distribution. Based on the lifetime prediction probability distribution, the remaining usage time distribution of the current SSD state is calculated to obtain the SSD lifetime prediction distribution model.

5. The enterprise-level solid-state drive data distributed storage management method according to claim 1, characterized in that, The step of performing Markov chain health classification processing on the SSD lifetime prediction distribution model to obtain the SSD state transition probability matrix includes: The remaining usage time data in the SSD lifetime prediction distribution model is discretized and divided into health status categories to obtain five levels of SSD health status: excellent, good, normal, warning, and critical. Based on the SSD health status classification, the P / E cycle utilization rate is mapped to a state interval to obtain the P / E cycle utilization rate range corresponding to each health status. Based on the P / E cycle utilization range, the historical SSD state evolution data is statistically processed to obtain a state transition count matrix by performing state transition frequency statistical processing. The state transition counting matrix is ​​normalized using the maximum likelihood estimation algorithm to obtain the transition probability values ​​between each state. Based on the transition probability values ​​between each state, a matrix is ​​constructed and the state labeling process is performed to obtain the SSD state transition probability matrix.

6. The enterprise-level solid-state drive data distributed storage management method according to claim 1, characterized in that, The wear-aware scheduling process for distributed storage write load based on the SSD state transition probability matrix, resulting in an enterprise-level SSD write allocation strategy, includes: The SSD state transition probability matrix and the current write load request are subjected to Hidden Markov Model state inference processing to obtain the current health state sequence of each SSD unit. Based on the current health status sequence of the SSD unit, the written data is subjected to hot, warm and cold data hierarchical classification processing to obtain differentiated data access mode labels; Based on the differentiated data access mode labels, the write weight of each health state SSD is dynamically calculated to obtain the wear-aware weight allocation matrix. The wear perception weight allocation matrix and the write amplification control threshold are dynamically optimized to obtain the optimal write path selection result. Based on the optimized write path selection results, SSD load balancing adjustment and scheduling instruction generation are performed to obtain an enterprise-level SSD write allocation strategy.

7. The enterprise-level solid-state drive data distributed storage management method according to claim 1, characterized in that, The step of performing distributed management and redistribution of storage data according to the enterprise-level SSD write allocation strategy to obtain a solid-state drive wear-leveling storage scheme includes: The hot, warm and cold data tiering information written into the allocation strategy of the enterprise-level SSD is used for storage target SSD matching to obtain a data-SSD health status correspondence table. Based on the data-SSD health status correspondence table, a progressive migration path planning process is performed on the current stored data to obtain the data migration execution sequence between SSDs; Based on the data migration execution sequence between SSDs, the network topology shortest path calculation is performed to obtain an optimized data transmission path scheme. The optimized data transmission path scheme and system load monitoring data are processed for migration timing control to obtain a migration scheduling plan for low-load periods. Based on the low-load period migration scheduling plan, the P / E cycle distribution uniformity is evaluated in real time to obtain a solid-state drive wear leveling storage scheme.

8. The enterprise-level solid-state drive data distributed storage management method according to claim 7, characterized in that, The step of performing migration timing control processing on the optimized data transmission path scheme and system load monitoring data to obtain a low-load period migration scheduling plan includes: The optimized data transmission path scheme is input into the system load monitoring module for real-time IOPS performance index acquisition and processing to obtain the current load status data of the SSD array. Based on the current load status data of the SSD array, the historical load change patterns are periodically analyzed to obtain statistical results of system load fluctuation patterns. Based on the statistical results of the system load fluctuation pattern, low load time windows are identified and predicted to obtain a list of candidate time periods for the optimal migration opportunity. The optimal migration timing candidate time period list is matched and optimized with the data migration execution sequence to obtain the migration task time allocation matrix. Based on the migration task time allocation matrix, a single migration data volume limit and execution priority sorting process are performed to obtain a migration scheduling plan for low-load periods.

9. The enterprise-level solid-state drive data distributed storage management method according to claim 8, characterized in that, The step of performing time matching optimization processing on the optimal migration timing candidate time period list and the data migration execution sequence to obtain the migration task time allocation matrix includes: The duration of the time window in the optimal migration timing candidate time period list is matched with the migration task time requirement to obtain a set of executable migration tasks. Based on the set of executable migration tasks, the migration complexity of the data migration execution sequence between SSDs is evaluated and calculated to obtain the expected execution time data of each migration task; Based on the estimated execution duration data, the optimal migration timing candidate time period is allocated using the task binning algorithm to obtain a time period-task mapping table. The time period-task mapping table is compared with the current load status data of the SSD array for conflict detection and adjustment to obtain a conflict-free time allocation scheme. The migration task time allocation matrix is ​​obtained by performing matrix organization and task dependency labeling based on the conflict-free time allocation scheme.

Images