A storage redundancy closed-loop regulation method based on recent data loss probability

Abstract

The application discloses a storage redundancy closed-loop regulation method based on recent data loss probability, and the core lies in introducing a closed-loop feedback control framework into a storage system, regulating redundancy gradually, realizing reliability and performance stability, and avoiding "reconstruction storm" and operation and maintenance burden caused by centralized data reconstruction due to traditional static configuration. The method is based on four technical integrations: 1. state observable: based on hard disk failure prediction, the device health state is observed in real time. 2. target controllable: through a recent data loss probability model, long-term reliability is quantified as a recent risk margin. 3. regulation feasible: the structured design of the data strip makes the redundancy rate adjustable and the risk margin controllable. 4. execution without disturbance: through background gradual regulation, the disturbance to the front-end business performance is strictly bounded. The application realizes the reliability and I / O performance stability under failure, optimizes the storage efficiency, and is suitable for various scenes such as large-scale data centers and AI training.

Description

[0001] Instruction manual Technical Field

[0002] This invention relates to the field of computer storage system technology, and more specifically to a method for managing data redundancy in storage systems. Background Technology

[0003] The core of a storage system is to ensure data reliability while providing data access services. Redundancy technology is the key to achieving this goal. In 1988, the Redundant Independent Disk Array (RAID) technology was proposed. Through hard disk striping and parity checking, it achieved a combination of performance and reliability on an industrial scale for the first time. Its core idea is to tolerate the failure of a single or a small number of hard disks by introducing redundant hard disks [1]. Subsequently, erasure coding (EC) technology, which has higher storage efficiency, was introduced into storage systems from the field of communications. It converts K data blocks into K+M data blocks through mathematical coding and allows the recovery of the original data from any K blocks. It can support higher redundancy and flexibility [2].

[0004] RAID and EC together form a widely adopted and well-proven data protection operating mode—"failure-degradation-reconstruction": When a component failure renders a data block inaccessible, the entire stripe to which it belongs immediately enters a "degraded" state. At this time, the system's read performance may be impaired (if real-time decoding is required), and write performance will typically decrease. In response, a high-priority "reconstruction" task is triggered, the goal of which is to read all surviving data blocks, recalculate the lost content, and write it to a new storage location to restore redundancy. This process is a core part of the standard operation and maintenance process.

[0005] The prevalence and behavioral characteristics of this "failure-degradation-reconfiguration" pattern stem from two inherent design features of traditional redundancy technology:

[0006] ●Static redundancy configuration: The redundancy rate of a stripe and the layout of its internal data / verification units are usually fixed at creation and are difficult to adjust dynamically throughout its lifecycle. Therefore, regardless of how the health status of individual components changes, the system can only maintain its fixed redundancy level through a standardized process of "degradation-refactoring".

[0007] ●Limited stripe width: Due to the computational complexity of encoding and decoding, latency and the overhead of consistency management, the number of physical devices associated with a stripe is usually within a few dozen (common configurations such as 8+3, 16+4, etc.)

[10] . This means that the failure of a single hard drive will affect a fixed proportion of data units in the stripe, and the impact is relatively concentrated.

[0008] The above design implicitly assumes that hardware failure is a "low-probability event" for each stripe. However, in a large-scale storage environment consisting of tens of thousands of components, failure becomes a "high-probability event" or even an "inevitable event" that occurs continuously at the system level. At this point, the design premise deviates significantly from operational reality, revealing an irreconcilable internal contradiction: On the one hand, attempting to improve fault tolerance by increasing static redundancy units (e.g., changing from "16+4" to "16+10") will lead to a sharp decline in storage efficiency, creating a redundancy level with an insurmountable ceiling both engineeringly and economically. On the other hand, when the system continuously experiences high-frequency failures that contradict the design premise, the structural defects of "static redundancy + limited stripe width" are fully amplified, easily leading to performance fluctuations and reliability risks.

[0009] Therefore, after the system scales up, the following phenomenon can be observed:

[0010] ●Deterministic degradation of performance: A single device failure will immediately affect a fixed proportion of data units in all its stripes (e.g., about 5% in a 16+4 configuration). To repair these data, the system must initiate an independent and complete reconstruction calculation for each damaged stripe. This process requires centralized reading of all surviving data within the stripe (resulting in significant read amplification I / O) and competing with front-end services for shared network and storage bandwidth, inevitably causing a measurable and global performance degradation, a phenomenon known as “reconstruction storm” [3].

[0011] ●Significant associated failure risk: Dense reconstruction I / O will bring additional and continuous read and write pressure and heat load to a large number of already healthy hard drives in the cluster. Multiple reliability studies on large-scale production systems have shown that after experiencing peer hard drive failure and subsequent reconstruction pressure, the short-term failure rate of the remaining hard drives in the same failure domain will show a statistically significant increase, which makes it possible for an initial single failure to induce an associated failure chain[4].

[0012] ● High operational complexity: In a large-scale cluster consisting of tens of thousands of hard drives, hardware failure is a normal event that conforms to the probability model. The fixed data layout makes any device replacement, cluster expansion or load balancing operation equivalent to a large-scale data migration and re-encoding at the underlying level. As a result, the system will be in a dynamic fluctuation state of "partial stripe degradation - partial stripe reconstruction" for a long time. The focus of the operation and maintenance team has changed from dealing with discrete alarms to continuously managing a complex system with constantly fluctuating performance and reliability indicators, and the management cost increases non-linearly

[13] .

[0013] New types of workloads, such as large-scale artificial intelligence (AI) training and high-performance computing (HPC), amplify the impact of the aforementioned system behaviors due to their unique business characteristics and pose substantial risks to business continuity. These tasks are usually tightly coupled and coordinated by tens of thousands of expensive and power-consuming computing units (such as GPUs), and their efficiency is highly dependent on the continuity and stability of data supply. A single I / O performance jitter or data access delay caused by a storage "degradation" or "reconstruction storm" can be amplified through the communication link, significantly slowing down the training progress, and in severe cases, even causing the entire 10,000-calorie computing task to be suspended or restarted, resulting in huge losses

[14] . Industry analysis further points out that the scale of AI model parameters is growing exponentially, while its training efficiency is limited by the linear increase of storage bandwidth. This "scissors difference" makes the storage subsystem increasingly become the key bottleneck of the entire computing system.

[0014] Faced with these long-standing challenges, academia and industry have carried out a great deal of effective optimization work in parallel. These efforts mainly revolve around several clear directions: at the coding theory level, research focuses on designing local repair codes (LRCs) to complete repairs within subsets and reduce network traffic [5, 6]; and exploring the dynamic selection of stripe redundancy parameters based on the device failure rate history to optimize overall space efficiency

[12] . At the engineering implementation level, the focus is on using dedicated hardware such as DPUs and FPGAs to offload high-intensity encoding and decoding computations, freeing the reconstruction process from the host CPU [7]; and widely deploying machine learning-based hard disk failure prediction models in order to achieve a shift from passive response to proactive maintenance [8, 9]. At the system architecture level, attempts are made to buffer the impact of background tasks on front-end services by decoupling metadata and data paths, designing wider stripes, or introducing elastic redundancy strategies.

[0015] These extensive and in-depth research and engineering practices have greatly improved the robustness and efficiency of storage systems, laying a solid foundation for the technological evolution of the entire industry. They have significantly alleviated some of the pressure of the traditional "failure-degradation-reconstruction" model and achieved great success in scenarios such as general cloud storage and backup archiving. However, most of these optimizations are still incremental improvements made within the framework of static redundancy configuration and passive failure response. When facing scenarios with extreme requirements for performance stability, such as AI / HPC, the inherent defects of the existing technology framework (i.e., failures inevitably trigger degradation and centralized reconstruction) become insurmountable bottlenecks. The industry sometimes still needs to rely on highly complex and costly solutions such as full-stack deep collaborative design and over-configuration. This clearly shows that incremental improvements within the existing technology framework have matured and are beginning to face the challenge of diminishing marginal returns.

[0016] It is worth noting that underlying technologies such as fault prediction, resource pooling, and hardware offloading have matured, laying the enabling foundation for overturning the traditional passive "fault-degradation-reconfiguration" model and establishing a new proactive dynamic redundancy control method. However, the output of these prediction technologies is essentially a probability estimate with uncertainty, and fault events in large-scale environments exhibit significant correlation risks. Traditional passive models cannot effectively utilize or manage this imperfect information. Therefore, exploring a new proactive dynamic redundancy control method that can fully utilize predictive information, manage its uncertainty, and operate robustly under fault correlation and resource constraints is not only necessary but also technically feasible. This invention is proposed against this backdrop. Summary of the Invention

[0017] (a) Technical problems to be solved

[0018] The technical problem this invention aims to solve is to overcome the "failure-degradation-reconstruction" pattern defects caused by the "static redundancy" architecture of existing storage systems. In large-scale distributed environments, this defect can lead to significant performance fluctuations (reconstruction storms), increased risk of associated failures, and increased operational complexity. This invention aims to provide a redundancy management method that can mitigate the concentrated impact of failures into minor disturbances that the system can withstand, thereby ensuring both continuous data reliability and highly stable business I / O operation.

[0019] (II) Technical Solution

[0020] This invention provides a closed-loop control method for storage redundancy based on the probability of recent data loss, such as... Figure 1 As shown. Its core is to quantify the reliability target into an observable and controllable "risk margin" variable, and dynamically adjust the redundancy rate of the data strip through a continuous "perception-decision-execution" cycle, so as to construct a closed-loop feedback control loop, so that the system risk margin automatically converges and stabilizes at the target value.

[0021] 1. Core elements of a closed-loop control framework

[0022] ● Control objectives: To achieve the system's preset long-term reliability targets (such as MTTDL). target This is converted into the setpoint for closed-loop control. Based on the risk margin H... i The definition of (t) is given by H, whose target baseline value is H. target =0 (corresponding to the risk reaching the theoretical compensation trigger point P) target The long-term goal of the control system is to drive the risk margin H of each data strip. i (t) toward H target It approaches and stabilizes within a preset safe operating range. This range is determined by the target risk margin H. target and preset upper and lower thresholds Hlow H high Common definition, namely (H) low H high ), where H low <H target <H high By maintaining H i (t) Within this interval, the system can guarantee that its recent data loss risk is no higher than that of P. target The derived safety level ultimately achieves the goal of long-term reliability.

[0023] ●Controlled object: A storage pool consisting of multiple data stripes. The number of valid verification particles M for each data stripe. i (t) is a direct control variable.

[0024] ●Observer and observed variables:

[0025] ○ Observed variable: Risk margin (H) i (t)). This variable represents the "estimated probability of recent data loss" as assessed in real-time by the system. With a "target probability threshold" (P) determined by long-term reliability goals and system design. target The standardized scalar feedback signal is obtained after comparison. Its mathematical definition is:

[0026]

[0027] ○ Observer Implementation: This is handled by the state sensor module. It periodically collects multi-dimensional operational metrics of the storage devices (such as SMART attributes and I / O performance metrics), and uses a built-in device failure prediction model to output an estimated failure probability for each device within a future evaluation window.

[0028] To calculate the observed variable—risk margin—the state perceptron performs the following steps:

[0029] ■Status Query: Retrieve the core control variable for each data stripe from the metadata module—the number of currently active check particles M. i (t).

[0030] ■ Risk Calculation: For each data strip i, the set of estimated failure probabilities of associated devices is calculated. With the currently active number of verification particles M i Using (t) as input, the conditional probability estimate of data loss occurring in the future window for this strip is calculated through a recent data loss probability model.

[0031] ■Standardized Mapping: Finally, based on the above risk margin definition formula, Mapped to a standardized, dimensionless risk margin H i (t). The state sensor outputs the calculated risk margin, along with the necessary control context (such as data stripe identifiers), to the health controller.

[0032] ●Controller:

[0033] ○ Input: Receive the observation risk margin H from the state perceptron i (t) and related control contexts (such as data stripe identifier, last adjustment time, etc.).

[0034] ○ Control objective: To control H i (t) Adjust to the safe operating range (H) low H high Within ) and make its long-term mean approach H target .

[0035] ○ Controller Implementation: This is handled by the health controller module. It first determines H... i (t) Whether it deviates from the safe operating range, and calculate the risk margin deviation e. i (t)=H target -H i (t) is used as a reference. Then, based on the built-in preset control algorithm (such as threshold control, PI control, etc., see Example 3 for details) and constraints such as cooling period, a decision is made on whether to generate a control command and the command content (such as "add a check particle to strip i"). In the threshold control algorithm, when H i (t)<H low Redundancy is added when H is triggered. i (t)>H high And when the conditions are met, redundancy reduction is triggered, thereby driving H. i (t) Return to the safe range. The introduction of the cooling-off period mechanism is key to ensuring closed-loop stability.

[0036] Output: Generates a redundancy adjustment instruction for a specific data stripe (i.e., adjusts the number of validation particles M). i (t) instructions), and send them to the policy executor.

[0037] ●Actuators and control variables:

[0038] ○Control variable: Number of validation particles M per data strip i (t), adjusting this value changes the redundancy rate of the strip.

[0039] ○ Actuator Implementation: This is handled by the policy executor module. It translates the control instructions from the health controller into specific sequences of state operations on the verification particles, and executes them through the following modules in the coordination logic resource layer:

[0040] ■ Metadata module: Call its atomic operation interface to complete the verification of particle state changes (such as from "unallocated" to "reserved" and then to "active") and physical resource binding.

[0041] ■Task Management Module: Submit background encoding or reconstruction tasks to this module. This module is responsible for scheduling the data module to perform actual data reading, writing, encoding, and decoding calculations based on global resource constraints (upper limit of background task resource ratio ω). The strategy executor itself does not directly schedule the data module, nor does it perform specific data operations, thus ensuring the decoupling of control logic and data operations.

[0042] ● Disturbance and immunity design:

[0043] ○Main disturbances: sudden hard drive failure, sudden load changes, etc.

[0044] ○Anti-interference mechanism:

[0045] ■ Real-time decoding: When the data module encounters a hard drive failure while processing front-end read I / O, it directly uses the surviving data in the same stripe to decode and recover in real time. This serves as a fault-tolerant branch of the standard I / O processing flow, does not trigger degradation, and is transparent to the business.

[0046] ■ Decoupling of Status and Resources: Redundancy adjustments only change the status of metadata, while actual data operations are executed asynchronously in the background, avoiding blocking of foreground I / O.

[0047] ■Global resource constraints: The aggregate resource consumption (such as I / O bandwidth) of all background tasks is limited to a preset ratio ω.

[0048] Specifically, such as Figure 2 As shown in the flowchart on the left, the closed-loop feedback loop continuously executes the following three steps in a loop:

[0049] ●Health Sensing (Perception): The state sensor calculates the current risk margin H for each data strip. i (t).

[0050] ●Dynamic decision-making (control): Health controllers compare H i (t) and H target Based on the deviation and the built-in algorithm, control instructions are generated (such as "add a check particle to strip i").

[0051] ●Controlled Execution: The strategy executor receives control instructions and coordinates the metadata module, data module, and task management module of the logic resource layer to complete the state change and data writing of the verification particles in a controlled background task manner.

[0052] The execution results of this period change the state of the controlled object, which in turn affects the risk margin perceived in the next period, thus forming a closed loop, enabling the system to automatically maintain the risk margin within the target range.

[0053] 2. Data stripes: Structured design of the controlled object

[0054] Data stripes: As the basic data organization and control unit in the method described in this invention, the structured design of data stripes is the basis for dynamically adjusting the redundancy rate. For example... Figure 5 As shown, unlike traditional static stripes, a data strip contains K data particles and at most M... max There are 1,000 check particles. At any time t, its redundancy level is determined by the number of check particles M that are actually active. i (t) determines the corresponding redundancy rate R. i (t)=M i (t) / K quantifies its redundancy protection level. This value can be 0 ≤ M i (t)≤M max Within the range, it is dynamically adjusted according to control commands. For M i The regulation of (t) is the specific way to adjust the strip redundancy rate.

[0055] ●Strip width K: The choice of K requires a trade-off between multiple dimensions, including fault impact granularity, control fineness, metadata overhead, and encoding / decoding complexity. The method described in this invention can be adapted to a wide range of scenarios, from smaller K values ​​(e.g., 8, close to traditional RAID) to larger K values ​​(e.g., 256, wide stripes).

[0056] ●M max : for M i (t) A clear upper bound is presupposed. This boundedness is a mathematical prerequisite for the stable design of the controller, ensuring that the solution space of the closed-loop control is always bounded and efficient.

[0057] ●Particles: such as Figure 3 As shown, the data particles and check particles are collectively referred to as particles, which are the basic logical units constituting data stripes and the smallest operational units for redundancy rate control; each particle is dynamically mapped to a storage area of ​​the physical storage device; wherein, the check particles have three states: "unallocated", "reserved", and "active", and check particles in the "active" state are counted in the number of effective check particles M. i (t).

[0058] 3. Core Model, Control Algorithm, and Parameter System

[0059] 3.1 State Observation Model: Estimation of Recent Data Loss Probability and Calculation of Risk Margin

[0060] ● Hard disk failure prediction model: This invention integrates it and uses it as the basis of the observer. It takes hard disk operating indicators (SMART, I / O latency, etc.) as input and outputs the failure probability estimate of each hard disk in the future evaluation window T.

[0061] ● Recent Data Loss Probability Model: The system simultaneously monitors multiple assessment windows (e.g., T1 = 4 hours, T2 = 24 hours, T3 = 7 days) to address risks at different time scales. For data stripe i, at time t, given the set of failure probability estimates for its associated hard drive... And the current redundancy configuration M i (t), the model evaluates the conditional probability estimate of a data loss event occurring in the worst-case time window for this strip, denoted as . In engineering implementation, the Poisson distribution approximation is often used for efficient calculations (see Appendix A).

[0062] ●Risk margin calculation: The estimated value... Substituting the values ​​into the risk margin definition formula yields the current risk margin for that band.

[0063] 3.2 Dynamic Model of the Controlled Object

[0064] Within a discrete-time control framework (control period T) c The evolution of risk margin for each data strip can be described using a standard engineering dynamic model:

[0065] H i [k+1]=H i [k]+γ·u i [k]+d i [k]

[0066] in:

[0067] ●H i [k] represents the risk margin of the i-th periodic strip.

[0068] ●γ>0 is the globally uniform control gain, representing the theoretical impact of a unit control action on the risk margin.

[0069] ●u i [k]∈{-1, 0, +1} is the control command (reduce, maintain, increase check particles) output by the controller to strip i in the k-th period.

[0070] ●d i [k] is the lumped disturbance term, which includes all unmodeled dynamics such as the natural evolution of equipment failure probability and model estimation errors. Its magnitude is bounded, i.e., |d i [k]|≤d max .

[0071] 3.3 Control Algorithm and Global Resource Constraints

[0072] ●Control Algorithm: The core of the controller (health controller) is based on the observed risk margin deviation e. i [k]=H target -H i [k], and combined with the cooling period constraint (the interval between two adjustments must be at least T). cool Time), decision control instructions u i [k]. The algorithm can be a simple threshold control or a more complex proportional-integral (PI) control, as shown in Example 3. The parameters of any algorithm must satisfy the stability condition.

[0073] ●Global resource constraints: The total I / O bandwidth consumed by all background control tasks is strictly limited within each control cycle:

[0074]

[0075] Where c(+1) represents the bandwidth cost of adding a verification particle, ω is the preset upper limit of the background task bandwidth ratio (usually ω≤5%), and B total This represents the total system bandwidth. This constraint is the mathematical guarantee of the "performance without disturbance" principle.

[0076] 3.4 Systematic Parameter Design Framework

[0077] All the key operating parameters of this method constitute a complete system constrained by theory, and its design must follow the parameter constraint system in the appendix.

[0078] 4. System Architecture: Four-Layer Collaborative Design

[0079] like Figure 4 As shown, the present invention adopts a four-layer design in system architecture, consisting of a strategy interface layer, a redundancy control layer, a logical resource layer, and a physical resource layer, thereby achieving complete decoupling of control logic, data state, and physical resources.

[0080] ●Policy Interface Layer: This layer serves as the interface for administrators to configure storage pool policies. Its core is the storage pool management module. This module is responsible for:

[0081] ○ Receive and persist the regulation strategy parameters of the storage pool, including long-term reliability targets, data stripe width, etc.;

[0082] It provides a unified view of logical storage space, shielding the details of underlying redundancy dynamic adjustments. These parameters serve as static inputs and target bases for closed-loop control of the redundancy control layer.

[0083] ● Redundant Control Layer: The core of closed-loop feedback control, comprising three modules: a state sensor, a health controller, and a policy executor, corresponding to the observer, controller, and executor, respectively. Among them:

[0084] ○ Strategy Executor: As the execution endpoint of control instructions, it receives the output of the health controller and transforms it into a specific operation sequence that can be coordinated by various modules of the logical resource layer (metadata module, task management module) to ensure the safe, atomic and controlled execution of the control actions.

[0085] ●Logical Resource Layer: Provides logical resources and service support to ensure the safe, efficient, and uninterrupted execution of control commands. It includes:

[0086] ○ Metadata module: This is the authoritative component for maintaining the globally consistent state of the system, specifically responsible for:

[0087] ■ Information Maintenance: Persistently store and manage the following core mapping and configuration information: Policy Configuration: such as the target risk margin of the storage pool, trigger threshold, stripe width, etc.; Topology Mapping: the mapping relationship between logical volume addresses and data stripes, and the mapping relationship between particles within the data stripe and physical storage devices; Real-time Status: the current status of each particle and the number of currently active verification particles in its respective stripe.

[0088] ■ Service Provision: Provides atomic state query and update interfaces for all other modules in the system. For example, the state sensor reads from it to calculate risk margin; the data module obtains the particle physical position from it for data reading, writing, or real-time decoding; and the policy executor uses it to atomically update particle states to complete redundancy adjustments.

[0089] ■Resource Coordination: When responding to resource allocation requests, coordinate the physical resource management module and atomically complete the binding of logical resources to physical resources and the updating of metadata.

[0090] ■Consistency assurance: Ensuring the high availability and strong consistency of the metadata service itself is the foundation for achieving reliable closed-loop control.

[0091] ○ Data Module: This is the core of data processing in the logical resource layer, responsible for all calculations related to data particles. For example... Figure 2 As shown, its main functions include:

[0092] ■Standard I / O processing: Responds to standard read and write requests for data stripes from applications and completes data access operations.

[0093] ■ Real-time fault recovery: When processing read requests, if a storage device failure is encountered, the system automatically uses surviving granules within the same stripe to perform real-time decoding, recover and return user data. This process is transparent to the business and does not trigger system degradation.

[0094] ■Background Control Calculation: As a schedulable resource, this involves the recalculation and writing of verification data required for redundancy rate adjustment. These tasks are uniformly scheduled by the task management module and run using the remaining bandwidth of the foreground I / O (limited by the global constraint ω) to ensure "uninterrupted execution."

[0095] ■ Through the above design, the data module integrates three data paths: normal I / O, fault recovery, and background computing, achieving unified management of resources and logic.

[0096] ○ Task Management Module: Used to schedule and manage background tasks generated by redundant control, such as the calculation and writing of verification data. Based on task priority and preset global resource constraints (total I / O bandwidth percentage ω), it ensures that the aggregate resource consumption of all control-related background tasks does not exceed the constraints.

[0097] ●Physical Resource Layer: Provides a unified resource abstraction, shielding hardware differences, including:

[0098] The physical resource management module manages all physical storage resources (HDD, SSD, etc.) and provides an interface for the metadata module to allocate and release physical resources.

[0099] With this architecture, all high-overhead operations involving data movement and computation are executed as background tasks with strictly limited resource consumption at the logical resource layer. This allows the redundant control layer to focus on decision-making, fundamentally avoiding the performance impact on front-end business caused by centralized data reconstruction in traditional models.

[0100] (III) System Resilience Design

[0101] To ensure the system remains safe and controllable under non-ideal conditions such as prediction bias, resource contention, and sudden failures, this invention incorporates the following resilience mechanisms:

[0102] ● Robust observation: Supports confidence information for fault probability estimates, and can switch to a "conservative estimation" safety mode to avoid misjudgments caused by prediction bias.

[0103] ●Resource overload protection: Background tasks are subject to hard global bandwidth constraints ω and are scheduled with dynamic priority; when foreground I / O latency exceeds the limit, automatic degradation and control actions are taken, and only emergency risks are responded to.

[0104] ● Associated risk compensation: By using covariance analysis to identify the synchronous increase of risks within the same fault domain, the cooldown period is temporarily shortened, the trigger threshold is lowered, and the redundancy construction is accelerated.

[0105] ● Parameter Adaptation: Within the stability boundary defined in Appendix C, H is adjusted based on performance feedback. low T coolParameters are fine-tuned online in small steps to adapt to long-term environmental evolution.

[0106] The above strategies work together to ensure that the system can maintain basic data security and service availability even in extreme scenarios. For detailed mathematical models and parameter designs, please refer to Appendix E.

[0107] (iv) Advantages and positive effects

[0108] ● Stable performance: The centralized reconstruction after a failure is transformed into gradual compensation during the risk period. Combined with real-time decoding and background bandwidth constraints, the performance impact is suppressed to within 5%, effectively suppressing the "reconstruction storm".

[0109] ●Preventive protection: Based on fault prediction, redundancy is proactively added before actual equipment failure, and a redundancy buffer is built to significantly reduce the probability of data loss and the risk of associated faults.

[0110] ● Storage efficiency optimization: Redundancy rate is dynamically adjusted according to health status to avoid static high redundancy waste, and storage efficiency is improved by more than 30% under the same reliability.

[0111] ●Simplified Operation and Maintenance: Achieve full-cycle autonomy from risk perception to decision-making to compensation, significantly reducing manual intervention and lowering the complexity of large-scale cluster operation and maintenance. Attached Figure Description

[0112] Appendix Figure 1 Block diagram of storage redundancy closed-loop control model

[0113] Appendix Figure 2 Storage Redundancy Closed-Loop Control and Transparent Recovery Flowchart

[0114] Appendix Figure 3 Schematic diagram of particle dynamic management and state transition

[0115] Appendix Figure 4 Overall Architecture and Module Interaction Diagram of Storage Redundancy Closed-Loop Control System

[0116] Appendix Figure 5 Comparison chart of static strip organization and fault impact of data strips in this invention: Detailed Implementation

[0117] The present invention will now be described in further detail with reference to several embodiments.

[0118] Example 1: A Closed-Loop Control Method for Storage Redundancy Based on Recent Data Loss Probability

[0119] This embodiment is based on the three core steps of the method described in claim 1: health perception, dynamic decision-making, and controlled execution.

[0120] 1.1 System Architecture and Initial Configuration

[0121] As attached Figure 4 As shown, the system architecture in this embodiment includes four layers: a policy interface layer, a redundancy control layer, a logical resource layer, and a physical resource layer. The redundancy control layer is the core of closed-loop feedback control, including a state sensor (observer), a health controller (controller), and a policy executor (actuator). The logical resource layer includes a metadata module, a data module, and a task management module. The metadata module is the center of the system's global state; it specifically maintains the following types of core information and provides atomic access interfaces for all other modules:

[0122] ●Control strategy parameters: such as the target risk margin of the storage pool, compensation threshold, stripe width, etc. These are the static basis for the health controller's decision-making.

[0123] ● Logical-physical mapping relationship: This includes two parts: first, the mapping from user logical addresses to data stripes and their internal particles; second, the dynamic mapping from particles to physical storage device locations. This mapping is the basis for data modules to perform I / O and real-time recovery.

[0124] ●Dynamic runtime state: The most critical information is the number of currently active check particles for each data stripe, and the state of each check particle (e.g., "unassigned," "reserved," "active"). Additionally, it includes contextual information such as the last adjustment time required to implement the cooldown mechanism.

[0125] ● The metadata module, through its atomic update and consistency guarantee mechanism, ensures the correctness of the above information under concurrent access, which is the foundation for the stable operation of closed-loop control.

[0126] Users configure the following parameters for the new storage pool through the storage pool management module: based on long-term reliability targets (Mean Time To Loss (MTTDL)). target The data stripe width K is set to 256, and the static redundancy level M is set to... base =4, maximum number of particles verified M max =16. Based on the theoretical constraints in Appendices C and D, the system collaboratively designs and initializes all operating parameters:

[0127]

[0128]

[0129] Initialization complete: All stripes S i M i (0)=M base =4, all 4 check particles are in an "active" state, the rest are M max - 4 check particles are in an "unassigned" state.

[0130] 1.2 Full-process simulation of closed-loop control

[0131] We track specific data strips S1, S2, and S3, and deduce their application over multiple control periods k (t = k·T). c The state evolution under )

[0132] 1.2.1 Initial State (k=0)

[0133] ●Health Sensing: All associated hard drives are healthy, and the estimated probability of failure is [value missing]. (Corresponding to an annualized failure rate of approximately 0.5%, based on statistics from high-quality hard drives).

[0134] ● The state sensor calculates an estimate of the probability of recent data loss:

[0135] λ(0)=(K+M(0))·p base =260×10 -4 =0.0260

[0136] Calculated using Poisson distribution:

[0137]

[0138] Based on the definition of risk margin:

[0139]

[0140] ●Dynamic decision-making: The health controller compares H(0) = 2.63 with the threshold H low = -0.5. Since H(0) > H low Decision: No adjustment (u=0).

[0141] ●System status: M(0)=4, H(0)=2.63, in a highly efficient and silent state.

[0142] 1.2.2 Disturbance Sequence A: Gradual Increase in Single-Disk Risk and Unique Compensation

[0143] Striped S1 hard drive D a As early warnings emerge, the estimated probability of failure gradually increases. The table below lists the states and decisions for critical cycles (the values ​​are completely consistent with the original text):

[0144]

[0145]

[0146] 1 Since the strip was created and not adjusted, the cooling period constraint is automatically satisfied.

[0147] 2 M has been updated to 5, therefore λ = 260 × 10-4 +0.7 = 0.7260.

[0148] 3 M = 5, λ = 260 × 10 -4 +1 = 1.0260.

[0149] Controlled execution process (initial compensation, cycle 3):

[0150] 1. The policy executor receives the instruction from the health controller to "add 1 check particle to strip S1".

[0151] 2. The strategy executor calls the atomic interface of the metadata module, completing the process within a single transaction:

[0152] ■ Allocate an idle check particle logic identifier;

[0153] ■ Update its status from "Unassigned" to "Reserved";

[0154] ■ Bind a free physical storage space to the particle through the physical resource management module;

[0155] ■ Update the last adjustment timestamp of strip S1 (used for cooldown period determination).

[0156] 3. After the transaction is successful, the strategy executor submits a background coding task to the task management module with the lowest priority.

[0157] 4. The task management module schedules the task within the remaining bandwidth limit based on the global bandwidth constraint ω = 5%.

[0158] 5. Data module execution task: Obtain the physical positions of all 256 data particles and 4 "active" check particles in stripe S1 from the metadata module, read the data, calculate the value of the new check particle, and write it to the bound storage space.

[0159] 6. After the write operation is complete, the policy executor calls the metadata module again to atomically update the status of the verification particle to "active". At this point, the number of active verification particles M in stripe S1 officially becomes 5.

[0160] D a Actual failure (sudden) scenarios (failures after period 21 in the table):

[0161] At some point after the cooling-off period ends, hard drive D... a Physical failure occurred

[0162] ●Transparent Recovery (Independent Mechanism): Application reads D aThe data request arrives at the data module. The data module queries the metadata module and finds that the target particle is mapped to the faulty device D. a Then, the physical locations of all other surviving particles in S1 (including 5 check particles) are obtained, and user data is restored through real-time decoding. This process is a read-only operation, does not modify metadata status, does not change control variable M, does not trigger background reconstruction tasks, and is transparent to the business.

[0163] Closed-loop response (control cycle after a fault):

[0164] ●Health Sensing: Faulty Disk D a The probability is 1, and the current M = 5:

[0165] λ = 1.0260, H≈2.63

[0166] ●Dynamic decision-making: H = 2.63 > H low Decision: No adjustment.

[0167] Uniqueness verification: For the same risk source D a Compensation is triggered only once when p = 0.6; thereafter the redundancy level is increased to M = 5, and even if p increases to 1.0, the system risk margin is still covered by the cooling period and redundancy buffer, and no secondary compensation is generated.

[0168] 1.2.3 Disturbance Sequence B: Accumulation of Risk Across Multiple Markets and Limited Compensation

[0169] Consider stripe S2, with 5 hard drives (D) within the same fault domain. b ~D f Risks are rising simultaneously. Initial M=4, cooling-off period has passed.

[0170]

[0171] Controlled execution: exactly the same as in section 1.2.2, the policy executor coordinates the metadata module, task management module, and data module to complete the addition of verification particles in a manner constrained by background bandwidth.

[0172] 1.2.4 Disturbance Sequence C: Sudden Single-Disk Failure (No Warning)

[0173] Consider stripe S3, initial M=4, all devices are healthy. Hard disk D g Sudden physical failure (without warning).

[0174] ● Transparent recovery is triggered immediately after a failure: When an application read request encounters a faulty disk, the data module queries the metadata and uses other surviving particles within the stripe to decode and recover the data in real time. The metadata is not modified, no reconstruction task is triggered, and the business is completely unaware of the failure.

[0175] ●Next control cycle:

[0176] ○ The remaining equipment p=10 -4 λ = 1.0259.

[0177] ○

[0178] ○H<H low The cooling-off period has passed. Decision: Add a check particle (u = +1), M: 4 → 5.

[0179] ●After compensation:

[0180] ○M=5, λ=1.0260, H≈2.63.

[0181] ○ The risk has returned to a safe range and will not be adjusted further.

[0182] Conclusion: Sudden failures will inevitably trigger compensation, and a single compensation can reduce the risk to a safe level.

[0183] 1.3 Summary of Examples

[0184] This embodiment fully demonstrates the "sensing-decision-execution" cycle of closed-loop control through three typical perturbation sequences, and verifies the following core mechanisms:

[0185] ●Risk Quantification: By utilizing the probability of recent data loss and risk margin, vague equipment health predictions are transformed into clear, dimensionless control feedback signals.

[0186] ● Gradual adjustment: Only one check particle is adjusted at a time, and a mandatory cooldown period T is required between two adjustments. cool To avoid system oscillations.

[0187] ● Disturbance isolation: Real-time front-end decoding and back-end control tasks are separated, with the latter subject to a global bandwidth constraint of ω=5%, ensuring stable business performance.

[0188] ●Rational compensation: Compensation for a single risk source is limited to one time; compensation for multiple risk sources is limited to a certain number of times; and immediate response is required for sudden failures.

[0189] This embodiment demonstrates that the method effectively avoids the traditional passive "fault-degradation-reconstruction" model, achieving proactive, stable, and autonomous storage redundancy management. Any technical solution that dynamically adjusts the data redundancy rate based on real-time health status feedback and follows the aforementioned control logic and constraints, regardless of its bandwidth, falls within the protection scope of this invention.

[0190] Example 2: Implementation in a medium-width stripe (K=32) storage system

[0191] This embodiment shows that the method of the present invention is also applicable to smaller K values, only requiring adjustment of the control parameters according to the guidance in Appendix D.

[0192] System parameters (based on Appendices C and D):

[0193] ●K=32,M base =4, M max =12.

[0194] ●P target =1 / 6! ≈0.001389 (unchanged).

[0195] ●Control period T c = 0.5 hours (due to the single adjustment time τ) adjust ∞K).

[0196] ●Cooldown period T cool =12 hours (with T) c (proportional).

[0197] ●Other parameters: γ=0.3, ΔH th =0.8, ω=5%.

[0198] Regulation process (increased risk in individual trading units):

[0199] ● Initially, M = 4, H ≈ 11.84, which is safe.

[0200] ●When p=0.6, H≈-0.77<H low This triggers compensation (M: 4→5).

[0201] ●After compensation, when p = 0.7, H ≈ 1.71 > H low No adjustment.

[0202] ● Real-time decoding and recovery in case of failure, closed-loop without additional adjustments.

[0203] Conclusion: The strip width K is an adjustable design parameter within the framework of this invention, which can be adapted to T. c T cool With equal time parameters, this method can be flexibly deployed in storage systems of different sizes and performance requirements.

[0204] Example 3: PI Controller Implementation Example

[0205] This embodiment demonstrates a specific implementation using a proportional-integral (PI) continuous control algorithm. Its architecture is the same as in Embodiment 1 (see attached). Figure 4 Only the decision logic of the health controller is changed to a PI control law.

[0206] 3.1 Control Algorithm Design

[0207] Input, setpoint, and deviation:

[0208] ●Observed variable: Overall system risk margin H g [k] = min i H i [k].

[0209] ●Set point: Target risk margin H target =0.

[0210] ● Deviation: e[k]=H target -H g [k]=-H g [k].

[0211] PI control law:

[0212] The controller calculates the global redundancy adjustment ΔR in each control cycle k. g [k]:

[0213]

[0214] in:

[0215] ●K p >0 represents the proportionality coefficient.

[0216] ●K i >0 represents the integral coefficient.

[0217] ● This is the cumulative sum of the deviations (integral term).

[0218] Parameter tuning and stability constraints:

[0219] Controller parameter K p and K i The tuning must satisfy the stability condition:

[0220]

[0221] Where γ = 0.3 is the global control gain (same as in Example 1), T c =2 hours is the control cycle (same as in Example 1), K p =0.1, K i =0.005, Verification: K p =0.1 < 2 / 0.3 ≈ 6.67, and K i =0.005 < 0.1 / 2 = 0.05, which satisfies the condition.

[0222] Control quantity decomposition (from global to strip):

[0223] ●Identify the set of strips to be adjusted: Select H i [k]<Htarget The strips with a value of 0 form a set.

[0224] ● Calculate weights: For strips Its adjustment weight w i It is directly proportional to its risk margin deviation (the greater the deviation, the higher the priority):

[0225]

[0226] ● Calculate the strip check particle increment: the check particle increment ΔM assigned to strip i. i for:

[0227]

[0228] Where K = 256 is the strip width (using the configuration of Example 1), This indicates rounding up, ensuring the adjustment amount is an integer and at least 1 (when ΔR). g (when [k] > 0).

[0229] ● Apply cooling-off period and boundary constraints: For each strip to be adjusted, only when the time since the last adjustment... Hours (same as in Example 1), and after adjustment M i No more than M max Adjustments are only performed when the value is 16. Otherwise, the ΔM of that band... i Set to zero.

[0230] ● Apply global resource constraints: All aggregated resource consumption to be adjusted must meet bandwidth constraints.

[0231]

[0232] Where c(ΔM) i ) indicates adjustment ΔM i The I / O bandwidth required for each verification particle. If the constraint is exceeded, the I / O bandwidth is calculated according to the weight w. i Reduce the adjustment amount in descending order until the constraint is met.

[0233] 3.2 Implementation Example

[0234] Assume that during a certain control period k, the overall risk margin H of the system is... g [k] = -0.1 (i.e., e[k] = 0.1), and the cumulative sum of historical deviations There are two data stripes WS A and WS B The risk margin is lower than the target value, as shown in the following details:

[0235] ●WS A H A[k] = -0.3, K = 256, M A [k] = 5, Hours (greater than T) cool ).

[0236] ●WS B H B [k] = -0.2, K = 256, M B [k] = 4, Hours (greater than T) cool ).

[0237] Step 1: Calculate the global adjustment amount

[0238]

[0239] Step 2: Determine the set to be adjusted and the set of weights to be adjusted. Total deviation: (-H) A )+(-H B = 0.3 + 0.2 = 0.5. Weight:

[0240]

[0241] Step 3: Calculate the increment of check particles for each strip

[0242]

[0243] Adjusted M′ A =5+2=7≤M max M′ B =4+2=6≤M max None of them crossed the boundary.

[0244] Step 4: Check the cooldown period (met).

[0245] Step 5: Check global resource constraints. Assuming the bandwidth cost of adding a verification particle is c0, the total cost of this adjustment is (2+2)c0 = 4c0. The system needs to verify that 4c0 ≤ ωB. total If the conditions are met, an instruction is generated; otherwise, the weights are reduced proportionally.

[0246] Final output command: WS A Add 2 check particles, for WS B Two additional verification particles were added.

[0247] Controlled execution: After receiving the instruction, the strategy executor coordinates the modules of the logic resource layer to asynchronously and controllably execute the operation of adding verification particles for the two stripes in the same manner as in Example 1. The entire process is strictly limited by the background resource constraint ω.

[0248] This embodiment demonstrates that the closed-loop control framework proposed in this invention possesses sufficient flexibility and scalability at the control algorithm level. Any control algorithm conforming to the stability theorem and parameter constraints in the appendix (including but not limited to threshold control, PI control, model predictive control, etc.) can be seamlessly integrated, thereby meeting the differentiated requirements for control accuracy and stability in different scenarios.

[0249] Example 4: Integration Implementation in the Distributed Object Storage System Ceph

[0250] This embodiment demonstrates how the core method of this invention can be integrated into an existing complex system in a minimally invasive manner. During implementation, the design of all control parameters of the system must follow the parameter constraint system in Appendix C, the control logic must satisfy the stability theorem in Appendix B, and the calculation of risk margin must be based on the theory in Appendix A.

[0251] 4.1 Design Overview

[0252] To dynamically increase the number of valid parity blocks in Ceph's erasure coding (EC) pool and reuse existing Ceph data paths, the core strategy is "pre-provisioning of encoding dimensions and on-demand calculation and activation of parity blocks":

[0253] ● Coding pre-provisioning: During the Ceph EC pool creation phase, i.e., according to (K,M) max Configure encoding parameters. K and M max The selection should be based on the scenario guidance in Appendix D and meet the boundary constraints in Appendix C.

[0254] ● On-demand computation and writing: Daily writes only process currently active M. current One verification block.

[0255] ●Dynamic Activation: When closed-loop control decisions require additional redundancy, controlled verification block calculations and activations are performed in the background. The resource consumption of this background task is strictly limited by the global constraint ω.

[0256] 4.2 Key Implementation Steps

[0257] System configuration parameter initialization: When creating an EC pool, the administrator needs to specify parameters, and these parameters must be designed to satisfy a joint constraint set:

[0258] ● The number of data blocks K (strip width) is selected based on the scenario trade-offs in Appendix D.

[0259] ●Maximum number of parity blocks M max It must meet the maximum disaster recovery constraints in Appendix C.

[0260] ●Long-term reliability target MTTDL target Used to derive P target .

[0261] ●Control period T c Cooling period T cool The upper limit of the backend bandwidth ratio, ω, etc., must meet all the constraints in Appendix C regarding time scale, stability, operational feasibility, and resource isolation.

[0262] Health perception and decision-making are integrated into the Monitor:

[0263] ●Health Sensing Plugin: Periodically (period T) c Collect OSD metrics and calculate the risk margin H(t) for each PG. The calculation must be based on the recent data loss probability model (Poisson approximation) and the definition of risk margin.

[0264] ● Redundancy control plugin: Compares H(t) with the target value. If H(t) < H low And satisfy the cooling period T cool If this is the case, an instruction to add a check block is generated. The parameter design of the control algorithm (such as threshold control) must satisfy the stability theorem in the appendix.

[0265] Controlled execution:

[0266] ●The main OSD received a command to increase redundancy.

[0267] ● Atomized updates of metadata, reserving resources.

[0268] ● Submit a lowest priority background coding task to the central scheduler. The resource consumption budget of this task is limited by the global constraint ω.

[0269] ● Background tasks perform independent encoding calculations and writes.

[0270] ● After a successful write, the metadata is atomically updated, a new check block is activated, and the cooldown timer is reset.

[0271] Transparent Recovery: Fully reuses Ceph's existing EC decoding process to achieve real-time data recovery in the event of a fault. This process is independent of closed-loop control.

[0272] This embodiment demonstrates the engineering feasibility and universality of the method of the present invention. This embodiment shows that the closed-loop feedback control method protected by claim 1 can be effectively integrated into existing mature distributed storage systems without disrupting their core architecture, thus obtaining the benefits of forward-looking redundancy control.

[0273] Example 5: System Resilience Strategies and Degradation Modes

[0274] This embodiment demonstrates how to ensure basic data security and service availability through preset resilience strategies when the system encounters abnormal scenarios such as high predictive uncertainty, concentrated outbreaks of failures, or resource shortages.

[0275] 5.1 Switching to a safe mode when forecast uncertainty increases dramatically

[0276] ● Triggering condition: Detection of more than X% of hard drives with a confidence interval width of their estimated failure probability. If the threshold is exceeded, or if the short-term failure rate output by the predictive model deviates significantly from the actual observed failure rate (e.g., p < 0.01 in the chi-square test).

[0277] ●Execution action:

[0278] ■ The health controller automatically switches to safe mode.

[0279] ■ The state sensor calculates the risk margin H i When (t), a conservative estimate is uniformly adopted. Perform the calculation.

[0280] ■ Lower threshold H of target risk margin low Temporarily increase ΔH safe (e.g., adjust from -0.5 to 0).

[0281] ■ Send an alert that says "Prediction uncertainty has increased, enter safe mode".

[0282] ●Recovery conditions: When the prediction uncertainty index returns to normal and continues for Y control cycles, the system will automatically switch back to standard mode.

[0283] 5.2 Background task degradation during resource overload

[0284] ●Triggering condition: The task management module detects that the average waiting time of the background task queue exceeds the threshold. Or the P99.9 latency of the front-end I / O exceeds the business SLA.

[0285] ●Execution action:

[0286] ■ The task management module dynamically reduces the scheduling weight of background tasks to a minimum.

[0287] ■Health controller suspends all actions due to risk margin H i (t) is in [H low H high The "optimization" adjustment command is triggered within the specified range.

[0288] ■ Only allow risk margin (A more stringent threshold, such as -1.0), performs a redundancy increase operation on the stripes.

[0289] ■ Record and age out adjustment instructions that are waiting in line due to insufficient resources to avoid instruction backlog.

[0290] ●Recovery conditions: Once the queue waiting time and foreground latency return to normal, gradually restore the normal decision-making logic of the background task scheduling weight and health controller.

[0291] 5.3 Correlation Compensation When Fault Domain Risk Increases

[0292] ●Triggering condition: Through statistical analysis, the estimated probability of N or more hard drives failing within the same fault domain (e.g., rack) within Z hours is found. A significant synchronous increase (e.g., the increase exceeds the threshold δ).

[0293] ●Execution action:

[0294] ■ Mark this fault domain as having "increased associated risk".

[0295] ■ For all data stripes mapped to devices within this fault domain, the health controller uses a modified risk margin H′ when making decisions. i (t)=H i (t)-ΔH domain (ΔH domain >0) and H low The comparison makes it easier for compensation to be triggered.

[0296] ■ The cooling period T of the strip within this domain can be temporarily extended. cool Shorten by a certain percentage.

[0297] ●Recovery conditions: After the hard drive risk indicators in the fault domain return to normal and remain so for W hours, the associated risk markers are cleared.

[0298] References

[0299] [1]PATTERSON DA, GIBSON G, KATZ R HA case for redundant arrays ofinexpensive disks(RAID)[C] / / Proceedings of the 1988ACM SIGMOI) international conference on Management of data.1988: 109-116.

[0300] [2]PLANK JSA tutorial on Reed-Solomon coding for fault-tolerance inRAID-like systems[J].Software: Practice and Expericnce, 1997.27(9)995-1012.

[0301] [3]KHAN O,BURNS R,PLANK J S,etc.Rethinking erasure codes for cloudfile systems:Minimizing I / O for recovery and degraded reads[C] / / Proceedingsof the 10th USENIX Conference on File and Storage Technologies.San Jose,CA,USA USENIX Association,2012.1-14.

[0302] [4]JIANG W,HU C,ZHOU Y,et al.Are disks the dominant contributor forstorage failures?A comprehensive study of storage subsystem failurecharacteristics[C] / / Proceedings of the 6th USENIX conference on File andStorage Technologies(FAST′08)2008:1-15

[0303] [5]SATHIAMOORTHY M,ASTERIADIS S,BORTHAKUR D,et al.Xoring elephants:Novel erasure codes for big data[C] / / Proceedings of the VLDB Endowment.2013,6(5)325-336.

[0304] [6]WU Si,LIN Guantian,LEE Patrick P.C.,LI Cheng,XU Yinlong.OptimalWide Stripe Generation in Locally Repairable Codes via Staged Stripe Merging[C] / / Proceedings of the IEEE International Conference on DistributedComputing Systems(ICDCS).2024.

[0305] [7]ZHONG Kan,YU Zhiwang,LI Qiao,LUO Xianqiang,LONG Lir.gbo,TANYujuan,REN Ao,LIU Duo.DPC:DPU-accelerated High-Performanee File System Client[C] / / Proceedings of the 53rd International Conference on Parallel Processing(ICPP′24).Gotland,Sweden 2024.

[0306] [8]S.Lu,B.Luo,T.Patel,Y Yao,D.Tiwari,and W.Shi,“Making disk failurepredictions smarterl,”in Proc.18th USENIX Conf.File Storage Technol.(EAST′20),2020,pp.151-167.

[0307] [9]S.Han,PPC Lce,Z.Shen,C.He,Y.Liu,and T.Huang,“Toward adaptive diskfailure prediction via stream mining,”in Proc.40th IEEEInt.Conf.Distrib.Comput.Syst.(ICDCS),Jul.2020,pp.1-11.

[0308]

[10] SHEN Z,CAI Y,CHENG K,LEE PPC,LI X,HU Y,SHU J.A Survey of thePast,Present,and Future of Erasure Coding for Storage Systems[J]ACMTransactions on Storage,2024,20(4):Article 1.

[0309]

[11] Hai Zhou, Dan Feng, Yuchong Hu, Wei Wang, and Huadong Huang.CoRD: Combining Raid and Delta for Fast Partial Updates in Erasurc-Coded StorageClusters[C] / / Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis (SC′24) IEEE Press, 2024: 1-14. https / / doi.org / 10.1109 / SC41406.2024.00113

[0310]

[12] Saurabh Kadekodt, Francisco Maturana, Sanjith Athlur, Arif Merchant, KVRashmi, Gregory R.Ganger: Tiger: Disk-Adaptive Redundancy Without PlacementRestrictions.OSDI 2022: 413-429

[0311]

[13] LIU S, YE F, WU Q.Clustered Distributed Data Storage RepairingMultiple Failures[J].Entropy, 2025, 27(3):313.https: / / doi.org / 10.3390 / e27030313

[0312]

[14] WANG Y, SI II S, HE

[0313] appendix

[0314] Appendix A: Probability of Recent Data Loss, Risk Margin, and Uniqueness of Compensation

[0315] A.1 Definition and Engineering Estimation of Recent Data Loss Probability

[0316] For data stripe i, define:

[0317] ●K: Number of data particles (globally fixed)

[0318] ●M i (t): The number of check particles that are active for stripe i at time t.

[0319] ●n i (t)=K+M i (t): The total number of devices associated with stripe i

[0320] ● Estimated failure probability of device j within the future evaluation window T

[0321] The probability of recent data loss is defined as:

[0322]

[0323] In engineering implementation, the Poisson approximation is used for efficient calculation:

[0324]

[0325] The system monitors multiple time windows simultaneously (e.g., T1=4h, T2=24h, T3=7d), and the overall risk is the maximum value of each window.

[0326] A.2 Mathematical Construction of Risk Margin

[0327] Use directly As a control variable, it suffers from problems such as a wide numerical range, low sensitivity, and lack of control benchmark. Therefore, a risk margin H is constructed. i It must meet the following requirements:

[0328] ●Monotonicity: H i Follow Increases and strictly decreases;

[0329] ●Zero-value baseline: P exists target Make H i =0 when

[0330] ●High sensitivity: Sensitive to relative changes;

[0331] ●Numerical stability.

[0332] Theorem A.1 (Optimal Risk Margin Form) satisfies the above requirements in logarithmic form:

[0333]

[0334] This form possesses excellent properties such as minimizing KL divergence and corresponding Neyman-Pearson optimal test statistic.

[0335] Physical meaning:

[0336] ●H i >0: Risk is below the compensation trigger point; safe.

[0337] ●H i =0: The risk has reached the compensation trigger point and compensation is required immediately;

[0338] ●H i <0: The risk has exceeded the trigger point and is in a logically dangerous state.

[0339] A.3 The principle of uniqueness of compensation and the benchmark of standardization objectives

[0340] The principle of unique compensation: For the same risk source (such as the aging of a single device), the controller should perform a one-time, decisive compensation when the risk first reaches the critical point, and the compensation effect should cover the entire process of the risk source's continuous deterioration from the trigger point to complete failure. This principle stems from the limitations of resources, causal decision-making, and stability requirements.

[0341] Considering the current stripe configuration (K, M), the risk probability for a single device is p. The compensation uniqueness requirement states that after adding a check particle, even if the device immediately fails completely (p=1), the loss probability of the new configuration is still no higher than the loss probability at the time of compensation triggering.

[0342]

[0343] The critical trigger probability is obtained by solving:

[0344]

[0345] Corresponding theoretical target probability benchmark:

[0346]

[0347] In engineering implementation, to obtain a unified and stable control benchmark, the benchmark redundancy level M determined during system design is selected. base Substituting into the right-hand side of the above equation and approximating it as a global constant:

[0348]

[0349] All risk margin calculations for all stripes use this fixed P. target Its long-term average data loss time target MTTDL target It is related to the safety factor.

[0350] A.4 Standard Engineering Dynamic Model (SEDM)

[0351] Discrete-time control, period T c , in t k =kT c At time i, the dynamics of strip i is described by the following equation:

[0352] H i [k+1]=H i [k]+γ·u i [k]+d i [k],

[0353] in:

[0354] ●γ>0: Globally unified engineering control gain;

[0355] ●u i [k]∈{-1, 0, +1}: Control action (decrease, hold, increase check particles);

[0356] ●d i [k]: Lumped perturbation term, including natural evolution, estimation error, etc., satisfying |d i [k]|≤d max .

[0357] Overall status of the storage pool:

[0358] H[k+1]=H[k]+γ·u[k]+d[k], ||d[k]|| ∞ ≤d max .

[0359] Global resource constraints (per control cycle):

[0360]

[0361] Where c(+1)=α·K·S·(1+β) is the bandwidth consumption of adding a check particle, ω is the upper limit of the bandwidth ratio of the background task, and B total This represents the total I / O bandwidth of the system.

[0362] Under bandwidth constraints, the maximum number of stripes that can be adjusted concurrently per cycle is:

[0363]

[0364] Priority scheduling is used: by H i Sort in ascending order and prioritize compensating the strips with the lowest risk margin.

[0365] Appendix B: Closed-Loop Stability Analysis, Control Algorithm, and Resource Constraint Coordination

[0366] B.1 Control Algorithm Design

[0367] B.1.1 Threshold Control Algorithm

[0368] For strip i, the output u of control period k i [k]∈{-1, 0, +1} is determined by the following formula:

[0369]

[0370] Where H low H high Global upper and lower thresholds (ΔH) th =H high -H low (where hysteresis width) T represents the time elapsed since the last adjustment. cool This is the cooling-off period.

[0371] B.1.2 Global Coordination under Bandwidth Constraints

[0372] Let R be the total bandwidth available for background tasks per cycle. max =ωB total The bandwidth cost of adding a check particle is c(+1). The system executes the following in each cycle:

[0373] ● Collect all satisfaction Furthermore, the stripes whose cooling-off period has expired constitute the candidate set.

[0374] ●Press H i Sort in ascending order (lower risk margin, higher priority).

[0375] ● Allocate bandwidth resources sequentially until insufficient remaining bandwidth is available to perform the next adjustment:

[0376]

[0377] in This refers to the set of stripes for which adjustments have been approved.

[0378] B.2 Stability Theorem

[0379] B.2.1 Single-strip stability (assuming sufficient resources)

[0380] Theorem B.1 For a given SEDM model H i [k+1]=H i [k]+γu i [k]+d i [k](|d i [k]|≤d max For the strips, a threshold control law is applied if the following conditions are met:

[0381] ● Gain constraint: γ≥γ min >0;

[0382] ●Hysteresis constraint:

[0383] ●Cooling period constraints:

[0384] ●Lower bound triggering constraint: Then H i [k] operates with high probability and boundedness in the long term within [H]. low H high ]Inside.

[0385] Proof approach: Construct the Lyapunov function V i [k]=(H i [k]-h ref ) 2 H ref ∈[H low H high We analyze the expected drift during the adjustment and cooling phases separately. By combining the condition that the hysteresis width is greater than the cumulative worst-case disturbance during the cooling period, we can prove negative drift, thus ensuring that the system state is bounded near the target interval.

[0386] B.2.2 Overall stability of the storage pool (considering bandwidth constraints)

[0387] Theorem B.2 Suppose the storage pool contains N stripes, and the maximum number of concurrent adjustments per cycle is... Each band. Define the maximum negative deviation for the current period. minimum positive deviation (If no band is below the target, take ΔH) min =ΔH max If the concurrency capability is satisfied:

[0388]

[0389] Then there exists a finite time K1 such that for all k ≥ K1 and all i, we have:

[0390] H i [k]≥H low -ε′,

[0391] B.3 Summary of Stability and Parameter Relationship

[0392] Key parameter design constraints derived from stability analysis:

[0393]

[0394] All controller parameters (T) c Tcool ΔH th H low The system (including γ, ω, etc.) must be designed in conjunction with each other, and must simultaneously satisfy the above stability inequalities and the physical feasibility constraints in Appendix C in order to ensure the reliable and stable operation of the closed-loop system.

[0395] Appendix C: Parametric Constraint System, Physical Constraints, and Systematic Design

[0396] C.1 Parameter Classification

[0397]

[0398] C.2 Physical and Operational Feasibility Constraints

[0399]

[0400] in:

[0401] ●c(+1)=α·K·S·(1+β): Bandwidth cost of incrementing the check particle by 1

[0402] ● Maximum concurrent adjustments per cycle

[0403] C.3 Stability Constraints (from Appendix B)

[0404] To ensure closed-loop stability, the adjustable parameters of the controller must jointly satisfy the following inequalities:

[0405]

[0406] C.4 Systematic Design Process (Iterative Method)

[0407] Due to the strong coupling between parameters, parameter design must employ iterative approximation until all constraints are satisfied simultaneously.

[0408] Step 1: Determine immutable system parameters and business objectives: Hardware capabilities: B total , τ SLA T resp Reliability target: From MTTDL target Derivation of K and M base Appendix D weighs the options, and the initial selection is M. min M max Verify decoding delay, τ decode (K, M) max )≤τ SLA If not satisfied, decrease K or M. max Alternatively, a better encoding algorithm may be selected;

[0409] Step 2: Initialize controller parameters: Based on scenario experience, initially set T c Tcool ω, γ, ΔH th wait;

[0410] Step 3: Iterative verification and adjustment: Calculate L and τ adjust Check τ adjust <T c Verify all stability inequalities in C.3; if any constraint is not satisfied, adjust the relevant parameters (e.g., increase T). c Increase ΔH th shorten T cool Or increase γ);

[0411] Step 4: Calculate derived parameters: based on M base Get P target =1 / (M) base +2)! ; According to P target d max and the stability inequality determines H low H high .

[0412] C.5 Feasibility Conditions for Parameter Design

[0413] In summary, considering the above constraints, the necessary condition for the existence of a feasible solution for the parameters is:

[0414]

[0415] This condition implies conditions for K, ω, and T. c The fundamental limitation of the relationship among these three factors is the physical upper limit of the system's controllability.

[0416] Appendix D: Design Trade-offs and Extensibility of Strip Width K

[0417] D.1 K's Multidimensional Influence

[0418]

[0419] D.2 Determining the upper and lower bounds of K

[0420] Upper limit constraints (must be satisfied simultaneously):

[0421] ●Operation time constraints: in

[0422] ●Real-time decoding constraints: (Determined by the specific encoding algorithm).

[0423] ●Overall Limit:

[0424] Lower bound constraint:

[0425] ●Control effectiveness: ΔRmax =1 / K needs to be small enough to ensure fine-grained control. Take a typical ΔR. max ≤0.125, therefore

[0426] ●Statistical robustness: lower bound of experience

[0427] ●Comprehensive lower limit: K min =max(8, 16) = 16.

[0428] Feasible interval: K∈[16, K max ].

[0429] D.3 Coordination Design with Control System Parameters

[0430] ●Control period T c Strongly correlated with K: T c (K)=β·τ adjust (K)+T base , β>1.

[0431] ●Cooling period is usually related to T c Proportional: T cool (K)=ρ·T c (K), ρ∈[3,10], and must satisfy the stability inequality in Appendix C.

[0432] D.4 Typical Scenario Configuration Reference

[0433]

[0434] The strip width K is an adjustable "knob" within the framework of this invention, not a limiting condition. When K changes, T must be adjusted proportionally. c T cool Equal time parameters are used to ensure that τ is satisfied. adjust <T c And stability inequalities. Any scheme that dynamically adjusts the redundancy rate based on real-time health status feedback, regardless of the size of K, falls within the protection scope of this invention.

[0435] Appendix E: System Resilience Design and Degradation Strategies

[0436] E.1 Robust Observations for Prediction Uncertainty

[0437] The set of fault probability estimates acquired by the state sensor is Where σ ij (t) represents the standard deviation of the estimate. The system defines a conservative estimate. Where k is the coverage factor (e.g., k=2 corresponds to approximately 95% confidence). In the safety mode, the risk margin calculation is based on...

[0438] Define the predictive risk margin index Factors such as confidence interval width and historical prediction accuracy can be considered comprehensively. When At that time, a safe mode switch is triggered.

[0439] E.2 Mathematical Description of Resource Overload Coping Strategies

[0440] Let Q(t) be the average waiting time of the background task queue, and L tail (t) represents the foreground I / O P99.9 latency. The joint triggering condition for resource overload is:

[0441]

[0442] In degraded mode, the decision function of the health controller is modified as follows:

[0443]

[0444] Meanwhile, the scheduling weight w of the task management module bg Dynamically adjusted to w bg =min(w base , α / Q(t)), where α is a constant, to achieve negative feedback regulation.

[0445] E.3 Formal Identification of Associated Fault Risks

[0446] For the fault domain Define its risk covariance matrix Principal component analysis is used to extract the variance contribution rate η1 of the first principal component. When η1 > η th At that time, it was considered that there was a strong correlation between the risks of equipment within the domain. The correlation risk correction amount ΔH domain It can be designed as a monotonically increasing function of η1.

[0447] E.4 Parameter Adaptive Fine-Tuning Safety Boundary

[0448] For any adjustable parameter θ (such as H) low T cool ,γ), its online adjustment must comply with the preset safety boundary θ∈[θ min θ max This boundary is determined by the stability constraints in Appendix C. The tuning engine calculates the gradients of performance metrics (such as average risk margin and adjustment frequency) based on historical data within the time window, and uses gradient descent to explore within the safety boundary in small steps. All adjustments must be recorded to form a traceable log.

Claims

1. A closed-loop control method for storage redundancy based on the probability of recent data loss, characterized in that, The method is applied to a storage system built with data stripes as the basic unit; each data stripe contains K data particles and multiple pre-created check particles, wherein the number M of check particles that actually provide data protection is an adjustable variable; The method is executed cyclically according to a control cycle, and includes the following steps: Health sensing steps: Based on the collected storage device operating indicators, calculate the estimated failure probability of each device within the future evaluation window using the device failure prediction model; and for each data stripe, based on the recent data loss probability model, the estimated failure probability of the devices associated with the stripe and the number of validator particles M currently active in the stripe, calculate the estimated recent data loss probability of data loss occurring in the evaluation window for that stripe, and obtain its risk margin H accordingly. Dynamic decision-making steps: The risk margin of each data strip is compared with a preset target risk margin. Based on the deviation generated by the comparison and combined with the preset control time interval constraint, a preset control algorithm is used to generate a control instruction for adjusting the number of effective verification particles M of the strip. Controlled execution steps: In response to the control command, the state of the verification particles in the corresponding data strip is changed by atomically updating the metadata, so as to increase or decrease the number of verification particles that are actually effective; wherein, all verification data calculation and writing operations related to state changes are executed in the background task mode, and the aggregated resource consumption of such background tasks is subject to preset global constraints.

2. The method according to claim 1, characterized in that... : The recent data loss probability model is used to calculate the conditional probability estimate of data loss of the data strip within the evaluation window based on a given set of equipment failure probability estimates and the number of currently active verification particles M. The risk margin is a standardized scalar, which is the output of a function mapping between the estimated recent data loss probability and a target probability threshold. The mapping satisfies the following: the risk margin value decreases monotonically as the estimated recent data loss probability increases, and when the estimated recent data loss probability is equal to the target probability threshold, the risk margin takes a preset baseline value. The target probability threshold is derived from the system's long-term average data loss time target.

3. The method according to claim 1, characterized in that, The data particles and verification particles are collectively referred to as particles, which are the basic logical units that constitute data stripes and the smallest operational units for redundancy rate control. Each particle is dynamically mapped to a storage area of ​​the physical storage device. The verification particles have three states: "unallocated", "reserved", and "active". Whether they are included in the number of active verification particles M is determined by their current state.

4. The method according to claim 1, characterized in that, In the controlled execution steps, the operation of increasing the number of effective verification particles M specifically includes: atomically updating the status of at least one verification particle with a status of "unallocated" to "reserved" and binding it to physical storage resources; after the background task completes the calculation and writing of the verification data corresponding to the particle, atomically updating its status to "active"; wherein, the status change is achieved by updating metadata, without needing to migrate any existing user data in the data stripe.

5. The method according to claim 1, characterized in that, The preset control time interval constraint includes: for the same data strip, there must be at least a preset cooling period between two consecutive control commands that increase the number of valid verification particles M.

6. A storage redundancy closed-loop control system based on the probability of recent data loss, applied to a storage system constructed using data stripes as the basic unit, characterized in that, include: The strategy interface layer includes: The storage pool management module is configured to receive and persist the control strategy parameters of the storage pool. The control strategy parameters include at least the long-term average data loss time target and the stripe width, and provide these parameters as the basis for closed-loop control. At the same time, it provides a unified logical storage space to upper-layer applications. Redundant control layer, including: The status sensor is configured to: periodically collect the operating metrics of the storage devices, and calculate the estimated failure probability of each device within the future evaluation window based on the device failure prediction model; for each data stripe, query the metadata module to obtain the number of currently active check particles M, and calculate the estimated recent data loss probability of data loss occurring in the evaluation window for that stripe based on the set of estimated failure probability values ​​of the devices associated with that stripe through the recent data loss probability model; The estimated probability of recent data loss is converted into a standardized scalar risk margin H, and H is output to the health controller. The health controller is configured to: receive the current risk margin and related control status from the state sensor, compare the current risk margin with a preset target risk margin, and, in conjunction with the related control status (such as cooling period constraints), generate a redundancy rate adjustment command using a preset control algorithm based on the resulting risk margin deviation and control logic. The strategy executor is configured to: in response to the redundancy adjustment instruction, coordinate the metadata module and task management module of the logical resource layer to execute the redundancy adjustment instruction; The logical resource layer includes: The metadata module is configured to maintain global system state information related to redundancy control, including: (1) the logical attributes, physical mapping and state of data stripes and their internal components; (2) the mapping relationship between logical storage addresses and data stripes; (3) the strategy parameters for redundancy control; and to provide atomic state query, resource allocation and mapping record services for the state sensor, the health controller, the strategy executor and the data module. The data module is configured to: process standard read / write I / O requests for data stripes; when the standard read / write request fails due to storage device failure, use other surviving particles in the same stripe to decode and recover the requested data in real time; and perform verification data calculation and writing related to redundancy rate adjustment in a background task mode with globally constrained resource consumption. The task management module is configured to: uniformly schedule and manage the background tasks generated by the redundancy control, and ensure that the impact of all such background tasks on the front-end business is strictly limited according to the task priority and the preset global resource constraints (total I / O bandwidth ratio ω); The physical resource layer includes: The physical resource management module is configured to manage all physical storage resources and provide interfaces for the metadata module to allocate and release physical resources.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the storage redundancy closed-loop control method based on the probability of recent data loss as described in any one of claims 1 to 5.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the storage redundancy closed-loop control method based on the probability of recent data loss as described in any one of claims 1 to 5.

Images