An error correction storage method and system based on polynomial interpolation hybrid check

Abstract

The application relates to the technical field of distributed storage, and discloses a correction storage method and system based on polynomial interpolation mixed checking. On the basis of block storage of original data, a unified polynomial expression relationship is constructed based on the quantity relationship between data blocks and the generation relationship of the checking data, and a state quantity used for representing data recovery support capability is calculated and formed accordingly. By judging the value interval of the state quantity, different data failure processing modes are distinguished, and a corresponding data recovery strategy is selected to perform a recovery operation in the corresponding mode. The method makes the recovery process after data failure have definite processing basis and stable execution path, helps reduce the uncertainty of system resource occupation state caused by data recovery in a distributed storage environment with changing node scale, failure mode and running load, and thus helps maintain long-term stable operation of the system.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of distributed storage technology, and in particular to an error correction storage method and system based on polynomial interpolation hybrid verification. Background Technology

[0002] In the engineering operation of distributed storage systems, data is typically distributed across multiple storage nodes to meet capacity expansion and reliability requirements. As system scale increases and operating time grows, the number of storage nodes, operating load, and failure modes exhibit continuous changes. In practical applications, when a storage node fails or a data block becomes invalid, the system needs to perform corresponding data recovery operations to maintain data integrity and availability. However, in complex and dynamically changing operating environments, the recovery process after data failure often consumes significant system resources, with highly uncertain resource consumption levels that are difficult to maintain stable and controllable under different operating conditions.

[0003] In engineering practice, this uncertain resource occupancy state can adversely affect the normal operation of a system. For example, it can trigger network bandwidth contention, increase storage read / write pressure, and cause fluctuations in processing latency, thus limiting the system's continuous service capability under high load or multi-node failure conditions. Especially when node size, failure modes, and operating loads are constantly changing, the impact of the data recovery process on the overall system stability becomes more pronounced, easily becoming a significant factor restricting the long-term stable operation of distributed storage systems. Therefore, how to effectively support the data recovery process after data failure under complex and variable operating conditions, and avoid its unpredictable impact on the system's operating status, has become an objective and pressing technical problem in existing distributed storage technologies. Summary of the Invention

[0004] To address the aforementioned technical issues, a method and system for error correction storage based on polynomial interpolation hybrid verification is proposed. This method effectively supports the recovery process after data failure in distributed storage systems, thereby mitigating the uncertain impact of data recovery on system operation under complex operating conditions.

[0005] To achieve the above objectives, a first aspect of the present invention provides an error-correcting storage method based on polynomial interpolation hybrid verification, comprising the following steps:

[0006] Obtain the original data to be stored in the distributed storage system and divide the original data into several data blocks;

[0007] Based on the data blocks, the first data recovery support state quantity is calculated and formed. The first data recovery support state quantity is a composite technical quantity used to characterize the association consistency state and redundancy coverage state of the data blocks under the polynomial representation relationship. The first data recovery support state quantity is constructed based on the quantitative relationship of the data blocks and the generation relationship of the corresponding verification data to form a unified polynomial representation relationship, and the correspondence integrity between the data blocks and the verification data blocks is calculated based on the polynomial representation relationship.

[0008] Based on the value range of the first data recovery support state quantity, the data failure handling mode is determined. When the first data recovery support state quantity is in the first value range, a data recovery strategy based on local correlation is determined. When the first data recovery support state quantity is in the second value range, a data recovery strategy based on global correlation is determined.

[0009] Based on the determined data failure handling mode, select the corresponding data recovery strategy and perform data recovery processing based on the polynomial representation relationship;

[0010] Output the data recovery results.

[0011] A second aspect of the present invention provides an error-correcting storage system based on polynomial interpolation hybrid verification, comprising:

[0012] The data acquisition and segmentation module is used to acquire the raw data to be stored in the distributed storage system and divide the raw data into several data blocks;

[0013] The state quantity forming module, connected to the data acquisition and block segmentation module, is used to calculate and form the first data recovery support state quantity based on the data block. The first data recovery support state quantity is a composite technical quantity used to characterize the association consistency state and redundancy coverage state of the data block under the polynomial representation relationship. The state quantity forming module constructs a unified polynomial representation relationship based on the quantitative relationship of the data block and the generation relationship of the corresponding verification data, and calculates the correspondence integrity between the data block and the verification data block based on the polynomial representation relationship.

[0014] The processing mode determination module, connected to the state quantity formation module, is used to determine the data failure processing mode based on the value range of the first data recovery support state quantity. When the first data recovery support state quantity is in the first value range, it is determined to be a data recovery strategy based on local correlation. When the first data recovery support state quantity is in the second value range, it is determined to be a data recovery strategy based on global correlation.

[0015] The data recovery execution module, connected to the processing mode determination module, is used to select the corresponding data recovery strategy according to the determined data failure processing mode, and to perform data recovery processing based on the polynomial representation relationship.

[0016] The results output module is connected to the data recovery execution module and is used to output the data recovery results.

[0017] Compared with the prior art, the error correction and storage method and system based on polynomial interpolation hybrid verification provided by the present invention have the following technical effects:

[0018] By implementing the aforementioned error-correcting storage method based on polynomial interpolation hybrid verification, an intermediate state quantity characterizing the data recovery support state can be formed in a distributed storage system based on the unified association relationship between data blocks. This allows for the differentiation of data failure handling modes, ensuring that the data recovery process can be executed according to a predetermined mode regardless of the failure of different nodes. Therefore, the recovery operation after data failure can maintain a clear processing path and stable execution mode during system operation, thereby reducing the unpredictability of system resource occupancy during recovery and contributing to the stable operation of the distributed storage system under complex operating conditions. Attached Figure Description

[0019] Figure 1 A flowchart illustrating an error correction storage method based on polynomial interpolation hybrid verification provided in an embodiment of the present invention;

[0020] Figure 2 A schematic diagram of the overall processing flow on the encoding side of an error correction storage system based on polynomial interpolation hybrid verification, provided for an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram of a distributed storage data recovery decision-making process based on the relationship between node failure type and availability verification, provided by an embodiment of the present invention.

[0022] Figure 4 This is a schematic diagram of data segmentation provided in an embodiment of the present invention;

[0023] Figure 5 A schematic diagram of intra-group redundancy coding provided in an embodiment of the present invention;

[0024] Figure 6 A schematic diagram of enhanced ENIC redundancy coding provided in an embodiment of the present invention;

[0025] Figure 7 This is a structural block diagram of an error-correcting storage system based on polynomial interpolation hybrid verification, provided in an embodiment of the present invention.

[0026] Figure label:

[0027] Data acquisition and segmentation module-100, state quantity formation module-200, processing mode determination module-300, data recovery execution module-400, result output module-500. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0029] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0030] Example 1

[0031] This embodiment provides a specific implementation of an error-correcting storage method based on polynomial interpolation hybrid verification, applicable to the engineering operation environment of a distributed storage system. This distributed storage system typically includes multiple storage nodes and at least one control / scheduling node. The nodes are interconnected via a data network, and data is distributed across different storage nodes in the form of data blocks. During long-term operation, the number of nodes may expand or shrink, the operating load may fluctuate with business needs, and node failures may manifest as single-node failure, network isolation, short-term unavailability, disk damage, or data block loss, among other things.

[0032] Under the aforementioned complex operating conditions, data recovery operations after data failure are often unavoidable. However, the recovery process's demands on network bandwidth, disk read / write, and computing resources can fluctuate with changing operating conditions, leading to problems such as recovery tasks crowding out normal read / write resources, increased service latency, or decreased system stability. This embodiment provides an implementable processing flow that establishes recoverable associations and verification relationships during the data writing phase. Furthermore, it differentiates processing modes based on a unified state variable in the event of data failure, ensuring a clear recovery processing path under different operating conditions and mitigating the uncertain impact of the recovery process on the system's operational status.

[0033] This embodiment is applicable to engineering scenarios such as object storage, distributed file systems, block storage, or network-attached storage, and is especially suitable for deployment environments with dynamically changing node size, significant load fluctuations, and the need for long-term online operation.

[0034] The error correction and storage method described in this embodiment executes multiple processing stages sequentially in chronological order. The overall process includes stages such as data acquisition and segmentation, state variable formation, processing mode determination, data recovery execution, and result output. Each stage is logically connected, with the processing result of the previous stage serving as the input condition for the next stage, thus forming a complete data processing chain.

[0035] The overall process begins with acquiring and segmenting the data to be stored, forming unified data processing units. Then, based on these data blocks, a state variable characterizing the data recovery capability is constructed. Next, the corresponding data failure handling mode is determined based on the value range of this state variable. Under the determined processing mode, the corresponding data recovery operation is executed. Finally, the data recovery result is output for subsequent system use. These stages are executed sequentially during system operation, together forming a complete error correction and storage process. See also... Figure 1 This embodiment provides a flowchart of an error correction and storage method based on polynomial interpolation hybrid verification. For ease of implementation and reproduction, the process is divided into S100 to S500 according to the operation time axis. Each stage provides the executable actions, parameter ranges, and technical functions, and explains its input-output relationship with subsequent stages. The method includes the following steps:

[0036] S100. Obtain raw data and perform data block division: Obtain the raw data to be stored in the distributed storage system and divide the raw data into several data blocks.

[0037] In the S100 stage, the system acquires the raw data to be stored and divides it into several data blocks. The raw data can originate from client write requests, file uploads from upper-layer business systems, batch import tasks in data pipelines, etc. Data acquisition can be accomplished through the storage system's write interface, which can take the form of HTTP / HTTPS, RPC calls, message queue retrieval, or local file handle reading. To reduce the sensitivity of the write process to network jitter, a receive buffer is typically set up at the control node or entry node. The data stream is first written to a memory buffer or cache device before entering the block division process.

[0038] In one possible implementation, the data is divided sequentially: bytes are split from beginning to end to generate fixed-size data blocks; when the original data is less than a complete block, the remaining portion forms a tail block. The data block size is an engineering parameter and can be set between 4MB and 256MB: in scenarios with large objects such as object storage, 32MB to 256MB can be selected to reduce the amount of metadata; in scenarios with small files or random read / write operations, 4MB to 32MB can be selected to reduce the amplification of small requests. The choice of block size is usually determined based on engineering conventions such as system network bandwidth, disk throughput, and concurrent read / write patterns, and does not affect the core processing logic of this embodiment.

[0039] During the partitioning process, the system generates a block identifier for each data block (e.g., composed of object ID + block sequence number) and records basic attributes such as object size, number of blocks, block size, and write time in the metadata. This metadata can be stored in a distributed consistent storage or metadata service, and the write latency can be set within the range of 10ms to 500ms to adapt to different deployment scales. When asynchronous metadata writing is used, the number of write retries can be set to 1 to 5, with a single retry interval of 50ms to 500ms.

[0040] The technical role of the S100 stage is to transform the raw data into a unified data block unit and form traceable metadata information, providing operable input for subsequent "verification relationship organization" and "state quantity formation".

[0041] S200. Forming a first data recovery support state quantity based on data blocks: Based on the data blocks, a first data recovery support state quantity is calculated and formed. The first data recovery support state quantity is a composite technical quantity used to characterize the association consistency state and redundancy coverage state of the data blocks under the polynomial representation relationship. The first data recovery support state quantity is obtained by constructing a unified polynomial representation relationship based on the quantitative relationship of data blocks and the generation relationship of corresponding verification data, and calculating the correspondence integrity between data blocks and verification data blocks based on the polynomial representation relationship.

[0042] In phase S200, the system calculates and forms the first data recovery support state quantity based on the data blocks obtained in phase S100. This state quantity is a composite technical quantity used to characterize the correlation consistency state and redundancy coverage state of the data blocks under the polynomial representation relationship, and serves as the sole basis for determining the subsequent processing mode.

[0043] To make this step feasible, this embodiment decomposes the formation of the "first data recovery support state variable" into executable engineering operations. Specifically, the system performs the following set of operations to obtain the state variable:

[0044] First, the system constructs a unified polynomial representation based on the quantity relationship of data blocks and the generation relationship of verification data. In practical engineering, this "relationship" is not an abstract description, but is solidified as a set of structured metadata. For example, the following three types of searchable entries are established: first, "data block set entries," which record the block set and block sequence range corresponding to an object; second, "verification block entries," which record the identifier of the verification data block and its correspondence with the data block set; and third, "relationship index entries," which associate the data block set with the verification block entries to quickly locate the required input block during recovery. These entries can be stored in the metadata service or in the local index file of each storage node. The index update latency can be set from 10ms to 1s to accommodate high-concurrency writes.

[0045] Secondly, based on the aforementioned polynomial representation, the system calculates the "correspondence integrity between data blocks and verification data blocks" to obtain the first data recovery support state quantity. In practical engineering, this "correspondence integrity" can be implemented as the verification and counting of the following:

[0046] 1) For each data block, does a corresponding record exist that can be used for its recovery?

[0047] 2) Whether the corresponding verification data block has been successfully generated and written to the target node;

[0048] 3) Verify whether the readability of the data block is normal (e.g., block checksum passes, read request returns within timeout threshold);

[0049] 4) Whether the relation index entries are complete (e.g., no missing sequence numbers, no records pointing to empty nodes).

[0050] The above verification results can be organized into two basic indicators: one indicator reflects the completeness of the associated entries (corresponding to "association consistency status"); the other indicator reflects the coverage of the data block set by the verification data block (corresponding to "redundancy coverage status"). Subsequently, the system combines the two indicators to obtain the first data recovery support status quantity. To avoid the difficulty in implementing the "status quantity," this embodiment provides an executable output format: the first data recovery support status quantity is output as a comparable numerical status quantity, with a value range that can be set to a normalized interval of 0-100 or 0-1. The advantage of this numerical output is that it can be directly used for subsequent interval judgment and mode selection, and it facilitates log recording and runtime verification.

[0051] In one feasible implementation, the system calculates the state variable after each object write is completed; in another feasible implementation, the system triggers the calculation at a fixed time period, such as calculating the most recently written object set every 1 to 30 seconds, to reduce the calculation frequency under high concurrency. The triggering strategy is an engineering parameter selection and does not change the essence of state variable formation: it is based on the relationship of data block quantity, the relationship of verification data generation, the relationship of polynomial representation, and the corresponding integrity check.

[0052] The technical role of the S200 stage is to provide a unified, comparable, and traceable intermediate representation for subsequent processing, enabling the system to characterize the "current recovery support status" on the same scale under complex operating conditions, thereby providing a stable basis for the selection of recovery mode.

[0053] S300. Determine the data failure handling mode based on the value range: Determine the data failure handling mode based on the value range of the first data recovery support state quantity. When the first data recovery support state quantity is in the first value range, it is determined to be a data recovery strategy based on local correlation. When the first data recovery support state quantity is in the second value range, it is determined to be a data recovery strategy based on global correlation.

[0054] In phase S300, the system determines the data failure handling mode based on the value range of the first data recovery support state quantity. To eliminate the problem of the abstract and unrealistic nature of "first value range" and "second value range", this embodiment provides a clear method for setting the range and the engineering configuration form, and explains the basis for its setting.

[0055] In one possible implementation, the system pre-divides the value range of the first data recovery support state variable into two consecutive intervals and stores them in a configuration table. This configuration table can be written by operations personnel during deployment or generated by the system during initialization. The interval boundaries can be determined using empirical values ​​or based on system capacity planning, and are configurable engineering parameters. For example, when the state variable value range is 0–100, [0, 60) can be defined as the first interval, and [60, 100] as the second interval; when the state variable value range is 0–1, [0, 0.6) can be defined as the first interval, and [0.6, 1] as the second interval. The setting of interval boundaries is usually based on engineering stability considerations: when the state variable is in a lower interval, the system provides more certain support for correlation and redundancy within a local range; when the state variable is in a higher interval, the system has more sufficient and stable overall support conditions and can handle a larger range of recovery input sets. The above basis pertains to engineering experience and deployment planning and does not involve the derivation of the underlying mechanism.

[0056] After determining the interval, the system performs interval matching on the first data recovery support state variable calculated at the moment. The interval matching operation can be simply implemented as a comparison operation: read the interval boundaries from the configuration table, determine which interval the state variable belongs to, and output the corresponding mode identifier. The mode identifier can be in the form of an enumerated value or a string identifier, such as "Mode A / Mode B". This mode identifier is then written to the object metadata or to the recovery scheduling queue, serving as the control input for S400 phase strategy selection.

[0057] When the first data recovery support state value is within the first value range, the system determines to adopt a data recovery strategy based on local association relationships. In engineering terms, "local association relationship" means limiting the input data blocks and verification data blocks required for recovery to a relatively small association range, such as a block set within the same object, or a set of related blocks within the same placement domain, to reduce cross-node read range and concurrency pressure. When the first data recovery support state value is within the second value range, the system determines to adopt a data recovery strategy based on global association relationships. In engineering terms, "global association relationship" means allowing the recovery input set to span a wider range of associated records, such as allowing input selection from multiple block sets within the same object, or allowing selection of input block sets that meet the recovery conditions from a broader association index. The above "local / global" only reflects the difference in the size of the association range and does not introduce specific steps or algorithm details.

[0058] The technical role of the S300 phase is to transform state variables into executable processing modes, enabling subsequent recovery execution to proceed along a predetermined path under complex operating conditions, thereby reducing the uncertainty of the recovery process, which is characterized by "proliferation upon startup and uncontrollable resource consumption".

[0059] S400. Select a recovery strategy and perform data recovery processing: Based on the determined data failure handling mode, select the corresponding data recovery strategy and perform data recovery processing based on the polynomial representation relationship.

[0060] In phase S400, the system selects the corresponding data recovery strategy based on the data failure handling mode determined in phase S300, and performs data recovery processing based on the polynomial representation relationship. To ensure the reproducibility of this phase, this embodiment provides an engineering description from four aspects: failure detection, data reading, recovery execution, and write-back to disk.

[0061] First, there's failure detection and triggering. The system can identify data failures through node heartbeats, read / write error codes, block verification failures, or timeouts. The heartbeat period can be set from 100ms to 5s, and the timeout threshold can be set from 1s to 30s to adapt to different network conditions. When a data block read failure or verification failure is detected, the system generates a recovery task entry and sends it to the recovery queue. The recovery queue can be a memory queue, a message queue, or a persistent task table; to ensure the recovery tasks are traceable, in practice, each task is assigned a task ID and the trigger time, target block identifier, failure type, and associated index location are recorded.

[0062] Secondly, there's the strategy selection and input set determination. The system reads the pattern identifier recorded in the object's metadata, or recalculates the pattern identifier based on the current state, and selects either "local association recovery" or "global association recovery" accordingly. Under local association recovery, the system prioritizes retrieving available input blocks from the same association range as the target block; for example, it prioritizes selecting a set of blocks readable within the same placement domain to reduce cross-domain reads. Under global association recovery, the system allows selecting readable input blocks from a wider range of indexes to handle more complex failure scenarios. In engineering terms, this selection process manifests as: choosing different index query ranges and input candidate set sizes based on the pattern identifier, and performing readability checks on the candidate set (e.g., read request success rate, timeout conditions, and verification results).

[0063] Next comes the recovery execution and resource control. The system loads the selected input block set and corresponding check block set into the memory buffer based on polynomial representation relations, and completes the generation of reconstructed data blocks in memory. To control the disturbance to system resources during the recovery process, the recovery concurrency and single-task rate limit range can be set in engineering practice. For example, the recovery concurrency can be set to 1–64, and the single-task read bandwidth limit can be set to 10MB / s–500MB / s to adapt to clusters of different sizes. If the system detects that the current disk queue depth or network congestion reaches a preset threshold, the recovery task can be postponed or the concurrency reduced. This type of scheduling belongs to the engineering operation strategy and does not change the basic process of "selecting a strategy according to the mode and executing recovery".

[0064] Finally, there is the write-back and consistency update. The reconstructed data blocks can be written back to the original node (if the node recovers and becomes available) or to a standby node. The write-back method can be sequential writing with checksum generation. Checksum calculation can be performed before or after the write-back to verify the correctness of the write. After the write-back is complete, the system updates the metadata: marking the target block status as available, recording the new location, write-back time, and checksum information, and updating the recovery task status to complete or failed. The write-back timeout threshold can be set to 1 second to 60 seconds, and the number of retries can be set to 0 to 3 times to avoid prolonged resource occupation.

[0065] The technical role of the S400 stage is to enable the recovery process to be executed under mode control, with the retrieval range and execution method of the input set matching the current support status, thereby maintaining a controllable recovery path under changes in node size, fault mode, and load, and reducing the unpredictability of the recovery process's impact on system resource consumption.

[0066] S500 outputs data recovery results.

[0067] In the S500 phase, the system outputs the data recovery results. To ensure the output results are verifiable and executable, this embodiment defines the data recovery results as a set of recordable information, including at least: recovery task ID, target data block identifier, recovery completion status (success / failure), write-back location identifier, completion timestamp, and necessary error codes or failure reason descriptions. The output method can be writing to the system log, writing to the task table, reporting to the monitoring system, or returning the recovery status to the upper-layer application. Log writing can be asynchronous to reduce the impact on critical paths, and the asynchronous refresh period can be set to 100ms to 5s.

[0068] The technical role of the S500 phase is to provide traceable evidence for system operation, maintenance and subsequent scheduling, to make the results of recovery actions verifiable and auditable, and to provide reference status information for the system in the event of similar failures in the future.

[0069] In the various stages of the method described in this embodiment, there are multiple feasible implementation paths. To facilitate engineering selection, a comparative explanation of the key stages is provided below.

[0070] In the S100 data sharding process, fixed-size sharding is suitable for scenarios with large object sizes and primarily sequential read / write operations. Its advantages include a simple index structure and low block management overhead. Adaptive sharding is suitable for data types with large differences in object size or natural boundaries. Its advantages include reducing tail block waste and improving some access patterns. The choice between the two usually depends on the typical object size distribution and access characteristics of the system.

[0071] Regarding the state quantity calculation triggering method of S200, triggering by object write completion allows state quantities to closely correspond to object granularity, facilitating mode control for single object recovery; triggering by time period reduces the calculation frequency and is suitable for high-concurrency write scenarios. The choice between the two can be determined based on the write concurrency, the computing power of the control node, and the acceptable computational overhead.

[0072] Regarding the S300's range configuration options, static configuration during deployment facilitates operation and maintenance management and is suitable for clusters with relatively stable operating environments; adjustable configuration during operation allows for adaptation to changes in node size or long-term shifts in business load, making it suitable for environments with frequent elastic scaling. The choice between the two can be determined based on the system's operation and maintenance mode and change frequency.

[0073] Regarding the choice of recovery execution location in S400, executing the recovery task on the service node can reduce data migration, but may compete for resources with service read and write operations; executing the recovery task on a dedicated recovery node can reduce contention, but requires additional network transmission. The choice between the two can be determined based on the node's resource availability and network topology conditions.

[0074] The above comparisons all pertain to the selection of engineering implementation paths and do not change the overall logical chain of "state quantity - interval - mode - resumption execution" in this embodiment.

[0075] The core problem addressed in this embodiment is that in a distributed storage environment where node size, failure modes, and operating loads continuously change, the data recovery process after data failure has unpredictable and uncontrollable impacts on system resource consumption, thus affecting stable system operation. In engineering practice, this problem typically manifests as: unstable read range after recovery task triggering, diffusion of recovery concurrency and input set, leading to fluctuations in network and disk resource consumption, and consequently causing service latency jitter.

[0076] The solution in this embodiment is reflected in the coordinated combination of each step: S100 establishes a unified storage and recovery unit for data blocks, giving the recovery operation a clear target object; S200 solidifies the quantity and verification relationships of data blocks during the writing phase, and constructs a searchable polynomial representation relationship in the form of metadata. At the same time, it aggregates the two types of supporting information, association consistency and redundancy coverage, into the first data recovery supporting state quantity, so that the system obtains a comparable and recordable unified representation; S300 maps this state quantity to a clear value range and outputs a processing mode identifier, so that the recovery decision changes from "only triggered by the failure event" to "mode selection driven by the supporting state"; S400 performs recovery under mode control, and the input set retrieval range and execution method match the current supporting state, so that the recovery process has a predictable processing path; S500 outputs the recovery result and accumulates traceable information, so that the recovery behavior is verifiable and can be managed in a closed loop.

[0077] From an engineering perspective, the common approach in existing technologies is to directly initiate recovery upon detecting a failure and retrieve the necessary data as quickly as possible. However, the selection of the recovery input set often lacks a unified supporting state representation, easily leading to resource consumption fluctuations where "the more urgent the recovery, the greater the spread." This embodiment breaks through this conventional approach in its design philosophy: the determination of the recovery mode is no longer driven solely by the failure event itself, but rather by the first data recovery supporting state quantity as the technical hub, enabling the system to consistently determine the recovery processing mode under complex operating conditions. This combined approach helps maintain the controllability of the recovery processing path under changes in node size, failure mode, and operating load, mitigating the unpredictable impact of the recovery process on resource consumption, thereby supporting stable system operation.

[0078] It should be noted that there is a strict sequential dependency and input-output relationship between the steps in this embodiment. The data blocks and basic metadata output by S100 are the direct inputs for S200 to form the first data recovery support state quantity; if the data block quantity relationship and verification relationship record are missing, it is impossible to construct a searchable polynomial representation relationship, and it is also impossible to verify the corresponding integrity, thus resulting in the state quantity not being reliably formed. The first data recovery support state quantity output by S200 is the only input for S300 interval judgment; if the state quantity is not formed or the state quantity is not comparable, then "first value interval / second value interval" cannot be implemented, and the processing mode cannot be determined. The processing mode identifier output by S300 is the control input for S400 strategy selection; if S300 is skipped and recovery is performed directly, the retrieval range of the recovery input set will lack mode constraints, and resource consumption fluctuations are more likely to occur under complex operating conditions. The recovery result output by S400 is the basis for the output content of S500; if there is no task status and write-back information of the recovery process, the output cannot be verified and traced. Adjusting the order of steps, such as calculating state variables before S100 is segmented, will result in incomplete state variable data. Similarly, determining the processing mode before solidifying the relationship index in S200 will lead to a lack of traceable state input for mode selection. Both of these situations will weaken the feasibility and stability of the overall process. Therefore, executing S100 to S500 sequentially along the timeline forms a clear logical loop, ensuring project reproducibility.

[0079] Example 2

[0080] Building upon the basic process of Embodiment 1, Embodiment 2 further provides a set of composable preferred implementation methods to provide a more stable basis and more controllable resource usage boundaries for recovery decisions and execution after data failure in distributed storage environments where node size, failure modes, and operating loads continuously change. Embodiment 2 does not repeat the basic process description of writing, redundancy generation, and recovery from Embodiment 1, but only provides explanations where necessary: ​​On the one hand, the data blocks, verification data blocks, and their distribution information formed by the encoding side during the writing phase provide traceable input for the subsequent formation of the "first data recovery support state quantity"; on the other hand, after detecting a failure, the recovery side uses this state quantity as a single hub to complete interval mapping, mode determination, strategy selection, and recovery execution.

[0081] See appendix Figure 2 , Figure 2 This paper demonstrates the overall processing flow of an error correction storage system based on polynomial interpolation hybrid verification on the encoding side. The process starts from the writing of the original data, and sequentially completes data segmentation, multi-level verification generation and encoding result storage, ultimately forming a hybrid redundancy structure that can be used for subsequent recovery.

[0082] In step S1, the system first enters the initialization phase, obtains the original data to be stored, and simultaneously determines the basic parameters related to storage, including the number of storage nodes, the size of a single data block, and the number of data blocks within a stripe, providing a unified configuration basis for subsequent data segmentation and verification generation.

[0083] In step S2, the system segments the original data according to a preset data block size. The data block size can be selected from commonly used engineering values ​​such as 64MB or 128MB. After segmentation, multiple sequentially arranged data blocks are formed within a single stripe, typically represented as follows: , ,…, These data blocks serve as the basic input objects for subsequent encoding and verification calculations.

[0084] In step S3, the system performs intra-group local XOR check processing on the data blocks within the stripe. Specifically, according to the parity of the data block index or a preset grouping rule, the data blocks are divided into two local groups: an even group and an odd group. Within each local group, a byte-by-byte XOR operation is performed on the data blocks to generate corresponding intra-group check blocks, thereby forming the first layer of redundancy structure for local recovery.

[0085] In step S4, the system further executes an enhanced ENIC check generation process. This step constructs a polynomial representation relationship based on the data blocks within the stripe, generates an intermediate coefficient matrix through methods such as difference quotient calculation, and then evaluates the polynomial at preset dedicated interpolation points to generate an enhanced ENIC check block. The ENIC check block is used to assist in recovery when local checks are insufficient, enhancing the overall error correction capability of the stripe.

[0086] In step S5, the system performs a global redundancy check generation operation. Specifically, the strip polynomial is evaluated at multiple global interpolation points to generate several global redundancy check blocks. These check blocks are used to provide global recovery support in the event of multi-data-block failure or cross-local-group failure.

[0087] In step S6, the system organizes the original data blocks within the stripe, the check blocks within the group, the ENIC check blocks, and the global redundant check blocks in a unified manner, and distributes them to different storage nodes according to a preset data placement strategy. Simultaneously, the system writes the polynomial representation relationships, block index mappings, and check coverage relationships related to the encoding process into the metadata to support subsequent data location and recovery operations.

[0088] In step S7, the system completes the encoding process and enters the final stage. At this point, all data blocks within the stripe and their corresponding multi-level check blocks have been written into the distributed storage system, forming a complete hybrid redundant storage structure, providing a reliable foundation for subsequent recovery decisions and execution in data failure scenarios.

[0089] See appendix Figure 3 ,like Figure 3 As shown in the diagram, this illustration depicts the process of making and executing data recovery decisions based on node failure types and availability verification relationships after a data failure occurs in a distributed storage system. This process is used to select a data repair path that matches the current recovery support capabilities when different failure modes exist, and to perform cyclical processing when multiple failures exist.

[0090] In step T1, the system first detects the operating status of nodes in the current storage system, counts the number of faulty nodes, and determines the type of node fault. This determination is used to distinguish between single-node and multi-node faults, and to identify whether the faulty node contains a data node or a check node, thus providing a basis for subsequent repair path selection.

[0091] In step T2, when the detection result indicates a single-node failure, the system enters the single-node failure repair process. At this point, based on the different data types carried by the faulty node, it is further distinguished as a data node failure or a verification node failure, and the corresponding repair judgment branch is entered accordingly.

[0092] In step T3, for a single-node failure scenario, the system determines whether the failure is an intra-group failure, that is, whether the failed node is located in the same data block group or local verification group. This determination is used to determine whether repair can be completed locally, or whether cross-group or global verification resources need to be called to participate in the recovery.

[0093] In step T4, the system performs corresponding data repair operations based on the aforementioned judgment results. Specifically, this includes: when a data node failure occurs and the group repair conditions are met, recovery is performed using an intra-group XOR method; when a check node failure occurs and enhanced check relationships are involved, a repair method based on polynomial interpolation is used; when a global check node failure is involved, a repair method based on global check encoding is used; when only a data node failure occurs but the local repair conditions are not met, repair is performed using an XOR or enhanced check method; when the faulty node also contains a check node and cannot be directly repaired, a re-encoding operation is performed to rebuild the corresponding data and check relationships.

[0094] In step T5, after completing a repair operation, the system checks whether there are still any unrepaired data nodes or verification nodes. If there are still nodes to be repaired, the system returns to steps T1 to T4 and repeats the detection, judgment, and repair process for the remaining faulty nodes; if there are no nodes to be repaired, the current data recovery process ends.

[0095] In a preferred embodiment, the first data recovery support status quantity is formed based on historical data block storage status information and current data block distribution status information, wherein the historical data block storage status information includes historical data block integrity status and historical verification data availability status, and the current data block distribution status information includes the distribution relationship between data blocks and verification data blocks in storage nodes.

[0096] In one possible implementation, the input to the first data recovery support state quantity consists of two sources: one is "historical data block storage state information," and the other is "current data block distribution state information." To facilitate a consistent description of data block objects within a stripe on both the encoding and recovery sides, this embodiment uniformly represents the set of data blocks within a stripe as a vector-like input object:

[0097] ;

[0098] in, Indicates the first band within the strip i A data block or a data block fragment, nThis represents the total number of data blocks within a stripe. This notation system is used to unify the input objects of subsequent interpolation point sets, difference quotient coefficient matrices, and basis function vectors under the same semantic framework, enabling the aggregation, recording, and tracing of "historical data block integrity status, historical verification data availability status, and current distribution relationships" around the same data block index. The historical data block integrity status can be obtained by background inspection tasks or post-read verification tasks. Inspection tasks sample and read recently written data blocks at the object or stripe granularity, and perform verification and comparison. The sampling period can be set to 5 minutes to 24 hours, and the sampling ratio can be set to 0.1% to 10% to control inspection overhead and maintain statistical validity when system load changes. When sampling hits a stripe, the system records the readability flag, verification and consistency flag, and the timestamp of the most recent successful read for each data block within that stripe, thus forming a traceable historical integrity status. The availability status of historical verification data can be determined by the success rate of verification block reads, verification block checksum consistency, and timeout distribution. The statistical window can be set from 10 minutes to 6 hours to cover the impact of short-term load fluctuations and node jitter on the readability of verification blocks. The current data block distribution status information is directly obtained from the write placement results: when the write scheduler writes data blocks and verification data blocks to disk on each storage node, it records the node identifier, fault domain identifier (e.g., rack, availability zone), and cross-domain distribution ratio between blocks within the same stripe for each block, and writes this distribution relationship into the stripe metadata. To ensure direct module collaboration, data transfer between the metadata service and the status aggregation module can be achieved through an RPC interface connection or a message queue subscription connection. The RPC timeout can be set from 100ms to 2s, and the number of retries can be set from 1 to 3 to avoid amplifying system overhead during failures. When using a message queue, topics can be divided into "write events," "inspection update events," and "node topology change events," and the retention time can be set from 10 minutes to 24 hours to support traceability. See Appendix. Figure 4 , attached Figure 4 This is a diagram illustrating data segmentation, showing the structural relationship of original data being divided into multiple data blocks; in this embodiment, the attached diagram... Figure 4The data block sequence and its index shown are the basic objects of "current distribution status information" because the distribution relationship is associated with the node identifier by the block index as the primary key, thus enabling the distribution status to be quantified and used for the formation of state variables. By introducing a dual-source input of "historical + current", the state variables can simultaneously reflect the availability trend over a period of time and the current placement status during engineering operation. This allows the recovery side to avoid relying solely on instantaneous observations when node size and load continue to change, thereby reducing the suddenness and unpredictability of disk and network resource consumption under adverse conditions. From a creative expression perspective, this constraint explicitly organizes the information on which recovery decisions depend as a combination of "historical integrity / historical verification availability / current distribution relationship" inputs, reducing the substitutability of the scheme being interpreted as making decisions based solely on a single failure phenomenon.

[0099] In a preferred embodiment, the first data recovery support state quantity consists of at least two sub-state quantities, including a first correlation consistency sub-state quantity and a first redundancy coverage sub-state quantity, wherein the first correlation consistency sub-state quantity is used to characterize the completeness of the polynomial representation relationship between data blocks, and the first redundancy coverage sub-state quantity is used to characterize the coverage range of the verification data block on the data block.

[0100] In one possible implementation, the first data recovery support state variable is implemented as an "aggregate structure of at least two sub-state variables," each with a clear engineering meaning and computable scope. The first association consistency sub-state variable characterizes the completeness of the polynomial representation relationship between data blocks. Its execution method can be quantified using "completeness of representation relationship elements": when the system forms the polynomial representation relationship on the encoding side, it writes the interpolation point set, the data block index set within the stripe, and the intermediate representation elements used for evaluation or reconstruction into metadata; after reading this metadata, the recovery side checks whether these elements are complete, whether the references point to valid block identifiers, and whether the corresponding blocks are locatable in the current distribution state. To ensure that this consistency can be directly reproduced, the polynomial representation relationship can use a Lagrange form as a unified semantic basis, as shown in the following example:

[0101] ;

[0102] in, For the first in the strip i The representation of a data block over a finite field The interpolation basis function is uniquely determined by the set of interpolation points. The above expression does not require the implementer to use a specific algorithm to solve it, but it provides a verifiable object for determining "relational integrity": when the set of interpolation points is missing, the block index mapping is missing, or there is a point conflict, it can be identified as an incomplete relationship, thereby reducing the risk of uncontrollable rollback when the recovery side enters a mode requiring global input. The first redundant coverage sub-state variable is used to characterize the coverage range of the check data block on the data block. Its execution method can be quantified using a "coverage mapping table": when generating check blocks, the encoding side records the set of data blocks covered by each check block, the check block type identifier, and the placement node identifier. The recovery side counts the number of readable check blocks in the current distribution state and the proportion of the data blocks they cover, thus forming a numerical value for the coverage degree. See Appendix. Figure 5 , attached Figure 5 This embodiment provides a schematic diagram of intra-group redundancy coding. The diagram visually illustrates the boundary of "local coverage" through the connection relationship between local groups and intra-group check blocks; see appendix. Figure 6 , attached Figure 6 This embodiment provides a schematic diagram of enhanced ENIC redundancy coding, illustrating the computational relationship and placement positions between the enhanced check block and the stripe data block, thereby providing a structural source for the "enhanced coverage" of the coverage mapping table. To further improve the sufficiency of the disclosure, in an optional embodiment, the enhanced check block can be obtained through dedicated points, which can be... The verification block is constructed using the difference quotient coefficient matrix and the basis function vector. Its core computational object can be expressed in the following form:

[0103] , ;

[0104] in D For data block sets, points For the set of interpolation points, g(x) This represents the basis function vector at the specific point; this representation allows for a traceable intermediate object for the source of the "enhanced coverage". In one possible implementation, to give the basis function object a clear structural representation and facilitate vectorized operations and cache layout by the implementer, the basis function vector can be given in a product recursive form:

[0105] ;

[0106] Furthermore, the basis function vectors at multiple interpolation points can be constructed into a set of column vectors:

[0107] ;

[0108] The above expression clarifies the composition structure of the "basis function vector / basis function matrix," enabling the encoding and recovery sides to verify the dimension, order, and reference consistency of basis function objects when sharing the same set of interpolation points. This further supports the engineering-based judgment of the completeness of the polynomial representation relationship by the first correlation consistency sub-state variable. By decomposing the state variable into two dimensions, "correlation consistency" and "redundancy coverage," the recovery side can simultaneously consider "whether the representation relationship is complete" and "whether the redundancy is sufficient to support reconstruction" under complex operating conditions, avoiding high-cost recovery triggered by a single readability rate or a single fault count. The technical effect is that the recovery path selection is more stable and the probability of errors entering the re-strategy is lower. From the perspective of creative contribution, this limitation enables a single state variable to have composite semantics and is associated with... Figure 5 Appendix Figure 6 The two types of redundant structures shown correspond directly to each other, which strengthens the technical organization method of "hybrid verification + state quantity driven" in this application.

[0109] In a preferred embodiment, the first data recovery support state quantity is obtained by weighted combination calculation of the first associated consistency sub-state quantity and the first redundancy coverage sub-state quantity, and output in the form of a numerical range to represent the level of data recovery support capability.

[0110] In one possible implementation, the first associated consistency sub-state quantity and the first redundancy coverage sub-state quantity are first normalized to a uniform scale, and then weighted and combined to form the first data recovery support state quantity. Normalization can be implemented using a linear proportional approach, for example, mapping the consistency completeness to 0 to 100 using "completeness item count / required item count," and mapping the coverage degree to 0 to 100 using "proportion of data blocks covered by available verification." To ensure reproducibility, the completeness item count can at least include items such as the existence of the interpolation point set, the continuity of the block index set, the existence of the verification block coverage mapping table, and the resolvability of key reference fields. The weighted combination calculation can take the following form:

[0111] , ;

[0112] in S To provide support for the first data recovery state quantity, This is the first associated consistent sub-state quantity. This is the first redundant covered sub-state variable. , This refers to weights. Weights are configurable parameters for the project, which can be distributed by the configuration center or loaded from a local configuration file. The update cycle can be set from 10 minutes to 24 hours to avoid frequent adjustments that could cause mode instability. In a common deployment, A value of 0.4 to 0.7 can be used to emphasize the fundamental role of relational integrity in recoverability. A value of 0.3 to 0.6 can be used to reflect the constraints of redundant coverage on the recovery path range. This status variable is output as a numerical range and persistently recorded in the recovery task context or stripe metadata. The recorded fields can include at least {S value, timestamp, statistical window identifier}, where the statistical window identifier indicates the historical statistical range or sampling range corresponding to the value, thus facilitating audit traceability. See Appendix Figure 2 In the metadata recording stage of the encoding side, the overlay mapping and distribution information written by the encoding side provides a directly usable data source for the above-mentioned normalization and weighting. In terms of technical effect, the numerical interval output enables the recovery side to complete the subsequent mapping and mode selection with constant-level comparison under high concurrency, reducing decision-making overhead and reducing the introduction of additional CPU contention when the load fluctuates; from the perspective of creative contribution, the structured link of "composite sub-state quantity - weighted combination - interval output" makes the state quantity have both multi-dimensional semantics and the single-value hub characteristics that can be engineered and implemented, which strengthens the irreplaceability of the "single intermediate representation quantity driven hierarchical strategy" in the architectural scheme of this application.

[0113] In a preferred embodiment, the value range of the first data recovery support status quantity is divided based on the statistical results of historical operating status, and includes at least one upper limit range and at least one lower limit range, used to distinguish different data recovery support capability levels.

[0114] In one possible implementation, the interval boundary of the first data recovery support state quantity is not a fixed constant, but rather formed based on historical operational status statistics, allowing the interval to adapt to changes in long-term operational conditions. Historical statistical samples can come from recovery task logs and inspection statistics, including at least the S-value distribution, average recovery task time, recovery failure rate, and checksum readability within each time bucket. The time bucket length can range from 5 minutes to 1 hour, and the statistical period can range from 24 hours to 14 days to cover the impact of daily load cycles, maintenance windows, and scaling events. The interval division can be implemented using quantiles: S The historical distribution of values ​​yields the low quantile boundary. With high quantile boundary And thereby forming at least one lower limit interval and at least one upper limit interval, for example, S ≤ Defined as the lower limit interval, S ≥ Defined as the upper limit interval; to ensure feasibility, quantiles can be common engineering choices such as 20% and 80% or 10% and 90%, and the boundaries should be written into a configuration table. The configuration table fields should at least include {lower boundary, upper boundary, effective time}. When the system detects insufficient statistical sample size, "minimum sample size protection" can be enabled, for example, requiring the boundary to be updated only after the sample size reaches 100 to 10,000, to avoid boundary drift caused by small samples. See appendix. Figure 3 In the decision-making stage prior to "mode determination" in the recovery-side process shown, the interval boundary is read as a recovery-side decision configuration and participates in subsequent mode mapping, thereby ensuring that the recovery side has consistent hierarchical classification rules under different historical states. Technically, after the interval boundary is formed based on historical statistics, the system can maintain the stability of mode switching during long-term operation, reducing frequent switching caused by short-term fluctuations, thus reducing the uncertain impact of recovery tasks on resource consumption. From a creative contribution perspective, this constraint establishes a clear binding between "interval" and "historical operational statistics," giving the state-driven hierarchical decision-making a traceable data basis and adaptive capability, unlike static classification methods that rely solely on empirical thresholds.

[0115] In a preferred embodiment, based on the first data recovery support state quantity being in different value ranges, different data failure handling modes are determined respectively, and different value ranges correspond to different ranges of data block failure situations.

[0116] In one possible implementation, after detecting unreadable data blocks within the stripe, the recovery side first counts the number of failed data blocks, *r*, and compares this number with the number of data blocks in the stripe, *k*, to form a comparable failure range representation. The count can be based on read failure counts, unreadable block counts due to node unreachability, or missing block counts marked by metadata. One to ten verification samples can be performed within 1 to 30 seconds to handle fault jitter. Subsequently, the recovery side reads the first data recovery support state quantity *S* and determines its value range. When *S* is in the upper limit range, the allowed failure range is mapped to a wider range, for example, allowing entry into a processing mode that covers moderate failures. When *S* is in the lower limit range, the allowed failure range is mapped to a narrower range, for example, only allowing entry into a mild failure processing mode. To avoid the ambiguity of "directly outputting the final conclusion from a single threshold," this implementation uses an interval expression for the "number range." For example, mild failure can be defined as *r*=1, and moderate failure as 2≤r≤... Severe failure is defined as r> In one possible implementation, in order to make the above... It has a definite and reproducible engineering meaning. The number of data blocks within a strip is denoted as k, and the local group size is fixed according to the parity relationship of the index grouping. When k is even, the two local group sizes are equal.

[0117] ;

[0118] When k is odd, the two local groups are of unequal size:

[0119] , ;

[0120] This quantitative relationship is used to fix the boundaries of local groups, ensuring that the upper bound of the number of failures corresponding to "moderate failure" maintains a consistent caliber with the maximum size of the local group, thereby facilitating the application of this approach in the context of local groups. Figure 5 Under the local group structure shown, a consistent judgment is made on the local recovery coverage area.

[0121] The mappings for "upper limit interval - moderate coverage" and "lower limit interval - light coverage" are then written into a mode mapping table. The mapping table fields can include at least {interval identifier, allowed failure range, and corresponding processing mode identifier}. See appendix. Figure 5 The local group structure shown is... This corresponds to the upper bound of the maximum group size of the local group, thus ensuring consistency between "moderate failure" and "local group recovery capability boundary" in engineering terminology; see appendix. Figure 3 The recovery side branch process, as shown, enters the corresponding mode and executes recovery when the failure range falls within the allowable range; otherwise, it maintains a conservative mode or delays execution to avoid uncontrollable resource preemption during peak load periods. Technically, the joint constraint of the interval and the range of failure quantities allows the recovery side not only to "know which mode to use" but also to "know the scale of failures that the mode is allowed to cover," thus more controllably limiting the size of the input set and the range of concurrent recovery when the failure mode changes. From a creative contribution perspective, this constraint establishes a mapping between a single state variable interval and the actual failure scale, giving mode selection an engineering-interpretable boundary and reducing the likelihood of the solution being replaced by a simplified implementation that "only considers the number of failures or only considers state variables."

[0122] In a preferred embodiment, when forming the first data recovery support state quantity, the historical storage data state information is grouped according to the operating conditions of the storage system, and the first data recovery support state quantity is formed based on the data distribution characteristics under the corresponding operating conditions.

[0123] In one possible implementation, the system uses the operating condition as the historical statistical grouping key, enabling the first data recovery support state quantity to have different statistical benchmarks under different load and topology conditions. The operating condition can be determined by observable indicators periodically collected by the control node. These indicators can include at least the cluster's average CPU utilization, average disk queue depth, network egress bandwidth usage, and node change event frequency. The sampling period can be 1 to 60 seconds, and the operating condition refresh period can be 10 to 10 minutes. To avoid state quantity jitter caused by operating condition fluctuations, a hysteresis mechanism can be used for operating condition switching: a new operating condition needs to meet 2 to 10 consecutive sampling periods to take effect, and the retention time after the operating condition takes effect can be set to 30 seconds to 30 minutes. Historical stored data state information is grouped and stored according to {operating condition label, time bucket}, with a time bucket length of 5 minutes to 1 hour. When the first data recovery support state quantity is formed, the system first reads the historical statistical distribution corresponding to the current operating condition label to calculate the completeness baseline of the consistent sub-state quantity and the availability baseline of the covering sub-state quantity, and synthesizes it with the current distribution state information. See Appendix. Figure 2 During the coding phase shown, the placement distribution of data blocks and check blocks may differ under different operating conditions. Operating condition grouping can make the state variables more closely resemble the historical availability performance of that placement distribution; see appendix. Figure 3 As shown in the recovery-side execution phase, the state variables grouped by operating conditions allow for a more conservative mode selection under high load and a stronger recovery under low load, thereby mitigating the impact of recovery tasks on business load in engineering operations. The technical benefits are: the same redundant structure has different impacts on resource consumption under different operating conditions; grouping state variables by operating conditions reduces the probability of "falsely triggering high-cost recovery during high load periods"; from a creative contribution perspective, this constraint establishes a structured relationship between "state variable formation" and "operating conditions," making the state variables no longer static indicators but intermediate representations adaptable to dynamic operating conditions, enhancing the irreplaceability of the solution under complex and changing conditions.

[0124] In a preferred embodiment, before performing the operation of determining the data failure handling mode based on the first data recovery support state quantity, a consistency judgment is made on the first data recovery support state quantity formed in multiple consecutive time periods, and the corresponding data failure handling mode is determined only when the consistency meets the preset conditions.

[0125] In one possible implementation, the recovery side sets a consistency gate before the output processing mode to perform consistency judgment on the state quantity sequence formed within multiple consecutive time periods, thereby suppressing frequent mode switching caused by short-term jitter. The time period can be defined as a fixed time, such as a state quantity being formed once every 1 to 30 seconds, or it can be defined according to the recovery trigger event, i.e., a state quantity is formed once for each recovery task; the window length for consistency judgment can be set to 3 to 20 periods. The preset conditions can be implemented as a directly achievable counting comparison, such as requiring at least [number missing] within the window. Samples must fall within the same interval (where L is the window length), or the number of interval switching within the window must not exceed 0 to 2 times; this judgment only involves counting and comparison, avoiding the introduction of black-box computation. If consistency is not met, the recovery side maintains the previously determined mode or maintains a conservative mode, and the maintenance duration can be set from 10 seconds to 10 minutes to reduce jitter. See Appendix Figure 3 The recovery process shown enhances the temporal stability of the mode output by adding a consistency gate between "interval mapping" and "mode output." This prevents frequent reconfiguration of concurrency and rate limiting strategies during the recovery execution phase due to short-term fluctuations, thereby reducing sudden fluctuations in resource consumption. Technically, this consistency mechanism provides inertia and stability for mode switching, especially in scenarios involving node failure jitter, network outages, or load spikes, preventing additional reads and recalculations caused by switching recovery strategies back and forth between local and global scenarios. From a creative contribution perspective, this constraint further engineers "state-driven hierarchical selection" into an implementable control structure for "switching after stabilization," strengthening the technical organization of this application regarding the controllability of resource consumption.

[0126] In a preferred embodiment, under different data failure handling modes, the length of the data window or the range of historical data references involved in forming the first data recovery support state quantity are dynamically adjusted.

[0127] In one implementation, the system associates the "state variable formation caliber" with the "current mode," making the state variables in different modes have different sensitivities to historical and current data, thus balancing response speed and stability in engineering. The data window length can simultaneously support two calibers: a window N based on the number of objects and a window T based on the time length. For example, in conservative mode, to more quickly reflect availability degradation, the window can be set to N=100N to 2000 or T=1 minute to 20 minutes, making the state variables more sensitive to short-term declines. In enhanced mode, to smooth short-term fluctuations, the window can be set to N=2000 to 20000 or T=20 minutes to 6 hours, making the state variables more stable. The historical data reference range can be adjusted by the number of time buckets or by the range of operating conditions. For example, in conservative mode, only the most recent time bucket or the most recent statistics under the current operating condition are referenced; in enhanced mode, multiple time buckets are referenced and a weighted average is performed. The weighted average does not require a complex algorithm and can be implemented using a linear weight sequence with higher weights for closer data. See Appendix. Figure 2 As shown in the encoding-side data writing and metadata recording phases, the shorter the window, the more the state variables reflect the latest write distribution and the latest readable state; see appendix. Figure 3 As shown in the recovery mode selection phase, after the window and reference range are dynamically adjusted, the mapping of state variable intervals and consistency judgments are more likely to form a stable closed loop, avoiding the situation where "fixed state variable caliber leads to mode instability" under the same load conditions. Technically, this dynamic adjustment enables state variables to have matched statistical stability under different recovery modes, thereby reducing erroneous switching and repeated recovery caused by mode and caliber mismatch. From a creative contribution perspective, this limitation also incorporates the formation process of a "single intermediate representation quantity" into the hierarchical strategy closed loop, making the architectural solution not just "selecting a strategy using state variables," but rather "adjusting the state variable formation caliber according to the strategy," enhancing the systematicity and irreplaceability of the overall technical organization.

[0128] In a preferred embodiment, when a change in the number of storage nodes or a change in the data block distribution structure is detected, a recalculation of the first data recovery support state quantity is triggered.

[0129] In one implementation, the control node maintains a node topology table and listens for topology change events. The topology table records at least the node ID, online / offline time, health status, and fault domain identifier. When a change in the number of nodes is detected to exceed 5% to 20%, the status variables are recalculated. Changes in the data block distribution structure can be detected through placement policy update events, rebalancing task start events, or cross-domain migration ratio change events. For example, a recalculation is triggered when the proportion of stripes affected by rebalancing exceeds 1% to 10%, or when the cross-domain read ratio rises above a preset tolerance range in the past 5 minutes. After triggering, the system can choose between full recalculation or incremental recalculation: full recalculation is suitable for scenarios with extremely large changes or small cluster sizes, while incremental recalculation is suitable for large-scale cluster scenarios where only some fault domains are affected. During incremental recalculation, the metadata service uses an index to locate the affected stripe set and only refreshes the historical statistics and current distribution mapping for that set, thereby controlling the recalculation overhead. See Appendix Figure 2 With appendix Figure 3 The encoding-recovery closed loop embodied here means that topology and distribution changes directly alter the input of "current distribution state information" and the statistical basis of "redundancy coverage availability." If recalculation is not triggered, the recovery side may enter an inappropriate mode under expired state variables, leading to uncontrollable resource consumption. After recalculation is triggered, state variables and interval boundaries can be aligned with the latest distribution structure in a timely manner. The technical effect is that during scaling up and down, maintenance windows, and large-scale migrations, the recovery path selection is more in line with the current support capabilities, thereby reducing the impact of recovery tasks on the stable operation of the system. From the perspective of creative contribution, this limitation incorporates "dynamic topology / dynamic distribution" into the lifecycle management of state variables, enabling the solution to cover more complex changing conditions and reducing the substitution interpretation that is only applicable to static topology environments.

[0130] In a preferred embodiment, the data recovery result includes the data block group identifier to which the failed data block belongs, which is used to characterize the logical range of the failure location in the distributed storage system.

[0131] In one possible implementation, the recovery result, in addition to including "recovery successful / failed," also outputs the data block group identifier to which the failed data block belongs, providing traceable logical range location. The data block group identifier can be formed during the stripe organization stage on the encoding side. The group identifier can include the object ID, stripe ID, and index range within the group, or it can include the object ID, local group ID, and a summary of the block index set. On the encoding side, this group identifier is written into the metadata and associated with the data block and checksum block mapping table. See Appendix. Figure 5 The local group structure shown can be directly used as part of the group identifier to characterize whether the failure occurs in an even or odd number of elements and their logical boundaries; see appendix. Figure 4The data block sequence shown can be further characterized by the block index range, which can be used to represent the location segment of the failed block in the original data. After the recovery execution is completed, the recovery module output record can include at least {task identifier, failed block identifier, group identifier, target node identifier, timestamp}, and write this record to the task table or log system. To enhance feasibility, the system can establish a reverse index of "group identifier → block set" in the metadata service, enabling the subsequent scheduler to quickly locate the relevant block set when duplicate failures occur or when it is necessary to limit concurrent recovery within the same group. In terms of technical effect, the group identifier output gives the recovery result location semantics, which facilitates subsequent scheduling control in complex failure modes, such as limiting simultaneous triggering of global recovery within the same group or prioritizing the recovery of certain key logical ranges, thereby improving the controllability and manageability of the recovery process in long-term operation. From the perspective of creative contribution, this limitation expands the recovery output from "result status" to "result + logical range", making the recovery decision link and the subsequent scheduling link form a closed loop, reducing the possibility of the solution being simplified to an alternative implementation that only outputs success / failure.

[0132] In a further embodiment of this second example, preferably, an enhanced verification scheme is also included, which does not change the core idea of ​​the independent scheme. For example, in terms of enhanced verification, multiple dedicated points can be optionally set. ， ，…， Multiple enhanced check blocks are generated to improve the availability of coverage redundancy under multi-point failure conditions; in this case, the check block corresponding to each dedicated point is still available. This expression allows implementers to extend the generation path according to the same representation relationship. For example, in terms of distributed placement, the check blocks can be optionally placed asymmetrically based on the node's historical failure rate and the failure domain topology, making the covering sub-state variable more stable in high-risk domains; this is an optional implementation of the placement layer strategy and does not change the main link of "state variable—interval—mode". The above further implementation methods are embedded into the implementation scenarios corresponding to paragraphs such as "forming the covering sub-state variable" and "topology change triggering recalculation," thereby avoiding the stacking of unincluded content in a single location and maintaining the semantic closure of each paragraph.

[0133] Example 3

[0134] Building upon the aforementioned method embodiments, this embodiment provides a specific implementation of an error-correcting storage system based on polynomial interpolation hybrid verification. The system, deployed as a device or product in a distributed storage environment, provides stable decision-making support and a controllable execution path for the data recovery process after failure, even under continuously changing node scale, failure modes, and operating load. The system is structurally organized around a single technical hub—a first data recovery support state variable—and completes state variable formation, processing mode determination, data recovery execution, and result output through modular collaboration, thereby reducing the uncertainty of system resource consumption during the recovery process in engineering operations. Figure 7 A structural block diagram of wide-area flow monitoring based on ultrasonic signal analysis is provided as an example in this embodiment, such as... Figure 7 As shown, it includes:

[0135] The data acquisition and segmentation module 100 is used to acquire the original data to be stored in the distributed storage system and divide the original data into several data blocks;

[0136] The state quantity forming module 200, connected to the data acquisition and segmentation module 100, is used to calculate and form a first data recovery support state quantity based on the data block. The first data recovery support state quantity is a composite technical quantity used to characterize the association consistency state and redundancy coverage state of the data block under the polynomial representation relationship. The state quantity forming module 200 constructs a unified polynomial representation relationship based on the quantity relationship of the data blocks and the generation relationship of the corresponding verification data, and calculates the correspondence integrity between the data block and the verification data block based on the polynomial representation relationship.

[0137] The processing mode determination module 300 is connected to the state quantity forming module 200 and is used to determine the data failure processing mode based on the value range of the first data recovery support state quantity. When the first data recovery support state quantity is in the first value range, it is determined to be a data recovery strategy based on local correlation. When the first data recovery support state quantity is in the second value range, it is determined to be a data recovery strategy based on global correlation.

[0138] The data recovery execution module 400 is connected to the processing mode determination module 300 and is used to select the corresponding data recovery strategy according to the determined data failure processing mode, and perform data recovery processing based on the polynomial representation relationship.

[0139] The result output module 500 is connected to the data recovery execution module 400 and is used to output the data recovery result.

[0140] In an implementable system architecture, the error correction storage system includes at least a data acquisition and segmentation module 100, a state quantity generation module 200, a processing mode determination module 300, a data recovery execution module 400, and a result output module 500. Each module can be deployed on the same control node, or distributed across multiple management nodes or storage nodes according to their functions, and connected via at least one communication method. The communication method may include in-process function calls, shared memory, message queues, or remote procedure calls; when using remote procedure calls, the interface timeout can be set to 100ms to 2s, and the number of retries can be set to 1 to 3 times to avoid excessive system overhead when a node malfunctions.

[0141] The data acquisition and segmentation module 100 is used to acquire the raw data to be stored in the distributed storage system and divide the raw data into several data blocks. In one specific implementation, the module can be deployed on an access node or gateway node on the write path to receive write requests from clients or upper-layer applications. The module can internally set up a memory buffer to temporarily store input data and segment the raw data according to a preset block size, which can be selected between 64MB and 128MB to adapt to common engineering conditions such as sequential disk writes and network transmissions. After segmentation, the module assigns basic metadata such as block index and stripe identifier to each data block, writes the data block to the target storage node, and outputs the segmentation results and distribution information to the status variable formation module 200. The specific segmentation and writing process of this module can be referred to the relevant descriptions of the method steps in the foregoing embodiments, and will not be repeated here.

[0142] The state quantity forming module 200 is connected to the data acquisition and segmentation module 100 and is used to calculate and form a first data recovery support state quantity based on the data block. The state quantity forming module 200 can run as an independent service process or management component at the system level and maintain communication with the metadata service and the background inspection component. This module is used to collect historical data block storage state information and current data block distribution state information, and construct a unified polynomial representation relationship based on the data block quantity relationship and the corresponding verification data generation relationship, thereby calculating the correspondence integrity between data blocks and verification data blocks to form a first data recovery support state quantity used to characterize the associated consistency state and redundant coverage state. The specific calculation method, sub-state quantity composition, weighted combination, and interval output form of the state quantity can all be implemented with reference to the relevant descriptions in the foregoing embodiments, and will not be elaborated further in this embodiment three.

[0143] In engineering implementation, the state quantity formation module 200 can run periodically or when a trigger event occurs, such as when writing is completed, inspection updates, node topology changes, or a rebalancing task is initiated, triggering recalculation. The module can write the formed first data recovery supporting state quantity and its corresponding timestamp, statistical window identifier, and operating condition label into the state storage area or metadata service for subsequent modules to read.

[0144] The processing mode determination module 300 is connected to the state quantity forming module 200 and is used to determine the data failure processing mode based on the value range of the first data recovery support state quantity. In system implementation, the processing mode determination module 300 can operate as a control logic component. Its input is the numerical range result output by the state quantity forming module 200, and its output is a mode identifier indicating the data failure processing path. In one implementation, the module maintains a set of interval boundary configurations to map the state quantity to at least two processing modes. When the first data recovery support state quantity is in a first value range, the module outputs a data recovery strategy identifier based on local correlation; when the first data recovery support state quantity is in a second value range, the module outputs a data recovery strategy identifier based on global correlation. To improve the stability of the mode output, the module can also perform consistency judgment on the state quantity results over multiple consecutive time periods, updating the output mode only when a preset consistency condition is met. The interval mapping and consistency judgment logic of this module can be implemented with reference to the relevant descriptions in the foregoing embodiments, and will not be repeated here.

[0145] The data recovery execution module 400 is connected to the processing mode determination module 300 and is used to select the corresponding data recovery strategy according to the determined data failure processing mode, and perform data recovery processing based on the polynomial representation relationship. The data recovery execution module 400 can be deployed on the control node or storage node and maintains communication with the metadata service and storage node process. In the system implementation, when the module detects that a data block is unreadable or invalid, it first locates the input set participating in the recovery based on the metadata, including available data blocks and verification data blocks; then, according to the mode identifier output by the processing mode determination module 300, it selects the corresponding data recovery path and performs recovery calculation. The specific data recovery calculation process includes local recovery, enhanced verification participation, or global recovery, etc., which can be referred to the detailed description of the method steps in the foregoing embodiments, and will not be repeated here. To control the impact of the recovery process on system resources, the data recovery execution module 400 can set different concurrency and rate limiting parameters according to different processing modes, thereby limiting the occupation of disk and network resources by the recovery task under high load conditions.

[0146] The result output module 500 is connected to the data recovery execution module 400 and is used to output the data recovery result. In one specific implementation, the result output module 500 encapsulates the recovery execution result into a structured record and outputs it to a metadata service, scheduling system, or log system. The recovery result may include at least a recovery success flag, a failed data block identifier, and a rebuilt node identifier, and may further include the data block group identifier to which the failed data block belongs, to characterize the logical range of the failure location in the distributed storage system. By outputting recovery results containing logical range information, the system can provide a basis for subsequent scheduling control, statistical analysis, or auditing.

[0147] In a further embodiment, the system may also include a triggering unit for monitoring changes in node topology or data block distribution structure. When a change in the number of nodes or a change in the distribution structure is detected, the system triggers a recalculation of the first data recovery support state quantity. This triggering mechanism can be used to ensure that the state quantity remains consistent with the current system structure, thereby avoiding inappropriate recovery decisions based on outdated state quantities during topology changes.

[0148] Through the aforementioned system structure and module collaboration method, this embodiment three implements an error-correcting storage function based on polynomial interpolation hybrid verification in the form of a device or product. The connections between the various modules of the system are clear, the functional boundaries are well-defined, and all relevant technical features have feasible implementation methods. Based on this, those skilled in the art can complete the implementation and deployment of the system, and complete the recovery and result output after data failure in a distributed storage environment.

[0149] The above description is merely a preferred embodiment of this application and does not limit the patent scope of this invention. Any equivalent structural or procedural transformations made based on the description and drawings of this invention, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this invention.

Claims

1. A method for error correction and storage based on polynomial interpolation hybrid verification, characterized in that, include: Obtain the original data to be stored in the distributed storage system, and divide the original data into several data blocks; Based on the data block, a first data recovery support state quantity is calculated and formed. The first data recovery support state quantity is a composite technical quantity used to characterize the association consistency state and redundancy coverage state of the data block under the polynomial representation relationship. The first data recovery support state quantity is constructed based on the quantitative relationship of the data blocks and the generation relationship of the corresponding verification data to form a unified polynomial representation relationship, and the correspondence integrity between the data block and the verification data block is calculated based on the polynomial representation relationship. Based on the value range of the first data recovery support state quantity, a data failure handling mode is determined. When the first data recovery support state quantity is in the first value range, a data recovery strategy based on local correlation is determined. When the first data recovery support state quantity is in the second value range, a data recovery strategy based on global correlation is determined. Based on the determined data failure handling mode, select the corresponding data recovery strategy and perform data recovery processing based on the polynomial representation relationship; Output the data recovery results.

2. The method according to claim 1, characterized in that, The first data recovery support status quantity is formed based on historical data block storage status information and current data block distribution status information. The historical data block storage status information includes historical data block integrity status and historical verification data availability status, and the current data block distribution status information includes the distribution relationship between data blocks and verification data blocks in the storage nodes.

3. The method according to claim 2, characterized in that, The first data recovery support state quantity consists of at least two sub-state quantities, including a first correlation consistency sub-state quantity and a first redundancy coverage sub-state quantity. The first correlation consistency sub-state quantity is used to characterize the completeness of the polynomial representation relationship between data blocks, and the first redundancy coverage sub-state quantity is used to characterize the coverage range of the verification data block on the data block.

4. The method according to claim 3, characterized in that, The first data recovery support state quantity is obtained by weighted combination calculation of the first correlation consistency sub-state quantity and the first redundancy coverage sub-state quantity, and output in the form of a numerical range to represent the level of data recovery support capability.

5. The method according to claim 4, characterized in that, The value range of the first data recovery support status quantity is divided based on the statistical results of historical operation status, and includes at least one upper limit range and at least one lower limit range, which are used to distinguish different data recovery support capability levels.

6. The method according to claim 1, characterized in that, Based on the fact that the first data recovery support status quantity is in different value ranges, different data failure handling modes are determined respectively, and different value ranges correspond to different ranges of data block failure situations.

7. The method according to claim 1, characterized in that, Before performing the operation of determining the data failure handling mode based on the first data recovery support state quantity, a consistency judgment is made on the first data recovery support state quantity formed in multiple consecutive time periods. Only when the consistency meets the preset conditions is the corresponding data failure handling mode determined.

8. The method according to claim 1, characterized in that, Under different data failure handling modes, the length of the data window or the range of historical data references that participate in the formation of the first data recovery support state quantity are dynamically adjusted.

9. The method according to claim 1, characterized in that, The data recovery result includes the data block group identifier to which the failed data block belongs, which is used to characterize the logical range of the failure location in the distributed storage system.

10. An error-correcting storage system based on polynomial interpolation hybrid verification, characterized in that, include: The data acquisition and segmentation module is used to acquire the raw data to be stored in the distributed storage system and divide the raw data into several data blocks; A state quantity forming module, connected to the data acquisition and segmentation module, is used to calculate and form a first data recovery support state quantity based on the data block. The first data recovery support state quantity is a composite technical quantity used to characterize the association consistency state and redundancy coverage state of the data block under the polynomial representation relationship. The state quantity forming module constructs a unified polynomial representation relationship based on the quantitative relationship of the data blocks and the generation relationship of the corresponding verification data, and calculates the correspondence integrity between the data block and the verification data block based on the polynomial representation relationship. The processing mode determination module, connected to the state quantity forming module, is used to determine the data failure processing mode based on the value range of the first data recovery support state quantity. When the first data recovery support state quantity is in the first value range, it is determined to be a data recovery strategy based on local correlation. When the first data recovery support state quantity is in the second value range, it is determined to be a data recovery strategy based on global correlation. The data recovery execution module, connected to the processing mode determination module, is used to select the corresponding data recovery strategy according to the determined data failure processing mode, and to perform data recovery processing based on the polynomial representation relationship. The result output module is connected to the data recovery execution module and is used to output the data recovery results.

Images