Cloud storage data dynamic hierarchical self-healing method and system

Abstract

The invention relates to the technical field of cloud storage data, in particular to a dynamic hierarchical self-healing method and system for cloud storage data, and the system comprises a data collection and access module, a data preprocessing and feature engineering module, an intelligent prediction and decision module, a dynamic hierarchical storage execution module, and a fault self-healing and load balancing module. According to the method, the health state of the storage system is continuously monitored through the Kafka real-time data stream, the data access trend is predicted in combination with the LSTM model, and potential faults are warned in advance; the node state is judged through indexes such as TCP timeout and heartbeat loss rate, redistribution or storage hierarchy adjustment of affected data is triggered, fault non-inductive repair is achieved through a self-healing mechanism, and service continuity is guaranteed; through deep linkage of hierarchical storage and a fault recovery mechanism, the storage hierarchy of the data is actively adjusted according to the change of the data integrity in the recovery process, it is ensured that the optimal storage scheduling can be obtained while the data is recovered, and the self-healing capacity of the system is improved.

Description

Technical Field

[0001] This invention relates to the field of cloud storage data technology, specifically to a dynamic hierarchical self-healing method and system for cloud storage data. Background Technology

[0002] In today's information age, cloud computing and cloud storage technologies have been widely applied, providing users with convenient data storage and access services. However, traditional cloud storage systems face numerous challenges in data storage and management, especially with massive and highly volatile data volumes. With the rapid development of cloud computing technology, the demand for cloud storage systems is increasing, and users are placing higher demands on the reliability and cost-effectiveness of storage services. In current cloud storage technologies, dynamic hierarchical storage has become an efficient management tool, automatically adjusting data storage levels based on data importance and access frequency to maximize resource utilization. However, in practical applications, it has been found that cloud storage technology cannot self-heal in the event of data corruption or loss, and it cannot automatically trigger adjustments to storage levels, severely limiting the effectiveness and efficiency of data recovery. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for dynamic hierarchical self-healing of cloud storage data, so as to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a cloud storage data dynamic hierarchical self-healing method, characterized by comprising the following steps:

[0005] S1. Obtain historical data access records from the cloud storage service system through the data acquisition and access module, including data identifier, access timestamp, access type, data size, and user identifier;

[0006] S2, the data preprocessing and feature engineering module uses Python to clean and standardize the collected data, and normalize the time series data;

[0007] S3, the data preprocessing and feature engineering module extracts key features from the processed data and transforms the categorical data through one-hot encoding to prepare structured input for model training;

[0008] S4. The intelligent prediction and decision-making module trains the machine learning prediction model LSTM based on preprocessed historical data.

[0009] S5. Deploy the trained model to the cloud storage service system. The dynamic hierarchical storage execution module continuously monitors the data through Kafka real-time data stream.

[0010] S6, the dynamic hierarchical storage execution module predicts data access trends through the LSTM model and dynamically allocates data to the SSD hot storage, HDD warm storage, and tape cold storage levels through the Sealos distributed storage system, combining the importance of stored data, access frequency, and storage cost.

[0011] S7, the fault self-healing and load balancing module uses erasure coding technology combined with load balancing algorithms to automatically switch services to healthy nodes and redistribute the load when a node fails.

[0012] The load balancing algorithm uses the FRA-M algorithm to achieve rapid switching of failed nodes and optimized resource allocation through metadata backup and load balancing among management nodes.

[0013] S71. The data block is stored as K+F fragments after being encoded by erasure coding.

[0014] Where K is the number of original data fragments, and F is the number of redundant fragments;

[0015] S72. Monitor shard access requests through the Kafka real-time stream processing framework, record the number of accesses per unit time, and calculate access popularity;

[0016] Node-level access frequency This represents the total access frequency of all contained fragments.

[0017] Erasure coding (K+F sharding) is combined with multi-replica redundancy, enabling second-level takeover in case of failure through log replication, and data recovery conditions satisfy K≤number of available shards≤K+F;

[0018] Metadata is synchronized in real time between nodes via message passing, ensuring that backup nodes can take over immediately in the event of a primary node failure.

[0019] S73. Based on distributed consensus protocols such as Raft, metadata changes are implemented through log replication; node status is monitored through a heartbeat mechanism and TCP timeout, and the fault detection period is dynamically associated with a timeout threshold, with the timeout threshold Ttimeout set as follows:

[0020]

[0021] in, The average response time, Standard deviation, For correction coefficients, and ;

[0022] Ensure the accuracy and timeliness of fault detection;

[0023] S74. Assign weights Wi based on the real-time load status of nodes:

[0024]

[0025] in, The normalization coefficient is... For CPU utilization, For memory usage, For network bandwidth, The data integrity score for node i;

[0026] In the weighting formula Adding data integrity The system performs checks to link timeout thresholds and data corruption detection; it can respond quickly to data corruption or node failures and automatically optimize storage tiers and load distribution.

[0027] S75. Requests are allocated according to weight ratios. In weighted round-robin, the number of requests Ni for the i-th node satisfies:

[0028]

[0029] in, Let be the proportion of node i's weight to the total weight of all nodes. The total number of requests to be allocated;

[0030] S76. Dynamically adjust storage levels and data access frequency by combining the machine learning prediction model LSTM. With storage tier Mapping relationship:

[0031]

[0032] Determine node status using TCP timeout and heartbeat loss rate metrics:

[0033]

[0034] in, For fault detection cycle, This is the timeout threshold. The average response time, This is the network latency fluctuation coefficient;

[0035] S77, By optimizing metadata synchronization time and metadata synchronization time Reduce total switchover time:

[0036]

[0037] in, For failover time, For detection time;

[0038] S78, By controlling the fault detection cycle Timeout threshold and load balancing weight To maintain algorithm performance within a stable range; system stability metrics The formula is:

[0039]

[0040] in, Mean time between failures (MTBF) The average recovery time, These are dynamic weighting coefficients;

[0041] By adjusting the parameters maximize;

[0042] S79, Data Reliability R employs multi-replica and RAID redundancy technologies, combined with cross-regional data synchronization:

[0043]

[0044] in, For the number of nodes, Number of copies;

[0045] By storing data replicas on distributed nodes, we ensure that the failure of a single node does not affect data availability; in the formula, increasing the number of replicas m can exponentially improve reliability.

[0046] S8. The fault self-healing and load balancing module automatically triggers recovery and hierarchical adjustment when data is corrupted;

[0047] S81. When data corruption is detected, the dynamic hierarchical self-healing system automatically initiates the recovery process: using erasure coding technology, the corrupted data is reconstructed from available shards or replicas; during the recovery process, data consistency is ensured through the distributed consistency protocol Raft.

[0048] S82. After recovery, reassess the data access trends and importance: Call the LSTM prediction model, combine historical access records and data importance labels, and recalculate the data access frequency and storage level; according to the mapping relationship in S76, dynamically adjust the data to SSD hot storage, HDD warm storage or tape cold storage.

[0049] S83. Automatically update metadata and load balancing weights: Recalculate node weights using the FRA-M algorithm based on the recovered data status. And synchronize metadata to ensure that load balancing and failover mechanisms take effect in a timely manner;

[0050] S84. Record self-healing events and optimize the model: Record data corruption and recovery events in the historical log for retraining and parameter calibration of the LSTM model, by dynamically adjusting the timeout threshold. and weighting coefficients This will improve the accuracy of future predictions and self-healing.

[0051] Furthermore, in S4, training the machine learning prediction model LSTM specifically involves the following steps:

[0052] S41 and LSTM models are used to process time series data and are suitable for fault prediction scenarios.

[0053] S42. Divide the dataset into training and test sets in an 8:2 ratio, optimize the parameters using gradient descent, and improve generalization ability through cross-validation.

[0054] S43. After training, verify the model performance to ensure that the prediction error is within an acceptable range.

[0055] A cloud storage data dynamic hierarchical self-healing system includes: a data acquisition and access module, a data preprocessing and feature engineering module, an intelligent prediction and decision-making module, a dynamic hierarchical storage execution module, and a fault self-healing and load balancing module, wherein:

[0056] M1, Data Acquisition and Access Module: Responsible for acquiring data through multiple heterogeneous sources such as sensors, log files, and API interfaces, achieving full lifecycle coverage of thermal, warm, and cold data, and providing a complete raw data foundation for subsequent classification and self-healing;

[0057] M2, Data Preprocessing and Feature Engineering Module: Based on Python's Pandas and other data processing libraries, it cleans, standardizes, and extracts features from the raw data; it also normalizes time series data and transforms categorical data through techniques such as one-hot encoding, ultimately generating a structured and normalized dataset for model training;

[0058] M3, Intelligent Prediction and Decision Module: This is the core driving module, responsible for training an LSTM machine learning model using preprocessed historical data. This model can accurately predict future data access trends and potential system failure risks, providing intelligent decision-making basis for dynamic hierarchical storage and proactive early warning.

[0059] M4, Dynamic Tiered Storage Execution Module: This is the execution layer for the tiered strategy, deployed in the cloud storage service system. This module continuously senses the system status and data access patterns by accessing Kafka's real-time data stream, and receives the prediction results from the data preprocessing and feature engineering modules. It automatically and dynamically allocates data to the storage tier with the best performance by comprehensively considering three dimensions: data importance, real-time access frequency, and storage cost.

[0060] M5, Fault Self-Healing and Load Balancing Module: This module serves as the system's resilience layer, integrating erasure coding technology and the FRA-M intelligent load balancing algorithm. Through real-time heartbeat detection and metadata synchronization, this module can achieve service switching within seconds, rapid data reconstruction, and immediate optimized load allocation when a node failure is detected.

[0061] Compared with existing technologies, the beneficial effects of this invention are as follows: By analyzing data access frequency, importance, and storage cost using an LSTM time-series prediction model, data is automatically allocated to SSD (hot storage), HDD (warm storage), and tape (cold storage) layers. High-frequency access data is preferentially stored in the SSD layer, while low-frequency data is archived in the tape layer, achieving precise matching between storage resources and data characteristics. The invention employs erasure coding (K+F fragmentation redundancy) combined with the FRA-M load balancing algorithm. When a node fails, real-time metadata synchronization and log replication mechanisms enable second-level takeover of the failed node. The LSTM model continuously monitors data access trends and dynamically adjusts the data based on access frequency and importance. At the storage tier, maintain efficient system operation; continuously monitor the health status of the storage system through real-time Kafka data streams, and predict data access trends using LSTM models to provide early warnings of potential faults; use heartbeat mechanisms and TCP timeouts to detect node status, and dynamically associate timeout thresholds with fault detection cycles to ensure the accuracy and timeliness of fault detection; when a significant change in data access patterns is detected (such as a sudden drop in high-frequency data access volume) or a node failure, immediately initiate a tiered adjustment process: determine the node status through indicators such as TCP timeout and heartbeat loss rate, trigger the redistribution of affected data or storage tier adjustments, and achieve seamless fault repair through a self-healing mechanism to ensure business continuity;

[0062] By deeply integrating tiered storage with fault recovery mechanisms, the storage tier of data is proactively adjusted based on changes in data integrity during the recovery process. This ensures that data receives optimal storage scheduling while being recovered, enhancing the system's self-healing capabilities and improving the flexibility and reliability of data management. Attached Figure Description

[0063] Figure 1 This is a flowchart of the method of the present invention;

[0064] Figure 2 This is a flowchart of the automatic trigger recovery and hierarchical adjustment process of the present invention. Detailed Implementation

[0065] The technical solution of the present invention will now be described in detail through specific embodiments.

[0066] Please see Figure 1-2 This invention provides a method for dynamic hierarchical self-healing of cloud storage data, comprising the following steps:

[0067] S1. Data is acquired from multiple sources, including sensors, log files, and API interfaces, covering the entire lifecycle of thermal, warm, and cold data; achieving a balance between the breadth and depth of data acquisition, and providing a complete data foundation for subsequent graded self-healing.

[0068] S2, the data preprocessing and feature engineering module, uses Python's Pandas library to clean and standardize the collected data, handle missing values, outliers, and duplicate data, and correct format errors to ensure data consistency, integrity, and accuracy.

[0069] S3, the data preprocessing and feature engineering module extracts the mean, variance, and peak value from the processed data, and transforms the categorical data through one-hot encoding to prepare structured input for model training;

[0070] S4. The intelligent prediction and decision-making module trains the machine learning prediction model LSTM based on preprocessed historical data, specifically through the following steps:

[0071] S41 and LSTM models are used to process time series data and are suitable for fault prediction scenarios.

[0072] S42. Divide the dataset into training and test sets in an 8:2 ratio, optimize the parameters using gradient descent, and improve generalization ability through cross-validation.

[0073] S43. After training, verify the model performance and ensure that the prediction error is within an acceptable range;

[0074] S5. Deploy the trained model to the cloud storage service system. The dynamic hierarchical storage execution module continuously monitors it through real-time data streams from Kafka. By continuously monitoring the health status of the storage system, it provides early warnings of potential failures.

[0075] S6, the dynamic hierarchical storage execution module predicts data access trends through the LSTM model and dynamically allocates data to the SSD hot storage, HDD warm storage, and tape cold storage levels through the Sealos distributed storage system, combining the importance of stored data, access frequency, and storage cost.

[0076] By dynamically analyzing three dimensions of data access frequency, importance, and storage cost through the LSTM time series prediction model, automatic hierarchical storage of hot, warm, and cold data is achieved; the SSD hot storage layer carries high-frequency access data, the HDD warm storage layer stores medium-frequency data, and the magnetic tape cold storage layer archives low-frequency data, thereby improving the utilization of storage resources.

[0077] By automatically classifying and storing data according to its access frequency and importance level, it is possible to effectively reduce storage costs and improve the efficiency and cost-effectiveness of data storage while ensuring data access efficiency.

[0078] S7, the fault self-healing and load balancing module uses erasure coding technology combined with load balancing algorithms to automatically switch services to healthy nodes and redistribute the load when a node fails.

[0079] The load balancing algorithm uses the FRA-M algorithm to achieve rapid switching of failed nodes and optimized resource allocation through metadata backup and load balancing among management nodes.

[0080] S71. The data block is stored as K+F fragments after being encoded by erasure coding.

[0081] Where K is the number of original data fragments, and F is the number of redundant fragments;

[0082] Data is fragmented into K+F redundant blocks using erasure coding technology, and combined with the FRA-M load balancing algorithm to achieve second-level takeover of node failures; the fault recovery time is shortened to the second level.

[0083] S72. Monitor shard access requests through the Kafka real-time stream processing framework, record the number of accesses per unit time, and calculate access popularity;

[0084] Node-level access frequency This represents the total access frequency of all contained fragments.

[0085] Erasure coding (K+F sharding) is combined with multi-replica redundancy, enabling second-level takeover in case of failure through log replication, and data recovery conditions satisfy K≤number of available shards≤K+F;

[0086] Metadata is synchronized in real time between nodes via message passing, ensuring that backup nodes can take over immediately in the event of a primary node failure.

[0087] S73. Based on distributed consensus protocols such as Raft, metadata changes are implemented through log replication; node status is monitored through a heartbeat mechanism and TCP timeout, and the fault detection period is dynamically associated with a timeout threshold, with the timeout threshold Ttimeout set as follows:

[0088]

[0089] in, The average response time, Standard deviation, For correction coefficients, and ;

[0090] Ensure the accuracy and timeliness of fault detection;

[0091] By proactively issuing early warnings and using intelligent parameter tuning (such as α dynamic calibration), problems can be detected and addressed early, ensuring the continuous and stable operation of the system.

[0092] In the FRA-M algorithm, A is usually calibrated using historical data. The system can statistically analyze the mean and standard deviation of response times between nodes and dynamically adjust A to adapt to different time periods.

[0093] S74. Assign weights Wi based on the real-time load status of nodes:

[0094]

[0095] in, The normalization coefficient is... For CPU utilization, For memory usage, For network bandwidth, The data integrity score for node i;

[0096] In the weighting formula Adding data integrity The system performs checks to link timeout thresholds and data corruption detection; it can respond quickly to data corruption or node failures and automatically optimize storage tiers and load distribution.

[0097] S75. Requests are allocated according to weight ratios. In weighted round-robin, the number of requests Ni for the i-th node satisfies:

[0098]

[0099] in, Let be the proportion of node i's weight to the total weight of all nodes. The total number of requests to be allocated;

[0100] S76. Dynamically adjust storage levels and data access frequency by combining the machine learning prediction model LSTM. With storage tier Mapping relationship:

[0101]

[0102] Determine node status using TCP timeout and heartbeat loss rate metrics:

[0103]

[0104] in, For fault detection cycle, This is the timeout threshold. The average response time, This is the network latency fluctuation coefficient;

[0105] S77, By optimizing metadata synchronization time and metadata synchronization time Reduce total switchover time:

[0106]

[0107] in, For failover time, For detection time;

[0108] S78, By controlling the fault detection cycle Timeout threshold and load balancing weight To maintain algorithm performance within a stable range; system stability metrics The formula is:

[0109]

[0110] in, Mean time between failures (MTBF) The average recovery time, These are dynamic weighting coefficients;

[0111] By adjusting the parameters maximize;

[0112] S79, Data Reliability R employs multi-replica and RAID redundancy technologies, combined with cross-regional data synchronization:

[0113]

[0114] in, For the number of nodes, Number of copies;

[0115] By storing data replicas on distributed nodes, we ensure that the failure of a single node does not affect data availability; in the formula, increasing the number of replicas m can exponentially improve reliability.

[0116] S8. The fault self-healing and load balancing module automatically triggers recovery and hierarchical adjustment when data is corrupted;

[0117] S81. When data corruption is detected, the dynamic hierarchical self-healing system automatically initiates the recovery process: using erasure coding technology, the corrupted data is reconstructed from available shards or replicas; during the recovery process, data consistency is ensured through the distributed consistency protocol Raft.

[0118] S82. After recovery, reassess the data access trends and importance: Call the LSTM prediction model, combine historical access records and data importance labels, and recalculate the data access frequency and storage level; according to the mapping relationship in S76, dynamically adjust the data to SSD hot storage, HDD warm storage or tape cold storage.

[0119] S83. Automatically update metadata and load balancing weights: Recalculate node weights using the FRA-M algorithm based on the recovered data status. And synchronize metadata to ensure that load balancing and failover mechanisms take effect in a timely manner;

[0120] S84. Record self-healing events and optimize the model: Record data corruption and recovery events in the historical log for retraining and parameter calibration of the LSTM model, by dynamically adjusting the timeout threshold. and weighting coefficients To improve the accuracy of future predictions and self-healing;

[0121] Self-healing events are fed back into the LSTM model, continuously improving prediction and self-healing capabilities.

[0122] A cloud storage data dynamic hierarchical self-healing system includes: a data acquisition and access module, a data preprocessing and feature engineering module, an intelligent prediction and decision-making module, a dynamic hierarchical storage execution module, and a fault self-healing and load balancing module, wherein:

[0123] M1, Data Acquisition and Access Module: Responsible for acquiring data through multiple heterogeneous sources such as sensors, log files, and API interfaces, achieving full lifecycle coverage of thermal, warm, and cold data, and providing a complete raw data foundation for subsequent classification and self-healing;

[0124] M2, Data Preprocessing and Feature Engineering Module: Based on Python's Pandas and other data processing libraries, it cleans, standardizes, and extracts features from raw data; specifically, it handles missing values, outliers, and duplicate data, corrects format errors, and normalizes time series data. At the same time, it transforms categorical data through techniques such as one-hot encoding, and finally generates a structured and normalized dataset for model training, ensuring the consistency of data quality and model input.

[0125] M3, Intelligent Prediction and Decision Module: This is the core driving module, responsible for training an LSTM machine learning model using preprocessed historical data. This model can accurately predict future data access trends and potential system failure risks, providing intelligent decision-making basis for dynamic hierarchical storage and proactive early warning.

[0126] M4, Dynamic Tiered Storage Execution Module: This is the execution layer for the tiered strategy, deployed in the cloud storage service system. This module continuously senses the system status and data access patterns by accessing Kafka's real-time data stream, and receives the prediction results from the data preprocessing and feature engineering modules. It automatically and dynamically allocates data to the storage tier with the best performance by comprehensively considering three dimensions: data importance, real-time access frequency, and storage cost.

[0127] M5, Fault Self-Healing and Load Balancing Module: This module serves as the system's resilience layer, integrating erasure coding technology and the FRA-M intelligent load balancing algorithm. Through real-time heartbeat detection and metadata synchronization, this module can achieve service switching within seconds, rapid data reconstruction, and immediate load optimization and allocation when a node failure is detected, ensuring the system's high availability and continuity, and completing a fully automatic "self-healing" closed loop from fault detection to recovery.

[0128] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for dynamic hierarchical self-healing of cloud storage data, characterized in that, Includes the following steps: S1. Obtain historical data access records from the cloud storage service system through the data acquisition and access module, including data identifier, access timestamp, access type, data size, and user identifier; S2, the data preprocessing and feature engineering module uses Python to clean and standardize the collected data, and normalize the time series data; S3, the data preprocessing and feature engineering module extracts key features from the processed data and transforms the categorical data through one-hot encoding to prepare structured input for model training; S4. The intelligent prediction and decision-making module trains the machine learning prediction model LSTM based on preprocessed historical data. S5. Deploy the trained model to the cloud storage service system. The dynamic hierarchical storage execution module continuously monitors the data through Kafka real-time data stream. S6, the dynamic hierarchical storage execution module predicts data access trends through the LSTM model and dynamically allocates data to the SSD hot storage, HDD warm storage, and tape cold storage levels through the Sealos distributed storage system, combining the importance of stored data, access frequency, and storage cost. S7, the fault self-healing and load balancing module uses erasure coding technology combined with load balancing algorithms to automatically switch services to healthy nodes and redistribute the load when a node fails. The load balancing algorithm uses the FRA-M algorithm to achieve rapid switching of faulty nodes and optimized resource allocation through metadata backup and load balancing among management nodes. S71. The data block is stored as K+F fragments after being encoded by erasure coding. Where K is the number of original data fragments, and F is the number of redundant fragments; S72. Monitor shard access requests through the Kafka real-time stream processing framework, record the number of accesses per unit time, and calculate access popularity; Node-level access frequency This represents the total access frequency of all contained fragments. Erasure coding (K+F sharding) is combined with multi-replica redundancy, enabling second-level takeover in case of failure through log replication, and data recovery conditions satisfy K≤number of available shards≤K+F; Metadata is synchronized in real time between nodes via message passing, ensuring that backup nodes can take over immediately in the event of a primary node failure. S73. Based on distributed consensus protocols such as Raft, metadata changes are implemented through log replication; node status is monitored through a heartbeat mechanism and TCP timeout, and the fault detection period is dynamically associated with a timeout threshold, with the timeout threshold Ttimeout set as follows: ， in, The average response time, Standard deviation, For correction coefficients, and ; Ensure the accuracy and timeliness of fault detection; S74. Assign weights Wi based on the real-time load status of nodes: ， in, The normalization coefficient is... For CPU utilization, For memory usage, For network bandwidth, For the data integrity of node i; In the weighting formula Adding data integrity The system performs checks to link timeout thresholds and data corruption detection; it can respond quickly to data corruption or node failures and automatically optimize storage tiers and load distribution. S75. Requests are allocated according to weight ratios. In weighted round-robin, the number of requests Ni for the i-th node satisfies: ， in, Let be the proportion of node i's weight to the total weight of all nodes. The total number of requests to be allocated; S76. Dynamically adjust storage levels and data access frequency by combining the machine learning prediction model LSTM. With storage tier Mapping relationship: ， Determine node status using TCP timeout and heartbeat loss rate metrics: ， in, For fault detection cycle, This is the timeout threshold. The average response time, This is the network latency fluctuation coefficient; S77, By optimizing metadata synchronization time and metadata synchronization time Reduce total switchover time: ， in, For failover time, For detection time; S78, By controlling the fault detection cycle Timeout threshold and load balancing weight To maintain algorithm performance within a stable range; system stability metrics The formula is: ， in, Mean time between failures (MTBF) The average recovery time, These are dynamic weighting coefficients; By adjusting the parameters maximize; S79, Data Reliability R employs multi-replica and RAID redundancy technologies, combined with cross-regional data synchronization: ， in, For the number of nodes, Number of copies; By storing data replicas on distributed nodes, we ensure that the failure of a single node does not affect data availability; in the formula, increasing the number of replicas m can exponentially improve reliability. S8. The fault self-healing and load balancing module automatically triggers recovery and hierarchical adjustment when data is corrupted; S81. When data corruption is detected, the dynamic hierarchical self-healing system automatically initiates the recovery process: using erasure coding technology, the corrupted data is reconstructed from available shards or replicas; during the recovery process, data consistency is ensured through the distributed consistency protocol Raft. S82. Reassess data access trends and importance after recovery: Utilize the LSTM prediction model, combining historical access records and data importance tags, to recalculate data access frequency and storage level; dynamically adjust data storage to SSD hot storage, HDD warm storage, or tape cold storage based on the mapping relationship in S76; W i S83. Automatically update metadata and load balancing weights: Recalculate node weights using the FRA-M algorithm based on the recovered data status. And synchronize metadata to ensure that load balancing and failover mechanisms take effect in a timely manner; S84. Record self-healing events and optimize the model: Record data corruption and recovery events in the historical log for retraining and parameter calibration of the LSTM model, by dynamically adjusting the timeout threshold. and weighting coefficients This will improve the accuracy of future predictions and self-healing.

2. The cloud storage data dynamic hierarchical self-healing method according to claim 1, characterized in that: In S4, training the machine learning prediction model LSTM specifically involves the following steps: S41 and LSTM models are used to process time series data and are suitable for fault prediction scenarios. S42. Divide the dataset into training and test sets in an 8:2 ratio, optimize the parameters using gradient descent, and improve generalization ability through cross-validation. S43. After training, verify the model performance to ensure that the prediction error is within an acceptable range.

3. A cloud storage data dynamic hierarchical self-healing system, characterized in that, The system includes: a data acquisition and access module, a data preprocessing and feature engineering module, an intelligent prediction and decision-making module, a dynamic hierarchical storage and execution module, and a fault self-healing and load balancing module, among which: M1, Data Acquisition and Access Module: Responsible for acquiring data through multiple heterogeneous sources such as sensors, log files, and API interfaces, achieving full lifecycle coverage of thermal, warm, and cold data, and providing a complete raw data foundation for subsequent classification and self-healing; M2, Data Preprocessing and Feature Engineering Module: Based on Python's Pandas and other data processing libraries, it cleans, standardizes, and extracts features from the raw data; it also normalizes time series data and transforms categorical data through techniques such as one-hot encoding, ultimately generating a structured and normalized dataset for model training; M3, Intelligent Prediction and Decision Module: This is the core driving module, responsible for training an LSTM machine learning model using preprocessed historical data. This model can accurately predict future data access trends and potential system failure risks, providing intelligent decision-making basis for dynamic hierarchical storage and proactive early warning. M4, Dynamic Tiered Storage Execution Module: This is the execution layer for the tiered strategy, deployed in the cloud storage service system. This module continuously senses the system status and data access patterns by accessing Kafka's real-time data stream, and receives the prediction results from the data preprocessing and feature engineering modules. It automatically and dynamically allocates data to the storage tier with the best performance by comprehensively considering three dimensions: data importance, real-time access frequency, and storage cost. M5, Fault Self-Healing and Load Balancing Module: This module serves as the system's resilience layer, integrating erasure coding technology and the FRA-M intelligent load balancing algorithm. Through real-time heartbeat detection and metadata synchronization, this module can achieve service switching within seconds, rapid data reconstruction, and immediate optimized load allocation when a node failure is detected.

Images