Data rescue and filing system based on cloud platform

Abstract

The invention discloses a data rescue and archiving system based on a cloud platform, and relates to the technical field of data management, the system is composed of a plurality of functional modules, and the system comprises a data sensing and acquisition module which acquires operation data in real time by deploying an edge computing node, the operation data comprises sensor data, business logs and historical fault data, and the data sensing and acquisition module is used for acquiring the operation data; a transmission link is encrypted by a hybrid encryption technology and is transmitted to a cloud end in a standardized manner through a unified data interface; the digital twinborn simulation module is used for generating a multi-dimensional data state vector model based on a lightweight modeling engine, integrating a spatio-temporal data fusion algorithm to dynamically update a twinborn state, introducing a preposed disaster condition of a virtual-real interaction test, simulating an execution effect of a data recovery plan under different disaster conditions through a Monte Carlo method, and performing disaster spreading simulation; and the reinforcement learning optimization module adopts a deep reinforcement learning model.

Description

Technical Field

[0001] The present invention relates to the technical field of data management, and specifically to a data rescue and archiving system based on a cloud platform. Background Art

[0002] With the popularization of personal computers, users' demand for data security has increased day by day, which has promoted the emergence of data recovery software such as Norton Utilities; such software can scan hard disks and recover accidentally deleted or damaged files; therefore, solid-state drives (SSDs) have gradually replaced traditional mechanical hard disks as the mainstream storage medium; however, since SSDs use flash memory as the storage medium, their working principle is completely different from that of traditional magnetic hard disks, especially the mapping of logical addresses to physical addresses is achieved through the FTL (Flash Translation Layer); this change makes traditional magnetic-medium-based data recovery technologies ineffective and unable to be directly applied to SSDs; therefore, data recovery technologies for SSDs have to be innovated, and tools dedicated to reverse-engineering FTL have been developed to be able to analyze the internal data structure of SSDs, thereby increasing the success rate of data recovery.

[0003] Although traditional encryption methods can guarantee data security to a certain extent, they often have performance bottlenecks or compatibility issues, especially during the process of cloud data rescue and archiving, it is difficult to ensure encryption strength and real-time processing speed; in the process of building a digital model that accurately reflects the behavior of the physical world, existing technologies are difficult to process a large amount of heterogeneous data in a timely manner, as well as the balance between model simplification and accuracy, and at the same time lack virtual-real interaction tests for data rescue and archiving, and corresponding data solutions. Summary of the Invention

[0004] To solve the problems raised in the above background art, the present invention is realized through the following technical solutions: A data rescue and archiving system based on a cloud platform, comprising: A data perception and acquisition module, which collects operation data in real time by deploying edge computing nodes. The operation data includes sensor data, business logs, and historical fault data, encrypts the transmission link through hybrid encryption technology, and standardizes the transmission to the cloud through a unified data interface; A digital twin simulation module, which generates a multi-dimensional data state vector model based on a lightweight modeling engine, integrates a spatio-temporal data fusion algorithm to dynamically update the state of the twin body, introduces pre-disaster conditions for virtual-real interaction testing, and simulates the execution effects of data recovery plans under different disaster conditions through the Monte Carlo method to conduct disaster spread simulation; A reinforcement learning optimization module, which uses a deep reinforcement learning model, with inputs including multi-dimensional evaluation indicators, historical fault data, and business logs, and outputs the optimal strategy; The policy execution and feedback module dynamically allocates emergency resources through a two-layer networking network based on the optimal policy generated by reinforcement learning, and optimizes the data transmission path using a path planning algorithm.

[0005] Furthermore, collect operation data: Collect sensor data through Internet of Things devices or sensor networks; Convert the records of the application during the execution of its functions into business logs; Collect detailed information on system or equipment failures through the maintenance management system and record them as historical failure data.

[0006] Furthermore, compress the data and encrypt the transmission link: S101: Preprocess the operation data; insert time points to obtain time series operation data; S102: Use the Zstandard algorithm to perform preliminary compression on the preprocessed time series operation data; S103: On the basis of the Zstandard preliminary compression, use a machine learning model to input the time series operation data and output patterns and regularities; identify through AI predictive coding, and use the patterns and regularities in the data to predict future data points; S104: Combine the results of Zstandard compression with the output generated by AI predictive coding for optimization.

[0007] Furthermore, predict future data points: S103-1: Divide the data stream compressed by Zstandard into time windows and extract multi-dimensional features, including: statistical features, frequency domain features, and pattern markers; S103-2: Build a dynamic context-aware model, maintain the state vector in real time, and save the evolution patterns of the previous Z data points; S103-3: Adopt adaptive quantization residual coding, dynamically adjust the quantization step size, establish a parameter update dictionary, and only transmit non-zero parameter gradient blocks; S103-4: Cross-sensor collaborative prediction, maintain the spatial correlation matrix, and calculate the spatial correlation coefficient; S103-5: Maintain a candidate model pool, switch models based on online evaluation metrics, perform context-adaptive Range coding on the prediction residuals, use the context model to predict the confidence interval and quantization step size level.

[0008] Furthermore, build a lightweight modeling engine to generate a multi-dimensional data state vector model: Mesh simplification of 3D geometric models is carried out based on the QEM algorithm. A hierarchical optimization strategy is adopted to reduce the number of faces by iterative contraction of vertices. The Draco compression technology is used to encode the simplified model. For dynamic parameters, an LSTM prediction network is integrated to generate a temporal state vector, and the LBFGS quasi-Newton method is used to optimize the vertex positions to minimize the surface energy. The maximum geometric error and root mean square error between the 3D geometric model and the mesh-simplified model are calculated through verification by the Metro mesh algorithm. By deploying edge sensors and cloud platform API interfaces, physical environment data, business data, and network topology data are obtained in real time and subjected to multi-stage processing.

[0009] Furthermore, update the twin state: Fuse the multi-source data after preprocessing and registration. For data from different sources, a spatio-temporal data fusion algorithm is used to obtain the target state. Based on the fusion result, a lightweight modeling engine is used to calculate and update the state vector model of the digital twin, and a feedback loop is established to continuously monitor the changes in the actual system and feed the new observations back to the digital twin to continuously adjust its internal state for dynamic update.

[0010] Furthermore, simulate the execution effects of data recovery plans under different disaster conditions: Generate random combination scenarios through the Monte Carlo method to simulate the data island effect caused by physical damage to servers, deduce the feasibility of the cross-rack data migration path, and generate quantitative indicators based on the data loss estimation, backup period, deduplication threshold, and recovery time confidence interval.

[0011] Furthermore, output the optimal strategy: Concatenate the 10-dimensional evaluation vector generated by the autoencoder, the fault scalar value, and the LSTM hidden state vector to form a comprehensive state vector, which integrates static indicators, dynamic time series, and historical risks, providing multi-dimensional inputs for the digital twin; load the current strategy in the digital twin environment and simulate the system operation based on the comprehensive state vector; generate trajectory data in real time during the deduction process, record the state, action, and reward signal of each step to construct the interaction sequence required for reinforcement learning; use generalized advantage estimation to analyze the trajectory data and calculate the advantage value of each step action. GAE balances the bias and variance of the estimation by weighted averaging the short-term return and long-term value to quantify the direction of policy improvement.

[0012] Furthermore, optimize the data transmission path using path planning algorithms: Dynamically configure network paths through the OpenFlow protocol. The control layer detects network devices through the LLDP protocol, constructs a real-time topology map, creates independent virtual network slices according to the task type, including rescue slices and archival slices, maps the parameters to a predefined global policy framework, and increases the resource allocation of a certain type of task on this node through the policy parameters output by the reinforcement learning module.

[0013] The present invention provides a data rescue and archival system based on a cloud platform, having the following beneficial effects: (1) Through the hybrid encryption technology, the present invention realizes end-to-end encryption; and the dynamic path planning algorithm avoids high-risk nodes through real-time link state perception; adopting the Zstandard+AI prediction coding technology, combined with the local preprocessing of edge computing nodes, the data compression rate is increased by 40%, significantly reducing the transmission bandwidth requirement and the cloud storage cost.

[0014] (2) The digital twin simulation module of the present invention simulates multiple disaster scenarios, such as fires and cyberattacks, through the Monte Carlo method, generates quantitative recovery indicators, such as RTO and data loss rate; the reinforcement learning module dynamically optimizes the backup strategy based on historical failures and real-time data; automatically adjusts the backup frequency, shortens the recovery time, and controls the loss of key data integrity.

[0015] (2) The present invention distributes resources according to task priorities through a two-layer networking network (control layer + data layer), dynamically adjusts in combination with the deduplication threshold, reduces redundant storage, integrates a spatio-temporal data fusion algorithm to dynamically update the state of the twin, introduces pre-disaster conditions for virtual-real interaction testing, inputs preconditions for the data rescue and archival path strategy, and simulates the execution effects of data recovery plans under different disaster conditions through the Monte Carlo method for disaster spread simulation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the system flow of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0018] Please refer to Figure 1 , this embodiment provides a data rescue and archival system based on a cloud platform, and the system includes: The data perception and acquisition module collects operation data in real time by deploying edge computing nodes. The operation data includes sensor data, business logs, and historical fault data. It compresses the data using the Zstandard+AI prediction coding technology, encrypts the transmission link using a hybrid encryption technology, and standardizes the transmission to the cloud through a unified data interface; Selection and configuration of edge computing nodes: Select a suitable edge computing device according to the requirements of the application scenario, install the corresponding operating system, and configure network settings and dependencies; For example: In an intelligent agriculture project, multiple-source sensors are deployed to monitor the environmental conditions of farmland, such as soil humidity, temperature, light intensity, etc., and irrigation and farming activities are optimized based on this data to ensure that the data can be processed in real time and reduce the amount of data transmitted to the cloud; multiple edge computing nodes are deployed. Considering the inconvenient power supply and weather resistance requirements, a stable and low-power industrial-grade gateway is selected as the edge computing node, which supports multiple input and output interfaces and has good scalability and adaptability; to meet the requirements of small resource occupancy, high security, and easy maintenance, a lightweight Linux distribution is installed, and the SSH service is configured for remote management and maintenance; at the same time, necessary libraries and tools are installed, such as the Python runtime environment and its related data analysis packages for local data processing, and the system health status is regularly checked and the software is automatically updated by setting Cron jobs; according to actual needs, a static or dynamic IP address acquisition method is configured, and firewall rules are set to only allow access requests from trusted sources to enhance security; Sensor data: Information collected by various physical sensors, including but not limited to environmental or device operation parameters such as temperature, humidity, pressure, location, etc.; obtaining these data usually requires the use of Internet of Things devices or sensor networks; for example, in an industrial environment, temperature sensors can be deployed to monitor the working temperature of machines; Business logs: Log records generated by an application or system during the execution of its functions, including information such as user activities, transaction records, error reports, performance metrics, etc.; business logs are usually automatically generated by the built-in logging mechanism of the software, and specific log levels and content can also be configured according to needs; Historical fault data: Records the detailed information of system or device failures, including failure time, type, cause, solution measures, etc.; collected through maintenance management systems such as CMMS, event management tools, or manual records; for manufacturing enterprises, historical fault data can assist in analyzing the failure modes of equipment and provide a basis for preventive maintenance plans; Compressed data: S101: Preprocess the data. The data collected in real time from sources such as sensors, business logs, and historical fault records will undergo preliminary cleaning and formatting, and time points will be inserted to obtain time series operation data, ensuring data consistency and accuracy, removing duplicate data, filling missing values, and converting data types; S102: Use the Zstandard algorithm to perform preliminary compression on the preprocessed time series operation data, providing a high compression ratio while maintaining a relatively high compression and decompression speed, and adjusting according to actual needs to balance compression efficiency and resource consumption; Zstandard: A fast and efficient lossless compression algorithm; S103: On the basis of the preliminary Zstandard compression, apply AI prediction coding technology. Using a machine learning model, for the input time series operation data or repetitive pattern data, output patterns and rules; identify and utilize the patterns and rules in the data through AI prediction coding to predict future data points, thereby reducing the amount of data that needs to be stored or transmitted; S103-1: Divide the Zstandard-compressed data stream into time windows, extract multi-dimensional features, including: statistical features, frequency domain features, and pattern markers; for example, industrial sensor data is segmented by 5-second windows, and feature vectors such as mean, variance, slope, and the main frequency component of FFT are extracted; establish long-term dependencies through LSTM or Transformer architectures; S103-2: Build a dynamic context-aware model, maintain the state vector in real time, and save the evolution patterns of the previous 100 data points. The following is a pseudo-code example: def hybrid_predict(current_data, model): # Short-term prediction based on a statistical model ar_pred = autoregressive(current_data[-10:], order=3) # Deep learning model prediction dl_pred = model(torch.tensor(current_data).unsqueeze(0)) # Adaptive weight fusion error_stats = calculate_recent_errors() final_pred = (0.7*dl_pred + 0.3*ar_pred) * error_adjustment_factor return quantize(final_pred, precision=0.001) S103-3: Apply adaptive quantization residual coding to dynamically adjust the quantization step Q = ƒ, where Q is the data volatility; f is the prediction confidence; establish a parameter update dictionary and only transmit non-zero parameter gradient blocks; for example, transmit a 3KB model fine-tuning parameter package every 1000 data points; S103-4: Cross-sensor collaborative prediction, maintain the spatial correlation matrix, calculate the spatial correlation coefficient, the formula is: ; where, is the spatial correlation coefficient, specifically the spatial correlation coefficient between sensor k and sensor j, and the value range is [−1,1], which is used to quantify the linear correlation degree of the data sequences of two sensors; and are the observed values of sensor k and sensor j at the same time point; and are the arithmetic means of the observed values of sensor k and sensor j within the current time window; and are the standard deviations of sensor k and sensor j within the time window; when > 0.8, activate the joint prediction mode; otherwise, maintain the current state; S103-5: Maintain a candidate model pool, switch models based on online evaluation metrics, perform context-adaptive Range coding on the prediction residuals, use an 8-level context model, predict the confidence interval, and quantization step level; the code example is: ``` if rolling_MAPE > threshold: load_lightweight_model() elif data_distribution_shift_detected: activate_retraining_protocol() ``` S104: Combine the result of Zstandard compression with the output generated by AI predictive coding for optimization. Store the trained AI predictive model and the compressed residual data together. During decompression, both the predictive model and the compressed data are required to restore the original data; it can contain both AI predictive model information and the residual data after Zstandard compression; for example, a file header can be used to store the model information, followed by the compressed residual data; thus achieving a more efficient compression effect, not only leveraging the advantages of traditional compression algorithms but also using AI to discover and utilize the deep - level structure in the data, with optimized compression ratio and efficiency. Encrypted transmission link: Combine the high efficiency of symmetric encryption with the security of asymmetric encryption to achieve end - to - end secure transmission; the cloud server generates an RSA / ECC asymmetric key pair, i.e., public key encryption and private key decryption, and publishes the public key to the edge node through a digital certificate. The edge node randomly generates a symmetric key, encrypts this key with the cloud public key to generate an encrypted session key package, and verifies the legality of the public key through the digital certificate to prevent man - in - the - middle attacks; the edge node uses Zstandard to compress the original data and combines AI predictive coding to reduce redundancy; and uses the AES - GCM mode to encrypt the compressed data in blocks, while generating an integrity check tag to prevent tampering; encapsulate the encrypted data and the session key package according to the unified interface standard, and attach metadata such as timestamps and device IDs. Unified data interface standardization: Adopt the RESTful API architecture to define the interface specifications, support the HTTP / 2 protocol to improve transmission efficiency, and through the standardized design of the URI path, achieve the unified encoding of device identifiers and data types, ensuring the interface compatibility of different edge nodes. Define a standardized data structure based on JSON Schema, which is compatible with heterogeneous data sources such as sensor data and business logs, and achieve multimodal data fusion through unified metadata fields. Establish a transmission channel through the HTTPS / TLS 1.3 protocol. Forward secrecy guarantees the security of temporary sessions, dynamically selects the transmission path, switches to the backup link when an anomaly occurs, and adds random padding to the data packets to standardize the ciphertext length and resist traffic analysis attacks; after transmitting to the cloud, use the private key to decrypt the session key package to obtain the AES key, use the AES key to decrypt the data block, and verify the GCM tag to ensure data integrity. If the decryption fails or the check tag does not match, trigger an alarm and request retransmission, and at the same time update the key pool; after success, immediately destroy the symmetric encryption session key and initialization vector in memory to avoid the leakage risk caused by key residence. It should be noted that: the session key is automatically updated every hour to prevent the risk of leakage caused by long-term use; the asymmetric key rotates the key pair every 30 days, and combines the certificate revocation list to manage expired certificates; the hardware security module is used to store the master key to isolate the key from the business logic; The digital twin simulation module generates a multi-dimensional data state vector model based on a lightweight modeling engine, integrates a spatio-temporal data fusion algorithm to dynamically update the twin state, introduces pre-disaster conditions for virtual-real interaction testing, and simulates the execution effect of the data recovery plan under different disaster conditions through the Monte Carlo method to conduct disaster spread simulation; Construction of the lightweight modeling engine: The 3D geometric model is meshed and simplified based on the QEM algorithm, and a hierarchical optimization strategy is adopted to significantly reduce the complexity of the digital twin model; the number of faces is reduced by 70% through vertex pair iterative contraction; the simplified model is encoded using the Draco compression technology, and the compression rate can reach more than 80%. For dynamic parameters, an LSTM prediction network is integrated to generate a time series state vector, and the LBFGS quasi-Newton method is used to optimize the vertex position to minimize the surface energy to improve smoothness. The model consistency is verified by the Metro mesh algorithm, and the maximum geometric error and root mean square error between the 3D geometric model and the meshed and simplified model are calculated; Maximum geometric error: The maximum single-point geometric deviation of the corresponding points between the original model and the simplified model, usually measured by the Hausdorff distance, and the formula is: ; Among them, H(A,B) represents the Hausdorff distance, which is used to measure the maximum single-point geometric deviation between two point sets A and B; A and B are the vertex sets of the original model and the simplified model respectively; is the Euclidean distance between vertices; a and b are a point in set A and set B respectively; inf is the infimum, which refers to the minimum value in the set; sup is the supremum, which refers to the maximum value in the set; by traversing all vertices, the maximum deviation value is calculated to verify whether it meets the requirements; Root mean square error: By measuring the average statistical characteristics of the overall geometric deviation of the model, the formula is: ; Among them, RMSE is the root mean square error; N is the total number of vertices; and are the corresponding vertices of the original model and the simplified model respectively; N is the total number of vertices; i is the current vertex, i = 1, 2, 3,..., N; this error reflects the square mean of all vertex deviations and needs to meet the relative error range; Parameter lightweighting screens key parameters through gradient evaluation and channel gain analysis; Generate a multi-dimensional data state vector model: By deploying edge sensors and cloud platform API interfaces, obtain physical environment data, business data, and network topology data in real time, and improve data quality through multi-stage processing; Physical environment data includes computer room location, rack layout, and UPS current fluctuation data; Business data includes timestamp sequences of backup logs and heat maps of data access frequencies; Network topology data: Dynamically capture the IP distribution map of storage nodes and the link delay matrix based on the SDN controller; Combine time window sliding alignment for asynchronous data streams, eliminate dimensional differences through Z-score standardization, apply PCA dimensionality reduction to reduce redundant features, and improve the efficiency of subsequent model training. For high-dimensional time series data, adopt differential coding and AI prediction compression technology to achieve a 40% increase in compression rate. Selection and training of the AI prediction compression model: Select a suitable AI model, such as a prediction model based on neural networks, and use historical high-dimensional time series data to train the model; the goal of the model is to predict the data value at the next time step; Compression operation: Based on the prediction, only store the difference between the predicted value and the actual value (similar to the idea of differential coding) and the relevant parameters of the model (such as the weights of the neural network, etc.). In this way, a 40% increase in compression rate can be achieved; Because the predicted value and the actual value often have a certain correlation, the difference is relatively small and occupies less storage space; fuse geographical information through semantic understanding and GIS technology to solve data compliance and consistency. The processed data is encapsulated into a standardized JSON / Protobuf format and transmitted to the cloud through TLS 1.3 encryption to meet real-time and security requirements; Update the twin state: Fuse the multi-source data after preprocessing and registration. For data from different sources, adopt a spatio-temporal data fusion algorithm to obtain the target state close to the real situation. Based on the fusion result, use a lightweight modeling engine to calculate and update the state vector model of the digital twin. In order to make the digital twin more accurately reflect the actual situation, establish a feedback loop to continuously monitor the changes in the actual system and feed the new observation results back to the digital twin, thereby continuously adjusting its internal state to achieve dynamic update; Simulate the execution effects of data recovery plans under different disaster conditions: Generate random combination scenarios through the Monte Carlo method; simulate the data island effect caused by physical damage to the server, and deduce the feasibility of the cross-rack data migration path; through the estimation of the data loss volume, based on the backup period, deduplication threshold, and recovery time confidence interval, consider the relevant elements of the backup period during the process of generating quantitative indicators based on the backup period, deduplication threshold, and recovery time confidence interval; determine an appropriate backup period according to the system characteristics and requirements. For example, critical business data may need to be backed up daily or even more frequently, while relatively static data can have an appropriately extended period; within each backup period, monitor and record the data change situation, calculate the amount of newly added, modified, or deleted data, and analyze the impact of the backup period on data recovery to understand the data loss risk and resource consumption status under different periods; Incorporate the deduplication threshold and recovery time confidence interval into the analysis. After setting the deduplication threshold, perform deduplication operations during backup and calculate the deduplication effect, evaluate its impact on the recovery time and data loss volume, and find the appropriate balance point of the deduplication threshold; at the same time, define the recovery time confidence interval, collect the time data of multiple simulated data loss and recovery, calculate the mean and standard deviation, determine the confidence interval range, evaluate the system performance based on this, construct a comprehensive quantitative indicator including the weights of various elements, and adjust the system strategy according to the indicator, that is, the Monte Carlo simulation result, quantitative indicator generation; implement a feedback loop for continuously monitoring the actual system, and continuously input new observation data into the digital twin to adjust its internal state in a timely manner; For example: The main server of a data center is damaged due to a fire. It is necessary to evaluate the data recovery plan through digital twin simulation; collect physical parameters such as the temperature, smoke concentration, power status, and hard disk temperature of the damaged server in real time; obtain the server operation logs, retrieve the processing records of past similar fire incidents, and integrate the above data into a multi-dimensional state vector through a lightweight modeling engine. Align the real-time temperature data of the sensor, such as "the temperature in the computer room has soared to 80 °C, timestamp 2023-10-01 14:00"; with the "storage I / O interruption" event in the business log, timestamp 2023-10-01 14:05, and associate them to the same spatial location, such as "Server Cabinet No. 3 in Area C of the Computer Room"; Based on historical fire data and real-time sensor data, predict the physical state changes of the server, such as "it is expected that the hard disk will fail due to high temperature in 10 minutes", combine the incremental backup period and deduplication strategy, such as "data blocks with a repetition rate ≥ 80% are not stored", calculate the amount of unbacked data. For example, 20% of the newly added data in the last hour is unique data that needs to be recovered; inject conditions into the twin to simulate the real disaster environment: Scenario 1: Assume that the fire spread speed accelerates and the backup server needs to be started across regions; Result: Data recovery time was extended to 4 hours, but only 5% of the key business data and database master tables were lost due to deduplication technology; Scenario 2: Assume that the network bandwidth is insufficient and the incremental backup is interrupted; Result: Relying on full backup and recovery, the amount of data loss increased to 15%, but the system availability recovery time was shortened to 2 hours; Data loss: 5%-15%, adjusted according to the backup strategy; Recovery time objective: 2-4 hours, depending on network and backup integrity; System availability: 90%, critical services are restored first, and secondary services are restarted later; The reinforcement learning optimization module uses a deep reinforcement learning model, with inputs including multi-dimensional evaluation indicators, historical fault data, and business logs, and outputs the optimal strategy, including backup frequency, deduplication threshold, and recovery priority; The 10-dimensional evaluation vector, fault scalar value, and LSTM hidden state vector generated by the autoencoder are concatenated to form a comprehensive state vector, which integrates static indicators, dynamic time series, and historical risks, providing multi-dimensional input for the digital twin. The current strategy is loaded in the digital twin environment, such as equipment scheduling rules and disaster recovery plans, and the system operation is simulated based on the comprehensive state vector. During the deduction process, trajectory data is generated in real time, and the state, action, and reward signal of each step are recorded to construct the interaction sequence required for reinforcement learning. The generalized advantage estimation is used to analyze the trajectory data and calculate the advantage value of each action step. GAE balances the estimated deviation and variance by weighted average short-term returns and long-term values, and quantifies the direction of strategy improvement. Set the clipping objective function to limit the amplitude of the policy update to avoid excessive deviation from the current policy, maximize the objective function through gradient ascent, and ensure that the policy is iteratively optimized within a stable range; update the network parameters of the value function to make it more accurately predict the state value, that is, the expectation of long-term cumulative rewards; the update process combines the time difference error and trajectory data to improve the accuracy of the value assessment of the system state and provide a benchmark for subsequent policy optimization; For example: Optimize based on simulation results: If the evaluation indicator shows that "high temperature causes hard disk failure time to be shorter than the backup window", it is recommended to: shorten the incremental backup cycle from 1 hour to 30 minutes; increase off-site backup nodes to reduce the risk of network delay; otherwise, no action will be taken; If "the key data is lost due to an overly high deduplication threshold", adjust the threshold to 70% to balance storage efficiency and data integrity, and quickly initiate the optimized recovery process through the digital twin's rehearsal results. For example: prioritize the recovery of the core database: utilize the most recent full backup (24 hours ago) + the latest incremental backup (1 hour ago), combined with deduplication technology, to control data loss within 5%; and dynamically adjust resource allocation, based on the "network bandwidth fluctuation" predicted by the twin, to reserve a standby transmission channel in advance; otherwise, do not process. Load the current system state in the digital twin environment, inject disaster parameters, and perform K independent simulations for each emergency plan, randomly generating perturbation parameters each time. Example code snippet (pseudo-code): ```python for strategy in strategy_library: for i in range(1000): # Generate random perturbations network_delay = base_delay * (1 + np.random.uniform(-0.2, 0.2)) resource_available = np.random.choice([True, False], p=[0.9, 0.1]) # Execute the simulation and record the results result = digital_twin.simulate(strategy, network_delay, resource_available) save_to_database(result) ``` Record the detailed event chain of each simulation, such as "the failure occurred at 14:00 → plan A was initiated at 14:05 → the recovery was completed at 14:30"; when the deviation between the actual recovery result and the simulated value exceeds the threshold, such as |actual RTO - predicted RTO| > 15%, trigger model calibration; update the equipment failure parameters in the digital twin, adjusting the hard disk MTBF from 100,000 hours to 80,000 hours. The strategy execution and feedback module dynamically allocates emergency resources through a two-layer networking network based on the strategy generated by reinforcement learning, and optimizes the data transmission path using the path planning algorithm. Deployed in the cloud, it is responsible for global policy parsing and resource status monitoring; it receives the policy parameters output by the reinforcement learning module in real time, distributes them among the edge nodes and the core data center, executes specific operation instructions, supports software-defined network technology, dynamically configures network paths through the OpenFlow protocol, and the control layer detects the connection status of network devices, switches, and storage nodes through the LLDP protocol, constructs a real-time topology map, and creates independent virtual network slices according to the task type, including rescue slices and archive slices; The policy parameters may contain weight information for network bandwidth and computing resource allocation under different task types, such as rescue tasks and archive tasks; map the parameters to the predefined global policy framework. When the resource status monitoring shows that the computing resources of a certain edge node are approaching saturation, and the policy parameters output by the reinforcement learning module are to increase the resource allocation of a certain type of task on this node, the system will adjust according to the actual situation; Edge nodes: After receiving the policy parameters output by the reinforcement learning module, it will execute specific operation instructions according to these parameters. For example, if the policy parameters indicate to increase the data read / write priority for a certain type of storage node, the relevant controller in the edge node will adjust the read / write queue management policy of the storage device according to this parameter; For networked devices, if the policy parameters involve adjusting the weight of the network path, the network controller in the edge node will send instructions to network devices such as switches according to this weight parameter through the OpenFlow protocol to dynamically change the configuration of the network path to meet the task requirements; Core data center operations: In the core data center, the policy parameters may be used to adjust large-scale data processing and storage policies; for example, if the policy parameters indicate that more storage space needs to be reserved for the archive slice, the management module in the core data center will re-plan the allocation of storage resources according to this parameter, which may involve reconfiguration of disk arrays or data migration, etc.; Creating independent virtual network slices: The policy parameters may stipulate that the data transmission of the rescue slice needs to pass through a specific low-latency link. For example, when the load of a certain link exceeds the threshold set by the policy parameters, the controller will direct part of the traffic to other links through the OpenFlow protocol; when creating independent virtual network slices (such as rescue slices and archive slices), the policy parameters play a key role; for the rescue slice, the policy parameters may stipulate requirements such as higher bandwidth and lower latency; the SDN controller will configure the network devices through the OpenFlow protocol when creating the slice to ensure that the slice meets the task requirements. For different types of slices, the policy parameters may also involve regulations on security policies, quality of service (QoS), etc.; Recovery slice: A dedicated channel for high-priority data recovery; Archiving slice: A cold data transfer channel that allows bandwidth preemption scheduling; Through the path planning algorithm, the network status is sensed in real time. By collecting link metrics, bandwidth utilization, latency , packet loss rate ; Build a dynamic weight graph, and the formula is: ; In the formula, u and v are two different nodes used to build the dynamic weight graph, which reflects the connection strength or cost between two nodes in the network; α, β, and γ are the weight coefficients of the influence degrees of bandwidth utilization, latency, and packet loss rate, , and are dynamically adjusted according to the policy type; represents the bandwidth utilization of the link; τ is the latency of the link; is the packet loss rate of the link; For example: Path optimization example: Task: Transfer 1TB of disaster recovery data from the G1 regional center to the G4 regional archiving center; Initial path: G1 → G2 → G3 → G4, estimated time-consuming 2.5 hours; Dynamic adjustment: The G2 node has a sudden congestion, and its bandwidth utilization > 90% → Switch to the path of G1 → G5 → G4, time-consuming 2.1 hours; The actual RTO is accurately recorded to the second level; Resource utilization, CPU / storage / bandwidth; Number of path switches.

[0019] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution.

[0020] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units. They may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0021] As described above, it is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application.

Claims

1. A data rescue and archiving system based on a cloud platform, characterized in that: The system includes: A data perception and collection module that collects operation data in real time by deploying edge computing nodes. The operation data includes sensor data, business logs, and historical fault data. The hybrid encryption technology encrypts the transmission link and standardizes the transmission to the cloud through a unified data interface; A digital twin simulation module that generates a multi-dimensional data state vector model based on a lightweight modeling engine, integrates a spatio-temporal data fusion algorithm to dynamically update the twin state, introduces pre-disaster conditions for virtual-real interaction testing, and simulates the execution effect of data recovery plans under different disaster conditions through the Monte Carlo method to conduct disaster spread simulation; A reinforcement learning optimization module that uses a deep reinforcement learning model. The inputs include multi-dimensional evaluation indicators, historical fault data, and business logs, and the output is the optimal strategy; A policy execution and feedback module that dynamically allocates emergency resources through a two-layer networking network based on the optimal policy generated by reinforcement learning, and optimizes the data transmission path using a path planning algorithm.

2. The data rescue and archiving system based on a cloud platform according to claim 1, wherein: Collect operation data: Collect sensor data through Internet of Things devices or sensor networks; Convert the records of the application during the execution of its functions into business logs; Collect detailed information on system or equipment failures through the maintenance management system and record it as historical fault data.

3. The data rescue and archiving system based on a cloud platform according to claim 2, characterized in that: Compress data and encrypt the transmission link: S101: Preprocess the operation data; insert time points to obtain time series operation data; S102: Use the Zstandard algorithm to perform preliminary compression on the preprocessed time series operation data; S103: Based on the preliminary Zstandard compression, use a machine learning model with the input time series operation data to output patterns and rules; identify through AI predictive coding and use the patterns and rules in the data to predict future data points; S104: Combine the results of Zstandard compression with the output generated by AI predictive coding for optimization.

4. The data rescue and archiving system based on a cloud platform according to claim 3, wherein: Predict future data points: S103-1: Divide the data stream compressed by Zstandard into time windows and extract multi-dimensional features, including: statistical features, frequency domain features, and pattern markers; S103-2: Build a dynamic context-aware model, maintain the state vector in real time, and save the evolution patterns of the previous Z data points; S103-3: Adopt adaptive quantization residual coding, dynamically adjust the quantization step size, establish a parameter update dictionary, and only transmit non-zero parameter gradient blocks; S103-4: Cross-sensor collaborative prediction, maintain the spatial correlation matrix, and calculate the spatial correlation coefficient; S103-5: Maintain a candidate model pool, switch models based on online evaluation indicators, perform context-adaptive Range coding on the prediction residuals, use the context model to predict the confidence interval and quantization step size level.

5. A data rescue and archiving system based on a cloud platform according to claim 1, characterized in that: Construct a lightweight modeling engine to generate a multi-dimensional data state vector model: Mesh simplification of 3D geometric models is carried out based on the QEM algorithm. A hierarchical optimization strategy is adopted to reduce the number of faces by iterative contraction of vertices. The Draco compression technology is used to encode the simplified model. For dynamic parameters, an LSTM prediction network is integrated to generate a temporal state vector, and the LBFGS quasi-Newton method is used to optimize the vertex positions to minimize the surface energy. The maximum geometric error and root mean square error between the 3D geometric model and the mesh-simplified model are calculated through verification by the Metro mesh algorithm. By deploying edge sensors and cloud platform API interfaces, physical environment data, business data, and network topology data are obtained in real time and subjected to multi-stage processing.

6. The data rescue and archiving system based on a cloud platform according to claim 5, wherein: Update the twin state: Fuse the multi-source data after preprocessing and registration. For data from different sources, a spatio-temporal data fusion algorithm is adopted to obtain the target state. Based on the fused results, a lightweight modeling engine is used to calculate and update the state vector model of the digital twin. A feedback loop is established to continuously monitor the changes in the actual system and feed the new observations back to the digital twin to continuously adjust its internal state to achieve dynamic updates.

7. The data rescue and archiving system based on a cloud platform according to claim 6, characterized in that: Simulate the execution effects of data recovery plans under different disaster conditions: Random combined scenarios are generated through the Monte Carlo method to simulate the data island effect caused by physical damage to servers, and the feasibility of the cross-rack data migration path is deduced. Through the estimation of data loss volume, based on the backup period, deduplication threshold, and recovery time confidence interval, quantitative metrics are generated.

8. A data rescue and archiving system based on a cloud platform according to claim 1, characterized in that: Output the optimal strategy: Concatenate the 10-dimensional evaluation vector generated by the autoencoder, the fault scalar value, and the LSTM hidden state vector to form a comprehensive state vector, integrating static metrics, dynamic time series, and historical risks to provide multi-dimensional inputs for the digital twin. Load the current strategy in the digital twin environment and simulate the system operation based on the comprehensive state vector. Trajectory data is generated in real time during the deduction process, recording the state, action, and reward signal at each step to construct the interaction sequence required for reinforcement learning. Use generalized advantage estimation to analyze the trajectory data and calculate the advantage value of each step action. GAE balances the bias and variance of the estimation by weighted averaging the short-term return and long-term value to quantify the direction of policy improvement.

9. A data rescue and archiving system based on a cloud platform according to claim 1, characterized in that: Optimize the data transmission path using path planning algorithms: Dynamically configure the network path through the OpenFlow protocol. The control layer detects network devices through the LLDP protocol to construct a real-time topology map. Independent virtual network slices are created according to the task type, including rescue slices and archive slices. Map the parameters to the predefined global policy framework, and adjust the resource allocation of the task through the policy parameters output by the reinforcement learning module.

Images