Dynamic fault tolerance method and system for online deep reinforcement learning system
By combining operational statistics and dimensionality reduction checksum error resilience mechanisms, and dynamically adjusting the detection threshold, the robustness and reliability issues of online deep reinforcement learning systems in soft error environments are solved, achieving fault tolerance protection throughout the entire process.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-02-13
- Publication Date
- 2026-03-24
AI Technical Summary
Existing technologies in online deep reinforcement learning systems suffer from problems such as high hardware redundancy overhead, offline calibration methods being unable to adapt to dynamic learning, and fault-tolerant methods that only focus on the inference stage failing to cover the training process, resulting in insufficient robustness and reliability of the system in soft error environments.
By employing a forward process error resilience mechanism based on operational statistics and a reverse process error resilience mechanism based on dimensionality reduction checksums, the detection threshold is dynamically adjusted to cover the entire process of forward inference and reverse training, preventing error propagation and contamination, thus forming an adaptive error resilience framework.
It achieves fault-tolerant protection with low overhead, high robustness and reliability, and is applicable to a variety of online deep reinforcement learning algorithms. It does not require modification of the core algorithm, adapts to the dynamic changes of online learning, and improves the stability and reliability of the system in soft error environments.
Smart Images
Figure CN121724093A_ABST
Abstract
Description
Technical Field
[0001] This invention mainly relates to the fields of artificial intelligence and fault-tolerant computing technology, specifically a dynamic fault-tolerant method and system for online deep reinforcement learning systems. Background Technology
[0002] With the rapid development of artificial intelligence technology, deep reinforcement learning systems are being widely deployed in safety-critical fields such as autonomous vehicles, smart grids, and drone control due to their ability to make autonomous decisions in complex and dynamic environments. These systems typically require long-term online operation or continuous learning on resource-constrained edge computing devices (such as embedded GPUs, FPGAs, or dedicated AI accelerators) to adapt to environmental changes and user behavior.
[0003] However, such edge hardware is constantly exposed to harsh physical environments such as radiation, thermal effects, voltage fluctuations, and electrical noise, making it highly susceptible to "soft errors." Soft errors refer to transient bit flips caused by external disturbances (such as single-event upsets (SEUs) induced by cosmic rays). While they do not cause permanent hardware damage, they can corrupt intermediate results in the computation process, such as neuron output values, gradients, or weight updates. For large-scale deep neural networks (DNNs) with millions of parameters, their inherent redundancy can tolerate such errors to some extent. However, online deep reinforcement learning systems typically employ more compact network models with fewer parameters to meet real-time and power consumption constraints. This streamlined structure lacks sufficient redundancy, making it possible for even a single soft error to cause a sharp decline or complete collapse of policy performance.
[0004] Existing fault-tolerant techniques still have the following significant limitations when dealing with soft errors in online deep reinforcement learning systems: 1. Hardware redundancy methods (such as Triple Modular Redundancy, TMR): Fault tolerance is achieved by performing the same computation three times in parallel and then voting by majority vote. While theoretically reliable, this increases computational, storage, and power consumption by at least 200%, which is unacceptable for resource-constrained edge devices and seriously violates the original intention of deploying online RL systems.
[0005] 2. Algorithm-Based Fault Tolerance Based on Offline Calibration (ABFT): These methods (such as linear checksums) typically rely on pre-setting fixed fault tolerance thresholds or checksums based on weight and activation statistics collected during error-free offline training. However, a core characteristic of online reinforcement learning is its continuous online learning capability; its network parameters and activation distribution dynamically evolve with interaction with the environment. The static thresholds of offline calibration cannot adapt to this dynamic change, leading to a sharp increase in false positives (mistaking normal values for errors) or false negatives (failing to detect true errors) in the later stages of training, thus losing their protective effect.
[0006] 3. Fault-Tolerant Methods Focusing Only on the Inference Phase: Many existing works (such as Median Feature Selection, MFS) focus solely on protecting the model's inference (forward propagation) process. They suppress errors by inserting median filters between layers or pruning outlier activations. These methods typically require modifications to the original network structure, increasing deployment complexity. More importantly, they completely ignore the training process (backpropagation). In online RL, erroneous weight updates contaminate the model itself, and their destructive effects are persistent and cumulative, far more severe than a single inference error. Since the training data for online RL (i.e., the agent's interaction experience with the environment) is non-repeatable and non-replayable, once the training process is corrupted by errors, it cannot be recovered by simply repeating a damaged training batch as in supervised learning.
[0007] 4. Lack of versatility and adaptability: Some fault-tolerant solutions are customized for specific RL algorithms (such as Q-learning) or specific hardware platforms (such as drones), making it difficult to generalize their design to other RL paradigms (such as Actor-Critic, PPO) or general computing platforms. Furthermore, most existing methods lack flexible mechanisms for adjusting detection sensitivity, making it impossible to balance "high detection coverage" and "low false alarm rate" according to the actual application scenario. Summary of the Invention
[0008] The technical problem to be solved by this invention is: in view of the technical problems existing in the prior art, this invention provides a dynamic fault-tolerant method and system for online deep reinforcement learning systems that is simple in principle, low in overhead, reliable and robust, and has a wider range of applications.
[0009] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A dynamic fault-tolerant method for online deep reinforcement learning systems includes: Based on the forward process error resilience mechanism of runtime statistics, the threshold setting is used to adaptively adjust with the dynamic process of online training, without the need for any offline calibration data; The reverse process error resilience mechanism based on dimensionality reduction checksum is used to prevent erroneous weights from contaminating the model itself, thus ensuring the continuity and stability of the online learning process. By combining the aforementioned forward process error resilience mechanism based on operational statistics and the backward process error resilience mechanism based on dimensionality reduction checksums, an adaptive online deep reinforcement learning error resilience framework is formed.
[0010] As a further improvement to the method of the present invention: in the forward process error resilience mechanism based on runtime statistics, in each step of network training or inference, standard forward computation is first performed, and then a threshold check based on runtime statistics is applied to the output of each layer; the output exceeding the threshold is set to zero, forming an "error suppression output", which is used for computation of subsequent layers on the one hand, and for updating the runtime statistics of this layer on the other hand.
[0011] As a further improvement to the method of the present invention: in the forward process error resilience mechanism based on running statistics, in each layer of the deep reinforcement learning network, the neuron output value maintains its running mean and running standard deviation and other statistics; using the concentration inequality, upper and lower thresholds are dynamically generated based on these real-time updated running statistics; any neuron output value exceeding the adaptive threshold is regarded as a potential error and is set to zero, thereby blocking the spread of errors in forward propagation.
[0012] As a further improvement to the method of the present invention: the process of the forward process error resilience mechanism based on runtime statistics includes: Step S1: Input training data X; Step S2: Set the distance pruning ratio Pd, the norm pruning ratio Pn, and other standard pruning ratios Po; Step S3: Initialize model parameters W; Step S4: For each training epoch, perform the following operations: Step S5-Step S11; Step S5: Update model parameters W; Step S6: For each convolutional layer of the model, perform the following operations: Steps S7-S10; Step S7: Find the convolution kernels that conform to the norm pruning algorithm; Step S8: Find convolution kernels that meet the distance pruning algorithm; Step S9: Find convolution kernels that conform to other pruning algorithms; Step S10: Set the parameter values of the selected convolution kernel to 0; Step S11: End of step S6; Step S12: End of step S4.
[0013] As a further improvement to the method of the present invention: in the reverse process error elasticity mechanism based on dimensionality reduction checksum, after the weight update amount δW is calculated by standard backpropagation, the system constructs multiple dimensionality reduction checksums for the weight update amount δW; each checksum vector is compared with its own running statistics, and checksums exceeding the threshold will trigger the freezing operation of the corresponding weight.
[0014] As a further improvement to the method of the present invention: the error resilience mechanism of the reverse process based on dimensionality reduction checksum includes: Construct a dimension-reduced checksum; sum along each dimension of the weight tensor to obtain multiple checksum vectors; For each dimension's checksum vector, its runtime statistics are also maintained, and a dynamic threshold is set using the concentration inequality. If an element in a checksum vector exceeds its threshold, the current update of all weights corresponding to that checksum is frozen, and the old values are retained, thereby preventing erroneous weights from contaminating the model itself.
[0015] As a further improvement to the method of the present invention: after calculating the weight update amount δW, the system calculates the checksum of each dimension, applies a statistical threshold to each checksum for error detection, freezes all weight updates that are judged to be abnormal, and finally uses the Welford algorithm to update the running statistics of the checksum.
[0016] As a further improvement to the method of the present invention: the process of the reverse process error resilience mechanism based on dimensionality reduction checksum includes: Step S100: Calculate the weight update amount ; Step S200: For the weight tensor Each dimension Perform the following operations; Step S300: Calculate the checksum vector for this dimension. ; Step S400: Based on the operational statistics of this dimension ( , Using Chebyshev's inequality, calculate the threshold; Step S500: Find All indices j that exceed the threshold constitute the abnormal index set. ; Step S600: Determine the Cartesian product E = of all dimension anomaly indices. × × ... × ; Step S700: For the weights corresponding to all indices in E, reject the current update (freeze) and retain the old values; Step S800: Update the runtime statistics for each dimension using the valid checksum vector. Step S900: Output: Corrected weight update.
[0017] This invention further provides a dynamic fault-tolerant system for online deep reinforcement learning systems, comprising: The forward process error resilience module based on runtime statistics is used to adaptively adjust with the dynamic process of online training by utilizing the threshold setting, without the need for any offline calibration data; The reverse process error resilience module based on dimensionality reduction checksum is used to prevent erroneous weights from contaminating the model itself, ensuring the continuity and stability of the online learning process. An adaptive online deep reinforcement learning error resilience framework is formed by organically combining the aforementioned forward process error resilience module based on runtime statistics and the backward process error resilience mechanism module based on dimensionality reduction checksum.
[0018] As a further improvement to the system of the present invention, it also includes a model weight unit, an environment sampling unit, and a reward unit. The reverse process error elasticity module based on dimensionality reduction checksum outputs to the forward process error elasticity module based on runtime statistics. The forward process error elasticity module based on runtime statistics outputs to the model weight unit to complete parameter updates. After passing through the environment sampling unit, the model weight unit is output by the reward unit to the forward process error elasticity module based on runtime statistics and the reverse process error elasticity module based on dimensionality reduction checksum.
[0019] Compared with the prior art, the advantages of the present invention are as follows: 1. The dynamic fault-tolerant method and system for online deep reinforcement learning systems of the present invention are simple in principle, low in overhead, highly reliable and robust, and have a wider range of applications. This invention is the first fault-tolerant solution that can simultaneously provide protection for the entire process of forward inference and backward training (weight update) in online deep reinforcement learning, overcoming the major deficiency of existing methods that only focus on the inference stage. Furthermore, this invention dynamically adjusts the detection threshold entirely based on online runtime statistical data, perfectly adapting to the dynamic evolution characteristics of online reinforcement learning, without relying on offline data calibration, thus possessing extremely strong versatility and deployment flexibility.
[0020] 2. The present invention provides a dynamic fault-tolerant method and system for online deep reinforcement learning systems. Addressing the shortcomings of existing fault-tolerant methods for online deep reinforcement learning systems—including excessive hardware redundancy overhead, the inability of offline calibration methods to adapt to online dynamic learning, and the fact that existing algorithm-level fault tolerance only covers the inference stage while ignoring the training process—this invention offers a low-overhead, highly versatile, and error-tolerant method that requires no modification to the core reinforcement learning algorithm and can simultaneously cover the entire forward inference and backward training process. This method utilizes real-time collected runtime statistics during online training to dynamically configure adaptive error detection thresholds, effectively improving the robustness and reliability of the system in soft error environments. Attached Figure Description
[0021] Figure 1 This is a schematic diagram illustrating the overall principle of the present invention.
[0022] Figure 2 This is a schematic diagram of the workflow of the forward process error resilience module in a specific application example of the present invention.
[0023] Figure 3 This is a schematic diagram of the reverse process error elasticity module workflow in a specific application example of the present invention. Detailed Implementation
[0024] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0025] This invention presents an error resilience method based on adaptive statistical detection applicable to online deep reinforcement learning (DRM) systems. This invention offers a low-overhead, highly versatile error resilience solution that requires no modification to the core algorithm, adapts to the dynamic process of online learning, and simultaneously covers both forward inference and backward training. It ensures the reliability and security of online DRM systems in soft error environments.
[0026] like Figures 1-3 As shown, this invention discloses a dynamic fault-tolerant method for online deep reinforcement learning systems, comprising: Based on the forward process error resilience mechanism of runtime statistics, the system can adaptively adjust to the dynamic process of online training by setting a threshold, without the need for any offline calibration data. In each step of network training or inference, standard forward computation (MAC and activation) is performed first, and then a threshold check based on runtime statistics is applied to the output of each layer. Outputs exceeding the threshold are set to zero, forming an "error suppression output", which is used for the computation of subsequent layers and for updating the runtime statistics (mean and variance) of the current layer. The error resilience mechanism based on dimensionality reduction checksums is used to prevent erroneous weights from contaminating the model itself, ensuring the continuity and stability of the online learning process. Specifically, after calculating the weight update amount δW through standard backpropagation, the system constructs multiple dimensionality reduction checksums (such as row sums and column sums) for the weight update amount δW. Each checksum vector is compared with its own running statistics, and checksums exceeding the threshold will trigger the freezing operation of the corresponding weights.
[0027] By combining the aforementioned forward process error resilience mechanism based on operational statistics and the backward process error resilience mechanism based on dimensionality reduction checksums, a complete and adaptive online deep reinforcement learning error resilience framework is formed.
[0028] The aforementioned online deep reinforcement learning error resilience framework serves as a general "protective layer" and can be seamlessly integrated into various mainstream online deep reinforcement learning algorithms such as DQN, SAC, and PPO without requiring any modification to their core training logic.
[0029] In a specific application example, the process of the forward process error resilience mechanism based on runtime statistics includes: In each layer of a deep reinforcement learning network, statistics such as the running mean and running standard deviation are maintained for the neuron output values (or the multiplied, accumulated, and MAC output values). Using concentration inequalities such as Chebyshev or Cantelli, upper and lower thresholds are dynamically generated based on these real-time updated operational statistics. Any neuron output value exceeding this adaptive threshold is considered a potential error and is set to zero (i.e., "suppressed"), thereby preventing the spread of errors in forward propagation.
[0030] Specifically, after system initialization, in each training / inference step, forward computation is performed on each layer, statistical thresholds are applied for error detection and suppression, and the Welford algorithm is used to update the runtime statistics of that layer online. In practical applications, the detailed process may include: Step S1: Input training data X; Step S2: Set the distance pruning ratio Pd, the norm pruning ratio Pn, and other standard pruning ratios Po; Step S3: Initialize model parameters W; Step S4: For each training epoch, perform the following operations (steps S5-S11). Step S5: Update model parameters W; Step S6: Perform the following operations on each convolutional layer of the model (steps S7-S10). Step S7: Find the convolution kernels that conform to the norm pruning algorithm; Step S8: Find convolution kernels that meet the distance pruning algorithm; Step S9: Find convolution kernels that conform to other pruning algorithms; Step S10: Set the parameter values of the selected convolution kernel to 0; Step S11: End of step S6; Step S12: End of step S4.
[0031] In a specific application example, the process of the reverse process error resilience mechanism based on dimensionality reduction checksum includes: Constructing dimensionality-reduced checksums: This mechanism does not directly monitor the massive weight update tensor, but instead constructs dimensionality-reduced checksums for it; that is, summing along each dimension of the weight tensor to obtain multiple checksum vectors.
[0032] For each dimension of the checksum vector, its runtime statistics are also maintained, and a dynamic threshold is set using the concentration inequality.
[0033] If an element in a checksum vector exceeds its threshold, the current update of all weights corresponding to that checksum is frozen, and the old values are retained. This prevents erroneous weights from contaminating the model itself and ensures the continuity and stability of the online learning process.
[0034] Specifically, after calculating the weight update amount δW, the system calculates the checksum for each dimension, applies a statistical threshold to each checksum for error detection, freezes all weight updates deemed abnormal, and finally updates the runtime statistics of the checksum using the Welford algorithm. In practical applications, the detailed process may include: Step S100: Calculate the weight update amount ; Step S200: For the weight tensor Each dimension Perform the following operations; Step S300: Calculate the checksum vector for this dimension. ; Step S400: Based on the operational statistics of this dimension ( , Using Chebyshev's inequality, calculate the threshold; Step S500: Find All indices j that exceed the threshold constitute the abnormal index set. ; Step S600: Determine the Cartesian product E = of all dimension anomaly indices. × × ... × ; Step S700: For the weights corresponding to all indices in E, reject the current update (freeze) and retain the old values; Step S800: Update the runtime statistics for each dimension using the valid checksum vector (the part that did not trigger a freeze); Step S900: Output: Corrected weight update.
[0035] The method of this invention dynamically generates an adaptive statistical threshold using runtime statistics of neuron output values, and suppresses neuron outputs exceeding this threshold during forward propagation. A dimensionality-reduced checksum is constructed for the weight update tensor, and the runtime statistics of this checksum are used to dynamically generate an adaptive statistical threshold. Weight updates corresponding to checksums exceeding this threshold during backpropagation are frozen. The runtime statistics are updated in real-time using the Welford online algorithm, enabling the error detection threshold to adapt to the dynamic training process of online reinforcement learning. Suppressing neuron outputs specifically involves setting output values exceeding the threshold to zero; freezing weight updates specifically involves rejecting the current update and retaining the old weight values.
[0036] The method described in this invention utilizes runtime statistics of neuron output values and weight update values collected in real time during online reinforcement learning training to dynamically configure an error detection threshold (referred to as a "check") to cover both the forward inference and backward training processes of deep reinforcement learning. During the forward process, neuron outputs exceeding an adaptive statistical threshold set based on concentration inequalities (such as Chebyshev's inequality and Cantali's inequality) are suppressed (set to zero). During the backward process, a dimension-reduced checksum is constructed for the weight update tensor, and weight updates corresponding to checksums exceeding their runtime statistical thresholds are frozen, retaining their old values. This invention requires no modification to the core reinforcement learning algorithm, the threshold can adaptively adjust during online training, and allows users to balance false positive rate and detection coverage. This invention effectively improves the robustness and reliability of online deep reinforcement learning systems in soft error environments.
[0037] Combination Figure 1 As shown, the present invention further provides a dynamic fault-tolerant system for online deep reinforcement learning systems, comprising: The forward process error resilience module, based on runtime statistics, is used to adaptively adjust to the dynamic process of online training by utilizing threshold settings, without requiring any offline calibration data; it includes a multiply-accumulate unit and an activation calculation unit connected in sequence; The reverse process error resilience module based on dimensionality reduction checksum is used to prevent erroneous weights from contaminating the model itself, ensuring the continuity and stability of the online learning process; it includes a loss calculation unit, a gradient calculation unit, and a gradient update unit connected in sequence. An adaptive online deep reinforcement learning error resilience framework is formed by organically combining the aforementioned forward process error resilience module based on runtime statistics and the backward process error resilience mechanism module based on dimensionality reduction checksum.
[0038] Furthermore, it also includes a model weight unit, an environment sampling unit, and a reward unit. The reverse process error elasticity module based on dimensionality reduction checksum outputs to the forward process error elasticity module based on runtime statistics; the forward process error elasticity module based on runtime statistics outputs to the model weight unit to complete parameter updates; after passing through the environment sampling unit, the model weight unit is output by the reward unit to the forward process error elasticity module based on runtime statistics and the reverse process error elasticity module based on dimensionality reduction checksum. The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.
Claims
1. A dynamic fault-tolerant method for online deep reinforcement learning systems, characterized in that, include: Based on the forward process error resilience mechanism of runtime statistics, the threshold setting is used to adaptively adjust with the dynamic process of online training, without the need for any offline calibration data; The reverse process error resilience mechanism based on dimensionality reduction checksum is used to prevent erroneous weights from contaminating the model itself, thus ensuring the continuity and stability of the online learning process. By combining the aforementioned forward process error resilience mechanism based on operational statistics and the backward process error resilience mechanism based on dimensionality reduction checksums, an adaptive online deep reinforcement learning error resilience framework is formed.
2. The dynamic fault-tolerant method for online deep reinforcement learning systems according to claim 1, characterized in that, In the aforementioned forward process error resilience mechanism based on runtime statistics, in each step of network training or inference, standard forward computation is first performed, and then a threshold check based on runtime statistics is applied to the output of each layer; outputs exceeding the threshold are set to zero, forming "error suppression outputs". These error suppression outputs are used for computation of subsequent layers on the one hand, and for updating the runtime statistics of the current layer on the other hand.
3. The dynamic fault-tolerant method for online deep reinforcement learning systems according to claim 2, characterized in that, In the aforementioned forward process error resilience mechanism based on operational statistics, at each layer of the deep reinforcement learning network, the operational mean and operational standard deviation statistics of the neuron output values are maintained. Using concentration inequality, upper and lower thresholds are dynamically generated based on these real-time updated operational statistics. Any neuron output value exceeding the adaptive threshold is considered a potential error and is set to zero, thus preventing the spread of errors in forward propagation.
4. The dynamic fault-tolerant method for online deep reinforcement learning systems according to claim 3, characterized in that, The process of the forward process error resilience mechanism based on runtime statistics includes: Step S1: Input training data X; Step S2: Set the distance pruning ratio Pd, the norm pruning ratio Pn, and other standard pruning ratios Po; Step S3: Initialize model parameters W; Step S4: For each training epoch, perform the following operations: Step S5-Step S11; Step S5: Update model parameters W; Step S6: For each convolutional layer of the model, perform the following operations: Steps S7-S10; Step S7: Find the convolution kernels that conform to the norm pruning algorithm; Step S8: Find convolution kernels that meet the distance pruning algorithm; Step S9: Find convolution kernels that conform to other pruning algorithms; Step S10: Set the parameter values of the selected convolution kernel to 0; Step S11: End of step S6; Step S12: End of step S4.
5. The dynamic fault-tolerant method for online deep reinforcement learning systems according to any one of claims 1-4, characterized in that, In the reverse process error resilience mechanism based on dimensionality reduction checksums, after the weight update amount δW is calculated by standard backpropagation, the system constructs multiple dimensionality reduction checksums for the weight update amount δW; each checksum vector is compared with its own running statistics, and checksums exceeding the threshold will trigger the freezing operation of the corresponding weight.
6. The dynamic fault-tolerant method for online deep reinforcement learning systems according to claim 5, characterized in that, The reverse process error resilience mechanism based on dimensionality reduction checksum includes: Construct a dimension-reduced checksum; sum along each dimension of the weight tensor to obtain multiple checksum vectors; For each dimension's checksum vector, its runtime statistics are also maintained, and a dynamic threshold is set using the concentration inequality. If an element in a checksum vector exceeds its threshold, the current update of all weights corresponding to that checksum is frozen, and the old values are retained, thereby preventing erroneous weights from contaminating the model itself.
7. The dynamic fault-tolerant method for online deep reinforcement learning systems according to claim 6, characterized in that, After calculating the weight update amount δW, the system calculates the checksum of each dimension, applies a statistical threshold to each checksum for error detection, freezes all weight updates that are judged to be abnormal, and finally uses the Welford algorithm to update the runtime statistics of the checksum.
8. The dynamic fault-tolerant method for online deep reinforcement learning systems according to claim 7, characterized in that, The process of the reverse process error resilience mechanism based on dimensionality reduction checksum includes: Step S100: Calculate the weight update amount ; Step S200: For the weight tensor Each dimension Perform the following operations; Step S300: Calculate the checksum vector for this dimension. ; Step S400: Based on the operational statistics of this dimension ( , Using Chebyshev's inequality, calculate the threshold; Step S500: Find All indices j that exceed the threshold constitute the abnormal index set. ; Step S600: Determine the Cartesian product E = of all dimension anomaly indices. × × ... × ; Step S700: For the weights corresponding to all indices in E, reject the current update and retain the old values; Step S800: Update the runtime statistics for each dimension using the valid checksum vector. Step S900: Output: Corrected weight update.
9. A dynamic fault-tolerant system for online deep reinforcement learning systems, characterized in that, include: The forward process error resilience module based on runtime statistics is used to adaptively adjust with the dynamic process of online training by utilizing the threshold setting, without the need for any offline calibration data; The reverse process error resilience module based on dimensionality reduction checksum is used to prevent erroneous weights from contaminating the model itself, ensuring the continuity and stability of the online learning process. An adaptive online deep reinforcement learning error resilience framework is formed by organically combining the aforementioned forward process error resilience module based on runtime statistics and the backward process error resilience mechanism module based on dimensionality reduction checksum.
10. The dynamic fault-tolerant system for online deep reinforcement learning systems according to claim 9, characterized in that, It also includes a model weight unit, an environment sampling unit, and a reward unit. The reverse process error elasticity module based on dimensionality reduction checksum outputs to the forward process error elasticity module based on runtime statistics. The forward process error resilience module based on operational statistics outputs to the model weight unit to complete parameter updates; After passing through the environment sampling unit, the model weight unit is output by the reward unit to the forward process error resilience module based on runtime statistics and the reverse process error resilience module based on dimensionality reduction checksum.