A checkpoint and redundant deployment based neuron computer fault recovery method

Abstract

The application discloses a neuron computer fault recovery method based on a checkpoint and a redundant deployment, belongs to the technical field of neuron computers, and aims at the characteristics of the neuron computer hardware structure distribution, the state storage dispersion and the frequent dynamic state change, constructs an efficient consistency checkpoint mechanism suitable for the brain-like computing architecture by distinguishing static parameters from dynamic states, and includes an incremental checkpoint hardware mapping describing the positioning relationship of the dynamic state in the physical storage space, when the checkpoint saving is triggered, the execution of the target application task is paused, after the hardware instruction is completely ended and the dynamic state is stable, the hardware state is read item by item according to the mapping and the incremental checkpoint file is generated, when the hardware fault occurs, the model is redeployed and the dynamic state is recovered by using the initial checkpoint and the incremental checkpoint, and then the binding relationship between the application and the model is reconstructed, so that the application continues to execute from the fault interruption position.

Description

Technical Field

[0001] This invention belongs to the field of neural computer technology, specifically relating to a neural computer fault recovery method based on checkpoints and redundant deployment. Background Technology

[0002] Neuromorphic computing is a type of neuromorphic computing hardware that simulates the structure and operation of biological nervous systems. It executes spiking neural network models through large-scale neuron arrays to achieve high parallelism and low power consumption in neural computation. This type of hardware typically features asynchronous event-driven computation, near-memory or in-memory computation, and highly parallel neuron updates. During its operation, it generates a large number of dynamic neuron states that change over time, including synaptic weight changes, membrane potential accumulation, and intermediate pulse activity recordings. These dynamic states determine the model's trajectory and inference results, and are an indispensable part of the neural computation process.

[0003] However, as the scale of neural computers continues to expand, the probability of failure also increases significantly. When existing neural computers experience anomalies (such as computational unit failure, inter-chip communication failure, or memory array errors), the common approach is to reinitialize the model or reschedule application tasks. However, neural computers lack the ability to save the states of neurons during operation, and these states are continuously updated during model execution. Therefore, once a failure occurs, existing recovery methods cannot restore the neural application to its state before the failure, causing the application to have to start from scratch and preventing continuous computation.

[0004] Furthermore, although traditional computing systems have mature checkpointing techniques for saving program execution states, neural computers are distributed and heterogeneous. On the one hand, acquiring neuron states requires multiple network transmissions and memory copies, which incurs significant overhead. If a full save method is adopted, it will not only be extremely costly but also significantly affect system performance. On the other hand, the operating system of a neural computer cannot perceive the task execution status of the neural computer hardware in real time and cannot guarantee the consistency of neuron states. Therefore, neural computers cannot directly apply the checkpointing mechanism of traditional computing systems to achieve dynamic saving and restoration of neuron states.

[0005] In summary, existing technologies are still unable to combine the characteristics of neural computer hardware architecture with a checkpoint mechanism that supports the dynamic preservation and rapid recovery of neuronal state consistency. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, achieve continuous execution of neural network computing tasks, and improve system reliability, this invention adopts the following technical solution:

[0007] A method for fault recovery in neural networks based on checkpoints and redundant deployment includes the following steps:

[0008] Step S1: Based on the difference between static parameters and dynamic states in the neuronal computer, checkpoints are divided into initial checkpoints and incremental checkpoints. Initial checkpoints record the neural network structure and static parameters in the neuronal computer, while incremental checkpoints record the dynamically changing neuron states during the operation of the neuronal computer.

[0009] Step S2: In the neuron application startup phase, the neuron state distribution is obtained by analyzing the neuron application to indicate the location and organization of the dynamic state in the hardware storage space, and an incremental checkpoint hardware mapping is generated to provide a basis for the extraction of consistent incremental checkpoints.

[0010] Step S3: When the checkpoint saving operation is triggered during the operation of the neural network, the task execution of the corresponding neuron application is paused, and the hardware instruction is waited for completion to ensure that the dynamic state no longer changes. The dynamic state is extracted based on the incremental checkpoint hardware mapping to generate and save the incremental checkpoint, and then the operation is resumed.

[0011] Step S4: When a hardware failure occurs in the neuron computer, the neural network is redeployed through initial checkpoints and incremental checkpoints to restore static parameters and neuron dynamic states, so that the application can continue running from the point of failure interruption, thereby ensuring that the neuron application can continue to run without rebuilding or relearning.

[0012] Further, step S2 includes the following sub-steps:

[0013] Step S21: Parse the application description file and obtain the logical layout; read the description file of the neuron application, analyze the structural information of the neural network, neuron type, synaptic connection mode and memory layout of the model in the logical neuron computing kernel, and obtain the neuron state distribution at the logical level.

[0014] Step S22: Generate a set of logical addresses for incremental checkpoints. Obtaining logical layout information is the full data structure of the model. The system needs to further identify the dynamic states that will evolve over time during the model's operation and determine their corresponding logical storage areas. Therefore, it is necessary to identify the dynamic states that change during operation (such as membrane potential, synaptic weights, etc.) based on the layout information of the logical neuron calculation kernel and generate a set of logical addresses required to store incremental checkpoints, which is used to indicate the location range of these dynamic states in the logical storage space.

[0015] Step S23: Calculate the corresponding physical address set; combine the resource allocation results of the neural computer during actual operation to obtain the mapping relationship between the logical neuron computing kernel and the physical neuron computing kernel, and convert the logical address into the actual hardware physical address accordingly to obtain the physical address set of the incremental checkpoint, forming the incremental checkpoint neuron state mapping. The physical address set is used to indicate the actual storage location of the dynamic neuron state in the physical hardware.

[0016] Further, in step S3, a pause record is generated based on the identifier of the running neuron application, and the application identifier of the neuron application and the queue of waiting tasks waiting to be scheduled are written into the application pause table; during subsequent task scheduling, the application identifier of each task to be scheduled is checked; if the application identifier exists in the key set of the pause table, the task is added to the corresponding waiting task queue; otherwise, the task is added to the execution queue; only tasks in the execution queue are sent to the neuron computer for execution; when it is necessary to resume application execution, the waiting task queue of the target application is taken out from the pause table and all of them are put into the execution queue to resume the operation of the neuron application.

[0017] Furthermore, check the asynchronous waiting task queue corresponding to the target application to confirm that all previously issued instructions have returned execution results.

[0018] Furthermore, in step S3, after confirming that the hardware state is stable, the system reads the physical address of the dynamic state in the neuron computer according to the incremental checkpoint hardware mapping; by calling the read state interface provided by the neuron computer, it reads the membrane potential, synaptic weight and other dynamic neuron state values ​​item by item according to the address range defined in the mapping table, and organizes the read data and the neuron application execution progress into an incremental checkpoint file in a predetermined format for fault recovery.

[0019] Further, step S4 includes the following steps:

[0020] Step S41: Redeploy the neural network based on the initial checkpoint and incremental checkpoint; when a failure is detected in the target neuron computing kernel and / or related hardware, select an idle hardware unit with available computing and storage resources, redeploy the initial checkpoint on the new hardware resources, and restore the dynamic state based on the incremental checkpoint. In this way, the model running environment before the failure can be reconstructed on different hardware nodes.

[0021] Step S42: Rebind the neuron resources, connection relationships and task scheduling information of the original neural network to the newly deployed neural network instance, so that the logical neurons and physical neuron computing units can re-establish the correspondence, thereby restoring the application's running environment to a consistent state.

[0022] In step S43, the neuron computer operating system starts a new neural network instance, resumes the operation of the neural network and rebuilds the running progress based on the initial checkpoint and incremental checkpoint, so that the application can continue to run on the new hardware instance from the interruption position before the failure, realizing seamless recovery at the business level and continuity assurance at the system level.

[0023] Furthermore, in step S41, when a fault occurs, it is determined whether the application already has a redundantly deployed initial checkpoint instance. If it does, the most recent incremental checkpoint is directly loaded on the redundantly deployed neural network to restore the dynamic neuron state of the neural network. If it does not exist, an idle hardware unit with available computing and storage resources is selected, the initial checkpoint is redeployed on the new hardware resources, and the dynamic state is restored based on the incremental checkpoint.

[0024] Furthermore, in step S41, considering the propagation of faults, redundant deployment is carried out with priority across chassis, modules, and chips.

[0025] Furthermore, in step S42, the rebinding process includes updating the application instance's model handle, access address, task mapping table, and hardware interface parameters, enabling the application to access the new neural network instance in a transparent manner.

[0026] Further, in step S1, the internal storage of a single computational core of the neuronal computer is divided into multiple storage areas according to the neuron's operating characteristics, including an instruction storage area, a membrane potential and weight data storage area, and an axon and dendrite data storage area. Based on memory partitioning, a checkpoint data format is constructed, including the number of configuration blocks, the configuration block header, and the configuration block data. The configuration block header adopts a fixed-length format and includes a storage area identifier, a storage area base address, an address offset, a data width, and a data length, which are used to describe the precise location and data structure of the configuration block in hardware storage. The configuration block data stores the corresponding content according to the description in the configuration block header.

[0027] The advantages and beneficial effects of this invention are as follows:

[0028] This invention is adaptable to the distributed, heterogeneous neuron state storage structure of neuronal computers. By constructing a neuron state mapping, it presents the dynamic neuron states, originally scattered across multiple hardware storage units, in a unified logical manner, thus resolving the structural contradiction that traditional checkpointing techniques cannot be applied to neuronal hardware. This mechanism can provide universal support for different types of neuronal computers without modifying the underlying hardware architecture, offering fundamental state abstraction capabilities for neuromorphic computing platforms.

[0029] This invention divides checkpoints into initial checkpoints and incremental checkpoints, saving only the dynamic neuron states that change during operation, thus avoiding a full read of all hardware states. This significantly reduces the amount of data, access overhead, and system interference generated during checkpoint saving. Simultaneously, by pausing application execution and waiting for the hardware to complete the current instruction when saving checkpoints, it ensures that the dynamic state remains consistent during the reading process. This allows the model to accurately return to its pre-failure running state after fault recovery, improving recovery accuracy and system continuity.

[0030] During the fault recovery phase, this invention can restore the model structure, static parameters, and dynamic neuron states based on initial checkpoints and incremental checkpoints, eliminating the need for applications to restart from the initial state and effectively improving the continuous execution capability of neuromorphic computing tasks. Where hardware resources permit, this invention can also redundantly deploy initial checkpoints to further shorten recovery time and improve system reliability.

[0031] Overall, this invention provides, for the first time, a system-level, low-overhead, and highly consistent fault recovery mechanism for neuromorphic computing systems, significantly enhancing the reliability and engineering usability of neuromorphic computing systems. Attached Figure Description

[0032] Figure 1 This is a flowchart of the method in an embodiment of the present invention;

[0033] Figure 2 This is a schematic diagram of the memory partitioning of a single core in the neuron computing hardware in this embodiment of the invention;

[0034] Figure 3 This is a schematic diagram of the application description file structure in the neuron dynamic state mapping of this invention;

[0035] Figure 4 This is a schematic diagram of the neuron dynamic state mapping table structure in the neuron dynamic state mapping embodiment of the present invention;

[0036] Figure 5 This is a schematic diagram of the neuron hardware layout structure in the neuron dynamic state mapping of this invention;

[0037] Figure 6 This is a flowchart of the application paused by the neuron computer scheduling control in an embodiment of the present invention;

[0038] Figure 7 This is a schematic diagram of the redundant model deployment location selection in an embodiment of the present invention (board 1).

[0039] Figure 8 This is a schematic diagram of the redundant model deployment location selection in an embodiment of the present invention (board 2). Detailed Implementation

[0040] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0041] like Figure 1 As shown, this invention discloses a fault recovery method for neural computers based on checkpoints and redundant deployment. This method addresses the characteristics of distributed storage of dynamic neuron states, high access overhead, and frequent state updates in neural computers. By defining a hierarchical checkpoint structure, constructing a neuron state mapping, and employing a state extraction mechanism based on consistency control, it achieves effective preservation and rapid recovery of dynamic neuron states. Specifically, it includes the following steps:

[0042] Step S1: Based on the difference between static parameters and dynamic states in the neuron computer, checkpoints are divided into initial checkpoints and incremental checkpoints. Initial checkpoints record the neural network structure and static parameters in the neuron computer, while incremental checkpoints record the dynamically changing neuron states during the operation of the neuron computer.

[0043] This implementation method, based on the operational characteristics of neural computers—namely, "fixed static parameters and dynamic state evolving over time"—classifies checkpoints into two categories: initial checkpoints and incremental checkpoints, to achieve efficient and low-overhead data storage. Specifically, it includes:

[0044] First, the initial checkpoint is used to record the model structure and its static parameters, which are directly generated from the original model description file during application deployment. Since these parameters remain unchanged after model deployment, they only need to be generated once during application startup, avoiding additional overhead during subsequent operation.

[0045] Secondly, incremental checkpoints are used to save the constantly changing dynamic neuron states during operation, including but not limited to key data such as synaptic weights and membrane potentials that evolve over time. By saving only the changed parts rather than the complete hardware state, the data volume of checkpoints can be significantly reduced, improving storage efficiency.

[0046] like Figure 2 As shown, the internal storage of a single computational core in a neuronal computer can be divided into multiple storage areas based on the neuron's operational characteristics, including an instruction storage area, a membrane potential and weight data storage area, and axon and dendrite data storage area. Based on this memory partitioning, this embodiment designs a general checkpoint data format, comprising three parts: the number of configuration blocks, the configuration block header, and the configuration block data. The configuration block header uses a fixed-length format and includes fields such as storage area identifier, storage area base address, address offset, data width, and data length, used to describe the precise location and data structure of the configuration block in hardware storage; the configuration block data stores the corresponding content according to the description in the configuration block header.

[0047] Finally, the checkpoints are organized into binary files in the above format and saved for subsequent state recovery operations.

[0048] Step S2: During the application startup phase, the neuron state distribution is obtained through neuron application analysis in the neural computer to indicate the location and organization of the dynamic state in the hardware storage space, generating incremental checkpoint hardware mappings to provide a basis for the extraction of consistent incremental checkpoints; specifically including the following steps:

[0049] Step S21: Parse the application description file and obtain the logical layout. The system reads and parses the description file of the neuron application, extracting the logical structure information of the neuron application, including neuron type, number of neurons, synaptic connections, hierarchical structure, and the logical storage organization of various parameters. Further analysis of the model's memory layout in the logical neuron computation kernel determines the storage location and organization of various dynamic neuron states (e.g., membrane potential, synaptic weights) at the logical level, thereby obtaining a distribution map of logical neuron states. Figure 3 As shown, the application description file is... Figure 2 The storage area format describes the data in different storage areas.

[0050] Step S22: Generate a logical address set for incremental checkpoints. The logical layout information obtained in Step S21 is the full data structure of the model. The system needs to further identify the dynamic states that evolve over time during model operation and determine their corresponding logical storage areas. Based on this, a logical address set for dynamic states is constructed to describe the data range that needs to be included in the incremental checkpoints. This logical address set may include fields such as logical base address, offset, data width, and data length to accurately identify the storage location of dynamic states in the logical space.

[0051] Step S23: Calculate the corresponding physical address set. Combining the resource allocation information generated during the actual deployment phase of the neural network computer, the system obtains the mapping relationship between the logical neuron computing cores and the physical neuron computing cores. Based on this relationship, the logical address set of the dynamic state is mapped item by item to the corresponding physical address range, thereby obtaining the physical address set of the incremental checkpoint. This set is used to indicate the actual storage location of the dynamic neuron state in the physical hardware, such as... Figure 4 As shown, the neuron state mapping table includes information such as the physical coordinates of the neuron's computational kernel, the number of data blocks, the physical base address, the address offset, and the data length, which can be used as the basis for subsequent incremental checkpoint read operations. Figure 5 As shown, the neuron hardware is distributed on the neuron hardware chassis in the form of a two-dimensional grid. boardID represents the board ID, and x,y represent the position coordinates of the computing kernel on the board.

[0052] Step S3: When the checkpoint save operation is triggered during operation, pause the task execution of the corresponding neuron application, wait for the hardware instruction to be executed, ensure that the dynamic state no longer changes, extract the dynamic state based on the incremental checkpoint hardware mapping, generate and save the incremental check, and then resume operation.

[0053] This implementation method ensures consistent and accurate neuron state data when saving incremental checkpoints through three stages: application pausing, state stability verification, and dynamic hardware state reading based on mapping. Specifically, it includes the following steps:

[0054] Step S31: Pause the execution of the target neuron application. When a checkpoint save operation is triggered during application execution, the task scheduling module first pauses the task execution of the corresponding neuron application.

[0055] Specifically, such as Figure 6 As shown, the scheduling module generates a pause record based on the application identifier, including the application ID of the application and its queue of tasks waiting to be scheduled. Write the application pause table M. During subsequent task scheduling, the scheduling module checks the application ID of each task to be scheduled; if the ID exists in the key set of table M, the task is added to the corresponding table. Conversely, add the task to the execution queue. In China. Only. The tasks are distributed to the neural network computer for execution. When application execution needs to be resumed, the target application's task is retrieved from table M. And put them all into the execution queue In order to restore the normal operation of the application.

[0056] Step S32: Wait for the hardware instructions to complete execution to ensure the dynamic state is stable. After pausing the application, the system further confirms that all pending operations on the neural network computer have been completed to ensure that the dynamic neuron state does not continue to change during the reading process. Specifically, the system checks the asynchronous wait queue corresponding to the target application to confirm that all previously issued instructions have returned execution results.

[0057] Step S33: Extract dynamic states and generate incremental checkpoints based on hardware mapping. After confirming that the hardware state is stable, the system reads the physical address of the dynamic state in the neuron computer according to the incremental checkpoint hardware mapping established in step S2. Specifically, the system calls the read state interface provided by the neuron computer, reads membrane potentials, synaptic weights, and other dynamic neuron state values ​​item by item according to the address range defined in the mapping table, and organizes these data and the neuron application execution progress into an incremental checkpoint file in a predetermined format for subsequent fault recovery.

[0058] Step S4: When a hardware failure occurs in the neural network computer, the neural network is redeployed through initial checkpoints and incremental checkpoints to restore static parameters and neuron dynamic states, allowing the application to resume operation from the point of interruption. This ensures that the neural network application can continue running without rebuilding or relearning. Specifically, this includes the following steps:

[0059] Step S41: Redeploy the model based on the initial checkpoint and incremental checkpoints. When the system detects a failure in the target neuron's computational kernel, on-chip storage array, or its connection links, it first determines whether the corresponding application has a redundantly deployed initial checkpoint instance. If a redundant instance exists, the most recently saved incremental checkpoint is directly loaded onto the redundant model to restore dynamic neuron states such as synaptic weights and membrane potentials, achieving rapid model recovery.

[0060] like Figure 7 , Figure 8 As shown, considering the propagation of faults, redundancy deployment is prioritized across chassis, modules, and chips. Specifically, the diagram prioritizes the position of redundancy model candidate 1, followed by the position of redundancy model candidate 2, and finally the position of redundancy model candidate 3.

[0061] If no redundant instances exist, the resource allocation module selects a physical neuron computational core with available computing and storage resources, redeploys the initial checkpoint on it, and restores the dynamic state by loading incremental checkpoints. In this way, the model environment before the failure can be completely reconstructed on the new hardware node, ensuring the model has the same neuron state as before the failure.

[0062] Step S42: Rebind the application and the restored model. After the model is restored, the system needs to re-establish the association between the application and the new model instance.

[0063] Specifically, the system remaps the application's logical neuron resources, synaptic connections, and task scheduling information to the restored physical neuron computational kernels. The binding process includes updating the application instance's model handle, access address, task mapping table, and hardware interface parameters, enabling the application to transparently access the new model instance. After this step, the application's runtime environment is restored to a consistent state.

[0064] Step S43: Resume application execution and rebuild the running progress.

[0065] After rebinding the application and model, the neural network operating system creates a new application instance while retaining the original application ID, and restores the application's runtime progress based on the initial checkpoint and incremental checkpoints. The restored information includes the application's runtime context. Ultimately, the application can continue running on the new hardware instance from its point of interruption before the failure, achieving seamless recovery at the business level and ensuring continuity at the system level.

[0066] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for fault recovery in neural networks based on checkpoints and redundant deployment, characterized in that... Includes the following steps: Step S1: Based on the difference between static parameters and dynamic states in the neuronal computer, checkpoints are divided into initial checkpoints and incremental checkpoints. Initial checkpoints record the neural network structure and static parameters in the neuronal computer, while incremental checkpoints record the dynamically changing neuron states during the operation of the neuronal computer. Step S2: In the neuron activation phase, the neuron state distribution is obtained by analyzing the neurons to indicate the location of the dynamic state in the hardware storage space, and an incremental checkpoint hardware mapping is generated. Includes the following sub-steps: Step S21: Parse the application description file and obtain the logical layout; read the description file of the neuron application, analyze the structural information of the neural network, neuron type, synaptic connection mode and memory layout of the model in the logical neuron computing kernel, and obtain the neuron state distribution at the logical level. Step S22: Generate a logical address set for incremental checkpoints; based on the layout information of the logical neuron computation kernel, identify the dynamic states that change during operation, and generate a logical address set required to save the incremental checkpoints, which is used to indicate the location range of the dynamic states in the logical storage space. Step S23: Calculate the corresponding physical address set; combine the resource allocation results of the neural computer during actual operation to obtain the mapping relationship between the logical neuron computing kernel and the physical neuron computing kernel, and convert the logical address into the actual hardware physical address accordingly to obtain the physical address set of incremental checkpoints, forming the incremental checkpoint neuron state mapping. Step S3: When the checkpoint saving operation is triggered during the operation of the neural network, the task execution of the corresponding neuron is paused, and the hardware instruction is waited for completion. The dynamic state is extracted based on the incremental checkpoint hardware mapping to generate and save the incremental checkpoint, and then the operation is resumed. Step S4: When a hardware failure occurs in the neural network computer, the neural network is redeployed through initial checkpoints and incremental checkpoints to restore static parameters and the dynamic state of neurons, so that the running progress can continue from the point of failure interruption.

2. The neural network computer fault recovery method based on checkpoints and redundant deployment according to claim 1, characterized in that: In step S3, a pause record is generated based on the running neuron application identifier, and the application identifier of the neuron application and the queue of waiting tasks waiting to be scheduled are written into the application pause table. During subsequent task scheduling, the application identifier of each task to be scheduled is checked; if the application identifier exists in the pause table, the task is added to the corresponding waiting task queue; otherwise, the task is added to the execution queue; only tasks in the execution queue are sent to the neural computer for execution; when it is necessary to resume application execution, the waiting task queue of the target application is taken out from the pause table and all of them are put into the execution queue to resume the running of the neural application.

3. The neural network computer fault recovery method based on checkpoints and redundant deployment according to claim 2, characterized in that: Check the asynchronous waiting task queue corresponding to the target application to confirm that all previously issued instructions have returned execution results.

4. The neural network computer fault recovery method based on checkpoints and redundant deployment according to claim 1, characterized in that: In step S3, after confirming that the hardware state is stable, the system reads the physical address of the dynamic state in the neuron computer according to the incremental checkpoint hardware mapping; by calling the read state interface provided by the neuron computer, it reads the membrane potential, synaptic weight and other dynamic neuron state values ​​item by item according to the address range defined in the mapping table, and organizes the read data and the neuron application execution progress into an incremental checkpoint file in a predetermined format for fault recovery.

5. The neural network computer fault recovery method based on checkpoints and redundant deployment according to claim 1, characterized in that: Step S4 includes the following steps: Step S41: Redeploy the neural network based on the initial checkpoint and incremental checkpoint; when a failure is detected in the target neuron computing kernel and / or related hardware, select an idle hardware unit with available computing and storage resources, redeploy the initial checkpoint on the new hardware resources, and restore the dynamic state based on the incremental checkpoint. Step S42: Rebind the neuron resources, connection relationships and task scheduling information of the original neural network to the newly deployed neural network instance so that the logical neurons and physical neuron computing units can re-establish the correspondence. In step S43, the neuron computer operating system starts a new neural network instance, resumes the operation of the neural network and reconstructs the running progress based on the initial checkpoint and incremental checkpoint.

6. The neural network computer fault recovery method based on checkpoints and redundant deployment according to claim 5, characterized in that: In step S41, when a fault occurs, it is determined whether the application already has a redundantly deployed initial checkpoint instance. If it does, the most recent incremental checkpoint is directly loaded on the redundantly deployed neural network to restore the dynamic neuron state of the neural network. If it does not exist, an idle hardware unit with available computing and storage resources is selected, the initial checkpoint is redeployed on the new hardware resources, and the dynamic state is restored based on the incremental checkpoint.

7. A neural network computer fault recovery method based on checkpoints and redundant deployment according to claim 5, characterized in that: In step S41, redundant deployment is prioritized across chassis, modules, and chips.

8. A method for neural network computer fault recovery based on checkpoints and redundant deployment according to claim 5, characterized in that: In step S42, the rebinding process includes updating the application instance's model handle, access address, task mapping table, and hardware interface parameters, enabling the application to access the new neural network instance in a transparent manner.

9. A neural network computer fault recovery method based on checkpoints and redundant deployment according to claim 1, characterized in that: In step S1, the internal storage of a single computational core of the neuronal computer is divided into multiple storage areas according to the neuron's operating characteristics, including an instruction storage area, a membrane potential and weight data storage area, and an axon and dendrite data storage area. Based on the memory partitioning, a checkpoint data format is constructed, including the number of configuration blocks, the configuration block header, and the configuration block data. The configuration block header contains a storage area identifier, a storage area base address, an address offset, a data width, and a data length, which are used to describe the location and data structure of the configuration block in the hardware storage. The configuration block data stores the corresponding content according to the description in the configuration block header.

Images