Neuron and synapse unified four-level hierarchical storage management method and system in neural simulation

Abstract

The invention discloses a neuron and synapse unified four-level hierarchical storage management method and system in neural simulation. According to the method, neuron entities and synaptic entities are abstracted into unified entity objects, and the unified entity objects are uniquely identified by unified entity identifiers comprising entity types, global unique indexes and brain region identifiers to which the entities belong. Four storage levels including a virtual level, a dormancy level, an active level and a key level are defined, and the two types of entities are managed completely in a unified mode. Comparable importance scores for neurons and synapses are calculated by an importance evaluator, respectively, wherein non-linear enhancement protection is applied to long-term memory synapses in a consolidated state. The unified storage manager automatically performs upgrade and downgrade migration of the storage hierarchy according to the importance score. According to the method, the problem of dual-system coordination is eliminated, dynamic allocation between the neurons and the synapses of the graphic processing unit is realized, and the key memory is ensured not to be expelled under the storage pressure.

Description

Technical Field

[0001] This invention relates to the field of data management in high-performance computing, and in particular to a four-level hierarchical storage method and system for unified management of neuron entities and synapse entities in a large-scale biological neural network simulation system. Background Technology

[0002] In large-scale neural simulation systems, two types of core computational entities need to be managed. Neuron entities maintain dynamic information such as membrane potential, ion channel state, and firing history, with storage ranging from 68 bytes to 1500 bytes. Synaptic entities maintain connection weights, plasticity states, and short-term plasticity parameters, with storage ranging from approximately 40 to 80 bytes. Based on 86 billion neurons and an average of 7000 synapses per neuron, the total number of synapses is approximately 600 trillion.

[0003] Existing neural simulation systems suffer from the following shortcomings: First, neuron states and synaptic weights are managed independently by different modules, with independent storage strategies and migration mechanisms, leading to code duplication and coordination difficulties. Second, when a neuron in a certain region is migrated to slow storage, its associated synapses may still be in fast storage, causing data inconsistency. Third, the neuron budget and synaptic budget in the graphics processing unit's video memory are independent and cannot be dynamically allocated according to actual needs. Fourth, there is a lack of a unified importance assessment standard, making it impossible to compare the activity of neurons and the memory importance of synapses within the same framework. Fifth, important long-term memory synapses are stored together with ordinary temporary weights, which cannot ensure that critical memories are not evicted when storage resources are scarce. Summary of the Invention

[0004] The purpose of this invention is to provide a unified four-level hierarchical storage management method and system that treats neuronal entities and synaptic entities as the same type of entity for unified management, automatically allocates storage resources through a multi-dimensional importance assessment mechanism, and ensures the persistent protection of critical memories through a hierarchical strategy.

[0005] To achieve the above objectives, the present invention provides a unified entity storage management method in a neural simulation system, comprising the following steps:

[0006] Step (a) abstracts the neuronal and synaptic entities in the neural simulation system into unified entity objects. Each entity object is uniquely identified by a unified entity identifier, which includes an entity type field, a globally unique index field, and a brain region identifier field.

[0007] Step (b) defines a four-level storage hierarchy: Level 0 (virtual level), Level 1 (dormant level), Level 2 (active level), and Level 3 (critical level). This four-level storage hierarchy is completely unified for both the neuron and synapse entities, and the same management interface applies to both types of entities simultaneously.

[0008] Step (c) involves performing upgrade and downgrade operations on all the entity objects using a unified storage manager. The upgrade operation migrates the entity from a lower storage level to a higher storage level, and the downgrade operation migrates the entity from a higher storage level to a lower storage level.

[0009] Step (d) involves calculating an importance score for each entity object using an importance evaluator. For neuronal entities, the importance score is based on a weighted combination of firing frequency, number of synaptic modifications, number of consolidated synapses, and number of global workspace broadcast participations. For synaptic entities, the importance score is based on a weighted combination of absolute weights, synaptic tag capture state, recent activity time, and elastic weight consolidation Fisher information.

[0010] Step (e): The unified storage manager automatically drives storage tier migration based on the importance score: when the importance score of an entity exceeds the upgrade threshold, an upgrade is triggered; when the importance score of an entity is lower than the downgrade threshold, a downgrade is triggered.

[0011] Furthermore, the entities at the 0th virtual level do not occupy any physical storage space. Virtual neuronal entities are reconstructed on demand using neuron type templates and deterministic pseudo-random number seeds. Virtual synaptic entities have their basic weights calculated programmatically using connection path rules, requiring physical storage only when there are learned weight increments.

[0012] Furthermore, the entities at the third critical level are simultaneously stored in the graphics processing unit's video memory and the permanent database, employing a dual-write strategy to ensure data persistence. This critical level is used to store frequently active neurons and synapses that have consolidated long-term memories.

[0013] Furthermore, the synaptic marker capture state includes four states: baseline state, early long-term enhancement state, late long-term enhancement state, and consolidated state. When the synaptic entity is in the consolidated state, its importance score is non-linearly increased by multiplying it by a preset enhancement coefficient, ensuring that consolidated long-term memory synapses are not downgraded.

[0014] Furthermore, the upgrade operation is triggered by one of the following events: the entity in a virtual or dormant state receives an input pulse signal; the entity is undergoing synaptic plasticity changes; the entity participates in global workspace information broadcasting; an external request explicitly accesses the entity; or a periodic maintenance task reassesses and finds that an upgrade is necessary.

[0015] Furthermore, the downgrade operation adopts a two-factor elimination strategy that combines the least recently used strategy with the importance score. The downgrade priority is calculated by multiplying the inverse of the importance score by the idle time.

[0016] Furthermore, it also includes an anti-vibration cooling mechanism, whereby each entity will not participate in reverse migration decisions for a preset cooling period after performing an upgrade or downgrade operation.

[0017] Furthermore, the unified storage manager maintains storage budget limits for each level. When the usage of any level approaches its limit, a batch degradation operation is automatically triggered. If the target level for degradation also exceeds the budget, a deeper degradation operation is triggered in a cascading manner.

[0018] Furthermore, all large data migration operations of the unified storage manager are executed asynchronously in a background thread without blocking the main simulation calculation loop.

[0019] Furthermore, it also includes a checkpointing step, in which the unified storage manager periodically serializes the current storage level and state data of all entities into checkpoint files, supporting full recovery.

[0020] This invention also provides a unified hierarchical storage management system for neural simulation, comprising: a unified entity abstraction layer for encapsulating neuron entities and synaptic entities into unified entity objects; a four-level storage layer, including a virtual storage sublayer, a dormant storage sublayer, an active storage sublayer, and a critical storage sublayer; an importance assessment module, including a neuron importance evaluator and a synapse importance evaluator; a unified storage manager that automatically performs storage level migration based on importance scores; and a budget management module that monitors the resource usage of each storage layer and triggers automatic eviction.

[0021] The beneficial effects of this invention are as follows: First, by incorporating neurons and synapses into the same management framework, the coordination problem between the two systems is eliminated, and the amount of code is reduced by approximately 40%; Second, the graphics processing unit memory budget can be dynamically allocated between neurons and synapses, improving utilization by approximately 30%; Third, by consolidating the Fisher information perception protection mechanism through synaptic tagging capture state and elastic weights, it ensures that consolidated long-term memory synapses are not evicted under storage pressure; Fourth, the unified entity identifier design allows new entity types to be added only by implementing an importance evaluator, reducing expansion costs by approximately 80%. Attached Figure Description

[0022] Figure 1 This is a unified entity abstract model diagram for the present invention.

[0023] Figure 2 This is a schematic diagram of the four-level storage hierarchy of the present invention.

[0024] Figure 3 This is a flowchart of the neuron importance assessment calculation for this invention.

[0025] Figure 4 This is a flowchart of the synaptic importance assessment calculation and a schematic diagram of the synaptic marker capture state protection of the present invention.

[0026] Figure 5 This is a flowchart illustrating the upgrade / downgrade process of the unified storage manager in this invention.

[0027] Figure 6 This is a flowchart of the budget management and cascading degradation process of this invention.

[0028] Figure 7 This is a diagram illustrating the architecture of the unified hierarchical storage management system of this invention.

[0029] Figure 8 This is a flowchart of the checkpoint saving and restoration process of the present invention. Detailed Implementation

[0030] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0031] The core idea of ​​this invention is to abstract neurons and synapses into unified entities, using the same four-level storage hierarchy and a unified management interface for lifecycle management. Each entity calculates an importance score through an importance evaluator, and the storage manager automatically performs storage hierarchy upgrades and downgrades based on the score.

[0032] like Figure 1 As shown, this invention abstracts neuron entities and synapse entities into a unified entity object. Each entity is uniquely identified by a unified entity identifier, which includes an entity type field (with a value of neuron type or synapse type), a globally unique index field, and a brain region identifier field. This abstraction allows the storage manager to handle both types of entities using the exact same interface.

[0033] like Figure 2 As shown, the four-level storage hierarchy is defined as follows:

[0034] Level 0 (Virtual): Entities do not occupy any physical storage space. Neuronal entities in the virtual state are reconstructed on demand using type templates and deterministic pseudo-random number seeds. Synaptic entities in the virtual state have their basic weights calculated programmatically using connection path rules, requiring physical storage only when there are learned weight increments.

[0035] Level 1 dormancy: Stores minimal state data in non-volatile memory. Neurons retain approximately 16 bytes of minimal state, and synapses retain approximately 8 bytes of minimal state.

[0036] Level 2 activity: Based on the central processing unit's memory storage of complete state data of the entity. Neurons retain 40 to 1500 bytes of complete state, and synapses retain 40 to 80 bytes of complete state.

[0037] Level 3 Critical Level: Based on the graphics processing unit's video memory, store the complete state data of the entity and write it synchronously to the permanent database, using a dual-write strategy to ensure data persistence.

[0038] like Figure 3 and Figure 4 As shown, the importance assessment uses a multi-factor weighted scoring model.

[0039] Neuron importance scores are based on a weighted combination of firing frequency, number of synaptic modifications, number of consolidated synapses, number of global workspace broadcast participations, and connectivity. A score is instantaneously boosted if a neuron has firing activity within the most recent preset time window. Historical importance decays exponentially using a decay factor.

[0040] Synaptic importance scoring is based on a weighted combination of absolute weights, synaptic tag capture state, recent activity time, and resilience weighting of Fisher information. Synaptic tag capture state is the core evaluation factor. Baseline states are mapped to low scores, early long-term enhanced states to medium scores, late long-term enhanced states to high scores, and consolidated states are multiplied by a preset enhancement coefficient to obtain scores significantly higher than 1.0, ensuring that consolidated long-term memory synapses are always ranked last in the demotion process and never expelled.

[0041] Neuron scores and synaptic scores use the same scoring range, allowing the unified storage manager to use the same threshold to determine upgrades and downgrades.

[0042] like Figure 5 As shown, the upgrade operation is triggered by five events: arrival of an input pulse signal, occurrence of synaptic plasticity change, participation of global workspace broadcast, external explicit access request, and periodic maintenance task reassessment. The upgrade process includes a cooling mechanism to prevent the same entity from being repeatedly upgraded or downgraded within a short period of time.

[0043] The downgrade operation employs a two-factor elimination strategy combining the Least Recently Used (LRU) policy and importance scoring. Downgrade priority is the product of the inverse of the importance score and idle time. Synapses in a consolidated state (those captured by synapse tagging) will never be downgraded to the virtual level. Synapses with elastic weights that have consolidated Fisher information exceeding a threshold will at least remain at the dormant level. Entities undergoing plasticity processes are not involved in downgrades. Downgrades are performed in batches.

[0044] like Figure 6 As shown, the unified storage manager maintains strict budget limits for each level. When the usage of any level approaches its limit, the degradation priority of all entities in the current level is calculated, and the least important entities are selected for batch degradation based on priority. If the next level also exceeds its budget, a deeper degradation operation is triggered in a cascading manner.

[0045] All large data migration operations of the unified storage manager are executed asynchronously in a background thread, without blocking the main simulation calculation loop. The main simulation loop only blocks when actually accessing entities that are not yet ready.

[0046] like Figure 8 As shown, the checkpoint management module periodically serializes the state and hierarchy information of all entities in each storage layer into checkpoint files. The checkpoint saving process includes pausing background migration operations, serializing the entity state of each layer, recording the current storage level and metadata of all entities, and writing to the checkpoint file. The recovery process includes reading the checkpoint file, restoring the metadata index, restoring the entity state layer by layer according to priority, and rebuilding the internal state of the storage manager.

[0047] Example 1: In the independent management scheme, 50% of the graphics processing unit's memory is allocated to neurons and 50% to synapses. In the unified management scheme of this invention, the same budget pool is dynamically allocated according to actual needs, improving the graphics processing unit's memory utilization by approximately 30%.

[0048] Example 2: In a simulated long-term learning scenario, synapses consolidated through synaptic tagging are permanently maintained at a critical or active level. Under extreme storage pressure, the system preferentially evicts synapses in the baseline state and inactive neurons, resulting in zero loss of consolidated long-term memory synapses.

[0049] Example 3: When adding a new astrocyte entity type, only the importance evaluator of astrocytes needs to be implemented and registered with the unified storage manager. The migration logic, budget management logic, and checkpoint logic of the storage manager do not need to be modified.

[0050] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A unified entity storage management method in a neural simulation system, characterized in that, Includes the following steps: (a) Abstract neuronal entities and synaptic entities in the neural simulation system into unified entity objects, each of which is uniquely identified by a unified entity identifier, the unified entity identifier including an entity type field, a globally unique index field, and a brain region identifier field; (b) Define a four-level storage hierarchy, namely Level 0 virtual level, Level 1 dormant level, Level 2 active level and Level 3 critical level. The four-level storage hierarchy is completely unified for the neuron entity and the synapse entity, and the same management interface is applicable to both types of entities. (c) Perform upgrade and downgrade operations on all said entity objects through a unified storage manager, wherein the upgrade operation migrates the entity from a lower level to a higher level storage hierarchy, and the downgrade operation migrates the entity from a higher level to a lower level storage hierarchy; (d) Calculate the importance score for each of the entities using an importance evaluator, wherein for neuronal entities, the importance score is based on a weighted combination of firing frequency, number of synaptic modifications, number of consolidated synapses, and number of global workspace broadcast participations; and for synaptic entities, the importance score is based on a weighted combination of absolute weight, synaptic tag capture state, recent activity time, and elastic weight consolidation Fisher information. (e) The unified storage manager automatically drives storage hierarchy migration based on the importance score. When the importance score of an entity exceeds the upgrade threshold, an upgrade is triggered. When the importance score of an entity is lower than the downgrade threshold, a downgrade is triggered.

2. The method according to claim 1, characterized in that: The entities at the 0th virtual level do not occupy any physical storage space. The neuronal entities in the virtual state are reconstructed on demand using neuron type templates and deterministic pseudo-random number seeds. The synaptic entities in the virtual state have their basic weights calculated programmatically using connection path rules. Physical storage is only required when there are weight increments generated by learning.

3. The method according to claim 1, characterized in that: The entities at the third critical level are stored simultaneously in the graphics processing unit's video memory and the permanent database. A dual-write strategy is used to ensure data persistence. The critical level is used to store high-frequency active neurons and synapses that carry long-term memories.

4. The method according to claim 1, characterized in that: The synaptic marker capture state includes four states: baseline state, early long-term enhancement state, late long-term enhancement state, and consolidated state. When the synaptic entity is in the consolidated state, its importance score is non-linearly increased by multiplying it by a preset enhancement coefficient to ensure that the consolidated long-term memory synapse is not downgraded.

5. The method according to claim 1, characterized in that: The upgrade operation is triggered by one of the following events: the entity in a virtual or dormant state receives an input pulse signal; the entity is undergoing synaptic plasticity changes; the entity participates in global workspace information broadcasting; an external request explicitly accesses the entity; or a periodic maintenance task reassesses and finds that an upgrade is necessary.

6. The method according to claim 1, characterized in that: The downgrade operation adopts a two-factor elimination strategy that combines the least recently used strategy with the importance score. The downgrade priority is calculated by multiplying the inverse of the importance score by the idle time. Entities with lower importance scores and longer idle times are downgraded first.

7. The method according to claim 1, characterized in that: It also includes an anti-vibration cooling mechanism, which prevents each entity from participating in reverse migration decisions within a preset cooling time after performing an upgrade or downgrade operation, thus preventing the same entity from repeatedly upgrading or downgrading in a short period of time.

8. The method according to claim 1, characterized in that: The unified storage manager maintains storage budget limits at various levels, including the graphics processing unit (GPU) memory limit, the central processing unit (CPU) memory limit, the non-volatile memory limit, the active entity count limit, and the critical entity count limit. When the usage of any level approaches its limit, a batch downgrade operation is automatically triggered. If the target level for downgrading also exceeds the budget, a deeper downgrade operation is triggered in a cascading manner.

9. The method according to claim 1, characterized in that: All large data migration operations of the unified storage manager are executed asynchronously in a background thread without blocking the main simulation calculation loop, which only blocks and waits when actually accessing entities that are not yet ready.

10. The method according to claim 1, characterized in that: It also includes a checkpointing step, in which the unified storage manager periodically serializes the current storage level and status data of all entities into checkpoint files, supporting the complete recovery of the storage state and level allocation of all entities from the checkpoint files.

11. The method according to claim 1, characterized in that: The design of the unified entity identifier allows for the addition of a new entity type by simply implementing a corresponding importance evaluator for the new type and registering it with the unified storage manager, without modifying the migration logic, budget management logic, and checkpoint logic of the unified storage manager.

12. A unified hierarchical storage management system for neural simulation, characterized in that, include: The unified entity abstraction layer encapsulates neuronal entities and synaptic entities into entity objects with a unified entity identifier, which includes entity type, globally unique index, and brain region identifier. The four-level storage layer includes a virtual storage sublayer, a hibernation storage sublayer, an active storage sublayer, and a critical storage sublayer. The virtual storage sublayer maintains the metadata index of virtual entities and does not allocate physical storage space. The hibernation storage sublayer stores the minimum state data of entities based on non-volatile memory. The active storage sublayer stores the complete state data of entities based on CPU memory. The critical storage sublayer stores the complete state data of entities based on graphics processing unit video memory and writes it synchronously to a permanent database. The importance assessment module includes a neuron importance evaluator and a synapse importance evaluator, which produce comparable importance scores; A unified storage manager connects the four storage tiers and the importance assessment module, and automatically performs upgrade and downgrade migrations of entities between storage tiers based on importance scores; The budget management module monitors the resource usage of each storage layer and triggers automatic eviction based on the least recent use policy and importance when resources are scarce.

13. The system according to claim 12, characterized in that: The synaptic importance evaluator in the importance assessment module applies a nonlinear enhancement to synaptic entities in a consolidated state, making their importance score significantly higher than that of synapses in other states, thus ensuring that consolidated long-term memory synapses are preferentially kept in higher-level storage layers.

14. The system according to claim 12, characterized in that: The unified storage manager receives storage capacity parameters and budget limit parameters from each level through a configuration file, enabling the same system to adapt to hardware configurations of different scales.

15. The system according to claim 12, characterized in that: It also includes a checkpoint management module, which periodically serializes the state and hierarchy information of all entities in each storage layer into checkpoint files and supports complete recovery of the system state from the checkpoint files.

Images