Simulation system supporting rapid large-scale brain simulation

Abstract

The invention discloses a simulation system supporting fast large-scale brain simulation, and belongs to the technical field of brain simulation, the simulation system comprises a hardware equipment layer for providing a plurality of hardware equipment resources; the data communication layer is used for providing multiple types of communication modes among the computing nodes; the operation abstraction layer provides multiple types of neuron models, synaptic models, connection rules and learning rules; the API layer is used for providing an API interface and receiving a user demand through the API interface; the hardware abstraction layer calls a corresponding hardware kernel; and the network abstraction layer is used for recording the topological structure of the brain simulation network, carrying out resource allocation and neuron cluster mapping, then creating the brain simulation network on hardware equipment, and finally training or executing the brain simulation network. Cluster hardware resources are fully utilized to realize faster and larger-scale brain simulation, and the problems that resource scheduling and resource allocation are lacked, parallelism of equipment is not fully utilized, communication time consumption is too long, and supported hardware and interfaces are limited are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of brain simulation, and in particular relates to a simulation system supporting rapid large-scale brain simulation. Background Art

[0002] At present, the development of traditional artificial intelligence represented by deep learning has encountered a bottleneck. It is not only difficult to achieve general artificial intelligence similar to the human brain, but also requires a lot of energy. The human brain represents the pinnacle of biological intelligence. Brain simulation not only reflects the most advanced artificial intelligence, but also inspires new developments in artificial intelligence technology. The reference to the spiking neural network of the human brain provides new ideas for general artificial intelligence. The human brain contains about 86 billion neurons. To achieve artificial intelligence similar to the human brain, it is first necessary to achieve brain simulation similar to the scale of the human brain. Creating and running large-scale brain models has become an important interdisciplinary research topic. However, there is currently a lack of effective simulation systems that support fast and large-scale brain simulation, so it is currently impossible to fully utilize computing resources to achieve faster and larger-scale brain simulation.

[0003] Some brain simulation-related simulation systems have been proposed in the prior art: for example, the NEURON simulation system focuses on simulating biological neuron models, providing a powerful and flexible environment for the biological model implementation of electrical and chemical signals in neurons and neural networks, and attempts to simulate complex biological neuron models; for example, the Brain simulation system focuses on simplicity and flexibility, and it can develop related models through equations and calculation formulas, making it convenient for users to develop complex neuron models and reducing the time spent on user code writing; for example, the GeNN (GPU enhanced Neuronal Networks) simulation system implements a 4.13×10-3D-CNN model of the macaque visual cortex on a single GPU by dynamically generating synapses. 6 For example, the NEST (NEural Simulation Tool) simulation system focuses on the system dynamics and scale effects of spiking neural networks rather than complex neuron models, and can perform brain simulation on distributed clusters.

[0004] Among them, the NEURON simulation system and the Brain simulation system are mainly aimed at modeling neuron models, which facilitates researchers in related neurobiology to develop complex neuron models that are closer to biological neurons. However, these two simulation systems lack support for multi-threading and multi-process, making it difficult to perform large-scale brain simulation. Although the GeNN simulation system has achieved brain simulation at the level of millions of neurons on a single GPU by dynamically generating synapses, the GeNN simulation system only supports brain simulation on a single GPU and does not support distributed computing clusters, making it difficult to achieve larger-scale brain simulation. Although the NEST simulation system supports brain simulation on distributed clusters, the network parallelism needs to be improved, the communication between cluster computing nodes takes a long time, and there is a lack of support for resource allocation. Therefore, the NEST simulation system is difficult to fully utilize hardware resources for fast and large-scale brain simulation. Summary of the invention

[0005] The purpose of the present invention is to provide a simulation system that supports fast and large-scale brain simulation, making full use of cluster hardware resources to achieve faster and larger-scale brain simulation, solving the problems in the prior art of lack of resource scheduling and resource allocation, underutilization of device parallelism, long communication time, and limited support for hardware and interfaces.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] A simulation system supporting fast large-scale brain simulation, the simulation system supporting fast large-scale brain simulation comprising a hardware device layer, a data communication layer, an operation abstraction layer, an API layer, a hardware abstraction layer and a network abstraction layer, wherein:

[0008] The hardware device layer is used to provide multiple hardware device resources;

[0009] The data communication layer is used to provide multiple types of communication modes between computing nodes in the brain simulation network;

[0010] The operation abstraction layer is used to provide multiple types of neuron models, synapse models, connection rules and learning rules;

[0011] The API layer is used to provide a predefined API interface and receive user requirements through the API interface;

[0012] The hardware abstraction layer is used to call the corresponding hardware kernel according to the hardware device specified in the user requirement;

[0013] The network abstraction layer is used to first record the topological structure of the brain simulation network provided in the user's requirements, and perform resource allocation and neuron cluster mapping according to the information of the specified hardware device and the topological structure of the brain simulation network, and then create a brain simulation network on the corresponding hardware device according to the type of neuron model, synaptic model, connection rule, learning rule and communication method specified in the user's requirements, and finally train or execute the brain simulation network on the specified hardware device according to the user's requirements.

[0014] Several optional methods are also provided below, but they are not intended to be additional limitations on the above-mentioned overall solution, but are merely further supplements or preferences. Under the premise that there are no technical or logical contradictions, each optional method can be combined with the above-mentioned overall solution separately, and multiple optional methods can also be combined.

[0015] Preferably, the hardware device resources provided by the hardware device layer include: hardware devices of the central processing unit CPU, graphics processing unit GPU and brain-like computing unit BCU type.

[0016] Preferably, the communication mode provided by the data communication layer is determined by communication rules, communication operators and communication libraries.

[0017] Preferably, the types of the communication rules include All_Reduce, Broadcast, Reduce, All_Gather, Reduce_Scatter, All_To_All and Point_To_Point;

[0018] The communication operator is used to encode the pulse information between the computing nodes, and the encoding method is divided into dense coding and sparse coding;

[0019] The communication library supports the MPI communication library and the NCCL communication library.

[0020] Preferably, the types of the neuron model include IF neuron model, LIF neuron model, HH neuron model and IZH neuron model;

[0021] The types of the synapse models include Static_synapse static synapse, Dynamic_synapse dynamic synapse and RealTime_synapse real-time synapse;

[0022] The types of the connection rules include One_To_One connection, All_To_All connection, Fixed_Prob connection, Fixed_PreNum connection, Fixed_PostNum connection and Fixed_Total_Number connection;

[0023] The learning rule is the STDP learning rule.

[0024] Preferably, the API interface includes a device specification interface, a model networking interface and a model operation interface;

[0025] The device specifying interface is used by the user to specify the type and quantity of hardware devices;

[0026] The model networking interface is used for a user to input a topological structure of a brain simulation network, wherein the topological structure includes a neuron model of neurons in a neuron cluster, the number of neurons, and parameters of neurons, and specifies a synaptic model, a connection rule, and a type of a communication method;

[0027] The model running interface is used by the user to input training parameters or execution parameters, and the training parameters include learning rules.

[0028] Preferably, the network abstraction layer is further used to complete the initialization of the specified hardware devices according to the types and quantities of the hardware devices specified in the user requirements, and read the storage space of each hardware device.

[0029] Preferably, the resource allocation and neuron cluster mapping according to the information of the designated hardware device and the topological structure of the brain simulation network include:

[0030] According to the storage space of each hardware device and the topological structure of the brain simulation network, the closely connected neuron clusters in the brain simulation network are allocated to the same hardware device to complete resource allocation;

[0031] According to resource allocation, a process number is assigned to each neuron cluster, the neuron cluster is mapped to a specified hardware device, and proxy neurons are established for neurons connected by synapses across hardware devices.

[0032] Preferably, the step of creating a brain simulation network on a corresponding hardware device includes:

[0033] If resource allocation is not called in user needs, the brain simulation network is created on the specified hardware device according to the recorded topological structure of the brain simulation network in a polling manner; if resource allocation is called in user needs, the brain simulation network is created on the specified hardware device according to the recorded topological structure of the brain simulation network in accordance with resource allocation and neuron cluster mapping.

[0034] Preferably, when creating the brain simulation network on the corresponding hardware devices, all hardware devices create neuron clusters and synaptic connections in parallel.

[0035] Preferably, the network abstraction layer is also used to draw a pulse emission diagram and / or a membrane voltage diagram according to the training or execution results of the brain simulation network, and output the pulse emission diagram and / or the membrane voltage diagram through the API interface.

[0036] The present invention provides a simulation system that supports fast and large-scale brain simulation. The simulation system supports computer cluster systems and supports resource scheduling, allowing neuron clusters to be created and simulated on specified hardware devices. The simulation system separates model parameter / variable management and model construction to a certain extent, which is convenient for actually creating a brain simulation network on a hardware device after resource allocation is completed, rather than actually creating a brain simulation network first, then allocating resources, and creating a brain simulation network again, resulting in repeated network creation. In the resource allocation of the simulation system, closely connected neuron clusters are allocated to the same hardware device, reducing the amount of communication between hardware devices, reducing the extra consumption of synaptic connections across hardware devices, and ensuring load balancing of each hardware device, thereby achieving faster and larger-scale brain simulation. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A block diagram of a simulation system supporting fast large-scale brain simulation according to the present invention;

[0038] Figure 2 This is the pulse emission diagram output by the experiment of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0041] Existing simulation systems focus too much on the establishment of load neuron models, have limited support for large-scale brain simulation, do not fully utilize the parallelism of cluster devices, waste storage space, and take too long to communicate. Therefore, this embodiment proposes a simulation system that supports fast large-scale brain simulation. The simulation system supports computer cluster systems and resource scheduling, and neuron clusters can be created and simulated on specified hardware devices.

[0042] like Figure 1As shown, a simulation system of this embodiment that supports fast large-scale brain simulation includes: a hardware device layer, a data communication layer, an operation abstraction layer, an API layer, a hardware abstraction layer and a network abstraction layer.

[0043] The hardware device layer provides hardware device resources called by the simulation system; the data communication layer completes the mutual communication between computing nodes in the distributed cluster; the operation abstraction layer stores basic inseparable operations, including neuron models, synapse models, etc.; the API layer provides users with pre-defined related API (Application Program Interface) interfaces; the hardware abstraction layer abstracts hardware resources and calls different hardware kernels according to the hardware specified by the user; the network abstraction layer completes the encapsulation of the operation abstraction and realizes the support for the API layer. Rapid large-scale brain simulation is achieved through this simulation system. The following is a detailed description of each layer in the simulation system.

[0044] (1) Hardware device layer, used to provide multiple hardware device resources.

[0045] The hardware device resources in this embodiment can be independent hardware devices or hardware resources of a cluster system. Corresponding to the common scenarios of brain simulation, the hardware device layer of this embodiment presets hardware device resources including: hardware devices of the central processing unit (CPU), graphics processing unit (GPU) and brain-inspired computing unit (BCU).

[0046] Users can actually create brain simulation networks on different hardware devices such as CPU, GPU and BCU for brain simulation. In addition, considering the extended application of the simulation system, the hardware device layer also supports intermediate representation RI (Intermediate Representation) files for hardware expansion.

[0047] (2) Data communication layer, which is used to provide multiple types of communication methods between computing nodes in the brain simulation network. The communication methods are determined by communication rules, communication operators and communication libraries.

[0048] (2.1) Communication rules: The types of communication rules executed by the simulation system include All_Reduce, Broadcast, Reduce, All_Gather, Reduce_Scatter, All_To_All and Point_To_Point, which are used to complete the communication of pulse emission between hardware devices.

[0049] Communication rule All_Reduce: Reduce is a general term for a series of simple computing operations, and All_Reduce applies the same Reduce operation to all processes. All_Reduce operations can be completed through Reduce+Broadcast operations on a single node.

[0050] Communication rules Broadcast: The Broadcast operation is to broadcast the input of a node to other nodes. It is often used to initialize network parameters in distributed machine learning.

[0051] Communication rules Reduce: Reduce is called reduction operation, which is a general term for a series of simple operations, which can be subdivided into: SUM, MIN, MAX, PROD, LOR and other types of reduction operations. Reduce means to reduce / simplify, because its operation obtains an input element array on each process, and after executing the operation, it will get fewer simplified elements.

[0052] Communication rule All_Gather: The Gather operation collects data from multiple senders to a single node. Gather can be understood as the reverse Scatter. The Gather operation collects data from multiple processes to one process, rather than distributing data from one process to multiple processes. In many cases, sending multiple elements to multiple processes is also useful, that is, in the scenario of many-to-many communication mode, then the All_Gather operation is needed. For a set of data distributed to all processes, All_Gather will collect all data to all processes. From the most basic point of view, All_Gather is equivalent to a Gather operation followed by a Bcast operation.

[0053] Communication rule Reduce_Scatter: The Reduce_Scatter operation will first sum the input of the process, then split it by the number of cards in the 0th dimension, and distribute the data to the corresponding cards.

[0054] Communication rule All_To_All: All_To_All is a full exchange operation. Through All_To_All communication, each node can obtain the value of other nodes. When using All_To_All, each process will send a message to any process, and each process will also receive a message from any process. The receive buffer and send buffer of each process are an array divided into several data blocks. The specific operation of All_To_All is: send the jth block of data in the send buffer of process i to process j, and process j puts the data block received from process i in the ith block position of its own receive buffer.

[0055] Communication rule Point_To_Point: Point_To_Point is point-to-point communication, which is carried out between two nodes.

[0056] It is easy to understand that the above are some relatively common communication rules provided in this embodiment. In practical applications, other communication rules, such as Scatter, Gather, etc., may also be selected as needed.

[0057] (2.2) The communication operator is used to encode the pulse information between the computing nodes. The encoding methods are divided into dense coding and sparse coding. In practical applications, the encoding method can be selected to reduce the amount of data transmission and the communication time.

[0058] When the neuron pulse firing rate is high, dense coding is called. Dense coding uses binary 0 and 1 to indicate whether the corresponding neuron fires a pulse. When it is 1, it indicates that the neuron corresponding to the binary bit fires a pulse. When it is 0, it indicates that the neuron corresponding to the binary bit does not fire a pulse. When the neuron pulse firing rate is low, sparse coding is called. The index of the neuron is used to indicate whether the neuron fires a pulse. When the index of the neuron is in the data, it indicates that the neuron fires a pulse. When the index of the neuron is not in the data, it indicates that the neuron does not fire a pulse. The encoding and decoding of the pulse data is completed in the above manner, which facilitates the transmission of the pulse data.

[0059] (2.3) The communication library supports MPI (Multi Point Interface) communication library and NCCL (NVIDIA Collective Communications Library) communication library, and supports multiple communication interfaces such as infiniband ("infinite bandwidth" technology, abbreviated as IB), PCIE (bus interface, PCI-Express), and NvLink.

[0060] (3) Operation abstraction layer, which is used to provide multiple types of neuron models, synapse models, connection rules and learning rules.

[0061] (3.1) The types of neuron models include IF (Integrate-and-Fire) neuron model, LIF (leaky-Integrate-and-Fire) neuron model, HH (Hodgkin-Huxley) neuron model and IZH (Izhikevich) neuron model. In other embodiments, other types of neuron models may be introduced, such as FHN (Fitzhugh-Nagumo) neuron model, ML (Morris-Lecar) neuron model, etc., and even a custom neuron model.

[0062] (3.2) The types of synaptic models include Static_synapse, Dynamic_synapse and RealTime_synapse. Static_synapse remains unchanged during the brain simulation process; Dynamic_synapse is mainly used for model training, and the weights of synapses are constantly changing during the training process. RealTime_synapse is mainly used to reduce the consumption of storage space by synapses. Synapses are not generated in the storage space, but are generated and destroyed in real time as needed during the simulation process, thereby reducing the consumption of storage space. In RealTime_synapse, only the delay of synaptic connections is saved in advance, but the weights of synaptic connections are not saved. The weights of synaptic connections are generated on demand during the execution of the brain simulation network. And the on-demand generation can be to randomly generate weights based on the source neuron ID and target neuron ID of each synaptic connection in the pulse propagation, and the randomly generated weights are true random numbers or pseudo-random numbers.

[0063] (3.3) The types of connection rules include One_To_One connection, All_To_All connection, Fixed_Prob connection, Fixed_PreNum connection, Fixed_PostNum connection and Fixed_Total_Number connection, which are convenient for users to establish connections between neuron clusters.

[0064] The connection function performs different processing according to different connection types. When connecting in the One_To_One connection mode, the connection is made one-to-one in the order of the neuron IDs in the source neuron cluster and the target neuron cluster. When connecting in the All_To_All connection mode, all neurons in the source neuron cluster and the target neuron cluster are traversed, and each pair of neurons is connected. When connecting in the Fixed_Prob connection mode, for each postsynaptic neuron, the probability of it forming a connection with the neurons in the presynaptic neuron cluster is fixed. When connecting in the Fixed_PreNum connection mode, each neuron in the postsynaptic neuron cluster is randomly connected to a fixed number of neurons in the presynaptic neuron cluster. When connecting in the Fixed_PostNum connection mode, each neuron in the presynaptic neuron cluster is randomly connected to a fixed number of neurons in the postsynaptic neuron cluster. When connecting in the Fixed_Total_Number connection mode, the number of synaptic connections between the two neuron clusters is fixed, and a fixed number of neuron IDs are randomly selected in the source neuron cluster and the target neuron cluster, and then connected one by one.

[0065] (3.4) The learning rule is the STDP (spike-timing dependent plasticity) learning rule. The system supports STDP learning rules such as Nearest-neighbour STDP and Temporal all-to-all STDP, and completes the update of synaptic weights during training.

[0066] (4) API layer, which is used to provide a predefined API interface and receive user requirements through the API interface. The API interface of this embodiment includes a device specification interface, a model networking interface and a model operation interface.

[0067] (4.1) Device specification interface, used by users to specify the type and quantity of hardware devices, providing users with hardware resource management capabilities.

[0068] (4.2) Model networking interface, used by users to input the topology of the brain simulation network. The topology includes the neuron model of the neurons in the neuron cluster, the number of neurons and the parameters of the neurons, and specifies the synaptic model, connection rules, learning rules and the type of communication method.

[0069] When users create a neuron cluster, they need to specify the neuron model and number of neurons in the neuron cluster, and set the relevant parameters of the neurons, including resting potential, initial membrane voltage, time constant, etc. Users set the synaptic model, specifically setting the synaptic model to static synapse, dynamic synapse or real-time synapse, excitatory synapse or inhibitory synapse, specifying synapses connecting out of the neuron cluster and synapses connecting to the neuron cluster, and specifying the connection rules. At the same time, it is necessary to configure the relevant parameters of the connection rules. The synaptic connection in the model network supports the slicing function. Users can specify that a part of the neuron cluster is connected to a part of another neuron cluster, and also support a part of the neuron cluster to be connected to a part of the neuron cluster itself; users specify the external input of the model, whether it is a DC input or a Poisson input, and set the input parameters. At the same time, it is necessary to specify the communication rules, communication operators and communication libraries in the communication mode. In addition, users can specify whether to call resource allocation.

[0070] (4.3) Model operation interface, used by users to input training parameters or execution parameters. If the model needs to be trained, users can set the model's learning rules, training parameters, training time and other training parameters according to their needs to train the model; if the model does not need to be trained, users can set the model execution time and other execution parameters to start brain simulation directly.

[0071] The training time here refers to the number of training times (the number of training cycles). The execution time also refers to the number of execution times (the number of execution cycles), which has a corresponding biological time of brain simulation, for example, one execution corresponds to 0.1ms of biological time.

[0072] In addition, in order to visualize the model simulation results, the API layer also provides a result display and saving interface, through which the model simulation results can be output and displayed. At the same time, in order to facilitate the application of the simulation system, the simulation system of this embodiment can also pre-import the classic brain model previously established by relevant researchers, and set up an existing model library including Microcircuit model, Macaque model, etc., so that users can directly call the existing brain simulation model for training or execution through the model networking interface.

[0073] (5) Hardware abstraction layer, which is used to call the corresponding hardware kernel according to the hardware device specified in the user's requirements. The hardware abstraction layer mainly consists of: CPU_Kernel CPU kernel, GPU_Kernel GPU kernel, BCU_Kernel brain-like chip kernel, IR (Intermediate Representation), which corresponds to the hardware device resources of the hardware device layer.

[0074] The hardware abstraction layer calls different kernels to create and simulate models according to the hardware device type specified by the user. In addition, the IR framework supports the extension of different types of hardware and can generate IR files to save the network built by the user, which is convenient for model creation and simulation on different types of hardware. IR is a hardware-independent intermediate representation. If hardware extension is required, other hardware can parse IR.

[0075] (6) Network abstraction layer, which mainly includes device management module, model parameter / variable management module, resource allocation module, model construction module, executor, trainer, and drawing library.

[0076] (6.1) Device management module: completes the initialization of hardware devices and reads the storage space of hardware devices according to the hardware device type and number specified by the user in the API layer device specification.

[0077] (6.2) Model parameter / variable management module: records and saves the topological structure and input parameters of the brain simulation network defined by the user in the API layer model networking interface. This module completes the recording and saving of the respective network parameters, but does not actually create the brain simulation network on the hardware device. Network creation is very time-consuming. The advantage of doing this is that it avoids the brain simulation network being created on the hardware device before resource allocation, and then re-creating it after resource allocation is completed, resulting in a waste of resources. At the same time, it is also convenient to pass the relevant parameters of the network to the resource allocation module to ensure that resource allocation is completed before the brain simulation network is created on the hardware device.

[0078] (6.3) Resource allocation module: runs resource allocation algorithm and resource scheduling to perform resource allocation and neuron cluster mapping based on the information of specified hardware devices and the topological structure of the brain simulation network.

[0079] The resource allocation algorithm allocates closely connected neuron clusters to the same hardware device according to the topological structure of the user-defined brain simulation network and the device storage space, reduces the communication volume between hardware devices, reduces the extra consumption of synaptic connections across hardware devices, and ensures the load balancing of each hardware device, thereby achieving faster and larger-scale brain simulation. The resource allocation algorithm running in this embodiment is an existing method, such as the resource allocation method mentioned in the Chinese patent document with reference application number CN2022116912007.

[0080] After running the resource allocation algorithm, the neuron clusters are respectively allocated to the hardware devices specified by the user. During resource scheduling, the process number of the allocated hardware device is specified for each neuron cluster, and the neuron cluster is mapped to the corresponding hardware device, so that the neuron cluster is created and simulated on the specified hardware device. In addition, proxy neurons are established for neurons connected by synapses across hardware devices to facilitate pulse transmission across devices during simulation. Resource scheduling has two main functions. The first is to facilitate the system's support for computer clusters (multiple hardware devices perform brain simulation at the same time), and the second is to support the creation of specified neuron clusters by specified devices to facilitate the actual creation of brain simulation networks based on the results of the resource allocation algorithm.

[0081] (6.4) Model construction module: Create a brain simulation network on the corresponding hardware device according to the types of neuron models, synaptic models, connection rules, learning rules, and communication methods specified in the user's requirements. This embodiment separates model parameter / variable management and model construction to a certain extent, making it convenient to actually create a brain simulation network on the hardware device after resource allocation is completed.

[0082] When actually creating a brain simulation network, if the user does not call resource allocation, the network is actually created on the hardware device specified by the user in a polling manner according to the network parameters saved in the model parameter / variable management module; if the user calls resource allocation, the network is actually created on the device specified by the user according to the result of resource allocation and the network parameters saved in the model parameter / variable management.

[0083] When actually creating a brain simulation network, all devices create neuron clusters in parallel. For any hardware device a, assuming that there are k neuron clusters on the hardware device, k threads are enabled to create k neuron cluster matrices. The neuron cluster matrix is ​​a two-dimensional matrix of pi*qi, where pi represents the number of neurons contained in the i-th neuron cluster, and qi represents the number of parameters contained in each neuron. Pi*qi threads are enabled to complete the initialization of the matrix value. And all hardware devices create synaptic connections in parallel. For any hardware device a, the total number of synapses s to be created is determined according to the connection rules and parameters, s threads are enabled to complete the synapse creation, and a synaptic connection matrix is ​​generated according to the synapses. This embodiment quickly completes model construction through multi-threaded parallelism, completes the actual creation of the brain simulation network on the hardware device, and the system makes full use of parallelism, enabling multiple processes and multiple threads at the same time to speed up brain simulation.

[0084] (6.5) Executor: Relying on the brain simulation network that has been actually created, brain simulation is started according to the execution time set by the user, and the number of processes of the hardware devices specified by the user are enabled. All hardware devices perform brain simulation in parallel. For any hardware device a, the total number of threads of the neurons of the hardware device is enabled to update the neuron parameters; the neuron parameters are updated to determine whether the neuron emits a pulse. For all neurons that emit a pulse, the synaptic connection matrix is ​​retrieved to determine the neurons to which the synapses are connected. The synapses transmit the pulses emitted by the neurons, and the brain simulation is repeated in a cycle.

[0085] (6.6) Trainer: Train the brain simulation network according to the learning rules, training parameters, and training time selected by the user, and update the synaptic weights.

[0086] (6.7) Drawing library: used to draw pulse emission diagrams and / or membrane voltage diagrams according to the training or execution results of the brain simulation network, and support the output of the pulse emission diagrams and / or membrane voltage diagrams through the result display and storage interface in the API interface, so that the simulation system of this embodiment supports sending the training or execution results to the web page in real time and displaying the simulation status on the web page.

[0087] In order to further illustrate the advantages of the simulation system supporting rapid large-scale brain simulation provided by the present invention, a specific example is provided below for illustration.

[0088] The established simulation system was used to conduct a large-scale brain simulation of 910 million scales on a computer cluster with 22 hardware devices (NVIDIA A100) and simulate four seconds of biological time. The information of each layer in the simulation system was set as follows:

[0089] Hardware device layer: The hardware device is 22 NVIDIA A100 GPUs.

[0090] Data communication layer: The communication rule is Point_To_Point; the communication operator is sparse coding; the communication library is the MPI library.

[0091] Operation abstraction layer: The neuron model is the LIF neuron model; the synapse model is the Static_synapse static synapse; the connection rule is the Fixed_PostNum connection.

[0092] API layer: device specification interface, specify the hardware device as GPU, the number of hardware devices as 22; model networking interface, create 20,000 neuron clusters, the neuron model is the LIF neuron model, and set the relevant parameters of the LIF neuron. Establish connections between neuron clusters, the connection rule is Fixed_PostNum, the synapse model is Static_synapse, set relevant parameters, set call resource allocation; model running interface, the model does not need to be trained, and the execution time is set to 4 seconds. Result display and saving interface, save the pulse emission data and pulse emission diagram of the specified neuron cluster.

[0093] Hardware abstraction layer: calls the GPU_Kernel kernel.

[0094] Network abstraction layer: Device management module, initializes 22 NVIDIA A100 GPUs according to the hardware devices specified by the user, and reads the device storage space information; model parameter / variable management module records, saves and manages the user's brain simulation network; resource allocation algorithm allocates resources according to the device storage space information and brain simulation network information to obtain resource allocation results; resource scheduling maps neuron clusters to corresponding hardware devices according to the resource allocation results; model construction module actually creates a brain simulation network on the hardware device; the executor enables multiple processes and multiple threads to start brain simulation; the drawing library draws the pulse emission diagram.

[0095] The pulse emission diagram of the simulation system output in the experiment is shown in the figure Figure 2 As shown. Figure 2 It can be seen that, relying on the designed simulation system, 22 NVIDIA A100 GPUs can perform large-scale brain simulation of 910 million scales, and it only takes 360.692 seconds to simulate four seconds of biological time. This shows that the simulation system designed by the present invention can well support fast large-scale brain simulation.

[0096] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0097] The above-mentioned embodiments only express several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A simulation system supporting rapid large-scale brain simulation, characterized in that: The simulation system supporting fast large-scale brain simulation includes a hardware device layer, a data communication layer, an operation abstraction layer, an API layer, a hardware abstraction layer and a network abstraction layer, wherein: The hardware device layer is used to provide multiple hardware device resources; The data communication layer is used to provide multiple types of communication modes between computing nodes in the brain simulation network; The operation abstraction layer is used to provide multiple types of neuron models, synapse models, connection rules and learning rules; The API layer is used to provide a predefined API interface and receive user requirements through the API interface; The hardware abstraction layer is used to call the corresponding hardware kernel according to the hardware device specified in the user requirement; The network abstraction layer is used to first record the topological structure of the brain simulation network provided in the user's requirements, and perform resource allocation and neuron cluster mapping according to the information of the specified hardware device and the topological structure of the brain simulation network, and then create a brain simulation network on the corresponding hardware device according to the type of neuron model, synaptic model, connection rule, learning rule and communication method specified in the user's requirements, and finally train or execute the brain simulation network on the specified hardware device according to the user's requirements.

2. The simulation system supporting rapid large-scale brain simulation according to claim 1, characterized in that: The hardware device resources provided by the hardware device layer include: central processing unit CPU, graphics processing unit GPU and brain-like computing unit BCU type hardware devices.

3. The simulation system supporting rapid large-scale brain simulation as claimed in claim 1, characterized in that: The communication mode provided by the data communication layer is determined by communication rules, communication operators and communication libraries.

4. The simulation system supporting rapid large-scale brain simulation as claimed in claim 3, characterized in that: The types of communication rules include All_Reduce, Broadcast, Reduce, All_Gather, Reduce_Scatter, All_To_All and Point_To_Point; The communication operator is used to encode the pulse information between the computing nodes, and the encoding method is divided into dense coding and sparse coding; The communication library supports the MPI communication library and the NCCL communication library.

5. The simulation system supporting rapid large-scale brain simulation according to claim 1, characterized in that: The types of the neuron models include IF neuron model, LIF neuron model, HH neuron model and IZH neuron model; The types of the synapse models include Static_synapse static synapse, Dynamic_synapse dynamic synapse and RealTime_synapse real-time synapse; The types of the connection rules include One_To_One connection, All_To_All connection, Fixed_Prob connection, Fixed_PreNum connection, Fixed_PostNum connection and Fixed_Total_Number connection; The learning rule is the STDP learning rule.

6. The simulation system supporting rapid large-scale brain simulation according to claim 1, characterized in that: The API interface includes a device specification interface, a model networking interface and a model operation interface; The device specifying interface is used by the user to specify the type and quantity of hardware devices; The model networking interface is used for a user to input a topological structure of a brain simulation network, wherein the topological structure includes a neuron model of neurons in a neuron cluster, the number of neurons, and parameters of neurons, and specifies a synaptic model, a connection rule, and a type of a communication method; The model running interface is used by the user to input training parameters or execution parameters, and the training parameters include learning rules.

7. The simulation system supporting rapid large-scale brain simulation according to claim 1, characterized in that: The network abstraction layer is also used to complete the initialization of the specified hardware devices according to the types and quantities of the hardware devices specified in the user requirements, and read the storage space of each hardware device.

8. The simulation system supporting rapid large-scale brain simulation according to claim 7, characterized in that: The resource allocation and neuron cluster mapping according to the information of the designated hardware device and the topological structure of the brain simulation network include: According to the storage space of each hardware device and the topological structure of the brain simulation network, the closely connected neuron clusters in the brain simulation network are allocated to the same hardware device to complete resource allocation; According to resource allocation, a process number is assigned to each neuron cluster, the neuron cluster is mapped to a specified hardware device, and proxy neurons are established for neurons connected by synapses across hardware devices.

9. The simulation system supporting rapid large-scale brain simulation according to claim 1, characterized in that: The step of creating a brain simulation network on a corresponding hardware device includes: If resource allocation is not called in user needs, the brain simulation network is created on the specified hardware device according to the recorded topological structure of the brain simulation network in a polling manner; if resource allocation is called in user needs, the brain simulation network is created on the specified hardware device according to the recorded topological structure of the brain simulation network in accordance with resource allocation and neuron cluster mapping.

10. The simulation system supporting rapid large-scale brain simulation according to claim 1, characterized in that: When the brain simulation network is created on the corresponding hardware devices, all the hardware devices create neuron clusters and synaptic connections in parallel.

11. The simulation system supporting rapid large-scale brain simulation according to claim 1, characterized in that: The network abstraction layer is also used to draw a pulse emission diagram and / or a membrane voltage diagram according to the training or execution results of the brain simulation network, and output the pulse emission diagram and / or the membrane voltage diagram through the API interface.

Images