A whole-process parallel acceleration brain simulation method and system

Abstract

The application discloses a full-process parallel acceleration brain simulation method and system, and is applied to a GPU, wherein the full-process parallel acceleration brain simulation method comprises the following steps: receiving cluster parameters and synapse parameters input by a user; creating a cluster table in parallel according to the cluster parameters; creating a routing table in parallel according to the synapse parameters; creating a pulse emission matrix and a pulse receiving matrix for each neuron; and performing brain simulation based on the cluster table, the routing table and the pulse emission matrix and the pulse receiving matrix. The application provides a parallel processing possibility and is more suitable for the many-core characteristics of the GPU.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of brain simulation, and particularly relates to a full-process parallel acceleration brain simulation method and system. BACKGROUND

[0002] Brain simulation is a technology for simulating the operation mechanism of a biological brain by software means, and is a necessary means for researching future general intelligence, brain disease diagnosis based on a digital twin brain and other new technologies. Compared with current mainstream deep learning networks, a brain simulation network has the characteristic of extremely large scale. In mainstream deep learning networks, such as a VIT model in the visual field, generally has a parameter quantity of 100 to 200 million, a Galactica model proposed by Meta in 2022 in the language field has a parameter quantity of 120 billion, and a GPT-3 language model proposed by OpenAI in 2020 has a parameter quantity of 170 billion. In the field of brain simulation, the human brain contains about 8.6 billion neurons, and the number of synapses reaches 100 trillion, and the parameter quantity of a simulated neural network in software will reach the order of magnitude of 10 million.

[0003] A traditional brain simulation network needs to rely on a supercomputer to complete. To achieve simulation close to the actual running speed of a biological brain, the simulation system needs to have super-high parallelism and communication capability. In recent years, GPU has been popularized in the field of dynamics-related computing. GPU has the characteristic of many cores, and compared with the single-core processing capability emphasized by CPU, GPU emphasizes increasing the number of processing units on a single graphics card. The A100 graphics card launched by NVIDIA in 2021 has 108 SM processing units, can provide 221184 independent computing threads, and has much higher parallelism than CPU. The many-core characteristic of GPU is suitable for the deployment of a large number of neurons and synapses in a brain simulation network.

[0004] In the European Human Brain Project, the NEST brain simulation system commonly used by supercomputers uses a CPU core as the smallest processing unit, and its parallel processing capability is limited by the number of CPU cores. Due to the communication limitation between CPUs, it cannot be quickly parallelized like GPU under the same processing core condition. NEST supports using GPU to simulate the network in 2022, but cannot support full-process parallelization, and a large number of processes need to be completed by CPU and memory. There are problems such as slow network creation speed. GeNN simulation system was proposed by the University of Sussex in the United Kingdom in 2018 and was updated to version 4.8 in 2022. However, compared with the GPU version of NEST launched in the same year, GeNN has slower simulation speed using GPU, and creating a network needs code generation and other cumbersome steps, and the overall execution efficiency is lower than that of NEST.

[0005] Because of numerous parameters and complex structure, it is difficult to support GPU parallel in the whole process of creating and executing the network. At present, the mainstream GPU brain simulation system can only support parallel in neuron update and pulse transmission, but the network creation step becomes complex. The network creation of one billion neurons often takes a day. To solve the problem of brain simulation without full-process parallel, a set of brain simulation network data structure adapting to the characteristics of GPU many-core is needed to support the rapid creation of network and simulation by using multi-thread and memory features. The main problems are:

[0006] The data structure in the brain simulation system cannot fully utilize the many-core characteristics of the GPU. The current brain simulation data structure on the GPU side is often based on the CPU version for transplantation, without designing a complete structure supporting network creation, neuron update and pulse processing in the whole process based on the GPU, resulting in the thread usage rate of the GPU not reaching the best. SUMMARY

[0007] One of the purposes of the present application is to provide a full-process parallel acceleration brain simulation method, which provides parallel processing possibilities and is more suitable for the many-core characteristics of the GPU.

[0008] To achieve the above purpose, the technical solution adopted by the present application is:

[0009] A full-process parallel acceleration brain simulation method applied to a GPU, the full-process parallel acceleration brain simulation method comprising:

[0010] Receiving cluster parameters and synapse parameters input by a user;

[0011] Creating a cluster table in parallel according to the cluster parameters;

[0012] Creating a routing table in parallel according to the synapse parameters;

[0013] Creating a pulse emission matrix and a pulse reception matrix for each neuron in parallel;

[0014] Performing brain simulation based on the cluster table, the routing table, and the pulse emission matrix and the pulse reception matrix.

[0015] The following also provides several optional modes, but not as an additional limitation to the above overall scheme, just a further supplement or preferred, without technical or logical contradiction, each optional mode can be combined with the above overall scheme, and can also be combined between multiple optional modes.

[0016] As a preferred, the cluster parameters include the total number of clusters, the number of neurons in each cluster, the number of neuron parameters, and the initial parameters of neurons;

[0017] The synapse parameters include a total number of synapse information, and each piece of synapse information contains a source cluster index, a target cluster index, a connection rule, a weight distribution, and a time delay distribution.

[0018] Preferably, the creating a cluster table according to the cluster parameters comprises:

[0019] The cluster parameters are a table with a length of L, L being a total number of clusters, and the i-th element of the table representing information of the i-th cluster, including a number of neurons M_i, a number of neuron parameters N_i, and neuron initial parameters;

[0020] A cluster table with a length of L is initialized, L threads are started to initialize each element in the cluster table in parallel, and the i-th element of the cluster table represents a cluster matrix of the i-th cluster, the cluster matrix of the i-th cluster having a first dimension with a length of N_i and a second dimension with a length of M_i.

[0021] L cluster matrices are processed in parallel by starting L threads, and M_i threads are started in the i-th thread to fill neuron initial parameters of M_i neurons of the i-th cluster into the cluster matrix of the i-th cluster.

[0022] Preferably, the creating a routing table according to the synapse parameters in parallel comprises:

[0023] Temporary source neuron tables, temporary target neuron tables, temporary weight tables, and temporary time delay tables corresponding to each two clusters are created according to the synapse parameters.

[0024] The maximum time delay value of each neuron and the number of synapses as a source neuron are counted according to the temporary neuron tables and the temporary time delay tables.

[0025] A one-dimensional routing table initialized as empty is initialized for each cluster, the routing table having a length of the number of neurons in the cluster, and an element in the routing table being a one-dimensional target address table having a length of the maximum time delay value of the corresponding neuron, and the element in the target address table being a [target neuron index, weight] key-value pair.

[0026] The routing table is filled according to the temporary source neuron tables, the temporary target neuron tables, the temporary weight tables, and the temporary time delay tables.

[0027] Preferably, the creating temporary source neuron tables, temporary target neuron tables, temporary weight tables, and temporary time delay tables corresponding to each two clusters according to the synapse parameters comprises:

[0028] A number of threads equal to a total number of synapse information in the synapse parameters are started, and all threads process all synapse information in parallel, and each thread has a process as follows:

[0029] obtaining corresponding synaptic information, the synaptic information including source cluster index, target cluster index, connection rule, weight distribution and time delay distribution;

[0030] calculating the number of synapses s_c needed to be created between two clusters according to the connection rule;

[0031] generating a temporary source neuron table with length of s_c and a temporary target neuron table with length of s_c by random number generator, the elements in the temporary source neuron table are the neuron indexes contained in the source cluster index, the elements in the temporary target neuron table are the neuron indexes contained in the target cluster index, meanwhile, generating a temporary weight table with length of s_c and a temporary time delay table with length of s_c by random number generator according to the weight distribution and the time delay distribution.

[0032] As a preferred, the filling the routing table according to the temporary source neuron table, the temporary target neuron table, the temporary weight table and the temporary time delay table comprises:

[0033] starting a same number of parallel threads as the total number of synapses, wherein the s th thread handles the process as follows: finding the temporary source neuron table, the temporary target neuron table, the temporary weight table and the temporary time delay table with the starting index less than and closest to s;

[0034] taking the element with index [s-starting index] in the temporary source neuron table as the index of the routing table to obtain the target address table, taking the element with index [s-starting index] in the temporary time delay table as the first dimension index value of the target address table, and writing the elements with index [s-starting index] in the temporary weight table and the temporary target neuron table into the target address table as [target neuron index, weight] key-value pairs.

[0035] As a preferred, the creating the pulse emission matrix and the pulse reception matrix for each neuron comprises:

[0036] starting a same number of parallel threads as the total number of clusters, and initializing a one-dimensional pulse reception matrix corresponding to a cluster for each thread, the first dimension of the pulse reception matrix has a length of the number of neurons of the cluster, and the element is a pulse number accumulation value with an initial value of 0;

[0037] starting a same number of parallel threads as the total number of neurons, and initializing a two-dimensional pulse emission matrix corresponding to a neuron for each thread, the first dimension of the pulse emission matrix has a length of the maximum time delay value of the neuron, the second dimension has a length of the number of synapses of the neuron as a source neuron, and the element is a [target neuron index, weight accumulation value] key-value pair, the target neuron index in the key-value pair is the index of the target neuron connected by the neuron as a source neuron, and the weight accumulation value in the key-value pair has an initial value of 0.

[0038] As preferred, the brain simulation based on the cluster table, the routing table and the pulse sending matrix and the pulse receiving matrix comprises:

[0039] 1) initializing the current time step as 0;

[0040] 2) starting M threads to collect pulses in parallel, taking the jth element in the ith pulse receiving matrix as the pulse received by the jth neuron in the ith cluster at the current time step, and emptying the receiving matrix, M being the total number of neurons;

[0041] 3) starting M parallel threads, each of which updates the neuron parameters of the corresponding neuron in the cluster table, and judges whether the current membrane voltage in each updated neuron parameter is greater than the threshold value, if there is a neuron whose current membrane voltage is greater than the threshold value, it indicates that the neuron needs to send a pulse and step 4) is executed, otherwise step 6) is executed;

[0042] 4) for the jth neuron of the ith cluster which needs to send a pulse, querying the jth target address table of the ith routing table, taking out all key-value pairs, taking the target address table index as the time delay value, and combining the target neuron index and the weight in the pulse information [time delay value, target neuron index, weight];

[0043] 5) filling the pulse information into the pulse sending matrix corresponding to the jth neuron of the ith cluster, taking the time delay value in the pulse information as the first dimension index of the pulse sending matrix to obtain s_j elements, determining the element with the same target neuron index in the obtained s_j elements according to the target neuron index in the pulse information, and adding the weight in the pulse information to the weight accumulation value in the key-value pair of the determined element, and adding the weights in the multiple pulse information corresponding to the same target address table to the same pulse sending matrix;

[0044] 6) starting M parallel threads, each of which starts s_j*max_delay_j parallel threads,

[0045] each of the s_j*max_delay_j parallel threads corresponds to an element of the pulse sending matrix, if the element processed by the thread is the element with the first dimension index of 0 in the pulse sending matrix, the weight accumulation value of the processed element is added to the element with the first dimension index of the target neuron index in the pulse receiving matrix, and then the weight accumulation value in the element is set to 0, in addition, each thread moves the processed element one bit forward, finally, if the element processed by the thread is the element with the first dimension index of (max_delay_j)-1 in the pulse sending matrix, the weight accumulation value in the element is set to 0, wherein max_delay_j is the first dimension length of the pulse receiving matrix, and s_j is the second dimension length of the pulse receiving matrix;

[0046] 7) current time step is increased by 1;

[0047] 8) determine whether the current time step is equal to the simulation time step, if not, execute step 2) to continue the simulation, if yes, end the flow.

[0048] The application provides a full-process parallel acceleration brain simulation method, supports creation of a cluster table and a routing table in a parallel manner, has neuron and synapse data structures adapted to GPU many-core characteristics, so that fast parameter addressing and updating processes are realized based on the structures.

[0049] The second object of the application is to provide a full-process parallel acceleration brain simulation system, which provides parallel processing possibilities and is more suitable for GPU many-core characteristics.

[0050] To achieve the above object, the technical scheme adopted by the application is:

[0051] A full-process parallel acceleration brain simulation system, comprising a memory, a video memory and a GPU, wherein:

[0052] The memory is used to store cluster parameters and synapse parameters input by a user;

[0053] The GPU is used to create a cluster table in the video memory according to the cluster parameters, create a routing table in the video memory according to the synapse parameters, and create a pulse emission matrix and a pulse receiving matrix for each neuron in the video memory, and then perform brain simulation based on the cluster table, the routing table and the pulse emission matrix and the pulse receiving matrix.

[0054] Preferably, the full-process parallel acceleration brain simulation system further comprises a parameter input interface and a running command interface;

[0055] The parameter input interface is used to receive the cluster parameters and the synapse parameters input by the user and forward them to the memory for storage;

[0056] The running command interface is used to receive a running command and a simulation time step input by the user, and the GPU starts running after recognizing the running command.

[0057] The application provides a full-process parallel acceleration brain simulation system, supports creation of a cluster table and a routing table in a parallel manner, has neuron and synapse data structures adapted to GPU many-core characteristics, so that fast parameter addressing and updating processes are realized based on the structures. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 A flow chart of a whole-process parallel acceleration brain simulation method of the present application;

[0059] Figure 2 A structural schematic diagram of a whole-process parallel acceleration brain simulation system of the present application. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terminology used in the description of the present application herein only for the purpose of describing specific embodiments and is not intended to limit the present application.

[0062] In order to solve the problem that the data structure taking neuron as the minimum unit does not match the many-core characteristics of GPU, resulting in complex network creation, parameter indexing and low parallel efficiency. At the same time, in order to solve the problem of low parallel efficiency in the main processes such as membrane voltage updating, routing addressing and pulse transmission process in brain simulation. The embodiment proposes a whole-process parallel acceleration brain simulation method, which realizes the complete expression of brain simulation on GPU, including the data structure related to neuron, synapse and pulse processing, and realizes the complete process of fast creation of these data structures on GPU memory, while providing a complete simulation process for the data structure, including neuron parameter indexing and updating, pulse routing information lookup, pulse data storage, emission and reception, etc., to realize parallel acceleration simulation based on GPU.

[0063] For the convenience of understanding, first, the terms used in the present application are explained:

[0064] Brain simulation system: a software system that provides users with basic interfaces for creating brain simulation networks, including interfaces for creating neuron clusters, inter-cluster synapses, simulation and result query, and calling hardware resources such as GPU to complete network deployment and simulation.

[0065] Neuron: a simulated neuron that imitates the actual operation mechanism of biological brain neurons, containing neuron parameters such as membrane voltage, threshold, membrane time constant and conductance, and parameter update logic, and is the smallest independent calculation unit in the simulation system, each neuron has its own unique index value.

[0066] Neuron cluster (cluster): a collection of multiple neurons, generally containing tens of thousands of neurons in one cluster, is the smallest unit of brain simulation network created by the user through the simulation system, and each cluster has its own unique index value.

[0067] Synapse: the connection between neurons, the user inputs the number of synapses between clusters, connection rules, weight distribution values and other parameters through the simulation system, and the simulation system converts the distribution values into actual values when deploying the network and creates actual inter-neuron connections.

[0068] Delay: a synapse parameter representing the time taken for a pulse to propagate over the synapse, measured in time steps.

[0069] Time step: a discrete time unit used by the simulation system to simulate real time during the simulation process, and one time step can represent 0.1 milliseconds or 1 milliseconds of real time according to the simulation accuracy. In each time step, the parameters of the entire network will be updated once.

[0070] Thread: refers to GPU thread in this context.

[0071] As shown in Figure 1 , the full-process parallel acceleration brain simulation method of the embodiment includes the following steps:

[0072] S1, receiving user input cluster parameters and synapse parameters.

[0073] Cluster parameters: usually represented as a table with a length of L, representing L clusters, i.e. L is the total number of clusters, and the i-th element in the table contains information: the number of neurons M_i in cluster i, the number of neuron parameters N_i, and the initial neuron parameters.

[0074] Wherein, the initial neuron parameters are the initial membrane voltage, time constant and other numerical values input by the user, and the user generally inputs information in units of clusters, i.e. the initial neuron parameters in the cluster are consistent.

[0075] Synapse parameters: usually represented as a table with a length of P, each element in the table represents synapse information between two clusters, P is the total number of synapse information, and each synapse information contains: source cluster index, target cluster index, connection rule, weight distribution, and delay distribution.

[0076] S2, creating a cluster table according to the cluster parameters.

[0077] S2.1, obtaining a cluster parameter table with a length of L, L is the total number of clusters, and the i-th element in the table represents the information of the i-th cluster, including the number of neurons M_i, the number of neuron parameters N_i, and the initial neuron parameters.

[0078] S2.2, start a thread to initialize a cluster table with length L, and start L threads to initialize each element in the cluster table in parallel, the i-th element in the cluster table represents the cluster matrix of the i-th cluster (also called cluster matrix i), the first dimension of the cluster matrix of the i-th cluster has length N_i, and the second dimension has length M_i. The total number of neurons is denoted as M.

[0079] S2.3, start L threads to process each cluster matrix in the cluster table in parallel, and start M_i threads in the i-th thread to fill the neuron initial parameters of the M_i neurons of the i-th cluster into the cluster matrix of the i-th cluster in parallel. Usually the neuron initial parameters in the cluster are consistent, so for each thread in the M_i threads, read the element with index k in the temporary parameter array storing N_i parameters, and fill it into the element with index k in the first dimension and index corresponding to the thread index in the second dimension of the i-th matrix.

[0080] S3, create a routing table in parallel according to the synaptic parameters.

[0081] S3.1, create a temporary source neuron table, a temporary target neuron table, a temporary weight table and a temporary delay table corresponding to each two clusters according to the synaptic parameters.

[0082] Initialize the total number of synapses S to 0. Start P threads, all threads process all synaptic information in parallel, where the c-th thread obtains the synaptic information with index c in the synaptic parameters. Each thread calculates the number of synapses s_c that need to be created between two clusters according to the connection rules in the obtained synaptic information, and accumulates the number of synapses of all threads to obtain S.

[0083] Generate a temporary source neuron table with length s_c and a temporary target neuron table with length s_c by a random number generator, the elements in the temporary source neuron table are the neuron indexes contained in the source cluster index, and the elements in the temporary target neuron table are the neuron indexes contained in the target cluster index. The random number generator generates random numbers according to the source cluster index and the target cluster index, and the generated random numbers are within the range of the neuron indexes contained in the source cluster index and the target cluster index.

[0084] Each thread generates a temporary weight table with length s_c and a temporary delay table with length s_c by a random number generator according to the weight distribution and the delay distribution. A total of 4*P temporary tables are generated for each synaptic information, and the starting index of each table is recorded. The random number generator generates random numbers according to the weight distribution and the delay distribution, and the generated random numbers are within the range of the weight distribution and the delay distribution.

[0085] S3.2, according to the temporary neuron table and the temporary delay table, count the maximum delay value of each neuron and the number of synapses as a source neuron.

[0086] S3.1, initialize temporary source neuron table and temporary delay table: start S parallel threads, each thread queries one element in temporary source neuron table and temporary delay table, and start one thread to count the maximum delay value max_delay_j and the number of synapses s_j for each neuron. Here max_delay_j and s_j represent the maximum delay value and the number of synapses for the jth neuron in the ith cluster. To simplify the calculation, the neuron index in each cluster is continuous, that is, the jth neuron in the ith cluster refers to the same neuron as the jth neuron in the total neurons.

[0087] S3.3, initialize routing table: start L parallel threads, each thread initializes a one-dimensional routing table with empty elements for each cluster, the length of the routing table is the number of neurons in the cluster, and the elements in the routing table are one-dimensional target address tables, the length of the target address table is the maximum delay value of the corresponding neuron, that is, for the jth target address table, the table length is max_delay_j of the jth neuron, and the elements in the target address table are [target neuron index, weight] key-value pairs.

[0088] S3.4, fill in the routing table according to the temporary source neuron table, temporary target neuron table, temporary weight table and temporary delay table.

[0089] Start S parallel threads, where the s th thread processes as follows: find the temporary source neuron table, temporary target neuron table, temporary weight table and temporary delay table whose starting index is less than and closest to s.

[0090] Take the element in the temporary source neuron table with index [s-starting index (here, the starting index of the temporary source neuron table found)] as the index of the routing table to get the target address table, take the element in the temporary delay table with index [s-starting index (here, the starting index of the temporary delay table found)] as the first dimension index value of the target address table, and write the elements in the temporary weight table and temporary target neuron table with index [s-starting index (here, the starting index of the temporary weight table and temporary target neuron table found, respectively)] into the target address table as [target neuron index, weight] key-value pairs.

[0091] S4, create a pulse firing matrix and a pulse receiving matrix for each neuron.

[0092] Pulse receiving matrix (referred to as receiving matrix): used to store the pulses received by the neuron, there are L receiving matrices. Start L parallel threads, for the ith thread, initialize a one-dimensional receiving matrix corresponding to the ith cluster, the first dimension length is the number of neurons in the ith cluster M_i, that is, the index of the first dimension is the neuron index, and the element is the pulse number cumulative value, the initial value is 0.

[0093] Firing matrix (referred to as firing matrix for short): used for storing the pulses to be fired by neurons, and there are M firing matrices. M parallel threads are started, and for the jth thread, a two-dimensional firing matrix corresponding to the jth neuron is initialized, the first dimension has a length of the maximum delay value max_delay_j of the neuron, the second dimension has a length of the number of synapses s_j of the neuron as a source neuron, and the element is a [target neuron index, weight cumulative value] key-value pair. The target neuron index in the key-value pair is the index of the target neuron connected by the neuron as a source neuron, and the weight cumulative value in the key-value pair is initially 0.

[0094] S5, brain simulation based on the cluster table, the routing table, and the pulse firing matrix and the pulse receiving matrix.

[0095] S5.1, initialize the current time step to 0.

[0096] S5.2, start M threads to collect pulses in parallel, take the jth element in the ith receiving matrix as the pulse received by the jth neuron in the ith cluster at the current time step, and empty the receiving matrix.

[0097] S5.3, start M parallel threads, and each thread updates the neuron parameters of the corresponding neuron in the cluster table. It is judged whether the current membrane voltage in each updated neuron parameter is greater than the threshold value. If there is a neuron whose current membrane voltage is greater than the threshold value, it indicates that the neuron needs to fire a pulse and step S5.4 is performed, otherwise step S5.6 is performed.

[0098] It should be noted that the main update of the neuron parameters of the corresponding neuron in the cluster table is to update the membrane voltage value of the neuron in the cluster table according to the parameter update logic of the neuron, and the membrane voltage is a conventional technique in the update of the neuron pulse transmission parameter, which will not be described in detail in this embodiment.

[0099] S5.4, for the jth neuron of the ith cluster that needs to fire a pulse, query the jth target address table of the ith routing table, take out all key-value pairs, take the target address table index as the delay value, and form [delay value, target neuron index, weight] pulse information with the corresponding key-value pair.

[0100] If there are multiple key-value pairs in the target address table, multiple pulse information is finally obtained, the delay value in each pulse information is the target address table index, and the target neuron index and the weight in the pulse information are the elements under the target address table index, that is, the [target neuron index, weight] key-value pair.

[0101] S5.5, fill the pulse information into the sending matrix corresponding to the jth neuron of the ith cluster, take the time delay value in the pulse information as the first dimension index of the sending matrix to obtain s_j elements, determine the element with the same target neuron index from the s_j elements according to the target neuron index in the pulse information, accumulate the weight in the pulse information to the weight accumulation value in the key-value pair of the determined element, and accumulate the weights in the multiple pulse information corresponding to the same target address table to the same sending matrix.

[0102] S5.6, start M parallel threads, each thread starts s_j*max_delay_j threads, each thread in the s_j*max_delay_j parallel threads corresponds to an element of the pulse sending matrix, if the element processed by the thread is the element with the first dimension index of 0 in the pulse sending matrix, the weight accumulation value of the processed element is accumulated to the element with the first dimension index of the target neuron index in the pulse receiving matrix, and then the weight accumulation value in the element is set to 0, in addition, each thread moves the processed element by one bit, finally, if the element processed by the thread is the element with the first dimension index of (max_delay_j)-1 in the pulse sending matrix, the weight accumulation value in the element is set to 0.

[0103] S5.7, the current time step is incremented by 1.

[0104] S5.8, determine whether the current time step is equal to the simulation time step, if not, execute step S5.2, continue simulation, if yes, end the process.

[0105] It should be noted that steps S2-S5 are GPU running steps, in actual operation, the GPU starts the actual network creation and simulation process according to the user instruction, the user instruction can be input through the external interface, and the input parameters also include the simulation time step.

[0106] The neuron, synapse and pulse processing related data structure designed by the application, and the pulse sending and receiving scheme realized according to the data structure, can obviously improve the parallel efficiency of network creation, pulse weight acquisition, time delay update, routing transmission and receiving process in the GPU, and realize the parallel of the whole brain simulation process (including neuron creation, synapse creation, neuron update and pulse transmission) on the GPU. The advantages of the parallel implementation method proposed for different stages of the simulation process are as follows:

[0107] 1. Network creation stage (including neuron creation and synapse creation): generally, the system needs to sequentially construct neurons, synapses and the like according to user input information. The neuron and synapse table structure designed by the application is more suitable for the characteristics of the GPU many-core, can be quickly initialized, and can be created and filled with neuron and synapse parameters in parallel.

[0108] 2. Neuron parameter update phase: Generally, the system processes the parameter addressing and parameter updating process of each neuron as a minimum independent unit. The method of the present application takes the parameter as the minimum unit, stores the same parameters of all neurons in the same location, and can quickly address and quickly update.

[0109] 3. Pulse transmission phase: The general system processes pulses by unit of neuron in the pulse transmission phase, which needs to spend more time processing routing logic. Based on the many-core characteristics of GPU, the present application simplifies and speeds up the processes of routing, pulse storage and reception through multi-thread parallel processing.

[0110] In order to further illustrate the advantages of the full-process parallel acceleration brain simulation method proposed by the present application, the following will be described through a specific experiment.

[0111] The open source Microcircuit network is used for comparison test. In order to improve the discrimination of experimental results, the network specification parameters in the actual test are adjusted as follows: a total of 8 neuron clusters are consistent with the original Microcircuit network, the total number of neurons is increased from 77,000 to 770,000, the total number of synapses is increased from 240 million to 240 million, that is, the network size is increased by 10 times, but the network structure remains unchanged. The simulation biological time is 1s, that is, 10000 time steps. The comparison of brain simulation system (referred to as system), hardware type, hardware name, and neural network creation and simulation time results is shown in Table 1.

[0112] Table 1 Comparison of creation and simulation time statistics

[0113]

[0114] As can be seen from Table 1, the method of the present application is superior to the speed of the mainstream brain simulation system on the market in simulation creation and running time, which proves that the present application can truly realize full-process parallel and has good market application prospect.

[0115] As Figure 2 shown, in another embodiment, a full-process parallel acceleration brain simulation system is provided, the full-process parallel acceleration brain simulation system comprising a memory, a video memory and a GPU, wherein:

[0116] The memory is used to store the cluster parameters and synapse parameters input by the user;

[0117] The GPU is used to create a cluster table in the video memory according to the cluster parameters, create a routing table in the video memory according to the synapse parameters, and create a pulse emission matrix and a pulse reception matrix for each neuron in the video memory, and then perform brain simulation based on the cluster table, the routing table and the pulse emission matrix and the pulse reception matrix.

[0118] Specific definitions of the whole-process parallel accelerated brain simulation system can refer to the definitions of the whole-process parallel accelerated brain simulation method in the above, which will not be repeated here.

[0119] In one specific embodiment, the whole-process parallel accelerated brain simulation system further comprises a parameter input interface and a running command interface.

[0120] The parameter input interface is configured to receive the cluster parameters and the synaptic parameters input by the user and forward the parameters to the memory for storage.

[0121] The running command interface is configured to receive the running command and the simulation time step input by the user, and the GPU starts running after recognizing the running command.

[0122] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not contradict, they should be considered as the scope of the present disclosure.

[0123] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the present application. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A full-pipeline parallel acceleration brain simulation method applied to a GPU, characterized in that, The full-process parallel acceleration brain simulation method comprises: receiving user input cluster parameters and synaptic parameters; creating a cluster table in parallel according to the cluster parameters; creating a routing table in parallel according to the synaptic parameters; creating a pulse emission matrix and a pulse receiving matrix for each neuron; performing brain simulation based on the cluster table, the routing table, and the pulse emission matrix and the pulse receiving matrix, comprising: 1) initializing the current time step to 0; 2) starting M threads in parallel to collect pulses, taking the jth element in the ith pulse receiving matrix as the pulse received by the jth neuron in the ith cluster at the current time step, and emptying the pulse receiving matrix, where M is the total number of neurons; 3) starting M parallel threads, each of which updates the neuron parameters of the corresponding neuron in the cluster table, and judges whether the current membrane voltage in each updated neuron parameter is greater than the threshold value, if there is a neuron whose current membrane voltage is greater than the threshold value, it indicates that the neuron needs to emit a pulse and step 4) is performed, otherwise step 6) is performed; 4) for the jth neuron of the ith cluster that needs to emit a pulse, querying the jth target address table of the ith routing table, taking out all key-value pairs, taking the target address table index as the time delay value, and combining the target address table index and the corresponding key-value pair to form a [time delay value, target neuron index, weight] pulse information; 5) filling the pulse information into the pulse emission matrix corresponding to the jth neuron of the ith cluster, taking the time delay value in the pulse information as the first dimension index of the pulse emission matrix to obtain s_j elements, determining the element with the same target neuron index from the obtained s_j elements according to the target neuron index in the pulse information, and accumulating the weight in the pulse information to the weight accumulation value in the determined element, and accumulating the weights in the multiple pulse information corresponding to the same target address table to the same pulse emission matrix; 6) starting M parallel threads, each of which starts s_j*max_delay_j parallel threads, each of which corresponds to an element of the pulse emission matrix, if the element processed by the thread is the element with the first dimension index of 0 in the pulse emission matrix, the weight accumulation value of the processed element is accumulated to the element with the first dimension index of the target neuron index in the pulse receiving matrix, and then the weight accumulation value in the element is set to 0, in addition, each thread moves the processed element one bit forward, finally, if the element processed by the thread is the element with the first dimension index of (max_delay_j)-1 in the pulse emission matrix, the weight accumulation value in the element is set to 0, where max_delay_j is the first dimension length of the pulse receiving matrix, and s_j is the second dimension length of the pulse receiving matrix; 7) increasing the current time step by 1; 8) judging whether the current time step is equal to the simulation time step, if not, step 2) is performed to continue the simulation, and if yes, the process is ended.

2. The full-pipeline parallel accelerated brain simulation method of claim 1, wherein, The cluster parameters include the total number of clusters, the number of neurons in each cluster, the number of neuron parameters, and the initial parameters of the neurons. The synaptic parameters include a total number of synaptic information, and each piece of synaptic information contains a source cluster index, a target cluster index, a connection rule, a weight distribution, and a time delay distribution.

3. The full-pipeline parallel accelerated brain simulation method of claim 1, wherein, The creating a cluster table according to the cluster parameters comprises: obtaining a cluster parameter table with a length of L, L being a total number of clusters, and an i-th element in the table representing information of an i-th cluster, containing a number of neurons M_i, a number of neuron parameters N_i, and neuron initial parameters; initializing a cluster table with a length of L, starting L threads to initialize each element in the cluster table in parallel, and an i-th element in the cluster table representing a cluster matrix of the i-th cluster, the cluster matrix of the i-th cluster having a first dimension with a length of N_i and a second dimension with a length of M_i; starting L threads to process the L cluster matrices in parallel, wherein M_i threads are started in an i-th thread to fill neuron initial parameters of M_i neurons of the i-th cluster into the cluster matrix of the i-th cluster.

4. The full-pipeline parallel accelerated brain simulation method of claim 1, wherein, The creating a routing table according to the synaptic parameters comprises: creating a temporary source neuron table, a temporary target neuron table, a temporary weight table, and a temporary time delay table corresponding to each two clusters according to the synaptic parameters; counting a maximum time delay value of each neuron and a number of synapses of the neuron as a source neuron according to the temporary source neuron table and the temporary time delay table; initializing a one-dimensional routing table with a length of a number of neurons in each cluster, and an element in the routing table being a one-dimensional target address table with a length of the maximum time delay value of the corresponding neuron, and the target address table having an element of a [target neuron index, weight] key-value pair; filling the routing table according to the temporary source neuron table, the temporary target neuron table, the temporary weight table, and the temporary time delay table.

5. The full-pipeline parallel accelerated brain simulation method of claim 4, wherein, The creating a temporary source neuron table, a temporary target neuron table, a temporary weight table, and a temporary time delay table corresponding to each two clusters according to the synaptic parameters comprises: starting a number of threads equal to a total number of synaptic information in the synaptic parameters, and all the threads processing all the synaptic information in parallel, and each thread processing as follows: obtaining corresponding synaptic information containing a source cluster index, a target cluster index, a connection rule, a weight distribution, and a time delay distribution; calculating a number s_c of synapses to be created between two clusters according to the connection rule; generating a temporary source neuron table with a length of s_c and a temporary target neuron table with a length of s_c by a random number generator, an element in the temporary source neuron table being a neuron index contained in the source cluster index, and an element in the temporary target neuron table being a neuron index contained in the target cluster index, and simultaneously generating a temporary weight table with a length of s_c and a temporary time delay table with a length of s_c by the random number generator according to the weight distribution and the time delay distribution.

6. The full-pipeline parallel accelerated brain simulation method of claim 4, wherein, The filling the routing table according to the temporary source neuron table, the temporary target neuron table, the temporary weight table, and the temporary time delay table comprises: starting a number of parallel threads equal to a total number of synapses, and an s-th thread processing as follows: finding temporary source neuron tables, temporary target neuron tables, temporary weight tables, and temporary time delay tables with starting indexes less than and closest to s; Take the element with index [s-start index] in the temporary source neuron table as the index of the routing table to obtain a target address table, take the element with index [s-start index] in the temporary time delay table as the first dimension index value of the target address table, and write the elements with index [s-start index] in the temporary weight table and the temporary target neuron table into the target address table as a [target neuron index, weight] key-value pair.

7. The full-pipeline parallel accelerated brain simulation method of claim 1, wherein, The method further includes: starting a same number of parallel threads as the total number of neurons, and initializing a two-dimensional pulse emission matrix corresponding to each neuron, wherein the first dimension of the pulse emission matrix has a length of a maximum time delay value of the neuron, the second dimension of the pulse emission matrix has a length of a number of synapses of the neuron as a source neuron, and each element of the pulse emission matrix is a [target neuron index, weight accumulation value] key-value pair. The full-process parallel acceleration brain simulation system includes a memory, a video memory, and a GPU.

8. A full-pipeline parallel accelerated brain simulation system, characterized by, The memory is configured to store cluster parameters and synapse parameters input by a user. The GPU is configured to create a cluster table in the video memory according to the cluster parameters, create a routing table in the video memory according to the synapse parameters, and create a pulse emission matrix and a pulse reception matrix for each neuron in the video memory, and then perform brain simulation based on the cluster table, the routing table, and the pulse emission matrix and the pulse reception matrix. The brain simulation based on the cluster table, the routing table, and the pulse emission matrix and the pulse reception matrix includes: 1) initializing a current time step as 0; 2) starting M threads in parallel to collect pulses, taking the jth element of the ith pulse reception matrix as a pulse received by the jth neuron of the ith cluster at the current time step, and clearing the pulse reception matrix, where M is the total number of neurons; 3) starting M parallel threads, each of which updates a neuron parameter of a corresponding neuron in the cluster table, and determines whether a current membrane voltage in each updated neuron parameter is greater than a threshold value, if there is a neuron whose current membrane voltage is greater than the threshold value, it is indicated that the neuron needs to emit a pulse and step 4) is performed, otherwise, step 6) is performed; 4) for the jth neuron of the ith cluster that needs to emit a pulse, querying the jth target address table of the ith routing table, taking out all key-value pairs, taking the target address table index as a time delay value, and forming [time delay value, target neuron index, weight] pulse information with the corresponding key-value pair. ​ 5) filling the pulse information into the pulse emission matrix corresponding to the jth neuron of the ith cluster, taking the time delay value in the pulse information as the first dimension index of the pulse emission matrix to obtain s_j elements, determining the element with the same target neuron index from the s_j elements according to the target neuron index in the pulse information, accumulating the weight in the pulse information to the weight accumulation value in the key-value pair of the determined element, and accumulating the weights in the multiple pulse information corresponding to the same target address table to the same pulse emission matrix; 6) starting M parallel threads, each thread starting s_j*max_delay_j parallel threads, each thread in the s_j*max_delay_j parallel threads corresponding to an element of the pulse emission matrix, if the element processed by the thread is the element with the first dimension index of 0 in the pulse emission matrix, the weight accumulation value of the processed element is accumulated to the element with the first dimension index of the target neuron index in the pulse reception matrix, and then the weight accumulation value in the element is set to 0, in addition, each thread moves the processed element one bit forward, finally, if the element processed by the thread is the element with the first dimension index of (max_delay_j)-1 in the pulse emission matrix, the weight accumulation value in the element is set to 0, wherein max_delay_j is the first dimension length of the pulse reception matrix, and s_j is the second dimension length of the pulse reception matrix; 7) increasing the current time step by 1; 8) judging whether the current time step is equal to the simulation time step, if not, executing step 2) to continue the simulation, and if yes, ending the process.

9. The full-pipeline parallel accelerated brain simulation system of claim 8, wherein, The whole-process parallel acceleration brain simulation system further comprises a parameter input interface and a running command interface; The parameter input interface is used to receive the cluster parameters and synapse parameters input by the user and forward them to the memory for storage; The running command interface is used to receive the running command and simulation time step input by the user, and the GPU starts running after recognizing the running command.

Images