A brain simulation network resource allocation method based on network structure and transfer algorithm

Abstract

This invention discloses a brain simulation network resource allocation method based on network structure and transfer algorithm, belonging to the field of brain simulation technology. The method includes: extracting brain network information and abstracting cluster resource information; when cluster resources are sufficient, calculating the connectivity of neuron clusters in the brain model network structure based on the brain network information, and pre-allocating neuron clusters based on the connectivity of neuron clusters; using the pre-allocation result as the initial value for the transfer algorithm, iteratively optimizing the allocation of neuron clusters to obtain the iterative optimization result; merging all neuron clusters allocated to the same computing node into one neuron cluster based on the iterative optimization result to obtain the final resource allocation result; and actually creating the brain simulation network based on the resource allocation result to complete the brain simulation network resource allocation. This invention's resource allocation method reduces the storage space consumption of the brain model and improves the speed of brain simulation.

Description

Technical Field

[0001] This invention belongs to the field of brain simulation technology, specifically relating to a brain simulation network resource allocation method based on network structure and transfer algorithm. Background Technology

[0002] With the development of distributed parallel computing technology, the scale of brain simulation can be increased by distributing neural networks across computing clusters. Achieving fast brain simulation on computing clusters requires consideration not only of load balancing among computing nodes but also of communication between them. Currently, large-scale brain simulations on computing clusters often employ simple random or round-robin allocation methods to assign neurons to corresponding hardware. This results in significant communication traffic between computing nodes, causing a large amount of time to be wasted on data communication during brain simulation. Furthermore, it can lead to some computing nodes being overloaded, with computation time and storage consumption far exceeding those of other computing nodes, thus limiting the scale and speed of brain simulation and failing to fully utilize the computing power and storage space of the computing cluster. Therefore, large-scale brain simulation lacks a good resource allocation scheme. How to better allocate neurons in the brain model to reduce communication traffic between computing nodes while ensuring load balancing and reducing storage space consumption is a pressing problem that needs to be solved in large-scale brain simulation.

[0003] In frameworks supporting large-scale brain simulation, the NEST brain simulation framework, as the core simulator of the European Human Brain Project (HBP), is based on a parallel computing architecture of CPU supercomputing. It proposes a method for resource allocation using a simple polling algorithm, which sequentially distributes neurons to each thread according to their indices. While the polling algorithm evenly distributes neurons across the cluster, ensuring a relatively consistent number of neurons per computing node and thus balancing computing power to some extent, it does not consider synapse distribution. The communication latency caused by an unreasonable synapse distribution becomes the main bottleneck in the simulation.

[0004] Based on the GeNN brain simulation framework, a research team from the University of Sussex published a paper in Nature in 2021, proposing a data structure that maps brain-like networks to GPU memory using neuron clusters as units. By dynamically generating synapses, they achieved a scheme to simulate a large-scale network (a monkey brain with 4 million neurons) on a single GPU. Based on the NeuronGPU brain simulation framework, a research team from the University of Cagliari implemented a scheme for simulation on multiple GPUs, but this requires users to manually allocate neuron clusters on each GPU. Both the GeNN and NeuronGPU brain simulation frameworks map networks to GPU memory using neuron clusters, enabling GPU-based parallel accelerated computation. Furthermore, by operating on a cluster basis, neurons within a cluster do not need to communicate remotely, reducing communication overhead to some extent. However, the GeNN brain simulation framework only supports running on a single GPU and cannot achieve large-scale simulations of hundreds of millions of neurons. While the NeuronGPU brain simulation framework achieves multi-GPU communication based on MPI technology and proposes a method for pulse transmission between GPUs, it requires users to manually create neuron clusters on each GPU, increasing the difficulty of creating networks across multiple GPUs. Furthermore, it's challenging for users to manually match the neural network structure with the current cluster's computing power to achieve an optimal allocation scheme. A research team from Sun Yat-sen University proposed a graph partitioning method to reduce the total communication load in brain simulations. This method moves neurons from the nodes with the highest communication load to nodes with lower communication loads, thereby reducing the maximum communication load on a single GPU. However, this method only considers reducing the total communication load of brain simulations without considering communication load balancing, which may lead to excessive communication load on individual computing nodes, causing communication congestion and hindering the achievement of fast brain simulations. Summary of the Invention

[0005] The purpose of this invention is to provide a brain simulation network resource allocation method based on network structure and transfer algorithm, which reduces the storage space consumption of brain models and improves the speed of brain simulation.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A method for allocating brain simulation network resources based on network structure and transfer algorithms, comprising:

[0008] Brain network information is extracted based on the input brain model network structure, including a list of neuron clusters and a matrix of synaptic connection information.

[0009] Based on the abstract cluster resource information of the computing cluster, the cluster resource information includes a list of storage spaces;

[0010] If the cluster resources are deemed insufficient based on brain network information and cluster resource information, a reminder is generated and the process ends; otherwise, proceed to the next step.

[0011] The degree of connection between neuron clusters in the brain model network structure is calculated based on brain network information, and neuron clusters are pre-assigned based on the degree of connection between neuron clusters.

[0012] The pre-allocation results are used as the initial values ​​for the transfer algorithm, which iteratively optimizes the allocation of neuron clusters to obtain the iterative optimization results.

[0013] Based on the iterative optimization results, all neuron clusters allocated to the same computing node are merged into one neuron cluster to obtain the final resource allocation result;

[0014] Based on the resource allocation results, a brain simulation network is actually created, and the resource allocation of the brain simulation network is completed.

[0015] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0016] Preferably, the step of determining whether cluster resources are sufficient based on brain network information and cluster resource information includes:

[0017] Calculate the total resources R required for brain simulation without considering long synaptic connections based on brain network information. i ;

[0018] Calculate the total number of available resources R based on cluster resource information. c ;

[0019] If R i >R c If R... i ≤R c If the cluster resources are sufficient without considering remote synaptic connections, then resource allocation will continue.

[0020] Preferably, the step of calculating the connectivity of neuron clusters in the brain model network structure based on brain network information includes:

[0021] Calculate the tightness of connection between two clusters of neurons:

[0022]

[0023] In the formula, D ij S represents the degree of connection between neuron clusters pi and pj. pi S represents the number of neurons in the neuron cluster pi. pj k represents the number of neurons in neuron cluster pj.ij k represents the number of synapses connecting neuron cluster pi to neuron cluster pj. ji This represents the number of synapses connecting neuron cluster pj to neuron cluster pi.

[0024] Based on the connectivity between the two neuron clusters, calculate the connectivity between the neuron clusters and the computing nodes in the computing cluster:

[0025]

[0026] In the formula, CD pa S represents the degree of connection between neuron cluster p and computation node a. p k represents the number of neurons in a neuron cluster p. pa k represents the number of synapses connecting neuron cluster p to computation node a. ap This represents the number of synapses connecting all neuron clusters on node a to neuron cluster p.

[0027] Preferably, the pre-allocation of neuron clusters based on the connectivity density of neuron clusters includes:

[0028] S401: Select computation node a of the unassigned neuron cluster;

[0029] S402: Select the cluster p of neurons without assigned computing nodes;

[0030] S403: Calculate the storage space consumption of computing node a if the neuron cluster p is placed into computing node a;

[0031] S404: If the storage space consumption is less than or equal to the available storage space of computing node a, then place the neuron cluster p into computing node a and proceed to step S406; if the storage space consumption is greater than the available storage space of computing node a, then computing node a is considered allocated, and neuron cluster p is still considered as a neuron cluster of an unallocated computing node and proceed to step S405.

[0032] S405: Determine if all compute nodes have completed pre-allocation: If all compute nodes have completed pre-allocation, it is determined that there are insufficient cluster resources when considering remote synaptic connections, and a resource shortage message is displayed, and the program ends; if there are still compute nodes that have not completed pre-allocation, proceed to step S401.

[0033] S406: Determine if all neuron clusters have been allocated: If all neuron clusters have been allocated, the pre-allocation is complete. Record the index of each neuron cluster to the computing node and end the pre-allocation. If there are still neuron clusters that have not been allocated, proceed to step S407.

[0034] S407: Select the unassigned neuron cluster p that is most closely connected to the existing neuron cluster on computing node a, and proceed to step S403.

[0035] Preferably, the calculation of the storage space consumption of computing node a includes:

[0036] This includes the storage space consumed by all neurons and all internal synaptic connections stored on computing node a, as well as the additional storage space consumed by remote synaptic connections to computing node a.

[0037] Preferably, the step of using the pre-allocation result as the initial value for the transfer algorithm, and then iteratively optimizing the allocation of neuron clusters to obtain the iterative optimization result, includes:

[0038] S501: Use the pre-allocation result as the initial iterative optimization result of the migration algorithm, and initialize the number of computing nodes that have been traversed by the iterative optimization to N=0;

[0039] S502: Based on the current iterative optimization results, select the computing node a with the most remote synaptic connections;

[0040] S503: Select computing node b with few untraced remote synapses and a lot of redundant storage space in this iteration of optimization;

[0041] S504: Select the cluster of neurons p that maximizes the benefit when moved from computing node a to computing node b;

[0042] S505: Determine whether the storage space of computing node b will overflow if the neuron cluster p is moved to computing node b: If the storage space of computing node b does not overflow, proceed to step S506; if the storage space of computing node b overflows, proceed to the storage space balancing process.

[0043] S506: Determine whether the remote synaptic connections of computing node a have decreased: if they have decreased, proceed to step S508; if they have not decreased, proceed to step S507.

[0044] S507: Let the number of computation nodes that have been traversed in the iterative optimization be N = N + 1, and determine whether N is greater than the number of computation nodes: if it is greater, proceed to step S509; if it is not greater, proceed to step S503.

[0045] S508: Set the number of computation nodes that have been traversed in the iterative optimization to N = 0, move neuron cluster p, and if there is a neuron cluster q that needs to be moved, move neuron cluster q at the same time to complete the current iterative optimization, and continue to the next iterative optimization, and proceed to step S502.

[0046] S509: Output the final iterative optimization result; iterative optimization complete.

[0047] Preferably, the storage space balancing process includes the following steps:

[0048] S510: Select computing node c with few untraced remote synapses and a lot of storage space redundancy during this storage space balancing process;

[0049] S511: Select the cluster of neurons q that maximizes the benefit when moved from computing node b to computing node c;

[0050] S512: Determine whether computation node c is computation node a and whether neuron cluster q is neuron cluster p: If both conditions are met, proceed to step S515; otherwise, proceed to step S513.

[0051] S513: Determine whether the storage space of computing node c overflows if the neuron cluster q is moved to computing node c: If the storage space of computing node c overflows, proceed to step S515; otherwise, proceed to step S514.

[0052] S514: Set the number of computing nodes that have been traversed during this storage space balancing process to M=0, record the neuron cluster q that needs to be moved from computing node b to computing node c, and proceed to step S505.

[0053] S515: Let the number of compute nodes traversed during this storage space balancing process be M = M + 1, and determine whether M is greater than the number of compute nodes: if it is greater, proceed to step S503; if it is not greater, proceed to step S510.

[0054] Preferably, the computing node with few remote synapses and abundant redundant storage space is determined as follows:

[0055] Obtain the number of remote synapses X and the storage space consumption U of the compute node. Subtract the storage space consumption U from the total storage space T of the compute node to obtain the storage space redundancy of the compute node. Perform a weighted summation on the number of remote synapses X and the storage space redundancy (TU). The weighted summation is calculated as follows:

[0056] J = w1(Max(X)-X) + w2(TU)v

[0057] In the formula, J is the weighted summation result of the computing nodes, w1 and w2 represent the remote synapse weight and storage space weight, respectively, and Max(X) represents the maximum number of remote synapses among all computing nodes;

[0058] Since the units for synapse number and storage space differ and thus have different value ranges, v is used to unify the value range. The formula for calculating v is as follows:

[0059]

[0060] In the formula, Max(T) represents the maximum total storage space among all computing nodes;

[0061] The computing node with the largest weighted sum is selected as the computing node with fewer remote synapses and more redundant storage space.

[0062] Preferably, the calculation of the revenue includes:

[0063]

[0064] In the formula, I p S represents the gain after the neuron cluster p moves. p represents the number of neurons in neuron cluster p, and k represents the reduction in the number of remote synapses of neuron cluster p as it migrates from one computing node to another.

[0065] Preferably, merging all neuron clusters assigned to the same computing node into one neuron cluster includes:

[0066] Based on the iterative optimization results, all neuron clusters assigned to the same computing node are determined, and the total number of neurons on the computing node is determined based on the neuron cluster information list. A reconstructed neuron cluster with the same number of neurons as the total number of neurons is constructed. Based on the synaptic connection information matrix, the synaptic connections of the reconstructed neuron cluster are adjusted to be equivalent to the synaptic connections of the neuron clusters on the computing node before merging.

[0067] This invention provides a brain simulation network resource allocation method based on network structure and transfer algorithms. It places tightly connected clusters of neurons on the same computing node, reducing storage space consumption caused by remote synaptic connections and the time spent creating such connections. This reduces the storage space consumption of the brain model by more than 8%. By reducing remote synaptic connections and achieving balanced communication between computing nodes, communication congestion is prevented. Furthermore, by merging neuron clusters to reconstruct the network, the parallel computing capabilities of the computing nodes are fully utilized, enabling rapid brain simulation and improving brain simulation speed by more than 8%. Attached Figure Description

[0068] Figure 1 This is a logic block diagram of the brain simulation network resource allocation method based on network structure and transfer algorithm of the present invention.

[0069] Figure 2 This is a flowchart of the brain simulation network resource allocation method based on network structure and transfer algorithm of the present invention;

[0070] Figure 3 This is a flowchart of the pre-allocation process of the present invention;

[0071] Figure 4 This is a flowchart illustrating the iterative optimization of the migration algorithm of this invention. Detailed Implementation

[0072] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0073] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0074] This embodiment proposes a brain simulation network resource allocation method based on network structure and transfer algorithms, addressing the relationships between neuron clusters and the distribution of cluster resources. This method can automatically match neuron clusters with computing nodes based on known network information through resource mapping between the brain model information graph and the computing cluster resource graph, before the large-scale brain-like model network structure is established. This distributes the large-scale brain simulation network across the computing cluster without requiring users to manually allocate neurons within the cluster. This not only reduces the difficulty of creating neural networks within the cluster but also solves the problems of unbalanced load, excessive communication volume, and excessive storage space consumption in traditional large-scale brain-like simulations.

[0075] like Figure 1 As shown, this embodiment constructs a network information graph based on the basic parameters of the brain model network input by the user, constructs a cluster resource information graph based on the computing cluster, and then obtains the resource allocation results through preliminary resource detection, resource pre-allocation, migration iterative optimization, and network reconstruction, thereby completing the allocation of neuron clusters in the computing cluster, realizing load balancing of computing nodes, making full use of the parallel computing capabilities of computing nodes, balancing the communication volume between computing nodes, and reducing the communication volume, thus enabling rapid large-scale brain simulation.

[0076] like Figure 2 As shown, the brain simulation network resource allocation method based on network structure and transfer algorithm in this embodiment specifically includes the following steps:

[0077] S1. Extract brain network information based on the input brain model network structure.

[0078] When extracting brain network information, the brain model network structure input by the user is saved, but the network is not actually built in memory. The brain network information extracted based on the brain model network structure includes a list of neuron cluster information (the list records the number of neurons in each neuron cluster) and a synaptic connection information matrix (the matrix records the number of synaptic connections between any two neuron clusters).

[0079] S2. Abstract cluster resource information based on the computing cluster.

[0080] During extraction, the computing nodes in the computing cluster are first indexed and identified. Based on the abstract cluster resource information of the computing cluster, a storage space list is obtained (this list records the storage space of each computing node).

[0081] S3. If, based on brain network information and cluster resource information, it is determined that cluster resources are insufficient without considering remote synapses, a reminder will be generated and the program will end; otherwise, proceed to the next step.

[0082] To avoid invalid resource allocation, a preliminary check is performed on the resources before allocation, that is, to determine whether the cluster resources are sufficient. The specific judgment logic is as follows:

[0083] S301. Calculate the total resources R required for brain simulation without considering remote synaptic connections, based on brain network information. i In the initial resource detection, only the cluster resources are determined to be sufficient to meet the storage space requirements of internal synaptic connections. That is, the storage space consumed by all neurons and all internal synaptic connections in the brain model network structure is taken as the total resource number R. i .

[0084] S302. Calculate the total number of available resources R based on the cluster resource information. c The total number of available resources in a cluster should be understood as the sum of the storage space of all computing nodes in the cluster.

[0085] S303, Based on the total resources R required for brain model simulation without considering remote synaptic connections. i The total number of available resources R that the computing cluster can utilize. c Preliminary assessment of whether resources meet the demand: If R i >R c If R... i ≤R c If the cluster resources are sufficient without considering remote synaptic connections, then resource allocation will continue.

[0086] S4. Calculate the connectivity of neuron clusters in the brain model network structure based on brain network information, and pre-allocate neuron clusters based on this connectivity. During this process, when considering remote synapses, it checks if cluster resources are sufficient; if not, a warning is generated and the program terminates; otherwise, proceed to the next step.

[0087] This embodiment does not perform iterative optimization on the random allocation result, but rather pre-allocates resources by considering the tightness of the connections of the neuron clusters to obtain a relatively better allocation result, and then performs iterative optimization, thereby effectively improving the iterative optimization result.

[0088] The tightness of connections between neuronal clusters is not determined directly by the number of synapses connecting two clusters, but rather by the following formula, which provides a more stable and reliable assessment:

[0089]

[0090] In the formula, D ij S represents the degree of connection between neuron clusters pi and pj. pi S represents the number of neurons in the neuron cluster pi. pj k represents the number of neurons in neuron cluster pj. ij k represents the number of synapses connecting neuron cluster pi to neuron cluster pj. ji This represents the number of synapses connecting neuron cluster pj to neuron cluster pi.

[0091] Typically, a computing node may contain one or more clusters of neurons. Therefore, this embodiment proposes a connection strength between computing nodes and neuron clusters based on the connectivity of these clusters. Since the number of neurons on computing node a is a fixed value, the connection strength between the neuron clusters and the computing nodes in the computing cluster is calculated based on the connectivity of the two neuron clusters as follows:

[0092]

[0093] In the formula, CD pa S represents the degree of connection between neuron cluster p and computation node a. p k represents the number of neurons in a neuron cluster p. pa k represents the number of synapses connecting neuron cluster p to computation node a. ap This represents the number of synapses connecting all neuron clusters on node a to neuron cluster p.

[0094] Pre-allocation of neuron clusters based on the density of their connections can reduce the additional storage and communication costs caused by remote synapses, thus facilitating traffic and load balancing. Figure 3 As shown, the specific pre-allocation process is as follows:

[0095] S401: Select computation node a of the unassigned neuron cluster.

[0096] S402: Select the cluster of neurons p that has not been assigned computing nodes.

[0097] It should be noted that in this embodiment, computing node 'a' does not specifically refer to a certain computing node; 'a' is merely an identifier chosen for ease of description. For example, other identifiers such as 'b' or 'c' could also be used. Similarly, neuron cluster 'p' does not specifically refer to a certain neuron cluster; 'p' is also merely an identifier chosen for ease of description.

[0098] S403: Calculate the storage space consumption of computing node a if the neuron cluster p is placed into computing node a.

[0099] In this embodiment, the storage space consumption caused by remote synaptic connections is dynamically considered during resource allocation to reduce storage space consumption. For example, the calculation of the storage space consumption of computing node a includes the storage space consumed by all neurons and all internal synaptic connections stored on computing node a (the number of internal synaptic connections multiplied by a preset fixed value), and also includes the additional storage space consumption caused by remote synaptic connections to computing node a (the number of externally connected remote synapses multiplied by a preset fixed value). The two neurons in the remote synaptic connection are not on the same computing node.

[0100] S404: If the storage space consumption is less than or equal to the available storage space of computing node a, then place the neuron cluster p into computing node a and proceed to step S406; if the storage space consumption is greater than the available storage space of computing node a, then computing node a is considered allocated, and neuron cluster p is still considered as a neuron cluster of an unallocated computing node, and proceed to step S405.

[0101] S405: Determine if all computing nodes have completed pre-allocation: If all computing nodes have completed pre-allocation, but there are still neuron clusters that have not been allocated, then it is determined that there are insufficient cluster resources when considering remote synaptic connections, and a resource shortage message is displayed, and the program ends; if there are still computing nodes that have not completed pre-allocation, then proceed to step S401.

[0102] S406: Determine if all neuron clusters have been allocated: If all neuron clusters have been allocated, the pre-allocation is complete. Record the index of each neuron cluster assigned to the computing node and end the pre-allocation. If there are still neuron clusters that have not been allocated, proceed to step S407.

[0103] S407: Select the unassigned neuron cluster p that is most closely connected to the existing neuron cluster on computing node a, and proceed to step S403.

[0104] In other embodiments, the pre-allocation process can also select the neuron cluster that is placed into a certain computing node first through different strategies, such as selecting the neuron cluster that has the most connections with other neuron clusters, or selecting the two neuron clusters that are most closely connected.

[0105] S5. Use the pre-allocation result as the initial value for the transfer algorithm. The transfer algorithm iteratively optimizes the allocation of neuron clusters to obtain the iterative optimization result. For example... Figure 4 As shown, the specific process is as follows:

[0106] S501: Use the pre-allocation result as the initial iterative optimization result of the migration algorithm, and initialize the number of computational nodes that have been traversed by the iterative optimization to N=0.

[0107] S502: Based on the current iterative optimization results, select the computing node a with the most remote synaptic connections.

[0108] S503: Select computing node b that has few untraversed remote synapses and a lot of redundant storage space in this iteration of optimization.

[0109] When determining computing node b, the number of remote synapses and the amount of storage space redundancy of each computing node not traversed in this iteration optimization are obtained. Based on the pre-determined remote synapse weights corresponding to the number of remote synapses and the storage space weights corresponding to the amount of storage space redundancy, the number of remote synapses and the amount of storage space redundancy are weighted and summed. The computing node with the largest weighted sum is taken as computing node b with fewer remote synapses and more storage space redundancy.

[0110] To prevent the migration of neuron clusters to computing nodes with full storage space during iterative optimization, which would burden storage space balance, and to ensure that the maximum number of remote synapses on computing nodes is effectively reduced, the number of remote synapses and storage space redundancy of the computing node are considered when selecting the computing node to which the neuron clusters are to be migrated, and remote synapse weights and storage space weights are set. In step S503, the remote synapse weight is set to be greater than the storage space weight, for example, a remote synapse weight of 80% and a storage space weight of 20%.

[0111] S504: Select the cluster of neurons p that maximizes the benefit when moved from computing node a to computing node b.

[0112] In this embodiment, the moving neuron cluster does not result in the largest reduction in the number of remote synapses, but rather in the reduction in the number of remote synapses divided by the number of neurons contained in that neuron cluster. That is, the gain is the average gain per neuron when moving the neuron cluster.

[0113]

[0114] In the formula, I p S represents the gain after the neuron cluster p moves. p represents the number of neurons in neuron cluster p, and k represents the reduction in the number of remote synapses of neuron cluster p as it migrates from one computing node to another.

[0115] S505: Determine whether the storage space of computing node b overflows if the neuron cluster p is moved to computing node b (the same storage space consumption mentioned in step S403, including the storage space consumption of neuron synapses inside the computing node, as well as the storage space consumption of remote synaptic connections): If the storage space of computing node b does not overflow, proceed to step S506; if the storage space of computing node b overflows, proceed to the storage space balancing process, i.e., step S510.

[0116] S506: Determine whether the remote synaptic connections of computing node a have decreased: if they have decreased, proceed to step S508; if they have not decreased, proceed to step S507.

[0117] S507: Let the number of computational nodes that have been traversed in the iterative optimization be N = N + 1, and determine whether N is greater than the number of computational nodes: if it is greater, proceed to step S509; if it is not greater, proceed to step S503.

[0118] S508: Set the number of computation nodes that have been traversed in the iterative optimization to N = 0, move neuron cluster p, and if there is a neuron cluster q that needs to be moved, move neuron cluster q at the same time to complete the current iterative optimization, and continue to the next iterative optimization, and proceed to step S502.

[0119] S509: Output the final iterative optimization result; iterative optimization complete.

[0120] S510: Select the computing node c with few untraversed remote synapses and a lot of storage space redundancy during this storage space balancing process.

[0121] The logic for determining computing node c in step S510 is the same as that in step S503. The difference is that, in order to reduce the impact of storage space balance on resource allocation optimization, when selecting the computing node to which the neuron cluster should migrate, both the number of remote synapses and storage space redundancy of the computing node are considered. In step S510, the weight of remote synapses is less than the weight of storage space. For example, the weight of remote synapses is 20%, and the weight of storage space is 80%.

[0122] The specific process for determining the computing nodes in steps S503 and S510 is as follows:

[0123] Obtain the number of remote synapses X and the storage space consumption U of the compute node. Subtract the storage space consumption U from the total storage space T of the compute node to obtain the storage space redundancy of the compute node. Perform a weighted summation on the number of remote synapses X and the storage space redundancy (TU). The weighted summation is calculated as follows:

[0124] J = w1(Max(X)-X) + w2(TU)v

[0125] In the formula, J is the weighted summation result of the computing nodes, w1 and w2 represent the remote synapse weight and storage space weight, respectively, and Max(X) represents the maximum number of remote synapses among all computing nodes;

[0126] Since the units for synapse number and storage space differ and thus have different value ranges, v is used to unify the value range. The formula for calculating v is as follows:

[0127]

[0128] In the formula, Max(T) represents the maximum total storage space among all computing nodes;

[0129] The computing node with the largest weighted sum is selected as the computing node with fewer remote synapses and more redundant storage space.

[0130] S511: Select the cluster of neurons q that maximizes the profit when moved from computing node b to computing node c. The profit calculation in step S511 is the same as the profit calculation in step S504, and will not be described again here.

[0131] S512: Determine whether computation node c is computation node a and whether neuron cluster q is neuron cluster p: If both conditions are met, proceed to step S515; otherwise, proceed to step S513.

[0132] S513: Determine whether the storage space of computing node c overflows if the neuron cluster q is moved to computing node c (the same storage space consumption mentioned in step S403, including the storage space consumption of neuron synapses inside the computing node, as well as the storage space consumption of remote synaptic connections): If the storage space of computing node c overflows, proceed to step S515; otherwise, proceed to step S514.

[0133] S514: Set the number of computing nodes that have been traversed during this storage space balancing process to M=0, record the neuron cluster q that needs to be moved from computing node b to computing node c, and proceed to step S505.

[0134] S515: Let the number of compute nodes traversed during this storage space balancing process be M = M + 1, and determine whether M is greater than the number of compute nodes: if it is greater, proceed to step S503; if it is not greater, proceed to step S510.

[0135] In the above steps, steps S501 to S509 are iterative optimization processes, and steps S510 to S515 are storage space balancing processes. In this embodiment, when iteratively optimizing the migration of neuron clusters, the computational nodes to which the neuron clusters are to be migrated are selected, while also considering computational nodes with fewer remote synaptic connections and more redundant storage space, rather than simply considering computational nodes with fewer remote synaptic connections. Furthermore, resource allocation not only considers minimizing total communication volume but also ensuring balanced communication volume, thereby achieving rapid large-scale brain simulation.

[0136] S6. Based on the iterative optimization results, all neuron clusters allocated to the same computing node are merged into one neuron cluster to obtain the final resource allocation result.

[0137] This embodiment determines the neuron clusters assigned to the same computing node based on the optimized resource allocation results, and determines the total number of neurons on that computing node based on the neuron cluster information list, constructing a reconstructed neuron cluster with the same number of neurons as the total number of neurons. Based on the synaptic connection information matrix, the synaptic connections of the reconstructed neuron cluster are made equivalent to those of the original neuron cluster. In other words, neuron clusters on the same computing node are merged into one cluster, fully utilizing the parallel computing capabilities of the computing node.

[0138] S7. Based on the resource allocation results, actually create the brain simulation network and complete the resource allocation of the brain simulation network.

[0139] Based on the resource allocation results, this embodiment creates a brain simulation network to perform rapid large-scale brain simulation and monitors relevant data such as pulse firing and membrane voltage changes.

[0140] Large-scale neuromorphic networks, due to their large parameter scale and complex structure, currently primarily distribute neurons across different computing nodes through round-robin allocation, random uniform allocation, or manual allocation. While round-robin and random uniform allocation evenly distribute neurons across each node in the cluster, ensuring some degree of computational power balance, they do not consider synaptic distribution, easily leading to excessively long communication times between different computing nodes in large-scale brain simulations, becoming a major bottleneck limiting simulation speed. Manual allocation increases the difficulty of creating networks on multiple GPUs, and users also find it difficult to manually achieve optimal allocation. This invention addresses these problems by automatically allocating clusters of neurons with a large number of synaptic connections to the same node based on known network and cluster information. This reduces the additional storage space and communication costs caused by remote synapses, and considers achieving balanced communication and load, solving the problems of uneven computational power distribution and excessive communication in traditional large-scale neuromorphic simulations.

[0141] Experimental Example

[0142] To verify the proposed brain simulation network resource allocation method based on network structure and transfer algorithm, relevant experiments were conducted.

[0143] A balanced stochastic brain simulation network, in which a neuron cluster is randomly connected to three other neuron clusters, was tested on a computing cluster containing 6 computing nodes (NVIDIA A100) for a 1-second biological time-lapse brain simulation.

[0144] The resource allocation results proposed in this invention are compared with the simulation results of random uniform allocation and round-robin allocation. The experimental results are shown in Table 1.

[0145] Table 1 Brain Simulation Results

[0146]

[0147] The experimental results show that, under different network scales, the network creation time and network simulation time of the resource allocation method proposed in this invention are reduced compared with the network creation time and network simulation time of the random uniform allocation and round-robin allocation methods, significantly improving the brain simulation speed, with a network simulation time improvement of more than 8.71%. Moreover, the storage space occupied by the resource allocation method of this invention is smaller than that of the other two methods, with the maximum storage space reduced by more than 8.37% and the average storage space reduced by more than 11.11%.

[0148] In another embodiment, this application also provides a brain simulation network resource allocation device based on network structure and transfer algorithm, including a processor and a memory storing a plurality of computer instructions, wherein the computer instructions, when executed by the processor, implement the steps of the brain simulation network resource allocation method based on network structure and transfer algorithm.

[0149] For specific limitations on the brain simulation network resource allocation device based on network structure and transfer algorithm, please refer to the limitations on the brain simulation network resource allocation method based on network structure and transfer algorithm above, which will not be repeated here.

[0150] The memory and processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory stores a computer program that can run on the processor, which implements the method in the embodiments of the present invention by running the computer program stored in the memory.

[0151] The memory may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory stores the program, and the processor executes the program upon receiving an execution instruction.

[0152] The processor may be an integrated circuit chip with data processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0153] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0154] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A brain simulation network resource allocation method based on network structure and migration algorithm, characterized in that, The brain simulation network resource allocation method based on network structure and migration algorithm comprises: extracting brain network information from the input brain model network structure, wherein the brain network information comprises a neuron cluster information list and a synapse connection information matrix; abstracting cluster resource information from the computing cluster, wherein the cluster resource information comprises a storage space list; if it is determined that the cluster resource is insufficient based on the brain network information and the cluster resource information, a prompt is generated and the process is ended; otherwise, the next step is performed; calculating the connection tightness of the neuron clusters in the brain model network structure based on the brain network information, and pre-allocating the neuron clusters based on the connection tightness of the neuron clusters; taking the pre-allocation result as an initial value of the migration algorithm, and iteratively optimizing the allocation of the neuron clusters by the migration algorithm to obtain an iterative optimization result; merging all neuron clusters allocated to the same computing node into one neuron cluster based on the iterative optimization result to obtain a final resource allocation result; actually creating a brain simulation network based on the resource allocation result to complete the brain simulation network resource allocation.

2. The network structure and migration algorithm based brain simulation network resource allocation method of claim 1, wherein, The determination of whether the cluster resource is sufficient based on the brain network information and the cluster resource information comprises: According to the brain network information, the total resource number R required for brain simulation when not considering long-range synaptic connections i ; According to the cluster resource information, a total number of available resources R is calculated c ; If R i > R c , it is determined that the cluster resource is insufficient, a prompt is generated and ended; if R i ≤ R c , it is determined that the cluster resource is sufficient when the remote synapse connection is not considered, and the resource allocation is continued.

3. The network structure and migration algorithm based brain simulation network resource allocation method of claim 1, wherein, The calculation of the connection tightness of the neuron clusters in the brain model network structure based on the brain network information comprises: calculating the connection tightness of two neuron clusters; wherein D ij denotes the tightness of the connection between neuron cluster pi and neuron cluster pj, S pi denotes the number of neurons of neuron cluster pi, S pj denotes the number of neurons of neuron cluster pj, k ij denotes the number of synapses from neuron cluster pi to neuron cluster pj, k ji denotes the number of synapses from neuron cluster pj to neuron cluster pi; calculating the connection tightness of the neuron clusters and the computing nodes in the computing cluster based on the connection tightness of the two neuron clusters; where CD pa represents the tightness of the connection between neuron cluster p and computing node a, S p represents the number of neurons in neuron cluster p, k pa represents the number of synapses from neuron cluster p to computing node a, k ap represents the number of synapses from all neuron clusters on computing node a to neuron cluster p.

4. The network structure and migration algorithm based brain simulation network resource allocation method of claim 1, wherein, The pre-allocation of the neuron clusters based on the connection tightness of the neuron clusters comprises: S401: selecting a computing node a that has not allocated neuron clusters; S402: selecting a neuron cluster p that has not allocated computing nodes; S403: calculating the storage space consumption of the computing node a if the neuron cluster p is put into the computing node a; S404: if the storage space consumption is less than or equal to the available storage space of the computing node a, the neuron cluster p is put into the computing node a, and step S406 is entered; if the storage space consumption is greater than the available storage space of the computing node a, the computing node a is regarded as having been allocated, and the neuron cluster p is still regarded as a neuron cluster that has not allocated computing nodes, and step S405 is entered; S405: determining whether all computing nodes have completed pre-allocation: if all computing nodes have completed pre-allocation, it is determined that the cluster resource is insufficient when considering remote synapse connections, a resource shortage is prompted, and the program is ended; if there are still computing nodes that have not completed pre-allocation, step S401 is entered; S406: determining whether all neuron clusters have completed allocation: if all neuron clusters have completed allocation, pre-allocation is completed, the index identifier of each neuron cluster allocated to a computing node is recorded, and pre-allocation is ended; if there are still neuron clusters that have not been allocated, step S407 is entered; S407: selecting an unallocated neuron cluster p that is most tightly connected to the existing neuron clusters on the computing node a, and entering step S403.

5. The network structure and migration algorithm based brain simulation network resource allocation method of claim 4, wherein, The calculation of the storage space consumption of the computing node a comprises: including the storage space consumed by all neurons and all internal synapse connections stored on the computing node a, and including additional storage space consumption caused by remote synapse connections connected to the computing node a.

6. The network structure and migration algorithm based brain simulation network resource allocation method of claim 1, wherein, The pre-allocation result is taken as an initial value of the migration algorithm, and allocation of the neuron cluster is iteratively optimized by the migration algorithm to obtain an iterative optimization result, comprising: S501: taking the pre-allocation result as an initial iterative optimization result of the migration algorithm, initializing the number of calculation nodes that have been traversed in the iterative optimization N=0; S502: selecting a calculation node a with the most remote synapse connections according to the current iterative optimization result; S503: selecting a calculation node b with few remote synapses and redundant storage space that has not been traversed in the current iterative optimization; S504: selecting a neuron cluster p that has the maximum benefit when moved from the calculation node a to the calculation node b; S505: determining whether the storage space of the calculation node b is full if the neuron cluster p is moved to the calculation node b: if the storage space of the calculation node b is not full, proceeding to step S506; if the storage space of the calculation node b is full, proceeding to a storage space balancing process; S506: determining whether the remote synapse connections of the calculation node a are reduced: if yes, proceeding to step S508; if no, proceeding to step S507; S507: setting the number of calculation nodes that have been traversed in the iterative optimization N=N+1, and determining whether N is greater than the number of calculation nodes: if yes, proceeding to step S509; if no, proceeding to step S503; S508: setting the number of calculation nodes that have been traversed in the iterative optimization N=0, moving the neuron cluster p, and simultaneously moving a neuron cluster q if there is a neuron cluster q that needs to be moved, completing the current iterative optimization and proceeding to the next iterative optimization, and proceeding to step S502; S509: outputting a final iterative optimization result, and completing the iterative optimization.

7. The network structure and migration algorithm based brain simulation network resource allocation method of claim 6, wherein, The storage space balancing process comprises the following steps: S510: selecting a calculation node c with few remote synapses and redundant storage space that has not been traversed in the current storage space balancing process; S511: selecting a neuron cluster q that has the maximum benefit when moved from the calculation node b to the calculation node c; S512: determining whether the calculation node c is the calculation node a and whether the neuron cluster q is the neuron cluster p: if both conditions are met, proceeding to step S515; otherwise, proceeding to step S513; S513: determining whether the storage space of the calculation node c is full if the neuron cluster q is moved to the calculation node c: if the storage space of the calculation node c is full, proceeding to step S515; otherwise, proceeding to step S514; S514: setting the number of calculation nodes that have been traversed in the current storage space balancing process M=0, recording the neuron cluster q that needs to be moved from the calculation node b to the calculation node c, and proceeding to step S505; S515: setting the number of calculation nodes that have been traversed in the current storage space balancing process M=M+1, and determining whether M is greater than the number of calculation nodes: if yes, proceeding to step S503; if no, proceeding to step S510.

8. The network structure and migration algorithm based brain simulation network resource allocation method according to claim 6 or 7, characterized in that, The calculation node with few remote synapses and redundant storage space is determined in the following manner: The remote synapse quantity X and the storage space consumption U of the computing node are obtained, the storage space redundancy of the computing node is obtained by subtracting the storage space consumption U from the total storage space T of the computing node, the weighted sum of the remote synapse quantity X and the storage space redundancy (T-U) is obtained, and the calculation method of the weighted sum is as follows: J = w1(Max(X)-X) + w2(T-U) v In the formula, J is the weighted sum result of the computing node, w1 and w2 represent the remote synapse weight and the storage space weight respectively, and Max(X) represents the maximum remote synapse quantity in all computing nodes. Because the synapse quantity and the storage space have different units and different value ranges, v is used to unify the value range, and the calculation formula of v is as follows: In the formula, Max(T) represents the maximum total storage space in all computing nodes. The computing node with the maximum value of the weighted sum result is taken as the computing node with less remote synapses and more storage space redundancy.

9. The network structure and migration algorithm based brain simulation network resource allocation method of claim 6, wherein, The calculation of the revenue includes: In the formula, I p represents the benefit after the neuron cluster p moves, S p represents the number of neurons of the neuron cluster p, and k represents the reduction amount of the number of remote synapses when the neuron cluster p is migrated from one computing node to another computing node.

10. The network structure and migration algorithm based brain simulation network resource allocation method of claim 1, wherein, The merging of all neuron clusters allocated to the same computing node into one neuron cluster includes: According to the iterative optimization result, all neuron clusters allocated to the same computing node are determined, the total neuron quantity on the computing node is determined according to the neuron cluster information list, the reconstructed neuron cluster with the neuron quantity equal to the total neuron quantity is constructed, and the synapse connection of the reconstructed neuron cluster is adjusted to be equivalent to the synapse connection of the neuron cluster before merging on the computing node according to the synapse connection information matrix.

Images