A spiking neural network heterogeneous acceleration device supporting custom neuron models

Abstract

The application discloses a kind of support custom neuron model's pulse neural network heterogeneous acceleration device, including the uniform scheduling module, primary cache area and cluster module connected in turn, cluster module includes the task control center module and computing core array interconnected;Uniform scheduling module is based on the neuron model of pulse neural network and its network information, control the memory read-write of primary cache area, and during the work task is distributed downward, according to current neural network situation, select to send adjacent neuron data in network and request in specified cluster;Primary cache area is used to save the neuron and its network information configured by user;Task control center module calls computing resource to operate according to task and corresponding data;Computing core array includes a group of array formed by neuron module based on data exchange link, the task and corresponding data obtained are distributed to each neuron module for calculation, and feedback calculation result.

Description

Technical Field

[0001] This invention belongs to the fields of neural networks and integrated circuit technology, and specifically relates to a heterogeneous acceleration device for spiking neural networks that supports custom neuron models. Background Technology

[0002] Deep learning has been widely applied in numerous fields such as image recognition, sound recognition, object detection and tracking, smart healthcare, and autonomous driving, demonstrating remarkable efficiency. However, while deep learning boasts high accuracy, it also suffers from numerous drawbacks, such as enormous computational costs and insufficient general intelligence. Neuromorphic computing based on spiking neural networks, due to its working mechanism being closer to that of the biological brain, is considered a promising approach to overcome the shortcomings of deep learning and become the best way to solve artificial intelligence problems. Therefore, meeting the requirements of high performance, low power consumption, and scalability remains a major challenge for neuromorphic computing systems.

[0003] Due to the numerous types of neurons and synapses, and the fact that spiking neural networks are still in the research stage, most current spiking neural network hardware platforms only support simple neuron models and cannot construct complex networks due to flexibility and implementation difficulties. Platforms that rely primarily on CPUs for computation suffer from slow processing speeds and high power consumption.

[0004] In conclusion, it is particularly important to design a heterogeneous acceleration device that can effectively support user-defined spiking neural network models. Summary of the Invention

[0005] To address the shortcomings of existing technologies, achieve a balance between time and resource consumption, improve data transmission efficiency, and enhance hardware computing efficiency, this invention adopts the following technical solution:

[0006] A heterogeneous acceleration device for spiking neural networks that supports custom neuron models includes a unified scheduling module, a first-level cache, and a cluster module connected in sequence. The cluster module includes a task control center module and a computing core array that are interconnected.

[0007] The unified scheduling module, based on the neuron model and network information of the spiking neural network, controls the memory read / write control signals and read / write addresses of the first-level cache, and during the downward allocation of work tasks, selects to send adjacent neuron data and requests in the network to the specified cluster according to the current neural network status.

[0008] The first-level cache is used to store the user-configured neurons and their network information;

[0009] The task control center module calls computing resources to perform calculations based on the task and corresponding data.

[0010] The computing core array includes an array of neuron modules based on data exchange links, which facilitates simple data exchange between neuron modules, reduces the resource overhead of the top-level unified scheduling center, achieves a balance between time and resource overhead, distributes the acquired tasks and corresponding data to each neuron module for computation, and feeds back the computation results.

[0011] Furthermore, the task and corresponding data include the weights and impulse information of the neural network as external data input, wherein the weights enter from the neuron modules at the outer corners of the array and are transmitted from the periphery to the center, and the impulse information enters from the neuron modules at the inner corners of the array and is transmitted from the center to the periphery. In order to improve the efficiency of data transmission within the array, the present invention specifies the transmission principle within the array: external data is preferentially transmitted to the neuron modules at the opposite corners.

[0012] Furthermore, each neuron module contains a buffer and a counter. Based on the counter's count, the one-way congestion status of the link is categorized as smooth, congested, and blocked. The routing selection under different conditions is as follows:

[0013] When the links along the X-axis and the Y-axis are both open, data is transmitted preferentially along the shortest path first along the X-axis and then along the Y-axis.

[0014] When the X-axis link is congested and the Y-axis link is unobstructed, data is transmitted via the shortest path that first extends along the Y-axis and then along the X-axis.

[0015] When the Y-axis link is congested and the X-axis link is unobstructed, data is transmitted via the shortest path that first extends along the X-axis and then along the Y-axis.

[0016] When the link in either direction is blocked and no more data can be stored, the only option is to choose the link in the other direction to transmit the data.

[0017] When both the X and Y axes are blocked, the data waits in the buffer of the neuron module until the link is unblocked.

[0018] Furthermore, to prevent data corruption when adjacent neuron module links arrive simultaneously, an arbitration algorithm based on link priority is constructed. Once a link in a neuron module's buffer successfully hands over data, its priority becomes the lowest in subsequent arbitrations. That is, the priority of each link is not fixed, but becomes the lowest after the handshake is completed, and is adjusted accordingly based on the current status of other links. When data from multiple links arrive simultaneously, the neuron module's buffer completes the handshake sequentially according to the current priority. Even if a previously high-priority link receives new data, it must wait for all other links to complete their handshakes.

[0019] Furthermore, the neuron module includes a cache, a memory, a weight accumulation module, a secondary cache, a stack control module, and a computation library module;

[0020] The buffer is used to store data transmitted from other neuron modules via a link. If the destination of the data is this neuron module, it is stored in the corresponding memory; otherwise, the link continues to transmit the data according to the relative position between this neuron module and the target address.

[0021] The memory includes a weight storage memory and a pulse storage memory;

[0022] The weight accumulation module reads data from the weight memory and pulse memory, performs accumulation calculations at the same time step, and uses the result of accumulating all weights at the same time as the stimulation signal for the neuron module to perform neuron calculation.

[0023] The secondary cache includes an operator storage memory and a parameter storage memory, used to store calculation formulas and calculation parameters;

[0024] The stack control module includes a state machine and a stack. The state machine reads data from the operator storage memory and the parameter storage memory according to the post-order expression and stores the data into the stack. The stack puts the data in its contents into the calculation library according to the corresponding order, obtains the calculation result and puts it back into the stack, until the post-order expression ends and the final value is obtained. It is compared with the threshold. If it is greater than or equal to the threshold, a pulse is issued and the weight value is cleared to zero. If it is less than the threshold, the current weight value is maintained and carried over to the next round of calculation.

[0025] The computation library module is used for the computation and firing of pulses by neurons. It consists of a set of operation modules. In addition to simple arithmetic operations, it also has complex operation rules such as square root and power operation. It also has a convenient addition function. New operation rules can be added by directly adding new operation modules without changing the logic of the internal circuit.

[0026] Furthermore, the state machine of the stack control module provides a way to judge different events in different states and determine the next action state. The state machine includes the following states:

[0027] Empty state: When the start calculation (the complete calculation formula is stored in RAM and the system starts calculation) signal is pulled high, the system enters the formula data reading state and the memory reading signal storing the formula is pulled high.

[0028] Formula data reading state: Enters working state, reads data from the memory storing formulas. If the read instruction is placeholder information, then enters parameter data reading state; if the read instruction is operator information, then enters operation stack state, and the stack output signal changes from 0 to 1.

[0029] Reading parameter data status: At this time, the enable signal of the parameter storage memory is emitted once; if the stack is not full, it enters the operation stack status;

[0030] Operation stack state: If the stack output signal is 0, return to the formula data reading state; otherwise, enter the docking calculation library state. The stack output enable emits a pulse, and the calculation library enable emits a pulse after one clock cycle and transfers the value in the stack to the calculation library.

[0031] Interconnected computing library status: When the computing library enable signal is pulled high, it enters the waiting state for the computing library to return data, and then the internal status control signal is reset to low level;

[0032] Waiting for the computation library to return data status: When the computation library completes the calculation signal and goes high, it means that the calculation result has been returned, so proceed to the next judgment: If the signal of completing a complete formula calculation in the neuron module does not respond, enter the data return stack state, the memory enabling signal storing the formula goes high, and then return and read the formula data status; if the signal of completing a complete formula calculation in the neuron module responds, enter the write-back data state, and then write data into the memory.

[0033] Write-back data status: The Runge-Kutta equation of one operation is obtained, and the result is compared with the threshold. If it is less than the threshold, the corresponding variable in the storage parameter memory is updated; otherwise, a pulse signal is issued and the variable value is reset.

[0034] Furthermore, if the post-order expression is calculated simply using the above method, its computational efficiency is relatively low compared to existing computing devices for fixed-model neurons. Therefore, this invention proposes an optimization scheme based on the binary tree property of the post-order expression, decomposing the post-order expression and parallelizing the internal calculations of the formula, thereby greatly improving its computational speed.

[0035] The variables, constants, and symbols of the objective function are decomposed to obtain the post-order expression; for example, f(x)=(x+1)*(x-3); then the post-order expression is f(x)=x1+x3-*;

[0036] Based on the units composed of constants, variables and corresponding symbols of the objective function, the post-order expression is decomposed into multi-level expressions; for example, after optimization, it is decomposed into f(x1)=x11+ and f(x2)=x23- and f(x)=f(x1)*f(x2);

[0037] Based on the expression level from low to high, they are placed into neuron modules and calculated sequentially. For example, at the first time step, the operations of f(x1) and f(x2) are placed into two computational kernels respectively. After obtaining the results of the first two formulas, the operation of f(x) = f(x1) * f(x2) at the next time step is performed. In this way, operations of the same level can run in parallel, which greatly saves computation time and improves the working efficiency of the computing device.

[0038] Furthermore, the task control center module includes a task station, a computing library control center, a database, and an intra-cluster destination instruction module. The task station is connected to the unified scheduling center, the computing library control center, and the intra-cluster destination instruction module, respectively, and the database is connected to the computing library control center and the intra-cluster destination instruction module, respectively.

[0039] The task station obtains instructions from the unified scheduling module and, based on the reading of the instructions, allows the database to store the corresponding data in the first-level cache.

[0040] The computing database control center is used to control the reading and writing of the database;

[0041] The database includes a formula memory, a weight memory, and a pulse memory, which are used to store formula, weight, and pulse information, respectively. The formula memory is connected to each neuron module of the array, the weight memory is connected to the neuron modules at the outer corners of the array, and the pulse memory is connected to the neuron modules at the inner corners of the array.

[0042] The cluster-specific instruction module is connected to the neuron module at the inner corner of the array.

[0043] Furthermore, the task station includes a queue for storing tasks and a state machine for controlling task retrieval. The state machine includes the following states:

[0044] Empty state: The state when power is off; it enters the idle state when the reset signal is high.

[0045] Idle state: At this time, the task station has not acquired a new task and is allowed to read a new task from the task FIFO queue. When a new task instruction is read, it is parsed and the process jumps to the corresponding new state machine.

[0046] Load configuration file: Import the data from the scheduling center (including neuron formulas, weight data packets, and impulse information packets) into the database, and return to the idle state after storage is complete;

[0047] Initialize array (i.e. load array): Put the formula stored at the specified location into the formula storage area (stores operator RAM and parameter RAM) in the specified neuron module, and return to the idle state after completion;

[0048] Neuron operation: The specified neuron module performs the operation, and returns to the idle state directly after issuing the instruction;

[0049] Intra-cluster data routing: Based on different data types, data enters the array from different array entrances, specifies the starting neuron to transmit pulse data to the target location, and returns to the idle state directly after issuing the instruction.

[0050] Sending information outside the cluster: Send a request to the unified scheduling module with the processed data (updated data), wait for the other party's request, and return to the idle state when the data exchange is completed;

[0051] The computing library control center writes data when the state machine is loading the configuration file; and reads data when the state machine is loading the array, routing data within the cluster, or sending information outside the cluster.

[0052] Furthermore, the task allocation method of the unified scheduling module is a round-robin scheduling mechanism, the purpose of which is to reduce the workload of a single computing core and, when hardware resources are sufficient, to distribute tasks evenly among the computing cores.

[0053] The advantages and beneficial effects of this invention are as follows:

[0054] This invention provides a heterogeneous acceleration device for spiking neural networks that supports custom neuron models. Compared to traditional heterogeneous computing platforms for spiking neural networks, the neuron modules exchange data via 4x4 links, making the processing of spiking neural network data more flexible and achieving a better balance between time and resources. The neuron model is computed using stack operations, enabling the calculation of more complex, user-defined arbitrary equations beyond LIF and Izhikevich models. The invention employs the Runge-Kutta method to update the neuron dynamics equations. The hardware computation process utilizes a binary tree representation method, and by leveraging the characteristics of binary trees, complex post-order expressions are optimized in parallel, thereby improving the device's hardware computation efficiency. Attached Figure Description

[0055] Figure 1 This is a connection diagram between the unified scheduling module and the cluster in an embodiment of the present invention.

[0056] Figure 2 This is a structure within a cluster in an embodiment of the present invention.

[0057] Figure 3This is the state transition diagram of the task station in this embodiment of the invention.

[0058] Figure 4 This is a flowchart of the content of the neuron module in an embodiment of the present invention.

[0059] Figure 5 This is the state transition diagram of the arbitration algorithm for inter-array link priority fairness in this embodiment of the invention.

[0060] Figure 6 This is a state transition diagram of the stack control module of the neuron module in this embodiment of the invention.

[0061] Figure 7 This is the state transition diagram for parallel computation in an embodiment of the present invention.

[0062] Figure 8 This is a flowchart of the method in an embodiment of the present invention. Detailed Implementation

[0063] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0064] like Figure 1 As shown, a heterogeneous acceleration device for spiking neural networks (SNNs) supporting custom neuron models is presented. This SNN acceleration device consists of numerous SNN neuron kernels, thus requiring the design of their interconnections and scheduling strategies. The device includes a unified scheduling module, a first-level buffer, and a cluster module connected sequentially. The cluster module is as follows... Figure 2 As shown, it includes a task control center module and a computing kernel array formed by connecting multiple neuron modules; each neuron module includes a secondary cache, a computing kernel, a stack module, and a post-expression operation control module.

[0065] The unified scheduling module controls the memory read / write control signals and read / write addresses of the first-level cache. During the downward allocation of tasks, it selects to send data and requests of adjacent neurons in the network to the specified cluster based on the current network situation. The task allocation method is a round-robin scheduling mechanism, which aims to reduce the workload of a single computing core and, when hardware resources are sufficient, distribute tasks evenly among the computing cores. The first-level cache is used to record and store the user-configured neurons and their network information.

[0066] In this embodiment of the invention, the user designs the network on a PC, inputs the neuron model and its network information, and sends the spiking neural network data to a Field Programmable Gate Array (FPGA) via a high-speed serial computer extended bus standard (PCIE, peripheral component interconnect express). In the FPGA, commands are sent to the unified scheduling module via the AXI (Advanced eXtensible Interface, a bus protocol). The unified scheduling module stores the data describing the network structure in a level-one buffer. Upon receiving an allocation command, the unified scheduling module begins operation. However, the space in the level-one buffer is limited, and it is impossible to import all the data into the FPGA board at once. Therefore, the data is imported sequentially according to the time steps of the neural network. After the calculation result is returned to the PC, the data for the next time step is overwritten into the level-one buffer.

[0067] The cluster contains a task control center module and... Figure 2 The neural modules connected in a 4x4 array do not require external resources. They distribute tasks and data packets from the unified scheduling center to the task control center module within the cluster. This module then calls upon resources within the cluster to process the data according to the specified content, enabling efficient data flow and computation.

[0068] The task control center module includes a task station, a computing library control center, a database, and an intra-cluster destination instruction module. The database includes a formula memory, a weight memory, and a pulse memory, which are used to store formula, weight, and pulse information, respectively. The task station is connected to the unified scheduling center, the computing library control center, and the intra-cluster destination instruction module, respectively, and the database is connected to the computing library control center and the intra-cluster destination instruction module, respectively.

[0069] In embodiments of the present invention, such as Figure 1 As shown, the 16 neuron computational nuclei are divided into a group called a cluster (the structure within the cluster is as follows). Figure 2 (The area indicated by the dashed box). All clusters are controlled by a unified scheduling module, which provides data pathways to enable orderly data exchange between clusters. The data exchange between the unified scheduling module and the clusters mainly includes neuron formulas, spike information, and weight values; each cluster exchanges data with the unified scheduling module through a handshake protocol.

[0070] Network data is distributed to various clusters via a unified scheduling module. Commands sent to the unified scheduling module include the neuron model, its corresponding weights, spike information, and time step information. These commands are then sent to the specified cluster number and the neuron module number within that cluster. When the unified scheduling center completes a handshake protocol with the specified cluster and the handshake signal goes high, the unified scheduling module sends a command to load the configuration file to the task station. When the task reads this command, it allows the specified data in the first-level buffer to be stored in the database of the specified cluster. When the user inputs data from a PC and lays out the neuron network on the FPGA board, the above steps are repeated. The task station reads the data, the state machine enters the array loading state, and the relevant information of the neurons is placed into the neuron model. After the user inputs the start calculation command, the above steps are repeated, and the neuron module begins calculation. During this process, the neuron module performs data interaction within the cluster and information flow between clusters, ultimately outputting the classification result.

[0071] The task station module exchanges instructions with the unified scheduling module. The module mainly contains a FIFO for storing tasks and a state machine that controls task retrieval. This state machine includes the following seven states: Figure 3 As shown:

[0072] Idle state: The state when power is off. When the reset signal is high, it enters the idle state (no task).

[0073] Idle state (no task): At this time, the task station has not acquired any new tasks, and is allowed to read new tasks from the task FIFO. When a new task instruction is read, it is parsed, and the process jumps to the new state machine.

[0074] Load configuration file: Import the data from the scheduling center (including neuron formulas, weight data packets, and impulse information packets) into the database, and return to the idle state after storage is complete.

[0075] Initialize array (i.e. load array): Place the formula stored at the specified location into the formula storage area (operator RAM and parameter RAM) within the specified neuron, and return to the idle state after completion.

[0076] Neuron operation: The specified neuron performs the operation, and returns to the idle state immediately after issuing the instruction.

[0077] Intra-cluster data routing: Based on different data types, data enters the array from different array entrances, specifies the starting neuron to transmit pulse data to the target location, and returns to the idle state directly after issuing the instruction.

[0078] Sending information outside the cluster: Send the updated data (the data that has been processed) to the unified scheduling module, wait for the other party's request, and return to the idle state when the data exchange is completed.

[0079] The computing library control center is used to control the reading and writing of the database. When the state machine is loading the configuration file, it writes data; when the state machine is loading the array, routing data within the cluster, or sending information outside the cluster, it reads data.

[0080] The computational kernel array connects multiple neuron modules in the form of an on-chip network, which facilitates simple data exchange between neuron modules, reduces the resource overhead of the top-level unified scheduling center, and achieves a balance between time and resource overhead.

[0081] In this embodiment of the invention, multiple flexibly callable neuron computing kernels are built within the cluster to form a computing kernel array; the computing kernel array is a neuron array formed by connecting adjacent neuron modules through data exchange links. Each computing kernel (i.e., neuron module) in the array has a stack and computing library built to perform spiking neural network operations and returns the results layer by layer.

[0082] Neuron modules are used for neuron computation and firing impulses, such as... Figure 4 As shown, it includes a buffer, memory, a weight accumulation module, a secondary buffer, a stack control module, and a computation library module. The buffer stores data transmitted from other neuron modules via links. If the data's destination is this neuron module, it is stored in the corresponding RAM; otherwise, the link continues based on the relative position between this module and the target address. The memory includes a weight RAM and a pulse RAM. The weight accumulation module reads data from the weight RAM and pulse RAM, performs accumulation calculations at the same time step, and uses the results of accumulating all weights at the same moment as the stimulation signal for the neuron module to perform neuron computation. The secondary buffer includes an operator RAM and a parameter RAM, used to store calculation formulas, etc. The computational parameters and stack control module consist of a state machine and a stack space. The state machine reads data from the operator RAM and parameter RAM according to the subsequent expression, stores the data on the stack, and then puts the data from the stack into the computation library to obtain the computation result, which is then put back into the stack. This process continues until the subsequent expression ends, obtaining the final value. This value is compared with a threshold; if it is greater than or equal to the threshold, a pulse is emitted and the value is cleared; if it is less than the threshold, the current value is retained and carried over to the next round of computation. The computation library module is used for neuron computation and pulse emission. It consists of a set of computation modules, including simple arithmetic operations as well as complex operations such as square roots and exponentiation. It also features a convenient addition function; new computation rules can be added directly by adding new computation modules without changing the internal circuit logic.

[0083] The formula storage corresponding to the database is connected to all neuron modules, the weight storage is connected to the neuron modules at the four corners of the array, and the pulse storage is connected to the four neuron modules in the center of the array. Through the data exchange link, data that does not belong to this neuron module is transmitted to the target neuron module through a specified route; the intra-cluster target instruction module is connected to the four neuron modules in the center of the array.

[0084] In this embodiment of the invention, data transmitted from the computational kernel of the connected neurons, or from the database, will enter the buffer for identification. If the address of the target neuron is the current neuron module, it will be stored in the RAM of the corresponding data type. Otherwise, it will be placed on the corresponding data exchange link according to the relative position of the target location and the current neuron.

[0085] All 16 neuron modules within the cluster are connected to the formula storage RAM, facilitating the loading of corresponding formulas by the neuron modules and reducing bandwidth usage between neuron links. Adjacent neuron computation modules are connected by links to transmit data. The RAM storing weights is connected to the neuron modules at the four corners of the array; therefore, weight values ​​are primarily transmitted from the periphery towards the center. The RAM storing pulses is connected to the four neuron modules in the center of the array; therefore, instructions within the cluster are primarily transmitted from the center outwards. Because these two types of data originate from outside the array, they are referred to as external data. Pulse information sent from within a neuron computation module to other modules within the cluster after computation is completed is called internal data.

[0086] Meanwhile, internal and external data transmission within the array share the inter-module link. To improve the efficiency of data transmission within the array, this invention specifies the transmission principle within the array: external data intended to be transmitted to the diagonally opposite module preferentially selects path 2 (e.g., ...). Figure 2 As shown in the diagram, path 2 carries a greater amount of external data than path 1, while path 3 has the least routing pressure. Each neuron's computational core has a buffer with an internal read / write counter. Based on this counter, the one-way congestion status of the link in this invention is categorized into three states: smooth, congested, and blocked. The routing selection will differ under each of these conditions.

[0087] When the links along the X-axis and the Y-axis are both open, data is transmitted preferentially along the shortest path, first along the X-axis and then along the Y-axis.

[0088] When the X-axis link is congested and the Y-axis link is unobstructed, data is preferentially transmitted via the shortest path that first extends along the Y-axis and then along the X-axis.

[0089] When the Y-axis link is congested and the X-axis link is unobstructed, data is preferentially transmitted via the shortest path that first extends along the X-axis and then along the Y-axis.

[0090] When a link in either direction is blocked and cannot store more data, the only option is to choose a link in the other direction to transmit data.

[0091] When both the X and Y axes are blocked, the data waits in the buffer of the neuron module until the link is unblocked.

[0092] Therefore, if the starting and ending positions of internal data are in different rows and columns, under normal circumstances (assuming the data flow path is unobstructed), path 3 is preferred, followed by path 1, and lastly path 2. When they are in the same row or column, data is transmitted along that row or column.

[0093] To prevent data corruption when adjacent neuron module links arrive simultaneously, and to ensure fairness, this invention employs an arbitration algorithm. Once one link in a neuron module's buffer successfully hands over data, its priority becomes the lowest in subsequent arbitrations. In other words, the priority of each link is not fixed but becomes the lowest after the handshake is completed, and is adjusted according to the current status of other links. This way, when multiple links arrive simultaneously, the buffer can complete handshakes sequentially according to their current priority. Even if a previously high-priority link receives new data, it will wait until all other links have completed their handshakes before receiving it. Figure 5 As shown, when a data handshake request occurs on the link, it jumps to the corresponding state. The four state machines correspond to four state operations where the link priority drops to the last position.

[0094] This invention stores the 0 / 1 signals emitted by the pulse in a designated storage unit. When the task control center module needs to allocate weights to the neuron module, it will first access the pulse information in the storage unit as the input pulse signal. If the pulse signal is 0, then it is not necessary to transmit the weight of the corresponding synapse to the neuron array, thereby reducing the link transmission pressure in the array.

[0095] The stack control module includes a state machine, which provides a way to judge different events in different states and determine the next action state. For example... Figure 6 As shown, this state machine contains the following eight states:

[0096] Empty state: When the start calculation (the complete calculation formula is stored in RAM and the system starts the calculation instruction) signal is pulled high, the system enters the formula data reading state and the RAM reading signal storing the formula is pulled high.

[0097] Formula Data Reading State: Enters working state, reads data from the RAM storing the formula. If the read instruction is a placeholder, then enters parameter data reading state. If the read instruction is a calculator, then enters the operation stack state, and the stack output signal changes from 0 to 1.

[0098] Parameter data read status: At this time, the enable signal for the RAM storing parameters is emitted once. If the stack is not full, it enters the operation stack status.

[0099] Operation stack state: If the stack output signal is 0, return to the formula data reading state. Otherwise, enter the docking calculation library state, the stack output enable emits a pulse, and the calculation library enable emits a pulse after one clock cycle and transfers the value in the stack to the calculation library.

[0100] Interconnection to the computing library status: When the computing library enable signal is pulled high, it enters the waiting state for the computing library to return data, and then the internal status control signal is reset to low level.

[0101] Waiting for the computation library to return data status: When the computation library completes the calculation signal and goes high, it means that the calculation result has been returned, so proceed to the next judgment: If the signal of completing a complete formula calculation in the neuron module does not respond, enter the data return stack state, the RAM enable signal storing the formula goes high, and then return and read the formula data status; if the signal of completing a complete formula calculation in the neuron module responds, enter the write-back data state, and then write data into RAM.

[0102] Write-back data status: The Runge-Kutta equation of one operation is obtained, and the result is compared with the threshold. If it is less than the threshold, the corresponding variable in the storage parameter RAM is updated; otherwise, a pulse signal is issued and the variable value is reset.

[0103] If the postorder expression is calculated using the above method, its computational efficiency is relatively low compared to existing computing devices for fixed-model neurons. Therefore, this invention proposes an optimization scheme based on the binary tree properties of the postorder expression, such as... Figure 7 As shown, the complex post-expression is decomposed into operations, realizing parallelization of the internal calculations of the formula, which greatly improves its calculation speed:

[0104] If the objective function is f(x) = (x+1)*(x-3), then the postorder expression is f(x) = x1 + x3 - *;

[0105] After optimization, it is decomposed into f(x1) = x11+, f(x2) = x23-, and f(x) = f(x1)*f(x2);

[0106] At the first moment, the operations of f(x1) and f(x2) are placed into two computation kernels respectively. After obtaining the results of the first two formulas, the operation of f(x) = f(x1) * f(x2) at the next moment is performed. The above operation greatly saves computation time and improves the working efficiency of the computing device.

[0107] like Figure 8 As shown, a heterogeneous acceleration method for spiking neural networks that supports custom neuron models includes the following steps:

[0108] Step S1: The user (PC) sends the network configuration parameters to the FPGA board;

[0109] Step S2: The FPGA board obtains and controls the reading and writing of the user-configured neurons and their network information in the first-level cache memory through the unified scheduling center, based on the neuron model and network information of the spiking neural network. During the downward allocation of work tasks, it selects to send the data and requests of adjacent neurons in the network to the specified cluster according to the current neural network status.

[0110] Step S3: The cluster includes a task controller and a neuron array. The task controller calls computing resources to perform calculations based on the task and corresponding data. The neuron array includes a group of neuron modules forming an array based on data exchange links to facilitate simple data exchange between neuron modules, reduce the overhead of the top-level unified scheduling center resources, and achieve a balance between time and resource overhead. The acquired tasks and corresponding data are allocated to each neuron module for calculation, and the feedback calculation results are sent to the user (PC) via the FPGA board.

[0111] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A heterogeneous acceleration device for spiking neural networks supporting custom neuron models, comprising a unified scheduling module, a first-level buffer, and a cluster module connected in sequence, characterized in that: The cluster module includes interconnected task control center modules and computing core arrays; The unified scheduling module, based on the neuron model and network information of the spiking neural network, controls the memory read and write of the first-level cache, and during the downward allocation of work tasks, selects to send adjacent neuron data and requests in the network to the specified cluster according to the current neural network status. The first-level cache is used to store the user-configured neurons and their network information; The task control center module calls computing resources to perform calculations based on the task and corresponding data. The computing core array includes an array of neuron modules based on a data exchange link, which distributes the acquired tasks and corresponding data to each neuron module for computation and feeds back the computation results; the neuron module includes a cache, a memory, a weight accumulation module, a secondary cache, a stack control module, and a computing library module; The buffer is used to store data transmitted from other neuron modules via a link. If the destination of the data is this neuron module, it is stored in the corresponding memory; otherwise, the link continues to transmit the data according to the relative position between this neuron module and the target address. The memory includes a weight storage memory and a pulse storage memory; The weight accumulation module reads data from the weight memory and pulse memory, performs accumulation calculations at the same time step, and uses the result of accumulating all weights at the same time as the stimulation signal for the neuron module to perform neuron calculation. The secondary cache includes an operator storage memory and a parameter storage memory, used to store calculation formulas and calculation parameters; The stack control module includes a state machine and a stack. The state machine reads data from the operator storage memory and the parameter storage memory according to the post-order expression and stores the data into the stack. The stack puts the data in its contents into the calculation library according to the corresponding order, obtains the calculation result and puts it back into the stack, until the post-order expression ends and the final value is obtained. It is compared with the threshold. If it is greater than or equal to the threshold, a pulse is issued and the weight value is cleared to zero. If it is less than the threshold, the current weight value is maintained and carried over to the next round of calculation. The computation library module, used for neuron computation and pulse firing, consists of a set of computation modules.

2. The heterogeneous acceleration device for spiking neural networks supporting custom neuron models according to claim 1, characterized in that: The task and corresponding data include the weights and impulse information of the neural network as external data input. The weights enter from the neuron modules at the outer corners of the array and are transmitted from the periphery to the center. The impulse information enters from the neuron modules at the inner corners of the array and is transmitted from the center to the periphery. External data is preferentially transmitted to the neuron modules at the opposite corners.

3. The heterogeneous acceleration device for spiking neural networks supporting custom neuron models according to claim 1, characterized in that: Each neuron module contains a buffer and a counter. Based on the counter's count, the one-way congestion status of the link is categorized as smooth, congested, and blocked. The routing selection under different conditions is as follows: When the links along the X-axis and the Y-axis are both open, data is transmitted preferentially along the shortest path first along the X-axis and then along the Y-axis. When the X-axis link is congested and the Y-axis link is unobstructed, data is transmitted via the shortest path that first extends along the Y-axis and then along the X-axis. When the Y-axis link is congested and the X-axis link is unobstructed, data is transmitted via the shortest path that first extends along the X-axis and then along the Y-axis. When the link in either direction is blocked and no more data can be stored, the only option is to choose the link in the other direction to transmit the data. When both the X and Y axes are blocked, the data waits in the buffer of the neuron module until the link is unblocked.

4. The heterogeneous acceleration device for spiking neural networks supporting custom neuron models according to claim 1, characterized in that: An arbitration algorithm based on link priority is constructed. When a neuron module successfully hands over a link, its priority becomes the lowest in the subsequent arbitration, and is adjusted accordingly based on the current status of other links. When data from multiple links arrive simultaneously, the neuron module completes the handshake sequentially according to the priority of its current state.

5. The heterogeneous acceleration device for spiking neural networks supporting custom neuron models according to claim 1, characterized in that: The state machine of the stack control module provides a way to judge different events in different states and determine the next action state. The state machine includes the following states: Empty state: When the start calculation signal is received and pulled high, it enters the formula data reading state, and the memory storing the formula is pulled high; Formula data reading state: Enters working state, reads data from the memory storing formulas. If the read instruction is placeholder information, then enters parameter data reading state; if the read instruction is operator information, then enters operation stack state, and the stack output signal changes from 0 to 1. Read parameter data status: At this time, the enable signal of the parameter storage memory is emitted once; If the stack is not full, enter the stack operation state; Operation stack state: If the stack output signal is 0, return to the formula data reading state; otherwise, enter the docking calculation library state. The stack output enable emits a pulse, and the calculation library enable emits a pulse after one clock cycle and transfers the value in the stack to the calculation library. Interconnected computing library status: When the computing library enable signal is pulled high, it enters the waiting state for the computing library to return data, and then the internal status control signal is reset to low level; Waiting for the computation library to return data status: When the computation library completes the calculation signal and goes high, it means that the calculation result has been returned, so proceed to the next judgment: If the signal of completing a complete formula calculation in the neuron module does not respond, enter the data return stack state, the memory enabling signal storing the formula goes high, and then return and read the formula data status; if the signal of completing a complete formula calculation in the neuron module responds, enter the write-back data state, and then write data into the memory. Write-back data status: The Runge-Kutta equation of one operation is obtained, and the result is compared with the threshold. If it is less than the threshold, the corresponding variable in the storage parameter memory is updated. Conversely, a pulse signal is emitted, and the variable value is reset.

6. The heterogeneous acceleration device for spiking neural networks supporting custom neuron models according to claim 1, characterized in that: The post-order expression is decomposed into operations, and the internal calculations of the formula are parallelized: The variables, constants, and symbols of the objective function are decomposed to obtain the decomposed postorder expression; Based on the units composed of objective function constants, variables and corresponding symbols, the post-order expression is decomposed into multi-level expressions; Based on the expression level from low to high, they are placed into neuron modules and calculated sequentially.

7. The heterogeneous acceleration device for spiking neural networks supporting custom neuron models according to claim 1, characterized in that: The task control center module includes a task station, a computing library control center, a database, and an intra-cluster destination instruction module. The task station is connected to the unified scheduling center, the computing library control center, and the intra-cluster destination instruction module, respectively. The database is connected to the computing library control center and the intra-cluster destination instruction module, respectively. The task station obtains instructions from the unified scheduling module and, based on the reading of the instructions, allows the database to store the corresponding data in the first-level cache. The computing database control center is used to control the reading and writing of the database; The database includes a formula memory, a weight memory, and a pulse memory, which are used to store formula, weight, and pulse information, respectively. The formula memory is connected to each neuron module of the array, the weight memory is connected to the neuron modules at the outer corners of the array, and the pulse memory is connected to the neuron modules at the inner corners of the array. The cluster-specific instruction module is connected to the neuron module at the inner corner of the array.

8. A heterogeneous acceleration device for spiking neural networks supporting custom neuron models according to claim 7, characterized in that: The task station includes a queue for storing tasks and a state machine for controlling task retrieval. The state machine includes the following states: Idle state: The state when power is off; it enters the idle state when the reset signal is high. Idle state: At this time, the task station has not acquired any new tasks and is allowed to read new tasks from the task queue. When a new task instruction is read, it is parsed and the process jumps to the corresponding new state machine. Load configuration file: Import data from the scheduling center into the database, and return to idle state after storage is complete; Initialize array: Place the formula stored at the specified location into the formula storage area within the specified neuron module, and return to the idle state after completion; Neuron operation: The specified neuron module performs the operation, and returns to the idle state directly after issuing the instruction; Intra-cluster data routing: Based on different data types, data enters the array from different array entrances, specifies the starting neuron to transmit pulse data to the target location, and returns to the idle state directly after issuing the instruction; Sending information outside the cluster: Send a request to the unified scheduling module after processing the data, wait for the other party's request, and return to the idle state when the data exchange is completed; The computing library control center writes data when the state machine is loading the configuration file; and reads data when the state machine is loading the array, routing data within the cluster, or sending information outside the cluster.

9. A heterogeneous acceleration device for spiking neural networks supporting custom neuron models according to claim 1, characterized in that: The unified scheduling module uses a round-robin scheduling mechanism to allocate tasks, which aims to reduce the workload of a single computing core and, when hardware resources are sufficient, distribute tasks evenly across the computing cores.

Images