Low-power Spiking Neural Network On-chip Learning System Based on Temporal Encoding

Abstract

The present invention discloses a low-power spiking neural network on-chip learning system based on time encoding, which includes a sorting module and a computing module. The sorting module is used to receive a plurality of groups of input data that have been time-encoded and arrange the input data in ascending or descending order. The computing module is used to receive the input data and weight data pre-stored and matching the input data; quantize the input data into a preset constant based on a predetermined rule to obtain quantized input data; judge whether a neuron will be excited based on the weight data, and if it is excited, calculate the output pulse time using the quantized input data and the weight data, and update the weight data using a linearization rule. By quantizing the input data, the hardware operation difficulty is reduced, and an optimized weight update rule is adopted to facilitate hardware design and reduce the amount of computation. It has the advantages of low power consumption, small area and high speed; it improves the operation speed of the spiking neural network and can realize on-chip learning on hardware.

Description

Technical Field

[0001] The present invention belongs to the field of hardware acceleration of artificial intelligence algorithms, and particularly relates to a low-power spiking neural network on-chip learning system based on time encoding. Background Art

[0002] In recent years, many neural networks for the recognition of handwritten digit pictures have been implemented on hardware. Among them, networks represented by convolutional neural networks are widely used. However, operations such as convolutional pooling require a large amount of computation, which consumes a large amount of hardware resources and is not conducive to low-power design. To address the above problems, spiking neural networks (SNNs) have emerged in recent years.

[0003] The emergence of spiking neural networks is inspired by the structure of the biological brain. The biological brain can process information efficiently and with low power in an unsupervised form. Therefore, spiking neural networks that simulate the human brain have emerged, and this network is also widely regarded as the third generation of neural networks. SNNs propagate information between neurons with discrete spikes, and the corresponding neuromorphic hardware can be implemented at low power, which is very important for battery-constrained devices.

[0004] There are mainly three methods for training SNNs: conversion from artificial neural networks (ANNs), supervised learning methods, and unsupervised learning methods. Although the converted SNNs and the SNNs converted using supervised learning methods such as backpropagation have achieved certain results, their complex algorithms make it very difficult to implement online learning of these methods in hardware.

[0005] How to achieve on-chip learning, reduce system power consumption, and reduce hardware overhead is a problem that needs to be solved currently. Summary of the Invention

[0006] The purpose of the present invention is:

[0007] To overcome the deficiencies in the hardware implementation of general neural networks, and provide a low-power spiking neural network on-chip learning system based on time encoding, which can effectively reduce power consumption, reduce hardware overhead, and can also achieve on-chip learning.

[0008] Specifically, it is realized by the following technical solutions:

[0009] A low-power spiking neural network on-chip learning system based on time encoding, comprising:

[0010] A sorting module, configured to receive a plurality of groups of input data that have been time encoded, and arrange the input data in ascending or descending order;

[0011] A calculation module, configured to receive the input data and pre-stored weight data that matches the input data; quantize the input data into a preset constant based on a predetermined rule to obtain quantized input data; determine whether a neuron will fire based on the weight data, and if it fires, calculate the output pulse time using the quantized input data and the weight data, and update the weight data using a linearization rule.

[0012] The advantages of the invention are as follows: A pulsed neural network using time coding is implemented on hardware, and it has the function of online learning; through design and improvement, the hardware system has characteristics such as low computational complexity, small area, and low power consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic diagram of the overall hardware architecture of the present invention.

[0014] Figure 2 It is a schematic diagram of the improved nine-input sorting tree structure of the present invention.

[0015] Figure 3 It is a schematic diagram of the arithmetic unit architecture of the present invention.

[0016] Figure 4 It is a schematic diagram of the CU structure of the present invention.

[0017] Figure 5 It is a schematic diagram of the weight update architecture of the present invention.

[0018] Figure 6 It is a schematic diagram of the sorting process of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0019] Based on the above problems, a low-power pulsed neural network on-chip learning system based on time coding is proposed, which mainly includes a sorting module and a calculation module.

[0020] Among them, the sorting module is configured to receive several groups of input data that have been time-coded, and arrange the input data in ascending or descending order;

[0021] The calculation module is configured to receive the input data and pre-stored weight data that matches the input data; quantize the input data into a preset constant based on a predetermined rule to obtain quantized input data; determine whether a neuron will fire based on the weight data, and if it fires, calculate the output pulse time using the quantized input data and the weight data, and update the weight data using a linearization rule.

[0022] The input data is quantized into preset constants according to preset rules, which changes the original method that requires calculating the exponential function value. Therefore, the calculation efficiency can be greatly improved, facilitating hardware implementation. At the same time, the weight update is performed using a linearization rule, which changes the problem in the existing method that the exponential calculation of the weight increment leads to low calculation efficiency. Meanwhile, the calculation accuracy is improved, and the input quantization can independently select the quantization fineness, facilitating hardware implementation, and the quantization fineness can be selected according to requirements.

[0023] In other words, different from the prior art, the processing process of data quantization is added, which solves the problems of high hardware requirements, difficult implementation, and low calculation efficiency in the existing method.

[0024] As Figure 1 shown, in a further embodiment, the SNN used in the low-power pulsed neural network on-chip learning system based on time encoding of the present invention has N in input axons and N out excitatory neurons. It mainly includes the following modules:

[0025] Sorting module, used to sort the data using time encoding from large to small or from small to large.

[0026] Calculation module, mainly used to calculate the time of pulse generation according to the quantized input data and find the first-occurring pulse.

[0027] Weight update module, updates the corresponding weight according to the first-occurring pulse.

[0028] Threshold and classification update module, updates the corresponding threshold according to the first-occurring pulse and classifies the corresponding output neuron; storage unit, stores the input data, sorting result, weight, threshold, and output classification.

[0029] The working process is as follows: Since the entire system can perform on-chip learning, the working mode is divided into a learning mode and an inference mode.

[0030] In the learning mode, the input data is sorted by the sorting module, and the data is input into the arithmetic unit from large to small. The arithmetic unit obtains the neuron with the first-occurring pulse through calculation, and then inputs the neuron index into the weight update module, threshold, and classification update module to update the corresponding weight and threshold, and label the output neuron.

[0031] In the inference mode, the sorting and calculation processes remain unchanged, but the weight, threshold, and label are no longer updated. At this time, the threshold and classification update module only outputs the label corresponding to the neuron with the first-occurring pulse. This label is the result of image classification.

[0032] The input pixels are first passed to the input pulse buffer. When all the input pixel values are stored in the input pulse buffer, the input sorting module sorts them in descending order along with the input pixel values. The sorted pixels and the corresponding pixel indices are stored in the sorted pixel buffer. Once the first row of memory in the sorted pulse buffer is full, the calculation module starts to work. The calculation module calculates for each neuron at each input pulse time to determine whether there is a neuron output. The whole system is divided into a learning mode and an inference mode.

[0033] In the learning mode, the minimum output index of the minimum-finding module is used to update the synaptic weights and the membrane threshold. According to the simplified STDP learning rule, the synaptic weights connected to the neuron with the minimum output are updated, and the weight update index and the membrane threshold (membrane potential threshold) of the neuron are increased by the membrane potential threshold increment Δv. Another task in the learning mode is to assign a class to each neuron. Only the neuron with the minimum output is assigned a class each time, because only the neuron with the minimum output can determine the output of the network. This minimum output is used to compare with the previously saved minimum output value in the class value memory.

[0034] If this minimum output is less than the previous value, we update the class of the neuron and save the minimum output value in the class value memory, otherwise we do not update. In the inference mode, no values are updated. The weight update module does not work. The membrane threshold and the class update module can still work, but the membrane threshold is not updated. This module only gives the output of the network based on the neuron whose minimum pulse time is found.

[0035] According to one aspect of the present application, the obtaining process of the predetermined rule is as follows:

[0036] Obtain the pixel values to be processed, normalize them to the interval [0, 1] to obtain the normalized pixel values. For any normalized pixel value, calculate the difference between it and 1, calculate the even power of this difference, and further obtain the value of the even power of the natural exponent e for this difference;

[0037] For all the normalized pixel values, calculate the upper and lower limits of the even power of the natural exponent e, and divide them into several intervals, and the endpoint values of each interval are multiples of 2;

[0038] Obtain the normalized pixel values corresponding to each endpoint value through the inverse operation of the above calculation process, and use them as the independent variable interval endpoints to form several independent variable intervals; use the endpoint value or the arithmetic mean of two adjacent endpoint values as the quantization target value to form several quantization constant values corresponding one-to-one to the independent variable intervals, and construct the predetermined rule with the independent variable intervals and the quantization constant values.

[0039] At runtime, find the independent variable interval corresponding to the normalized pixel value and read the quantization constant value corresponding to the independent variable interval, i.e., obtain the quantized input data.

[0040] As Figure 2 shown, in one embodiment, the predetermined rule may specifically be:

[0041] When the input data ≤ 0.125, the quantization constant value Z q = 2;

[0042] When 0.125 ≤ input data ≤ 0.3125, the quantization constant value Z q = 1.5;

[0043] When the input data ≥ 0.3125, the quantization constant value Z q = 1.

[0044] Z q is the quantized input data. After the data is quantized, without affecting the calculation accuracy, the operation difficulty is greatly reduced and the consumption of area resources is reduced.

[0045] Based on this research finding, there is no specific relationship between the computational complexity and the accuracy. The solution proposed in this application not only reduces the complexity but also improves the accuracy.

[0046] According to one aspect of this application, the sorting process of the sorting module is as follows:

[0047] According to the set of prime factors x t × y k × z j of the number of input data groups, construct an α-input sorting tree and a β-input sorting tree;

[0048] where α < β, β - α ≤ constant C, α, β ∈ {x a , y b , z c}, where 1 ≤ a ≤ t, 1 ≤ b ≤ k, 1 ≤ c ≤ j; α, β, a, b, c are all positive integers;

[0049] Divide the N in groups of input data into N in / α vectors, each vector consisting of α data, and save them to the storage unit;

[0050] Sequentially send the Nin / α vectors to the α-input sorting tree,

[0051] Divide the N in groups of input data into N in / β vectors, each vector consisting of β data, and save them to the storage unit;

[0052] Send N in / β vectors to the β-input sorting tree in sequence,

[0053] Repeat the above sorting process, and finally obtain the sorting of N in input pixels.

[0054] In a certain embodiment, if there are 12 groups of data, they can be first split into pairs and divided into 6 groups of data. After each group is sorted, they are then split into 4 groups of data in sequence and sorted again. Repeating the above process can sort all 12 groups of data.

[0055] Through this design, it is convenient for hardware implementation, saves hardware resources, and improves calculation efficiency at the same time. Through factorization, sorting trees with similar input numbers are obtained. And using sorting trees with similar input numbers can be reused, which is convenient for reducing area resources. And this method is faster than bubble sorting, and compared with parallel sorting, the speed is basically the same but the resources are greatly reduced.

[0056] As Figure 3 described, according to one aspect of the present application, the computing module includes K cal computing units and a minimum finding module,

[0057] Each time K cal neurons are calculated, and each time a pulse input is received, K cal computing units perform parallel calculations,

[0058] After the calculation is completed, observe whether there is an output pulse. If so, stop the calculation,

[0059] Otherwise, continue with the next input pulse until an output is generated or all input pulses have been input;

[0060] If there are multiple output pulses in one round, the minimum finding module compares and obtains the minimum input pulse.

[0061] Specifically, the computing module in this system is divided into two parts: computing and finding the minimum value. Since there are n out neurons in the output, theoretically these n out neurons should be calculated in parallel, but it consumes too much in hardware implementation. Therefore, n cal computing units are used, and only n cal neurons are calculated at a time. Each time a pulse input is received, n cal computing units perform parallel calculations, and n out / n calAfter a round, observe whether there is an output pulse. If there is, stop the calculation. If not, continue with the next input pulse until an output is generated or all input pulses have been input. If there are multiple output pulses in a round, then to find the minimum module, compare which of these pulses is smaller, that is, which occurs earlier.

[0062] The calculation module consists of a control block and n cal control units CU. The control unit CU can calculate one output neuron each time, so it can calculate n cal excitatory neurons at each input pulse. The calculation module accesses the sorted buffer and two pixels, the pixels quantized in the previous text, and sends them to the control units CUs. The corresponding synaptic weights, membrane thresholds, and intermediate results of the operations are also sent to the control units CUs. Since there are n out neurons in the output, theoretically these n out neurons should be calculated in parallel, but it consumes too much in hardware implementation. Therefore, n cal calculation modules are used, and only n cal neurons are calculated at a time. If a neuron emits a pulse, the control unit CU will send a stop signal. When the calculation of this round ends, the calculation process will stop. If not, continue with the next input pulse until an output is generated or all input pulses have been input. If there are multiple output pulses in a round, then to find the minimum module, compare which of these pulses is smaller, that is, which occurs earlier. Due to the input quantization scheme, two multipliers are not required in the control unit CU, but only shifters and adders are needed, and the structure is as Figure 4 shown.

[0063] According to one aspect of the present application, the process of updating weight data according to the linearization rule includes:

[0064] Read the pre-stored upper weight limit w max and lower weight limit w min of each synapse, the original weight w old of each synapse, and the weight update coefficient η.

[0065] Calculate the time difference between the pre-synaptic neuron and the post-synaptic neuron, and determine whether the time difference is greater than zero.

[0066] If it is greater, then the weight increment Δw of the synapse = η(w max - w old );

[0067] Conversely, the weight increment Δw of the synapse = -η(w old - w min ).

[0068] By adopting an improved weight update rule, the operation difficulty is greatly reduced, the area is decreased, the power consumption is lowered, and the accuracy is also improved. In the existing algorithms, the weight update rule requires exponential operations, which greatly increases the difficulty of hardware design. The weight update module simplifies the exponential weight update for time encoding at the hardware level, and the new weight update rule also has upper and lower bounds to avoid weight update explosion.

[0069] As Figure 5 shown, due to the simplified weight update rule, when an appropriate learning rate is selected, only adders, subtracters, and shifters are needed in the weight update unit. First, the corresponding synaptic weights are extracted from the weight memory and enhanced or weakened according to the weight update exponent; then the new synaptic weights are stored back in the weight memory. This process of updating a single synaptic weight consumes 4 clock cycles.

[0070] According to one aspect of the present application, it further includes a threshold and classification update module.

[0071] In the learning mode, calculate and store the increase in the membrane threshold ΔV of the neuron corresponding to the minimum output value, and compare the increase in the membrane threshold ΔV with a predetermined value. If it is greater than the predetermined value, assign a new class to the neuron and update the predetermined value of the neuron. In this way, the membrane voltage threshold value of the neuron is increased, thereby preventing the neuron from being continuously strengthened.

[0072] The threshold and classification update module in the low-power spiking neural network on-chip learning system based on time encoding is used to update the threshold and classification. In the learning mode, the membrane threshold of the neuron with the minimum output increases by Δv. Compare the current minimum output value with the previous minimum output value of the neuron to determine whether to assign a new class to the neuron and update the minimum output value of the neuron. In the inference mode, the module gives the output of the network through the specified class.

[0073] In a further embodiment, the process of determining whether a neuron will fire based on the weight data is specifically

[0074] the sum of the current weight data. If it is greater than the firing threshold, it will fire.

[0075] In a further embodiment, the process of calculating the output pulse time using the quantized input data and weight data is specifically:

[0076] The output pulse time t out = ln((the sum of the products of the weight data and the quantized input data) / (the difference between the sum of the weight data and the firing threshold)).

[0077] As Figure 6As shown, in a further embodiment, the sorting process may also be carried out in the following manner:

[0078] The sorting process of the sorting module is as follows:

[0079] According to the set of prime factors x t ×y k ×z j of the number of input data groups, construct an α input sorting tree and a β input sorting tree;

[0080] where the difference between α and β is less than a predetermined value, and α and β are factors or products of factors of the number of input data groups;

[0081] Divide the N in groups of input data into N in / α vectors, each vector having α data, save them to the storage unit, and sequentially send the N in / α vectors to the α input sorting tree to obtain N in / α sorted vectors;

[0082] When N in / α is a multiple of α, split the N in / α sorted vectors into N in / α 2 vectors, and sequentially send them to the α input sorting tree; obtain N in / α 2 sorted vectors;

[0083] When N in / α is not a multiple of α, fill in vectors outside the range of several groups of input data to make it satisfy that N in / α is a multiple of α, and then split it into N in / α 2 vectors, and sequentially send them to the α input sorting tree; obtain N in / α 2 sorted vectors;

[0084] Judge whether N in / α 2 is a multiple of β, and send the N in / α 2 sorted vectors to the β input sorting tree;

[0085] When N in / α 2 is not a multiple of β, fill in vectors outside the range of several groups of input data to make it satisfy that N in / α 2 is a multiple of β, and sequentially send them to the β input sorting tree.

[0086] For example, for a 24*24 input data, since 576 = 2 6 *3 2 , first use an 8-input sorting tree to divide N in input pixel values into N in / 8 vectors, each vector consisting of 8 data, and save them back to the storage unit. Then split the 72 sorted vectors into 9 groups, each group having 8 vectors, and send every 8 vectors to the 8-input sorting tree again to obtain N in / 64 = 9 vectors each consisting of 64 data. Then send every 9 vectors to 9-input sorting trees, and so on until the sorting of N in input pixels is obtained.

[0087] In the above embodiments, 576 and 72 are both multiples of 8 and 9. When they do not meet the above conditions, it can be achieved by filling invalid input data.

[0088] According to one aspect of the present application, the sorting process of the sorting module can also be carried out in the following way:

[0089] Obtain the number of groups of input data, factorize it, and obtain at least two factors α and β whose difference is less than a predetermined value and the product of positive integer powers is equal to the number of groups of input data; sequentially construct an α-input sorting tree and a β-input sorting tree;

[0090] Divide N in groups of input data into vectors each containing α or β data, and send them to the α-input sorting tree or the β-input sorting tree so that each group of data is sorted;

[0091] Then divide the sorted several groups of vectors into α or β groups again, each group containing α or β vectors, and send them to the α-input sorting tree or the β-input sorting tree;

[0092] Repeat the above process until N in groups of input data are sorted.

[0093] Therefore, when designing the input data, the number of groups of input data N in can be designed in the form of α λ ×β τ = N in .

[0094] At the first sorting, obtain α (λ-1) ×β τ vectors;

[0095] At the second and subsequent sortings, the already sorted vectors can be directly input into the sorting tree. For example, α (λ-2) ×β τα vectors in each group are input into the sorting tree,

[0096] and so on. Finally, β vectors in each group are input into the β-input sorting tree.

[0097] λ and τ are the number of sorting times. By designing the number of input data groups, hardware savings can be achieved, the area can be reduced, and the reuse times and calculation speed can be improved. If it is designed into three prime factors, the above method can also be used for calculation.

[0098] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept scope of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all belong to the protection scope of the present invention.

Claims

1. A low-power spiking neural network on-chip learning system based on time encoding, characterized in that, it includes: A sorting module for receiving several groups of input data that have been time-encoded and arranging the input data in ascending or descending order; A calculation module for receiving the input data and weight data pre-stored and matching the input data; Quantize the input data into a preset constant based on a predetermined rule to obtain quantized input data; Judge whether a neuron will fire based on the weight data. If it fires, calculate the output pulse time using the quantized input data and weight data, and update the weight data using a linearization rule; The obtaining process of the predetermined rule is: Obtain the pixel value to be processed, normalize it to the interval [0, 1] to obtain the normalized pixel value. For any normalized pixel value, calculate the difference between it and 1, calculate the even power of the difference, and further obtain the value of the even power of the natural exponential e; For all normalized pixel values, calculate the upper and lower limits of the even power of the natural exponential e and divide them into several intervals, and the endpoint values of each interval are multiples of 2; Obtain the normalized pixel values corresponding to each endpoint value through the inverse operation of the above calculation process, and use them as the endpoints of the independent variable interval to form several independent variable intervals; use the endpoint value or the arithmetic mean of two adjacent endpoint values as the quantization target value to form several quantization constant values corresponding one-to-one with the independent variable interval, and construct the predetermined rule with the independent variable interval and the quantization constant value; During operation, find the independent variable interval corresponding to the normalized pixel value and read the quantization constant value corresponding to the independent variable interval to obtain the quantized input data; The process of updating the weight data by the linearization rule includes: Read the upper limit w of the weights of the pre-stored synapses max , the lower limit w min , the original weights w of each synapse old , and the weight update coefficient η Calculate the time difference between the pre-synaptic neuron and the post-synaptic neuron and judge whether the time difference is greater than zero, If it is greater, the synaptic weight increment ∆w = η (w max - w old ); Conversely, the synaptic weight increment ∆w = -η (w old - w min ).

2. The low-power spiking neural network on-chip learning system based on time encoding according to claim 1, characterized in that, The sorting process of the sorting module is as follows: Based on the set of prime factors x of the number of groups of the input data t ×y k ×z j , construct an α input sorting tree and a β input sorting tree; where α < β, β - α ≤ constant C, α, β ∈ {x a , y b , z c}, where 1 ≤ a ≤ t, 1 ≤ b ≤ k, 1 ≤ c ≤ j; α, β, a, b, c, t, j, k are all positive integers; Divide N in groups of input data into N in / α vectors, where each vector consists of α data and is saved to the storage unit; Send N in / α vectors to the α-input sorting tree in sequence, Divide N in groups of input data into N in / β vectors, where each vector consists of β data and is saved to the storage unit; sequentially send N in / β vectors to the β-input sorting tree Repeat the above sorting process, and finally obtain the sorting of N in input pixels.

3. The low-power spiking neural network on-chip learning system based on time encoding according to claim 2, characterized in that, The calculation module includes K cal calculation units and a minimum-finding module, Each time K is calculated cal neurons, and each time a pulse input is received, K cal computing units perform parallel computing After the calculation, observe whether an output pulse occurs. If so, stop the calculation, On the contrary, continue with the next input pulse until an output is generated or all input pulses have been input; If there are multiple output pulses in one round, find the minimum module to compare and obtain the minimum input pulse.

4. The low-power spiking neural network on-chip learning system based on time encoding according to any one of claims 1 to 3, characterized in that, It further includes a threshold and classification update module, In the learning mode, calculate and store the membrane threshold increase amount ∆V of the neuron corresponding to the minimum output value, and compare the membrane threshold increase amount ∆V with a predetermined value. If it is greater than the predetermined value, assign a new class to the neuron and update the predetermined value of the neuron.

5. The low-power spiking neural network on-chip learning system based on time encoding according to claim 1, characterized in that, The sorting process of the sorting module is as follows: Based on the set of prime factors x of the number of groups of the input data t ×y k ×z j , construct an α input sorting tree and a β input sorting tree; wherein, the difference between α and β is less than a predetermined value, and α and β are factors or the product of factors of the number of input data groups; Divide N in groups of input data into N in / α vectors, each vector having α data, and save them to the storage unit Send N in / α vectors to the α-input sorting tree in sequence to obtain N in / α sorted vectors; When N in / α is a multiple of α, split N in / α sorted vectors into N in / α 2 vectors, and send them to the α-input sorting tree in sequence; obtain N in / α 2 sorted vectors; When N in / α is not a multiple of α, fill in several vectors outside the input data range to make it satisfy N in / α is a multiple of α, and then split it into N in / α 2 vectors, and send them to the α-input sorting tree in sequence; obtain N in / α 2 sorted vectors; Determine N in / α 2 to see if it is a multiple of β. Send the in / α 2 sorted vectors to the β-input sorting tree; When N in / α 2 is not a multiple of β, fill in several vectors outside the input data range to make N in / α 2 a multiple of β, and send them to the β-input sorting tree in sequence.

6. The on-chip learning system of a low-power spiking neural network based on time encoding as described in claim 1, characterized in that, the sorting process of the sorting module is as follows: obtain the number of input data groups, factorize it, and obtain at least two factors α and β whose difference is less than a predetermined value and the product of positive integer powers is equal to the number of input data groups; sequentially construct an α-input sorting tree and a β-input sorting tree; Divide N in groups of input data into vectors each containing α or β data, and send them to an α-input sorting tree or a β-input sorting tree, so that each group of data is sorted; then divide the sorted several groups of vectors into α or β groups again, each group contains α or β vectors, and send them to the α-input sorting tree or the β-input sorting tree; Repeat the above process until N in The group of input data is sorted.

Images