Neural network optimization method and neural network computing device integrating sparse distributed memory and spiking neural network

Abstract

This disclosure provides a neural network optimization method suitable for neural network computing devices integrating sparse distributed memory (SDM) and spiking neural networks (SNN). The method generates a Fibonacci sequence based on the structure of the input data, and then generates multiple constant-weight codes with M-of-N properties based on this sequence to be stored as an address index matrix in the SDM. By calculating the similarity between the input data and the address index matrix, target neurons in the SNN can be selectively activated, thereby significantly reducing computational resource consumption and power consumption. This disclosure also provides a neural network computing device for implementing this method.

Claims

1. A neural network optimization method applicable to neural networks integrating Sparse Distributed Memory (SDM) and Spiking Neural Network (SNN), the method comprising: The Fibonacci sequence is generated based on the input data structure of the SNN; multiple bit segments of different lengths are generated based on the Fibonacci sequence, wherein the values ​​of multiple bits in each bit segment are initialized to 0; multiple constant codes with M-of-N characteristics are generated based on these multiple bit segments, where M represents the number of bits with a value of 1 in each constant code, and N represents the total number of bits in each constant code; these multiple constant codes are stored as an address index matrix in the SDM; based on the input data and the address index matrix with dispersion characteristics stored in the SDM, the correspondence between the input data and multiple neurons of the SNN is obtained to determine the multiple target neurons to be activated in the SNN, thereby reducing the resources consumed by the SNN when performing operations.

2. The neural network optimization method as described in claim 1, wherein generating the Fibonacci sequence based on the input data structure of the SNN comprises: The number of segments is determined based on the dimensions of the input data; And generating multiple values ​​of the Fibonacci sequence based on the number of segments, wherein the total number of the multiple values ​​is equal to the number of segments, wherein generating multiple bit segments of different lengths based on the Fibonacci sequence includes: setting the respective lengths of the multiple bit segments based on the multiple values ​​of the Fibonacci sequence to generate the multiple bit segments, wherein the total number of bits in each bit segment is equal to its length, and the total number of bit segments generated is equal to the number of segments.

3. The neural network optimization method as described in claim 2, wherein generating the Fibonacci sequence based on the input data structure of the SNN further includes: Perform a base-2 first logarithm operation on the dimension of the input data to obtain the number of segments for the multiple bit segments; Perform a second logarithmic operation to base 2 on the number of segments to obtain the first seed value of the Fibonacci sequence; obtain the second seed value of the Fibonacci sequence based on the dimension and the first seed value; and generate multiple values ​​of the Fibonacci sequence based on the first seed value and the second seed value.

4. The neural network optimization method as described in claim 3, wherein the second seed value is calculated by the following formula: b = Round((n-55a) / 88), where b is the second seed value, n is the dimension, a is the first seed value, and Round represents rounding, wherein generating multiple values ​​of the Fibonacci sequence based on the first seed value and the second seed value of the Fibonacci sequence includes: The first two values ​​of the Fibonacci sequence are set as the first seed value and the second seed value, and each subsequent value is set as the sum of the first two values ​​of the subsequent value.

5. The neural network optimization method as described in claim 2, wherein generating multiple bit segments of different lengths based on the Fibonacci sequence further includes: The length of each bit segment is set to the value corresponding to that bit segment among multiple values ​​in the Fibonacci sequence; A first quantity corresponding to each bit segment is set according to the length of each bit segment; according to the first quantity corresponding to each bit segment, one or more first bits of the first quantity in each bit segment are set to 1, wherein the position of the one or more first bits in each bit segment conforms to a predetermined rule; and multiple random permutations are performed on all bit segments using each bit segment as a permutation unit to generate multiple constant weight codes with a number equal to the number of neurons in the SNN.

6. The neural network optimization method as described in claim 1, wherein obtaining the correspondence between the input data and multiple neurons of the SNN based on the input data and the address index matrix with scattering characteristics stored in the SDM includes: Receive and accumulate multiple input pulse signals corresponding to the input data to form an input binary vector; calculate multiple similarities between the input binary vector and multiple address index columns of the address index matrix; set a masking threshold to filter multiple specific address index columns with similarity higher than the masking threshold; and generate a binary mask based on the multiple specific address index columns to determine multiple target neurons to be activated in the SNN.

7. The neural network optimization method as described in claim 6, further comprising: Perform the inference phase of the SNN, wherein all parameters of the SNN remain unchanged and all neurons of the SNN are activated without being restricted by the binary mask; obtain the most active neuron that outputs the most pulses in the inference phase from all neurons of the SNN; if the most active neuron is one of multiple target neurons corresponding to the binary mask, increase the mask threshold by a predetermined difference. And if the most active neuron is not one of the multiple target neurons corresponding to the binary mask, then the mask threshold is reduced by the predetermined difference.

8. The neural network optimization method as described in claim 6, wherein calculating multiple similarities between the input binary vector and multiple address index columns of the address index matrix includes calculating multiple inner products between the input binary vector and each address index column, wherein generating the binary mask includes: When the inner product corresponding to a specific address index column is greater than or equal to the masking threshold, the bit in the binary mask corresponding to the specific address index column is set to 1; otherwise, it is set to 0.

9. The neural network optimization method as described in claim 6, wherein calculating multiple similarities between the input binary vector and multiple address index columns of the address index matrix includes calculating multiple Hamming distances between the input binary vector and each address index column, wherein generating the binary mask includes: When the Hamming distance corresponding to a specific address index column is less than or equal to the masking threshold, the bit in the binary mask corresponding to the specific address index column is set to 1; otherwise, it is set to 0.

10. The neural network optimization method as described in claim 1, wherein the neural network computing device comprises: The SDM address decoder is used to store the address index matrix; A neuron-based processing unit; A shared neuronal memory used to store the weights and neuron parameters of the SNN; The method further includes an input / output interface for receiving the input data, wherein obtaining the correspondence between the input data and multiple neurons of the SNN includes: receiving the input data with n dimensions through the input / output interface and converting the input data into multiple corresponding input pulse signals; calculating multiple similarities between an input binary vector and multiple address index columns of the address index matrix through the SDM address decoder, wherein the input binary vector is accumulated through the multiple input pulse signals; filtering the multiple similarities based on a masking threshold through the SDM address decoder to generate a binary mask, wherein the binary mask is used to indicate multiple target neurons to be activated in the SNN, wherein the method further includes: obtaining the multiple target neurons through the neuron processing processor based on the binary mask, and obtaining the weights and neuron parameters corresponding to each target neuron from the shared neuron memory; and activating the multiple target neurons through the neuron processing processor according to the weights and neuron parameters of each target neuron to perform the operation.

11. The neural network optimization method as described in claim 10, further comprising, after obtaining the plurality of target neurons: The neural network processor obtains multiple non-target neurons from the multiple neurons of the SNN, excluding the multiple target neurons. Furthermore, by applying a clock gate control signal to the multiple non-target neurons through the neuron processing processor to shut down the clock signal of each non-target neuron, the energy consumption of the SNN is further reduced.

12. A neural network computing device integrating sparse distributed memory (SDM) and spiking neural network (SNN), the neural network computing device comprising: An SDM address decoder includes: an address index circuit unit; an arithmetic unit; a neuron processing processor configured to activate one or more neurons of the SNN to perform operations; a shared neuron memory configured to store the weights and neuron parameters of each neuron of the SNN; and an input / output interface configured to receive input data corresponding to the SNN and convert the input data into a plurality of corresponding input impulse signals. The address index circuit unit is configured to generate a Fibonacci sequence based on the structure of the input data. The address index circuit unit is further configured to generate a plurality of bit segments of different lengths based on the Fibonacci sequence. The address index circuit unit is further configured to generate a plurality of constant-similarity codes with M-of-N characteristics based on the plurality of bit segments, where M represents the number of bits with a value of 1 in each constant-similarity code, and N represents the total number of bits in each constant-similarity code. The address index circuit unit is further configured to store the plurality of constant-similarity codes as an address index matrix. The computation unit is configured to obtain the correspondence between the input data and multiple neurons of the SNN based on the input data and the address index matrix with distributed characteristics stored in the SDM, so as to determine the multiple target neurons to be activated in the SNN, thereby reducing the resources consumed by the SNN when performing the operation.

13. The neural network computing apparatus of claim 12, wherein the address indexing circuit unit is further configured to: determine a number of segments based on the dimension of the input data; and generate a plurality of values ​​of the Fibonacci sequence based on the number of segments, wherein the total number of the plurality of values ​​is equal to the number of segments, wherein when a plurality of bit segments of different lengths are generated based on the Fibonacci sequence, the address indexing circuit unit is further configured to: set the respective lengths of the plurality of bit segments based on the plurality of values ​​of the Fibonacci sequence to generate the plurality of bit segments, wherein the total number of bits in each bit segment is equal to its length, and the total number of bit segments generated is equal to the number of segments.

14. The neural network computing apparatus of claim 13, wherein the address indexing circuit unit is further configured to: perform a base-2 first logarithmic operation on the dimension of the input data to obtain the number of segments of the plurality of bit segments; perform a base-2 second logarithmic operation on the number of segments to obtain a first seed value of the Fibonacci sequence; obtain a second seed value of the Fibonacci sequence based on the dimension and the first seed value; and generate a plurality of values ​​of the Fibonacci sequence based on the first seed value and the second seed value of the Fibonacci sequence.

15. The neural network computing apparatus of claim 14, wherein the second seed value is calculated by the following formula: b = Round((n-55a) / 88), where b is the second seed value, n is the dimension, a is the first seed value, and Round represents rounding, wherein when multiple values ​​of the Fibonacci sequence are generated based on the first seed value and the second seed value, the address index circuit unit is further configured to: set the first two values ​​of the Fibonacci sequence as the first seed value and the second seed value, and set each subsequent value as the sum of the first two values ​​of the subsequent value.

16. The neural network computing apparatus of claim 13, wherein when generating a plurality of bit segments of different lengths according to the Fibonacci sequence, the address indexing circuit unit is further configured to: set the length of each bit segment to a value corresponding to the bit segment among a plurality of values ​​in the Fibonacci sequence; set a first quantity corresponding to each bit segment according to the length of each bit segment; set one or more first bits of the plurality of bits in each bit segment to 1 according to the first quantity corresponding to each bit segment, wherein the position of the one or more first bits in each bit segment conforms to a predetermined rule; and perform multiple random permutations on all bit segments as permutation units to generate a plurality of constant weight codes equal in number to the number of neurons in the SNN.

17. The neural network computing apparatus of claim 12, wherein when a correspondence between the input data and a plurality of neurons of the SNN is obtained based on the input data and an address index matrix with distributed characteristics stored in the SDM, the computing unit is configured to: receive and accumulate a plurality of input impulse signals corresponding to the input data to form an input binary vector; calculate a plurality of similarities between the input binary vector and a plurality of address index columns of the address index matrix; set a masking threshold to filter a plurality of specific address index columns having similarities higher than the masking threshold; and generate a binary mask based on the plurality of specific address index columns to determine a plurality of target neurons to be activated in the SNN.

18. The neural network computing apparatus of claim 17, wherein the computing unit is further configured to: perform an inference phase of the SNN, wherein in the inference phase, all parameters of the SNN remain unchanged, and all neurons of the SNN are activated without being restricted by the binary mask; obtain the most active neuron among all neurons of the SNN that outputs the most pulses in the inference phase; if the most active neuron is one of a plurality of target neurons corresponding to the binary mask, increase the mask threshold by a predetermined difference; and if the most active neuron is not one of a plurality of target neurons corresponding to the binary mask, decrease the mask threshold by the predetermined difference.

19. The neural network computing apparatus of claim 17, wherein when calculating multiple similarities between the input binary vector and multiple address index columns of the address index matrix, the computing unit is configured to calculate multiple inner products between the input binary vector and each address index column, wherein when generating the binary mask, the computing unit is configured to: set the bit in the binary mask corresponding to the specific address index column to 1 when the inner product corresponding to a specific address index column is greater than or equal to the mask threshold, otherwise set it to 0.

20. The neural network computing apparatus of claim 17, wherein when calculating multiple similarities between the input binary vector and multiple address index columns of the address index matrix, the computing unit is configured to calculate multiple Hamming distances between the input binary vector and each address index column, wherein when the binary mask is generated, the computing unit is configured to: set the bit in the binary mask corresponding to the specific address index column to 1 when the Hamming distance corresponding to a specific address index column is less than or equal to the mask threshold, otherwise set it to 0.

21. The neural network computing apparatus of claim 12, wherein when the correspondence between the input data and a plurality of neurons of the SNN is obtained: the input / output interface is configured to receive the input data having n dimensions and convert the input data into a plurality of corresponding input impulse signals; the SDM address decoder is configured to calculate a plurality of similarities between an input binary vector and a plurality of address index columns of the address index matrix, wherein the input binary vector is accumulated through the plurality of input impulse signals; the SDM address decoder is further configured to filter the plurality of similarities based on a masking threshold to generate a binary mask, wherein the binary mask is used to indicate a plurality of target neurons to be activated in the SNN, wherein the neuron computing processor is further configured to obtain the plurality of target neurons based on the binary mask and obtain weights and neuron parameters corresponding to each target neuron from the shared neuron memory.

22. The neural network computing apparatus of claim 21, wherein after obtaining the plurality of target neurons, the neural network processor is further configured to: obtain a plurality of non-target neurons other than the plurality of target neurons in the plurality of neurons of the SNN; and apply a clock gate control signal to the plurality of non-target neurons to shut down the clock signal of each non-target neuron, thereby further reducing the energy consumption of the SNN.

23. The neural network computing apparatus of claim 12, wherein the neuron computing processor is further configured to activate the plurality of target neurons to perform the computing operation based on the weights and neuron parameters of each target neuron.

Images