Deep sparse coding methods for spiking neural networks

Abstract

The present application proposes a deep sparse coding method for a pulse neural network, which includes: N sparse coding layers; for the first N-1 sparse coding layers, each sparse coding layer corresponds to a sliding window cropping layer and a reconstruction arrangement layer; the sliding window cropping layer is located before the sparse coding layer, and the reconstruction arrangement layer is located after the sparse coding layer; a sparse coding layer and a corresponding sliding window cropping layer and a reconstruction arrangement layer together constitute a sparse coding module. The present application uses a novel deep structure of a multi-layer sparse coding module, which can effectively extract the features of the input signal and convert it into a pulse sequence, and at the same time complete the dimensionality reduction of the input data. The use of this coding method can effectively reduce the computational pressure of the subsequent pulse neural network and bring performance improvement.

Description

Technical Field

[0001] The present application relates to coding technology, and in particular to a deep sparse coding method for pulse neural networks. Background Art

[0002] The spiking neural network is called the third generation of neural networks. Compared with the second generation of neural networks: deep neural networks, the spiking neural network uses spiking neurons that are more similar to biological brains, using spikes or action potentials to transmit information. The spiking neural network is highly nonlinear and event driven, and has great potential in information processing capabilities and power consumption.

[0003] The input of the spiking neural network is a multi-dimensional pulse sequence. The number of dimensions corresponds to the number of neurons in the spiking neural network input. The pulse sequence is a sequence of 1 and 0, where 1 represents a pulse at this moment and 0 represents no pulse. The samples that the spiking neural network needs to process, such as pictures, text, videos, electrical signals, etc., have their own formats. For example, pictures are composed of pixels, and pixels are divided into three channels of RGB. Each pixel in each channel is represented by a value from 0 to 255. The process of converting samples into pulse sequences is called spike encoding.

[0004] Taking a single-channel image (grayscale image) as an example, the most commonly used pulse encoding methods in the industry are introduced: ① Frequency encoding (rate encoding): Normalize the brightness (value range is 0 to 1) as the probability of the pixel pulse emission. The emission probability of 255 brightness is 1, that is, it is continuously emitted, and the emission probability of 0 brightness is 0, that is, it is never emitted. ② Delay encoding (latency encoding), normalize the brightness, take the reciprocal as the delay time of pulse emission, 255 brightness is emitted first, 0 brightness is emitted last (or not emitted), and each pixel in the sequence will only emit one pulse. In addition to the above two methods, researchers will also use personalized encoding methods according to specific needs, but they are basically simple format conversions and do not have (or very limited) information extraction functions.

[0005] In artificial intelligence, information extraction is usually achieved by neural networks. For example, convolutional neural networks (CNN) and multi-layer perceptrons (MLP), but these neural networks usually rely on supervised learning or self-supervised learning back propagation (BP) for training, and few unsupervised learning methods are suitable for these networks. Unsupervised learning is an important branch of machine learning. Compared with supervised learning, it has three significant advantages. First, unsupervised learning uses easy-to-obtain unlabeled data and has low training costs. Second, unsupervised learning is not prone to overfitting, and the features extracted are usually more versatile, that is, the neural network obtained by unsupervised training on data set A can also show good information extraction capabilities on the same type of data set B. Third, unsupervised learning does not have a back propagation step, which is more biologically credible.

[0006] Sparse coding is an unsupervised algorithm that aims to find a set of basis vectors D for a set of samples X so that X can be represented as a sparse linear combination of basis vectors.

[0007] In neural network training, the sample set X is usually very large, and the sparse coding problem cannot be solved directly, so an iterative method is generally used. The process of continuously updating the dictionary through iteration is called dictionary learning. Summary of the invention

[0008] In view of the above problems, the present application aims to propose a deep sparse coding method for pulse neural networks.

[0009] The deep sparse coding method for a pulse neural network of the present application comprises: N sparse coding layers;

[0010] For the first N-1 sparse coding layers, each sparse coding layer corresponds to a sliding window cropping layer and a reconstruction arrangement layer; the sliding window cropping layer is located before the sparse coding layer, and the reconstruction arrangement layer is located after the sparse coding layer; a sparse coding layer and a corresponding sliding window cropping layer and a reconstruction arrangement layer together constitute a sparse coding module;

[0011] For each sparse coding module, each input information is firstly sampled by the sliding window cropping layer to obtain a sampled signal, then the sparse coding layer performs sparse coding on the sampled signal, and finally the reconstruction and arrangement layer rearranges the sparse coding result;

[0012] For each sparse coding module, when sliding window sampling is performed, for each input signal, starting from the starting point, sliding window sampling is performed according to a predetermined route, a predetermined step length, and a predetermined size to form a first number of block signals; the first number of block signals constitute a sample set; when sparse coding is performed, a dictionary is used to perform sparse coding on each block signal in the sample set, each block signal corresponds to a block sparse coefficient, and a first number of block sparse coefficients are obtained; wherein the vector of the dictionary is a second number; each block sparse coefficient corresponds to a second number of dimensional vectors; when rearrangement is performed, the first number of block sparse coefficients are rearranged into a second number of matrices, each matrix includes a first number of values, and the first number of values ​​respectively correspond to the corresponding components of the first number of block sparse coefficients; the second number of matrices are the second number of output signals generated by the input signal;

[0013] Each sparse coding module is executed independently of each other, and the output signal of the previous sparse coding module is used as the input signal of the next sparse coding module;

[0014] The Nth sparse coding layer performs sparse coding again on the output signal of the N-1th sparse coding module, and binarizes the sparse coefficients to obtain a pulse sequence;

[0015] Wherein N is a natural number greater than 1.

[0016] Preferably, dictionaries of various sparse coding modules are independent of each other, step sizes of various sparse coding modules are independent of each other, and predetermined sizes of various sparse coding modules are also independent of each other.

[0017] Preferably, at least one of the multiple sparse coding modules further includes a pooling layer, and the pooling layer is arranged after the reconstruction arrangement layer or before the sliding window cropping layer of the module.

[0018] Preferably, the input signal of the first sparse coding module is a one-dimensional or multi-dimensional signal.

[0019] Preferably, the input signal of the first sparse coding module is a two-dimensional signal, and when sliding window cropping is performed on the two-dimensional signal, the cropping is performed according to an S-shaped trajectory starting from the upper left and ending at the lower right.

[0020] Preferably, the dictionary of each sparse coding layer is updated by online learning; after sparse coding, a small batch dictionary learning algorithm is used to update the dictionary of each sparse coding layer.

[0021] Preferably, the dictionary of each sparse coding layer is updated by layer-by-layer pre-training; starting from the first sparse coding layer, the dictionaries corresponding to the N sparse coding layers are trained layer by layer.

[0022] This application uses a new deep structure of multi-layer sparse coding modules, which can effectively extract the features of the input signal and convert it into a pulse sequence, while also reducing the dimensionality of the input data. Using this encoding method can effectively reduce the computational pressure of subsequent pulse neural networks and bring performance improvements. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A schematic diagram of a flow chart of a deep sparse coding method for a spiking neural network of the present application;

[0024] Figure 2 Schematic diagram of cutting out a sample set of a two-dimensional signal;

[0025] Figure 3 Schematic diagram for rearranging sparse coefficients of a two-dimensional signal;

[0026] Figure 4 Schematic diagram of sparse coefficient pulse conversion;

[0027] Figure 5 Schematic diagram of cutting out a sample set of a one-dimensional signal;

[0028] Figure 6 Schematic diagram of rearranging sparse coefficients of a one-dimensional signal;

[0029] Figure 7 A schematic diagram of cropping a sample set of a three-dimensional signal;

[0030] Figure 8 Schematic diagram of rearranging the sparse coefficients of a three-dimensional signal. DETAILED DESCRIPTION

[0031] As an unsupervised learning method, sparse coding has a strong feature extraction function. Its learning goal is to find a set of dictionaries to make the sparse coefficients of the samples as sparse as possible. After training, the basis vectors in the dictionary contain common features in the sample library, and the sparse coefficients of the samples represent the amount of various features contained in the sample. The sparse coefficients are sparse as the name implies, and after binarization, a pulse sequence can be obtained. The features of the samples are converted into timing information, which is included in the moment of pulse emission. Blocks containing similar features will emit pulses at the same time, which is exactly the same as the synchronous spiking mechanism in the brain.

[0032] The present application is a deep sparse coding neural network, which is composed of multiple sparse coding modules with similar structures. Each sparse coding module must contain a sparse coding layer. Except for the last module, the remaining modules must contain a sliding window cropping layer and a reconstruction arrangement layer. The last module must contain a zero crossing detection layer. In addition, each block may also contain a pooling layer and an activation layer. The function of the sparse coding module is described below, taking an image signal as an example: the signal is sent to the sliding window sampling layer, such as Figure 2 As shown, the window slides along an S-shaped trajectory from the upper left to the lower right, and samples the signal at the same time, sampling K blocks from image I (the output of the previous layer) to form a sample set I={I1, I2, …, I K}, the distance between two samples is the stridelength. After the sliding window sampling layer, the sample set can be pooled and reduced in dimension using the pooling layer. Next, the sample set is sent to the sparse coding layer, and sparse coding is performed through the dictionary D to obtain the sparse coefficients {α1, α2, …, α K}, the subscript corresponds to the block number. The sparse coefficient of the block is a vector, α i = [α i1 , α i2 , … , α iM ] indicates that the second subscript corresponds to the dictionary vector, and M is the total number of vectors. If you want to introduce more nonlinear features to the sparse coefficients, you can use the activation layer to perform nonlinear transformation on the sparse system. Finally, the sparse coefficients are sent to the reconstruction and permutation layer, such as Figure 3 As shown in the figure, the images are rearranged into M matrices, which correspond to the dictionary vectors one by one and serve as the input of the next layer. Each matrix contains K values, which correspond to the blocks one by one, so that the contextual topological relationship of the original image is preserved. After reconstructing the permutation layer, if further dimensional compression is desired, a pooling layer can be used.

[0033] After the signal passes through a sparse coding module, the number of signal channels will increase, and the size of the signal will decrease. The reduction factor is positively correlated with the block edge length and step size. After passing through multiple sparse coding modules, when the size of the image is reduced to a suitable size, the image set is sent to the last sparse coding module, and the sparse coefficients are binarized through the zero-crossing detection layer to obtain a pulse sequence.

[0034] In the pulse coding process described above, the dictionary used in pulse coding needs to be pre-trained. The dictionary learning process is described below. This application is a neural network, and its training parameters are the dictionary used in the sparse coding process. Learning can be done through two modes: layer-by-layer pre-training or online learning. The online learning mode is first described below.

[0035] Online learning means updating the parameters after each sample is received, and learning is performed while the neural network completes pulse coding. If online learning is used, it is only necessary to update the dictionary of the layer using a conventional small batch dictionary learning algorithm after completing sparse coding.

[0036] This algorithm is unsupervised learning and does not require back propagation, so layer-by-layer pre-training can be used during training. This method cannot complete pulse coding during pre-training, but it requires less computation than online learning. The approach is to train the dictionary layer by layer starting from the first layer. When training the nth layer sparse coding dictionary, the dictionaries from 1 to n-1 layers are fixed, and the sample image is input into the first layer. The output image of the n-1 layer is obtained as the sample set, and the dictionary of the nth layer is updated by the dictionary learning method, and iterated until convergence. In this way, the training of all dictionaries is completed layer by layer.

[0037] When building a neural coding framework, the number of layers of the deep sparse coding network, the side length of the sliding window sampling, the step size, whether to add a pooling layer, the pooling layer parameters, the sparse coding dictionary size, etc. can all be freely set.

[0038] The encoding method of the present application is described below using a two-dimensional signal of a 3-channel 32*32 pixel image as an example.

[0039] 1. Sliding window sampling layer 1: Starting from the upper left corner, sliding window sampling is performed with a side length of 6 and a step length of 1 to obtain 3*27*27 6*6 blocks.

[0040] 2. Pooling layer 1: Perform 2*2 pooling on the blocks to obtain 3*27*27 3*3 blocks.

[0041] 3. Sparse coding layer 1: The first sparse coding, the number of dictionary vectors = 12, obtains 3*27*27 12-dimensional vectors.

[0042] 4. Reconstruction permutation layer 1: Rearrange the sparse coefficients to obtain 3*12 27*27 images.

[0043] 5. Sliding window sampling layer 2: Sliding window sampling is performed with a side length of 6 and a step size of 1 to obtain 3*12*22*22 6*6 blocks.

[0044] 6. Pooling layer 2.1: Perform 2*2 pooling on the blocks to obtain 3*12*22*22 3*3 blocks.

[0045] 7. Sparse coding layer 2: The number of dictionary vectors = 12, resulting in 3*22*22 12-dimensional vectors.

[0046] 8. Reconstruction and permutation layer 2: The sparse coefficients are rearranged to obtain 3*12*12 22*22 images.

[0047] 9. Pooling layer 2.2: Perform 2*2 pooling on the image set to obtain 3*12*12 11*11 images.

[0048] 10. Sparse coding layer 3: Number of dictionary vectors = 150, resulting in 3*12*12 150-dimensional sparse vectors

[0049] 11. Activation layer 3: The sparse vectors are fed into the activation function sequentially, and the dimension of the sparse vectors remains unchanged.

[0050] 12. Zero-crossing detection layer: Detect the zero-crossing of the sparse vector and obtain a 3*12*12 pulse sequence with 150 time steps.

[0051] The encoding method of the present application is described below using a two-dimensional signal of a 3-channel 225*225 pixel image as an example.

[0052] 1. Sliding window sampling layer 1: Sliding window sampling with side length 6 and step length 3. Number of signals 3*74*74, size 6*6.

[0053] 2. Pooling layer 1: 2*2 pooling is performed on the blocks. The number of signals is 3*74*74, and the size is 3*3.

[0054] 3. Sparse coding layer 1: Sparse coding is performed with a dictionary of 12 vectors. The number of signals is 3*74*74, and the size is 12.

[0055] 4. Reconstruction and permutation layer 1: Reconstruction and permutation, the number of signals is 3*12 and the size is 74*74.

[0056] 5. Sliding window sampling layer 2: pad the edge of the 74*74 image with zeros for 1 row and 1 column to obtain a 75*75 image. Sliding window sampling is performed with a side length of 6 and a step size of 3. The number of signals is 3*12*24*24, and the size is 6*6.

[0057] 6. Pooling layer 2: Perform 2*2 pooling on the blocks. Number of signals: 3*12*24*24, size: 3*3.

[0058] 7. Sparse coding layer 2: Sparse coding is performed with a dictionary with 12 vectors, 3*12*24*24 signals, and size 12.

[0059] 8. Reconstruction and permutation layer 2: Reconstruction and permutation, the number of signals is 3*12*12, and the size is 24*24.

[0060] 9. Sliding window sampling layer 3: side length 6, step length 3. Number of signals 3*12*12*7*7, size 6*6

[0061] 10. Pooling layer 3: Perform 2*2 pooling on the blocks. Number of signals: 3*12*12*7*7, size: 3*3.

[0062] 11. Sparse coding layer 3: Sparse coding is performed with a dictionary with 12 vectors, the number of signals is 3*12*12*7*7, and the size is 12.

[0063] 12. Reconstruction and permutation layer 3: Reconstruction and permutation, signal quantity 3*12*12*12, size 7*7.

[0064] 13. Sparse coding layer 4: sparse coding is performed with a dictionary of size 75, the number of signals is 3*12*12*12, and the size is 75.

[0065] 14. Activation layer 4: The sparse coefficients are fed into the activation function, and the dimension remains unchanged.

[0066] 15. Zero-crossing detection layer: Detect the zero-crossing of sparse coefficients and obtain a 3*12*12*12 pulse sequence with 75 time steps.

[0067] In order to demonstrate the generalization ability of one-dimensional data, the encoding method of the present application is explained below using an EEG signal with 19 channels and a length of 512 as an example.

[0068] 1. Sliding window sampling layer 1: Sliding window sampling is performed with a length of 16 and a step size of 4. The number of signals is 19*124 and the size is 16.

[0069] 2. Pooling layer 1: The pooling layer kernel size is 2. The number of signals is 19*124, and the size is 8.

[0070] 3. Sparse coding layer 1: Sparse coding is performed with a dictionary of 16 vectors. The number of signals is 19*124, and the size is 16.

[0071] 4. Reconstruction and permutation layer 1: Reconstruction and permutation, number of signals 19*16, size 124.

[0072] 5. Sparse coding layer 2: Sparse coding is performed with a dictionary of 144 vectors. The number of signals is 19*16, and the size is 144.

[0073] 6. Activation layer 2: The sparse coefficients are fed into the activation function without changing the dimension.

[0074] 7. Zero-crossing detection layer: Perform zero-crossing detection on the sparse coefficients to obtain 19*16 pulse sequences with a length of 144.

[0075] In the first example, a picture with 1024 three-channel pixels is converted into 432 pulse trains with a length of 150. In the second example, 50625 three-channel pixels are converted into 5184 pulse trains with a time step of 75. In the third example, an EEG signal consisting of 9728 floating-point numbers in 19 channels is converted into 304 pulse trains with a time step of 144. Different information compression and information extraction effects can be achieved by setting different parameters. The activation function can be set according to needs, such as ReLu, absolute value, sigmoid, softplus, etc., and its activation threshold can be changed by adding a bias to the above function.

Claims

1. A deep sparse coding method for a spiking neural network, comprising: N sparse coding layers; For the first N-1 sparse coding layers, each sparse coding layer corresponds to a sliding window cropping layer and a reconstruction permutation layer; The sliding window cropping layer is located before the sparse coding layer, and the reconstruction arrangement layer is located after the sparse coding layer; A sparse coding layer, a corresponding sliding window cropping layer, and a reconstruction arrangement layer together constitute a sparse coding module; For each sparse coding module, each input information is firstly sampled by the sliding window cropping layer to obtain a sampled signal, then the sparse coding layer performs sparse coding on the sampled signal, and finally the reconstruction and arrangement layer rearranges the sparse coding result; For each sparse coding module, when performing sliding window sampling, for each input signal, starting from a starting point, sliding window sampling is performed according to a predetermined route, a predetermined step length, and a predetermined size to form a first number of block signals; the first number of block signals constitute a sample set; exist When performing sparse coding, a dictionary is used to perform sparse coding on each block signal in a sample set, each block signal corresponds to a block sparse coefficient, and a first number of block sparse coefficients are obtained; wherein the dictionary has a second number of vectors; each block sparse coefficient corresponds to a second number of dimensional vectors; when performing rearrangement, the first number of block sparse coefficients are rearranged into a second number of matrices, each matrix includes a first number of values, and the first number of values ​​respectively correspond to the corresponding components of the first number of block sparse coefficients; the second number of matrices is the second number of output signals generated by the input signal; Each sparse coding module is executed independently of each other, and the output signal of the previous sparse coding module is used as the input signal of the next sparse coding module; The Nth sparse coding layer performs sparse coding again on the output signal of the N-1th sparse coding module, and binarizes the sparse coefficients to obtain a pulse sequence; Wherein, N is a natural number greater than 1; The input signal of the first sparse coding module is a two-dimensional signal, which is a picture. When the picture is cropped with a sliding window, the picture is cropped according to an S-shaped trajectory starting from the upper left and ending at the lower right.

2. The deep sparse coding method for a spiking neural network according to claim 1, characterized in that: The dictionaries of the sparse coding modules are independent of each other, the step sizes of the sparse coding modules are independent of each other, and the predetermined sizes of the sparse coding modules are also independent of each other.

3. The deep sparse coding method for a spiking neural network according to claim 1, characterized in that: At least one of the multiple sparse coding modules also includes a pooling layer, and the pooling layer is arranged after the reconstruction arrangement layer or before the sliding window cropping layer of the module.

4. The deep sparse coding method for a spiking neural network according to claim 1, characterized in that: The dictionary of each sparse coding layer is updated by online learning; after sparse coding, each sparse coding layer uses a small batch dictionary learning algorithm to update the dictionary of the sparse coding layer.

5. The deep sparse coding method for a spiking neural network according to claim 1, characterized in that: The dictionary of each sparse coding layer is updated by pre-training layer by layer; starting from the first sparse coding layer, the dictionaries corresponding to N sparse coding layers are trained layer by layer.

Images