An Adaptive Spiking Neural Network Structure

Abstract

The present invention discloses an adaptive spiking neural network structure, which includes an input layer, an excitation layer, an inhibition layer and an output layer. The excitation layer is a three-layer network structure, the first layer of which forms a fully connected structure with the input layer, and the third layer of which forms a fully connected structure with the output layer; the inhibition layer is a three-layer network structure, and its connection to the excitation layer is a one-to-one connection, while the connection from the inhibition layer to the excitation layer is a one-to-many connection; specifically, the input end of the inhibition layer is connected to each layer of spiking neurons in the excitation layer in a one-to-one manner, and the output end is fully connected to other neurons in the corresponding excitation layer to form a circular connection. The spiking neural network constructed by the present invention has self-organization and self-adaptive capabilities, and can adaptively adjust the network structure according to the input data.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence, and particularly relates to an adaptive spiking neural network structure. Background Art

[0002] With the gradual understanding of the brain structure and information processing methods, people have established a simplified and abstract mathematical model - artificial neural network based on biological neural networks. The artificial neural network mimics the information processing method of the biological neural network and has achieved great success in the fields of pattern recognition, automatic control, machine learning, etc.

[0003] The spiking neural network is currently the most biologically interpretable artificial neural network, and has stronger biogenicity compared with traditional feedforward neural networks and deep learning neural networks. The research on spiking neural networks is of great significance to brain-like intelligence. The existing spiking neural network structures can be mainly divided into feedforward type, recurrent type, and hybrid network structures. Among them, the feedforward type network adopts a fully connected method. Although this network can achieve good learning effects, the training time is long and the process is complex, and it is basically impossible to achieve fast online learning.

[0004] Therefore, designing an adaptive spiking neural network structure is beneficial to promoting the research and application of spiking neural networks, and has important significance for the artificial intelligence industry.

[0005] Regarding the spiking neural network structure, some scholars have conducted research. The literature "Unsupervised learning of digit recognition using spike-timing-dependent plasticity" uses a fully connected feedforward spiking neural network structure to achieve the recognition of MNIST handwritten digits, but the training time is relatively long and it is not applicable to large-scale networks. "Supervised learning in spiking neural networks with ReSuMe: sequence learning, classification, and spiking shifting" uses a spiking neural network structure with a liquid state machine structure and completes its training, but this structure requires more parameters to be trained. Summary of the Invention

[0006] The purpose of the present invention is to provide a spiking neural network structure that can achieve fast learning in view of the problems existing in the above-mentioned prior art.

[0007] The technical solution for achieving the purpose of the present invention is: an adaptive spiking neural network structure, which includes an input layer, an excitation layer, an inhibition layer, and an output layer;

[0008] The input layer is used to provide pulse signals for the entire spiking neural network;

[0009] The excitation layer is used to reduce the network depth while ensuring memory information;

[0010] The inhibition layer is used to inhibit the excessive behavior of the excitation layer;

[0011] The output layer is used to output a pulse sequence.

[0012] Furthermore, the excitation layer has a three-layer network structure. The first layer of the excitation layer forms a fully connected structure with the input layer, and the third layer of the excitation layer forms a fully connected structure with the output layer;

[0013] The inhibition layer has a three-layer network structure. Its connection to the excitation layer is a one-to-one connection, while the connection from the inhibition layer to the excitation layer is a one-to-many connection; specifically, the input end of the inhibition layer is connected to each pulse neuron of the excitation layer in a one-to-one manner, and the output end is fully connected to other neurons of the corresponding excitation layer to form a loop connection; when the pulse neuron of the inhibition layer fires a pulse, it will inhibit the other pulse neurons of the corresponding excitation layer from firing pulses.

[0014] Furthermore, in the initial state of this network structure, there is no connection between the three layers of the excitation layer. During simulation, through the action of the input signal and the connection algorithm, partial connections will be formed, and the number of these partial connections ensures that the total number of connections exceeds half of the number of full connections.

[0015] Furthermore, the determination method of the partial connections is as follows:

[0016] According to the pulse firing situation of the pulse neurons, determine the connection structure between each pulse neuron of the excitation layer and the pulse neurons of the next layer, specifically including:

[0017] Map the time when the pulse neuron fires a pulse to a weight value:

[0018]

[0019] In the formula, λ is the mapped weight value, λ max is the maximum weight value, t is the time when the pulse is fired, and T max is the longest time when the pulse is fired;

[0020] The connection between the current pulse neuron of the excitation layer and the next pulse neuron is expressed as:

[0021]

[0022] Among them,

[0023]

[0024] Wherein, p ki represents the probability that the k-th neuron in the current layer is connected to the i-th neuron in the next layer, s represents the total number of pulses fired by the neurons in the current layer, n is the number of neurons in the current layer, α is a regulation parameter, and p th is the connection probability threshold, ∑λ is the total weight corresponding to the firing time, and λ th is the threshold of the time weight.

[0025] Compared with the prior art, the present invention has the following remarkable advantages: 1) Optimize the structure of the spiking neural network, enabling it to adaptively change the structure according to the input data, accelerating the speed of subsequent training; 2) Introduce three excitation layers, while ensuring the memory information, reducing the network depth; 3) By introducing an inhibition layer, suppress the excessive behavior of the excitation layer and enhance the generalization ability of the spiking neural network; 4) Perform adaptive structure network connection between the excitation layers, apply full connection between the input layer and the excitation layer, and under the condition of ensuring all received data, reduce the network complexity and accelerate the network training speed.

[0026] The present invention will be further described in detail below with reference to the accompanying drawings. Description of the Drawings

[0027] Figure 1 It is the overall structure diagram of the spiking neural network in an embodiment.

[0028] Figure 2 It is the internal connection diagram of the excitation layer network in an embodiment.

[0029] Figure 3 It is the connection diagram between the excitation layer and the inhibition layer in an embodiment. Specific Embodiments

[0030] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0031] In one embodiment, in combination with Figure 1 , an adaptive spiking neural network structure is provided, which includes an input layer, an excitation layer, an inhibition layer and an output layer;

[0032] The input layer is used to provide pulse signals for the entire spiking neural network;

[0033] The excitation layer is used to reduce the network depth while ensuring the memory information;

[0034] The inhibition layer is used to suppress the excessive behavior of the excitation layer;

[0035] The output layer is used to output a pulse sequence.

[0036] Further, in one embodiment, in combination with Figure 2 and Figure 3 , the excitation layer is a three-layer network structure. The first layer of the excitation layer forms a fully connected structure with the input layer, and the third layer of the excitation layer forms a fully connected structure with the output layer;

[0037] The inhibition layer is a three-layer network structure. Its connection to the excitation layer is a one-to-one connection, while the connection from the inhibition layer to the excitation layer is a one-to-many connection; specifically, the input end of the inhibition layer is connected to each pulse neuron of the excitation layer in a one-to-one manner, and the output end is fully connected to other neurons of the corresponding excitation layer to form a loop connection; when the pulse neuron of the inhibition layer emits a pulse, it will inhibit other pulse neurons of the corresponding excitation layer from emitting pulses.

[0038] Based on the above description, the formation process of this neural network is as follows:

[0039] 1) Neural network structure initialization: Set the number of layers of the pulse neural network, the number of neurons in each layer, and the connection status of neurons between layers; 2) Adaptive inter-layer network structure: In the initial network, the excitation layer networks are not connected. According to the pulse emission situation of neurons in the previous layer, connections are generated in a layer-by-layer recursive manner; 3) Inhibition layer construction: It corresponds one-to-one with the excitation layer network structure and serves as the feedback of the excitation layer network to other neurons in this layer.

[0040] Further, in one embodiment, the number of pulse neurons in the input layer is the same as the number of input data pixel points.

[0041] Further, in one embodiment, the number of pulse neurons in the excitation layer, inhibition layer, and output layer is determined by the number of pulse neurons in the input layer and the set scaling factor between layers.

[0042] Here, by way of example, for an MNIST image with a picture pixel of 28*28, the number of neurons in the input layer is 28*28 = 784, the number of neurons in the excitation layer is 14*14*3 = 588, and the number of neurons in the output layer is 7*7 = 49.

[0043] Further, in one embodiment, when the network structure is in the initial state, there is no connection between the three layers of the excitation layer. During simulation, through the action of the input signal and the connection algorithm, partial connections will be adaptively formed in a layer-by-layer recursive manner, and the total number of these partial connections ensures that it exceeds half of the full connection number.

[0044] Further, in one embodiment, the determination method of the partial connections is as follows:

[0045] Determine the connection structure between the pulse neurons in each layer of the excitation layer and the pulse neurons in the subsequent layer according to the pulse firing situation of the pulse neurons, specifically including:

[0046] Map the time when the pulse neuron fires a pulse to a weight:

[0047]

[0048] In the formula, λ is the mapped weight, λ max is the maximum weight value, t is the time when the pulse is fired, and T max is the longest time when the pulse is fired;

[0049] The connection between the pulse neurons in the current layer and the pulse neurons in the next layer in the excitation layer is expressed as:

[0050]

[0051] Among them,

[0052]

[0053] In the formula, p ki represents the probability of connection between the kth neuron in the current layer and the ith neuron in the next layer, s represents the total number of pulses fired by the neurons in the current layer, n is the number of neurons in the current layer, α is a regulation parameter, and p th is the connection probability threshold, ∑λ is the total weight corresponding to the firing time, and λ th is the threshold of the time weight.

[0054] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art of this industry should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification only illustrates the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. An adaptive pulsed neural network structure, characterized in that, The structure includes an input layer, an excitation layer, an inhibition layer, and an output layer; The input layer is used to provide pulse signals for the entire spiking neural network; The excitation layer is used to reduce the network depth while ensuring memory information; The inhibition layer is used to inhibit the excessive behavior of the excitation layer; The output layer is used to output a pulse sequence; The excitation layer is a three-layer network structure. The first layer of the excitation layer forms a fully connected structure with the input layer, and the third layer of the excitation layer forms a fully connected structure with the output layer; The inhibition layer is a three-layer network structure. Its connection to the excitation layer is a one-to-one connection, while the connection from the inhibition layer to the excitation layer is a one-to-many connection; specifically, the input end of the inhibition layer is connected to each pulse neuron of the excitation layer in a one-to-one manner, and the output end is fully connected to other neurons of the corresponding excitation layer to form a circular connection; when the pulse neuron of the inhibition layer emits a pulse, it will inhibit the other pulse neurons of the corresponding excitation layer from emitting pulses; The number of pulse neurons in the input layer is the same as the number of input data pixel points; The number of pulse neurons in the excitation layer, inhibition layer, and output layer is determined by the number of pulse neurons in the input layer and the set scaling factor between layers; In the initial state of this network structure, there is no connection between the three layers of the excitation layer. During simulation, through the action of the input signal and the connection algorithm, partial connections will be adaptively formed in a layer-by-layer recursive manner, and the total number of these partial connections is guaranteed to exceed half of the full connection number.

2. The adaptive pulsed neural network structure according to claim 1, wherein The determination method of the partial connections is as follows: According to the pulse emission situation of the pulse neurons, determine the connection structure between each pulse neuron of the excitation layer and the pulse neurons of the next layer, specifically including: Map the time when the pulse neuron emits a pulse to a weight value: ； Wherein, is the weight of the mapping, is the maximum value of the weight, is the time of pulse emission, is the longest time of pulse emission; The connection between the current layer of pulse neurons in the excitation layer and the next layer of pulse neurons is expressed as: ； Wherein, ； In the formula, represents the probability that the k-th neuron in the current layer is connected to the i-th neuron in the next layer, s represents the total number of spikes emitted by the neurons in the current layer, n is the number of neurons in the current layer, is a regulation parameter, is the connection probability threshold, is the total weight corresponding to the firing time, is the threshold of the time weight.

Images