An image classification method based on a multi-stage complementary pulse neural network model
By introducing multi-level activation design and complementary potential energy into the spiking neural network, the gradient vanishing problem was solved, the accuracy and precision of image classification were improved, the low energy consumption characteristics were maintained, and the expressive power was expanded.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UNIV
- Filing Date
- 2024-09-06
- Publication Date
- 2026-05-29
AI Technical Summary
Existing spiking neural networks suffer from the vanishing gradient problem in image classification tasks, resulting in lower accuracy than traditional deep neural networks and higher energy consumption.
By employing a multi-level complementary spiking neural network model, and redesigning the LIF neuron structure, introducing multi-level activation design and complementary potential energy, the gradient vanishing problem is improved, thereby enhancing the accuracy of image classification.
While maintaining low energy consumption, the performance of spiking neural networks was improved, the accuracy and precision of image classification were enhanced, information loss was reduced, and expressive power was expanded.
Smart Images
Figure CN119206323B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence, and specifically to an image classification method based on a multi-level complementary spiking neural network model. Background Technology
[0002] Since its release, ChatGPT has demonstrated unprecedented capabilities, sparking an AI boom. Multimodal large-scale models are gradually becoming the future trend. However, building and maintaining these large models is a resource-intensive task. Deep neural networks consume enormous amounts of energy, so adopting more energy-efficient hardware, algorithmic innovation, and promoting the use of green energy have gradually become indispensable parts of model research and application.
[0003] Recently, researchers have been inspired by the mechanism by which biological neurons use impulses for computation and communication, and have built a novel neural network based on this. Although traditional artificial neural networks (ANNs) also draw on the principles of biological nervous systems, they differ fundamentally from the human brain in their structural composition, neuronal computation mechanisms, and learning strategies. The human brain can perform complex cognitive tasks, such as multi-object recognition, logical reasoning, and motor control, with extremely low energy consumption (approximately 2 watts), while a conventional computer requires up to 250 watts of energy to complete the same complex object recognition task. Spike Neural Networks (SNNs) simulate the operating mechanism of biological neural networks, using impulse signal transmission instead of continuous value activation, which more closely resembles the natural working mode of biological nervous systems. This significantly optimizes the utilization of computational resources, demonstrating superior generalization performance and temporal data processing capabilities.
[0004] The core feature of SNNs, spike communication, not only reflects the dynamic spatiotemporal relationships between neurons but also promotes efficient utilization of computational resources, enhancing the model's performance and efficiency in complex temporal tasks. This spike-based activation mechanism, combined with temporal encoding and asynchronous processing characteristics, constitutes the unique energy efficiency and adaptability advantages of SNNs, making them particularly suitable for tasks involving multiple types of information (multimodal). SNNs can effectively integrate input signals from different modes, improving the accuracy and response speed of task execution, effectively alleviating the problems faced by ANNs in processing multimodal data, such as high computational costs, large storage requirements, and insufficient temporal information processing, further expanding the application boundaries and performance potential of neural network technology.
[0005] Despite the numerous advantages of spiking neural networks (SNNs), the discretized, binary pulse sequences that act as both inputs and outputs of spiking neurons lead to a severe vanishing gradient problem when directly training them. This significantly limits the performance of directly trained SNNs and hinders further research. While binary spikes enable faster information transmission, they represent a more limited amount of information compared to floating-point numbers. This limitation can cause different values in an ANN to exhibit the same pulse behavior in a SNN, further exacerbating the vanishing gradient problem. Furthermore, since most SNNs employ the Leaky Integrate-and-Fire (LIF) model, this structure is ill-suited for handling temporal gradients at larger time steps, resulting in excessively rapid gradient descent.
[0006] The information gaps caused by the aforementioned issues have, to some extent, interfered with the accuracy of SNN networks in image classification tasks. Currently, the accuracy of SNN networks in image classification tasks is generally lower than that of ANN networks with the same network structure. Summary of the Invention
[0007] To address the problems and shortcomings of existing technologies, this invention proposes an image classification method based on a multi-level complementary spiking neural network model. This method redesigns LIF neurons and adopts a novel multi-level complementary neuron structure, which improves the performance of the SNN network and enhances the accuracy of image classification while ensuring the low energy consumption of the SNN.
[0008] The objective of this invention is achieved through the following technical solution: an image classification method based on a multi-level complementary spiking neural network model, comprising:
[0009] Obtain the image classification dataset;
[0010] Constructing a multi-level complementary spiking neural network model includes: using the traditional LIF neuron structure to construct an SNN network model; reconstructing the LIF neuron structure in the SNN network model and introducing a multi-level activation design, wherein the multi-level activation design uses multiple small neuron structures to construct a new multi-level neuron structure; after introducing the multi-level activation design, complementary potential energy is then introduced to realize the complementary activation design, thus obtaining the multi-level complementary neuron structure.
[0011] The multi-level complementary spiking neural network model is trained using an image classification dataset, and the trained multi-level complementary spiking neural network model is used for image classification.
[0012] Furthermore, in the multi-level activation design, a new multi-level neuron structure is constructed using multiple small neuron structures. The membrane voltage accumulation process under the new structure replaces the membrane voltage accumulation process under the original LIF model. The specific formula is as follows:
[0013]
[0014] Where τ is the membrane time constant, u l [t] represents the membrane voltage of the multi-level neuron structure in the l-th layer at time t. This represents the membrane voltage of the i-th small neuron in layer l at time t-1. This represents the current in the i-th small neuron structure of layer l at time t.
[0015] Furthermore, after introducing the multi-level activation design, complementary potential energy is then introduced to realize the complementary activation design, including:
[0016] The complementary activation design introduces an additional "complementary potential" condition into the LIF structure to mitigate the membrane voltage decay after pulse activation. The membrane voltage decay process under the new structure replaces the decay process under the original LIF model, as shown in the following formula:
[0017]
[0018] in V represents the membrane voltage of the i-th small neuron in layer l at time t. th The membrane voltage threshold representing the neurons in this layer, s l [t] represents whether a pulse is generated at time t in layer l; This represents the magnitude of the "complementary potential" of the i-th small neuron in layer l at time t, where α represents the learning rate. In the formula, when a neuron in layer l pulses at time t, its corresponding membrane voltage u is... l [t] will decay, through additional supplementation To slow the rate of decline.
[0019] Furthermore, the magnitude of the complementary potential energy is dynamically adjusted according to the pulse activation status, as shown in the following formula:
[0020]
[0021] in This represents the magnitude of the "complementary potential" of the i-th small neuron in layer l at time t, where β is a hyperparameter used to control the growth rate of the complementary potential.
[0022] Furthermore, it also includes: adjusting the backpropagation function for the current multi-level complementary neuron structure, with the specific formula as follows:
[0023]
[0024] Where L represents the loss value, The gradient of the weights in the l-th layer. Let T be the gradient of the bias of the l-th layer, T represent the time step, and K represent the number of small neurons in the multi-level activation.
[0025] The present invention also provides an image classification device based on a multi-level complementary spiking neural network model, comprising:
[0026] The data acquisition module is used to acquire image classification datasets;
[0027] The model building module is used to build a multi-level complementary spiking neural network model, including: building an SNN network model using the traditional LIF neuron structure; reconstructing the LIF neuron structure in the SNN network model and introducing a multi-level activation design, wherein the multi-level activation design uses multiple small neuron structures to build a new multi-level neuron structure; after introducing the multi-level activation design, complementary potential energy is then introduced to realize the complementary activation design, thus obtaining the multi-level complementary neuron structure.
[0028] The model training module is used to train the multi-level complementary spiking neural network model using an image classification dataset. The trained multi-level complementary spiking neural network model is then used for image classification.
[0029] The present invention also provides an electronic device, including a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-described image classification method based on a multi-level complementary spiking neural network model.
[0030] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described image classification method based on a multi-level complementary spiking neural network model.
[0031] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described image classification method based on a multi-level complementary spiking neural network model.
[0032] The beneficial effects of this invention are as follows:
[0033] Modifying existing SNN networks can yield new SNN networks with better performance and higher accuracy while maintaining the same network backbone. Specifically, the core structure of the original SNN network is preserved, and only the LIF neuron model is adjusted. By introducing a multi-level activation design, the binarization expressive power of the original SNN is expanded, improving the gradient vanishing problem. Furthermore, the multi-level activation design also expands the width of neurons by approximately the inverse, increasing the information representation capability of spatial gradients and further reducing the occurrence of gradient vanishing. By introducing a complementary activation design, additional complementary potential energy is compensated on top of the original membrane voltage, improving the decay rate of temporal gradients. This allows the SNN network to maintain temporal gradients even at longer time steps, thereby capturing more information during training, reducing information loss, and ultimately improving the overall performance of the SNN network. For example, in an image recognition scenario, this method can be used to modify only the LIF structure of the original SNN network. The improved network structure can perform image recognition more accurately while maintaining the low-energy consumption characteristics of SNNs. In summary, this method can improve SNN networks, making the computational complexity roughly the same before and after the improvement, but with better performance and accuracy. Attached Figure Description
[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0035] Figure 1 This is a flowchart illustrating the overall workflow of the present invention;
[0036] Figure 2 This is the neuronal structure after the introduction of a multi-level activation mechanism in this invention;
[0037] Figure 3 This is the neuronal structure after the introduction of the pulse activation mechanism in this invention;
[0038] Figure 4 These are schematic diagrams of the SNN network structure before and after the improvement of the present invention; wherein, (a) is a schematic diagram of the SNN network structure before the improvement, and (b) is a schematic diagram of the SNN network structure after the improvement.
[0039] Figure 5 This is a table showing the accuracy of various image datasets before and after the improvements made in this invention;
[0040] Figure 6 This is a schematic diagram of an electronic device structure provided in an embodiment of the present invention. Detailed Implementation
[0041] The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments. The objectives and effects of the present invention will become clearer from this description. The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Unless otherwise specified, the features of the following embodiments and implementation methods can be combined with each other. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.
[0042] Depend on Figure 1 As shown, this invention provides an image classification method based on a multi-level complementary spiking neural network model, the specific steps of which include:
[0043] Step 1: Construct an SNN network model using the traditional LIF neuron structure;
[0044] In one embodiment, based on the LIF model, the SNN network model to be constructed is first determined. Here, Spike-ResNet (an SNN network) is taken as an example, and its structure is as follows: Figure 4 As shown in (a) above. This step is mainly used to determine the backbone of the SNN network, and subsequent steps will not adjust the backbone structure.
[0045] Step 2: Reconstruct the LIF neuron structure in the SNN network model and introduce a multi-level activation design. The multi-level activation design uses multiple small neuron structures to construct a new multi-level neuron structure.
[0046] In one embodiment, the LIF neuron structure of the SNN network model is reconstructed, introducing a multi-level activation design. For example... Figure 2 As shown, the optimized neuron is formed by the accumulation and combination of multiple small neurons. Compared with the traditional single-neuron structure's binary output, this structure has stronger expressive power, reduces information loss, and improves gradient vanishing. The corresponding formula is as follows:
[0047]
[0048] Where τ is the membrane time constant, u l [t] represents the membrane voltage of the multi-level neuron structure in the l-th layer at time t. This represents the membrane voltage of the i-th small neuron in layer l at time t-1. This represents the current in the i-th small neuron structure of layer l at time t.
[0049] Step 3: After introducing the multi-level activation design, complementary potential energy is then introduced to realize the complementary activation design, thus obtaining the multi-level complementary neuron structure.
[0050] In one embodiment, a complementary activation setting is further introduced based on step 2. For example... Figure 3 As shown, each improved small neuron structure receives additional complementary potential energy on top of the original membrane voltage. In the traditional LIF model, the membrane voltage decays faster with increasing time step, and the time gradient gradually disappears. The complementary potential energy effectively improves this problem of vanishing time gradients. The corresponding formula is as follows:
[0051]
[0052] in V represents the membrane voltage of the i-th small neuron in layer l at time t. th The membrane voltage threshold representing the neurons in this layer, s l [t] represents whether a pulse is generated at time t in layer l. This represents the magnitude of the "complementary potential" of the i-th small neuron in layer l at time t, where α represents the learning rate. In the formula, when a neuron in layer l pulses at time t, its corresponding membrane voltage u is... l [t] will decay, through additional supplementation To slow the rate of decline.
[0053] The complementary activation setting introduces complementary potential energy. During the propagation of the SNN network, this complementary potential energy needs to be dynamically adjusted. After a pulse signal is generated, the complementary potential energy needs to be replenished; if no pulse signal is generated, the complementary potential energy should be weakened accordingly. This ensures the overall propagation trend of the network while also reducing the membrane voltage decay rate and mitigating the gradient vanishing effect over large time steps. The specific formula is as follows:
[0054]
[0055] in, This represents the magnitude of the "complementary potential" of the i-th small neuron in layer l at time t, where β is a hyperparameter used to control the growth rate of the complementary potential. l [t] = 1 indicates that a pulse signal has been generated. That is, the complementary potential energy of the i-th small neuron in the l-th layer at the current time step will be increased to a certain extent with the hyperparameter β as the learning rate. If there is no impulse signal, then It will mimic the membrane voltage decay process to a certain extent and reduce it.
[0056] Step 4: Train the multi-level complementary spiking neural network model using the image classification dataset. The trained multi-level complementary spiking neural network model is then used for image classification.
[0057] Specifically, during the training of the spiking neural network, because the structure of the original LIF model has been modified, the entire backpropagation process also needs to be updated. The specific formula is as follows:
[0058]
[0059] in The gradient of the weights in the l-th layer. The gradient of the bias in layer l. T represents the time step, and K represents the number of small neurons in the multi-level activation, which is usually set to 3.
[0060] After the above steps, we have basically constructed the backbone of a multi-level complementary spiking neural network, such as... Figure 4 As shown, where s l [t] represents the input of layer l at time t, O l [t] represents the output of layer l at time t, Conv is a convolutional layer, BN is a batch-normal layer, LIF is a LIF spiking neuron structure, and our is a multi-level complementary spiking neuron structure, which is the improved neuron structure of this invention. Figure 4 This embodiment uses Spike-ResNet (an SNN network structure) as an example, wherein... Figure 4 (a) in the image represents Spike-ResNet, which uses the LIF neuron model. After the above steps, we obtain the improved Spike-ResNet as shown below. Figure 4 As shown in (b) of the diagram, the our module represents the improved multi-level complementary spiking neuron structure in this embodiment. It can be seen that the overall network skeleton structure remains unchanged before and after the improvement; therefore, the order of magnitude of computation remains essentially the same. However, through our multi-level complementary mechanism design, we can effectively improve problems such as gradient vanishing in the LIF model. Next, we only need to train the new network structure, and the training process is no different from that before the improvement.
[0061] A comparative experiment was conducted on the Spike-ResNet network before and after the improvements to test the effectiveness of the multi-level pulse complementarity structure. To ensure training fairness, the same random seed, architecture, loss function, weight initialization, and all hyperparameters were used for both networks. Furthermore, to ensure the accuracy and generality of the tests, multiple image datasets, including CIFAR-10, CIFAR-100, ImageNet, and DVS-Gesture, were selected for testing. The test results are as follows: Figure 5 As shown, the horizontal axis represents different datasets, and the vertical axis represents the accuracy achieved by the LIF structure (left) and the multi-level pulse complementarity structure (right) on the current dataset. It can be seen that the improved network structure has a certain performance improvement on different image datasets.
[0062] In summary, after all the steps are completed, a multi-level complementary spiking neural network structure with better accuracy and similar energy consumption compared to the original spiking neural network can be obtained.
[0063] The present invention also provides an image classification device based on a multi-level complementary spiking neural network model, comprising:
[0064] The data acquisition module is used to acquire image classification datasets;
[0065] The model building module is used to build a multi-level complementary spiking neural network model, including: building an SNN network model using the traditional LIF neuron structure; reconstructing the LIF neuron structure in the SNN network model and introducing a multi-level activation design, wherein the multi-level activation design uses multiple small neuron structures to build a new multi-level neuron structure; after introducing the multi-level activation design, complementary potential energy is then introduced to realize the complementary activation design, thus obtaining the multi-level complementary neuron structure.
[0066] The model training module is used to train the multi-level complementary spiking neural network model using an image classification dataset. The trained multi-level complementary spiking neural network model is then used for image classification.
[0067] It should be noted that the device embodiment shown in this embodiment matches the content of the above method embodiment, and the content of the above method embodiment can be referred to, and will not be repeated here.
[0068] Figure 6 This is a schematic diagram of an electronic device structure provided in an embodiment of the present invention. Please refer to... Figure 6 The electronic device provided in this embodiment includes a memory and a processor, wherein the memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions. When the program instructions are loaded and executed by the processor, the method of the present invention is implemented.
[0069] It should be noted that, in addition to Figure 6 In addition to the memory and processor shown, electronic devices may include other hardware depending on their actual functions, which will not be elaborated further.
[0070] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.
[0071] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.
[0072] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.
[0073] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0074] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0075] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0076] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0077] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.
Claims
1. An image classification method based on a multi-level complementary spiking neural network model, characterized in that, include: Obtain the image classification dataset; Constructing a multi-level complementary spiking neural network model includes: using the traditional LIF neuron structure to construct an SNN network model; reconstructing the LIF neuron structure in the SNN network model and introducing a multi-level activation design, wherein the multi-level activation design uses multiple small neuron structures to construct a new multi-level neuron structure; after introducing the multi-level activation design, complementary potential energy is then introduced to realize the complementary activation design, thus obtaining the multi-level complementary neuron structure. The multi-level complementary spiking neural network model is trained using an image classification dataset, and the trained multi-level complementary spiking neural network model is used for image classification. The multi-level activation design uses multiple small neuron structures to construct a new multi-level neuron structure. The membrane voltage accumulation process under the new structure replaces the membrane voltage accumulation process under the original LIF model. The specific formula is as follows: in The membrane time constant is express At this moment, the first Membrane voltage of a multi-level neuronal structure in a layer, represent time Layer, number Membrane voltage of a small neuron structure represent time Layer, number The current in a small neuron structure; After introducing a multi-level activation design, complementary potential energy is then introduced to realize a complementary activation design, including: The complementary activation design introduces an additional "complementary potential" condition into the LIF structure to mitigate the membrane voltage decay after pulse activation. The membrane voltage decay process under the new structure replaces the decay process under the original LIF model, as shown in the following formula: in express time Layer, membrane voltage of the i-th small neuron, This represents the membrane voltage threshold of neurons in that layer. represent layer Whether a pulse is generated at any given moment; It indicates time The magnitude of the "complementary potential" of the i-th small neuron in the layer. Represents the learning rate, in the formula when layer When a neuron generates a pulse at a given time, its corresponding membrane voltage A decline will occur, through additional supplementation To slow the rate of decline; The magnitude of the complementary potential energy is dynamically adjusted according to the pulse activation status, and the specific formula is as follows: in express time The magnitude of the "complementary potential" of the i-th small neuron in the layer. This is a hyperparameter used to control the growth rate of the complementary potential.
2. The image classification method based on a multi-level complementary spiking neural network model according to claim 1, characterized in that, Also includes: The backpropagation function is adjusted for the current multi-level complementary neuron structure, and the specific formula is as follows: in Represents the loss value. No. The gradient of the layer weights, For the first The gradient of the layer bias, where T represents the time step and K represents the number of small neurons in the multi-level activation.
3. An image classification device based on a multi-level complementary spiking neural network model, characterized in that, To implement the method according to any one of claims 1-2, comprising: The data acquisition module is used to acquire image classification datasets; The model building module is used to build a multi-level complementary spiking neural network model, including: building an SNN network model using the traditional LIF neuron structure; reconstructing the LIF neuron structure in the SNN network model and introducing a multi-level activation design, wherein the multi-level activation design uses multiple small neuron structures to build a new multi-level neuron structure; after introducing the multi-level activation design, complementary potential energy is then introduced to realize the complementary activation design, thus obtaining the multi-level complementary neuron structure. The model training module is used to train the multi-level complementary spiking neural network model using an image classification dataset. The trained multi-level complementary spiking neural network model is then used for image classification.
4. An electronic device, comprising a memory and a processor, characterized in that, The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the image classification method based on a multi-level complementary spiking neural network model as described in any one of claims 1-2.
5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements an image classification method based on a multi-level complementary spiking neural network model as described in any one of claims 1-2.
6. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the image classification method based on a multi-level complementary spiking neural network model as described in any one of claims 1-2.