Construction method of reversible spiking neural network supporting parallel computing

Abstract

The invention belongs to the technical field of spiking neural networks. The invention provides a construction method of a reversible pulse neural network supporting parallel computing. According to the embodiment of the invention, the reversible calculation structure is introduced, the neuron state of an intermediate layer does not need to be explicitly stored in the forward propagation process of the network, and the reconstruction of the activation value can be realized through the reversible mapping relation in the backward propagation stage. In the design stage of the reversible structure, hardware mapping characteristics are fully considered, and forward calculation and reverse reconstruction processes of reversible mapping are simultaneously incorporated into a parallel execution framework of a configurable calculation array. Through collaborative design of a data path and control logic, forward propagation and reverse reversible reconstruction can be efficiently and parallelly operated in the same array, and inter-stage calculation idle time and data waiting time are remarkably shortened. On the premise of ensuring reversibility, streamlined and parallel execution of forward and reverse calculation is realized, and the throughput rate and the training speed of the system are integrally improved.

Description

Technical Field

[0001] This disclosure relates to the field of spiking neural network technology, and in particular to a method for constructing a reversible spiking neural network that supports parallel computing. Background Technology

[0002] Spiking neural networks (SNNs), as third-generation neural networks inspired by the brain, possess high biological interpretability and excellent robustness. Thanks to their event-driven and spiking characteristics, SNNs exhibit significant low-power advantages when deployed on edge devices. However, SNNs face considerable challenges during the training phase. Since information is transmitted and processed in the form of non-differentiable spiking sequences, existing training methods often employ surrogate gradient techniques to approximate differentiation, thereby achieving end-to-end supervised training. Although this method outperforms traditional artificial neural network to SNN conversion methods in terms of accuracy and time latency, deployment on edge devices remains difficult. This is mainly because the training process relies on backpropagation (BPTT) to process spatiotemporal information, resulting in high computational latency and huge memory overhead, making it difficult to meet the real-time and energy efficiency requirements of resource-constrained environments.

[0003] Reversible computation techniques were first proposed in ANNs (Application-Oriented Neural Networks) and initially applied to unsupervised learning tasks. The core idea is to construct reversible modules to reconstruct the intermediate activation values ​​required for forward propagation during the backpropagation phase, thereby significantly reducing memory overhead. Subsequently, this technique was introduced into the SNN (Small Neural Networks) field, alleviating to some extent the high memory consumption problem in BPTT (Backpropagation-to-Tribulation) processes. However, existing reversible structures typically employ a serial stacking approach, making it difficult to efficiently coordinate with the parallel computing mechanisms of hardware. This results in increased time latency during training and inference phases after introducing reversible mechanisms, limiting its application potential in efficient deployment scenarios.

[0004] The high memory consumption of spiking neural networks (SNNs) stems from several factors. Firstly, it increases with network depth; deeper networks require storing more parameters and intermediate activations. Secondly, SNN memory increases with the simulation time step T, as SNNs need to store T times the number of intermediate activations and membrane potentials. To address this memory consumption issue, researchers have proposed various schemes to reduce it. For example, OTTT, SLTT, and EfficientLIF-Net reduce memory requirements by decreasing state storage or sharing membrane potentials in the time dimension, but these methods typically sacrifice some accuracy or expressive power. Subsequently, reversible spiking neural networks (RevSNNs) were proposed, achieving backpropagation without explicitly storing intermediate states through reversible mapping, thus significantly reducing memory consumption. However, existing reversible SNN methods still suffer from the following problems: 1. Reducing the trade-off between memory usage and accuracy. Existing methods such as OTTT, SLTT, and EfficientLIF-Net typically sacrifice network accuracy when reducing training memory usage. This means that while pursuing memory optimization, the model's performance on complex tasks may degrade.

[0005] 2. Trade-off between reducing memory usage and computation speed: Although S-ResVSNN performs well in maintaining the accuracy of SNN, the serial dependency in its reversible blocks makes it difficult to parallelize the computation process, and the training and inference speeds are not significantly improved compared to conventional SNN.

[0006] 3. Trade-offs between software and hardware deployment. Existing methods often do not fully consider hardware implementation efficiency during algorithm design. For example, while T-RevSNN can save GPU memory by performing time-step inverse calculations at the neuron level, it introduces additional computational load, leading to a significant increase in power consumption, which is not conducive to deployment on edge devices or low-power platforms.

[0007] Therefore, it is necessary to improve one or more of the problems existing in the above-mentioned related technical solutions.

[0008] It should be noted that this section is intended to provide background or context for the technical solutions of this disclosure as set forth in the claims. The description herein does not constitute an admission that it is prior art simply because it is included in this section. Summary of the Invention

[0009] The purpose of this disclosure is to provide a method for constructing a reversible spiking neural network that supports parallel computing, thereby overcoming, to at least some extent, one or more problems caused by the limitations and defects of related technologies.

[0010] According to embodiments of this disclosure, a method for constructing a reversible spiking neural network supporting parallel computing is provided, comprising: A reversible spiking neural network is constructed by concatenating multiple reversible blocks; each reversible block contains two complementary first sub-functions F and second sub-functions G. The output of the previous reversible block is used as the input of the next reversible block. After the first sub-function F of the previous layer is calculated, it is simultaneously fed into the second sub-function G of the first layer and the first sub-function F of the second layer for parallel calculation. Establish a forward parallel scheduling mechanism and perform forward propagation computation; Establish a reverse parallel scheduling mechanism and perform reverse reconstruction to obtain the input for each reversible block; Based on the input of each invertible block, the gradient of the input of each invertible block is obtained, so as to calculate the weight gradient of the first subfunction F and the weight gradient of the second subfunction G; Update the weights of the first subfunction F and the second subfunction G based on the weight gradients of the first subfunction F and the second subfunction G. After multiple rounds of forward propagation, backward reconstruction, and parameter updates, the final reversible spiking neural network is obtained.

[0011] Furthermore, when the first sub-function F and the second sub-function G adopt a convolutional structure, the first sub-function F and the second sub-function G have the same structure and are composed of sequentially connected spiking neuron coding layers, convolutional layers and batch normalization layers. When the first subfunction F and the second subfunction G employ attention modules, the first subfunction F is a pulse self-attention mechanism layer, consisting of linear layers and matrix multiplication modules; the second subfunction G is a multilayer perceptron module, consisting of linear layers.

[0012] Furthermore, for the first-level reversible block, the input... Divided into the first input section along the channel direction Second input section Perform the following transformation:

[0013]

[0014] Then the first output part is obtained Second output section Assembled according to channels ,Will To input the output into the loss calculation function, or to...

[0015] Furthermore, each reversible block performs forward computation sequentially, so the output of the nth reversible block is:

[0016]

[0017] The output of the (n+1)th layer reversible block is:

[0018]

[0019] in, This is the first output part of the nth layer reversible block. This is the second output part of the nth layer reversible block. This is the first output part of the (n-1)th level reversible block. This is the second output part of the (n-1)th level reversible block. This is the first output part of the (n+1)th layer invertible block. This is the second output part of the (n+1)th reversible block.

[0020] Furthermore, since the second sub-function G in the second output part of the nth layer and the first sub-function F in the first output of the (n+1)th layer have the same input, the forward parallel scheduling mechanism is configured as follows: If it is in a GPU, the first sub-function F and the second sub-function G will be mapped to different thread groups or independent GPUs for execution. In a hardware accelerator, the first sub-function F and the second sub-function G are mapped to two independent computing units, and computing resources are dynamically allocated to different sub-functions through a scheduler to achieve parallelization at the network level.

[0021] Furthermore, during backpropagation, the reversibility relation is used to reconstruct the intermediate features; where the output of the (n+1)th reversible block is:

[0022]

[0023] The output of the nth layer reversible block is:

[0024] .

[0025] Furthermore, the reverse parallel scheduling mechanism is configured as follows: when After obtaining, and They are deployed on different computing units and executed in parallel. In a GPU, either thread parallelism or stream parallelism is used. In a hardware accelerator, the first sub-function F and the second sub-function G are deployed on independent computing cores for execution, thereby achieving acceleration in backpropagation as well.

[0026] when After obtaining, and They are deployed on different computing units and executed in parallel. If it is in a GPU, the first sub-function F and the second sub-function G will be mapped to different thread groups or independent GPUs for execution. In a hardware accelerator, the first sub-function F and the second sub-function G are mapped to two independent computing units, and computing resources are dynamically allocated to different sub-functions through a scheduler to achieve parallelization at the network level.

[0027] Furthermore, the parameter update process is as follows:

[0028]

[0029]

[0030]

[0031]

[0032]

[0033] in, , , , , Let x1 be the gradient. Let x2 be the gradient. Let y1 be the gradient. Let y2 be the gradient. For the first subfunction F, its relationship to gradient, For the second subfunction G pair gradient, This represents the total error.

[0034] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: In the embodiments of this disclosure, the above-described method for constructing a reversible spiking neural network supporting parallel computing achieves several advantages. Firstly, by introducing a reversible computational structure, the neuron states of intermediate layers do not need to be explicitly stored during the forward propagation of the network. Activation values ​​can be reconstructed through reversible mapping relationships during the backward propagation stage. This significantly reduces the consumption of GPU memory and on-chip storage resources, making it suitable for memory-constrained embedded and edge computing platforms. Secondly, the reversible structure fully considers hardware mapping characteristics during the design phase, incorporating both the forward computation and backward reconstruction processes of the reversible mapping into a parallel execution framework of a configurable computing array. Through the collaborative design of data paths and control logic, forward propagation and reverse reversible reconstruction can run efficiently in parallel on the same array, significantly reducing computational idle time and data waiting time between stages. While ensuring reversibility, the pipelined and parallel execution of forward and reverse computations is achieved, improving the overall system throughput and training speed. Thirdly, strict reversible mapping relationships and numerical consistency are maintained at the algorithm level, ensuring that no additional numerical errors are introduced during parallelization and hardware acceleration. This significantly improves computational speed while maintaining the accuracy of the serial reversible structure. Attached Figure Description

[0035] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0036] Figure 1 The diagram illustrates the steps of a method for constructing a reversible spiking neural network that supports parallel computing, according to an exemplary embodiment of this disclosure. Figure 2 A flowchart illustrating the forward and backward computation of a reversible block in an exemplary embodiment of this disclosure is shown; Figure 3 This diagram illustrates the forward parallel and backward computation processes when two reversible blocks are connected in series in an exemplary embodiment of this disclosure. Figure 4 This diagram illustrates the construction of functions F and G in an exemplary embodiment of this disclosure. Figure 5 This diagram illustrates how reversible blocks are assigned to different positions on a pulsating array in an exemplary embodiment of this disclosure. Figure 6 This diagram illustrates a reversible block deployed in a GPU in an exemplary embodiment of this disclosure. Detailed Implementation

[0037] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0038] Furthermore, the accompanying drawings are merely illustrative diagrams of embodiments of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0039] This example implementation provides a method for constructing a reversible spiking neural network that supports parallel computing. (See reference...) Figure 1 As shown, the method for constructing the reversible spiking neural network that supports parallel computing may include: In step S101: a reversible spiking neural network is constructed by concatenating multiple reversible blocks; wherein each reversible block contains two complementary first sub-functions F and second sub-functions G; In step S102: the output of the previous reversible block is used as the input of the next reversible block. After the first sub-function F of the previous layer is calculated, it is simultaneously fed into the second sub-function G of the first layer and the first sub-function F of the second layer for parallel calculation. In step S103: a forward parallel scheduling mechanism is established, and forward propagation computation is performed; In step S104: a reverse parallel scheduling mechanism is established and reverse reconstruction is performed to obtain the input of each reversible block; In step S105: Based on the input of each invertible block, the gradient of the input of each invertible block is obtained, so as to calculate the weight gradient of the first sub-function F and the weight gradient of the second sub-function G; In step S106: Update the weights of the first sub-function F and the second sub-function G according to the weight gradient of the first sub-function F and the weight gradient of the second sub-function G; In step S107: After multiple rounds of forward propagation, backward reconstruction and parameter update, the final reversible spiking neural network is obtained.

[0040] The aforementioned method for constructing a reversible spiking neural network supporting parallel computing achieves several advantages. Firstly, by introducing a reversible computational structure, the neuron states of intermediate layers do not need to be explicitly stored during forward propagation. Activation values ​​can be reconstructed through reversible mapping relationships during backward propagation. This significantly reduces the consumption of GPU memory and on-chip storage resources, making it suitable for memory-constrained embedded and edge computing platforms. Secondly, the reversible structure fully considers hardware mapping characteristics during the design phase, incorporating both forward computation and backward reconstruction processes into a parallel execution framework of a configurable computing array. Through the collaborative design of data paths and control logic, forward propagation and reverse reversible reconstruction can run efficiently in parallel on the same array, significantly reducing computational idle time and data waiting time between stages. While ensuring reversibility, pipelined and parallel execution of forward and reverse computations is achieved, improving overall system throughput and training speed. Thirdly, strict reversible mapping relationships and numerical consistency are maintained at the algorithm level, ensuring that no additional numerical errors are introduced during parallelization and hardware acceleration. This significantly improves computational speed while maintaining the accuracy of the serial reversible structure.

[0041] Below, we will refer to Figures 1 to 6 The steps of the above-described method for constructing a reversible spiking neural network that supports parallel computing in this example embodiment will be described in more detail.

[0042] In step S101, a reversible spiking neural network is constructed by concatenating multiple reversible blocks; wherein each reversible block contains two complementary first sub-functions F and second sub-functions G.

[0043] Specifically, within the neural network framework, a reversible spiking neural network is constructed, consisting of multiple reversible blocks connected in series. Each reversible block contains two complementary sub-modules. (i.e., the first subfunction F) and (i.e., the second sub-function G), its input Divided into the first input section along the channel direction Second input section Perform the following transformation:

[0044]

[0045] Received , Then splice them together according to the channels. , specifically Figure 2 As shown. The obtained y can be used as the output and fed into the loss calculation function, or it can be directly fed into the next layer of the invertible block, or it can be fed into the invertible block after being boosted through a one-dimensional convolution. and It consists of spiking neuron coding layers, convolutional layers, and batch normalization layers, or it can be composed of subnetworks of a Transformer. For example... Figure 4 As shown.

[0046] In step S102, the output of the previous reversible block is used as the input of the next reversible block. After the first sub-function F of the previous layer is calculated, it is simultaneously fed into the second sub-function G of the first layer and the first sub-function F of the second layer for parallel calculation.

[0047] Specifically, the output of each reversible block serves as the input of the next reversible block, and inter-layer data dependencies are achieved through reversible mapping relationships, ensuring that the network retains reversibility and reconfigurability even after multiple layers are stacked.

[0048] More specifically, multiple reversible blocks are stacked sequentially to form a deep network: First layer:

[0049]

[0050] Second layer:

[0051] ... Level n-1:

[0053]

[0054] nth level:

[0055]

[0056] It is not difficult to see that the first layer After the calculation was completed, it was sent to the first layer. The second layer Parallel computing, such as Figure 3 As shown. And so on, for the (n-1)th layer... After the calculation is completed, it is sent to the (n-1)th layer. and the nth layer Parallel computing.

[0057] In step S103, a forward parallel scheduling mechanism is established and forward propagation computation is performed.

[0058] Specifically, a forward parallel scheduling mechanism should be established.

[0059] Based on the parallel relationship in step S102, this application designs a parallel scheduling mechanism for forward computation: On a GPU, the F function (i.e., the first sub-function F) and the G function (i.e., the second sub-function G) can be mapped to different thread groups or independent GPUs for execution, such as... Figure 6 As shown, on a GPU device, this application maps the independent sub-functions F and G in the reversible neural network to different parallel computing resources of the GPU for execution. Specifically, the GPU contains multiple parallel execution units, such as Streaming Multiprocessor (SM), thread blocks, and multiple CUDA Streams. Utilizing these hardware parallel structures, this application encapsulates the F and G functions into independent GPU kernels and schedules their execution asynchronously on different Streams or different thread block groups.

[0060] In a dedicated hardware accelerator (such as an FPGA, ASIC, or a self-developed neural network acceleration chip), this application splits the reversible module into two decoupled sub-computation paths, which are then mapped to two independent hardware computing units (Processing Elements, PEs) for execution. Figure 5 As shown.

[0061] This mechanism significantly reduces the sequential stages of waiting dependencies and accelerates the forward propagation of reversible networks.

[0062] Perform forward propagation computation.

[0063] Each reversible block performs forward computation sequentially.

[0064] Both the forward computation of the F and G functions support interchangeable implementations of convolutional and attention structures, thus allowing them to be applied to different network architectures (including ResNet and Transformer types).

[0065] Specifically, when F and G use a convolutional structure, such as Figure 2 As shown, the ResNet channel structure can be modified into a reversible form; the F / G functions are composed of spiking neuron layers, convolutional layers, and batch normalization layers, respectively.

[0066] When F and G use attention modules, such as Figure 3 As shown, a Transformer-style invertible structure can be constructed. The F function is the impulse self-attention mechanism layer, and the G function is the multilayer perceptron module. Both are composed of linear layers, with the multilayer attention mechanism module adding a matrix multiplication module.

[0067] This design balances versatility and scalability across different network architectures.

[0068] In step S104, a reverse parallel scheduling mechanism is established and reverse reconstruction is performed to obtain the input of each reversible block.

[0069] Specifically, reverse reconstruction is performed.

[0070] During backpropagation, the reversible relationship is used to reconstruct intermediate features without storing intermediate activations.

[0071] Let the output be ( , If the answer is ), then the reverse calculation is as follows: nth level:

[0072]

[0073] Level n-1:

[0074] ... Second layer:

[0076]

[0077] First layer:

[0078]

[0079] This process allows for the reconstruction of the original input x layer by layer, achieving lossless reverse recovery.

[0080] Establish a reverse parallel scheduling mechanism.

[0081] Similar to the forward phase, when After obtaining, and It can be executed in parallel on different computing units. In GPU implementations, thread parallelism or streaming parallelism can be used; in hardware accelerators, the F and G functions can be deployed on independent computing cores for execution, thus achieving acceleration in backpropagation as well.

[0082] In steps S105 to S107, the gradient of the input of each reversible block is obtained based on the input of each reversible block, so as to calculate the weight gradient of the first sub-function F and the weight gradient of the second sub-function G; the weights of the first sub-function F and the second sub-function G are updated based on the weight gradients of the first sub-function F and the second sub-function G; after multiple rounds of forward propagation, backward reconstruction and parameter update, the final reversible spiking neural network is obtained.

[0083] Specifically, the parameters need to be updated.

[0084] The following example uses a reversible block. Based on steps S101 and S103, the output is... , It is known. Therefore, each input, and its gradient, can be obtained using the following formula.

[0085]

[0086]

[0087]

[0088]

[0089]

[0090]

[0091] Here, L is the total error. The same applies to the others. It is the gradient of the weights of the F function. This is the gradient of the weights of the G function. Using the functions mentioned above, we can sequentially obtain the initial input, its gradient, and the gradients of the weights. After obtaining these, we can update the weights of the F and G functions.

[0092] Complete network training and inference.

[0093] After multiple rounds of forward propagation, backward reconstruction, and parameter updates, a performance-optimized reversible spiking neural network model is finally obtained. This model maintains the memory advantages of the reversible structure while significantly improving computational speed and hardware execution efficiency through a parallel scheduling mechanism. Specifically, the same reversible computational path is used in both the network training and inference phases, and the forward and backward structures are kept symmetrical during inference to achieve high-energy-efficiency, low-latency online learning.

[0094] In one specific embodiment, the ResNet architecture and its reversible form were deployed on a single GPU.

[0095] Table 1

[0096] Table 2

[0097] As shown in Tables 1 and 2, this embodiment tested two different datasets (CIFAR10 and CIFAR100), with 200 training rounds for each dataset. ResNet34 represents the traditional ResNet network, where 34 indicates its number of layers. RevSResNet37 is a traditional reversible spiking neural network with 37 layers, and ParaRevSResNet37 is the parallel reversible spiking neural network proposed in this application, also with 37 layers. To ensure consistent network parameters, IF neurons were used throughout. It can be seen that the proposed parallel reversible spiking neural network outperforms the original RevSResNet network in both training and inference speeds, with minimal decrease in accuracy. The parallel reversible spiking neural network also reduces memory usage during training / inference.

[0098] The aforementioned method for constructing a reversible spiking neural network supporting parallel computing achieves several advantages. Firstly, by introducing a reversible computational structure, the neuron states of intermediate layers do not need to be explicitly stored during forward propagation. Activation values ​​can be reconstructed through reversible mapping relationships during backward propagation. This significantly reduces the consumption of GPU memory and on-chip storage resources, making it suitable for memory-constrained embedded and edge computing platforms. Secondly, the reversible structure fully considers hardware mapping characteristics during the design phase, incorporating both forward computation and backward reconstruction processes into a parallel execution framework of a configurable computing array. Through the collaborative design of data paths and control logic, forward propagation and reverse reversible reconstruction can run efficiently in parallel on the same array, significantly reducing computational idle time and data waiting time between stages. While ensuring reversibility, pipelined and parallel execution of forward and reverse computations is achieved, improving overall system throughput and training speed. Thirdly, strict reversible mapping relationships and numerical consistency are maintained at the algorithm level, ensuring that no additional numerical errors are introduced during parallelization and hardware acceleration. This significantly improves computational speed while maintaining the accuracy of the serial reversible structure.

[0099] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0100] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0101] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A method of constructing a reversible spiking neural network supporting parallel computation, characterized by, The application relates to a reversible pulse neural network, and relates to a method for training the reversible pulse neural network. The reversible pulse neural network is constructed according to a plurality of reversible blocks in series, wherein each reversible block comprises two complementary first and second sub-functions F and G. The output of a previous layer of reversible blocks is taken as the input of a next layer of reversible blocks, and the first sub-function F of the previous layer is calculated, and then the output is simultaneously sent to the second sub-function G of the first layer and the first sub-function F of the second layer for parallel calculation. A forward parallel scheduling mechanism is established, and forward propagation calculation is performed. A reverse parallel scheduling mechanism is established, and reverse reconstruction is performed to obtain the input of each reversible block. The gradient of the input of each reversible block is obtained according to the input of each reversible block, so as to calculate the weight gradient of the first sub-function F and the weight gradient of the second sub-function G. The weight of the first sub-function F and the weight of the second sub-function G are updated according to the weight gradient of the first sub-function F and the weight gradient of the second sub-function G. After multiple rounds of forward propagation, reverse reconstruction and parameter updating, the final reversible pulse neural network is obtained.

2. The method of claim 1, wherein the method further comprises: When the first sub-function F and the second sub-function G adopt a convolution structure, the first sub-function F and the second sub-function G have the same structure and are composed of a pulse neuron coding layer, a convolution layer and a batch normalization layer connected in series. When the first sub-function F and the second sub-function G adopt an attention module, the first sub-function F is a pulse self-attention mechanism layer composed of a linear layer and a matrix multiplication module, and the second sub-function G is a multi-layer perception module composed of a linear layer.

3. The method of claim 2, wherein the method further comprises: For the 1st layer reversible block, input is divided into a first input part and a second input part in the channel direction, and the following transformation is performed: Then the first output part is obtained Second output section Assembled according to channels ,Will To input the output into the loss calculation function, or to... It is fed into the reversible block of the next layer.

4. The method of constructing a reversible spiking neural network supporting parallel computation according to claim 3, wherein, The nth layer of reversible blocks is sequentially executed for forward calculation, and the output of the nth layer of reversible blocks is: The output of the (n+1)th layer of reversible blocks is: wherein, is a first output portion of an n-th layer reversible block, is a second output portion of the n-th layer reversible block, is a first output portion of an n-1-st layer reversible block, is a second output portion of the n-1-st layer reversible block, is a first output portion of an n+1-st layer reversible block, is a second output portion of the n+1-st layer reversible block.

5. The method of claim 4, wherein the method further comprises: Since the second sub-function G in the second output part of the nth layer and the first sub-function F in the first output of the (n+1)th layer are the same input, the forward parallel scheduling mechanism is configured as: If in a GPU, the first sub-function F and the second sub-function G are respectively mapped to different thread groups or independent GPUs for execution. If in a hardware accelerator, the first sub-function F and the second sub-function G are mapped to two independent calculation units, and the calculation resources are dynamically allocated to different sub-functions by a scheduler to realize parallelization of network levels.

6. The method of claim 5, wherein the method further comprises: In reverse propagation, the reversible relationship is used to reconstruct the intermediate features, wherein the output of the (n+1)th layer of reversible blocks is: The output of the nth layer of reversible blocks is: 。 7. The method of claim 6, wherein the method further comprises: Since the second sub-function G in the second output part of the nth layer and the first sub-function F in the first output of the (n+1)th layer are the same input, the reverse parallel scheduling mechanism is configured as: If in a GPU, the first sub-function F and the second sub-function G are respectively mapped to different thread groups or independent GPUs for execution. If in a hardware accelerator, the first sub-function F and the second sub-function G are mapped to two independent calculation units, and the calculation resources are dynamically allocated to different sub-functions by a scheduler to realize parallelization of network levels.

8. The method of claim 7, wherein the method further comprises: The parameter updating process is: wherein , , , , is the gradient of x1, is the gradient of x2, is the gradient of y1, is the gradient of y2, is the gradient of the first sub-function F with respect to its , is the gradient of the second sub-function G with respect to , is the overall error.

Images