Gesture recognition system based on adaptive batch channel normalized spiking neural network

Abstract

The invention discloses a gesture recognition system based on a self-adaptive batch channel normalized spiking neural network. The gesture recognition system is characterized by comprising a data acquisition module, a preprocessing module and a gesture recognition module which are connected in sequence, the gesture recognition module is provided with an input layer, a pulse neural network, a full connection layer FC and an adaptive weighted nuclear norm layer AWNN which are connected in sequence; the spiking neural network is provided with a convolution input layer, the convolution input layer is connected with a batch channel normalization network module, and the batch channel normalization network module is provided with m network blocks which are sequentially connected end to end; the data acquisition module is used for acquiring original gesture data with a time sequence attribute; the preprocessing module is used for preprocessing the original gesture data to obtain standard gesture data; and the gesture recognition module is used for performing gesture recognition operation on the standard gesture data to obtain a gesture recognition result. The method has the advantages that redundant time sequence noise can be suppressed on the whole, and power consumption is reduced.

Description

Technical Field

[0001] This invention relates to the field of spiking neural network technology, and in particular to a gesture recognition system based on an adaptive batch channel normalized spiking neural network. Background Technology

[0002] During the training of deep spiking neural networks (SNNs), due to the randomness of parameter initialization and the variations in the input pulse sequence across time and samples, the feature distribution received by the hidden layers continuously drifts during training iterations. This phenomenon is commonly referred to as Internal Covariate Shift (ICS). In SNNs, the ICS problem takes on a more complex form: distribution drift not only manifests in changes in activation amplitude but also directly affects the dynamics of membrane potential and the temporal characteristics of pulse firing rate, resulting in pulse firing imbalance, timescale shift, and deep neuron "silencing" or over-firing. This phenomenon weakens the unique temporal characteristics of spiking neural networks, reducing the stability and accuracy of SNN training.

[0003] Normalization techniques are widely considered a key component in mitigating covariate shift issues within deep SNN training networks, as well as stabilizing training and accelerating convergence. However, only a few normalization methods have been proposed for SNNs. Summarizing existing research, current SNN normalization methods mainly fall into two categories: The first category performs independent normalization only along the time or channel dimension. For example, the Batch Normalized Temporal Transformer (BNTT) performs independent normalization at each discrete time step. This method ignores inter-channel structural information, making it less suitable for sparse impulse mechanisms and small-batch training. Wu et al. proposed the neuron normalization technique NeuNorm, which relies solely on spatial location normalization along the channel dimension, effectively mitigating the impulse firing imbalance problem in SNNs. However, it cannot capture feature changes in the time and sample dimensions, making it difficult to handle complex temporal structures, significant sample differences, or deep networks. The second category mainly performs joint normalization by aggregating data in the batch and time dimensions, such as Threshold Dependent Batch Normalization (tdBN) and Time-Effective Batch Normalization (TEBN). These methods can effectively acquire information across samples and time steps, suppressing distribution drift caused by temporal accumulation to some extent. However, in deep networks or with short time steps, they may overlook the structured information within a single sample. In spiking neural networks, batch normalization primarily captures global pulse firing information and membrane potential changes across samples and time dimensions, serving as a global steady-state regulation mechanism. Channel normalization, on the other hand, focuses more on the relative pulse firing intensity relationships between different channels within a single sample, enhancing the unique key feature structures of a single sample while maintaining pulse sparsity. The two are functionally complementary, but currently, there is no method in the SNN field that simultaneously integrates channel, time, and batch dimensions for normalization.

[0004] Limitations of Existing Technologies: Spiking Neural Networks (SNNs) have shown great potential in neuromorphic computing due to their event-driven and low-power characteristics. However, the discrete pulse firing mechanism and temporal accumulation dynamics in SNNs lead to problems such as gradient instability, temporal covariate shift, and feature redundancy during direct training. Existing research mainly normalizes and balances neuronal pulse firing in a single batch dimension, neglecting the global structural correlation of SNNs in the temporal-feature dimension. Summary of the Invention

[0005] The present invention provides a gesture recognition system based on an adaptive batch channel normalized spiking neural network, which can suppress redundant timing noise and reduce power consumption as a whole.

[0006] To achieve the above objectives, this invention provides a gesture recognition system based on an adaptive batch channel normalized spiking neural network, which, crucially, includes the following steps:

[0007] It is equipped with a data acquisition module, a preprocessing module, and a gesture recognition module connected in sequence;

[0008] The gesture recognition module is provided with an input layer, a spiking neural network (SNN), a fully connected layer (FC), and an adaptive weighted kernel norm layer (AWNN) connected in sequence. The spiking neural network (SNN) is provided with a convolutional input layer, which is connected to a batch channel normalization network module. The batch channel normalization network module is provided with m network blocks connected end to end.

[0009] The data acquisition module is used to acquire raw gesture data with time-series attributes;

[0010] The preprocessing module is used to preprocess the original gesture data to obtain standard gesture data that conforms to network input.

[0011] The gesture recognition module is used to perform gesture recognition operations on the standard gesture data to obtain gesture recognition results.

[0012] Through the above design, this invention calculates the characteristics of spiking neurons in both the batch and channel dimensions using a batch-channel normalization layer. Then, it adjusts the spiking activation distribution between different channels and samples using adaptive weights, effectively mitigating the temporal covariate offset problem during training and stabilizing gradient propagation. Furthermore, to enable more stable and continuous iterative optimization of the learned key temporal-feature structures during training, this invention proposes an adaptive weighted kernel norm (AWNN) regularization strategy. This strategy adaptively learns the weights of each structural component to impose low-rank structural constraints on the SNN's temporal-feature matrix, effectively preserving its core spatiotemporal structure, suppressing redundant temporal noise, and reducing power consumption.

[0013] Preferably, the m network blocks have the same structure, each having a first convolutional layer, a first batch of channel normalization layers, a first neuron layer, a second convolutional layer, a second batch of channel normalization layers, a second neuron layer, a pooling layer, and an addition unit connected in sequence, with a third batch of channel normalization layers connected in parallel between the inputs of the first convolutional layer and the addition unit.

[0014] Preferably, the first batch of channel normalization layers and the second batch of channel normalization layers have the same structure, and both are provided with batch-time joint dimension (N×T, H, W) processing units and channel-space dimension (C, H, W) processing units in parallel. The output of the batch-time joint dimension processing unit and the channel-space dimension processing unit are connected to the input of the same weighted fusion unit.

[0015] Preferably, the batch-time joint dimension processing unit first calculates the current batch channel normalized layer input signal along the (N×T, H, W) axis. mean and variance The expression is:

[0016] ;

[0017] ;

[0018] Then use the mean. and variance For input signal After normalization, the expression is:

[0019] ;

[0020] in, This represents the input signal of the current batch channel normalization layer. , The total time step is represented by i, the batch-time index is represented by m, and the total number of batches-time is represented by m∈N×T; For hyperparameters, ; A small constant set to ensure numerical stability. This represents the normalized output of the batch-time joint dimension processing unit;

[0021] The channel-space dimension processing unit first calculates the input signal of the normalized layer of the current batch of channels along the (C, H, W) axis. mean and variance The expression is:

[0022] ;

[0023] ;

[0024] Then use the mean. and variance For input signal After normalization, the expression is:

[0025] ;

[0026] in, Indicates the total number of channels. Indicates the channel index. This represents the normalized output of the channel-space dimension processing unit.

[0027] Preferably, the weighted fusion unit uses learning parameters. The normalized outputs along the (N×T, H, W) and (C, H, W) axes are weighted and fused as follows:

[0028] ;

[0029] Therefore, the final normalized output expression of the batch channel normalization layer is:

[0030] ;

[0031] in, These are the learning parameters.

[0032] The batch channel normalization layer considers both batch normalization and channel normalization, thus preserving the relative importance of channels within each time step while maintaining temporal consistency. Then,

[0033] This invention introduces learnable parameters, enabling the network to automatically adjust the fusion ratio of batch normalization and channel normalization, making the normalization operation more flexible and adaptable to different data and network structures.

[0034] Preferably, the loss of the gesture recognition module includes temporally effective training loss. and adaptive weighted norm loss The expression is:

[0035] ;

[0036] ;

[0037] ;

[0038] in, Indicates the total time step. For time step index, Indicates the target label, Represents cross-entropy loss, It is the output of time step t. The shape is [N,D], where N is the batch size and D is the output feature dimension; For batch indexing, ; The total number of singular values. This is the singular value index, and the superscript T is the matrix transpose; Indicates global adaptive weights; , An initialization of a vector of length R consisting entirely of 1s is equivalent to the standard nuclear norm. express The rank of the matrix, ; This represents the time-feature matrix of the k-th sample;

[0039] During training By automatically adjusting through gradient descent, large singular values ​​corresponding to the core structure of the time-feature matrix are given greater weight during network updates, while small singular values ​​corresponding to noise or redundant information are given less weight. This allows the network model to dynamically filter out information from the more important time-feature structures.

[0040] By adding the hyperparameter λ to adjust the proportion of the regularization term, the total loss function of the gesture recognition module is thus adjusted. for:

[0041] .

[0042] This invention applies low-rank structure constraints to the time-feature matrix, filtering out redundant noise and preserving the core spatiotemporal structure of the pulse sequence as a whole. Simultaneously, it introduces learnable adaptive weights, enabling the model to dynamically adjust the constraint strength based on data sparsity and noise levels, thereby enhancing both robustness and temporal feature representation capabilities.

[0043] The beneficial effects of this invention are as follows: This invention proposes a novel adaptive spatiotemporal backpropagation batch channel normalization method, which is implemented based on the batch channel normalization layer in the gesture recognition module. This method simultaneously calculates the temporal structural features of the batch dimension, time dimension, and channel dimension during the normalization process, and adaptively adjusts the contributions of different dimensions during spatiotemporal backpropagation, enabling the network to simultaneously learn the global pulse firing characteristics across samples and time steps, as well as the structured channel information within a single sample.

[0044] Furthermore, because the batch channel normalization layer aBCN adds a spatiotemporal structured representation within the samples during the feature normalization stage, the network forms an output representation with a clear temporal-feature structure during task execution. However, current training methods based on time-step error accumulation, such as TET loss, mainly focus on prediction errors at local moments and lack global constraints on the overall temporal-feature structure. Therefore, to enable the network to achieve more stable and continuous iterative optimization of the learned key temporal-feature structures during training, this invention further proposes an adaptive weighted nuclear norm training method, which is implemented based on the adaptive weighted nuclear norm layer AWNN. This method, based on adaptive batch channel normalization, applies structured regularization constraints to the network's output temporal-feature matrix and assigns greater weights to temporal structural features with higher discriminative value. Through the backpropagation mechanism, the AWNN can adaptively learn and update the weights of each structural component, thereby effectively preserving its core spatiotemporal structure while maintaining the sparsity of the impulse signal, and suppressing redundant temporal noise and reducing power consumption as a whole. Attached Figure Description

[0045] Figure 1 This is a diagram illustrating the overall framework of the gesture recognition system in this embodiment.

[0046] Figure 2 The training accuracy curves of batch normalization and batch channel normalization techniques in the embodiments are shown on (a) CIFAR10, (b) CIFAR100, and (c) CIFAR10DVS.

[0047] Figure 3 The following are test accuracy curves of batch normalization and batch channel normalization techniques in the embodiments on (a) CIFAR10, (b) CIFAR100, and (c) CIFAR10DVS.

[0048] Figure 4 In this example, the spectral energy concentration and effective rank of features and logits learned using the CE, TET and AWNN methods on the CIFAR10DVS dataset are shown.

[0049] Figure 5 The figure shows a comparison of the hierarchical average burst firing rates of SNNs trained on (a) CIFAR10, (b) CIFAR100, and (c) CIFAR10-dvs using AWNN, CE, and TET methods, respectively, in the examples. Detailed Implementation

[0050] The present invention will be further described in detail below with reference to the accompanying drawings and specific examples. The following embodiments or drawings are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0051] A gesture recognition system based on an adaptive batch channel normalized spiking neural network is provided, which includes a data acquisition module, a preprocessing module and a gesture recognition module connected in sequence.

[0052] like Figure 1 As shown, the gesture recognition module is configured with an input layer, a spiking neural network (SNN), a fully connected layer (FC), and an adaptive weighted kernel norm layer (AWNN) connected in sequence. The spiking neural network (SNN) has a convolutional input layer connected to a batch channel normalization network module. The batch channel normalization network module has m network blocks connected end-to-end. The m network blocks have the same structure, each having a first convolutional layer, a first batch channel normalization layer, a first neuron layer, a second convolutional layer, a second batch channel normalization layer, a second neuron layer, a pooling layer, and an addition unit connected in sequence. A third batch channel normalization layer is connected in parallel between the inputs of the first convolutional layer and the addition unit.

[0053] The data acquisition module is used to acquire raw gesture data with time-series attributes;

[0054] The preprocessing module is used to preprocess the original gesture data to obtain standard gesture data that conforms to network input.

[0055] The gesture recognition module is used to perform gesture recognition operations on the standard gesture data to obtain gesture recognition results.

[0056] The gesture recognition module ultimately calculates the probability distribution values ​​corresponding to various gesture category labels, and outputs the gesture category label corresponding to the maximum probability distribution value as the gesture recognition result.

[0057] The first and second neuron layers use the LIF iterative model, and the iterative expression is:

[0058] ;

[0059] in, A constant representing the decay of membrane potential. This represents the membrane potential at time t. This represents the presynaptic input at time t;

[0060] ;

[0061] in, This represents the binary impulse output of neuron j at time t. As weight, Here, b represents the neuron index, and b represents the bias. , Indicates the batch processing axis. Indicates the channel axis. Indicates spatial axes;

[0062] when At this time, the neuron emits a pulse and resets the membrane potential to 0, that is... Therefore, the LIF iterative model expression in the space and time domains is:

[0063] ;

[0064] ;

[0065] in, Indicates the launch threshold. This represents the membrane potential of the nth layer neuron at time t. This represents the binary pulse output of the nth layer neuron at time t. This represents the presynaptic input of the nth layer neuron at time t. This is the activation function.

[0066] The first batch of channel normalization layers and the second batch of channel normalization layers have the same structure. Both are equipped with batch-time joint dimension (N×T, H, W) processing units and channel-space dimension (C, H, W) processing units in parallel. The output of the batch-time joint dimension processing unit and the channel-space dimension processing unit are connected to the input of the same weighted fusion unit.

[0067] The batch-time joint dimension processing unit first calculates the current batch channel normalization layer input signal along the (N×T, H, W) axis. mean and variance The expression is:

[0068] ;

[0069] ;

[0070] Then use the mean. and variance For input signal After normalization, the expression is:

[0071] ;

[0072] in, This represents the input signal of the current batch channel normalization layer. , The total time step is represented by i, the batch-time index is represented by m, and the total number of batches-time is represented by m∈N×T; For hyperparameters, ; A small constant set to ensure numerical stability. This represents the normalized output of the batch-time joint dimension processing unit;

[0073] The channel-space dimension processing unit first calculates the input signal of the normalized layer of the current batch of channels along the (C, H, W) axis. mean and variance The expression is:

[0074] ;

[0075] ;

[0076] Then use the mean. and variance For input signal After normalization, the expression is:

[0077] ;

[0078] in, Indicates the total number of channels. Indicates the channel index. This represents the normalized output of the channel-space dimension processing unit.

[0079] The weighted fusion unit uses learning parameters. The normalized outputs along the (N×T, H, W) and (C, H, W) axes are weighted and fused as follows:

[0080] ;

[0081] Therefore, the final normalized output expression of the batch channel normalization layer is:

[0082] ;

[0083] in, These are the learning parameters.

[0084] The loss of the gesture recognition module includes temporal effective training loss. and adaptive weighted norm loss The expression is:

[0085] ;

[0086] ;

[0087] ;

[0088] in, Indicates the total time step. For time step index, Indicates the target label, Represents cross-entropy loss, It is the output of time step t. The shape is [N,D], where N is the batch size and D is the output feature dimension; For batch indexing, ; The total number of singular values. This is the singular value index, and the superscript T is the matrix transpose; Indicates global adaptive weights; , An initialization of a vector of length R consisting entirely of 1s is equivalent to the standard nuclear norm. express The rank of the matrix, ; This represents the time-feature matrix of the k-th sample;

[0089] During training By automatically adjusting through gradient descent, large singular values ​​corresponding to the core structure of the time-feature matrix are given greater weight during network updates, while small singular values ​​corresponding to noise or redundant information are given less weight. This allows the network model to dynamically filter out information from the more important time-feature structures.

[0090] By adding the hyperparameter λ to adjust the proportion of the regularization term, the total loss function of the gesture recognition module is thus adjusted. for:

[0091] .

[0092] The total loss of the gesture recognition module is relative to the SNN synaptic weight matrix. The gradient expression is:

[0093] ;

[0094] ;

[0095] ;

[0096] in, The output value of the presynaptic input of the output layer after passing through softmax is the output value. For target label The unique hot code; This represents the input pulse vector of the k-th sample at time step t; ; The input pulse at time step t; Describe the SNN synaptic weight matrix, satisfying ;

[0097] Therefore, the gradient of the total loss with respect to the weights W is:

[0098] .

[0099] Next, the effectiveness of the present invention was verified using the residual network ResNet19 on the static datasets CIFAR10 and CIFAR100. The effectiveness was also verified using VGGSNN on the neuromorphic datasets CIFAR10-DVS and N-Caltech101. The results were compared with the latest research findings, validating the advantages of the proposed method in terms of accuracy and power consumption. Specific parameter settings are shown in Table 1.

[0100] A. Dataset

[0101] 1) CIFAR10 / 100: The CIFAR10 / 100 dataset consists of 60,000 32×32 images with 3 channels and 10 / 100 classes. There are 50,000 training images and 10,000 test images.

[0102] 2) CIFAR10-DVS: The CIFAR10-DVS dataset is an event-driven version adapted from the popular static dataset CIFAR10. This dataset uses a dynamic vision sensor to convert 10,000 frame-based images across 10,000 categories into an event stream. Since the CIFAR10-DVS dataset does not have separate training and test sets, this dataset is divided into 9k training images and 1k test images, and the spatial resolution is reduced from 128×128 to 48×48.

[0103] 3) N-Caltech 101: The N-Caltech 101 dataset is also converted from the original version of Caltech 101, with slight modifications to the object classes to avoid confusion. N-Caltech 101 consists of 100 object classes plus one background class. It is divided into a 90% training set and a 10% test set.

[0104] Table 1 Experimental Parameter Settings

[0105]

[0106] Accuracy Assessment: Compared with existing state-of-the-art research methods, the aBCN model proposed in this invention achieves superior TOP1 accuracy. Specifically, on CIFAR10, this invention achieves 96.11% accuracy at T=2, 96.38% accuracy at T=4, and 96.52% accuracy at T=6. On CIFAR100, it achieves 79.75% accuracy at T=2, 80.32% accuracy at T=4, and 80.51% accuracy at T=6. On CIFAR10DVS, it achieves 84.8% accuracy at T=10. On N-Caltech101, it achieves 84.10% accuracy at T=10.

[0107] Performance comparison analysis of normalization methods: such as Figure 2 , Figure 3 As shown, CIFAR10 / 100 was trained using SNN-ResNet19, and CIFAR10DVS was trained using VGGSNN. The training accuracy and test accuracy for the first 100 epochs were recorded, and the results of tdBN and aBCN are displayed. Other experimental settings are shown in Table 1. First, as... Figure 2 , Figure 3 As shown, during the training phase, aBCN exhibits faster convergence speed and higher training accuracy across all datasets. On CIFAR-10, aBCN achieves a training accuracy of 90.59% at epoch 39 and a test accuracy of 90.22% at epoch 22; while tdBN only achieves 86.75% training accuracy and 83.52% test accuracy at the same epoch. On CIFAR-100, aBCN achieves a training accuracy of 90.33% at epoch 53 and a test accuracy of 70.42% at epoch 26; while tdBN only achieves 87.20% training accuracy and 65.71% test accuracy at the same epoch. On CIFAR-10DVS, aBCN achieves a training accuracy of 90.12% at epoch 25 and a test accuracy of 79.10% at epoch 40; while tdBN only achieves 86.33% training accuracy and 70.30% test accuracy at the same epoch.

[0108] Through observation Figure 2It can be observed that on CIFAR-10 and CIFAR-100, aBCN quickly widens the performance gap with tdBN in the early stages of training and maintains its advantage throughout subsequent training, indicating that it can more effectively mitigate gradient oscillations during feature normalization and promote the network's learning of discriminative features. On CIFAR10-DVS, the training curve of BN shows significant fluctuations, while aBCN exhibits a smoother and more stable upward trend, indicating that aBCN is more robust to event-driven datasets. Through observation... Figure 3 It can be observed that during the testing phase, aBCN achieved consistent and stable improvements in generalization performance across all three datasets. Compared to tdBN, aBCN not only achieved higher performance in final test accuracy but also effectively suppressed drastic fluctuations in test accuracy throughout the training process. This advantage is particularly pronounced on the CIFAR10-DVS dataset, which exhibits significant temporal sparsity and noise. This indicates that aBCN can adaptively preserve key temporal structure information during normalization, thereby enhancing the model's generalization ability to complex spatiotemporal patterns.

[0109] Structured constraint analysis: Figure 4 (a), (b), and (c) show a comparison of the energy concentration ratio of the feature layer before classification on the top-k (k∈{1,2,3}) using the CE, TET, and AWNN methods, respectively. Figure 4 (d) shows a comparison of the effective rank reff of the feature layer before classification. Figure 4 (e), (f), and (g) represent comparisons of the energy concentration ratio of the original output (logits) of the classification layer on the top-k (k∈{1,2,3}) using the CE, TET, and AWNN methods, respectively. Figure 4 (h) shows a comparison of the effective rank reff of the original output logits of the classification layer.

[0110] To investigate how the use of CE, TET, and AWNN methods affects the geometric properties of learned features, this invention analyzes the singular value spectra of the original output logits of the pre-classification feature layer and the classification layer using energy concentration ratio (ECR) and effective rank. Furthermore, this invention uses ECR to quantify the proportion of total spectral energy captured by the top-k singular values. All spectral calculations are based on Frobenius-normalized representations to eliminate the influence of differences in overall numerical scale, such as variations in activation strength caused by different training algorithms or regularization methods, and differences in numerical magnitude between different network layers, thereby ensuring that the spectral analysis reflects the representation structure itself rather than the magnitude of the amplitude.

[0111] like Figure 4As shown, TET produces a relatively high representation rank and a more uniform spectral energy distribution, indicating the existence of a redundant subspace. Conversely, CE exhibits stronger energy concentration on the dominant singular components, but this also leads to a decrease in effective rank, meaning the representation space is excessively compressed. Notably, AWNN achieves a good balance between these two extremes. Figure 4 As shown in (a)-(c) and (e)-(g), compared with TET, AWNN significantly improves the spectral energy concentration of the first three singular components in both the feature space and the logits space, which proves its ability to suppress redundant high-rank structures.

[0112] Meanwhile, AWNN maintains a moderate effective rank, higher than CE, thus avoiding excessive dimensionality collapse and preserving sufficient discriminative representation capacity. Crucially, these trends remain consistent in the accuracy-matched experimental setting, ruling out the possibility that the reduction in effective rank is due to performance degradation. In summary, the results demonstrate that AWNN effectively filters redundant noise and preserves the core spatiotemporal structure of the pulse sequence by imposing a low-rank structural constraint on the time-feature matrix.

[0113] Formula for calculating effective rank:

[0114] , ;

[0115] in, This represents the i-th singular value. H represents the normalized weight corresponding to the i-th singular value, and H represents the spectral entropy. The effective rank represents the number of effective dimensions that actually participate in the changes in the characterization matrix.

[0116] Spectral entropy is used to measure the uniformity of the distribution of singular values. If all energy is concentrated on a few singular values ​​→ small H → it indicates "concentrated structure, low dimension". If energy is uniformly distributed on multiple singular values ​​→ large H → it indicates "dispersed structure, high dimension".

[0117] Formula for calculating spectral energy percentage:

[0118]

[0119] in, This represents the spectral energy corresponding to the i-th singular value. This represents the total energy captured in the first k principal directions. Represents the total energy in all directions. This represents the energy concentration ratio of the first k singular values. It is used to measure how much of the overall representation's energy is explained by the top k principal subspaces. A high ECR indicates that information is concentrated in the lower-dimensional subspace, while a low ECR indicates that energy is distributed in multiple directions.

[0120] Pulse firing rate analysis: such as Figure 5 As shown, the average burst rate of different loss functions (CE, TET, and AWNN) at each network layer is compared on the CIFAR-10, CIFAR-100, and CIFAR10-DVS datasets when using ResNet19 and VGGSNN networks, respectively. It can be observed that compared to CE and TET, the proposed AWNN significantly reduces the burst rate in almost all network layers, and this trend is consistent across all three datasets and network architectures. Specifically, on static image datasets such as CIFAR-10 and CIFAR-100, TET induces a significantly higher burst rate in mid-to-deep networks, easily generating redundant bursts, while AWNN effectively suppresses excessive burst activation in mid-to-high layers while maintaining classification performance, resulting in a sparser and more balanced burst distribution. This advantage is even more pronounced on CIFAR10-DVS: AWNN maintains the lowest average burst rate in all layers, indicating that it can more fully utilize the temporal sparsity of the event data itself and avoid invalid or redundant burst propagation. From an energy consumption perspective, since the energy consumption of spiking neural networks is highly correlated with the number of pulse firings, the AWNN method uses a lower pulse firing rate, resulting in lower computational energy consumption. This indicates that AWNN does not sacrifice performance by simply suppressing neuronal activity, but rather guides the network to focus on key temporal structures with discriminative power through an adaptive weighting mechanism, thereby reducing redundant pulses while maintaining effective information transmission. Therefore, Figure 5 From the perspective of hierarchical pulse firing rate, it is demonstrated that AWNN effectively suppresses deep redundant pulses while maintaining the pulse firing of key feature layers, thereby significantly reducing overall energy consumption without increasing model complexity.

[0121] Energy Consumption Assessment: In Artificial Neural Networks (ANNs), each basic operation involves a multiplication-accumulation (MAC) process. For a given network structure, the total number of MAC operations in an ANN can be directly calculated and remains constant during inference.

[0122] In contrast, spiking neural networks (SNNs) perform accumulation (AC) operations only upon receiving input pulses during computation, without involving multiplication. The number of AC operations in an SNN can be estimated by summing the average pulse firing activity of each network layer and the number of synaptic connections layer by layer. Therefore, the number of synaptic operations (SOP) is used as a metric for the computational complexity of an SNN. The input undergoes feature mapping through a non-pulsating convolutional layer, which does not involve time dimension or pulse generation; its computation is primarily based on MAC operations. Subsequently, the features are fed into intermediate spiking neuron layers, which consist of event-driven AC operations. This metric is widely used in the energy efficiency analysis of spiking neural networks and can effectively reflect the computational overhead in actual hardware implementations.

[0123] Here, that is and This represents the average burst rate of the (n-1)th convolutional layer and the fully connected layer.

[0124] Ablation Experiments: As shown in Table 2, the ablation experiments of the proposed aBCN and AWNN methods, based on ResNet-19 and VGGSNN network structures respectively, were conducted on the CIFAR-10, CIFAR-100, and CIFAR10-DVS datasets. Using the baseline as a reference, on CIFAR-10, introducing AWNN and aBCN alone improved the Top-1 accuracy to 96.01% and 95.67% respectively, while using them together further improved it to 96.38%, showing a significant performance synergy effect. On the more challenging CIFAR-100 dataset, AWNN and aBCN brought accuracy improvements of approximately 4.2% and 4.5% respectively, and the combined strategy further improved the performance to 80.32%, significantly reducing the discrimination error in complex category scenarios. For the event-driven CIFAR10-DVS dataset, both settings also achieved an accuracy of 84.80%, validating the robustness of the proposed method on temporally sparse data. This also demonstrates the good complementarity between the two methods in temporal modeling and feature normalization. In summary, this ablation experiment fully demonstrates the generalization ability of the proposed method under different network structures and data modalities, as well as the key contribution of the synergistic effect of AWNN and aBCN to the overall performance improvement.

[0125] Table 2 shows the ablation studies conducted using the two proposed methods, with each experiment repeated three times. Results are expressed as mean and standard deviation.

[0126]

[0127] Table 2 shows the ablation results of the proposed aBCN and AWNN methods on the CIFAR-10, CIFAR-100, and CIFAR10-DVS datasets, based on ResNet-19 and VGGSNN network structures, respectively. Using the baseline as a reference, on CIFAR-10, introducing AWNN and aBCN alone improves the Top-1 accuracy to 96.01% and 95.67%, respectively, while using them together further improves it to 96.38%, showing a significant performance synergy effect. On the more challenging CIFAR-100 dataset, AWNN and aBCN bring accuracy improvements of approximately 4.2% and 4.5%, respectively, and the combined strategy further improves the performance to 80.32%, significantly reducing the discrimination error in complex category scenarios. For the event-driven CIFAR10-DVS dataset, both settings also achieved an accuracy of 84.80%, validating the robustness of the proposed method on temporally sparse data. This also demonstrates the good complementarity between the two methods in temporal modeling and feature normalization. In summary, this ablation experiment fully demonstrates the generalization ability of the proposed method under different network structures and data modalities, as well as the key contribution of the synergistic effect of AWNN and aBCN to the overall performance improvement.

[0128] Table 3 presents the experimental results of ablation analysis on the weight coefficients λ in AWNN on the CIFAR-10, CIFAR-100, and CIFAR10-DVS datasets. It can be observed that on all datasets, the model performance after introducing AWNN is superior to SDT and TET, indicating that the adaptive weighted nuclear norm has universal effectiveness in improving the discriminative power of temporal representations. Furthermore, AWNN exhibits a certain degree of robustness to the value of λ, maintaining stable performance over a relatively wide parameter range.

[0129] Table 3 shows the ablation studies conducted using the two proposed methods, with each experiment repeated three times. Results are expressed as mean and standard deviation.

[0130]

[0131] As shown in Table 3, on static image datasets like CIFAR-10 and CIFAR-100, the highest accuracy is achieved when λ=0.07. This indicates that in scenarios with high class complexity or stronger semantic discrimination requirements, moderately enhancing the constraint on low-rank temporal structure helps suppress redundant features and improve model performance. In contrast, on the event-driven CIFAR10-DVS dataset, the highest accuracy is achieved when λ=0.01. This suggests that for data with inherently high temporal sparsity and event-driven characteristics, excessively strong kernel norm constraints may suppress effective impulse information, while milder structure regularization is more conducive to preserving key event temporal patterns.

[0132] In summary, this ablation experiment demonstrates that AWNN can adaptively balance structural constraint strength and information retention capability under different data modalities by adjusting λ, thereby achieving stable and controllable performance improvement. This result not only verifies the rationality of the AWNN design but also provides a clear basis for parameter selection for its practical application in different tasks and data types.

[0133] This invention rethinks the normalization method of SNNs and proposes an adaptive batch-channel normalization method to alleviate key problems such as gradient instability, temporal covariate shift, and impulse redundancy during SNN training. Addressing the shortcomings of existing methods that primarily focus on single-dimensional statistics or local time-step supervision while neglecting global temporal-feature structure correlation, this paper proposes an adaptive spatiotemporal backpropagation batch-channel normalization method, aBCN. By adaptively adjusting the contributions of batch and channel dimensions, it effectively stabilizes training dynamics and enhances the global impulse representation capability across samples and time steps. Building upon this, an adaptive weighted nuclear norm training method, AWNN, is further proposed. This method applies structured low-rank constraints to the network's output temporal-feature matrix, guiding the model to retain important core spatiotemporal structures while maintaining impulse sparsity, thereby suppressing redundant temporal noise and reducing energy consumption. The proposed method can be widely applied to existing methods, providing a general and scalable solution for efficient, stable, and structure-aware SNN training, and can further reduce the power consumption of spiking neural networks without increasing computational complexity.

[0134] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A gesture recognition system based on an adaptive batch channel normalized spiking neural network, characterized in that: It is equipped with a data acquisition module, a preprocessing module, and a gesture recognition module connected in sequence; The gesture recognition module is provided with an input layer, a spiking neural network (SNN), a fully connected layer (FC), and an adaptive weighted kernel norm layer (AWNN) connected in sequence. The spiking neural network (SNN) is provided with a convolutional input layer, which is connected to a batch channel normalization network module. The batch channel normalization network module is provided with m network blocks connected end to end. The data acquisition module is used to acquire raw gesture data with time-series attributes; The preprocessing module is used to preprocess the original gesture data to obtain standard gesture data; The gesture recognition module is used to perform gesture recognition operations on the standard gesture data to obtain gesture recognition results.

2. The gesture recognition system based on an adaptive batch channel normalized spiking neural network according to claim 1, characterized in that: The m network blocks have the same structure, each consisting of a first convolutional layer, a first batch of channel normalization layers, a first neuron layer, a second convolutional layer, a second batch of channel normalization layers, a second neuron layer, a pooling layer, and an addition unit connected in sequence. A third batch of channel normalization layers is connected in parallel between the inputs of the first convolutional layer and the addition unit.

3. The gesture recognition system based on an adaptive batch channel normalized spiking neural network according to claim 2, characterized in that: The first and second neuron layers use the LIF iterative model, and the iterative expression is: ； in, A constant representing the decay of membrane potential. This represents the membrane potential at time t. This represents the presynaptic input at time t; ； in, This represents the binary impulse output of neuron j at time t. As weight, Here, b represents the neuron index, and b represents the bias. , Indicates the batch processing axis. Indicates the channel axis. Indicates spatial axes; when At this time, the neuron emits a pulse and resets the membrane potential to 0, that is... Therefore, the LIF iterative model expression in the space and time domains is: ； ； in, Indicates the launch threshold. This represents the membrane potential of the nth layer neuron at time t. This represents the binary pulse output of the nth layer neuron at time t. This represents the presynaptic input of the nth layer neuron at time t. This is the activation function.

4. The gesture recognition system based on an adaptive batch channel normalized spiking neural network according to claim 2, characterized in that: The first batch of channel normalization layers and the second batch of channel normalization layers have the same structure. Both are equipped with batch-time joint dimension processing units and channel-space dimension processing units in parallel. The output of the batch-time joint dimension processing unit and the channel-space dimension processing unit are connected to the input of the same weighted fusion unit.

5. The gesture recognition system based on an adaptive batch channel normalized spiking neural network according to claim 4, characterized in that: The batch-time joint dimension processing unit first calculates the current batch channel normalization layer input signal along the (N×T, H, W) axis. mean and variance The expression is: ； ； Then use the mean. and variance For input signal After normalization, the expression is: ； in, This represents the input signal of the current batch channel normalization layer. , The total time step is represented by i, the batch-time index is represented by m, and the total number of batches-time is represented by m∈N×T; For hyperparameters; A constant set to ensure numerical stability. This represents the normalized output of the batch-time joint dimension processing unit; The channel-space dimension processing unit first calculates the input signal of the normalized layer of the current batch of channels along the (C, H, W) axis. mean and variance The expression is: ； ； Then use the mean. and variance For input signal After normalization, the expression is: ； in, Indicates the total number of channels. Indicates the channel index. This represents the normalized output of the channel-space dimension processing unit.

6. The gesture recognition system based on an adaptive batch channel normalized spiking neural network according to claim 5, characterized in that: The weighted fusion unit uses learning parameters. The normalized outputs along the (N×T, H, W) and (C, H, W) axes are weighted and fused as follows: ； Therefore, the final normalized output expression of the batch channel normalization layer is: ； in, These are the learning parameters.

7. The gesture recognition system based on an adaptive batch channel normalized spiking neural network according to claim 1, characterized in that: The loss of the gesture recognition module includes temporal effective training loss. and adaptive weighted norm loss The expression is: ； ； ； in, Indicates the total time step. For time step index, Indicates the target label, Represents cross-entropy loss, It is the output of time step t. The shape is [N,D], where N is the batch size and D is the output feature dimension; For batch indexing, ; The total number of singular values. This is the singular value index, and the superscript T is the matrix transpose; Indicates global adaptive weights; , ; express The rank of the matrix, ; This represents the time-feature matrix of the k-th sample; By adding the hyperparameter λ to adjust the proportion of the regularization term, the total loss function of the gesture recognition module is thus adjusted. for: 。 8. The gesture recognition system based on an adaptive batch channel normalized spiking neural network according to claim 7, characterized in that: The total loss of the gesture recognition module is relative to the SNN synaptic weight matrix. The gradient expression is: ； ； ； in, The output value of the presynaptic input of the output layer after passing through softmax is the output value. For target label The unique hot code; This represents the input pulse vector of the k-th sample at time step t; ; The input pulse at time step t; Describe the SNN synaptic weight matrix, satisfying ; Therefore, the gradient of the total loss with respect to the weights W is: 。

Images