Neuromorphic image recognition method based on pulse-gated graph enhanced attention network

Abstract

The application discloses a neuromorphic image recognition method based on a pulse-gated graph enhanced attention network, and the method comprises the following steps: an image acquisition module acquires image data with time sequence characteristics; an input layer in the pulse-gated graph enhanced attention network converts the image data into a continuous feature sequence; a neuron layer converts the continuous feature sequence into a binary pulse sequence; a pulse-gated time attention module adaptively adjusts the time importance of the continuous feature sequence and the pulse sequence to obtain time sequence attention; a graph enhanced spatial attention module captures the structural spatial correlation of the continuous feature sequence to obtain a spatial attention graph; an attention fusion module fuses and enhances the time sequence attention, the spatial attention graph and the pulse sequence to obtain an enhanced pulse sequence; and a prediction and recognition module classifies and recognizes the enhanced pulse sequence to obtain an image recognition result. The method can effectively reduce redundant time steps in an SNN without affecting performance or parallelism, and improves image enhancement efficiency.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a neuromorphic image recognition method based on pulse-gated graph-enhanced attention networks. Background Technology

[0002] Existing spiking neural network (SNN) architectures still struggle to effectively capture the inherent spatiotemporal dependencies in dynamic neuromorphic data. Neuromorphic sensors, such as dynamic visual sensors (DVS), generate asynchronous event streams that encode spatial and temporal information, posing a significant challenge to traditional SNNs. To achieve competitive recognition accuracy on benchmark datasets such as CIFAR10-DVS, N-Caltech101, and DVS-Gesture, existing SNN models typically require relatively large temporal simulation windows, such as 10–40 time steps. However, as the time step size decreases, performance degrades significantly due to the lack of temporal integration and limitations in feature encoding. Therefore, designing low-latency, high-accuracy SNNs remains a critical and unsolved problem in neuromorphic computing.

[0003] Recent research has attempted to improve the modeling of spatiotemporal dependencies in SNNs through enhanced neuron models, hybrid recurrent architectures, and optimized temporal training strategies. However, these methods often treat the spatial or temporal domains independently, neglecting their coupling effects. Spatial augmentation methods, such as convolutional SNNs or recurrent SNNs, can extract static texture features but struggle to capture global temporal correlations. Therefore, the challenge lies in achieving spatiotemporal feature learning without sacrificing low latency or computational efficiency. Attention mechanisms—originally introduced in deep learning—have recently been explored in SNNs to enhance selective feature learning. For example, spatiotemporal or channel attention modules have shown the potential to suppress redundant impulses and amplify significant activations. However, existing SNN attention frameworks typically require additional time steps or introduce synchronization constraints, thereby weakening the impulse-driven and asynchronous nature of SNNs.

[0004] Disadvantages of existing technologies: Existing SNN attention frameworks cannot effectively reduce redundant time steps in SNNs without affecting performance or parallelism, resulting in low image enhancement efficiency. Summary of the Invention

[0005] This invention provides a neuromorphic image recognition method based on pulse-gated graph-enhanced attention network, which can effectively reduce redundant time steps in SNN and improve image enhancement efficiency without affecting performance or parallelism.

[0006] To achieve the above objectives, this invention provides a neuromorphic image recognition method based on a pulse-gated graph-enhanced attention network, the key of which is:

[0007] Step 1: Construct a neuromorphic image recognition system based on a pulse-gated graph enhanced attention network. This neuromorphic image recognition system is equipped with an image acquisition module and a pulse-gated graph enhanced attention network.

[0008] The pulse-gated graph-enhanced attention network is configured with an input layer, a neuron layer, a pulse-gated temporal attention module, a graph-enhanced spatial attention module, an attention fusion module, and a prediction and recognition module.

[0009] The output of the input layer is connected to the input of the neuron layer, the pulse-gated temporal attention module, the graph-enhanced spatial attention module, and the prediction and recognition module, respectively. The output of the neuron layer is connected to the input of the pulse-gated temporal attention module and the attention fusion module, respectively. The outputs of the pulse-gated temporal attention module and the graph-enhanced spatial attention module are connected to the input of the attention fusion module. The output of the attention fusion module is connected to the input of the prediction and recognition module.

[0010] Step 2: The image acquisition module acquires neuromorphic image data x with time-series characteristics and transmits it to the pulse-gated graph enhanced attention network;

[0011] Step 3: The input layer in the pulse-gated graph-enhanced attention network acquires the neuromorphic image data x, transforms the neuromorphic image data x into a continuous feature sequence X, and then passes it to the neuron layer, the pulse-gated temporal attention module, and the graph-enhanced spatial attention module;

[0012] Step 4: The neuron layer converts the continuous feature sequence X into a binary pulse sequence S and passes it to the pulse-gated temporal attention module and the attention fusion module;

[0013] Step 5a: The pulse-gated temporal attention module adaptively adjusts the temporal importance of the continuous feature sequence X and the pulse sequence S through pulse-driven gating to obtain temporal attention. And pass it to the attention fusion module;

[0014] Step 5b: The graph-enhanced spatial attention module constructs a dynamic graph topology to capture the structured spatial correlations of the continuous feature sequence X beyond the local receptive field, thereby obtaining a spatial attention graph. And pass it to the attention fusion module;

[0015] Step 6: The attention fusion module integrates the temporal attention... Spatial attention diagram The pulse sequence S is fused and enhanced to obtain the enhanced pulse sequence Y, which is then passed to the prediction and recognition module.

[0016] Step 7: The prediction and recognition module classifies and recognizes the enhanced pulse sequence Y to obtain the image recognition result y.

[0017] Through the above design, this invention integrates a pulse-gated temporal attention module and a graph-enhanced spatial attention module to jointly capture the dynamic spatiotemporal dependencies in neuromorphic object recognition tasks such as image recognition. The pulse-gated temporal attention module introduces a pulse-driven gating mechanism to recalibrate the temporal feature flow while maintaining asynchronous firing dynamics; the graph-enhanced spatial attention module employs graph-based spatial reasoning to enhance the feature interactions between neurons at each time step. This method can effectively reduce redundant time steps in SNNs without affecting performance or parallelism, improving not only image enhancement and recognition efficiency but also image recognition accuracy.

[0018] Preferably, the pulse-gated temporal attention module is provided with a spatiotemporal average pooling layer, a multilayer perceptron (MLP), a gated signal generation unit, a multiplication unit, and a one-dimensional convolutional layer, wherein the output of the one-dimensional convolutional layer has a built-in sigmoid activation function.

[0019] The input of the spatiotemporal average pooling layer is connected to the output of the input layer, and the output of the spatiotemporal average pooling layer is connected to the first input of the multiplication unit via a multilayer perceptron (MLP).

[0020] The input terminal of the gate signal generation unit is connected to the output terminal of the neuron layer, and the output terminal of the gate signal generation unit is connected to the second input terminal of the multiplication unit.

[0021] The output of the multiplication unit is connected to the input of the one-dimensional convolutional layer, and the output of the one-dimensional convolutional layer is connected to the input of the attention fusion module.

[0022] The pulse-gated time attention module dynamically recalibrates the importance in the time dimension by fusing continuous feature information with a pulse-driven gating signal.

[0023] Preferably, in step 5a, the pulse-gated time attention module generates temporal attention. The specific steps are as follows:

[0024] Step A1.1: The spatiotemporal average pooling layer performs spatiotemporal average pooling on the continuous feature sequence X to aggregate spatial information and generate a compact temporal descriptor. , , The set of real numbers is represented and then processed by a multilayer perceptron (MLP) to obtain the perceptual data. And pass it to the multiplication unit;

[0025] Step A1.2: The gate signal generation unit first calculates the average activation of the pulse sequence S in the spatial and channel dimensions to obtain the pulse-based gate signal G. Then, the gating signal G is broadcast along the channel dimension to obtain the broadcast signal. , And pass it to the multiplication unit;

[0026] Step A2: The multiplication unit processes the sensed data. and broadcast signals Element-wise multiplication is performed, followed by one-dimensional depthwise convolution through a one-dimensional convolutional layer, and finally activation by the sigmoid activation function to obtain temporal attention. ;

[0027] The pulse-gated time attention module generates temporal attention. The calculation expression is as follows:

[0028] ;

[0029] ;

[0030] in, This represents the sigmoid activation function; This represents a one-dimensional depthwise convolution operation used to capture local temporal dependencies; This represents a multilayer perceptron, which is used for... Perform dimensionality reduction and dimensionality expansion operations by channel; This represents element-wise multiplication;

[0031] The expression for calculating the pulse-based gated signal G is as follows:

[0032] ;

[0033] in, Indicates batch, Indicates time, Indicates the total number of channels. Indicates the channel index. Indicates the length of the feature space. Indicates the length index. Indicates the width of the feature space. Indicates the width index. Represents a pulse sequence. This indicates a pulse-based gated signal.

[0034] Pulse-based gating signals Used to characterize the average number of activation pulses at each time step.

[0035] The key to the pulse-gated temporal attention module lies in the introduction of a pulse-gated mechanism that bridges the gap between continuous feature dynamics and discrete neural events. By modulating feature-based weights with a global firing rate G, the pulse-gated temporal attention module acts as a filter with activity-aware capabilities: it suppresses time frames where neural activity is negligible (noise) and amplifies moments of high neuronal synchronization (salient events). This allows the network to prioritize information-intensive time steps, thereby improving its robustness in the temporal dimension and aligning the attention mechanism with the sparse, event-driven characteristics of biological visual systems.

[0036] Preferably, the graph-enhanced spatial attention module is provided with a flattening layer, a convolutional projection layer, a graph convolutional network, a pointwise convolutional layer and a raw sequence recovery layer connected in sequence, and the output of the raw sequence recovery layer has a built-in sigmoid activation function.

[0037] The graph convolutional network is configured with a downsampling layer, a graph convolutional module, and an upsampling layer connected in sequence. The graph convolutional module includes L graph convolutional layers connected end to end, and the output of each graph convolutional layer has a built-in ReLU activation function.

[0038] The graph-enhanced spatial attention module models complex spatial relationships between different regions of the feature map by projecting the feature map onto a graph structure and performing graph convolution.

[0039] Preferably, in step 5b, the graph-enhanced spatial attention module generates a spatial attention graph. The specific steps are as follows:

[0040] Step B1: The flattening layer acquires the neuromorphic image data x, and then flattens the neuromorphic image data x in the batch and time dimensions to obtain flattened data. , And pass it to the convolutional projection layer;

[0041] Step B2: The convolutional projection layer uses 1×1 convolution to perform feature projection, obtaining projection data. And pass it to the graph convolutional network;

[0042] Step B3: The downsampling layer in the graph convolutional network will convert the projected data... Sampling is performed in the spatial dimension onto a grid consisting of N nodes. To obtain the node feature set , Then the node feature set Construct as a dynamic normalized adjacency matrix , The expression is:

[0043] ;

[0044] in, Represents the ReLU activation function. The query projection representing node features. The key projection representing the node features. , , To query the projected weights, For key projection weights, The superscript T indicates matrix transpose. This represents the dimension of the key vector used for scaling. This is the identity matrix used to introduce self-loops; express The angle matrix is ​​used for symmetric normalization. Total number of batches The total time step is denoted by ; this form enables the graph structure to adapt to input features, thereby facilitating the modeling of nonlocal spatial relationships.

[0045] Step B4: The graph convolution module processes the normalized adjacency matrix. For multi-layer graph convolution operations, the expression for a single-layer graph convolution operation is:

[0046] ;

[0047] in, Indicates the first The node feature matrix of the layer , ; The trainable weight matrix of the graph convolutional layer. ; This is an optional bias term;

[0048] The output node features of the last convolutional layer After reshaping by the upsampling layer, the data is upsampled back to the original spatial scale to obtain a refined feature map. , Then it is passed to the pointwise convolutional layer;

[0049] Step B5: The pointwise convolutional layer processes the feature map. Pointwise convolution is performed, then the sequence is reshaped back to its original format via a sequence recovery layer, and finally activated by a sigmoid activation function to obtain the spatial attention feature map. The expression is:

[0050] ;

[0051] in, This represents pointwise convolution. This indicates that the output will be generated by Reshape back to the original sequence format ; This represents the sigmoid activation function.

[0052] The spatial attention feature map This graph augmentation method encodes the importance of each spatial location. It offers a fundamental advantage over traditional local convolutional methods. The graph-augmented spatial attention module effectively captures the global structured relationships and non-local dependencies between different regions of the feature map by constructing a dynamic graph based on feature similarity, which is crucial for a comprehensive understanding of visual information. This mechanism significantly enhances the spatial representation capabilities of the SNN front end. It enables the network to selectively focus on spatial information most relevant to the context, overcoming the inherent locality limitations of standard convolution, thus achieving superior feature extraction and more robust performance in complex visual tasks.

[0053] Preferably, in step 6, the attention fusion module first uses a weighted summation method to combine the temporal attention. Spatial attention diagram The sequences are combined, and then a comprehensive spatiotemporal attention map is generated using the sigmoid function. Based on this map, the pulse sequence S is gated to obtain the enhanced pulse sequence Y, expressed as follows:

[0054] ;

[0055] ;

[0056] in, Indicates the weighted parameters. ; for and The combined features, For element-wise multiplication, This is the sigmoid function. , .

[0057] This integration strategy ensures that the network can selectively amplify significant impulse signals in both the temporal and spatial domains, thereby achieving more efficient information propagation and feature representation within the SNN. Gating the original impulses guarantees that the output retains a binary or rate-coded impulse form, thus maintaining compatibility with subsequent SNN layers.

[0058] Preferably, the image recognition result y obtained by the prediction and recognition module is a probability distribution value vector corresponding to various image category labels, and the image category label corresponding to the maximum probability distribution value is output as the final result.

[0059] Preferably, the neuromorphic image data x is either gesture image data, animal image data, or vehicle image data.

[0060] The neuromorphic image data x is obtained by capturing images using a dynamic vision sensor, namely a DVS camera.

[0061] The data includes hand gesture images such as rock, paper, and scissors; animal image data includes images of animals such as rabbits, cats, tigers, rhinos, and elephants; and vehicle image data includes images of vehicles such as airplanes, cars, ships, and motorcycles.

[0062] In practical applications, the pulse-gated graph enhanced attention network can be trained by selecting appropriate image data according to specific recognition needs, thereby achieving accurate recognition of the corresponding image categories.

[0063] The beneficial effects of this invention are:

[0064] (1) A pulse-gated graph-enhanced attention network framework is proposed to jointly capture the dynamic spatiotemporal dependencies of neuromorphic object recognition. By integrating pulse-driven temporal modulation and graph-based spatial reasoning in the input stage, this invention enhances feature representation while retaining the asynchronous and energy-efficient characteristics of SNN.

[0065] (2) A pulse-gated temporal attention module was designed to recalibrate the temporal feature flow through pulse-based gating; and a graph-enhanced spatial attention module was designed to model nonlocal spatial interactions through learnable graph topology. These two modules together suppress redundant background activity and highlight significant neuromorphic events, thereby expanding the effective receptive field of the model.

[0066] (3) The present invention can effectively reduce redundant time steps in SNN without affecting performance or parallelism, which not only improves image enhancement and recognition efficiency, but also improves image recognition accuracy. Attached Figure Description

[0067] Figure 1 This is a flowchart of the method of the present invention;

[0068] Figure 2 This is a block diagram of the pulse-gated graph-enhanced attention network structure in the embodiment;

[0069] Figure 3 The examples illustrate the visualization of neural system data event streams and attention maps based on impulse firing rate.

[0070] Figure 4 This is a visualization of the effective receptive field (ERF) in the embodiment. Detailed Implementation

[0071] The present invention will be further described in detail below with reference to the accompanying drawings and specific examples. The following embodiments or drawings are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0072] like Figure 1 As shown, a neuromorphic image recognition method based on pulse-gated graph-enhanced attention network includes the following steps:

[0073] Step 1: Construct a neuromorphic image recognition system based on a pulse-gated graph enhanced attention network. This neuromorphic image recognition system is equipped with an image acquisition module and a pulse-gated graph enhanced attention network.

[0074] like Figure 2 As shown, the pulse-gated graph-enhanced attention network is configured with an input layer, a neuron layer, a pulse-gated temporal attention module, a graph-enhanced spatial attention module, an attention fusion module, and a prediction and recognition module;

[0075] The output of the input layer is connected to the input of the neuron layer, the pulse-gated temporal attention module, the graph-enhanced spatial attention module, and the prediction and recognition module, respectively. The output of the neuron layer is connected to the input of the pulse-gated temporal attention module and the attention fusion module, respectively. The outputs of the pulse-gated temporal attention module and the graph-enhanced spatial attention module are connected to the input of the attention fusion module. The output of the attention fusion module is connected to the input of the prediction and recognition module.

[0076] The pulse-gated temporal attention module is equipped with a spatiotemporal average pooling layer, a multilayer perceptron (MLP), a gated signal generation unit, a multiplication unit, and a one-dimensional convolutional layer. The output of the one-dimensional convolutional layer has a built-in sigmoid activation function.

[0077] The input of the spatiotemporal average pooling layer is connected to the output of the input layer, and the output of the spatiotemporal average pooling layer is connected to the first input of the multiplication unit via a multilayer perceptron (MLP).

[0078] The input terminal of the gate signal generation unit is connected to the output terminal of the neuron layer, and the output terminal of the gate signal generation unit is connected to the second input terminal of the multiplication unit.

[0079] The output of the multiplication unit is connected to the input of the one-dimensional convolutional layer, and the output of the one-dimensional convolutional layer is connected to the input of the attention fusion module.

[0080] The graph-enhanced spatial attention module is configured with a flattening layer, a convolutional projection layer, a graph convolutional network, a pointwise convolutional layer, and a raw sequence recovery layer connected in sequence. The output of the raw sequence recovery layer has a built-in sigmoid activation function.

[0081] The graph convolutional network is configured with a downsampling layer, a graph convolutional module, and an upsampling layer connected in sequence. The graph convolutional module includes L graph convolutional layers connected end to end, and the output of each graph convolutional layer has a built-in ReLU activation function.

[0082] Step 2: The image acquisition module acquires neuromorphic image data x with time-series characteristics and transmits it to the pulse-gated graph enhanced attention network;

[0083] Step 3: The input layer in the pulse-gated graph-enhanced attention network acquires the neuromorphic image data x, transforms the neuromorphic image data x into a continuous feature sequence X, and then passes it to the neuron layer, the pulse-gated temporal attention module, and the graph-enhanced spatial attention module;

[0084] Step 4: The neuron layer converts the continuous feature sequence X into a binary pulse sequence S and passes it to the pulse-gated temporal attention module and the attention fusion module;

[0085] Step 5a: The pulse-gated temporal attention module adaptively adjusts the temporal importance of the continuous feature sequence X and the pulse sequence S through pulse-driven gating to obtain temporal attention. And pass it to the attention fusion module;

[0086] Step 5b: The graph-enhanced spatial attention module constructs a dynamic graph topology to capture the structured spatial correlations of the continuous feature sequence X beyond the local receptive field, thereby obtaining a spatial attention graph. And pass it to the attention fusion module;

[0087] Step 6: The attention fusion module integrates the temporal attention... Spatial attention diagram The pulse sequence S is fused and enhanced to obtain the enhanced pulse sequence Y, which is then passed to the prediction and recognition module.

[0088] Step 7: The prediction and recognition module classifies and recognizes the enhanced pulse sequence Y to obtain the image recognition result y.

[0089] The Pulse Gated Graph Enhanced Attention Network framework enhances spatiotemporal feature extraction in spiking neural networks by combining bio-inspired pulsation dynamics with data-driven attention mechanisms. Its goal is to simultaneously model temporal dependencies and nonlocal spatial relationships, thereby alleviating the limited representational capabilities of traditional pulsation-based models.

[0090] The pulse-gated graph-enhanced attention network integrates two complementary attention modules, such as... Figure 2 As shown, the pulse-gated temporal attention module adaptively adjusts temporal importance through pulse-driven gating. In contrast, the graph-enhanced spatial attention module constructs dynamic graph topologies to capture structured spatial correlations beyond local receptive fields. The attention mechanism optimizes neuronal communication by selectively adjusting synaptic weights, thereby improving the detection of salient information in noisy environments. To improve representation efficiency without increasing computational cost, the pulse-gated graph-enhanced attention framework models temporal modulation and spatiotemporal coupling based on graph reasoning only at the SNN input layer. The generated attention-enhanced pulse signals are then fed into the backbone network, i.e., the prediction and recognition module, based on a CNN or Transformer architecture. This approach improves the stability of SNNs, reduces redundancy, and preserves biological plausibility, thus providing a scalable mechanism for advanced visual understanding in neuromorphic computing.

[0091] In step 5a, the pulse-gated temporal attention module generates temporal attention. The specific steps are as follows:

[0092] Step A1.1: The spatiotemporal average pooling layer performs spatiotemporal average pooling on the continuous feature sequence X to aggregate spatial information and generate a compact temporal descriptor. Then, the data is processed by a multilayer perceptron (MLP) to obtain the sensing data. And pass it to the multiplication unit;

[0093] Step A1.2: The gating signal generation unit first calculates the average activation of the pulse sequence S in the spatial and channel dimensions to obtain a pulse-based gating signal G, and then broadcasts the gating signal G along the channel dimension to obtain a broadcast signal. And pass it to the multiplication unit;

[0094] Step A2: The multiplication unit processes the sensed data. and broadcast signals Element-wise multiplication is performed, followed by one-dimensional depthwise convolution through a one-dimensional convolutional layer, and finally activation by the sigmoid activation function to obtain temporal attention. ;

[0095] The pulse-gated time attention module generates temporal attention. The calculation expression is as follows:

[0096] ;

[0097] ;

[0098] in, This represents the sigmoid activation function. This represents a one-dimensional depthwise convolution operation used to capture local temporal dependencies; This represents a multilayer perceptron. This represents element-wise multiplication;

[0099] The expression for calculating the pulse-based gated signal G is as follows:

[0100] ;

[0101] in, Indicates batch, Indicates time, Indicates the total number of channels. Indicates the channel index. Indicates the length of the feature space. Indicates the length index. Indicates the width of the feature space. Indicates the width index. Represents a pulse sequence. This indicates a pulse-based gated signal.

[0102] In step 5b, the graph-enhanced spatial attention module generates a spatial attention graph. The specific steps are as follows:

[0103] Step B1: The flattening layer acquires the neuromorphic image data x, and then flattens the neuromorphic image data x in the batch and time dimensions to obtain flattened data. And pass it to the convolutional projection layer;

[0104] Step B2: The convolutional projection layer uses 1×1 convolution to perform feature projection, obtaining projection data. And pass it to the graph convolutional network;

[0105] Step B3: The downsampling layer in the graph convolutional network will convert the projected data... Sampling is performed in the spatial dimension onto a grid consisting of N nodes. To obtain the node feature set , Then the node feature set Construct as a dynamic normalized adjacency matrix The expression is:

[0106] ;

[0107] in, Represents the ReLU activation function. The query projection representing node features. The key projection represents the node feature, and the superscript T indicates matrix transpose. This represents the dimension of the key vector used for scaling. This is the identity matrix used to introduce self-loops; express The angle matrix is ​​used for symmetric normalization. Total number of batches The total time step is denoted by ; this form enables the graph structure to adapt to input features, thereby facilitating the modeling of nonlocal spatial relationships.

[0108] Step B4: The graph convolution module processes the normalized adjacency matrix. For multi-layer graph convolution operations, the expression for a single-layer graph convolution operation is:

[0109] ;

[0110] in, Indicates the first The node feature matrix of the layer , ; The trainable weight matrix of the graph convolutional layer. ; This is an optional bias term;

[0111] The output node features of the last convolutional layer After reshaping by the upsampling layer, the data is upsampled back to the original spatial scale to obtain a refined feature map. , Then it is passed to the pointwise convolutional layer;

[0112] Step B5: The pointwise convolutional layer processes the feature map. Pointwise convolution is performed, then the sequence is reshaped back to its original format via a sequence recovery layer, and finally activated by a sigmoid activation function to obtain the spatial attention feature map. The expression is:

[0113] ;

[0114] in, This represents pointwise convolution. This indicates that the output will be generated by Reshape back to the original sequence format ; This represents the sigmoid activation function.

[0115] In step 6, the attention fusion module first uses a weighted summation method to combine the temporal attention. Spatial attention diagram The sequences are combined, and then a comprehensive spatiotemporal attention map is generated using the sigmoid function. Based on this map, the pulse sequence S is gated to obtain the enhanced pulse sequence Y, expressed as follows:

[0116] ;

[0117] ;

[0118] in, Indicates the weighted parameters. for and The combined features, For element-wise multiplication, This is the sigmoid function.

[0119] The image recognition result y obtained by the prediction and recognition module is a probability distribution value vector corresponding to various image category labels, and the image category label corresponding to the maximum probability distribution value is output as the final result.

[0120] In addition to image recognition tasks, this invention can also be applied to other neuromorphic object recognition tasks.

[0121] The performance of the present invention will be verified through specific experiments.

[0122] (1) Datasets and Experimental Setup: The proposed pulse-gated graph-enhanced attention network was evaluated on three benchmark datasets for neuromorphic object recognition. First, the event-driven image classification dataset CIFAR10-DVS was used, created by scanning each sample in the static CIFAR10 dataset using a dynamic visual sensor (DVS). Next, the N-Caltech101 dataset was used, created by moving Caltech101 samples in front of a DVS camera to capture corresponding event streams. Finally, the DVS-Gesture dataset was used, created by recording event streams of 11 gestures and actions using a DVS camera under different lighting conditions. To accommodate the characteristics and complexity of various neuromorphic datasets, different SNN backbone networks were selected for each dataset. For the neuromorphic datasets CIFAR10-DVS and N-Caltech101, VGGSNN was used as the backbone network. For the DVS-Gesture dataset, QKFormer was used as the backbone network. During training, the experiment used the hyperparameters listed in Table 1, including learning rate, total number of training rounds, spiking neuron threshold, and leakage rate.

[0123] Table 1. Hyperparameters used in the experiment

[0124]

[0125] (2) Comparative experiment: The results of the comparative experiment between the present invention and the existing method are shown in Table 2:

[0126] Table 2. Experimental comparison of neuromorphic datasets CIFAR10-DVS, N-Caltech101, and DVS-Gestrue

[0127]

[0128] As shown in Table 2, the proposed pulse-gated graph-enhanced attention network achieves state-of-the-art performance on three challenging neuromorphic benchmarks. On the CIFAR10-DVS dataset, the accuracy of this invention reaches 87.7% at T=10, a 2.8% improvement over the previous best result of 84.9%, and outperforms Transformer-based SNNs, such as STSA at 79.9%. Similarly, for the N-Caltech101 dataset, the invention achieves a novel state-of-the-art accuracy of 89.2% at T=8, 4.6% higher than the suboptimal SMA model at T=14 (84.6%). This performance improvement is particularly significant considering the sparsity and event-driven nature of these datasets. Furthermore, on the DVS-Gesture image recognition benchmark, the invention achieves 99.2% accuracy at T=16, demonstrating the good generalization ability of the attention mechanism even in high-speed spatiotemporal tasks.

[0129] The main contribution of this invention lies in its robustness under low-latency constraints, meeting the requirements of energy-efficient neuromorphic computing. Unlike previous methods that relied on large time windows, such as T=16 or 10, to accumulate spatiotemporal information, this invention maintains high accuracy while significantly reducing the time step. For example, on the CIFAR10-DVS dataset, this invention achieves an accuracy of 87.1% at T=6, surpassing the best result of 84.9% achieved by other methods at T=10. On the N-Caltech101 dataset, this invention achieves 87.7% accuracy at T=4, outperforming the best result of other methods, such as SMA at 84.6%, at T=14, further highlighting its efficiency. This consistent accuracy validates that the unified pulse-gated temporal attention module and graph-enhanced spatial attention module can effectively model spatiotemporal dynamics with minimal integration overhead, thereby achieving energy-efficient, low-latency neuromorphic object recognition.

[0130] (3) Ablation Experiments: To quantify the contributions of the proposed pulse-gated temporal attention module and graph-enhanced spatial attention module, ablation experiments were conducted on the CIFAR10-DVS dataset. Table 3 compares four variants: a baseline SNN without attention, Method 1 containing only the pulse-gated temporal attention module, Method 2 containing only the graph-enhanced spatial attention module, and the complete pulse-gated graph-enhanced attention framework. Introducing only the pulse-gated temporal attention module improved the test accuracy from 85.7% to 87.4%, while the number of parameters and FLOPs remained essentially unchanged. This indicates that adaptive pulse-gated temporal modulation can effectively enhance temporal integration without adding additional computational burden. Introducing only the graph-enhanced spatial attention module achieved a similar performance improvement of 87.4%, with a slight increase in the number of floating-point operations from 13.7 G to 14.0 G. Combining the pulse-gated temporal attention module and the graph-enhanced spatial attention module in the pulse-gated graph-enhanced attention framework further improved the accuracy to 87.7%, confirming that temporal attention and spatial attention are complementary and can jointly improve recognition performance with negligible overhead.

[0131] Table 3 Ablation Experiment

[0132]

[0133] Figure 3 Attention maps obtained from the impulse firing rate of cat samples in the CIFAR10-DVS dataset are presented. The baseline SNN exhibits scattered and noisy activations, with weak responses in the red box region, indicating insufficient localization of discriminative structures such as the head and torso contours. In contrast, the proposed method generates denser and more continuous activations along object boundaries while suppressing background responses. These observations suggest that the impulse-gated temporal attention module facilitates the consistent temporal propagation of salient impulses, while the graph-enhanced spatial attention module further aggregates them onto graph-connected pixels, resulting in a more object-aware attention map. The qualitative results are consistent with the quantitative results reported in Table 3, indicating that the proposed module guides the network toward semantically meaningful neuromorphic events.

[0134] (4) Power Consumption Comparison Analysis: This invention estimates the power consumption of a single image inference under a 45 nm CMOS process model, where the power consumption of the 32-bit multiply-accumulate (MAC) operation is 4.6 pJ, and the power consumption of the pulse accumulation (AC) operation is 0.9 pJ. For each model, the total number of MAC and AC operations is counted, and the power consumption is calculated as E = MAC × 4.6 pJ + AC × 0.9 pJ. All data are from the CIFAR10-DVS dataset, and the image input size is 48 × 48 pixels to ensure fair comparison.

[0135] Table 4 Power Consumption Comparison

[0136]

[0137] As shown in Table 4, the ANN baseline model performs 1.4 G MACs of computation, consuming 6.3 mJ of energy per inference. In contrast, this invention reduces computational cost by replacing intensive multiplication operations with pulse-driven accumulation operations. With a 10-time-step configuration, this invention requires only 0.4 G MACs and 0.9 G ACs of computation, reducing power consumption to 2.6 mJ. Notably, reducing the number of time steps results in a linear decrease in computational load: at 8 time steps, power consumption drops to 2.1 mJ; at 6 time steps, power consumption drops to 1.60 mJ, representing a 74.6% reduction compared to the ANN baseline model. In conclusion, the energy analysis demonstrates that this invention fully leverages the inherent efficiency of AC-dominated SNN computation while maintaining scalability across different time-step budgets, further proving its suitability for neuromorphic edge deployments.

[0138] (5) Receptive Field Analysis: In neuromorphic object recognition, integrating contextual information through a larger receptive field remains a challenge. The Effective Receptive Field (ERF) was used as a visualization tool to analyze the performance of the proposed impulse-gated graph-enhanced attention framework. The ERF visualization is shown below. Figure 4 As shown, the images were generated using 500 images randomly selected from the CIFAR10-DVS dataset. This invention exhibits a larger ERF during the network phases, and in the final phase, the ERF of the pulse-gated graph-enhanced attention framework almost covers the entire input image, demonstrating its strong global context modeling capabilities and providing a reasonable explanation for the observed performance improvement.

[0139] Conclusion: This invention proposes a novel architecture—the pulse-gated graph-enhanced attention framework—aimed at addressing the limitations of existing spiking neural networks in modeling complex spatiotemporal dependencies. This framework combines a pulse-gated temporal attention module for adaptive temporal recalibration with a graph-enhanced spatial attention module for nonlocal spatial reasoning, effectively suppressing redundant background noise and enhancing significant neuromorphic events without affecting asynchronous firing dynamics. Extensive evaluations on benchmark datasets including CIFAR10-DVS, N-Caltech101, and DVS-Gesture demonstrate that the pulse-gated graph-enhanced attention framework achieves competitive performance. Furthermore, this invention exhibits superior robustness in low-latency settings, maintaining excellent performance with only four time steps, and reduces power consumption by 74.6% compared to traditional artificial neural networks. These findings confirm that the pulse-gated graph-enhanced attention framework is a scalable and efficient approach, providing an effective solution for advancing real-time, event-driven neuromorphic intelligent systems.

[0140] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A neuromorphic image recognition method based on pulse-gated graph-enhanced attention network, characterized in that, Includes the following steps: Step 1: Construct a neuromorphic image recognition system based on a pulse-gated graph enhanced attention network. This neuromorphic image recognition system is equipped with an image acquisition module and a pulse-gated graph enhanced attention network. The pulse-gated graph-enhanced attention network is configured with an input layer, a neuron layer, a pulse-gated temporal attention module, a graph-enhanced spatial attention module, an attention fusion module, and a prediction and recognition module. The output of the input layer is connected to the input of the neuron layer, the pulse-gated temporal attention module, the graph-enhanced spatial attention module, and the prediction and recognition module, respectively. The output of the neuron layer is connected to the input of the pulse-gated temporal attention module and the attention fusion module, respectively. The outputs of the pulse-gated temporal attention module and the graph-enhanced spatial attention module are connected to the input of the attention fusion module. The output of the attention fusion module is connected to the input of the prediction and recognition module. Step 2: The image acquisition module acquires neuromorphic image data x with time-series characteristics and transmits it to the pulse-gated graph enhanced attention network; Step 3: The input layer in the pulse-gated graph-enhanced attention network acquires the neuromorphic image data x, transforms the neuromorphic image data x into a continuous feature sequence X, and then passes it to the neuron layer, the pulse-gated temporal attention module, and the graph-enhanced spatial attention module; Step 4: The neuron layer converts the continuous feature sequence X into a binary pulse sequence S and passes it to the pulse-gated temporal attention module and the attention fusion module; Step 5a: The pulse-gated temporal attention module adaptively adjusts the temporal importance of the continuous feature sequence X and the pulse sequence S through pulse-driven gating to obtain temporal attention. And pass it to the attention fusion module; Step 5b: The graph-enhanced spatial attention module constructs a dynamic graph topology to capture the structured spatial correlations of the continuous feature sequence X, thereby obtaining a spatial attention graph. And pass it to the attention fusion module; Step 6: The attention fusion module integrates the temporal attention... Spatial attention diagram The pulse sequence S is fused and enhanced to obtain the enhanced pulse sequence Y, which is then passed to the prediction and recognition module. Step 7: The prediction and recognition module classifies and recognizes the enhanced pulse sequence Y to obtain the image recognition result y.

2. The neuromorphic image recognition method based on pulse-gated graph-enhanced attention network according to claim 1, characterized in that: The pulse-gated temporal attention module is equipped with a spatiotemporal average pooling layer, a multilayer perceptron (MLP), a gated signal generation unit, a multiplication unit, and a one-dimensional convolutional layer. The output of the one-dimensional convolutional layer has a built-in sigmoid activation function. The input of the spatiotemporal average pooling layer is connected to the output of the input layer, and the output of the spatiotemporal average pooling layer is connected to the first input of the multiplication unit via a multilayer perceptron (MLP). The input terminal of the gate signal generation unit is connected to the output terminal of the neuron layer, and the output terminal of the gate signal generation unit is connected to the second input terminal of the multiplication unit. The output of the multiplication unit is connected to the input of the one-dimensional convolutional layer, and the output of the one-dimensional convolutional layer is connected to the input of the attention fusion module.

3. The neuromorphic image recognition method based on pulse-gated graph-enhanced attention network according to claim 2, characterized in that: In step 5a, the pulse-gated temporal attention module generates temporal attention. The specific steps are as follows: Step A1.1: The spatiotemporal average pooling layer performs spatiotemporal average pooling on the continuous feature sequence X to aggregate spatial information and generate a temporal descriptor. Then, the data is processed by a multilayer perceptron (MLP) to obtain the sensing data. And pass it to the multiplication unit; Step A1.2: The gating signal generation unit first calculates the average activation of the pulse sequence S in the spatial and channel dimensions to obtain a pulse-based gating signal G, and then broadcasts the gating signal G along the channel dimension to obtain a broadcast signal. And pass it to the multiplication unit; Step A2: The multiplication unit processes the sensed data. and broadcast signals Element-wise multiplication is performed, followed by one-dimensional depthwise convolution through a one-dimensional convolutional layer, and finally activation by the sigmoid activation function to obtain temporal attention. ; The pulse-gated time attention module generates temporal attention. The calculation expression is as follows: ； ； in, This represents the sigmoid activation function. This represents a one-dimensional depthwise convolution operation. This represents a multilayer perceptron. This represents element-wise multiplication; The expression for calculating the pulse-based gated signal G is as follows: ； in, Indicates batch, Indicates time, Indicates the total number of channels. Indicates the channel index. Indicates the length of the feature space. Indicates the length index. Indicates the width of the feature space. Indicates the width index. Represents a pulse sequence. This indicates a pulse-based gated signal.

4. The neuromorphic image recognition method based on pulse-gated graph-enhanced attention network according to claim 1, characterized in that: The graph-enhanced spatial attention module is configured with a flattening layer, a convolutional projection layer, a graph convolutional network, a pointwise convolutional layer, and a raw sequence recovery layer connected in sequence. The output of the raw sequence recovery layer has a built-in sigmoid activation function. The graph convolutional network is configured with a downsampling layer, a graph convolutional module, and an upsampling layer connected in sequence. The graph convolutional module includes L graph convolutional layers connected end to end, and the output of each graph convolutional layer has a built-in ReLU activation function.

5. The neuromorphic image recognition method based on pulse-gated graph-enhanced attention network according to claim 4, characterized in that: In step 5b, the graph-enhanced spatial attention module generates a spatial attention graph. The specific steps are as follows: Step B1: The flattening layer acquires the neuromorphic image data x, and then flattens the neuromorphic image data x in the batch and time dimensions to obtain flattened data. And pass it to the convolutional projection layer; Step B2: The convolutional projection layer uses 1×1 convolution to perform feature projection, obtaining projection data. And pass it to the graph convolutional network; Step B3: The downsampling layer in the graph convolutional network will convert the projected data... Sampling is performed in the spatial dimension onto a grid consisting of N nodes. To obtain the node feature set , Then the node feature set Construct as a dynamic normalized adjacency matrix The expression is: ； in, Represents the ReLU activation function. The query projection representing node features. The key projection represents the node feature, and the superscript T indicates matrix transpose. This represents the dimension of the key vector used for scaling. This is the identity matrix used to introduce self-loops; express The angle matrix, Total number of batches This represents the total time step. Step B4: The graph convolution module processes the normalized adjacency matrix. For multi-layer graph convolution operations, the expression for a single-layer graph convolution operation is: ； in, Indicates the first The node feature matrix of the layer , ; Here is the weight matrix of the graph convolutional layer. ; For bias; The output node features of the last convolutional layer After reshaping by the upsampling layer, the feature map is upsampled to the original spatial scale to obtain the feature map. , Then it is passed to the pointwise convolutional layer; Step B5: The pointwise convolutional layer processes the feature map. Pointwise convolution is performed, then the sequence is reshaped back to its original format via a sequence recovery layer, and finally activated by a sigmoid activation function to obtain the spatial attention feature map. The expression is: ； in, This represents pointwise convolution. This indicates that the output will be generated by Reshape back to the original sequence format ; This represents the sigmoid activation function.

6. The neuromorphic image recognition method based on pulse-gated graph-enhanced attention network according to claim 4, characterized in that: In step 6, the attention fusion module first uses a weighted summation method to combine the temporal attention. Spatial attention diagram The sequences are combined, and then a spatiotemporal attention map is generated using the sigmoid function. Based on this map, the pulse sequence S is gated to obtain the enhanced pulse sequence Y, expressed as follows: ； ； in, Indicates the weighted parameters. for and The combined features, For element-wise multiplication, This is the sigmoid function.

7. The neuromorphic image recognition method based on pulse-gated graph-enhanced attention network according to claim 1, characterized in that: The image recognition result y obtained by the prediction and recognition module is a probability distribution value vector corresponding to various image category labels, and the image category label corresponding to the maximum probability distribution value is output as the final result.

8. The neuromorphic image recognition method based on pulse-gated graph-enhanced attention network according to claim 1, characterized in that: The neuromorphic image data x may be gesture image data, animal image data, or vehicle image data.

Images