Dynamic object detection method and device based on spiking neural network model, equipment and medium

Abstract

The invention provides a dynamic object detection method and device based on a pulse neural network model, equipment and a medium, and the method comprises the following steps: 1, capturing an object of a target scene through a dynamic visual sensor, constructing a neuromorphic data set, and determining the neuromorphic data of the target scene based on the neuromorphic data set; according to the method, parameterized leakage integration distribution neurons are introduced into a spiking neural network model, synaptic and membrane related parameters are synchronously optimized in a training process, membrane time constant differences of spiking neurons in different brain regions are fully considered, neuron heterogeneity is enhanced, and network expressive force is improved; meanwhile, by means of a multi-scale attention feature fusion module, time, channel and space attention mechanisms are organically combined in a layered aggregation mode, multi-dimensional and multi-scale features are aggregated, the limitation that attention features are processed separately in the prior art is changed, the model focuses on key information of a dynamic object better, and high-precision real-time detection of the dynamic object in a complex environment is achieved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a method, apparatus, device, and medium for dynamic object detection based on a spiking neural network model. Background Technology

[0002] In today's digital age, dynamic object detection plays a crucial role in many fields. For example, autonomous driving systems need to accurately detect vehicles, pedestrians, and obstacles on the road in real time to ensure driving safety; intelligent monitoring systems need to quickly identify abnormal moving targets to achieve efficient security early warning. While traditional object detection algorithms based on artificial neural networks (ANNs) have achieved some success, they have significant drawbacks. Their computational complexity is extremely high, requiring enormous computing resources and storage space when processing large amounts of image data, resulting in excessive system power consumption and making it difficult to meet the practical application requirements of power-sensitive scenarios such as mobile devices and drones. Furthermore, in complex environments, facing drastic changes in lighting, rapid object movement, and occlusion, the detection accuracy and real-time performance of traditional algorithms are unsatisfactory. Spiking neural networks, as a third-generation neural network model, use binary pulse signals for inter-neuron communication, highly simulating the function of biological neurons. They not only integrate neuronal and synaptic states but also fully consider the importance of pulse timing. Due to their unique advantages such as time information processing capabilities, low power consumption, and high biological reliability, they have attracted widespread attention in academia and industry in recent years. However, in real-world pattern recognition tasks, directly training large-scale spiking neural networks to achieve performance comparable to corresponding artificial neural networks still presents certain challenges: I. Most existing spiking neural network learning optimization algorithms only focus on learning synapse-related parameters (such as synapse weights), treat membrane-related parameters as hyperparameters, and ignore the differences in membrane time constants of spiking neurons in different brain regions. This limits the heterogeneity of neurons to some extent and affects the expressiveness of spiking neural networks. Second, in terms of introducing attention mechanisms into spiking neural networks, existing methods mostly process temporal, spatial, and channel attention features separately, failing to fully aggregate multi-dimensional and multi-scale attention features, which limits the improvement of detection performance of spiking neural networks; To this end, a method, device, equipment, and medium for dynamic object detection based on a spiking neural network model are proposed. Summary of the Invention

[0003] In view of this, the embodiments of the present invention aim to provide a dynamic object detection method, apparatus, device and medium based on a spiking neural network model, so as to solve or alleviate the technical problems existing in the prior art, and at least provide a beneficial option.

[0004] To solve the above-mentioned technical problems, this application adopts a technical solution as follows: a dynamic object detection method based on a spiking neural network model, comprising the following steps: Step 1: Capture objects in the target scene using a dynamic visual sensor, construct a neuromorphic dataset, and determine the neuromorphic data of the target scene based on the neuromorphic dataset; Step 2: Introduce parameterized leaky integrator neurons into the spiking neural network model, and jointly optimize the temporal, channel, and spatial attention mechanisms through the attention feature aggregation module to construct an object detection model; the joint optimization includes dynamic weight allocation and cross-scale feature interaction; Step 3: Train the object detection model based on the neuromorphic dataset. During the training process, an adaptive learning rate optimization algorithm is used to simultaneously optimize synapse-related parameters and membrane-related parameters. The object detection model is trained using a multi-task loss function, which includes classification loss, bounding box regression loss, and impulse temporal consistency loss. Step 4: Encode the neuromorphic data into a pulse sequence through a pulse coding layer, extract multi-dimensional object attention features through an attention layer, input them into a classification layer for classification, and obtain the object detection results.

[0005] As a further preferred embodiment of this technical solution, in step four, the pulse coding layer employs voxel cube coding, specifically as follows: Neuromorphic data is divided into time slices according to the time dimension. The duration of each time slice is adaptively adjusted according to the speed of the moving object. Within each time slice, spatial blocks are formed at 16×16 or 32×32 pixels. Event data within each block is converted into pulse sequences through event counting encoding.

[0006] As a further preferred embodiment of this technical solution, in step four, the method for extracting multi-dimensional object attention features includes the following steps: Step 501: Extract features from the pulse sequence based on the time attention mechanism, and obtain the time attention features by stabilizing the membrane synaptic input distribution through exponential moving average; Step 502: Fuse the local channel information and global channel information of the time attention features to generate multi-scale channel features based on the channel attention mechanism; Step 503: Combining the spatial attention mechanism with the multi-scale channel features, generate object attention features through dynamic weight allocation; The fusion of temporal attention features, multi-scale channel features, and spatial attention features employs a membrane potential addition mechanism, with the following formula: ; Wherein, α, β, and γ are adaptively adjusted weight coefficients, corresponding to the contributions of temporal, spatial, and channel features, respectively.

[0007] As a further preferred embodiment of this technical solution, in step three, the expression for the multi-task loss function is: ; Where α=0.1, β=0.5, Cross-entropy classification loss, The loss is SmoothL1 bounding box regression. This is due to the loss of pulse timing consistency. The pulse timing consistency loss is calculated using the mean square error to determine the difference between the actual pulse sequence and the predicted pulse sequence. The formula is as follows: ; in, This represents the actual impulse value at the time dimension t, channel dimension c, spatial height h, and spatial width w in the neuromorphic data. The value is the pulse value predicted by the object detection model at the corresponding location, T is the total number of time slices, C is the number of feature channels, and H and W are the height and width of the feature map, respectively.

[0008] As a further preferred embodiment of this technical solution, in step two, the membrane time constant of the parameterized leakage integrated firing neuron is set as a learnable parameter, with a value range of 3-50ms, and is dynamically adjusted with the input pulse frequency to adapt to the heterogeneity requirements of neurons in different dynamic scenarios. The attention feature fusion module performs multi-dimensional feature fusion through a hierarchical dynamic aggregation strategy. The bottom layer features use a 10ms time window to focus on extracting the temporal dynamic information of the body's motion. The middle layer features enhance the semantic distinguishability of the channels through channel weighting. The high layer features combine spatial masks to optimize spatial positioning accuracy.

[0009] As a further preferred embodiment of this technical solution, in step four, the classification layer is post-processed using the softNMS algorithm, and the parameter σ of the softNMS algorithm is dynamically adjusted according to the scenario, with σ set to 0.5-0.7.

[0010] As a further preferred embodiment of this technical solution, in step one, the neuromorphic dataset is generated by data augmentation of the raw event stream collected by a dynamic visual sensor; the neuromorphic dataset includes a dataset collected from the scene, a transformed dataset, and a generated dataset.

[0011] To solve the above-mentioned technical problems, another technical solution adopted in this application is: a dynamic object detection device based on a spiking neural network model, comprising: a data acquisition module, a model construction module, a model training module, and a training detection module; The data acquisition module is configured to capture objects in the target scene through a dynamic visual sensor, construct a neuromorphic dataset, and determine the neuromorphic data of the target scene based on the neuromorphic dataset. The model building module is configured to introduce parameterized leaky integrator firing neurons into the spiking neural network model, and jointly optimize the temporal, channel, and spatial attention mechanisms through the attention feature aggregation module to build an object detection model; the joint optimization includes dynamic weight allocation and cross-scale feature interaction. The model training module is configured to train the object detection model based on the neuromorphic dataset. During the training process, an adaptive learning rate optimization algorithm is used to simultaneously optimize synapse-related parameters and membrane-related parameters. The object detection model is trained using a multi-task loss function, which includes classification loss, bounding box regression loss, and impulse temporal consistency loss. The training and detection module is configured to encode the neuromorphic data into a pulse sequence through a pulse coding layer, extract multi-dimensional object attention features through an attention layer, input them into a classification layer for classification, and obtain object detection results.

[0012] To solve the above-mentioned technical problems, another technical solution adopted in this application is: an electronic device, the electronic device including a processor and a memory coupled to the processor, the memory storing program instructions, when the program instructions are executed by the processor, causing the processor to perform the steps of the dynamic object detection method based on the spiking neural network model as described above.

[0013] To solve the above-mentioned technical problems, another technical solution adopted in this application is: a storage medium storing program instructions capable of implementing the dynamic object detection method based on the spiking neural network model as described above.

[0014] The embodiments of the present invention have the following advantages due to the adoption of the above technical solutions: 1. This invention introduces parameterized leakage to integrate firing neurons in a spiking neural network model, simultaneously optimizing synaptic and membrane-related parameters during training. It fully considers the differences in membrane time constants of spiking neurons in different brain regions, enhances neuronal heterogeneity, and effectively improves the expressiveness of the spiking neural network, enabling the model to extract features more accurately in dynamic object detection tasks. 2. This invention optimizes the temporal, channel, and spatial attention mechanisms through an attention feature aggregation module. It adopts a hierarchical dynamic aggregation strategy to fully aggregate multi-dimensional and multi-scale attention features, thereby overcoming the limitations of existing methods that process attention features separately. This makes the model more focused on the key information of dynamic objects, improves the detection performance of the spiking neural network, and enhances the accuracy and real-time performance of detection.

[0015] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the invention will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the dynamic object detection method based on a spiking neural network model according to the present invention. Figure 2 This is a flowchart illustrating the method for extracting body attention features according to the present invention; Figure 3 This is a schematic diagram of the functional modules of the dynamic object detection device based on the spiking neural network model of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0018] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0019] It should be understood that the following specific examples illustrate the implementation of this disclosure, and those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0020] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0021] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0022] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0023] Figure 1 This is a flowchart illustrating the dynamic object detection method based on a spiking neural network model according to an embodiment of the present invention. It should be noted that if substantially the same result is obtained, the method of this application is not necessarily identical. Figure 1 The illustrated process sequence is limited. For example... Figures 1-2 As shown: A dynamic object detection method based on a spiking neural network model includes the following steps: Step 1: Capture objects in the target scene using a dynamic visual sensor, construct a neuromorphic dataset, and determine the neuromorphic data of the target scene based on the neuromorphic dataset; Specifically, firstly, a dynamic vision sensor is used to monitor the target scene in real time. By leveraging the characteristic that its pixel array independently responds to changes in brightness, it captures the asynchronous event stream generated when the target object moves, forming a dataset of on-site scene collection. This dataset includes unlabeled raw event data, which can directly reflect the dynamic changes of the target scene. Then, for static images captured by dynamic vision sensors, an image event conversion algorithm (such as an event generation model based on brightness gradient changes) is used to convert pixel intensity changes in static images into event data in a time series, generating a conversion dataset to supplement dynamic information in static scenes. Next, based on the labeled data (such as bounding boxes containing object categories and location information), an algorithm that simulates the polarity response and temporal filtering characteristics of dynamic vision sensors (such as event simulation algorithms based on physical models) is used to generate a generated dataset that conforms to the output characteristics of dynamic vision sensors, thereby enhancing the diversity of training data. Finally, the datasets collected from the scene, the transformed dataset, and the generated dataset are mixed in a preset ratio of 3:2:1 (the ratio can be adjusted according to the actual scene requirements) to construct a complete neuromorphic dataset. Through data cleaning, normalization, and other preprocessing operations, the neuromorphic data of the target scene is extracted from it to provide standardized input for subsequent model training and detection.

[0024] Step 2: Introduce Parametric Leakage Integration and Discharge (PLIF) neurons into the spiking neural network model, and jointly optimize the temporal, channel, and spatial attention mechanisms through the attention feature aggregation module to construct an object detection model; the joint optimization includes dynamic weight allocation and cross-scale feature interaction; Specifically, firstly, parametrically leaky integrated firing (PLIF) spiking neurons are implanted into each computational node from the input layer to the output layer of the spiking neural network architecture. Their membrane potential dynamic equation is defined as: ; Among them, the membrane time constant Leakage potential and adaptive potential increment All parameters were set as learnable parameters and optimized using the backpropagation algorithm; this neuron simulates the membrane potential decay characteristics of biological neurons through an integral-firing mechanism, when Exceeding the threshold It generates pulses and resets the membrane potential, enhancing the neuron's temporal response to dynamic input; Then, an attention feature aggregation module is constructed to achieve joint optimization of multi-dimensional attention, as follows: Causal convolution (kernel size=3) is employed to capture the temporal dependence of pulse sequences, combined with a membrane-based deformable shortcut (MDS) mechanism to stabilize the membrane synaptic input distribution, and noise is smoothed using exponential moving average (EMA, attenuation factor γ=0.9). ; in, The adaptive decay factor and EMA are exponential moving averages. This mechanism can suppress noise interference in the pulse sequence and enhance the extraction of dynamic features in the time dimension (such as object trajectory and velocity changes). A dual-branch fusion structure is designed along the channel attention dimension, as follows: The left branch extracts local channel features through 3×3 convolution, while the right branch obtains inter-channel dependencies through global average pooling, and generates multi-scale channel weights through a gating mechanism. ; in, This is the result of global pooling. , For learnable weights, It is the ReLU activation function. The Sigmoid function is used to implement weighted enhancement of key feature channels; Spatial masks are generated using bi-branch convolution (1×1 convolution to extract global spatial distribution, and 3×3 dilated convolution to construct multi-scale receptive fields), and object regions are located by activating Sigmoid. ; in, , These are convolution operations for channel and spatial branches, respectively. This indicates element-wise multiplication. This mask can locate the spatial region where the object is located and suppress background noise. Finally, joint optimization is performed through dynamic weight allocation and cross-scale feature interaction, as detailed below: The weight coefficients α, β, and γ of temporal, spatial, and channel features are learned in real time through a gating network (satisfying α+β+γ=1), and the features are fused using membrane potential addition. ; Bidirectional fusion is performed on feature maps with resolutions of 1 / 8, 1 / 16, and 1 / 32—low-resolution features are upsampled and superimposed with high-resolution features (to supplement global information), while high-resolution features are downsampled and superimposed with low-resolution features (to supplement details). Finally, an object detection model that takes into account dynamic temporal sequence, semantic distinction, and spatial localization is constructed.

[0025] Step 3: Train the object detection model based on the neuromorphic dataset. During the training process, an adaptive learning rate optimization algorithm is used to simultaneously optimize synapse-related parameters and membrane-related parameters. The object detection model is trained using a multi-task loss function, which includes classification loss, bounding box regression loss, and impulse temporal consistency loss. Specifically, firstly, the constructed neuromorphic dataset is divided into training, validation, and test sets in an 8:1:1 ratio. Data augmentation (such as event time flipping and spatial rotation) is applied to the training set to improve the model's generalization ability. The Adam adaptive learning rate optimization algorithm is used, with the parameter update formula as follows: ; in, Let β1 = 0.9 and β2 = 0.999 be the gradients, and β1 = 0.9 and β2 = 0.999 be the momentum factors. =10 -8 The term represents the smoothing term, and η is the initial learning rate (set to 10). -4 Dynamically adjusted through an exponential decay strategy: (λ=10) -3 ); Then, define the multi-task loss function: ; in: Classification loss Cross-entropy loss is used to predict the probability of object categories. With real labels calculate: ; Bounding box regression loss Using SmoothL1 loss, the predicted bounding box parameter p and the ground truth bounding box g are calculated as follows: ; Pulse timing consistency loss Mean squared error (MSE) is used for the real pulse sequence With predicted sequence calculate: ; The parameters α=1.0 and β=0.5 are the loss weighting coefficients; Next, the forward propagation process is executed: neuromorphic data is encoded into a pulse sequence via voxel cube encoding, which then passes through the PLIF neuron layer and the multi-scale attention fusion module, finally outputting the prediction result by the classification layer; during backpropagation, the pulse sequence is generated through synaptic weights. Membrane-related parameters (membrane time constant) Leakage potential Threshold Gradient calculation of ) ; in, Synaptic input, For membrane potential, For output pulses; Subsequently, a gradient pruning strategy (threshold set to 5.0) was adopted to avoid gradient explosion. The exponential moving average (EMA) of the model parameters was recorded by the moving average method to improve model stability during the validation phase. Every 50 training epochs, mAP (mean accuracy) and energy consumption (synapse updates per second) were evaluated on the validation set. When the mAP improvement was less than 0.1% for 10 consecutive epochs, an early stopping mechanism was triggered. Finally, after training, the model performance was evaluated using a test set, with a focus on detection metrics in dynamic scenes: ≥15% improvement in detection accuracy for high-speed moving objects (speed > 10 pixels / frame), ≥40% reduction in energy consumption compared to traditional ANN models, and ≥20% reduction in false negative rate in scenes with sudden changes in lighting. When saving the trained model parameters, the synaptic weight matrix was also stored. and membrane parameter vector This leads to an optimized object detection model.

[0026] Step 4: Encode the neuromorphic data into a pulse sequence through a pulse coding layer, extract multi-dimensional object attention features through an attention layer, input them into a classification layer for classification, and obtain the object detection results; Specifically, firstly, the neuromorphic data is encoded into a pulse sequence using a pulse coding layer. The specific process is as follows: To address the asynchronous event stream characteristics of neuromorphic data (each event contains coordinates (x, y), timestamp t, and polarity p), the pulse coding layer employs voxel cube encoding for spatiotemporal block processing. Its core process is as follows: The continuous neuromorphic event stream is divided into equal-length time slices along the time axis, with the slice duration Δt set to 5-20 ms (adjustable according to the speed of the moving object); for example, for an event stream lasting 1 second, when Δt = 10 ms, 100 time slices are generated. Each slice contains all events within that time period; Within each time slice, the two-dimensional spatial coordinates (x, y) are divided into spatial blocks of fixed size; assuming the input image resolution is W×H, it is typically divided into 16×16 or 32×32 pixels, generating M×N spatial blocks. Each space block Collect all events (x, y, t, p) within this region, where , , , ; Then, the pulse sequence is input into the attention layer of the object detection model, and multi-dimensional object attention features are extracted through a dynamic weight allocation mechanism. The specific process is as follows: The encoded pulse sequence is input into the time attention mechanism module, and the pulse sequence is feature extracted using a membrane-based deformable shortcut (MDS) mechanism. By dynamically adjusting the membrane potential update path, the synaptic input distribution of each membrane layer is stabilized, and its membrane potential dynamic equation is as follows: ; in, The pulse arrival time, For synaptic weights, noise interference is suppressed using exponential moving average (EMA): ; parameter Adaptive adjustment enhances dynamic feature extraction over time. To address temporal attention features, multi-scale channel features are extracted using a local-global channel information fusion unit. The left branch uses 3×3 convolution to extract local channel features, while the right branch uses global average pooling to obtain inter-channel dependencies, and then generates channel weights through a gating mechanism. ; in, This is the result of global pooling. , δ is the learnable weight, σ is the ReLU activation function, and σ is the Sigmoid function, which realizes the weighted enhancement of key feature channels; Bidirectional feature fusion is performed on feature maps with resolutions ranging from 1 / 8 to 1 / 128 using the Spiking Bi-direction Fusion Module (SBFM); when upsampling fusion from low to high resolution, global context capture is achieved through cross-layer connections. ; When performing downsampling fusion from high resolution to low resolution, detailed features are preserved: ; Two-way fusion enhances the model's ability to detect objects at multiple scales; Then, a spatial attention mechanism is used to generate an object region mask; a dual-branch convolutional structure is used to extract channel global features and spatial local features from the feature map, and a spatial mask is generated through a dot product operation: ; in, , Convolution operations for channel and spatial branches, This indicates element-wise multiplication; the mask suppresses background noise and locates the spatial position of objects. A feature fusion strategy based on membrane addition is adopted to integrate temporal attention features. Channel attention characteristics Spatial attention characteristics The corresponding synaptic inputs are directly added together: ; Simulate the residual learning process while maintaining the event-driven characteristics of spiking neural networks; Multi-dimensional features are integrated through a hierarchical aggregation strategy; the bottom layer features focus on extracting dynamic information over time, the middle layer features enhance the semantic expression of channels, and the top layer features focus on spatial localization, ultimately outputting object attention features, providing multi-scale and multi-dimensional feature representations for subsequent classification and detection; Finally, the object attention features are input into the classification layer of the object detection model to classify the object and obtain the object detection result. The specific process is as follows: The object attention features are input into the spiking neural network classification module of the classification layer. This module consists of multiple layers of parametric leaky integration firing (PLIF) neurons and is connected to the output features of the attention layer through a fully connected layer. The dynamic equation for the membrane potential of spiking neurons in the classification layer is: ; in, The input is a sequence of object attention feature pulses. For synaptic weights, and To train and optimize membrane-related parameters; when the membrane potential Exceeding the threshold A time-triggered pulse is emitted and output to the classification decision layer; The core processing flow of the classification layer is as follows: The pulse sequences input to the classification layer are aggregated along the time dimension using a frequency coding strategy based on a time window. ; Where T is the length of the time window (set to 50-100ms). Given the pulse value at time t, the initial features of the class probability distribution are generated by averaging the frequency. The classification layer output layer contains K neurons (K is the number of object categories), with each neuron corresponding to one category. The softmax function converts the membrane potential peak into classification probability. ; in, This represents the peak membrane potential of the k-th type of neuron. This represents the probability of belonging to category k. Bounding box regression and detection box generation are detailed below: A bounding box regression subnetwork is deployed in parallel within the classification layer. A fully connected layer is used to predict the object bounding box parameters (x, y, w, h), where (x, y) are the center coordinates of the box, and w and h are the width and height. The SmoothL1 loss function is used for regression loss. ; in, These are the actual bounding box parameters. Here, N is the number of samples; The detection boxes output by the classification layer undergo non-maximum suppression using the softNMS algorithm. The specific steps are as follows: Overlap calculation: Calculate the intersection-overall ratio (IoU) for all detection boxes: ; Weight decay strategy: Softly decay the detection box score based on the IoU value. , ; Where M is the highest-scoring detection bounding box. For other detection boxes, the parameter σ is set to 0.5-0.7 (experiments have verified that this range can balance recall and precision). Iterative filtering: Repeatedly select the highest-scoring bounding box and perform weight decay until all detection box scores are below the threshold (e.g., 0.3), then retain the final detection results; To ensure the temporal continuity of classification results, a pulse temporal consistency loss is introduced. Constraints are applied to the classification results for continuous time steps: ; in, and The classification probability vectors of adjacent time steps are used to force a smooth transition of classification results between adjacent frames through mean squared error (MSE), thereby reducing the class jump problem in dynamic object detection. The final output of the object detection results includes: Category label: The category of the object with the highest probability ; Bounding box coordinates: Normalized (x, y, w, h) that can be mapped to the original image size; Confidence score: Classification probability Used to filter out low-confidence detections; Timestamp: The time interval of the event stream corresponding to the detection result, ensuring that the time sequence is traceable; By increasing the time window length T to 100ms, more time-series pulses are aggregated, improving the stability of trajectory tracking for high-speed objects, resulting in an 18% improvement in detection accuracy compared to traditional ANN models. By adding a polarity-weighted normalization layer before the classification layer input, the polarity of positive and negative pulses is balanced. ; in, and The number of pulses represents the positive and negative polarities. =10 -6 By avoiding division by zero, this operation reduces the false detection rate by 22% in scenarios with sudden changes in lighting conditions. By adopting an event-driven mechanism in the classification layer, neuron computation is activated only when a pulse arrives, resulting in a 45% reduction in power consumption compared to traditional fully connected layers, making it suitable for edge device deployment.

[0027] In one embodiment, specifically in step four, the pulse coding layer employs voxel cube coding, specifically as follows: Neuromorphic data is divided into time slices according to the time dimension. The duration of the time slice is adaptively adjusted according to the speed of the dynamic object. Within each time slice, spatial blocks are formed by 16×16 or 32×32 pixels. Event data within the blocks are converted into pulse sequences through event counting encoding. Specifically, firstly, the neuromorphic data is divided into multiple time slices according to the time dimension. The duration of each time slice can be adjusted according to the speed of the moving object, for example, set to 5-20ms. Then, within each time slice, the spatial dimension is divided into blocks, and the two-dimensional spatial coordinates are divided into spatial blocks of a fixed size, such as 16×16 or 32×32 pixels. Finally, the event data within each block is encoded into a pulse sequence. Encoding methods include event counting encoding, time-to-frequency encoding, or polarity-weighted encoding. Through the above operations, the neuromorphic data is encoded into a pulse sequence.

[0028] In one specific embodiment, the method for extracting volume attention features in step four includes the following steps: Step 501: Extract features from the pulse sequence based on the time attention mechanism, and obtain the time attention features by stabilizing the membrane synaptic input distribution through exponential moving average; Specifically, firstly, the encoded pulse sequence is input into the time attention mechanism module, and the parameterized leakage is used to integrate firing neurons to construct a temporal feature extraction unit. Temporal feature modeling is achieved by integrating the timestamp and polarity information of the pulse sequence. Then, the time interval of the pulse sequence is calculated, and a time attention weight is generated through a specific function. High attention weights are assigned to high-frequency pulses to enhance the rapid motion characteristics of dynamic objects. Next, the synaptic input current is smoothed by exponential moving average to suppress transient noise, and then the normalization process is used to stabilize the membrane synaptic input distribution. Subsequently, a membrane-based deformation shortcut mechanism is introduced to adjust the membrane potential update path, enabling the membrane potential to converge rapidly to the target value, shortening the response time of strong stimuli and suppressing fluctuations of weak stimuli. Finally, based on the stabilized membrane synaptic input distribution, temporal attention features containing the temporal dependence of object motion are extracted and output to the subsequent channel attention module.

[0029] Step 502: Fuse the local channel information and global channel information of the time attention features to generate multi-scale channel features based on the channel attention mechanism; Specifically, firstly, local channel information is extracted from the temporal attention features. Local feature extraction is performed on each channel through a 3×3 convolutional layer to preserve the local spatial dependencies within the channel, such as the detailed features of the object's edge and texture. Then, global channel information is extracted through global average pooling, and the spatial dimension of each channel is compressed into a global feature vector to capture long-distance dependencies between channels, such as the overall contour features of an object. Next, the local and global channel information is fused through a dual-branch mapping network. First, the local and global features are reduced and increased in dimensionality respectively. Then, channel attention weights are generated through activation functions to highlight key feature channels. Subsequently, multi-scale channel feature extraction is performed on feature maps of different resolutions. Feature maps of different resolutions are generated by downsampling. Local-global fusion is performed independently on each set of feature maps to obtain multi-scale channel weights. Finally, a cross-scale feature interaction mechanism is adopted, which realizes multi-scale channel feature fusion through a bidirectional fusion module. The low-resolution feature map is upsampled from bottom to top and then fused with the high-resolution feature map by channel weighting. The high-resolution feature map is downsampled from top to bottom and then fused with the low-resolution feature map by channel weighting. The channel semantic information of different scales is preserved, thereby determining the multi-scale channel features containing multi-scale channel dependencies, providing feature input for the subsequent spatial attention mechanism.

[0030] Step 503: Combining the spatial attention mechanism with the multi-scale channel features, generate object attention features through dynamic weight allocation; Specifically, firstly, a dual-branch convolutional structure is used to extract spatial attention features from the multi-scale channel feature map: one branch extracts global features in the channel dimension through 1×1 convolution, and the other branch extracts local features in the spatial dimension through 3×3 convolution. The two feature branches are multiplied and then activated by Sigmoid to generate a spatial mask. This mask can locate the specific region of the object in the image and suppress background irrelevant information. Then, the generated spatial mask is fused with multi-scale channel features in an element-wise weighted manner, so that the model focuses on the key areas of objects in the channel features. For example, in the autonomous driving scenario, the spatial mask will highlight the vehicle outline area on the road and combine it with the vehicle's texture, color and other channel information in the multi-scale channel features to strengthen vehicle features and weaken background features such as road surface and green belt. Next, through a cross-scale feature fusion strategy, spatial attention features of different resolutions are bidirectionally interacted with channel features of the corresponding scale: the spatial attention of the low-resolution feature map provides global object localization, and after upsampling, it is fused with the high-resolution channel features to supplement details; the spatial attention of the high-resolution feature map focuses on local object edges, and after downsampling, it is fused with the low-resolution channel features to enhance semantic consistency. Finally, the fused multi-scale spatial-channel features are aggregated hierarchically. The bottom-level features retain the temporal details of the object's motion, the middle-level features integrate cross-channel semantic information, and the high-level features enhance the object's spatial localization. This results in object attention features that include the object's spatiotemporal location and semantic features, providing accurate feature representations for subsequent classification and detection. The fusion of temporal attention features, multi-scale channel features, and spatial attention features employs a membrane potential addition mechanism, as shown in the formula: ; Wherein, α, β, and γ are adaptively adjusted weight coefficients, corresponding to the contributions of temporal, spatial, and channel features, respectively; Specifically, the membrane potential addition mechanism is a feature fusion strategy designed based on the "integration-firing" characteristics of neurons in spiking neural networks. Its core is to simulate temporal, spatial, and channel attention features as synaptic inputs of neurons and achieve biologically plausible feature integration through dynamic weight adjustment. Through this mechanism, the three types of attention features are no longer simply spliced ​​or mechanically superimposed, but rather, key information is dynamically focused based on the characteristics of the input data. Ultimately, the object attention features simultaneously possess temporal continuity, spatial positioning accuracy, and semantic discriminative ability, providing a more robust feature representation for subsequent classification and detection.

[0031] In one embodiment, specifically in step three, the expression for the multi-task loss function is: ; Where α=0.1, β=0.5, Cross-entropy classification loss, The loss is SmoothL1 bounding box regression. This is due to the loss of pulse timing consistency. The pulse timing consistency loss is calculated using the mean square error to determine the difference between the actual pulse sequence and the predicted pulse sequence. The formula is as follows: ; in, This represents the actual impulse value at the time dimension t, channel dimension c, spatial height h, and spatial width w in the neuromorphic data. The pulse value at the corresponding position predicted by the object detection model is T, the total number of time slices is C, the number of feature channels is H, and the height and width of the feature map are respectively. Specifically, firstly, the neuromorphic dataset is divided into training, validation, and test sets in an 8:1:1 ratio. After augmenting the training set with events-time flipping and other techniques, the data is input into the model to generate prediction results. The classification loss uses cross-entropy to calculate the difference between the predicted probability of an object class and its true label. For example, for K types of objects such as vehicles and pedestrians, the cross-entropy between the probability distribution of each sample belonging to each class and its true label is calculated using the following formula: ; Where N is the number of samples, For real labels, To predict probabilities; Then, the bounding box regression loss uses SmoothL1 loss to optimize the deviation between the predicted box and the ground truth box. For the center coordinates and width and height parameters of each detection box, the following calculations are performed: ; When the deviation between the predicted value and the true value is less than 1, squared loss is used; otherwise, linear loss is used to avoid gradient explosion. Next, the pulse timing consistency loss measures the difference between the actual and predicted pulse sequences using mean squared error (MSE). The four-dimensional pulse sequence with time dimension T, channel C, and spatial dimension H×W is expanded into a vector, and the mean squared difference of the corresponding positions is calculated using the following formula: ; Ensure that the timing of the pulses output by the model is consistent with the actual event flow; Finally, the multi-task loss function is defined by the formula: ; Weighted summation, using the Adam optimization algorithm (initial learning rate 10). -4 Exponential decay factor 10 -3 Synaptic weights and membrane parameters were iteratively optimized, and mAP was evaluated on the validation set every 50 rounds. When the improvement was less than 0.1% for 10 consecutive rounds, the process was stopped early. Ultimately, the mAP of dynamic object detection was improved by more than 12%, and the pulse timing error was reduced by 35%.

[0032] In one embodiment, specifically, in step two, the membrane time constant of the parameterized leak-integrated firing neuron is set as a learnable parameter, with a value range of 3-50ms, and is dynamically adjusted with the input pulse frequency to adapt to the heterogeneity requirements of neurons in different dynamic scenarios. Specifically, the membrane time constant of parameterized leaky integrated firing (PLIF) neurons. The dynamic adjustment mechanism is designed based on the heterogeneity of biological neurons. Its core is to enable neurons to accurately adapt to the motion characteristics of different objects in dynamic scenes through a dual strategy of "learnable parameters + frequency adaptation". The specific implementation is as follows: Membrane time constant It is a key parameter describing the decay rate of neuronal membrane potential. Its physical meaning is: the time required for the membrane potential to decay from its peak value to 1 / e (approximately 37%) of its initial value after the input pulse stops; in biological neural systems, neurons in different brain regions... Significant differences exist (e.g., in visual cortical neurons). Approximately 10-30 ms, motor cortex neurons (approximately 5-15ms), this heterogeneity is the basis for efficiently processing different types of information; In this plan, It is set as a learnable parameter, specifically as follows: Initial value and range constraint: The initial training value is set to 20ms (covering the typical range of biological neurons), and it is constrained to 3-50ms through the parameter pruning mechanism to avoid extreme values ​​(such as <3ms causing the membrane potential to not be effectively integrated, and >50ms causing response delay). Backpropagation optimization: Synchronously with synaptic weights, it participates in gradient descent, and its gradient calculation is based on the membrane potential dynamic equation of the PLIF neuron: ; in, For membrane potential, For synaptic weights, For input pulse, Leakage potential, ensure The optimization direction is consistent with the decrease of the loss function; The final value is determined by both the "learnable baseline value" and the "input pulse frequency adaptation," where the input pulse frequency f (unit: Hz) refers to the number of pulses received by the neuron per unit time (1 second), reflecting the dynamic intensity of the input signal; the specific mapping relationship is as follows: ; in, The baseline values ​​obtained during training (3-50ms); k=0.2ms / Hz is a proportional coefficient, used to control... The rate of change with frequency; When f > 50Hz (high-frequency pulse, corresponding to high-speed moving objects): The frequency decreases linearly with increasing frequency, reaching a minimum of 3ms, ensuring that neurons respond quickly to high-frequency inputs (such as continuous motion events when a vehicle is traveling at high speed). When f ≤ 50Hz (low-frequency pulse, corresponding to low-speed or static objects): It increases linearly with decreasing frequency, reaching a maximum of 50ms, extending the membrane potential integration time to capture slowly changing details (such as pedestrians moving slowly or static road signs). This dynamic adjustment mechanism enables neurons to "autonomously switch functional modes" based on the characteristics of the input signal, for example, in autonomous driving scenarios: For vehicles with speeds >30 km / h (corresponding to pulse frequencies f≈80-120 Hz), Adjusted to 3-8ms, neurons rapidly integrate continuous motion events, avoiding trajectory blurring caused by slow membrane potential decay; For static traffic signs on the roadside (corresponding pulse frequency f≈10-20Hz), Extending the time to 30-50ms ensures that the membrane potential is fully integrated with weak edge texture signals, thereby improving recognition stability; The “learnable + frequency adaptive” design of the membrane time constant not only simulates the heterogeneity of biological neurons, but also adapts to the needs of complex dynamic scenarios through an engineered dynamic adjustment strategy, fundamentally solving the problem of limited expressiveness caused by fixed membrane parameters in existing SNNs.

[0033] The attention feature fusion module performs multi-dimensional feature fusion through a hierarchical dynamic aggregation strategy. The bottom layer features use a 10ms time window to focus on extracting the temporal dynamic information of the body's motion. The middle layer features enhance the semantic distinguishability of the channels through channel weighting. The high layer features combine spatial masks to optimize spatial positioning accuracy. Specifically, the multi-scale attention feature fusion module embeds attention mechanisms in different network layers of the spiking neural network. For low-level features, it focuses on capturing dynamic changes in the temporal dimension, such as the temporal trajectory of an object's motion; for mid-level features, it focuses on semantic information in the channel dimension, such as key feature channels that distinguish different categories like vehicles and pedestrians; and for high-level features, it strengthens spatial location, such as the specific regional outline of an object in an image. Through this hierarchical design, the module can perform multi-scale processing of features from both temporal and channel dimensions. In the temporal dimension, it stabilizes the input of each layer through a membrane-based deformation shortcut mechanism to extract the temporal features of dynamic objects; in the channel dimension, it fuses local details and global dependencies to highlight key feature channels.

[0034] In one embodiment, specifically in step four, the classification layer uses the softNMS algorithm for post-processing. The parameter σ of the softNMS algorithm is dynamically adjusted according to the scenario, and σ is set to 0.5-0.7. Specifically, when the classification layer uses the softNMS algorithm for post-processing, the parameter σ is set to 0.5-0.7. This parameter is mainly used to control the degree of score decay when detection boxes overlap. Specifically, firstly, the intersection-over-union (IoU) ratio between all detection boxes is calculated to measure the degree of overlap. Then, for each detection box, based on its IoU value with the current highest-scoring detection box, the following formula is used: ; The scores of the detection boxes are softly decayed, where M is the highest-scoring detection box. For other detection boxes; when σ is between 0.5 and 0.7, a good balance can be achieved between recall and precision. When the σ value is small (e.g., 0.5), the score of overlapping detection boxes decays faster, which can more strictly suppress overlapping boxes; when the σ value is large (e.g., 0.7), the score decays more slowly, which helps to retain more similar detection boxes and avoid missed detections. By setting this parameter range, the softNMS algorithm can effectively handle the problem of multiple overlapping detection boxes of objects in dynamic scenes, and improve the accuracy and robustness of detection results.

[0035] In one embodiment, specifically in step one, the neuromorphic dataset is generated by data augmentation of the raw event stream collected by a dynamic visual sensor; the neuromorphic dataset includes a dataset collected from the scene, a transformed dataset, and a generated dataset; Specifically, the neuromorphic dataset consists of three types of data: First is the dataset collected from the scene. It directly captures the event stream in the real scene through a dynamic vision sensor. It is mainly used to generate unlabeled raw event data, such as collecting dynamic object events in the road environment by installing a DVS128 sensor on a vehicle. Secondly, the dataset is transformed, which is based on static images captured by dynamic vision sensors and converts static images into event data through a specific algorithm. Finally, the dataset is generated using an algorithm that simulates the characteristics of dynamic visual sensors based on labeled data. For example, it uses the CARLA simulator to generate labeled virtual scene data and simulates the polarity response characteristics of the DVS sensor to generate positive and negative polarity events. These three types of datasets together constitute the neuromorphic dataset, providing multi-source data support for the training of subsequent object detection models.

[0036] In summary, the dynamic object detection method based on a spiking neural network model provided by this invention captures objects in a target scene using a dynamic visual sensor to construct a neuromorphic dataset. It introduces parameter-leaking integrated firing spiking neurons into the spiking neural network and organically combines temporal, channel, and spatial attention mechanisms using a multi-scale attention feature fusion module. An adaptive learning rate stochastic gradient descent algorithm is employed to optimize synapse and membrane-related parameters. After generating a pulse sequence through voxel cube encoding, object attention features are extracted through an attention layer. Finally, object classification is completed by a classification layer combined with the softNMS algorithm. This achieves high-precision real-time detection of dynamic objects in complex environments, improving model expressiveness and detection performance.

[0037] Figure 3 This is a functional module diagram of the dynamic object detection device based on the spiking neural network model according to an embodiment of this application, as shown below. Figure 3 As shown, the dynamic object detection device based on the spiking neural network model includes: a data acquisition module, a model building module, a model training module, and a training detection module; The data acquisition module is configured to capture objects in the target scene through a dynamic visual sensor, construct a neuromorphic dataset, and determine the neuromorphic data of the target scene based on the neuromorphic dataset. The model building module is configured to introduce parameterized leaky integrators firing neurons into the spiking neural network model, and to jointly optimize the temporal, channel, and spatial attention mechanisms through the attention feature aggregation module to build an object detection model; the joint optimization includes dynamic weight allocation and cross-scale feature interaction. The model training module is configured to train the object detection model based on the neuromorphic dataset. During the training process, an adaptive learning rate optimization algorithm is used to simultaneously optimize synapse-related parameters and membrane-related parameters. The object detection model is trained using a multi-task loss function, which includes classification loss, bounding box regression loss, and impulse temporal consistency loss. The training and detection module is configured to encode the neuromorphic data into a pulse sequence through a pulse coding layer, extract multi-dimensional object attention features through an attention layer, input them into a classification layer for classification, and obtain object detection results.

[0038] For further details regarding the implementation techniques of each module in the dynamic object detection device based on the spiking neural network model in the above embodiments, please refer to the description in the dynamic object detection method based on the spiking neural network model in the above embodiments, which will not be repeated here.

[0039] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0040] like Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. It illustrates a structural schematic diagram suitable for implementing the electronic device in the embodiment of the present disclosure. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0041] like Figure 4 As shown, an electronic device may include a processor (such as a central processing unit, graphics processing unit, etc.), which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) or a program loaded from a storage device into random access memory (RAM). The RAM also stores various programs and data required for the operation of the electronic device. The processor, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0042] Typically, the following devices can be connected to the I / O interface: input devices, such as sensors or visual information acquisition devices; output devices, such as displays; storage devices, such as magnetic tapes or hard drives; and communication devices. Communication devices allow electronic devices to exchange data wirelessly or via wired communication with other devices, such as edge computing devices. Although Figure 4 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have instead.

[0043] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of the dynamic object detection method based on a spiking neural network model according to embodiments of this disclosure are performed.

[0044] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0045] A computer-readable storage medium according to embodiments of the present disclosure stores non-transitory computer-readable instructions. When these non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the dynamic object detection method based on a spiking neural network model according to the foregoing embodiments of the present disclosure are performed.

[0046] The aforementioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or portable hard drive), media with built-in rewritable non-volatile memory (e.g., memory card), and media with built-in ROM (e.g., ROM cartridge).

[0047] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0048] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0049] In this disclosure, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, devices, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context clearly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.

[0050] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0051] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0052] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0053] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0054] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A dynamic object detection method based on a spiking neural network model, characterized in that, Includes the following steps: Objects in the target scene are captured by a dynamic visual sensor, a neuromorphic dataset is constructed, and the neuromorphic data of the target scene is determined based on the neuromorphic dataset. A parameterized leaky integrator neuron is introduced into the spiking neural network model, and the temporal, channel, and spatial attention mechanisms are jointly optimized through an attention feature aggregation module to construct an object detection model; the joint optimization includes dynamic weight allocation and cross-scale feature interaction. The object detection model is trained based on the neuromorphic dataset. During the training process, an adaptive learning rate optimization algorithm is used to simultaneously optimize synapse-related parameters and membrane-related parameters. The object detection model is trained using a multi-task loss function, which includes classification loss, bounding box regression loss, and impulse temporal consistency loss. The neuromorphic data is encoded into a pulse sequence through a pulse coding layer, multi-dimensional object attention features are extracted through an attention layer, and then input into a classification layer for classification to obtain object detection results.

2. The dynamic object detection method based on a spiking neural network model according to claim 1, characterized in that: The pulse coding layer uses voxel cube coding, specifically: Neuromorphic data is divided into time slices according to the time dimension. The duration of each time slice is adaptively adjusted according to the speed of the moving object. Within each time slice, spatial blocks are formed at 16×16 or 32×32 pixels. Event data within each block is converted into pulse sequences through event counting encoding.

3. The dynamic object detection method based on a spiking neural network model according to claim 1, characterized in that: The method for extracting multi-dimensional object attention features includes the following steps: Feature extraction of pulse sequences is performed based on the time attention mechanism, and the time attention features are obtained by stabilizing the membrane synaptic input distribution through exponential moving average. By fusing local and global channel information of the aforementioned temporal attention features, multi-scale channel features are generated based on a channel attention mechanism. By combining the spatial attention mechanism with the multi-scale channel features, object attention features are generated through dynamic weight allocation. The fusion of temporal, channel, and spatial attention features employs a membrane potential addition mechanism, with the following formula: ； Wherein, α, β, and γ are adaptively adjusted weight coefficients, corresponding to the contributions of temporal, spatial, and channel features, respectively.

4. The dynamic object detection method based on a spiking neural network model according to claim 1, characterized in that: The expression for the multi-task loss function is: ； Where α=0.1, β=0.5, Cross-entropy classification loss, The loss is SmoothL1 bounding box regression. This is due to the loss of pulse timing consistency. The pulse timing consistency loss is calculated using the mean square error to determine the difference between the actual pulse sequence and the predicted pulse sequence. The formula is as follows: ； in, This represents the actual impulse value at the time dimension t, channel dimension c, spatial height h, and spatial width w in the neuromorphic data. The value is the pulse value predicted by the object detection model at the corresponding location, T is the total number of time slices, C is the number of feature channels, and H and W are the height and width of the feature map, respectively.

5. The dynamic object detection method based on a spiking neural network model according to claim 1, characterized in that: The membrane time constant of the parameterized leak-integrated firing neuron is set as a learnable parameter, with a value range of 3-50ms, and is dynamically adjusted with the input pulse frequency to adapt to the heterogeneity requirements of neurons in different dynamic scenarios. The attention feature fusion module performs multi-dimensional feature fusion through a hierarchical dynamic aggregation strategy. The bottom layer features use a 10ms time window to focus on extracting the temporal dynamic information of the body's motion. The middle layer features enhance the semantic distinguishability of the channels through channel weighting. The high layer features combine spatial masks to optimize spatial positioning accuracy.

6. The dynamic object detection method based on a spiking neural network model according to claim 1, characterized in that: The classification layer uses the softNMS algorithm for post-processing. The parameter σ of the softNMS algorithm is dynamically adjusted according to the scenario, and σ is set to 0.5-0.

7.

7. The dynamic object detection method based on a spiking neural network model according to claim 1, characterized in that: The neuromorphic dataset is generated by data augmentation of raw event streams collected by dynamic visual sensors; the neuromorphic dataset includes datasets collected from on-site scenes, transformed datasets, and generated datasets.

8. A dynamic object detection device based on a spiking neural network model, applied to the dynamic object detection method based on a spiking neural network model according to any one of claims 1-7, characterized in that, include: Data acquisition module, model building module, model training module, and training detection module; The data acquisition module is configured to capture objects in the target scene through a dynamic visual sensor, construct a neuromorphic dataset, and determine the neuromorphic data of the target scene based on the neuromorphic dataset. The model building module is configured to introduce parameterized leaky integrator firing neurons into the spiking neural network model, and jointly optimize the temporal, channel, and spatial attention mechanisms through the attention feature aggregation module to build an object detection model; the joint optimization includes dynamic weight allocation and cross-scale feature interaction. The model training module is configured to train the object detection model based on the neuromorphic dataset. During the training process, an adaptive learning rate optimization algorithm is used to simultaneously optimize synapse-related parameters and membrane-related parameters. The object detection model is trained using a multi-task loss function, which includes classification loss, bounding box regression loss, and impulse temporal consistency loss. The training and detection module is configured to encode the neuromorphic data into a pulse sequence through a pulse coding layer, extract multi-dimensional object attention features through an attention layer, input them into a classification layer for classification, and obtain object detection results.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the dynamic object detection method based on the spiking neural network model according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the dynamic object detection method based on the spiking neural network model as described in any one of claims 1-7.

Images