Pulse neural network dynamic time step adjustment method based on multi-state feature fusion

Abstract

This invention discloses a dynamic time-step adjustment method for spiking neural networks based on multi-state feature fusion, comprising multiple types of perceptrons and a time-step adjustment decision module. The perceptrons are configured into three types: a task stage perceptron to characterize the task progress state of the input pulse sequence on a macroscopic time scale; a topology perceptron to describe the static structural features of the network, such as layer depth and channel size; and a temporal dynamic perceptron to characterize the semantic complexity changes and instantaneous activity characteristics of pulse activity on a microscopic time scale. The time-step adjustment decision module determines the time-step adjustment strategy of the spiking neural network based on the multi-state features to adjust the inference time window length of each layer of the network. This mechanism effectively reduces system inference latency and computational energy consumption while ensuring inference accuracy and adjustment stability, making it suitable for brain-like intelligent inference applications with high requirements for real-time performance and energy efficiency.

Description

Technical Field

[0001] This invention belongs to the field of spiking neural networks, specifically relating to a dynamic time step adjustment method for spiking neural networks based on multi-state feature fusion. Background Technology

[0002] Spiking Neural Networks (SNNs), as the third generation of artificial neural networks, demonstrate enormous potential in fields such as neuromorphic computing and low-power edge intelligence by simulating the pulse-based temporal information transmission and processing mechanism of biological neurons. Compared to traditional artificial neural networks, the discrete event-driven nature of SNNs gives them an inherent advantage in processing temporal data and enables them to achieve extremely high energy efficiency on dedicated neuromorphic hardware, providing a new technological path for developing real-time, low-power intelligent systems.

[0003] However, the inference performance of SNNs is highly dependent on the setting of their time step. Existing SNN models generally adopt a fixed time step inference mechanism, that is, keeping the time window length constant across all input samples and throughout the entire inference process. While this static setting simplifies design and implementation, it exposes a series of inherent defects in practical applications: First, when faced with input signals of dynamically changing complexity, a fixed step size often leads to "computational redundancy" or "insufficient information capture": for simple samples, an excessively long step size causes unnecessary iterative computations, increasing latency and energy consumption; for complex samples, an excessively short step size may not be able to accumulate enough impulse information, resulting in insufficient feature extraction and decreased recognition accuracy. Second, different network layers of SNNs have different sensitivities and processing requirements for temporal information, and a uniform static step size cannot adapt to this hierarchical dynamic characteristic, restricting the optimization of the overall expressive power of the network. Furthermore, existing methods lack multi-dimensional perception of task progress, semantic content changes, and the internal state of the network, making it impossible to intelligently allocate time resources according to actual inference needs.

[0004] For example, in autonomous driving scenarios, vehicles need to process multimodal temporal data from LiDAR, cameras, and other sources in real time for environmental perception and path planning. A fixed time step size struggles to balance the low-latency requirements of high-speed scenarios with the high-precision recognition under complex road conditions. In video action recognition tasks, subtle changes in human movements (such as rapid gesture switching) often contain crucial semantic information; a fixed time step size may lead to the loss of important temporal features, affecting recognition accuracy. Therefore, designing an intelligent time step adjustment mechanism within a Service-Oriented Neural Network (SNN) that deeply integrates task, topology, and temporal multi-dimensional perception to achieve a fundamental shift from "static fixed" to "dynamic adaptive" is of significant theoretical value and urgent practical necessity for fully unleashing the low-power potential of SNNs and improving their robustness and practicality in complex dynamic scenarios. Summary of the Invention

[0005] To address the aforementioned issues, this invention discloses a dynamic time step adjustment method for spiking neural networks (SNNs) based on multi-state feature fusion. By real-time sensing of complementary information from multiple dimensions, including macroscopic task progress, static network topology, and microscopic pulse dynamics, the method achieves adaptive optimization of the time step, thereby significantly improving the inference speed and energy efficiency of SNNs while maintaining inference accuracy. This method is suitable for neuromorphic intelligent computing scenarios sensitive to latency and power consumption. This invention not only overcomes the inherent shortcomings of SNNs but also provides key technical support for intelligent upgrades in fields such as edge intelligence and human-machine collaboration by being widely applicable to high real-time and high-dynamic application scenarios.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] The dynamic time step adjustment method for spiking neural networks based on multi-state feature fusion includes the following steps:

[0008] Step S1: In the feature extraction stage, three types of perceptrons are deployed in parallel to extract the running state features of the spiking neural network from different dimensions in real time. The three types of perceptrons are: (1) a task stage perceptron, used to extract task progress state features and capture the overall progress state of the task represented by the input pulse sequence from a macroscopic time scale; (2) a topology perceptron, used to extract network topology state features and encode the inherent static structural information of the network, such as the layer depth and channel size; and (3) a temporal dynamic perceptron, used to extract the temporal dynamic characteristics of pulse activity. The temporal dynamic perceptron is composed of a semantic entropy estimator and a pulse change rate estimator, which work together to characterize the instantaneous dynamic characteristics inside the network from a microscopic time scale.

[0009] Step S2: In the time step adjustment generation stage, based on the multi-state features obtained in step S1, the time step adjustment decision module fuses and maps the multi-state features to generate the time step adjustment of the spiking neural network.

[0010] Step S3: In the time step adjustment execution and model adaptive update stage, the running time window length of the spiking neural network is dynamically adjusted and the running process is executed according to the time step adjustment amount generated in step S2; a multi-objective performance constraint function is constructed based on the output result of the running process, and the relevant model parameters are jointly updated according to the function to form a closed-loop adjustment mechanism for time step adjustment amount generation, execution and model adaptive update.

[0011] Furthermore, the specific method for step S1 is as follows:

[0012] The entire process of step S1 is completed collaboratively by three types of perceptrons, which extract and quantify the multi-dimensional dynamic and static features of the spiking neural network in real time.

[0013] (1) The task-stage perceptron uses the input pulse sequences of each layer of the spiking neural network (SNN) as input. For the first... Layer, in time step The input pulse sequence is denoted as ,in For batch size, The input time step is defined as _D_, where _D_ is the feature dimension of the layer. The perceptron internally uses a GRU to update its internal state. The GRU updates its state at time steps _D_. Hidden state The update process (for the hidden layer dimension) involves several key steps. First, the update gate is calculated. :

[0014] ;

[0015] In this formula, It is the hidden state of the previous time step, carrying historical information. (Symbol) The vector concatenation operation combines the historical state with the current input. Connect them. , Update the trainable weight matrix and bias vector corresponding to the gate; for Function; Update Gate Its function is to control how much historical information should be retained in the current state. The closer its value is to 1, the more historical information is retained.

[0016] Next, calculate the reset gate. :

[0017] ;

[0018] The calculation method for resetting a door is similar to that for updating a door, but it uses independent parameters. and . and The trainable weight matrix and bias vector corresponding to the reset gate, the reset gate Its function is to determine how much historical information should be ignored or "reset" when calculating new candidate states. The closer its value is to 0, the more it means that the generation of new states depends on the current input rather than the historical state.

[0019] Then, the result of resetting the gate is used to calculate the candidate hidden state. :

[0020] ;

[0021] In this step, the symbol ⊙ represents element-wise multiplication. Reset gate. Firstly, regarding the historical situation Multiplication allows for selective filtering of historical information. The filtered historical state is then multiplied by the current input. Concatenate and pass through a weight matrix bias Process and generate a candidate state that contains new information at the current time. .

[0022] Finally, combined with the updated gate and candidate hidden state Calculate the final hidden state at the current time step. :

[0023] ;

[0024] It updates the gate As a weight, historical state and candidate states Perform a weighted summation. If If the value is close to 1, the final state primarily adopts the candidate state (i.e., new information); if If the value is close to 0, the final state primarily retains historical information. Ultimately, the hidden state at the current time step is... Output as a feature representation of the task's progress status.

[0025] (2) The topology perceptron is responsible for encoding the static structural information of the SNN network itself. For the first... Layer and its first Each channel has its topological identity defined as a tuple. The perceptron maintains a trainable embedding table. To learn this structural prior, in which It is the total number of network layers. It is the maximum number of channels. This refers to the dimension of the embedding vector (e.g., 16 dimensions). The corresponding topological feature vector is obtained by querying this embedding table. :

[0026] ;

[0027] For channels-independent layers, all channels within that layer share the same topological feature vector. In implementation, all channel vectors corresponding to that layer in the embedding table E are bound together for training. Therefore, any valid channel number c (e.g., c=0) can be used for querying. This embedding vector During training, it is optimized together with other network parameters, thereby learning structural prior knowledge at different locations in the network.

[0028] (3) The semantic entropy estimator aims to quantify the semantic uncertainty of neuron population activity. First, within a time window, the semantic entropy estimator is used to calculate the semantic uncertainty of the neuron population activity. The firing frequency of each neuron (That is, the number of pulses divided by the window length). Then, the firing frequencies of all neurons are normalized to obtain a probability distribution. :

[0029] ;

[0030] in, This represents the total number of neurons in this layer. It is a very small positive number used to prevent division by zero errors.

[0031] Based on this probability distribution, calculate the Shannon information entropy within this time window. As a static measure of semantic complexity:

[0032] ;

[0033] To capture the dynamic changes in complexity, the entropy change characteristics of adjacent time windows are further calculated. :

[0034] ;

[0035] This entropy change characteristic This is the output of the semantic entropy estimator. An increase in entropy value indicates increased information disorder and complexity; a decrease in entropy value indicates that information tends to be more ordered and less complex.

[0036] (4) The impulse rate of change estimator operates in parallel, quantifying the instantaneous dynamic changes in neuron population activity at the signal strength level, effectively complementing the semantic entropy estimator. This estimator defines a time window within which the impulse rate of change is calculated. Average pulse firing rate of layer neuron population :

[0037] ;

[0038] in, This represents the total number of neurons in that layer. The time window length, Indicates the first At time step, one neuron The pulse delivery state.

[0039] To capture the changing trend of activity, the estimator calculates the first-order absolute difference of the pulse rate between adjacent time windows as a feature of the pulse rate of change. :

[0040] ;

[0041] scalar characteristic This is the output of the pulse rate of change estimator. An increase in the value indicates a drastic change in the overall activation level of the neuron population, which may correspond to a change in input information or a switch in the internal state of the network; a decrease in the value indicates that the activation level tends to stabilize.

[0042] Explanation of the relationship with the semantic entropy estimator: The output of the semantic entropy estimator... The uncertainty change of the information carried by the impulse pattern was measured from an information theory perspective; while the output of the impulse change rate estimator... This measures the intensity change of the pulse signal from a signal processing perspective. Both methods, from different but complementary dimensions, jointly characterize the temporal dynamic factors in the SNN processing, providing more comprehensive perceptual information for the subsequent time-step adjustment decision module.

[0043] Furthermore, the specific method of S2 is as follows:

[0044] (1) State feature fusion: The above heterogeneous feature vectors are concatenated to form a unified joint representation vector. :

[0045] ;

[0046] This operation integrates information on task temporal dynamics, network structure priors, semantic complexity changes, and neuron population activity changes into a high-dimensional vector, providing comprehensive state awareness for subsequent adjustment decisions.

[0047] (2) Regulation factor prediction: The fused joint feature vector Input a lightweight multilayer perceptron (MLP) or linear mapping layer to predict a scalar regulation coefficient. This coefficient characterizes the degree to which the time step is scaled based on the current multi-state characteristics.

[0048] ;

[0049] in, , , , For trainable parameters, Activation function for Function, ensure output This makes the adjustment coefficient smooth and normalized.

[0050] (3) Time step demapping: The predicted adjustment coefficients are then demapped. Mapped to actual usable integer time steps This is used to control the SNN inference process in the next time window. The mapping function is defined as follows:

[0051] ;

[0052] in, and The preset minimum and maximum allowable time steps, This is the floor function. This step ensures that the output time step is within a preset reasonable range and meets the SNN computation requirement for integer time steps.

[0053] (4) Output and Iteration: The time step adjustment decision module outputs the predicted time step. This value is given to the SNN inference engine. It will serve as the direct basis for adjusting the time window length in step S3. The entire S2 process is repeated at each time step or every N time steps, achieving dynamic and adaptive temporal scaling.

[0054] Furthermore, the specific method of S3 is as follows:

[0055] The core of step S3 lies in forming a complete "perception-decision-execution-optimization" closed loop. The time-step adjustment strategy predicted in step S2 is executed, the resulting network performance is evaluated, and the entire system (including the perceptron in S1 and the time-step adjustment decision module in S2) is guided to evolve towards high precision, high efficiency, smoothness, and stability by optimizing the multi-objective loss function.

[0056] (1) Dynamic time window inference execution: The system executes in time steps At the beginning, read the integer time step predicted by step S2. And directly set it as the length of the next inference window, that is:

[0057] ;

[0058] Then in the window Within the window, the SNN is iteratively calculated with a fixed discrete time step (usually 1) optimized by step S2 to complete the accumulation and propagation of pulses, and finally the inference result of the time slice is produced at the end of the window.

[0059] (2) Multi-objective loss calculation and backpropagation: The system enters the multi-objective loss calculation and backpropagation stage to evaluate the network performance resulting from the above decisions and guide the optimization direction. This step defines a comprehensive multi-objective loss function. The weighted sum of the following four loss terms:

[0060] ;

[0061] in, , , These are adjustable hyperparameters used to balance the importance of different optimization objectives. After calculating the total loss, the gradient is used through backpropagation to not only update the classification weights of the SNN, but also to adjust the decision module in step S2 and the perceptron in step S1, optimizing all trainable parameters end-to-end.

[0062] (3) Iterative update of model parameters and closed-loop feedback: based on The calculated gradient is used to simultaneously update the following three parameters using an optimizer (such as Adam): First, update the parameters of the SNN backbone network to improve its basic classification ability; second, update the parameters of the time step adjustment decision module in step S2. , , , This enables it to make better time-step decisions; finally, the parameters in each perceptron in step S1 are updated. , , , , , (Embedded table E) enables it to extract features that are more useful for decision-making and optimization.

[0063] Furthermore, the four specific loss items mentioned above will be explained:

[0064] (1) Classification accuracy loss To encourage SNNs to output correct predictions, the specific formula is as follows:

[0065] ;

[0066] in, It is the cross-entropy loss function. This represents the predicted probability distribution of the output based on the input data for the current time slice. The true label for the data input in the current time slice.

[0067] (2) Reasoning efficiency loss To encourage shorter average inference time, the output of step S2 can be used directly. The specific formula is as follows:

[0068] ;

[0069] in, This is the size of the sliding window used to calculate the average time step. This loss prompts the time step adjustment decision module in step S2 to predict smaller time steps, thereby reducing computational cost.

[0070] (3) Temporal smoothness loss This penalty applies to significant fluctuations in the time step, ensuring the stability of the inference process. This loss is applied directly to the predicted sequence in step S2, and the specific formula is as follows:

[0071] ;

[0072] (4) Perception-Decision Consistency Loss: Connecting steps S1, S2, and S3. Encouraging the decision (time step) in step S2 to be logically consistent with the semantic dynamics and task progress perceived in step S1, the specific formula is as follows:

[0073] ;

[0074] in, It is a priori, differentiable reference function. For example, it can be defined as: when semantic entropy increases sharply ( (Large) or the task is in a critical phase (from When a certain state is decoded (in the process of processing), the reference function outputs a larger time step, and vice versa. This loss term ensures that the decisions of the time step adjustment decision module do not deviate from the basic perceptual logic.

[0075] The beneficial effects of this invention are as follows:

[0076] (1) This invention constructs a dynamic control architecture by integrating task phases, topology, and temporal dynamic multi-state features, transforming the time step size of the spiking neural network from a static preset to an adaptive adjustment. This mechanism utilizes a task phase perceptron, a topology perceptron, a semantic entropy estimator, and a pulse change rate estimator to collaboratively extract the dynamic characteristics and structural information of the input data, thereby achieving intelligent allocation of computing resources. This effectively overcomes the computational redundancy and insufficient information utilization caused by a fixed step size, significantly reducing latency and energy consumption while maintaining inference accuracy.

[0077] (2) This invention proposes a multi-objective joint training framework for dynamic time step optimization. By designing a composite objective function that integrates classification accuracy, efficiency, smoothness, and consistency loss, the framework performs end-to-end collaborative optimization of the SNN backbone network, perceptron, and time step adjustment decision module. This framework not only drives the model to improve task accuracy but also encourages the reduction of unnecessary computation steps through efficiency loss, suppresses step size jitter through smoothness loss, and maintains the consistency between step size decision and underlying dynamic logic through consistency loss, ultimately achieving collaborative optimization of accuracy, speed, and stability. Attached Figure Description

[0078] Figure 1 This is the overall flowchart of the present invention. Detailed Implementation

[0079] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention.

[0080] like Figure 1 As shown, the present invention discloses a dynamic time step adjustment method for spiking neural networks based on multi-state feature fusion, which includes the following steps:

[0081] Step S1: In the feature extraction stage, three types of perceptrons are deployed in parallel, specifically with four core components to extract the running state features of the spiking neural network from different dimensions in real time: (1) Task progress state features: the task stage perceptron captures the overall progress state of the task represented by the input pulse sequence from the macro time scale; (2) Network topology state features: the topology perceptron extracts the inherent static structural information of the network, such as the layer depth and channel size; (3) Temporal dynamic characteristics of pulse activity: the semantic entropy estimator and the pulse change rate estimator work together to characterize the instantaneous dynamic characteristics inside the network from the micro time scale.

[0082] Step S2: In the time step adjustment generation stage, based on the multi-state features obtained in step S1, the time step adjustment decision module fuses and maps the multi-state features to generate the time step adjustment of the spiking neural network.

[0083] Step S3: In the time step adjustment execution and model adaptive update stage, the running time window length of the spiking neural network is dynamically adjusted and the running process is executed according to the time step adjustment amount generated in step S2; a multi-objective performance constraint function is constructed based on the output result of the running process, and the relevant model parameters are jointly updated according to the function to form a closed-loop adjustment mechanism for time step adjustment amount generation, execution and model adaptive update.

[0084] This embodiment uses the classic MNIST handwritten digit recognition task as an application scenario, employing a spiking neural network (SNN) containing an input layer, two convolutional layers, and two fully connected layers as the backbone network. The system initialization parameters are as follows: batch size... The initial input time steps are 32. The minimum and maximum allowed time steps are 10. , The values ​​are 5 and 20, respectively, representing the time window lengths for calculating semantic entropy and impulse rate of change. The hidden layer dimension of the GRU in the task-stage perceptron is 5. The dimension of the topological embedding vector is 64. The weighting coefficients for efficiency, smoothness, and consistency loss in the multi-objective loss function are 16. , , The values ​​are set to 0.1, 0.05, and 0.05 respectively.

[0085] Step S1: Multi-state feature extraction stage

[0086] This step captures the runtime state of the SNN from different dimensions by having four perceptrons work together. Taking the system processing a mixed batch containing the complex digit "8" and the simple digit "1" as an example, the specific implementation is as follows:

[0087] (1) Task Stage Perceptron: The input pulse sequence of the first layer of the SNN at time step t (in =32 represents the batch size. =10 is the initial time step. =64 is the feature dimension. Three-dimensional tensor. Each row corresponds to a pulse sequence of a sample, and each column corresponds to a feature vector of a time step. Temporal modeling is performed using a single-layer GRU network, with the GRU hidden state dimension... =64. The GRU update formula is:

[0088] ;

[0089] ;

[0090] in, It is the hidden state of the previous time step (the first dimension 1 corresponds to the hidden state dimension of a single-layer GRU), carrying the task progress history information of the preceding time step. This represents the vector concatenation operation, which... Flattened by batch Afterwards, with ( 10 64) Concatenate along the feature dimension 10 A matrix of 128; , It is a learnable weight matrix with a dimension of 64×128 (the input dimension is 64+64=128, and the output dimension is the hidden layer dimension of 64). , The bias terms are all 64×1 in dimension and are initialized as all-zero vectors. yes The activation function is used to control the strength of the gate switch.

[0091] Next, calculate the candidate hidden state. :

[0092] ;

[0093] In this step, the symbol ⊙ represents element-wise multiplication, and the dimension of the operation is... , Consistency 10 64); Reset the door Historical states are obtained through element-wise multiplication. Perform selective filtering when When the value approaches 0, historical states are significantly suppressed, and the calculation of new states relies more on the current input; the filtered historical states and the current input Assemble as described above. 10 128, via the weight matrix (Dimensions 64×128) and bias A candidate state containing new information at the current time is generated after a (64×1) linear transformation. .

[0094] Finally, update the current hidden state. :

[0095] ;

[0096] It updates the gate As a weight (dimension B×10×64), for historical states and candidate states Perform a weighted summation. If If the value is close to 1, the final state primarily adopts the candidate state (i.e., new information); if If the value is close to 0, the final state primarily retains historical information. Ultimately, the hidden state at the current time step is... The output, representing the macroscopic progress of the task, has a dimension of 1×B×64. When processing complex strokes such as the digit "8", GRU captures long-range dependencies through the aforementioned gating mechanism, and its final hidden state... The temporal features indicating that "the current task is in the complex pattern recognition stage" are encoded and used as the output of the perceptron; while when processing the digit "1", The vector norm is significantly smaller, clearly distinguishing the progress status of different task difficulties.

[0097] (2) Static architecture knowledge of the topological perceptron encoding network. For a predefined, trainable embedding table... in =5 represents the total number of network layers. =16 is the maximum number of channels. =16 represents the embedding dimension. The embedding table is initialized using a normal distribution (mean 0, variance 0.01) to ensure that the parameters are within a reasonable range during the initial training phase. By querying this table, the topological feature vectors of a specified network layer and channel can be obtained. :

[0098] ;

[0099] For example, for the second convolutional layer ( =2) first channel ( =1), and its eigenvector is = For channels-insensitive layers (such as fully connected layers), the channel number c is fixed at 0, and the embedding vectors corresponding to all channels of this layer are bound and updated during training, meaning they share the same set of parameters, ensuring consistency of structural prior knowledge across channels. This vector is iteratively optimized along with the parameters of the SNN backbone and other perceptrons during model training, learning the structural characteristics of different locations in the network (different layers, different channels) through backpropagation. For example, convolutional layers require more time steps, while fully connected layers can be allocated fewer time steps of prior information.

[0100] (3) Semantic entropy estimator: within a time window (length) =5) Statistical analysis of neuron spur firing frequency within a time window, i.e., statistical analysis of the firing frequency of the first neuron within a time window. The firing frequency of each neuron (Number of pulses divided by window length). For a convolutional layer (number of neurons N=256), normalize the firing frequency of all neurons to form a discrete probability distribution. :

[0101] ;

[0102] in, This represents the total number of neurons in this layer. The aim is to ensure numerical stability and prevent division by zero errors when all neurons have not fired impulses.

[0103] Based on this probability distribution, the Shannon information entropy H(t) within this time window is calculated as a static measure of semantic complexity:

[0104] ;

[0105] Further calculation of entropy change characteristics of adjacent time windows :

[0106] ;

[0107] This entropy change characteristic This is the output of the semantic entropy estimator. An increase in entropy indicates increased information disorder and complexity; a decrease in entropy indicates that the information tends to be more ordered and the complexity decreases. When inputting the complex strokes of the digit "8," or when the input sample changes from the simple digit "1" to the complex digit "8," more neurons are typically needed to collaboratively participate in the representation in different modes, leading to an increased probability distribution. It becomes more uniform than the previous time step (when processing "1"). This makes the entropy value of the current window more uniform. The entropy value is significantly higher than that of the previous window. This results in a clearly positive value. This positive value is interpreted by the subsequent time step adjustment decision module as a key signal that "semantic complexity is increasing rapidly," and is one of the important bases for triggering the system to extend the inference time step.

[0108] (4) Impulse rate of change estimator: operates in the same network layer as the semantic entropy estimator, but focuses on the overall activity of the neuron population:

[0109] ;

[0110] in, This represents the total number of neurons in that layer. The time window length, Indicates the first At time step, one neuron The pulse firing state is determined; a double summation operation is used to count the total number of pulse firings of all neurons in this layer within the current time window, and then divided by the product of the total number of neurons and the window length to obtain the average pulse firing rate of the neuron population within this window. The value ranges from [0,1].

[0111] To capture the changing trend of activity, the estimator calculates the first-order absolute difference of the pulse rate between adjacent time windows as a feature of the pulse rate of change. :

[0112] ;

[0113] scalar characteristic This is the output of the pulse rate of change estimator. An increase in the value indicates a dramatic change in the overall activation level of the neuronal population, possibly corresponding to a shift in input information or a change in the network's internal state; a decrease in the value indicates that the activation level tends to stabilize. The transition from "1" to "8" is usually accompanied by a significant increase in the activity of the neuronal population. > Make It outputs a large positive value. This feature, from the perspective of energy (firing rate) variation, provides complementary validation for semantic entropy features.

[0114] Step S2: Time Step Adjustment Generation Stage

[0115] Using the state features output by the multiple perceptrons described in step S1 as input, the adjustment amount of each network layer at the next time step is dynamically generated through feature fusion and mapping. The specific implementation is as follows:

[0116] (1) State feature fusion: The above heterogeneous feature vectors are concatenated to form a joint representation vector from the output of S1. :

[0117] ;

[0118] The dimensions and batch adaptation logic of each feature are clearly defined: (B is the batch size, and 64 is the GRU hidden layer dimension). (16 is the dimension of the topological embedding vector). , (These are all scalar features, expanded to batch dimensions). The concatenation operation is performed along the feature dimensions, ultimately yielding a joint representation vector. (64+16+1+1=82 dimensions) This vector fully integrates multi-dimensional key information such as task temporal dynamics, network structure priors, and temporal dynamic characteristics of impulse activity. The temporal dynamic characteristics are characterized by changes in semantic complexity and changes in neuron population activity, providing comprehensive state awareness for subsequent layer-specific time step adjustment decisions.

[0119] (2) Regulation factor prediction: using MLP mapping To the adjustment coefficient The MLP structure is explicitly defined as "82-dimensional input layer → 32-dimensional hidden layer → 1-dimensional output layer". This coefficient represents the degree to which the time step of the corresponding network layer is scaled based on the current multi-state features.

[0120] ;

[0121] in, Dimensions , Dimensions , Dimensions , Dimensions (Output layer bias, initialized as an all-zero vector); Activation functions are used to introduce nonlinear mappings to alleviate the gradient vanishing problem, and their output values ​​are between [0, +∞). for Function, ensure output This makes the adjustment coefficient smooth and normalized.

[0122] (3) Time step demapping: The normalized adjustment coefficients are demapped. Mapped to actual usable integer time steps This is used to control the SNN inference process in the next time window of the corresponding network layer:

[0123] ;

[0124] in, and The preset minimum and maximum allowable time steps ( =5、 =20), This is a rounding function that converts continuous mapping results into integers, satisfying the SNN computation's strict requirement for integer time steps. For example, when... =5、 =20, if =0.6, then + =5 + 0.6 × 15 = 14, which is the next time window of this layer. The inference time step is 14.

[0125] (4) Iterative execution: Finally, the time step adjustment decision module outputs the predicted time step. This value is provided to the SNN inference engine. It will serve as the direct basis for adjusting the time window length in step S3. The entire S2 process is repeated at each time step or every N time steps to achieve dynamic, adaptive temporal scaling. For example, setting S2 to repeat every 5 time steps achieves dynamic adjustment.

[0126] Step S3: Time-step adjustment and model adaptive update phase

[0127] The core of this step is to form a closed loop of perception-decision-execution-optimization: execute the time step adjustment strategy predicted in step S2, evaluate the network performance it produces, and guide the entire system (including the perceptron in S1 and the time step adjustment decision module in S2) to evolve towards high precision, high efficiency, smoothness and stability by optimizing the multi-objective loss function.

[0128] (1) Dynamic reasoning: At the start of time step t+1, the system reads the integer time step predicted by step S2. And directly set it as the length of the next inference window, that is:

[0129] ;

[0130] in, The inference window length for the corresponding layer is defined as follows: [Window range is defined as follows] This interval is a continuous range of time steps without overlap or gaps, ensuring continuous processing of the pulse sequence. Within this window, the system iteratively calculates each layer of the SNN with a fixed discrete time step (usually 1 to ensure the precision of pulse propagation and avoid information loss), completing pulse accumulation and inter-layer propagation. Finally, at the end of the window, the outputs of each layer are summarized to produce the overall inference result for that time slice. For example, in the MNIST handwritten digit recognition task, the iterative SNN within the window completes pulse feature extraction and classification calculation, outputting the classification results for the digits "0-9".

[0131] (2) Multi-objective loss calculation: A comprehensive multi-objective loss function is defined. To simultaneously optimize multiple key performance aspects:

[0132] ;

[0133] in, , , For adjustable hyperparameters (such as =0.1、 =0.05、 =0.05), used to balance the importance of different optimization objectives. It can be flexibly adjusted according to task priority (such as low latency priority, high precision priority); after calculating the total loss, the gradient will penetrate the SNN backbone network, the time step adjustment decision module in step S2 and all perceptrons in step S1 through the backpropagation algorithm, optimizing all trainable parameters end-to-end and ensuring that each module is coordinated and adapted.

[0134] Specifically, classification loss For example, the actual label is the number "5"; efficiency loss. Used to penalize long step sizes; smoothing loss This is to penalize severe fluctuations in the time step, ensuring the stability of the inference process; consistency loss. Reference function Defined as: when >0.1 or When the norm is greater than the threshold, output a larger step size (e.g., 18); otherwise, output a smaller step size (e.g., 8).

[0135] (3) Parameter update: The Adam optimizer (learning rate 0.001) is used to synchronously update the SNN weights, time step adjustment decision module parameters (MLP weights) and perceptron parameters (GRU weights, embedding table) to achieve end-to-end training.

[0136] (4) Optimize convergence criteria: Set convergence conditions, such as "multi-objective total loss function". The parameter remains below the preset threshold of 0.001 for 5 consecutive rounds. If the above convergence condition is met, the iteration process is terminated and the inference result of the current round (such as the digit classification result) is output. If the convergence condition is not met, the process returns to the feature extraction stage of step S1 and continues to execute the closed-loop process of "feature extraction → time step adjustment generation → time step adjustment execution and model adaptive update stage" based on the currently updated parameters.

[0137] Finally, this effectively balances the accuracy and efficiency of SNN.

[0138] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A method for dynamic time step adjustment of a spiking neural network based on multi-state feature fusion, characterized in that, Includes the following steps: Step S1, Feature Extraction Stage: Three types of perceptrons are deployed in parallel to extract the running state features of the spiking neural network from different dimensions in real time; The three types of perceptrons are: (1) a task phase perceptron, used to extract task progress status features and capture the overall progress status of the task represented by the input pulse sequence from a macro time scale; (2) a topology perceptron, used to extract network topology status features and encode static structural information such as the inherent layer depth and channel size of the network; and (3) a temporal dynamic perceptron, used to extract the temporal dynamic characteristics of pulse activity. The temporal dynamic perceptron is composed of a semantic entropy estimator and a pulse change rate estimator, which work together to characterize the instantaneous dynamic characteristics inside the network from a micro time scale. Step S2, Time Step Adjustment Generation Stage: Based on the multi-state features obtained in Step S1, the multi-state features are fused and mapped by the time step adjustment decision module to generate the time step adjustment of the spiking neural network. Step S3, Time Step Adjustment Execution and Model Adaptive Update Stage: Based on the time step adjustment amount generated in step S2, the running time window length of the spiking neural network is dynamically adjusted and the running process is executed; based on the output results of the running process, a multi-objective performance constraint function is constructed, and the relevant model parameters are jointly updated according to the function, forming a closed-loop adjustment mechanism for time step adjustment amount generation, execution and model adaptive update.

2. The dynamic time step adjustment method for a spiking neural network based on multi-state feature fusion according to claim 1, characterized in that, Step S1 is achieved in the following manner: The overall process of step S1 is completed in parallel by three types of perceptrons, and the specific classification and component composition are as follows: the first type is the task stage perceptron, the second type is the topology perceptron, and the third type includes the semantic entropy estimator and the impulse change rate estimator. The four core components capture the running state characteristics of the spiking neural network from the macroscopic time scale, the static structure dimension, and the microscopic time scale, respectively. The semantic entropy estimator and the impulse change rate estimator work together to characterize the temporal dynamic characteristics of impulse activity.

3. The dynamic time step adjustment method for a spiking neural network based on multi-state feature fusion according to claim 2, characterized in that, The task phase perceptron is used to capture the macroscopic progress status features of the task, and is implemented in the following way: The task-stage perceptron takes the input pulse sequences of each layer of the spiking neural network as input; for the first layer... Layer, in time step The input pulse sequence is denoted as ,in For batch size, To input the number of time steps, This represents the feature dimension of the layer. The perceptron uses a GRU to update its internal state; the GRU updates its internal state at time steps. Hidden state ( The update process (for the hidden layer dimension) involves several key steps; first, the update gate is calculated. : ； In this formula, It is the hidden state of the previous time step, carrying historical information; symbols This represents a vector concatenation operation, which combines the historical state with the current input. Connect them; , Update the trainable weight matrix and bias vector corresponding to the gate; update the gate The closer the value is to 1, the more historical information is retained; Next, calculate the reset gate. : ； Reset the door using independent parameters and ; and The trainable weight matrix and bias vector corresponding to the reset gate, the reset gate The closer the value is to 0, the more it means that the generation of new states depends on the current input rather than the historical state; Then, the result of resetting the gate is used to calculate the candidate hidden state. : ； Here, the symbol ⊙ represents element-wise multiplication; reset gate Firstly, regarding the historical situation Multiplication allows for selective filtering of historical information; the filtered historical state is then multiplied by the current input. Concatenate and pass through a weight matrix bias Process and generate a candidate state that contains new information at the current time. ; Finally, combined with the updated gate and candidate hidden state Calculate the final hidden state at the current time step. : ； It updates the gate As a weight, historical states and candidate states Perform a weighted summation; if If it is close to 1, then it is a candidate state. The weights in the hidden state calculation are increased, while the historical state... The weight decreases accordingly; if If it is close to 0, then the final candidate state is... Increasing the weight in the hidden state calculation, historical state The weight of the hidden state at the current time step is increased accordingly; Output as a feature representation of the task's progress status.

4. The dynamic time step adjustment method for a spiking neural network based on multi-state feature fusion according to claim 2, characterized in that, The topology perceptron is used to encode the static structural information of the network, and is implemented in the following way: The topology perceptron is used to encode the static structure information of the SNN; for the first... Layer and its first Each channel defines its topological identity as a tuple. : Maintain a trainable parameter matrix ,in It is the total number of network layers. It is the maximum number of channels. It is the dimension of the embedded vector; By querying the embedding table, the corresponding topological feature vector can be obtained. : ； For layers that do not distinguish between channels, use layer numbers. Perform a query; this embedding vector During training, it is optimized together with other network parameters, thereby learning structural prior knowledge at different locations in the network.

5. The dynamic time step adjustment method for a spiking neural network based on multi-state feature fusion according to claim 2, characterized in that, The third type of sensor is used to characterize the temporal dynamics of impulse activity, and is implemented in the following way: (1) The semantic entropy estimator aims to quantify the semantic uncertainty of neuron population activity, characterized by: First, within a time window, the number of... The firing frequency of each neuron Subsequently, the firing frequencies of all neurons were normalized to obtain a probability distribution. : ； in, This represents the total number of neurons in this layer. It is a very small positive number to prevent division by zero errors; Based on this probability distribution, calculate the Shannon information entropy within this time window. As a static measure of semantic complexity: ； To capture the dynamic changes in complexity, the entropy change characteristics of adjacent time windows are further calculated. : ； This entropy change characteristic This is the output of the semantic entropy estimator; an increase in entropy value indicates an increase in information disorder and complexity; a decrease in entropy value indicates that information tends to be more ordered and the complexity decreases. (2) The pulse rate of change estimator is used to quantify the instantaneous activity change of pulse activity. The impulse rate of change estimator aims to quantify the instantaneous dynamic changes of neuronal population activity at the signal intensity level, serving as an effective supplement to the information content changes measured by semantic entropy. First, a time window is defined; within the window, the impulse rate of change is calculated. Average pulse firing rate of layer neuron population : ； in, This represents the total number of neurons in that layer. The time window length, Indicates the first A neuron at time step The pulse delivery status; To capture the changing trend of activity, the estimator calculates the first-order absolute difference of the pulse rate between adjacent time windows as a feature of the pulse rate of change. : ； scalar characteristic This is the output of the pulse rate of change estimator. An increase in the value indicates a drastic change in the overall activation level of the neuronal population, while a decrease in the value indicates that the activation level tends to stabilize.

6. The dynamic time step adjustment method for a spiking neural network based on multi-state feature fusion according to claim 1, characterized in that, Step S2 is achieved in the following manner: Its core is the time step adjustment decision module; this module takes the features output by the multiple perceptrons described in step S1 as input, and dynamically generates the adjustment amount of each network layer in the next time step through state feature fusion and mapping. Before the implementation of step S2, based on the output of step S1, its input features are clearly defined as: (1) Task progress status features from the task phase perceptron: (2) Semantic complexity variation characteristics from the semantic entropy estimator: (3) Network structure characteristics from the topology perceptron: (4) Instantaneous activity characteristics from the pulse rate of change estimator: ; The S2 process can be broken down into the following three sub-steps: (1) State feature fusion: First, the heterogeneous feature vectors are concatenated to form a unified joint representation vector. : ； (2) Prediction of time step adjustment: The fused joint feature vector Input a lightweight multilayer perceptron (MLP) or linear mapping layer to predict a scalar regulation coefficient. : ； in, , , , These are trainable parameters; For activation functions; for function; (3) Time step demapping: The predicted adjustment coefficient Mapped to actual usable integer time steps The mapping function is used to control the SNN inference process in the next time window; it is defined as follows: ； in, and The preset minimum and maximum allowable time steps, It is a rounding function; (4) Output and Iteration: Finally, the time step adjustment decision module outputs the predicted time step. This value is given to the SNN inference engine; it will serve as the direct basis for adjusting the time window length in step S3; the entire S2 process is repeated at each time step or every N time steps to achieve dynamic and adaptive temporal scale adjustment.

7. The dynamic time step adjustment method for a spiking neural network based on multi-state feature fusion according to claim 1, characterized in that, Step S3 is achieved in the following manner: (1) Dynamic time window inference execution: Transform the decision made in step S2 into the actual reasoning behavior of the SNN; in time step Initially, the system reads the integer time step predicted in step S2. ; The inference process of SNN is organized into a sliding time window of variable length; Window length It is dynamically determined by the following formula: ； That is, the predicted value is used directly as the length of the next inference window; the system then uses this window... Within the window, the SNN is iteratively calculated with a fixed discrete time step optimized by step S2 to complete the accumulation and propagation of pulses, and finally the inference result of the time slice is produced at the end of the window. (2) Multi-target loss calculation and backpropagation: Defined at time step Multi-objective loss function The weighted sum of the following four loss terms: ； in, , , These are adjustable hyperparameters used to balance the importance of different optimization objectives. After calculating the total loss, the gradient is used through backpropagation to not only update the classification weights of the SNN, but also to adjust the decision module in step S2 and the perceptron in step S1, thus optimizing all trainable parameters end-to-end. (3) Iterative update of model parameters and closed-loop feedback: based on The calculated gradient is used by the optimizer to synchronously update the following three parameters: First, update the parameters of the SNN backbone network to improve its basic classification ability; second, update the parameters of the time step adjustment decision module in step S2 to enable it to make better time step decisions; finally, update the parameters of each perceptron in step S1 to enable it to extract more useful features for decision-making and optimization. (4) Optimize convergence judgment: Compare the current loss value with the convergence condition to determine whether the model has reached the optimization convergence state; if the convergence condition is met, terminate the iteration process and output the final inference result; if the convergence condition is not met, repeat the above steps of "model parameter iterative update and closed-loop feedback".

8. The dynamic time step adjustment method for a spiking neural network based on multi-state feature fusion according to claim 7, characterized in that, The four loss terms of the multi-objective loss function in step (2) are constructed as follows: (1) Classification accuracy loss Encourage SNNs to output correct predictions: ； in, It is the cross-entropy loss function. This represents the predicted probability distribution of the output based on the input data for the current time slice. Input the true label of the data for the current time slice; (2) Reasoning efficiency loss Encourage shorter average inference time by directly utilizing the output of step S2. ； in, The sliding window size used to calculate the average time step is used; this loss prompts the time step adjustment decision module in step S2 to predict smaller time steps, thereby reducing the computational cost. (3) Temporal smoothness loss The penalty is for severe fluctuations in the time step, ensuring the stability of the inference process; this loss directly affects the predicted sequence in step S2. ； (4) Perception-Decision Consistency Loss: Connecting steps S1, S2, and S3; encouraging the decision made in step S2 to be logically consistent with the semantic dynamics and task progress perceived in step S1: ； in, Based on semantic entropy changes Pulse change rate Task progress characteristics The constructed differentiable function has an output logic consistent with the information processing requirements of the spiking neural network; this loss term ensures that the decisions of the time-step adjustment decision module do not deviate from the basic perception logic.

Images