Memristor SNN multi-mode fault tolerance method based on multi-task learning

Abstract

The invention provides a memristor SNN multi-mode fault tolerance method based on multi-task learning. The method comprises the following steps: step 1, constructing a multi-noise level task set, and simulating space fixed faults with different densities and distributions in a memristor array; 2, training a shared SNN backbone network model through a multi-task learning framework based on the multi-noise-level task set, and optimizing the universal feature extraction capability and robustness of the model; 3, an independent noise adaptation layer is configured for each noise task in the multi-noise-level task set and used for generating specific pulse modulation parameters of the task; step 4, the trained shared SNN backbone network is combined with the independent adaptation layer of each noise task, so that the backbone network can cooperatively work with the corresponding pulse modulation parameter when processing the input of any noise task, and robust reasoning across noise levels is realized; and step 5, deploying the SNN model parameters after joint optimization and the pulse modulation parameters of each noise task to a memristor cross array of a pulse-driven storage and calculation integrated architecture for hardware implementation. According to the method, through a multi-task learning framework and noise perception loss design, model trunk road training and specific task pulse modulation parameter adaptation are used for sending, and the robustness of SNN in a complex SAF environment is remarkably improved.

Description

Technical Field

[0001] This invention relates to a multimodal fault tolerance method for memristors, specifically a multimodal fault tolerance method for memristors based on multi-task learning SNN. Background Technology

[0002] Spiking Neural Networks (SNNs), with their event-driven characteristics and biological interpretability, have significant application value in low-power edge computing, neuromorphic chips, and high real-time processing systems. However, SNNs often rely on memristor crossbar arrays as in-memory computing carriers in hardware implementation. These devices are prone to fixed-type failures (SAFs) due to manufacturing defects and long-term operational degradation. SAFs manifest as abnormal locking of memristor cell conductance values ​​to high or low resistance states, and drifting of pulse firing thresholds, severely interfering with the spatiotemporal encoding mechanism of pulses in SNNs, resulting in information transmission distortion, decreased feature extraction capabilities, and unstable system output.

[0003] Existing fault-tolerance methods for SAF have the following limitations:

[0004] Traditional retraining methods rely on hardware redundancy or online parameter calibration to retrain the model. While this can partially restore performance, it requires frequent storage access, incurs high hardware overhead, and struggles to handle dynamic, high-density fault scenarios. Furthermore, these methods typically ignore the event-driven and impulse-sequential characteristics unique to SNNs, leading to significant performance degradation when processing spatiotemporal information tasks.

[0005] Fault-tolerant weight mapping based on artificial neural networks (ANNs): This method directly maps ANN weights to a memristor array and mitigates the effects of spontaneous emission (SAF) using redundancy or error correction circuitry. While this method offers some protection for ANNs, it cannot be adapted to the pulse timing coding mechanism of spontaneously generated neural networks (SNNs). For example, its static weight mapping method struggles to handle the temporal distribution differences in pulse delivery within SNNs, resulting in insufficient reliability in spatiotemporal tasks such as video and audio processing.

[0006] Furthermore, existing technologies generally suffer from the following problems:

[0007] Poor adaptability to single noise scenarios: Most methods are designed only for specific fault densities (such as 5%-10%), and are difficult to generalize to high noise environments (such as SAF above 30%).

[0008] Lack of time-series modeling capability: The time pulse sequence characteristics of SNN are not fully explored, resulting in insufficient optimization of pulse synchronization and time compactness by the fault compensation strategy.

[0009] Limited hardware compatibility: Traditional methods often rely on complex correction circuits or redundant designs, making it difficult to adapt to the low-power requirements of pulse-driven in-memory computing architectures. Summary of the Invention

[0010] This invention proposes a multi-mode fault-tolerant method for memristor SNN based on multi-task learning. By combining multi-noise level task modeling with pulse-driven in-memory computing integrated design, the reliability and fault tolerance of the system under non-ideal device conditions are improved.

[0011] The technical solution to achieve the purpose of this invention is as follows:

[0012] A multi-modal fault-tolerant method for memristor SNN based on multi-task learning includes:

[0013] Step 1: Construct a multi-noise-level task set to simulate spatially fixed faults with different densities and distributions in a memristor array;

[0014] Step 2: Based on the multi-noise level task set, train a shared SNN backbone network model using a multi-task learning framework;

[0015] Step 3: Configure an independent noise adaptation layer for each noise task in the multi-noise level task set to generate the pulse modulation parameters of the task;

[0016] Step 4: Combine the trained shared SNN backbone network model with the noise adaptation layer of each noise task, so that the backbone network can work in conjunction with the corresponding pulse modulation parameters when processing the input of any noise task, thereby achieving robust inference across noise levels.

[0017] Step 5: Deploy the jointly optimized SNN backbone network model parameters and the pulse modulation parameters of each noise task to a memristor cross array of a pulse-driven in-memory computing architecture for hardware implementation.

[0018] Furthermore, the construction of the multi-noise-level task set specifically includes: first, dividing the noise into K noise tasks according to multiple preset fault density levels, where the fault density range covers 5% to 30%; then, generating a spatial fault distribution template corresponding to each noise task based on the Poisson distribution; finally, using a generative adversarial network containing LSTM to extend the spatial fault distribution template in terms of spatiotemporal dimension to simulate complex fault modes and generate a dataset for multi-task training.

[0019] Furthermore, a shared SNN backbone network model is trained using a multi-task learning framework, specifically including: [the following is a list of SNN model parameters]. Initialize the backbone network; input all task data from the multi-noise-level task set into the backbone network for forward propagation; construct a global multi-task loss function. Optimize backbone network parameters.

[0020] Furthermore, the global multi-task loss function for:

[0021] ;

[0022] In the formula, K represents the total number of tasks. It is the dynamic weight of the k-th task. The time-encoded cross-entropy loss is the value corresponding to the k-th task. For the pulse synchronization loss corresponding to the k-th task, This is a measure of the difference in pulse timing distribution corresponding to the k-th task. For noise smoothing regularization, , These are the weights for time-coded cross-entropy loss and pulse synchronization loss, respectively.

[0023] Furthermore, the temporal encoding cross-entropy loss corresponding to the k-th task for:

[0024] ;

[0025] in, s represents the true label of the i-th sample in the k-th noisy task; ki ( Let be the pulse firing intensity of the i-th sample in the k-th task at time step . , These represent the time steps for the inner and outer loop summation, respectively; T is the length of the time window.

[0026] The pulse synchronization loss corresponding to the k-th task is:

[0027] ;

[0028] Among them, s k (t) represents the pulse firing intensity of the k-th noise task at time step t.

[0029] Furthermore, SNN model parameters Updated using gradient descent:

[0030] ;

[0031] Where α is the global learning rate and β is the weight decay coefficient, which are used during parameter optimization. This represents the gradient.

[0032] Furthermore, step 3 specifically includes: each noise adaptation layer maintaining a set of independent pulse modulation parameters for the corresponding k-th task. The parameters include at least the pulse firing time window length. and pulse firing threshold offset During training, the pulse modulation parameters are optimized simultaneously. With backbone network parameters Through the global multi-task loss function The backpropagation generates the parameter set for the task.

[0033] Furthermore, in step 4, the trained shared SNN backbone network model is combined with the noise adaptation layers of each noisy task. Specifically, this includes: for the input data of the k-th noisy task, the shared SNN backbone network model first performs feature calculation; during the calculation process of the SNN backbone network model, the dynamic behavior of its spiking neurons is determined by the noise adaptation layer parameters corresponding to the current task. Modulation is performed, specifically by using Controlling the time window for membrane potential integration, using Adjust the firing threshold of neurons.

[0034] Furthermore, step 5 specifically includes:

[0035] The weights of the shared SNN backbone network model are mapped to the conductance values ​​of each memristor in the memristor cross array; the pulse modulation parameters of each noise adaptation layer are... Stored in a separate configuration register; during inference, the corresponding configuration parameters are retrieved based on the noise environment. The pulse firing behavior of LIF neuron circuits on a modulated memristor array.

[0036] Furthermore, a surrogate gradient function is used during the training phase of the SNN backbone network model. Approximation, proxy gradient function for:

[0037] ;

[0038] Where a is an adjustable parameter, and X is the difference between the neuron membrane potential and the threshold.

[0039] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0040] This invention addresses the challenge of fixed-fault fault tolerance faced by spiking neural networks in in-memory computing hardware, proposing a systematic solution covering the entire process of modeling, training, and deployment. This solution overcomes existing technological bottlenecks through innovative designs at three levels. In dynamic noise modeling, a spatiotemporal joint fault generation method is proposed. By defining a fault density range of 5%-30% and a time window length of 3-5 pulse periods, a hybrid network of spatiotemporal convolutional sliding sampling and LSTM-GAN is combined to construct highly complex training samples capable of simulating the coupling effect between non-ideal device characteristics and pulse events. Regarding the multi-task training mechanism, the hardware adaptation process is innovatively transformed into a task optimization problem across noise levels. A hybrid parameter-sharing architecture and dynamic weight aggregation strategy are adopted. While maintaining parameter sharing in the backbone network, independent pulse modulation parameters are generated for each task through a noise adaptation layer. A multi-task loss function including time-coded cross-entropy, noise smoothing regularization, and uncertainty-driven weights is introduced, explicitly improving the model's temporal robustness and cross-task generalization ability. At the hardware deployment level, the gradient non-differentiability problem of discrete pulses in SNN is effectively solved by the surrogate gradient method. Combined with time compactness regularization, the convergence efficiency of the memristor array within a finite time window is significantly improved. Finally, parallel execution of parameter update and pulse delivery is realized on the pulse-driven in-memory computing architecture, which greatly reduces the system energy consumption. Attached Figure Description

[0041] Figure 1 This is a flowchart illustrating the overall multi-task learning process in an embodiment of the present invention.

[0042] Figure 2 This diagram illustrates the impact of introduced noise in an embodiment of the present invention.

[0043] Figure 3 This invention provides methods for constructing different noise tasks according to its embodiments.

[0044] Figure 4 This is a diagram illustrating the multi-task learning process in an embodiment of the present invention.

[0045] Figure 5 This is a schematic diagram of the circuit implementation structure of an embodiment of the present invention.

[0046] Figure 6 This is a T-SNE diagram showing the improved training process in an embodiment of the present invention.

[0047] Figure 7 This is a diagram illustrating the noise resistance performance of the optimized model in this embodiment of the invention. Detailed Implementation

[0048] To make the technical solutions and advantages of the present invention clearer, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0049] This invention overcomes the bottlenecks of existing technologies through the following innovative points:

[0050] (1) Construction of multi-noise level tasks: SAFs with different densities and distributions are abstracted into multiple noise level tasks, covering 5% to 30% of fault density, and a multi-task training set is constructed to simulate real fault environments.

[0051] (2) Joint training of shared backbone and adaptation layer: Design an SNN backbone network with shared parameters to extract cross-task general features; set up an independent noise adaptation layer for each noisy task to generate task-specific pulse modulation parameters (such as time window length and threshold offset).

[0052] (3) Noise-aware multi-task loss aggregation: A dynamically weighted multi-task loss function is proposed, which integrates time-encoded cross-entropy, impulse synchronization constraints and noise smoothing regularization terms to explicitly enhance the temporal robustness and cross-task generalization performance of SNN.

[0053] (4) Backbone network and adaptation layer collaborative reasoning: During the reasoning phase, the backbone network is combined with the adaptation layer parameters of the corresponding task to realize the adaptive pulse modulation of the same network for different noise environments.

[0054] (5) Pulse-driven in-memory computing deployment: The optimized network parameters are deployed on a memristor cross array. The surrogate gradient method is used to solve the gradient non-differentiability problem caused by the pulse discreteness of SNN. The computational efficiency within a finite time window is improved by time compaction regularization.

[0055] See Figure 1 Based on the above innovations, this invention provides a multi-modal fault-tolerant method for memristor SNN based on multi-task learning, the specific steps of which are as follows:

[0056] Step 1: Construct a multi-noise-level task set to simulate spatially fixed faults with different densities and distributions in a memristor array;

[0057] Step 1 first divides the noise task into K noise tasks based on multiple preset fault density levels, with the fault density range covering 5% to 30%; then, it generates a spatial fault distribution template corresponding to each noise task based on a Poisson distribution; finally, it uses a generative adversarial network containing LSTM to expand the spatiotemporal dimension of the fault distribution template to simulate complex fault modes and generate a dataset for multi-task training.

[0058] Step 2: Based on the multi-noise level task set, train a shared SNN backbone network model through a multi-task learning framework to optimize its general feature extraction capability and robustness.

[0059] Step 2 first involves setting the parameters of a standard SNN model. The backbone network is initialized; then, all task data from the multi-noise-level task set are input into the backbone network for forward propagation; finally, a global multi-task loss function is constructed. This loss function is used to optimize the backbone network parameters and is a weighted sum of the losses from each noisy task:

[0060] ;

[0061] In the formula It is the dynamic weight of the k-th task, while the other losses consist of time-coded cross-entropy loss, impulse synchronization loss, impulse temporal distribution difference measure, and noise smoothing regularization term.

[0062] Considering the temporal impulse characteristics of SNNs, the temporal encoded cross-entropy loss is defined as:

[0063] ;

[0064] Where: y ki s represents the true label of the i-th sample in the k-th noisy task; ki (t) represents the pulse firing intensity of the i-th sample in the k-th task at time step t; T is the length of the time window.

[0065] Introducing a time compactness constraint, the pulse synchronization loss is defined as:

[0066] ;

[0067] Among them, s k (t) represents the pulse firing intensity of the k-th noisy task at time step t, where T is the length of the time window. This loss forces the model to converge quickly within a finite time window.

[0068] A measure of pulse timing distribution difference is defined based on Wasserstein distance:

[0069] ;

[0070] Wherein, P(sk) is the pulse time sequence distribution of the k-th noisy task. For an ideal pulse timing distribution, W(·,·) represents the Wasserstein distance.

[0071] To constrain the output stability of tasks with adjacent noise levels, a noise smoothing regularization term is introduced:

[0072] ;

[0073] Where yk is the output of the k-th task.

[0074] Update the backbone network parameters using gradient descent. :

[0075] ;

[0076] Where α is the global learning rate and β is the weight decay coefficient. During parameter optimization, both the static weight mapping error of the ANN and the dynamic temporal encoding error of the SNN are considered, and joint optimization ensures the model's robustness in both the weight space and temporal space.

[0077] Step 3: Configure an independent noise adaptation layer for each noise task in the multi-noise level task set to generate pulse modulation parameters specific to that task;

[0078] Step 3 involves configuring an independent noise adaptation layer for each noise task. Specifically, each noise adaptation layer maintains a set of independent pulse modulation parameters for the corresponding k-th task. The parameters include at least the pulse firing time window length. and pulse firing threshold offset During training, the pulse modulation parameters are optimized simultaneously. With backbone network parameters Through the global multi-task loss function The backpropagation generates a task-specific set of parameters.

[0079] Step 4: Combine the trained shared SNN backbone network with the independent adaptation layers of each noisy task, so that the backbone network can work in conjunction with the corresponding pulse modulation parameters when processing the input of any noisy task, and achieve robust inference across noise levels.

[0080] In step 4, the joint operation of the backbone network and the adaptation layer is as follows: For the input data of the k-th noisy task, the shared SNN backbone network first performs feature calculation; during the calculation process of the backbone network, the dynamic behavior of its spiking neurons is determined by the noise adaptation layer parameters corresponding to that task. Modulation is specifically manifested in: using Controlling the time window for membrane potential integration, using Adjust the firing threshold of neurons; through the modulation, the same backbone network can adapt to the pulse timing characteristics under different noisy tasks.

[0081] Step 5: Deploy the jointly optimized SNN model parameters and the pulse modulation parameters of each noise task to a memristor cross array of a pulse-driven in-memory computing architecture for hardware implementation.

[0082] 3. The hardware implementation method for pulse-driven in-memory computing in step 5: Mapping the weight values ​​of the shared SNN backbone network to the conductance values ​​of each memristor in the memristor cross array; mapping the pulse modulation parameters of each noise adaptation layer... Stored in a separate configuration register; during inference, the corresponding configuration parameters are retrieved based on the noise environment in which the system operates. The pulse firing behavior of LIF neurons on a modulated memristor array is observed; to address the gradient problem caused by the discreteness of SNN pulses, a surrogate gradient function is employed during the training phase. An approximation is made, defined as follows:

[0083] ;

[0084] Where a is an adjustable parameter, and X is the difference between the neuron membrane potential and the threshold.

[0085] Please see Figure 2 , Figure 2 An illustration of the impact of introduced noise in an embodiment of the present invention. (See diagram below.) Figure 2 As shown, due to the introduction of SAF weight noise, the cumulative membrane potential of the pulse released by the previous neuron in the input of the next neuron is affected, which in turn affects the timing of the output pulse of the subsequent neuron. The pulse is released earlier (or later), which will affect the operation of the entire network.

[0086] Please see Figure 3 , Figure 3 This is a method for constructing different noise tasks according to an embodiment of the present invention, corresponding to step 1. As shown in the left half of the figure, the method first generates k-level noise convolution kernels based on Poisson distribution and constructs the spatiotemporal correlation of fault clusters through sliding sampling; on the right, an adversarial generative network (LSTM-GAN) is used to expand the diversity of noise patterns, with the generator taking Gaussian noise as input.

[0087] Please see Figure 4 , Figure 4 This is a schematic diagram of training a shared SNN backbone network in an embodiment of the present invention. It illustrates the process from task generation, task sampling, obtaining Ltime, Lsync, LSPD, and Lsmooth for each task, to aggregation to obtain Lglobal and then updating the backbone network.

[0088] Please see Figure 5 , Figure 5This is a circuit implementation diagram in an embodiment of the present invention. The circuit is used to load weight parameters optimized based on multi-task learning and to execute the spatiotemporal computation characteristics relied upon by the Spiking Neural Network (SNN). Through co-design from algorithm to hardware, this architecture maintains high robustness in the face of SAF (Simultaneous Automatic Fault) while achieving significant improvements in energy efficiency. The core of the system adopts an RRAM (Resistive Random Access Memory) cross-array structure, used as the analog storage and parallel computing unit for weights. The orange highlighted units in the diagram represent memristors with SAF faults, whose conductance is permanently locked in a high-resistivity state (HRS, typical value RHRS=100kΩ) or a low-resistivity state (LRS, RLRS=10kΩ), simulating common non-ideal states in real-world devices. The input signal first passes through an 8-bit pulse width modulation (PWM) encoding module, being converted into a digital pulse sequence with time characteristics. The pulse width is linearly related to the input value (range 20–200ns), thus embedding analog information into the time dimension. Subsequently, the driving circuit injects the pulse signal into the row lines of the RRAM array through a current mirror array, achieving a current matching accuracy of ±10% and ensuring stable transmission of the input signal. Unlike traditional ANN in-memory computing architectures, this design deeply aligns with the event-driven characteristics of SNNs: the analog current output from the RRAM array column lines is directly input to the Leaky Integrate-and-Fire (LIF) neuron circuit. This circuit integrates the input current using an RC filter to obtain the membrane potential. When the integrated voltage reaches a set threshold Vth, the voltage comparator triggers the output pulse, completing a full pulse firing process. This design eliminates the high-power analog-to-digital converter (ADC) stage found in traditional architectures, achieving a true "pulse-to-pulse" processing flow and significantly reducing overall energy consumption.

[0089] Please see Figure 6 , Figure 6 This is a T-SNE diagram showing the improved training process in this embodiment of the invention. Taking an MLP model as an example, by comparing the dimensionality reduction diagram of the output feature vector of the first fully connected layer, and observing the initial model output with added noise and the model output with added noise after training adaptation, it can be seen that the trained model can recover a certain classification ability in the presence of noise, with the intra-class distance reduced and the inter-class distance increased.

[0090] Please see Figure 7 , Figure 7This invention provides an embodiment of the optimized model's noise resistance. The figure illustrates the noise resistance performance of the optimized model in this embodiment. We compared the performance of the original model with two existing methods under 0% to 30% SAF noise conditions. Experimental results show that the model trained with our optimization exhibits good adaptability and robustness when facing different proportions of SAF noise. This means that our model can not only effectively resist error interference caused by SAF, but also has significantly enhanced generalization ability under different noise environments. Importantly, this improvement does not rely on additional error correction circuitry, thus simplifying system design and reducing costs. This demonstrates that the optimization strategy proposed in this solution can significantly improve the reliability of neural network models without increasing hardware complexity.

[0091] In summary, this invention provides a multi-modal fault-tolerant method for memristor-based SNNs based on multi-task learning. By dividing tasks into multiple noise levels and employing a mask-driven adversarial training mechanism, the traditional hardware-dependent single-cycle training process is transformed into a transferable multi-task joint optimization framework. This method effectively improves the robustness and generalization ability of spiking neural networks (SNNs) in the presence of high SAF fault density, significantly enhancing classification accuracy and model stability without introducing additional hardware resources. Experiments show that, compared to traditional retraining methods, this approach significantly reduces hardware resource overhead while maintaining performance, demonstrating promising engineering application prospects.

[0092] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-task learning-based memristor SNN multi-modal fault-tolerant method, characterized in that, The method comprises the following steps: Step 1: constructing a multi-noise-level task set to simulate different densities and distributions of spatial fixed faults in the memristor array; Step 2: training a shared SNN backbone network model through a multi-task learning framework based on the multi-noise-level task set; Step 3: configuring an independent noise adaptation layer for each noise task in the multi-noise-level task set to generate pulse modulation parameters of the task; Step 4: jointly optimizing the trained shared SNN backbone network model and the noise adaptation layer of each noise task, so that the backbone network works with the corresponding pulse modulation parameters when processing the input of any noise task, and realizes robust reasoning across noise levels; Step 5: deploying the optimized SNN backbone network model parameters and the pulse modulation parameters of each noise task to a memristor cross array of a pulse-driven storage-computing integrated architecture for hardware implementation.

2. The multi-task learning based memristor SNN multi-modal fault-tolerant method according to claim 1, characterized in that, The construction of the multi-noise-level task set specifically comprises the following steps: first, dividing K noise tasks according to a plurality of preset fault density levels, wherein the fault density range covers 5% to 30%; then generating a spatial fault distribution template corresponding to each noise task based on a Poisson distribution; finally, expanding the spatial fault distribution template in time and space dimensions by using a generative adversarial network containing LSTM to simulate complex fault modes and generate a data set for multi-task training.

3. The multi-task learning based memristor SNN multi-modal fault-tolerant method according to claim 1, characterized in that, Training a shared SNN backbone network model using a multi-task learning framework specifically includes: [The text abruptly ends here, likely due to an incomplete sentence or missing information.] Initialize the backbone network; input all task data from the multi-noise-level task set into the backbone network for forward propagation; construct a global multi-task loss function. Optimize backbone network parameters.

4. The multi-task learning based memristor SNN multi-modal fault-tolerant method according to claim 3, characterized in that, The global multi-task loss function is: ； where K is the total number of tasks, is the dynamic weight of the kth task, is the time encoding cross-entropy loss corresponding to the kth task, is the pulse synchronization loss corresponding to the kth task, is the pulse timing distribution difference measure corresponding to the kth task, is the noise-smoothing regularization term, , are the weights of the time encoding cross-entropy loss and the pulse synchronization loss, respectively.

5. The multi-task learning based memristor SNN multi-modal fault-tolerant method according to claim 4, characterized in that, The time encoding cross-entropy loss corresponding to the kth task is: ； wherein, is the true label of the i-th sample in the k-th noisy task; s ki ( ) is the spike intensity of the i-th sample in the k-th task at time step t, 、 denote the time steps of inner and outer loop summation, respectively; T is the length of the time window. The corresponding pulse synchronization loss of the kth task is ； where s k (t) is the spike rate of the kth noise task at time step t.

6. The multi-task learning based memristor SNN multi-modal fault tolerance method according to claim 3, characterized in that, SNN model parameters is updated by gradient descent method as: ； where a is a global learning rate, β is a weight decay coefficient, and denotes the gradient.

7. The multi-task learning based memristor SNN multi-modal fault-tolerant method according to claim 3, characterized in that, The step 3 specifically comprises: each noise adaptation layer maintains a set of independent pulse modulation parameters for the corresponding kth task , the parameters at least including pulse firing time window length and pulse firing threshold offset ; in the training process, the pulse modulation parameters and backbone network parameters are optimized synchronously, and the parameter set of the task is generated through the back propagation of the global multi-task loss function .

8. The multi-task learning based memristor SNN multi-modal fault-tolerant method according to claim 7, characterized in that, In step 4, the trained shared SNN backbone network model is combined with the noise adaptation layer of each noise task, specifically: for the input data of the kth noise task, first the feature calculation is performed by the shared SNN backbone network model; in the calculation process of the SNN backbone network model, the dynamic behavior of the pulse neuron is modulated by the noise adaptation layer parameters corresponding to the current task , specifically: using to control the time window of membrane potential integration, using to adjust the firing threshold of the neuron.

9. The multi-task learning based memristor SNN multi-modal fault-tolerant method according to claim 8, characterized in that, The step 5 specifically comprises: mapping the weight values of the shared SNN backbone network model to conductance values of each memristor in the memristor crossbar array; mapping the pulse modulation parameters of each noise adaptation layer to the pulse modulation parameters of the LIF neuron circuit on the memristor array are stored in independent configuration registers; during inference, the corresponding configuration parameters are called according to the noise environment , modulate the pulse firing behavior of the LIF neuron circuit on the memristor array.

10. The multi-task learning based memristor SNN multi-modal fault-tolerant method according to claim 9, characterized in that, In the SNN backbone network model training stage, a proxy gradient function is adopted Approximation is performed on the proxy gradient function is ； Wherein a is an adjustable parameter, and X is the difference between the membrane potential of the neuron and the threshold value.

Images