A fault injection method, system and device for SNN neurons

Abstract

The application discloses a fault injection method, system and device for SNN neurons, and relates to the field of artificial intelligence. According to the network topology structure, the last layer is determined as a reference layer and the layer before the reference layer is determined as an evaluation layer. According to the type of the reference layer, feature data is obtained, and the feature data includes at least one of the output pulse number of each neuron under the reference layer, the output pulse number of each neuron under the evaluation layer, the connection weight between each neuron under the reference layer and each neuron under the evaluation layer, and the score of each neuron under the reference layer. According to a preset evaluation strategy corresponding to the type, the feature data is processed to determine the score of each neuron under the evaluation layer, which represents the importance of each neuron. According to the score, the neurons are sorted to determine a target neuron according to the sorting result. The score of each neuron is determined layer by layer, so that the target neuron with higher importance is operated during subsequent fault injection, the key neurons are directly hit, and the safety of the SNN network is ensured.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, system, and apparatus for injecting faults into SNN neurons. Background Technology

[0002] Spiking Neural Networks (SNNs) are neural networks inspired by the basic operating principles of biological neurons. Their working principle is as follows: when a pulse reaches a postsynaptic neuron, the neuron's membrane potential changes. As pulses are continuously received until the membrane potential exceeds the neuron's threshold potential, the postsynaptic neuron also outputs a pulse. Subsequently, the neuron's membrane potential resets to its resting potential. The synaptic weight determines the rate of increase in membrane potential, and the relationship between the input pulse and the neuron's membrane potential change after passing through the synapse is determined by the neuron's impulse response equation. Therefore, neurons, synapses, and connection methods are the three important factors constituting an SNN. Common SNN models include the LIF model and the Izhikevich model. Currently, SNNs are widely used in artificial intelligence fields such as image recognition, speech recognition, and natural language processing.

[0003] In practical applications, although SNN hardware has good fault tolerance, due to the large number of neurons in the network, faults may occur in these neurons. This may manifest as neurons becoming more or less likely to fire pulses, or neurons being unable to fire pulses or firing pulses continuously. Accumulated faults can have a significant impact on the performance of SNN. Therefore, in practical applications, fault injection is used to simulate fault scenarios in the neurons of SNN.

[0004] However, existing fault injection methods mainly involve randomly selecting neurons for injection, or injecting neurons in descending order of pulse count, to detect the decline in the accuracy of the SNN network. However, this single fault injection method is not very effective. For example, when using the pulse count-based injection method for deep SNN networks, the injection results show that the network accuracy decreases more slowly with small injections than with the randomly selected neuron injection method. It is evident that the current fault injection method cannot accurately characterize the importance of neurons at each level, making subsequent fault injection unable to directly target key neurons, which is not conducive to the hardening of SNNs.

[0005] Therefore, how to provide an effective fault injection scheme for SNN neurons is an urgent problem to be solved. Summary of the Invention

[0006] In view of this, the present invention provides a method, system and device for fault injection into SNN neurons, which can determine the score of each intermediate layer of neurons by derivation, so that subsequent fault injection can directly target key neurons, which is conducive to taking corresponding measures to protect these important neurons, improving the fault tolerance of SNN network and ensuring the security of SNN network.

[0007] To address the aforementioned technical problems, this application provides a method for fault injection into SNN neurons, comprising:

[0008] S11: Based on the network topology of the SNN, determine the last layer under the network topology as the baseline layer, and determine the layer before the baseline layer as the layer to be evaluated;

[0009] S12: Obtain feature data according to the type of the reference layer. The feature data includes at least one of the following: the number of output pulses of each neuron in the reference layer, the number of output pulses of each neuron in the layer to be evaluated, the connection weight between each neuron in the reference layer and each neuron in the layer to be evaluated, and the score of each neuron in the reference layer.

[0010] S13: Process the feature data according to a preset evaluation strategy corresponding to the type to determine the score representing the importance of each neuron in the layer to be evaluated;

[0011] S14: Sort the neurons in the layer to be evaluated according to their scores, so as to determine the target neurons for fault injection based on the sorting results;

[0012] S15: Determine whether the current layer to be evaluated is the second layer. If not, proceed to S16.

[0013] S16: According to the network topology, determine the current layer to be evaluated as the new baseline layer, and the layer before the current layer to be evaluated as the new layer to be evaluated, and return to S12.

[0014] Furthermore, when the type of the reference layer is an output layer and the type of the layer to be evaluated is a LIF neuron layer, the feature data includes the number of output pulses of each neuron in the LIF neuron layer and the connection weights between each neuron in the layer to be evaluated and each neuron in the output layer.

[0015] Step S13 includes:

[0016] Based on the first preset relational expression and the feature data, the score of each neuron in the LIF neuron layer is determined; the first preset relational expression is:

[0017]

[0018] Where a is the index of each neuron in the LIF neuron layer and A is the total number of neurons, where A is an integer greater than 1. a Let be the number of output pulses of the a-th neuron, and scoreLIF. a The score of the a-th neuron in the LIF neuron layer;

[0019] b is the index of each neuron in the output layer, and B is the total number of neurons, where B is an integer greater than 1. This represents the boost weight after subtracting the minimum value from the connection weight between neuron a and neuron b, where the minimum value is the minimum value among the connection weights between neuron a and each neuron in the output layer.

[0020] Furthermore, when the type of the baseline layer is a LIF neuron layer and the type of the layer to be evaluated is a flattened layer, the feature data includes the score of each neuron in the LIF neuron layer and the connection weight between each neuron in the flattened layer and each neuron in the LIF neuron layer.

[0021] Step S13 includes:

[0022] Based on the second preset relational expression and the feature data, the score of each neuron in the flattened layer is determined; the second preset relational expression is:

[0023]

[0024] Where d is the index of the neuron under the flattened layer and D is the total number of neurons under the flattened layer, where D is an integer greater than 1; The scoreFlatten represents the connection weight between the d-th neuron in the flattened layer and the a-th neuron in the LIF neuron layer. d ScoreLIF represents the score of the d-th neuron. a This represents the score of the a-th neuron in the LIF neuron layer.

[0025] Furthermore, when the type of the baseline layer is a flattened layer and the type of the layer to be evaluated is a max pooling layer, the feature data includes the scores of each neuron under the flattened layer;

[0026] Step S13 includes:

[0027] The lower coordinates of the max pooling layer are determined based on its shape. The score of the d-th neuron is the same as the score of the d-th neuron under the flattened layer;

[0028] Among them, the This represents the channel index of the neuron in the max-pooling layer after dimensionality increase through the d-th neuron. This represents the height index of the neuron in the max-pooling layer after dimensionality increase through the d-th neuron. This represents the width index of the neuron in the max-pooling layer after dimensionality increase through the d-th neuron; and, and Based on the third preset relation, the third preset relation is:

[0029]

[0030] Where d is the index of the neuron under the flattened layer, H Z W is the maximum value of the height index used to describe the shape of the max-pooling layer. Z This is the maximum value of the width index used to describe the shape of the max-pooling layer.

[0031] Furthermore, when the type of the baseline layer is a LIF neuron layer and the type of the layer to be evaluated is not a flattened layer, the feature data includes the scores of each neuron under the LIF neuron layer;

[0032] Step S13 includes:

[0033] The scores of each neuron in the layer to be evaluated are determined to be equal to the scores of the corresponding neurons in the LIF neuron layer.

[0034] Furthermore, when the baseline layer is a max pooling layer, the feature data includes the number of output pulses of each neuron in the max pooling layer, the number of output pulses of each neuron in the layer to be evaluated, and the score of each neuron in the max pooling layer.

[0035] Step S13 includes:

[0036] The scores of each neuron in the layer to be evaluated are determined based on the fourth preset relation and the feature data; the fourth preset relation is:

[0037]

[0038] Where [C,H,W] represents the coordinate indices used to locate neurons, with C representing the channel index, H representing the height index, and W representing the width index; scoreL eva [C,H,W] represents the score of the neuron with coordinates [C,H,W] in the layer to be evaluated, L eva Indicates the layer to be evaluated, L bas W represents the max pooling layer. Z C represents the maximum value of the width index of the layer to be evaluated. ZH represents the maximum value of the channel index of the layer to be evaluated. Z This represents the maximum value of the height index of the layer to be evaluated;

[0039] This represents the number of output pulses of the neuron with coordinates [C,H,W] in the evaluation layer. The coordinates of the max pooling layer are: The number of output pulses of neurons This represents the number of output pulses of the neuron with coordinates [C, H-(Hmod2)+i, W-(Wmod2)+j] in the layer to be evaluated, where i and j represent the first and second size parameters required for sliding on the corresponding pooling window in the max-pooling layer, respectively. The coordinates of the max pooling layer are: The score of the neurons.

[0040] Furthermore, when the reference layer is a convolutional layer, the feature data includes the number of output pulses of each neuron under the convolutional layer and the score of each neuron under the convolutional layer;

[0041] Step S13 includes:

[0042] The scores of each neuron in the layer to be evaluated are determined based on the fifth preset relation and the feature data; the fifth preset relation is:

[0043]

[0044] Among them, scoreL eva [C in [H,W] indicates that the lower coordinates of the layer to be evaluated are [C in The scores of neurons in [H,W], C z The lower channel index C of the layer to be evaluated is indicated. in The maximum value of W z H represents the maximum value of the width index of the layer to be evaluated. Z C represents the maximum value of the height index of the layer to be evaluated. z ′ represents the lower channel index C of the convolutional layer. out The maximum value; [C out C in ,K h ,K w ] represents the convolution kernel used for convolution operations in the convolutional layer, w[C out C in ,K h ,K w ] represents the weights of the convolution kernel, K hK is the first parameter used to define the height of the convolution kernel. w The second parameter is used to define the width of the convolution kernel, and the third and fourth parameters are used to define the window size when the convolution kernel slides, respectively.

[0045] The lower coordinate of the convolutional layer is [C out The number of output pulses of neurons in the range [H+m, W+n], scoreL bas [C out [H+m,W+n] represents the lower coordinates of the convolutional layer as [C out The scores of neurons in the sequence [H+m,W+n].

[0046] Further, step S14 includes:

[0047] Sort the neurons in the layer to be evaluated in descending order of their scores;

[0048] When the current layer to be evaluated is determined to be the target layer for fault injection, the first Q neurons are selected as the target neurons for fault injection according to the sorting order of the neurons in the target layer, where Q is an integer not less than 1.

[0049] To address the aforementioned technical problems, the present invention also provides a fault injection system for SNN neurons, comprising:

[0050] The layer determination unit is used to determine the last layer under the network topology as the base layer and the layer before the base layer as the layer to be evaluated, based on the network topology of the SNN.

[0051] The acquisition unit is used to acquire feature data according to the type of the reference layer. The feature data includes at least one of the following: the number of output pulses of each neuron in the reference layer, the number of output pulses of each neuron in the layer to be evaluated, the connection weight between each neuron in the reference layer and each neuron in the layer to be evaluated, and the score of each neuron in the reference layer.

[0052] The scoring unit is used to process the feature data according to a preset evaluation strategy corresponding to the type, so as to determine the score representing the importance of each neuron in the layer to be evaluated;

[0053] The target neuron determination unit is used to sort the neurons in the layer to be evaluated according to their scores, so as to determine the target neuron for fault injection based on the sorting results.

[0054] The judgment unit is used to determine whether the current layer to be evaluated is the second layer. If not, the relocation unit is triggered.

[0055] The relocation unit is used to determine, according to the network topology, the current layer to be evaluated as the new reference layer, and the layer preceding the current layer to be evaluated as the new layer to be evaluated, and then return to the acquisition unit.

[0056] To address the aforementioned technical problems, the present invention also provides a fault injection device for SNN neurons, comprising:

[0057] Memory, used to store computer programs;

[0058] A processor is configured to implement the steps of the fault injection method for SNN neurons as described above when executing the computer program.

[0059] This application provides a method, system, and apparatus for fault injection into SNN neurons. Based on the SNN network topology, the last layer is designated as the baseline layer, and the layer preceding the baseline layer is designated as the evaluation layer. Feature data is obtained according to the type of the baseline layer. This feature data includes at least one of the following: the number of output pulses of each neuron in the baseline layer, the number of output pulses of each neuron in the evaluation layer, the connection weights between each neuron in the baseline layer and each neuron in the evaluation layer, and the score of each neuron in the baseline layer. The feature data is processed according to a preset evaluation strategy corresponding to the type to determine the score representing the importance of each neuron in the evaluation layer. The scores of each neuron are then sorted to determine the target neuron for fault injection. This scheme deduces the score of each intermediate layer's neurons layer by layer, allowing subsequent fault injection to focus on operating on higher-importance target neurons, directly targeting key neurons. This facilitates subsequent protection measures for these important neurons, improves the fault tolerance of the SNN network, and ensures SNN network security.

[0060] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0061] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0062] Figure 1 A flowchart of a fault injection method for SNN neurons provided by the present invention;

[0063] Figure 2A schematic diagram of an 11-layer SNN network topology provided by the present invention;

[0064] Figure 3 This is a schematic diagram of a convolution kernel sliding process provided by the present invention;

[0065] Figure 4 A schematic diagram of a fault injection system for SNN neurons provided by the present invention;

[0066] Figure 5 This is a schematic diagram of a fault injection device for SNN neurons provided by the present invention. Detailed Implementation

[0067] The core of this invention is to provide a method, system, and device for fault injection into SNN neurons. This method can deduce and determine the scores of neurons in each intermediate layer layer by layer, so that subsequent fault injection can directly target key neurons. This facilitates the implementation of corresponding protection measures for these important neurons, improves the fault tolerance of the SNN network, and ensures the security of the SNN network.

[0068] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0069] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0070] Please refer to Figure 1 , Figure 1 A flowchart of a fault injection method for SNN neurons provided in this application.

[0071] The fault injection method for SNN neurons includes:

[0072] S11: Based on the network topology of the SNN, determine the last layer under the network topology as the baseline layer, and determine the layer before the baseline layer as the layer to be evaluated.

[0073] In this embodiment, the specific structure of the SNN network topology is not particularly limited. Generally, it can be obtained by combining convolutional layers, LIF neuron layers, max pooling layers, flattening layers, and fully connected layers. Taking an 11-layer network topology as an example, please refer to... Figure 2 , Figure 2 This is a schematic diagram of an 11-layer SNN network topology provided by the present invention. The shape of each layer is marked, and the shape refers to the neurons. For example, the ninth flattening layer has 6272 neurons and its shape is (6272); the eighth max pooling layer has a shape of (128×7×7), which essentially marks the coordinate range of the neurons. The other layers are not described in detail. The neurons in layers 1 to 8 need to be located using three-dimensional coordinates. Layers 9 to 11 will reduce the dimensionality so that the neurons can be located using one-dimensional coordinates (i.e., the nth or nth position). It can be seen that the connection method of the 11-layer SNN network topology is convolutional layer-LIF neuron layer-convolutional layer-LIF neuron layer-max pooling layer-convolutional layer-LIF neuron layer-max pooling layer-flattening layer-LIF neuron layer-output layer. The flattening layer and the LIF neuron layer, and the LIF neuron layer and the output layer are connected in a fully connected manner.

[0074] The last layer in the network topology here refers to the output layer. Figure 2 Taking the network topology in the example, the layer before the output layer is the LIF neuron layer.

[0075] S12: Obtain feature data according to the type of the baseline layer. The feature data includes at least one of the following: the number of output pulses of each neuron in the baseline layer, the number of output pulses of each neuron in the layer to be evaluated, the connection weight between each neuron in the baseline layer and each neuron in the layer to be evaluated, and the score of each neuron in the baseline layer.

[0076] Specifically, before step S11, in order to obtain the number of output pulses of each layer of neurons in the trained SNN to be evaluated and the connection weights between neurons, the MNIST static dataset is input into the SNN and trained and tested using the spatiotemporal backpropagation algorithm. As a result, the trained SNN has an accuracy of 99.52%, and the number of output pulses and connection weights in the SNN processing are known.

[0077] S13: Process the feature data according to the preset evaluation strategy corresponding to the type to determine the score representing the importance of each neuron in the layer to be evaluated;

[0078] S14: Sort the neurons in the layer to be evaluated according to their scores, so as to determine the target neurons for fault injection based on the sorting results;

[0079] Specifically, multiple preset evaluation strategies were designed in advance to adapt to different network topologies, thereby reliably determining the scores representing the importance of each neuron in the layer to be evaluated;

[0080] S15: Determine whether the current layer to be evaluated is the second layer. If not, proceed to S16.

[0081] S16: Based on the network topology, determine the current layer to be evaluated as the new baseline layer, and the layer preceding the current layer to be evaluated as the new layer to be evaluated, and return to S12.

[0082] Understandably, the last layer (i.e., the output layer) and the first layer do not need to be evaluated, because the neurons selected during fault injection are mainly those in the intermediate layers excluding the last and first layers. Furthermore, the loop ends when it is determined that the current layer to be evaluated is the second layer.

[0083] In summary, this application provides a method for fault injection into SNN neurons. This method determines the score of neurons in each intermediate layer through layer-by-layer derivation, so that subsequent fault injection focuses on operating on target neurons with higher importance, directly targeting key neurons. This facilitates the subsequent implementation of corresponding protection measures for these important neurons, improves the fault tolerance of the SNN network, and ensures the security of the SNN network.

[0084] Based on the above embodiments:

[0085] In some embodiments, when the type of the base layer is an output layer and the type of the layer to be evaluated is a LIF neuron layer, the feature data includes the number of output pulses of each neuron in the LIF neuron layer and the connection weights between each neuron in the layer to be evaluated and each neuron in the output layer.

[0086] Step S13 includes:

[0087] Based on the first preset relational expression and feature data, the score of each neuron in the LIF neuron layer is determined; the first preset relational expression is:

[0088]

[0089] Where a is the index of each neuron in the LIF neuron layer and A is the total number of neurons, where A is an integer greater than 1. a Let be the number of output pulses of the a-th neuron, and scoreLIF. a This represents the score of the a-th neuron in the LIF neuron layer;

[0090] b is the index of each neuron in the output layer, and B is the total number of neurons, where B is an integer greater than 1. This represents the boost weight after subtracting the minimum value from the connection weight between neuron a and neuron b, where the minimum value is the minimum value among the connection weights between neuron a and each neuron in the output layer.

[0091] In this embodiment, considering that the output layer is the last layer, the classification result is directly output. Figure 2 Taking the network topology in the example, the 10th layer is the LIF neuron layer, and the 11th layer is the output layer. The output layer directly outputs the 10-class classification results. The output accuracy is directly related to the category output by the ten classes. The category output by the ten classes is directly related to the difference between each class. Therefore, the basis for judging the importance of neurons in the LIF neuron layer is the degree of contribution of a neuron to the difference between different classes. That is, the importance of a neuron in the 10th layer lies in the difference of its connection weights.

[0092] Specifically, the weights are raised to be non-negative. The principle behind the first preset relational expression is to obtain the positive contribution of each neuron in the LIF neuron layer to the output category by raising the connection weights to be non-negative, thereby obtaining the score of each neuron.

[0093] In some embodiments, when the type of the baseline layer is a LIF neuron layer and the type of the layer to be evaluated is a flattened layer, the feature data includes the scores of each neuron under the LIF neuron layer and the connection weights between each neuron under the flattened layer and each neuron under the LIF neuron layer.

[0094] Step S13 includes:

[0095] Based on the second preset relational expression and feature data, the score of each neuron in the flattened layer is determined; the second preset relational expression is:

[0096]

[0097] Where d is the index of the neuron under the flattened layer and D is the total number of neurons under the flattened layer, and D is an integer greater than 1; The scoreFlatten represents the connection weight between the d-th neuron in the flattened layer and the a-th neuron in the LIF neuron layer. d ScoreLIF represents the score of the d-th neuron. a This represents the score of the a-th neuron in the LIF neuron layer.

[0098] In this embodiment, the principle of setting the second preset relation is to use the absolute value of the connection weight between neurons in the LIF neuron layer and neurons in the flattening layer as the criterion to give the contribution of neurons in the flattening layer to the LIF neuron layer; where a is the index of each neuron in the LIF neuron layer and A is the total number of neurons, and A is an integer greater than 1.

[0099] In some embodiments, when the type of the baseline layer is a flattened layer and the type of the layer to be evaluated is a max pooling layer, the feature data includes the scores of each neuron under the flattened layer;

[0100] Step S13 includes:

[0101] The lower coordinates of the max pooling layer are determined based on its shape. The score of the d-th neuron is the same as the score of the d-th neuron under the flattened layer;

[0102] in, This represents the channel index of the neuron in the max-pooling layer after dimensionality increase through the d-th neuron. This represents the height index of the neuron in the max-pooling layer after dimensionality increase through the d-th neuron. This represents the width index of the neuron in the max-pooling layer after dimensionality increase through the d-th neuron; and, and Based on the third preset relation, the third preset relation is:

[0103]

[0104] Where d is the index of the neurons in the flattened layer, and H Z W is the maximum value of the height index used to describe the shape of the max-pooling layer. Z The maximum value of the width index used to describe the shape of the max-pooling layer.

[0105] In this embodiment, the setting mechanism of the third preset relation is based on the fact that the output of the max pooling layer undergoes a flattening process from the max pooling layer to the flattening layer. Therefore, the importance score of the neurons in the max pooling layer can be obtained by increasing the dimensionality of the neurons in the flattening layer.

[0106] In some embodiments, when the type of the baseline layer is a LIF neuron layer and the type of the layer to be evaluated is not a flattened layer, the feature data includes the scores of each neuron under the LIF neuron layer;

[0107] Step S13 includes:

[0108] The scores of each neuron in the layer to be evaluated are determined to be equal to the scores of the corresponding neurons in the LIF neuron layer.

[0109] In this embodiment, when the base layer is a LIF neuron layer, the layer to be evaluated can be a convolutional layer, such as... Figure 2As shown, in order to conform to the design mechanism of SNN network and simulate the characteristics of biological neurons, the score of each neuron in the layer to be evaluated is determined to be equal to the score of the corresponding neuron in the LIF neuron layer.

[0110] In some embodiments, when the baseline layer is a max pooling layer, the feature data includes the number of output pulses of each neuron in the max pooling layer, the number of output pulses of each neuron in the layer to be evaluated, and the score of each neuron in the max pooling layer.

[0111] Step S13 includes:

[0112] The scores of each neuron in the layer to be evaluated are determined based on the fourth preset relation and feature data; the fourth preset relation is:

[0113]

[0114] Where [C,H,W] represents the coordinate indices used to locate neurons, with C representing the channel index, H representing the height index, and W representing the width index; scoreL eva [C,H,W] represents the score of the neuron with coordinates [C,H,W] in the layer to be evaluated, L eva Indicates the layer to be evaluated, L bas W represents the max pooling layer. Z C represents the maximum value of the width index of the layer to be evaluated. Z H represents the maximum value of the channel index of the layer to be evaluated. Z This represents the maximum value of the height index of the layer to be evaluated;

[0115] This represents the number of output pulses of the neuron with coordinates [C,H,W] in the evaluation layer. The coordinates of the max pooling layer are: The number of output pulses of neurons This represents the number of output pulses of the neuron with coordinates [C, H-(Hmod2)+i, W-(Wmod2)+j] in the layer to be evaluated, where i and j represent the first and second size parameters required when sliding on the corresponding pooling window in the max pooling layer, respectively. The coordinates of the max pooling layer are: The score of the neurons.

[0116] In this embodiment, when the baseline layer is a max-pooling layer, the layer to be evaluated can be a LIF neuron layer, such as... Figure 2As shown, specifically, the mechanism for setting the fourth preset relation is based on the consideration that the max pooling process involves sliding a pooling window of a fixed size on the input feature map and selecting the maximum value within each pooling window as the output, thereby achieving dimensionality reduction and feature extraction. Therefore, when deriving the scores of each neuron before the max pooling layer, the number of pulses of each neuron before and after max pooling is combined for derivation. The importance of neurons in the max pooling layer is related to the contribution of the number of pulses in the window before pooling to the output pulses after pooling. Furthermore, since the number of pulses is included in the score of the current layer after deriving from the pooling layer, and the distribution of the number of pulses varies greatly, the square root value is taken to reduce the difference, ultimately yielding the fourth preset relation.

[0117] In some embodiments, when the base layer is a convolutional layer, the feature data includes the number of output pulses of each neuron under the convolutional layer and the score of each neuron under the convolutional layer;

[0118] Step S13 includes:

[0119] The scores of each neuron in the layer to be evaluated are determined based on the fifth preset relation and feature data; the fifth preset relation is:

[0120]

[0121] Among them, scoreL eva [C in [H,W] indicates that the coordinates of the layer to be evaluated are [C in The scores of neurons in [H,W], C z Indicates the lower channel index C of the layer to be evaluated. in The maximum value of W z H represents the maximum width index of the layer to be evaluated. Z C represents the maximum value of the height index of the layer to be evaluated. z ′ represents the lower channel index C of the convolutional layer. out The maximum value; [C out C in ,K h ,K w ] represents the convolution kernel used for convolution operations in a convolutional layer, w[C out C in ,K h ,K w ] represents the weights of the convolution kernel, K h K is the first parameter used to define the height of the convolution kernel. w The second parameter is used to define the width of the convolution kernel, and the third and fourth parameters are used to define the window size when the convolution kernel slides, respectively.

[0122] The lower coordinate of the convolutional layer is [Cout The number of output pulses of neurons in the range [H+m, W+n], scoreL bas [C out [H+m,W+n] represents the coordinates of the convolutional layer as [C out The scores of neurons in the sequence [H+m,W+n].

[0123] In this embodiment, to illustrate the setting mechanism of the fifth preset relation, the sliding process of the convolution kernel is first explained as follows:

[0124] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the sliding process of a convolution kernel provided by the present invention. For the sake of simplicity, a 2×2 convolution kernel is used as an example. Each channel corresponds to one convolution kernel. Figure 3 The channel 1 shown corresponds to the convolution kernel [w1, w2, w3, w4]. Therefore, for N channels of channel 1... 0,0 Specifically, it participates in four convolution processes; more specifically, in the blue window of channel 1, N... 0,0 Multiplying by w4, the sum of the products of this window and the corresponding elements of the convolution kernel is added to the sum of the products of the blue windows in other channels to get out1; in the orange window of channel 1, N 0,0 Multiplying by w3, the sum of the products of this window and the corresponding elements of the convolution kernel, plus the sum of the products of the orange windows in other channels, yields out2; in the yellow window of channel 1, N 0,0 Multiplying by w2, the sum of the products of the corresponding elements of this window and the convolution kernel, plus the sum of the products of the yellow windows in other channels, yields out3; in the green window of channel 1, N 0,0 Multiplying by w1, the sum of the products of this window and the corresponding elements of the convolution kernel, plus the sum of the products of the green windows of other channels, yields out4. Therefore, it can be seen that N 0,0 It contributes to all four outputs; W is obtained by summing the weights of each convolution kernel separately. sum Calculate the contribution of this neuron to the output under different convolutional kernel weights; then the total contribution of this neuron to the output is out1×(w4 / W). sum )+out2*(w3 / W sum )+out3×(w2 / W sum )+out4*(w1 / W sum ).

[0125] It is evident that for a given neuron, its contribution to the output is the sum of its contributions to the output of each convolution window. Therefore, its contribution to each convolution window is calculated and then summed. Similarly, when working backward from this layer, the number of pulses is included in the score of the current layer. However, the distribution of the number of pulses varies considerably. To reduce this variation, the square root of the number of pulses is calculated, thus obtaining the fifth preset relational expression, which is used to determine the score of each neuron in the layer to be evaluated.

[0126] Furthermore, as an example, with Figure 2 Taking the network topology as an example, and combining the above-mentioned preset evaluation strategy, the method for determining the score of neurons in each layer is given:

[0127] The 10th LIF neuron layer has 256 neurons, and the 11th output layer has 10 neurons. Therefore, according to the first preset relation, the score of the a-th neuron in the 10th layer is:

[0128]

[0129] The 9th flattened layer has 6272 neurons. According to the second preset relation, the score of the d-th neuron in the 9th layer is:

[0130]

[0131] The shape of the 9th layer is (6272), and the shape of the 8th layer is (128,7,7). Then H... Z =7, W Z =7, then according to the third preset relation, the coordinates of the neuron corresponding to the d-th neuron in the 9th layer after dimensionality upgrade are:

[0132]

[0133] in, The score of the neuron defined by this coordinate is the same as the score of the d-th neuron.

[0134] Layer 8 is a max pooling layer, and layer 7 is a LIF neuron layer with a shape of (128×14×14). Therefore, according to the fourth preset relational formula, the score of the neuron with coordinates [C,H,W] in layer 7 is:

[0135]

[0136] If the 6th layer is a convolutional layer with a shape of (128×14×14), then the score of each neuron in the convolutional layer is equal to the score of the corresponding neuron in the 7th LIF neuron layer.

[0137] If layer 5 is a max-pooling layer with a shape of (128×14×14), then according to the fifth preset relation, the score of each neuron in layer 5 is determined as follows:

[0138]

[0139] The fourth layer is a LIF neuron layer with a shape of (128×28×28). Based on the fourth preset relational formula, the score of the neuron with coordinates [C,H,W] in the fourth layer is determined as follows:

[0140]

[0141] If the third layer is a convolutional layer with a shape of (128×28×28), then the score of each neuron in the convolutional layer is equal to the score of the corresponding neuron in the fourth LIF neuron layer.

[0142] The second layer is a LIF neuron layer with a shape of (128×28×28). Based on the fifth preset relational formula, the score of each neuron in the second layer is determined as follows:

[0143]

[0144] In some embodiments, step S14 includes:

[0145] Sort the neurons in the layer to be evaluated in descending order of their scores;

[0146] When the current layer to be evaluated is determined to be the target layer for fault injection, the first Q neurons are selected as the target neurons for fault injection according to the sorting order of the neurons in the target layer, where Q is an integer not less than 1.

[0147] Specifically, the score represents the importance of neurons, and the above method can better enable fault injection into key neurons to monitor the accuracy of the SNN network, which is conducive to determining subsequent reinforcement of key neurons, thereby effectively improving the network's fault tolerance.

[0148] Still with Figure 2 Taking the SNN network topology in the example, fault injection is performed in each LIF neuron layer. First, dead neuron fault injection is performed (i.e., the neuron does not emit pulses). Table 1 shows the network accuracy after dead neuron fault injection in layers 10, 7, 4, and 2, respectively, using random injection, pulse injection, and the method described in this application to evaluate the neurons in this layer to determine the target neurons before injection. It can be seen that the method of dead neuron fault injection for target neurons in this application makes the decline in network accuracy greater than the decline in the other two methods, thus proving that these target neurons do play a more important role in maintaining the accuracy of the SNN network, which is conducive to subsequent targeted maintenance.

[0149] Table 1

[0150]

[0151]

[0152] Secondly, saturated neuron fault injection (i.e., making the neurons continuously emit pulses) is performed. Table 2 shows the network accuracy after saturated neuron fault injection in layers 10, 7, 4, and 2, respectively, using random injection, pulse injection, and injection after evaluating the neurons in this application to determine the target neurons. It can be seen that the method of saturated neuron fault injection for target neurons in this application can make the decline in network accuracy greater than the other two methods, thus proving that these target neurons do play a more important role in maintaining the accuracy of the SNN network, which is conducive to subsequent targeted maintenance.

[0153] Table 2

[0154]

[0155] At the same time, it should be noted that there are indeed a very small number of results in Table 2 that do not meet expectations under the condition of improper injection, because SNN itself has a certain degree of fault tolerance.

[0156] In summary, by using the method provided in this application to determine the target neurons and then injecting faults into them, the network can be implemented more quickly. This verifies the accuracy of the target neuron determination in the scheme of this application, which is beneficial for subsequent targeted reinforcement of these key neurons, thereby effectively improving the network's fault tolerance.

[0157] Please refer to Figure 4 , Figure 4 This is a schematic diagram of a fault injection system for SNN neurons provided by the present invention.

[0158] The fault injection system for SNN neurons includes:

[0159] The layer determination unit 21 is used to determine the last layer under the network topology as the base layer based on the network topology of the SNN, and to determine the layer before the base layer as the layer to be evaluated.

[0160] The acquisition unit 22 is used to acquire feature data according to the type of the baseline layer. The feature data includes at least one of the following: the number of output pulses of each neuron in the baseline layer, the number of output pulses of each neuron in the layer to be evaluated, the connection weight between each neuron in the baseline layer and each neuron in the layer to be evaluated, and the score of each neuron in the baseline layer.

[0161] The scoring unit 23 is used to process the feature data according to the preset evaluation strategy corresponding to the type in order to determine the score representing the importance of each neuron in the layer to be evaluated.

[0162] The target neuron determination unit 24 is used to sort the neurons in the layer to be evaluated according to their scores, so as to determine the target neurons for fault injection based on the sorting results.

[0163] Judgment unit 25 is used to determine whether the current layer to be evaluated is the second layer. If not, relocation unit 26 is triggered.

[0164] Relocation unit 26 is used to determine the current layer to be evaluated as the new base layer according to the network topology, and the layer before the current layer to be evaluated as the new layer to be evaluated, and then return to acquisition unit 22.

[0165] For a description of the fault injection system for SNN neurons provided in this application, please refer to the above-described embodiments of the fault injection method for SNN neurons, which will not be repeated here.

[0166] Please refer to Figure 5 , Figure 5 This is a schematic diagram of a fault injection device for SNN neurons provided by the present invention.

[0167] The fault injection device for SNN neurons includes:

[0168] Memory 31 is used to store computer programs;

[0169] The processor 32 is configured to implement the steps of the fault injection method for SNN neurons as described above when executing the computer program.

[0170] For a description of the fault injection device for SNN neurons provided in this application, please refer to the above-described embodiments of the fault injection method for SNN neurons, which will not be repeated here.

[0171] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0172] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0173] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for fault injection into SNN neurons, characterized in that, include: S11: Based on the network topology of the SNN, determine the last layer under the network topology as the baseline layer, and determine the layer before the baseline layer as the layer to be evaluated; S12: Obtain feature data according to the type of the reference layer. The feature data includes at least one of the following: the number of output pulses of each neuron in the reference layer, the number of output pulses of each neuron in the layer to be evaluated, the connection weight between each neuron in the reference layer and each neuron in the layer to be evaluated, and the score of each neuron in the reference layer. S13: Process the feature data according to a preset evaluation strategy corresponding to the type to determine the score representing the importance of each neuron in the layer to be evaluated; S14: Sort the neurons in the layer to be evaluated according to their scores, so as to determine the target neurons for fault injection based on the sorting results; S15: Determine whether the current layer to be evaluated is the second layer. If not, proceed to S16. S16: According to the network topology, determine the current layer to be evaluated as the new baseline layer, and the layer before the current layer to be evaluated as the new layer to be evaluated, and return to S12.

2. The fault injection method for SNN neurons as described in claim 1, characterized in that, When the type of the reference layer is an output layer and the type of the layer to be evaluated is a LIF neuron layer, the feature data includes the number of output pulses of each neuron in the LIF neuron layer and the connection weights between each neuron in the layer to be evaluated and each neuron in the output layer. Step S13 includes: Based on the first preset relational expression and the feature data, the score of each neuron in the LIF neuron layer is determined; the first preset relational expression is: Where a is the index of each neuron in the LIF neuron layer and A is the total number of neurons, where A is an integer greater than 1. a Let be the number of output pulses of the a-th neuron, and scoreLIF. a The score of the a-th neuron in the LIF neuron layer; b is the index of each neuron in the output layer, and B is the total number of neurons, where B is an integer greater than 1. This represents the boost weight after subtracting the minimum value from the connection weight between neuron a and neuron b, where the minimum value is the minimum value among the connection weights between neuron a and each neuron in the output layer.

3. The fault injection method for SNN neurons as described in claim 1, characterized in that, When the type of the baseline layer is a LIF neuron layer and the type of the layer to be evaluated is a flattened layer, the feature data includes the score of each neuron under the LIF neuron layer and the connection weight between each neuron under the flattened layer and each neuron under the LIF neuron layer. Step S13 includes: Based on the second preset relational expression and the feature data, the score of each neuron in the flattened layer is determined; the second preset relational expression is: Where d is the index of the neuron under the flattened layer and D is the total number of neurons under the flattened layer, where D is an integer greater than 1; The scoreFlatten represents the connection weight between the d-th neuron in the flattened layer and the a-th neuron in the LIF neuron layer. d ScoreLIF represents the score of the d-th neuron. a This represents the score of the a-th neuron in the LIF neuron layer.

4. The fault injection method for SNN neurons as described in claim 1, characterized in that, When the type of the baseline layer is a flattened layer and the type of the layer to be evaluated is a max pooling layer, the feature data includes the scores of each neuron under the flattened layer; Step S13 includes: The lower coordinates of the max pooling layer are determined based on its shape. The score of the d-th neuron is the same as the score of the d-th neuron under the flattened layer; Among them, the This represents the channel index of the neuron in the max-pooling layer after dimensionality increase through the d-th neuron. This represents the height index of the neuron in the max-pooling layer after dimensionality increase through the d-th neuron. This represents the width index of the neuron in the max-pooling layer after dimensionality increase through the d-th neuron; and, and Based on the third preset relation, the third preset relation is: Where d is the index of the neuron under the flattened layer, H Z W is the maximum value of the height index used to describe the shape of the max-pooling layer. Z This is the maximum value of the width index used to describe the shape of the max-pooling layer.

5. The fault injection method for SNN neurons as described in claim 1, characterized in that, When the type of the baseline layer is a LIF neuron layer and the type of the layer to be evaluated is not a flattened layer, the feature data includes the scores of each neuron under the LIF neuron layer; Step S13 includes: The scores of each neuron in the layer to be evaluated are determined to be equal to the scores of the corresponding neurons in the LIF neuron layer.

6. The fault injection method for SNN neurons as described in claim 1, characterized in that, When the baseline layer is a max pooling layer, the feature data includes the number of output pulses of each neuron in the max pooling layer, the number of output pulses of each neuron in the layer to be evaluated, and the score of each neuron in the max pooling layer; Step S13 includes: The scores of each neuron in the layer to be evaluated are determined based on the fourth preset relation and the feature data; the fourth preset relation is: Where [C,H,W] represents the coordinate indices used to locate neurons, with C representing the channel index, H representing the height index, and W representing the width index; scoreL eva [C,H,W] represents the score of the neuron with coordinates [C,H,W] in the layer to be evaluated, L eva Indicates the layer to be evaluated, L bas W represents the max pooling layer. Z C represents the maximum value of the width index of the layer to be evaluated. Z H represents the maximum value of the channel index of the layer to be evaluated. Z This represents the maximum value of the height index of the layer to be evaluated; This represents the number of output pulses of the neuron with coordinates [C,H,W] in the evaluation layer. The coordinates of the max pooling layer are: The number of output pulses of the neuron, spike Leva [C,H-(Hmod2)+i,W-(Wmod2)+j] represents the number of output pulses of the neuron with coordinates [C,H-(Hmod2)+i,W-(Wmod2)+j] in the layer to be evaluated, where i and j represent the first and second size parameters required for sliding on the corresponding pooling window in the max pooling layer, respectively. The coordinates of the max pooling layer are: The score of the neurons.

7. The fault injection method for SNN neurons as described in claim 1, characterized in that, When the baseline layer is a convolutional layer, the feature data includes the number of output pulses of each neuron under the convolutional layer and the score of each neuron under the convolutional layer; Step S13 includes: The scores of each neuron in the layer to be evaluated are determined based on the fifth preset relation and the feature data; the fifth preset relation is: Among them, scoreL eva [C in [H,W] indicates that the lower coordinates of the layer to be evaluated are [C in The scores of neurons in [H,W], C z The lower channel index C of the layer to be evaluated is indicated. in The maximum value of W z H represents the maximum value of the width index of the layer to be evaluated. Z C represents the maximum value of the height index of the layer to be evaluated. z ′ represents the lower channel index C of the convolutional layer. out The maximum value; [C out C in ,K h ,K w ] represents the convolution kernel used for convolution operations in the convolutional layer, w[C out C in ,K h ,K w ] represents the weights of the convolution kernel, K h K is the first parameter used to define the height of the convolution kernel. w The second parameter is used to define the width of the convolution kernel, and the third and fourth parameters are used to define the window size when the convolution kernel slides, respectively. The lower coordinate of the convolutional layer is [C out The number of output pulses of neurons in the range [H+m, W+n], scoreL bas [C out [H+m,W+n] represents the lower coordinates of the convolutional layer as [C out The scores of neurons in the sequence [H+m,W+n].

8. The fault injection method for SNN neurons as described in claim 1, characterized in that, Step S14 includes: Sort the neurons in the layer to be evaluated in descending order of their scores; When the current layer to be evaluated is determined to be the target layer for fault injection, the first Q neurons are selected as the target neurons for fault injection according to the sorting order of the neurons in the target layer, where Q is an integer not less than 1.

9. A fault injection system for SNN neurons, characterized in that, include: The layer determination unit is used to determine the last layer under the network topology as the base layer and the layer before the base layer as the layer to be evaluated, based on the network topology of the SNN. The acquisition unit is used to acquire feature data according to the type of the reference layer. The feature data includes at least one of the following: the number of output pulses of each neuron in the reference layer, the number of output pulses of each neuron in the layer to be evaluated, the connection weight between each neuron in the reference layer and each neuron in the layer to be evaluated, and the score of each neuron in the reference layer. The scoring unit is used to process the feature data according to a preset evaluation strategy corresponding to the type, so as to determine the score representing the importance of each neuron in the layer to be evaluated; The target neuron determination unit is used to sort the neurons in the layer to be evaluated according to their scores, so as to determine the target neuron for fault injection based on the sorting results. The judgment unit is used to determine whether the current layer to be evaluated is the second layer. If not, the relocation unit is triggered. The relocation unit is used to determine, according to the network topology, the current layer to be evaluated as the new reference layer, and the layer preceding the current layer to be evaluated as the new layer to be evaluated, and then return to the acquisition unit.

10. A fault injection device for SNN neurons, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the fault injection method for SNN neurons as described in any one of claims 1 to 8 when executing the computer program.

Images