Model optimization information generation method and device applied to spiking neural network model

Abstract

The embodiment of the invention discloses a model optimization information generation method and device applied to a spiking neural network model. A specific embodiment of the method comprises the following steps: carrying out model initialization on a spiking neural network model to generate a model file; for each piece of neuron layer description information in the neuron layer description information set, executing the following processing steps: determining neuron activeness corresponding to each piece of neuron description information in the neuron description information set included in the neuron layer description information; performing neuron activeness division on the neuron description information; generating an updated neuron description information set; generating sub-optimization information based on the updated neuron description information set; and according to the obtained sub-optimization information set, generating model optimization information for the spike neural network model. According to the embodiment, under the condition that the model precision is stable, the energy consumption attack simulation for the spike neural network model is realized.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and more particularly to a method and device for generating model optimization information applied to a spike neural network model. Background Art

[0002] Currently, spiking neural networks (SNNs) are becoming an energy-efficient alternative to traditional artificial neural networks (ANNs). However, as spiking neural networks are increasingly deployed in edge devices, security issues are also emerging. Currently, when dealing with the security of spiking neural networks, the usual approach is to improve the security of spiking neural networks by controlling adversarial attacks caused by input noise.

[0003] However, the inventors have found that when the above method is used to process the security of the spiking neural network, the following technical problems often occur:

[0004] First, when the spike activity of neurons increases, it usually leads to more memory access operations, which greatly increases energy consumption. Therefore, traditional methods have shortcomings in dealing with the balance between energy efficiency and accuracy of spiking neural networks. That is, it is difficult to achieve energy consumption attack simulation for spiking neural network models under the condition of stable model accuracy.

[0005] Second, existing security research mainly focuses on input perturbations, with less attention paid to the security of spiking neural network parameters. As a result, spiking neural networks are prone to the risk of malicious parameter manipulation when facing memory-based attacks.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0007] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0008] Some embodiments of the present disclosure propose a method and apparatus for generating model optimization information applied to a spike neural network model to solve one or more of the technical problems mentioned in the above background technology section.

[0009] In a first aspect, some embodiments of the present disclosure provide a method for generating model optimization information applied to a spike neural network model, the method comprising: initializing a model of a spike neural network model to generate a model file, wherein the model file comprises: a neuron layer description information set and an upper limit value of accuracy loss, the neuron layer description information in the neuron layer description information set comprises: a neuron description information set, an initial threshold change amount and a percentage of high-activity neurons, and the neuron description information in the neuron description information set comprises: an initial neuron threshold. For each neuron layer description information in the neuron layer description information set, the following processing steps are performed: determining the neuron activity corresponding to each neuron description information in the neuron description information set included in the neuron layer description information. According to the percentage of high-activity neurons included in the neuron layer description information and the neuron activity corresponding to the neuron description information, neuron description information in the neuron description information set included in the neuron layer description information is divided into neuron activity, and a high-activity neuron description information set and a low-activity neuron description information set are obtained. According to the initial threshold change amount, the high-activity neuron description information set and the low-activity neuron description information set included in the neuron layer description information, an updated neuron description information set is generated. Based on the updated neuron description information set, sub-optimization information is generated. Based on the obtained sub-optimization information set, model optimization information for the above-mentioned spike neural network model is generated.

[0010] In a second aspect, some embodiments of the present disclosure provide a model optimization information generation device for a spike neural network model, the device comprising: a model initialization unit, configured to perform model initialization on the spike neural network model to generate a model file, wherein the above-mentioned model file comprises: a neuron layer description information set and an upper limit value of accuracy loss, the neuron layer description information in the above-mentioned neuron layer description information set comprises: a neuron description information set, an initial threshold change and a percentage of high-activity neurons, and the neuron description information in the neuron description information set comprises: an initial neuron threshold. The processing unit is configured to perform the following processing steps for each neuron layer description information in the neuron layer description information set: determine the neuron activity corresponding to each neuron description information in the neuron description information set included in the neuron layer description information; divide the neuron description information in the neuron description information set included in the neuron layer description information according to the percentage of high-activity neurons included in the neuron layer description information and the neuron activity corresponding to the neuron description information, and obtain a high-activity neuron description information set and a low-activity neuron description information set; generate an updated neuron description information set according to the initial threshold change amount included in the neuron layer description information, the high-activity neuron description information set and the low-activity neuron description information set; generate sub-optimization information based on the updated neuron description information set. The generating unit is configured to generate model optimization information for the spike neural network model according to the obtained sub-optimization information set.

[0011] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0012] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0013] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the model optimization information generation method applied to the spike neural network model of some embodiments of the present disclosure, the energy consumption attack simulation for the spike neural network model can be realized under the condition of stable model accuracy. Specifically, the reason why it is difficult to realize the energy consumption attack simulation for the spike neural network model under the condition of stable model accuracy is that when the pulse activity of the neuron increases, it usually leads to more memory access operations, thereby greatly increasing energy consumption. Therefore, the traditional method has deficiencies in balancing the energy efficiency and accuracy of the spike neural network, that is, it is difficult to realize the energy consumption attack simulation for the spike neural network model under the condition of stable model accuracy. Based on this, the model optimization information generation method applied to the spike neural network model of some embodiments of the present disclosure, first, the spike neural network model is initialized to generate a model file, wherein the above-mentioned model file includes: a neuron layer description information set and an upper limit value of the accuracy loss, the neuron layer description information in the above-mentioned neuron layer description information set includes: a neuron description information set, an initial threshold change amount and a percentage of high-activity neurons, and the neuron description information in the neuron description information set includes: an initial neuron threshold. Set initial values ​​for the parameters of the spike neural network model to accelerate the training process. Secondly, for each neuron layer description information in the neuron layer description information set, the following processing steps are performed: the first step is to determine the neuron activity corresponding to each neuron description information in the neuron description information set included in the above neuron layer description information. Thus, preparation can be made for the subsequent division of high neuron description information and low activity description information. The second step is to divide the neuron description information in the neuron description information set included in the above neuron layer description information according to the percentage of high activity neurons included in the above neuron layer description information and the neuron activity corresponding to the neuron description information, and obtain a high activity neuron description information set and a low activity neuron description information set. In practice, since the impact of attacking high activity neurons and attacking low activity neurons on the accuracy of the model is different, the neurons are divided from the perspective of neuron activity. The third step is to generate an updated neuron description information set according to the initial threshold change amount included in the above neuron layer description information, the above high activity neuron description information set and the above low activity neuron description information set. Since the impact of attacking low activity neurons on the accuracy of the model is small and the energy consumption is increased less, different strategies are used to update high activity neurons and low activity neurons. The fourth step is to generate sub-optimization information based on the updated neuron description information set. Finally, based on the obtained sub-optimization information set, model optimization information for the above-mentioned spike neural network model is generated.By reducing the neuron threshold, the state of the neuron is affected, making its pulse excitation frequency more frequent under the same conditions, thereby increasing the energy consumption attack effect of the spiking neural network model without reducing the model accuracy, thereby effectively realizing the energy consumption attack simulation for the spiking neural network model under the condition of stable model accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0015] Figure 1 is a flow chart of some embodiments of a method for generating model optimization information applied to a spiking neural network model according to the present disclosure;

[0016] Figure 2 is a schematic diagram of the structure of some embodiments of a device for generating model optimization information applied to a spike neural network model according to the present disclosure;

[0017] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0020] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0024] refer to Figure 1 , shows a process 100 of some embodiments of a method for generating model optimization information applied to a spike neural network model according to the present disclosure. The method for generating model optimization information applied to a spike neural network model comprises the following steps:

[0025] Step 101, initializing the spike neural network model to generate a model file.

[0026] In some embodiments, the execution subject (e.g., a computing device) of the model optimization information generation method applied to the spike neural network model can perform model initialization on the spike neural network model to generate a model file. Wherein, the above-mentioned model file includes: a neuron layer description information set and an upper limit value of accuracy loss. The neuron layer description information in the above-mentioned neuron layer description information set includes: a neuron description information set, an initial threshold change amount, and a percentage of high-activity neurons. The neuron description information in the neuron description information set includes: an initial neuron threshold. Wherein, the above-mentioned upper limit value of accuracy loss is a preset maximum value of model accuracy loss. The above-mentioned initial threshold change amount represents the increase or decrease of the threshold for the initial neuron threshold. In practice, when the initial threshold change amount is a positive number, the initial threshold change amount represents the threshold increment for the initial neuron threshold. When the initial threshold change amount is a negative number, the initial threshold change amount represents the threshold decrement for the initial neuron threshold. The above-mentioned percentage of high-activity neurons is the percentage of neurons with higher neuron activity to the total number of neurons.

[0027] In practice, the above-mentioned execution entity can initialize the model parameters of the spike neural network model by a random initialization method to generate a model file.

[0028] Step 102: for each neuron layer description information in the neuron layer description information set, perform the following processing steps:

[0029] Step 1021 : determining the neuron activity corresponding to each neuron description information in the neuron description information set included in the neuron layer description information.

[0030] In some embodiments, the execution subject may determine the neuron activity corresponding to each neuron description information in the neuron description information set included in the neuron layer description information, wherein the neuron activity may be the frequency of pulses emitted by the corresponding neuron.

[0031] In practice, the execution subject may determine the neuron activity corresponding to each neuron description information in the neuron description information set included in the neuron layer description information through an Integrate-and-Fire Model (IFModel).

[0032] In some optional implementations of some embodiments, the execution subject determines the neuron activity corresponding to each neuron description information in the neuron description information set included in the neuron layer description information, which may include the following steps:

[0033] The first step is to determine the preset time window.

[0034] The preset time window may represent a preset time step. As an example, the preset time window may be 100 milliseconds.

[0035] The second step is to determine the number of pulses emitted by the neuron corresponding to the above neuron description information within a preset time window to obtain the target number of pulses.

[0036] The third step is to determine the quotient of the target pulse number and the preset time window as the neuron activity corresponding to the neuron description information.

[0037] As an example, the preset time window may be 100 milliseconds, and the target number of pulses may be 15, so the neuron activity is 15 / 100×1000=150 pulses / second.

[0038] Step 1022, based on the percentage of high-activity neurons included in the neuron layer description information and the neuron activity corresponding to the neuron description information, the neuron description information in the neuron description information set included in the neuron layer description information is divided into neuron activity to obtain a high-activity neuron description information set and a low-activity neuron description information set.

[0039] In some embodiments, the above-mentioned execution entity may divide the neuron description information in the neuron description information set included in the above-mentioned neuron layer description information according to the percentage of high-activity neurons included in the above-mentioned neuron layer description information and the neuron activity corresponding to the neuron description information, and obtain a high-activity neuron description information set and a low-activity neuron description information set.

[0040] In practice, the execution subject can determine the number of high-activity neurons according to the percentage of high-activity neurons included in the neuron layer description information, filter out the neuron description information that meets the filter conditions from the neuron description information set included in the neuron layer description information as high-activity neuron description information, obtain a high-activity neuron description information set, remove the high-activity neuron description information set from the neuron description information set, and obtain a low-activity neuron description information set. The above-mentioned filter condition is the neuron description information whose corresponding neuron activity is the activity of the first K neurons. K is the number of high-activity neurons.

[0041] In some optional implementations of some embodiments, the execution subject divides the neuron description information in the neuron description information set included in the neuron layer description information according to the percentage of high-activity neurons included in the neuron layer description information and the neuron activity corresponding to the neuron description information to obtain a high-activity neuron description information set and a low-activity neuron description information set, which may include the following steps:

[0042] In the first step, the number of high-activity neurons is determined according to the number of neuron description information in the neuron description information set included in the neuron layer description information and the percentage of high-activity neurons.

[0043] In practice, the execution entity may determine the number of high-activity neurons as the product of the number of neuron description information in the neuron description information set included in the neuron layer description information and the percentage of high-activity neurons.

[0044] As an example, the number of neuron description information in the neuron description information set included in the neuron layer description information is 10, and the percentage of high-activity neurons is 10%, then the number of high-activity neurons is 10×10%=1.

[0045] In the second step, the neuron description information in the neuron description information set included in the neuron layer description information is sorted according to the neuron activity corresponding to the neuron description information to obtain a neuron description information sequence.

[0046] In practice, the execution entity may arrange the neuron description information in the neuron description information set included in the neuron layer description information in descending order of neuron activity to obtain a neuron description information sequence.

[0047] The third step is to select neuron description information whose corresponding bit sequence is greater than or equal to the target bit sequence from the above neuron description information sequence as high-activity neuron description information to obtain a high-activity neuron description information set.

[0048] The target position sequence is equal to the number of the highly active neurons.

[0049] The fourth step is to select neuron description information whose corresponding bit sequence is smaller than the target bit sequence from the neuron description information sequence as low-activity neuron description information to obtain a low-activity neuron description information set.

[0050] Step 1023: Generate an updated neuron description information set according to the initial threshold change, the high-activity neuron description information set and the low-activity neuron description information set included in the neuron layer description information.

[0051] In some embodiments, the execution entity may generate an updated neuron description information set according to the initial threshold change amount, the high-activity neuron description information set, and the low-activity neuron description information set included in the neuron layer description information.

[0052] In practice, the execution subject may update the high-activity neuron description information set and the low-activity neuron description information set according to the initial threshold change amount included in the neuron layer description information to obtain an updated neuron description information set.

[0053] In some optional implementations of some embodiments, the execution subject generates an updated neuron description information set according to the initial threshold change amount, the high-activity neuron description information set, and the low-activity neuron description information set included in the neuron layer description information, which may include the following steps:

[0054] In the first step, according to the initial threshold change included in the above-mentioned neuron layer description information, each high-activity neuron description information in the above-mentioned high-activity neuron description information set is subjected to a first update process to generate updated high-activity neuron description information and obtain an updated high-activity neuron description information set.

[0055] In practice, the execution subject may perform a first update process on the high-activity neuron description information through a first update process formula to generate updated high-activity neuron description information. The first update process formula is: A+2B=C. In practice, A is the initial neuron threshold corresponding to the high-activity neuron description information, B is the initial threshold change included in the neuron layer description information, and C is the updated high-activity neuron description information.

[0056] In the second step, according to the initial threshold change included in the above-mentioned neuron layer description information, a second update processing is performed on each low-activity neuron description information in the above-mentioned low-activity neuron description information set to generate updated low-activity neuron description information and obtain an updated low-activity neuron description information set.

[0057] In practice, the execution subject may perform a second update process on the low-activity neuron description information through a second update process formula to generate updated low-activity neuron description information. The second update process formula is: D+B=E. In practice, D is the initial neuron threshold corresponding to the low-activity neuron description information, B is the initial threshold change included in the neuron layer description information, and E is the updated low-activity neuron description information.

[0058] In a third step, the updated high-activity neuron description information set and the updated low-activity neuron description information set are determined as the updated neuron description information set.

[0059] In practice, the execution entity may combine the updated high-activity neuron description information set and the updated low-activity neuron description information set into the updated neuron description information set.

[0060] Step 1024: Generate sub-optimization information based on the updated neuron description information set.

[0061] In some embodiments, the execution subject may generate sub-optimization information based on the updated neuron description information set, wherein the sub-optimization information is attack configuration information that meets preset requirements.

[0062] In practice, the above execution entity can find the lowest threshold by performing a bit-flip attack (BFA) on the updated neuron description information set to generate sub-optimization information.

[0063] In some optional implementations of some embodiments, the execution subject generates sub-optimization information based on the updated neuron description information set, which may include the following steps:

[0064] Based on the updated neuron description information set, perform the following update steps:

[0065] The first step is to generate a flipped neuron description information set based on the updated neuron description information set.

[0066] In practice, the execution subject may perform threshold flipping on the updated neuron description information set to obtain the flipped neuron description information set.

[0067] Optionally, each updated neuron description information in the above updated neuron description information set includes: an updated neuron threshold.

[0068] In some optional implementations of some embodiments, the execution subject generates a flipped neuron description information set according to the updated neuron description information set, which may include the following steps:

[0069] For each updated neuron description in the updated neuron description set, perform the following flipping steps:

[0070] Sub-step 1: performing a first numerical conversion on the updated neuron threshold included in the updated neuron description information to obtain a converted neuron threshold.

[0071] In practice, the above execution entity may convert the updated neuron threshold included in the updated neuron description information into binary to obtain the converted neuron threshold.

[0072] As an example, the updated neuron threshold may be 0.2, and the converted neuron threshold may be 00111110010011001100110011001101.

[0073] Sub-step 2: Determine the flip position information.

[0074] The flip position information indicates the position of the bit to be bit flipped in the neuron threshold after the conversion.

[0075] In practice, the execution subject may randomly select the positions of two binary bits in the converted neuron threshold in the converted neuron threshold as the flipping position information.

[0076] As an example, the above-mentioned flip position information may be the 10th bit and the 25th bit.

[0077] Sub-step 3: performing bit flipping processing on the converted neuron threshold according to the above flip position information to obtain the bit flipped neuron threshold.

[0078] In practice, the execution subject obtains the bit-flipped neuron threshold by performing bit flipping on the value at the flip position information in the converted neuron threshold.

[0079] As an example, the neuron threshold after conversion can be 001111100100110011001100110011001101, and the flip position information can be the 6th and 21st bits. The numbers of the 6th and 21st bits are changed from 0 to 1 or from 1 to 0. Therefore, the neuron threshold after bit flipping is 00111110011011001100110010001101.

[0080] Sub-step 4, performing a second numerical conversion on the bit-flipped neuron threshold to obtain the bit-flipped neuron threshold.

[0081] In practice, the above execution entity can convert the bit-flipped neuron threshold into a 32-bit floating point number to obtain the bit-flipped neuron threshold.

[0082] As an example, the bit flipped neuron threshold is 00111110011011001100110010001101, the flipped neuron threshold may be 0.23095703125.

[0083] Sub-step 5: performing threshold update on the updated neuron threshold included in the updated neuron description information according to the flipped neuron threshold, to obtain the flipped neuron description information in the above flipped neuron description information set.

[0084] The execution subject may update the post-flipped neuron threshold to the updated neuron threshold included in the updated neuron description information, and then determine the updated updated neuron description information as the post-flipped neuron description information in the post-flipped neuron description information set.

[0085] In the second step, based on the flipped neuron description information set, the above-mentioned spike neural network model is subjected to a neuron updating model test to generate the model accuracy.

[0086] In practice, the execution entity may input the flipped neuron description information set into the spike neural network model to generate model accuracy.

[0087] The third step is to determine the total threshold error based on the neuron description information set after flipping and the neuron description information set included in the above neuron layer description information.

[0088] In practice, the execution subject may obtain the total threshold error by performing a logical operation on the flipped neuron description information set and the neuron description information set included in the neuron layer description information.

[0089] In some optional implementations of some embodiments, the execution subject determines the total threshold error according to the flipped neuron description information set and the neuron description information set included in the neuron layer description information, which may include the following steps:

[0090] Sub-step 1: for each flipped neuron description information in the flipped neuron description information set, the difference between the flipped neuron threshold included in the flipped neuron description information and the initial neuron threshold included in the neuron description information corresponding to the flipped neuron description information is determined as the threshold difference.

[0091] In practice, for each flipped neuron description information in the flipped neuron description information set, the execution entity obtains a threshold difference by subtracting the flipped neuron threshold included in the flipped neuron description information from the initial neuron threshold included in the neuron description information corresponding to the flipped neuron description information.

[0092] Sub-step 2: determining the sum of the threshold difference values ​​in the obtained threshold difference value set as the total threshold error.

[0093] In practice, the execution subject adds up the threshold differences in the obtained threshold difference set to obtain a total threshold error.

[0094] In the fourth step, in response to determining that the total threshold error is less than the preset threshold error accuracy, sub-optimization information is generated according to the flipped neuron description information set.

[0095] The preset threshold error accuracy may be 1 / 256.

[0096] In practice, the above execution entity can parse the flipped neuron description information set to obtain sub-optimization information.

[0097] In the fifth step, in response to determining that the total threshold error is greater than or equal to the above-mentioned preset threshold error accuracy and the model accuracy is less than the above-mentioned accuracy loss upper limit value, a second update processing is performed on the flipped neuron description information set, and the updated flipped neuron description information set is used as the updated neuron description information set, and the above-mentioned update step is executed again.

[0098] In practice, for each flipped neuron description information in the above flipped neuron description information set, the above execution subject can perform a second update process on the above flipped neuron description information through a second update processing formula, and use the updated flipped neuron description information set as the updated neuron description information set to perform the above update step again. Wherein, the above second update processing formula is: D+B=E. In practice, D is the initial neuron threshold corresponding to the above flipped neuron description information, B is the initial threshold change included in the above neuron layer description information, and E is the updated flipped neuron description information.

[0099] In the sixth step, in response to determining that the total threshold error is greater than or equal to the above-mentioned preset threshold error accuracy and the model accuracy is greater than or equal to the above-mentioned accuracy loss upper limit value, determine the target neuron description information set according to the flipped neuron description information set and the updated neuron description information set, and use the determined target neuron description information set as the updated neuron description information set, and execute the above-mentioned update step again.

[0100] In practice, for each flipped neuron description information in the above-mentioned flipped neuron description information set, the above-mentioned execution entity can determine the target neuron description information set by performing a binary classification method on the flipped neuron threshold corresponding to the above-mentioned flipped neuron description information and the updated neuron threshold included in the updated neuron description information corresponding to the flipped neuron threshold in the updated neuron description information set, and use the determined target neuron description information set as the updated neuron description information set, and execute the above-mentioned updating step again.

[0101] The content of "some optional implementations in some embodiments" in step 1024, as an inventive point of the present disclosure, solves the second technical problem mentioned in the background technical problem, namely, "existing security research mainly focuses on input perturbations, and pays less attention to the security of spike neural network parameters, so that spike neural networks are prone to the risk of malicious manipulation of parameters when facing memory-based attacks." Based on this, first, based on the updated neuron description information set, the following update steps are performed: the first step is to generate a flipped neuron description information set based on the updated neuron description information set. Increase the model energy consumption through bit flip attacks. The second step is to perform a neuron update model test on the above-mentioned spike neural network model based on the flipped neuron description information set to generate model accuracy. Increase the model energy consumption through bit flip attacks without significantly reducing the accuracy. The third step is to determine the total threshold error based on the flipped neuron description information set and the neuron description information set included in the above-mentioned neuron layer description information. The fourth step is to generate sub-optimization information based on the flipped neuron description information set in response to determining that the total threshold error is less than the preset threshold error accuracy. In the fifth step, in response to determining that the total threshold error is greater than or equal to the above-mentioned preset threshold error accuracy and the model accuracy is less than the above-mentioned accuracy loss upper limit value, the flipped neuron description information set is subjected to a second update process, and the updated flipped neuron description information set is used as the updated neuron description information set, and the above-mentioned update step is performed again. In the sixth step, in response to determining that the total threshold error is greater than or equal to the above-mentioned preset threshold error accuracy and the model accuracy is greater than or equal to the above-mentioned accuracy loss upper limit value, the target neuron description information set is determined according to the flipped neuron description information set and the updated neuron description information set, and the determined target neuron description information set is used as the updated neuron description information set, and the above-mentioned update step is performed again. Through a progressive search strategy, the neuron threshold is gradually reduced until the accuracy loss upper limit value is reached. While maintaining the model accuracy, the obfuscated energy consumption characteristics are increased by modifying the stored neuron information, thereby resisting potential malicious attacks.

[0102] Step 103: Generate model optimization information for the spike neural network model based on the obtained sub-optimization information set.

[0103] In some embodiments, the execution entity may generate model optimization information for the spike neural network model based on the obtained sub-optimization information set.

[0104] In practice, the above-mentioned execution entity can merge each sub-optimization information in the obtained sub-optimization information set to obtain model optimization information for the above-mentioned spike neural network model.

[0105] In some optional implementations of some embodiments, the above method further includes:

[0106] The first step is to determine the computational load information and memory usage information of the above-mentioned spike neural network model based on the above-mentioned model optimization information, wherein the above-mentioned computational load information represents the amount of computing resources when the above-mentioned spike neural network model is optimized, and the above-mentioned memory usage information represents the model memory requirement when the above-mentioned spike neural network model is optimized.

[0107] In practice, the above-mentioned execution entity can optimize the above-mentioned spike neural network model through the above-mentioned model optimization information, and monitor the amount of computing resources and model memory requirements during the optimization process to determine the computing load information and memory usage information of the above-mentioned spike neural network model.

[0108] In the second step, in response to determining that the above-mentioned computing load information is less than the preset computing load value, the voltage of the central processing unit is reduced to adjust the system's computing resource allocation to the above-mentioned spike neural network model.

[0109] Among them, the above-mentioned preset computing load value is the amount of computing resources when the above-mentioned spike neural network model is running.

[0110] In practice, in response to determining that the above-mentioned computing load information is less than the preset computing load value, the above-mentioned execution subject can reduce the voltage of the central processing unit through dynamic voltage adjustment. By reducing the voltage, the corresponding power adjustment can be triggered. Specifically, when the voltage is reduced, the power of the central processing unit may be limited. In this case, the processing speed of the central processing unit will be further limited. When the processing speed is limited, the computing resources required for the spike neural network model can be reallocated. For example, due to the limited processing speed, the core computing resources corresponding to the spike neural network can be processed first. For example, data is transitive with the network structure of the spike neural network. Therefore, the computing resources corresponding to the neuron layer currently being processed can be used as core computing resources, and the computing resources corresponding to the neuron layer that has not been processed can be delayed. Loading. In this way, the voltage of the central processing unit can be adaptively adjusted according to the computing needs of the spike neural network, thereby achieving power consumption control.

[0111] The third step is to increase the voltage of the central processing unit in response to determining that the above-mentioned computing load information is greater than or equal to the preset computing load value to adjust the system's computing resource allocation to the above-mentioned spike neural network model.

[0112] In practice, in response to determining that the above-mentioned computing load information is greater than or equal to the preset computing load value, the above-mentioned execution subject can increase the voltage of the central processing unit through dynamic voltage adjustment. By increasing the voltage, the corresponding power adjustment can be triggered. Specifically, when the voltage is increased, the power of the central processing unit may be increased. In this case, the processing speed of the central processing unit will be further improved. When the processing speed is increased, the computing resources required for the spike neural network model can be reallocated. For example, due to the increase in processing speed, more computing resources corresponding to the spike neural network model can be enabled, such as allocating the corresponding computing resources to each neuron layer corresponding to the spike neural network model in full to speed up the calculation speed.

[0113] The fourth step is to reduce the virtual memory or physical memory allocated by the system to the above-mentioned spike neural network model in response to determining that the above-mentioned memory usage information is less than the preset allocated memory amount, so as to adjust the system's storage resource allocation to the above-mentioned spike neural network model.

[0114] The preset allocated memory amount is the memory requirement of the model, including the memory requirement for storing weights, activation values, and intermediate calculation results.

[0115] In practice, in response to determining that the above memory usage information is less than the preset allocated memory amount, the above execution entity can release the memory corresponding to the spike neural network model and some processes that can be closed through system calls or programming interfaces (such as Python's gc module), so as to reduce the virtual memory or physical memory allocated by the system to the above spike neural network model, thereby reducing the storage requirements for the spike neural network model parameters, intermediate calculation results and input data, so as to adjust the system's storage resource allocation to the above spike neural network model.

[0116] The fifth step, in response to determining that the above memory usage information is greater than or equal to the preset allocated memory amount, increase the virtual memory or physical memory allocated by the system to the above spike neural network model to adjust the system's storage resource allocation to the above spike neural network model.

[0117] In practice, in response to determining that the above memory usage information is greater than or equal to the preset allocated memory amount, the above execution subject can release the memory corresponding to the process that can be closed by the process, or divide the memory in an idle state to allocate it to the spike neural network model to meet the memory requirements of the spike neural network model, so as to achieve the purpose of storage resource allocation. At the same time, as the corresponding memory increases, the spike neural network model can load more required calculation data into the memory.

[0118] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the model optimization information generation method applied to the spike neural network model of some embodiments of the present disclosure, the energy consumption attack simulation for the spike neural network model can be realized under the condition of stable model accuracy. Specifically, the reason why it is difficult to realize the energy consumption attack simulation for the spike neural network model under the condition of stable model accuracy is that when the pulse activity of the neuron increases, it usually leads to more memory access operations, thereby greatly increasing energy consumption. Therefore, the traditional method has deficiencies in balancing the energy efficiency and accuracy of the spike neural network, that is, it is difficult to realize the energy consumption attack simulation for the spike neural network model under the condition of stable model accuracy. Based on this, the model optimization information generation method applied to the spike neural network model of some embodiments of the present disclosure, first, the spike neural network model is initialized to generate a model file, wherein the above-mentioned model file includes: a neuron layer description information set and an upper limit value of the accuracy loss, the neuron layer description information in the above-mentioned neuron layer description information set includes: a neuron description information set, an initial threshold change amount and a percentage of high-activity neurons, and the neuron description information in the neuron description information set includes: an initial neuron threshold. Set initial values ​​for the parameters of the spike neural network model to accelerate the training process. Secondly, for each neuron layer description information in the neuron layer description information set, the following processing steps are performed: the first step is to determine the neuron activity corresponding to each neuron description information in the neuron description information set included in the above neuron layer description information. Thus, preparation can be made for the subsequent division of high neuron description information and low activity description information. The second step is to divide the neuron description information in the neuron description information set included in the above neuron layer description information according to the percentage of high activity neurons included in the above neuron layer description information and the neuron activity corresponding to the neuron description information, and obtain a high activity neuron description information set and a low activity neuron description information set. In practice, since the impact of attacking high activity neurons and attacking low activity neurons on the accuracy of the model is different, the neurons are divided from the perspective of neuron activity. The third step is to generate an updated neuron description information set according to the initial threshold change amount included in the above neuron layer description information, the above high activity neuron description information set and the above low activity neuron description information set. Since the impact of attacking low activity neurons on the accuracy of the model is small and the energy consumption is increased less, different strategies are used to update high activity neurons and low activity neurons. The fourth step is to generate sub-optimization information based on the updated neuron description information set. Finally, based on the obtained sub-optimization information set, model optimization information for the above-mentioned spike neural network model is generated.By reducing the neuron threshold, the state of the neuron is affected, making its pulse excitation frequency more frequent under the same conditions, thereby increasing the energy consumption attack effect of the spiking neural network model without reducing the model accuracy, thereby effectively realizing the energy consumption attack simulation for the spiking neural network model under the condition of stable model accuracy.

[0119] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a device for generating model optimization information applied to a spike neural network model. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the model optimization information generation device applied to the spike neural network model can be specifically applied to various electronic devices.

[0120] like Figure 2 As shown, the model optimization information generating device 200 applied to the spike neural network model of some embodiments includes: a model initialization unit 201, a processing unit 202 and a generating unit 203. The model initialization unit 201 is configured to perform model initialization on the spike neural network model to generate a model file, wherein the model file includes: a neuron layer description information set and an upper limit value of accuracy loss, the neuron layer description information in the neuron layer description information set includes: a neuron description information set, an initial threshold change amount and a percentage of high-activity neurons, and the neuron description information in the neuron description information set includes: an initial neuron threshold. The processing unit 202 is configured to perform the following processing steps for each neuron layer description information in the neuron layer description information set: determine the neuron activity corresponding to each neuron description information in the neuron description information set included in the above neuron layer description information; divide the neuron description information in the neuron description information set included in the above neuron layer description information according to the percentage of high-activity neurons included in the above neuron layer description information and the neuron activity corresponding to the neuron description information, and obtain a high-activity neuron description information set and a low-activity neuron description information set; generate an updated neuron description information set according to the initial threshold change amount included in the above neuron layer description information, the above high-activity neuron description information set and the above low-activity neuron description information set; generate sub-optimization information based on the updated neuron description information set. The generation unit 203 is configured to generate model optimization information for the above spike neural network model according to the obtained sub-optimization information set.

[0121] It can be understood that the units recorded in the model optimization information generating device 200 applied to the spike neural network model are similar to the reference Figure 1Therefore, the operations, features and beneficial effects described above for the method are also applicable to the model optimization information generating device 200 applied to the spike neural network model and the units contained therein, and will not be repeated here.

[0122] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device (eg, a computing device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0123] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory 302 or a program loaded from a storage device 308 into a random access memory 303. Various programs and data required for the operation of the electronic device 300 are also stored in the random access memory 303. The processing device 301, the read-only memory 302, and the random access memory 303 are connected to each other via a bus 304. An input / output interface 305 is also connected to the bus 304.

[0124] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0125] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the read-only memory 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0126] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0127] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0128] The computer-readable medium may be included in the electronic device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: performs model initialization on the spike neural network model to generate a model file, wherein the model file includes: a neuron layer description information set and an upper limit value of accuracy loss, the neuron layer description information in the neuron layer description information set includes: a neuron description information set, an initial threshold change amount and a percentage of high-activity neurons, and the neuron description information in the neuron description information set includes: an initial neuron threshold. For each neuron layer description information in the neuron layer description information set, perform the following processing steps: determine the neuron activity corresponding to each neuron description information in the neuron description information set included in the neuron layer description information. According to the percentage of high-activity neurons included in the neuron layer description information and the neuron activity corresponding to the neuron description information, divide the neuron description information in the neuron description information set included in the neuron layer description information into neuron activity, and obtain a high-activity neuron description information set and a low-activity neuron description information set. According to the initial threshold change amount included in the neuron layer description information, the high-activity neuron description information set and the low-activity neuron description information set, an updated neuron description information set is generated. Based on the updated neuron description information set, sub-optimization information is generated. Based on the obtained sub-optimization information set, model optimization information for the spike neural network model is generated.

[0129] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0130] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0131] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor including a model initialization unit, a processing unit, and a generation unit. Among them, the names of these units do not constitute a limitation on the units themselves in certain cases, for example, the model initialization unit may also be described as a "unit for initializing the model of the spike neural network model to generate a model file".

[0132] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0133] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A method for generating model optimization information applied to a spiking neural network model, comprising: Initializing the spike neural network model to generate a model file, wherein the model file includes: a neuron layer description information set and an upper limit value of accuracy loss, the neuron layer description information in the neuron layer description information set includes: a neuron description information set, an initial threshold change amount and a percentage of high-activity neurons, and the neuron description information in the neuron description information set includes: an initial neuron threshold; For each neuron layer description in the neuron layer description set, perform the following processing steps: Determine the neuron activity corresponding to each neuron description information in the neuron description information set included in the neuron layer description information; According to the percentage of high-activity neurons included in the neuron layer description information and the neuron activity corresponding to the neuron description information, dividing the neuron description information in the neuron description information set included in the neuron layer description information into neuron activity to obtain a high-activity neuron description information set and a low-activity neuron description information set; Generate an updated neuron description information set according to the initial threshold change amount, the high-activity neuron description information set and the low-activity neuron description information set included in the neuron layer description information; Generate sub-optimization information based on the updated neuron description information set; Based on the obtained sub-optimization information set, model optimization information for the spike neural network model is generated.

2. The method according to claim 1, wherein: The method further comprises: Determine the computational load information and memory usage information of the spike neural network model according to the model optimization information, wherein the computational load information represents the amount of computational resources when the spike neural network model is optimized, and the memory usage information represents the amount of model memory required when the spike neural network model is optimized; In response to determining that the computational load information is less than a preset computational load value, reducing a voltage of a central processing unit to adjust a system's computational resource allocation to the spiking neural network model; In response to determining that the computational load information is greater than or equal to a preset computational load value, increasing a voltage of a central processing unit to adjust a system's computational resource allocation to the spike neural network model; In response to determining that the memory usage information is less than a preset allocated memory amount, reducing the virtual memory or physical memory allocated by the system to the spiking neural network model to adjust the system's allocation of storage resources to the spiking neural network model; In response to determining that the memory usage information is greater than or equal to a preset allocated memory amount, the virtual memory or physical memory allocated by the system to the spike neural network model is increased to adjust the system's allocation of storage resources to the spike neural network model.

3. The method according to claim 2, wherein: The determining of the neuron activity corresponding to each neuron description information in the neuron description information set included in the neuron layer description information comprises: Determine the preset time window; Determine the number of pulses emitted by the neuron corresponding to the neuron description information within a preset time window to obtain a target number of pulses; A quotient of the target pulse quantity and the preset time window is determined as the neuron activity corresponding to the neuron description information.

4. The method according to claim 3, wherein: The neuron description information in the neuron description information set included in the neuron layer description information is divided into neuron activity levels according to the percentage of high-activity neurons and the neuron activity corresponding to the neuron description information, to obtain a high-activity neuron description information set and a low-activity neuron description information set, including: Determine the number of high-activity neurons according to the number of neuron description information in the neuron description information set included in the neuron layer description information and the percentage of high-activity neurons; Sort the neuron description information in the neuron description information set included in the neuron layer description information according to the neuron activity corresponding to the neuron description information to obtain a neuron description information sequence; Filtering out neuron description information whose corresponding bit sequence is greater than or equal to the target bit sequence from the neuron description information sequence as high-activity neuron description information, and obtaining a high-activity neuron description information set, wherein the target bit sequence is equal to the number of the high-activity neurons; Neuron description information whose corresponding bit sequence is smaller than the target bit sequence is selected from the neuron description information sequence as low-activity neuron description information to obtain a low-activity neuron description information set.

5. The method according to claim 4, wherein: The step of generating an updated neuron description information set according to the initial threshold change amount, the high-activity neuron description information set, and the low-activity neuron description information set included in the neuron layer description information comprises: According to the initial threshold change amount included in the neuron layer description information, performing a first update process on each high-activity neuron description information in the high-activity neuron description information set to generate updated high-activity neuron description information, thereby obtaining an updated high-activity neuron description information set; According to the initial threshold value variation amount included in the neuron layer description information, performing a second update process on each low-activity neuron description information in the low-activity neuron description information set to generate updated low-activity neuron description information, thereby obtaining an updated low-activity neuron description information set; The updated high-activity neuron description information set and the updated low-activity neuron description information set are determined as the updated neuron description information set.

6. A model optimization information generation device applied to a spike neural network model, comprising: A model initialization unit is configured to perform model initialization on the spike neural network model to generate a model file, wherein the model file includes: a neuron layer description information set and an upper limit value of accuracy loss, the neuron layer description information in the neuron layer description information set includes: a neuron description information set, an initial threshold change amount and a percentage of high-activity neurons, and the neuron description information in the neuron description information set includes: an initial neuron threshold; The processing unit is configured to perform the following processing steps for each neuron layer description information in the neuron layer description information set: determine the neuron activity corresponding to each neuron description information in the neuron description information set included in the neuron layer description information; divide the neuron description information in the neuron description information set included in the neuron layer description information according to the percentage of high-activity neurons included in the neuron layer description information and the neuron activity corresponding to the neuron description information, so as to obtain a high-activity neuron description information set and a low-activity neuron description information set; generate an updated neuron description information set according to the initial threshold change amount included in the neuron layer description information, the high-activity neuron description information set and the low-activity neuron description information set; and generate sub-optimization information based on the updated neuron description information set; A generating unit is configured to generate model optimization information for the spike neural network model based on the obtained sub-optimization information set.

7. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images