Impulse neural network construction method

Abstract

The invention discloses a spiking neural network construction method, relates to the technical field of deep learning, and solves the problems that the performance of an existing spiking neural network on a computer vision task is insufficient and the search time is long when an NAS is used. The method comprises the following steps: S1, determining an application scene and a performance requirement of a spiking neural network; s2, collecting data corresponding to the application scene and preprocessing the data; s3, performing training quantization on the artificial neural network, and then converting the trained artificial neural network into a spiking neural network according to an equivalent mapping relation between spatial quantization of the artificial neural network and time quantization of the spiking neural network; s4, searching a pulse neural network with the best performance according to a peak neural structure search algorithm; s5, deploying the spiking neural network with the optimal performance to a specified application scene; according to the method, the optimal SNN architecture is automatically searched by introducing a spiking neural structure search method, so that the requirement on manual design is reduced, and the design efficiency and the network performance are improved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a pulse neural network construction method. Background Art

[0002] As a computing model inspired by the brain, the design concept of spiking neural network (SNN) is to simulate the way neurons in biological nervous systems transmit information through discrete and sparse spikes (pulses). This model is not only closer to the working mechanism of biological brains in theory, but also shows remarkable energy efficiency in practical applications. This feature of SNN is mainly due to its event-driven representation, that is, the calculation is triggered only when the input changes, which is in sharp contrast to the continuous and intensive calculation in traditional artificial neural networks (ANN).

[0003] With the development of deep learning, the choice of neural structure has an increasingly prominent impact on network performance. However, the network structure design of SNN currently mostly relies on manual design, which is not only time-consuming, but also easily limited by the designer's experience and knowledge, and difficult to adapt to rapidly changing application requirements. In addition, the advantages of SNN in computing and energy efficiency, especially the use of addition instead of weight multiplication in ANN, further reduces the computing intensity and energy consumption, making it potentially valuable for application in resource-constrained environments.

[0004] Despite the many advantages of SNN, its training process faces challenges, mainly due to the discreteness of the impulse function. This discreteness makes the design of SNN training algorithms complicated, especially in deep networks. To overcome this challenge, researchers have proposed a variety of methods, one of which is the ANN to SNN conversion method. This method attempts to use the mature training technology of ANN to initialize the weights of SNN, and then convert the ANN model into an SNN model through a specific conversion strategy.

[0005] However, the following technical problems still exist:

[0006] Limitations of manually designed network structures: The current network structure design of SNNs mainly relies on manual processes, which is not only time-consuming but also easily limited by the designer's experience and knowledge, making it difficult to adapt to diverse application scenarios.

[0007] Challenges in the training process: Due to the discreteness of the impulse function, the training process of deep SNN is complex and it is difficult to implement an efficient learning algorithm.

[0008] Efficiency issue of ANN to SNN conversion: Existing ANN to SNN conversion methods often achieve high performance but are accompanied by high latency, which limits the potential of SNN in real-time applications.

[0009] Computational cost of neural architecture search (NAS): When applying NAS in SNNs, the computational cost of searching the optimal network structure is high, which hinders the widespread application of NAS in SNN design. Summary of the invention

[0010] In order to solve the problems existing in the above-mentioned prior art, the present invention provides a method for constructing a pulse neural network to solve the problems that the existing pulse neural network has insufficient performance in computer vision tasks and has a long search time when using NAS.

[0011] A method for constructing a pulse neural network, comprising:

[0012] S1: Determine the application scenarios and performance requirements of spiking neural networks;

[0013] S2: Collect data corresponding to the application scenario and perform preprocessing;

[0014] S3: training and quantizing the artificial neural network, and then converting the trained artificial neural network into a spiking neural network according to the equivalent mapping relationship between the spatial quantization of the artificial neural network and the temporal quantization of the spiking neural network;

[0015] S4: A spiking neural network that searches for the best performing spiking neural network based on a spiking neural architecture search algorithm;

[0016] S5: Deploy the best performing spiking neural network to the specified application scenario.

[0017] Furthermore, the equivalent mapping relationship between the spatial quantization of the artificial neural network and the temporal quantization of the spiking neural network in S3 includes:

[0018] A uniform quantization function is constructed as the activation function of the artificial neural network. This function discretizes the activation values ​​of the neurons in the lth layer of the ReLU-activated neural network in the spatial dimension. The discretization formula is:

[0019]

[0020] in represents the spatial quantization value, b represents the number of bits (precision) of the quantizer, assuming that the time window of the spike neural network is T, the peak count should be {0,1,...,T}, then the state number of the artificial neural network is defined as {0,1,...,2 b -1}, the round operator indicates that the function rounds the number to the given number of digits, clip_threshold l Indicates decision input The clipping operator limits the elements in the array to the range between 0 and 2.b -1.

[0021] In a spiking neural network, at time step t, the total membrane charge of neuron i in layer l is It is expressed as:

[0022]

[0023] Among them, M prev represents the set of neurons in the previous layer, is the weight of the synaptic connection between the current neuron i and neuron j, is the bias term, input j (t) represents the input of neuron j from the previous layer at time t.

[0024] The expression of the spiking neuron is as follows:

[0025] The membrane potential at the current time t is and the membrane potential at the previous moment prev The potential difference expression is as follows:

[0026]

[0027] Where fire_threshold is the firing threshold, t is the time step, If it is a step function, the peak value of the trigger can be obtained as

[0028] In spiking neural networks, the quantized fixed value The discharge rate express:

[0029]

[0030] The floor operator returns the largest integer value that is less than or equal to the passed parameter.

[0031] Comparing Equation 5 with Equation 1, we can obtain an equivalent mapping between spatial quantization in artificial neural networks (ANN) and temporal quantization in spiking neural networks (SNN), and derive the activation equivalence between artificial neural networks and SNNs:

[0032]

[0033] Furthermore, the peak neural structure search algorithm in S4 includes:

[0034] Starting from the commonly used building blocks of SNN architectures related to previous application scenarios, peak residual blocks with different convolution kernel sizes are constructed. The training accuracy of different architectural combinations of neurons is used as an indicator to screen the optimal architecture from many architectural candidates.

[0035] Furthermore, the SNN includes a spike coding layer and a peak residual block SRB. The spike coding layer is used to convert the image mapping value obtained by previous processing into a peak value. The peak residual block SRB includes multiple nodes. The microstructure of each node includes a convolutional layer for extracting image features, peak neuron activation for generating spike signals for image feature processing, and residual connection for enhancing image information processing capabilities. At the same time, a learnable membrane threshold is used to adjust the pulse activation, and finally the network structure of the SNN and the parameter information of each layer, including weights, thresholds, biases, etc., are obtained.

[0036] Furthermore, the convolution layer is responsible for extracting the key features of the data, and can effectively capture feature information at different levels through the sliding operation of the convolution kernel on the data; the peak neuron activation gives full play to the unique advantages of peak neurons, efficiently processes the input signal, generates discrete spike signals, and realizes rapid transmission of information; the introduction of the residual connection effectively solves the common problems of gradient disappearance in deep networks, so that the network can maintain good performance during training. It directly transmits part of the input signal to the subsequent layers, thereby significantly enhancing the overall expression ability of the network.

[0037] Furthermore, the size of the convolution kernel in each SRB is set, different SNN neural structures are defined, and the quality of different architectures is evaluated based on the training accuracy and the number of peaks.

[0038] Furthermore, the peak neural structure search algorithm described in S4 also includes performance comparison by using a linear classifier, so as to objectively and accurately evaluate the advantages and disadvantages of different architectures, providing a strong basis for selecting the optimal SNN architecture.

[0039] The beneficial effects of the present invention include:

[0040] Automated SNN structure design: By introducing the spiking neural architecture search (NAS) method, the optimal SNN architecture is automatically found, reducing the need for manual design and improving design efficiency and network performance.

[0041] Improve SNN conversion efficiency: Design a low-latency, high-performance SNN conversion method to achieve fast and accurate conversion between ANN and SNN, ensuring advanced performance on various computer vision tasks.

[0042] Optimize the SNN training process: By demonstrating the equivalent mapping between the temporal quantization of SNN and the spatial quantization of ANN, the SNN training process is simplified, the quantization error is reduced, and the training efficiency is improved.

[0043] Reducing the computational cost of NAS: The performance of candidate architectures is evaluated with the aim of reducing the amount of computation in the NAS process and making NAS more feasible in SNN design. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flowchart of a method for constructing a pulse neural network involved in an embodiment of the present application.

[0045] Figure 2 This is a network architecture diagram of a pulse neural network construction process involved in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0047] A method for constructing a spiking neural network, such as Figure 1 As shown, including:

[0048] S1: Determine the application scenarios and performance requirements of spiking neural networks;

[0049] S2: Collect data corresponding to the application scenario and perform preprocessing;

[0050] S3: training and quantizing the artificial neural network, and then converting the trained artificial neural network into a spiking neural network according to the equivalent mapping relationship between the spatial quantization of the artificial neural network and the temporal quantization of the spiking neural network;

[0051] S4: A spiking neural network that searches for the best performing spiking neural network based on a spiking neural architecture search algorithm;

[0052] S5: Deploy the best performing spiking neural network to the specified application scenario.

[0053] In this embodiment, the target scenario is the field of government services. In this scenario, image data plays an important role, such as images of business licenses submitted by companies, images of certification materials submitted by citizens when applying for certificates, and scanned images of various documents in the process of government office work.

[0054] First, data collection and processing are carried out: various types of government data mainly in the form of pictures are collected from the government service system, such as pictures of business licenses of enterprises, pictures of citizens' ID cards, scanned copies of policy documents, etc. For each picture, the type of government service it belongs to (such as enterprise registration, personal social security processing, policy interpretation, etc.), the category of the subject involved (enterprise or individual), and the scope of key information are identified. For example, in the business license picture, the scope of key information can be the area where the company name, business scope, registered capital, etc. are located. The minimum rectangular area containing these key information can be determined by image recognition technology, and the four corner coordinates of the rectangle are used to refer to it (the upper left corner of the image is taken as the origin of the coordinate system). At the same time, these key information are extracted and marked for subsequent processing. Service types can also be referred to by numbers, such as 0 for enterprise registration and registration picture-related services, 1 for personal social security processing picture-related services, and 2 for policy interpretation picture-related services.

[0055] According to the constructed conversion network (based on the artificial neural network and pulse neural network conversion mapping technology in this patent), the construction and training process is completed with the help of a deep learning framework (such as Pytorch, etc.) and algorithms suitable for image data processing.

[0056] like Figure 2 As shown, during the calculation, the input of the first layer is the collected government image data, and several images are randomly selected as a group of inputs. The input of the subsequent layer is the output of the previous layer after processing the image.

[0057] In artificial neural networks (ANN), spatial quantization technology is used to quantize various types of information in the image. For example, text information and graphic information in the image are quantized and encoded according to their characteristics and importance, and the quantization results are finally output.

[0058] Subsequently, based on the activation equivalence between ANN and SNN, the ANN network is converted into a spiking neural network (SNN). During the conversion process, the output of each layer in the SNN form after processing the image information is made as close as possible to the output of each layer in the ANN form through equivalent mapping analysis of spatial quantization (mapping the image information to discrete finite values) and temporal quantization (analogous to the time series related quantization of the image information during processing); before the conversion, different quantization precisions are used for training to select an artificial neural network with higher accuracy.

[0059] After the conversion is completed, the peak neural network of the best performing SNN is automatically found using the peak neural architecture search (NAS) technique.

[0060] In the application of NAS technology, starting from the commonly used building blocks of SNN architecture related to previous government services, peak residual blocks (SRBs) with different convolution kernel sizes are constructed. The training accuracy of different architectural combinations of neurons is used as an indicator to screen the optimal architecture from many architectural candidates.

[0061] In its macroscopic skeleton, the first block is the spike coding layer, which converts the previously processed image mapping value into a peak value. The peak residual block SRB contains a specific number of nodes. Its microscopic structure includes convolutional layers (for extracting image features), peak neuron activations (generating spike signals for image feature processing) and residual connections (enhancing image information processing capabilities), etc. At the same time, the learnable membrane threshold is used to adjust the pulse activation, and finally the network structure of the SNN and the parameter information of each layer are obtained, including weights, thresholds, biases, etc.

[0062] Finally, the network is deployed on edge deployment hardware dedicated to government services (such as smart terminal devices in government service halls, etc.). The government system collects image data related to government services in real time (such as business processing image materials newly submitted by enterprises, document images of newly issued policies, etc.), and transmits the image information to the deployed hardware for calculation. After the calculation is completed, the processing suggestions for government service matters (such as whether the information in the image is complete and accurate, whether it meets the processing conditions, etc.), policy matching results (analysis of the correlation between the business in the image and the existing policies), service process optimization direction (how to optimize the processing process based on the image information processing analysis) and other prediction results can be obtained. Then, the prediction results are visualized through the government service platform or hall display screen, etc., so as to improve the efficiency, accuracy and humanization level of government services, ensure the authenticity and security of government information, and achieve comprehensive optimization of government office.

[0063] In another embodiment, the equivalent mapping relationship between the spatial quantization of the artificial neural network and the temporal quantization of the spiking neural network is calculated in S3, including:

[0064] A uniform quantization function is constructed as the activation function of the artificial neural network. This function discretizes the activation values ​​of the neurons in the lth layer of the ReLU-activated neural network in the spatial dimension. The discretization formula is:

[0065]

[0066] in represents the spatial quantization value, b represents the number of bits (precision) of the quantizer, assuming that the time window of the spike neural network is T, the peak count should be {0,1,...,T}, then the state number of the artificial neural network is defined as {0,1,...,2 b -1}, the round operator indicates that the function rounds the number to the given number of digits, clip_thresholdl Indicates decision input The clipping operator limits the elements in the array to the range between 0 and 2. b -1.

[0067] In a spiking neural network, at time step t, the total membrane charge of neuron i in layer l is It is expressed as:

[0068]

[0069] Among them, M prev represents the set of neurons in the previous layer, is the weight of the synaptic connection between the current neuron i and neuron j, is the bias term, input j (t) represents the input of neuron j from the previous layer at time t.

[0070] The expression of the spiking neuron is as follows:

[0071] The membrane potential at the current time t is and the membrane potential at the previous moment prev The potential difference expression is as follows:

[0072]

[0073] Where fire_threshold is the firing threshold, t is the time step, If it is a step function, the peak value of the trigger can be obtained as

[0074] In spiking neural networks, the quantized fixed value The discharge rate express:

[0075]

[0076] The floor operator returns the largest integer value that is less than or equal to the passed parameter.

[0077] Comparing Equation 5 with Equation 1, we can obtain an equivalent mapping between spatial quantization in artificial neural networks (ANN) and temporal quantization in spiking neural networks (SNN), and derive the activation equivalence between artificial neural networks and SNNs:

[0078]

[0079]

[0080] In another embodiment, the peak neural structure search algorithm described in S4 includes:

[0081] Starting from the commonly used building blocks of SNN architectures related to previous application scenarios, peak residual blocks with different convolution kernel sizes are constructed. The training accuracy of different architectural combinations of neurons is used as an indicator to screen the optimal architecture from many architectural candidates.

[0082] In another embodiment, the SNN includes a spike coding layer and a peak residual block SRB. The spike coding layer is used to convert the image mapping value obtained by previous processing into a peak value. The peak residual block SRB includes multiple nodes. The microstructure of each node includes a convolutional layer for extracting image features, peak neuron activation for generating spike signals for image feature processing, and residual connection for enhancing image information processing capabilities. At the same time, a learnable membrane threshold is used to adjust the pulse activation, and finally the network structure of the SNN and the parameter information of each layer, including weights, thresholds, biases, etc., are obtained.

[0083] In another embodiment, the convolution layer is responsible for extracting key features of the data, and can effectively capture feature information at different levels through the sliding operation of the convolution kernel on the data; the peak neuron activation gives full play to the unique advantages of the peak neuron, efficiently processes the input signal, generates discrete spike signals, and realizes rapid transmission of information; the introduction of the residual connection effectively solves the common problems of gradient disappearance in deep networks, so that the network can maintain good performance during training. It directly transmits part of the input signal to the subsequent layers, thereby significantly enhancing the overall expression ability of the network.

[0084] In another embodiment, the size of the convolution kernel in each SRB is set, different SNN neural structures are defined, and the quality of different architectures is evaluated based on the training accuracy and the number of peaks.

[0085] In another embodiment, the peak neural structure search algorithm described in S4 also includes comparing performance by using a linear classifier, so as to objectively and accurately evaluate the advantages and disadvantages of different architectures, providing a strong basis for selecting the optimal SNN architecture.

[0086] The above-mentioned embodiments only express the specific implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the protection scope of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the technical solution concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A method for constructing a pulse neural network, characterized in that: include: S1: Determine the application scenarios and performance requirements of spiking neural networks; S2: Collect data corresponding to the application scenario and perform preprocessing; S3: training and quantizing the artificial neural network, and then converting the trained artificial neural network into a spiking neural network according to the equivalent mapping relationship between the spatial quantization of the artificial neural network and the temporal quantization of the spiking neural network; S4: Searching for the best performing spiking neural network based on the spiking neural architecture search algorithm; S5: Deploy the best performing spiking neural network to the specified application scenario.

2. A method for constructing a pulse neural network according to claim 1, characterized in that: The equivalent mapping relationship between the spatial quantization of artificial neural networks and the temporal quantization of spiking neural networks in S3 includes: A uniform quantization function is constructed as the activation function of the artificial neural network. This function discretizes the activation values ​​of the neurons in the lth layer of the ReLU-activated neural network in the spatial dimension. The discretization formula is: in represents the spatial quantization value, b represents the number of bits of the quantizer, assuming that the time window of the spike neural network is T, the peak count should be {0,1,...,T}, then the state number of the artificial neural network is defined as {0,1,...,2 b -1}, the round operator indicates that the function rounds the number to the given number of digits, clip_threshold l Indicates decision input The clipping operator limits the elements in the array to the range between 0 and 2. b -1. In a spiking neural network, at time step t, the total membrane charge of neuron i in layer l is It is expressed as: Among them, M prev represents the set of neurons in the previous layer, is the weight of the synaptic connection between the current neuron i and neuron j, is the bias term, input j (t) represents the input of neuron j from the previous layer at time t. The expression of the spiking neuron is as follows: The membrane potential at the current time t is and the membrane potential at the previous moment prev The potential difference expression is as follows: Where fire_threshold is the firing threshold, t is the time step, If it is a step function, the peak value of the trigger can be obtained as In spiking neural networks, the quantized fixed value The discharge rate express: The floor operator returns the largest integer value that is less than or equal to the passed parameter. Comparing Equation 5 with Equation 1, we can obtain an equivalent mapping between spatial quantization in artificial neural networks (ANN) and temporal quantization in spiking neural networks (SNN), and derive the activation equivalence between artificial neural networks and SNNs:

3. A method for constructing a pulse neural network according to claim 1, characterized in that: S4 The peak neural structure search algorithm includes: Starting from the commonly used building blocks of SNN architectures related to previous application scenarios, peak residual blocks with different convolution kernel sizes are constructed. The training accuracy of different architectural combinations of neurons is used as an indicator to screen the optimal architecture from many architectural candidates.

4. A method for constructing a pulse neural network according to claim 3, characterized in that: The SNN includes a spike coding layer and a peak residual block SRB. The spike coding layer is used to convert the image mapping value obtained by previous processing into a peak value. The peak residual block SRB includes multiple nodes. The microstructure of each node includes a convolution layer for extracting image features, peak neuron activation for generating spike signals for image feature processing, and residual connection for enhancing the image information processing capability. At the same time, a learnable membrane threshold is used to adjust the pulse activation, and finally the network structure of the SNN and the parameter information of each layer, including weights, thresholds, and biases, are obtained.

5. A method for constructing a pulse neural network according to claim 4, characterized in that: The convolution layer is responsible for extracting the key features of the data and capturing feature information at different levels by sliding the convolution kernel on the data; the peak neuron activation processes the input signal to generate discrete spike signals to achieve information transmission; the introduction of the residual connection directly transmits part of the input signal to the subsequent layers, thereby enhancing the overall expression ability of the network.

6. A method for constructing a pulse neural network according to claim 3, characterized in that: Different SNN neural structures are defined according to the convolution kernel size in each SRB, and the quality of different architectures is evaluated based on the training accuracy and the number of peaks.

7. A method for constructing a pulse neural network according to claim 3, characterized in that: The peak neural structure search algorithm described in S4 also includes comparing performance by using a linear classifier to evaluate the advantages and disadvantages of different architectures, providing a basis for selecting the optimal SNN architecture.

Images