Data identification method based on spiking neural network model, computer equipment, readable storage medium and program product

Abstract

The invention relates to the technical field of data recognition, in particular to a data recognition method based on a spiking neural network model, computer equipment, a readable storage medium and a program product, and provides the data recognition method based on the spiking neural network model, which combines the spiking neural network model with an attention mechanism. The method enables the model to capture the features in the data more easily, carries out the better extraction of the pulse signal, improves the recognition processing efficiency of the model on the data, and remarkably improves the accuracy of the data features extracted by the model.

Description

[Technical field]

[0001] The present invention relates to the field of data recognition technology, and in particular to a data recognition method, computer equipment, readable storage medium and program product based on a pulse neural network model. [Background technology]

[0002] Dynamic Vision Sensor (DVS) is a new type of image sensor designed to simulate the working mode of biological vision system. Unlike traditional frame-based image sensors, DVS has higher temporal resolution and lower latency by capturing changing events in the scene instead of capturing complete images at fixed time intervals per second. As a new type of sensing device, DVS makes up for the problem of missing information in traditional images in high-speed motion scenes due to its low latency, high dynamic range, high temporal resolution and low power consumption.

[0003] Inputting dynamic vision sensor (DVS) data into spiking neural network (SNN) is a promising research direction, which can not only improve the real-time performance and response speed of machine vision systems, but also promote the application and development of intelligent systems in dynamic environments. However, the performance of SNN networks trained by directly inputting dynamic vision sensor data from dynamic vision sensors into SNN is not satisfactory and does not meet the high standards required for practical applications in real scenarios. The event data generated by DVS is sparse, which means that SNN may receive uneven input when processing, which may lead to information loss or inefficient processing. Due to the strong sparsity of dynamic vision sensor data, this method cannot effectively extract information from events. [Summary of the invention]

[0004] In order to solve the problem that the existing data recognition method cannot effectively extract key information from event data, the present invention provides a data recognition method based on a pulse neural network model, and the data recognition method includes the following steps:

[0005] Provide a spiking neural network model including an input layer, a convolutional layer, a fully connected layer, and an output layer connected in sequence;

[0006] Inputting external data information into the pulse neural network model, the pulse neural network model receives the external data information through an input layer, the input layer converts the external data information into first pulse information and outputs it to a convolution layer, the convolution layer extracts features from the first pulse information and outputs second pulse information according to a preset weight;

[0007] The convolutional layer is also connected to an attention module, which is also connected to the fully connected layer. The attention module receives the second pulse information output by the convolutional layer and recalculates the weight of the feature, and then outputs the third pulse information to the fully connected layer.

[0008] The fully connected layer performs weighted integration on the third pulse information and transmits it to the output layer, and the output layer generates the prediction results of the pulse neural network model about the input external data information.

[0009] Preferably, the external information received by the input layer includes dynamic visual sensing data obtained by a dynamic visual sensor.

[0010] Preferably, the spiking neural network model is trained by a gradient substitution method.

[0011] Preferably, the loss function in the training process of the pulse neural network model is calculated using a mean square error function.

[0012] Preferably, a sigmoid function is used instead of a step function during the training process of the pulse neural network model.

[0013] Preferably, the attention module uses a SE module, an ECA module or a CBAM module.

[0014] Preferably, LIF neurons, IF neurons or PLIF neurons are used in the spiking neural network model.

[0015] In order to solve the above technical problems, the present invention also provides a computer device, including a memory, a processor and a computer program stored on the memory, and the processor executes the computer program to implement the steps of the data recognition method based on the pulse neural network model.

[0016] In order to solve the above technical problems, the present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of a data recognition method based on a pulse neural network model as described in any one of claims 1 to 7 are implemented.

[0017] In order to solve the above technical problems, the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the data recognition method based on a pulse neural network model.

[0018] Compared with the prior art, the data recognition method, computer device, readable storage medium and program product based on the pulse neural network model provided by the present invention have the following advantages:

[0019] 1. The present invention provides a data recognition method based on a pulse neural network model, which realizes recognition processing on data by using a pulse neural network model, uses a convolutional layer as an intermediate layer in the pulse neural network model to process external data information input to the model, and then extracts important features in the data through the convolutional layer, eliminates redundant information, and compresses the data into a more meaningful pulse signal; the method uses a convolutional layer to replace a traditional encoding layer, which can significantly improve the efficiency of feature extraction and data representation. The convolutional layer can capture local features and discover basic patterns such as edges and textures, making the model more suitable for image processing;

[0020] At the same time, this method also connects an attention module to the convolutional layer of the model. The attention mechanism of the attention module can pay different attention to different features by calculating the importance of the features, so that the model can better understand and use the input data; the convolutional layer is good at extracting local features, but there may be certain limitations in capturing global context information. By connecting the attention module to the convolutional layer, it can capture a broader context relationship by paying attention to different parts of the entire input feature map, which enables the network to understand the structure and information of the data more comprehensively; by connecting the attention module, the model can adaptively select and emphasize the key parts of the input features according to the needs of the current task, thereby enhancing the model's ability to extract features from the data;

[0021] This method combines the pulse neural network model with the attention mechanism, making it easier for the model to capture features in the data and better extract pulse signals, thereby improving the model's efficiency in identifying and processing data. At the same time, it improves the accuracy of data features extracted by the model through global attention and adaptive learning.

[0022] 2. In a data recognition method based on a pulse neural network model provided by the present invention, the external information received by the input layer of the model includes dynamic visual sensing data obtained by a dynamic visual sensor. The dynamic visual sensor can capture changes in the scene with extremely high temporal resolution. The dynamic visual sensor only sends events where changes occur in the image. This can significantly reduce data redundancy and computational burden, but will increase the sparsity of dynamic visual sensing data. The data recognition method provided by the present invention improves the ability to extract and process features in the data, and can solve the problem of inability to effectively extract features due to the sparsity of dynamic visual sensing data. This allows the method to solve the problem of missing information in traditional images in high-speed motion scenes through dynamic visual sensors, and can quickly and effectively identify and process the dynamic visual sensing data generated by it.

[0023] 3. The pulse neural network model in the present invention is trained by the gradient substitution method. Since the output of the pulse neuron in the pulse neural network model is a discrete pulse signal, the traditional back propagation algorithm cannot be directly applied to the training of the pulse neural network model. However, effective training can be achieved through pulse coding, a specific forward propagation mechanism and suitable learning rules. In addition, the practicality and performance of the pulse neural network model can be improved through training strategies suitable for specific tasks.

[0024] 4. The loss function in the training process of the pulse neural network model in the present invention is calculated using the mean square error function. During the training process, a loss function is usually required to evaluate the difference between the network output and the target output. Since the pulse signal in the model is discrete, the pulse signal can be converted into frequency or pulse emission rate through the mean square error function, thereby calculating its mean square error, thereby optimizing the training process of the model.

[0025] 5. In the training process of the pulse neural network model in the present invention, the sigmoid function is used instead of the step function. In the pulse neural network model, the step function is usually used to generate pulses, and the output of the step function is used to calculate the loss function during the training process. However, the step function is prone to gradient vanishing or inability to perform effective gradient updates during training. Therefore, using a continuous and differentiable sigmoid function as a substitute for the step function can provide continuous output for the pulse neural network model, making the model easier to train and improving the convergence speed.

[0026] 6. The attention module in the present invention uses an SE module, an ECA module or a CBAM module. The SE module enhances the representation ability of the feature channel through a compression-excitation mechanism, and can dynamically adjust the importance of the channel to improve the representation ability of the model, thereby effectively improving the accuracy of the model. The ECA module aims to improve the computational efficiency of the channel attention mechanism. The ECA module does not require additional parameters, avoids a large fully connected layer, and calculates the relationship between channels by using one-dimensional convolution. The calculation is simple and fast, and is suitable for occasions requiring fast reasoning. The CBAM module combines channel attention and spatial attention, and improves the feature expression ability through a two-stage attention mechanism, taking into account the characteristics of channels and space, and can more comprehensively adjust the expression of feature maps, which helps to improve model performance. The present invention can meet the use requirements for different data by adjusting and using different attention modules, thereby improving the applicability of the method.

[0027] 7. In the pulse neural network model of the present invention, LIF neurons, IF neurons or PLIF neurons are used. IF neurons are simple and easy to use and suitable for basic research; LIF neurons are closer to biological neurons by introducing leakage characteristics and are suitable for models that require biological realism; PLIF neurons provide greater flexibility and adaptability through their piecewise linear characteristics and can simulate complex neuronal behaviors. By selecting a suitable neuron model, the computational complexity and accuracy can be balanced according to specific data processing tasks and requirements.

[0028] 8. The present invention also provides a computer device having the same beneficial effects as the above-mentioned data recognition method based on the pulse neural network model, which will not be elaborated here.

[0029] 9. The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed, it implements the above-mentioned data recognition method based on the pulse neural network model, and has the same beneficial effects as the above-mentioned data recognition method based on the pulse neural network model, which will not be elaborated here.

[0030] 10. The present invention also provides a computer program product, which has the same beneficial effects as the above-mentioned data recognition method based on the pulse neural network model, which will not be elaborated here.

Brief Description of the Drawings

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0032] Figure 1 It is a step flow chart of a data recognition method based on a pulse neural network model provided by the first embodiment of the present invention.

[0033] Figure 2 It is a flowchart of the steps of a training method for a pulse neural network model provided by the second embodiment of the present invention.

[0034] Figure 3 It is a performance comparison chart of three types of neurons provided in the first embodiment of the present invention on the SED data set.

[0035] Figure 4 This is a performance comparison chart of three different attention modules provided in the first embodiment of the present invention on the N-MNIST dataset.

[0036] Figure 54 is a performance comparison chart of three different attention modules provided in the first embodiment of the present invention on the DVS Gesture 128 dataset.

[0037] Figure 6 This is a performance comparison chart of three different attention modules provided in the first embodiment of the present invention on the CIFAR10-DVS dataset.

[0038] Figure 7 It is a schematic diagram of the structure of a computer device provided by the third embodiment of the present invention.

[0039] Figure 8 It is a schematic diagram of the structure of a computer-readable storage medium provided by the fourth embodiment of the present invention.

[0040] Fig. 9 It is a schematic diagram of the structure of a computer program product provided by the fifth embodiment of the present invention.

[0041] Description of the accompanying drawings:

[0042] 10. Computer device; 20. Computer readable storage medium; 30. Computer program product. [Specific implementation method]

[0043] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and implementation examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0044] Inputting dynamic vision sensor data into a spiking neural network can improve the real-time performance and response speed of the machine vision system. However, the performance of the network trained by directly inputting the dynamic vision sensor data of the dynamic vision sensor into the spiking neural network is not satisfactory and does not meet the high standards required for practical applications in real scenarios. The event data generated by the dynamic vision sensor is sparse, which may cause the network to receive uneven input during processing, which may lead to information loss or inefficient processing and inability to effectively extract information from the event.

[0045] To solve the problem that existing data identification methods cannot effectively extract key information from event data, please refer to Figure 1 The first embodiment of the present invention provides a data recognition method based on a pulse neural network model, and the data recognition method specifically includes the following steps:

[0046] S1: Provide a spiking neural network model including an input layer, a convolutional layer, a fully connected layer, and an output layer connected in sequence;

[0047] S2: Inputting external data information into the pulse neural network model, the pulse neural network model receives the external data information through the input layer, the input layer converts the external data information into first pulse information and outputs it to the convolution layer, the convolution layer extracts features in the first pulse information and outputs second pulse information according to preset weights;

[0048] S3: The convolution layer is also connected to an attention module, which is also connected to the fully connected layer. The attention module receives the second pulse information output by the convolution layer and recalculates the feature weights before outputting the third pulse information to the fully connected layer.

[0049] S4: The fully connected layer performs weighted integration on the third pulse information and transmits it to the output layer. The output layer generates the prediction results of the pulse neural network model about the input external data information.

[0050] It should be noted that spiking neural networks (SNNs) are a type of neural network model that processes information based on the mechanism of biological neurons emitting pulses. Unlike traditional artificial neural networks (ANNs) that use continuous activation values, spiking neural networks simulate the temporal dynamic characteristics of biological neurons and can process more complex time-dependent information. Unlike traditional networks, SNNs use discrete moments to send pulses and encode information through time sequences. This temporal nature gives spiking neural networks an advantage in processing dynamic information.

[0051] The present invention provides a data recognition method based on a pulse neural network model. The pulse neural network model is used to realize recognition processing on data. A convolution layer is used as an intermediate layer in the pulse neural network model to process external data information input to the model, and then the convolution layer is used to extract important features in the data and eliminate redundant information, thereby compressing the data into a more meaningful pulse signal. The method uses the convolution layer to replace the traditional encoding layer, which can significantly improve the efficiency of feature extraction and data representation. The convolution layer can capture local features and discover basic patterns such as edges and textures, making the model more suitable for image processing.

[0052] At the same time, this method also connects an attention module to the convolutional layer of the model. The attention mechanism of the attention module can pay different attention to different features by calculating the importance of the features, so that the model can better understand and use the input data; the convolutional layer is good at extracting local features, but there may be certain limitations in capturing global context information. By connecting the attention module to the convolutional layer, it can capture a broader context relationship by paying attention to different parts of the entire input feature map, which enables the network to understand the structure and information of the data more comprehensively; by connecting the attention module, the model can adaptively select and emphasize the key parts of the input features according to the needs of the current task, thereby enhancing the model's ability to extract features from the data;

[0053] This method combines the pulse neural network model with the attention mechanism, making it easier for the model to capture features in the data and better extract pulse signals, thereby improving the model's efficiency in identifying and processing data while also improving the accuracy of the data features extracted by the model.

[0054] It can be understood that the input layer in the pulse neural network model is responsible for receiving external data information (such as images or sensor signals) and converting it into pulse signals. Each input neuron in the input layer responds to changes in the input and emits pulses at a specific time. The frequency of pulse emission is usually proportional to the intensity of the input. The input layer and the subsequent convolutional layer are connected through synapses to realize the transmission of pulse signals.

[0055] The convolution layer receives pulse signals from the input layer and connects to the next layer through synapses. The connection weights between neurons in the convolution layer's convolution kernels can be adjusted through learning to enhance the response to specific features. The convolution layer is used to extract spatial features from the input signal and use the convolution kernel to perform local receptive field processing on the input pulse signal. The convolution operation of the convolution layer generates new pulse outputs by weighted summing of input pulses and performing them in time.

[0056] The fully connected layer integrates the output of the convolutional layer or other layers to provide a higher-level feature representation. In this layer, each neuron is connected to all neurons in the previous layer, and a new pulse output is generated by receiving pulse signals from different neurons and performing weighted summation, thereby enhancing the model's expressive power and feature combination capabilities.

[0057] The output layer is connected to the fully connected layer, and uses the third pulse information of the fully connected layer's output to make decisions. The neurons in the output layer calculate the firing frequency based on the pulse signal transmitted by the upstream layer to produce the final classification result or regression value. Each output neuron corresponds to a specific category or output value, generating the final decision or prediction of the model.

[0058] It can be understood that the external information received by the input layer includes dynamic visual sensor data obtained by dynamic visual sensors. Dynamic visual sensors can capture changes in scenes with extremely high temporal resolution. Dynamic visual sensors only send events where changes occur in the image, which can significantly reduce data redundancy and computational burden, but will increase the sparsity of dynamic visual sensor data. The data recognition method provided by the present invention improves the ability to extract and process features in the data, and can solve the problem of inability to effectively extract features caused by the sparsity of dynamic visual sensor data, so that the method can solve the problem of missing information in traditional images in high-speed motion scenes through dynamic visual sensors, and can quickly and effectively identify and process the dynamic visual sensor data generated by it.

[0059] Optionally, the pulse neural network model can also process various types of data, including time series data (sensor data, financial data), image data (visual information, event-driven cameras), sound and voice signals (audio signals, biometrics), biological signals (neural signals, electromyographic signals), control signals, text and sequence data, and multimodal data.

[0060] It should be noted that the dynamic visual sensor data recognition process includes two steps, namely data encoding and classification. The data encoding module is used to encode the frame-based event data into pulse form, and the classification module is used to classify the input data. The existing data recognition method usually uses an encoding module of the Poisson distribution encoding method. Poisson distribution is a commonly used encoding method, which is particularly suitable for processing random and sparse signals. However, the encoding method of Poisson distribution may damage the performance of the network. Although the sparsity of Poisson coding helps to reduce the computational complexity, the sparse pulse emission may lead to information loss, especially in high-frequency signals. Important information may be quickly sparse, thereby affecting the accuracy of learning. At the same time, the randomness of Poisson distribution will introduce noise, because the emission of pulses is based on probability, which may cause instability in neurons when processing information, thereby reducing the reliability and response speed of the network. In addition, there may be adaptability problems in encoding through Poisson distribution. For situations where the environment or input features change greatly, the fixed Poisson encoding method may not be able to adapt to the new input mode, limiting the flexibility and adaptability of the network. In this regard, this method uses a combination of input layer and convolution layer to replace the Poisson coding module to complete the data encoding of dynamic visual sensor data, avoiding damage to the performance of the pulse neural network model.

[0061] Optionally, LIF neurons, IF neurons or PLIF neurons are used in the spiking neural network model. IF neurons are simple and easy to use, suitable for basic research; LIF neurons are closer to biological neurons by introducing leakage characteristics, suitable for models that require biological realism; PLIF neurons provide greater flexibility and adaptability through their piecewise linear characteristics, and can simulate complex neuronal behaviors. By selecting a suitable neuron model, it is possible to balance computational complexity and accuracy according to specific data processing tasks and requirements.

[0062] Preferably, the method provides an attention-based spiking neural network model (Event Attention Spiking Neural Networks, EA-SNNs) using LIF neurons. In order to study the effects of different neurons on the spiking neural network model (EA-SNNs) in the method, this paper conducted comparative tests on three different neurons, LIF, IF and PLIF, on the SED dataset. The PLIF neuron is a newest neuron model, which is similar to the LIF neuron model. The difference is that the membrane time constant of the PLIF neuron model is no longer a hyperparameter, but a parameter that can be learned during network training. The final experimental results are as follows. Figure 3 As shown. Figure 3 The experimental results show that the best effect is the LIF neuron, with an accuracy of 96.6% on the SED data set. The highest accuracy of the IF neuron can reach 90%, but the accuracy fluctuates greatly during the test, and the results are not stable. PLIF is more stable than IF neurons, but the highest accuracy of PLIF neurons is only 86.67%, which is relatively low. According to the comparative experimental results, LIF neurons are used in the pulse neural network model in this method.

[0063] Optionally, the attention module uses an SE module, an ECA module, or a CBAM module. The SE module enhances the representation capability of feature channels through a compression-excitation mechanism, and can dynamically adjust the importance of channels to improve the representation capability of the model, effectively improving the accuracy of the model; the ECA module is designed to improve the computational efficiency of the channel attention mechanism. The ECA module does not require additional parameters, avoids large fully connected layers, and uses one-dimensional convolution to calculate the relationship between channels. The calculation is simple and fast, and it is suitable for occasions that require fast reasoning; the CBAM module combines channel attention and spatial attention, and improves the feature expression capability through a two-stage attention mechanism, taking into account the characteristics of channels and space, and can more comprehensively adjust the expression of feature maps, which helps to improve model performance. The present invention can meet the use requirements for different data by adjusting the use of different attention modules, thereby improving the applicability of the method.

[0064] Preferably, the SE module is used in the present method. In the EA-SNNs in the present method, the effects of three different attention mechanisms (ECA, CBAM and SE) on the N-MNIST dataset are first verified. The experimental results are as follows: Figure 4 As shown. It can be observed that adding the SE module network to the EA-SNNs model has the highest accuracy, with a maximum accuracy of 99.67%. The highest accuracy achieved using the ECA module and CBAM module is only 99.57% and 99.36%.

[0065] This paper compares the accuracy of the EA-SNNs model using the SE module and other advanced SNNs models or learning algorithms on the N-MNIST dataset. The final experimental results are shown in Table 1-1, which shows in detail the accuracy of different methods at different time steps T. It can be observed that on the N-MNIST dataset, the accuracy of most algorithms is about 99.5%. Compared with other algorithms, the EA-SNNs network model has the highest accuracy, reaching a recognition accuracy of 99.67%. In addition, EA-SNNs uses fewer steps than most algorithms, indicating that EA-SNNs has higher temporal accuracy. The accuracy of the studies Fang et al. (2021) and Xu et al. (2023) is similar to that of EA-SNNs, but the performance is slightly behind. This is because EA-SNNs integrates an attention module, which enhances the feature extraction ability of the network, thereby improving its performance.

[0066]

[0067] Table 1-1 Comparison of EA-SNNs and other algorithms on the N-MNIST dataset

[0068] Furthermore, this embodiment also evaluates the performance of three attention modules in EA-SNNs on the DVS Gesture 128 dataset. The experimental results are shown in Figure 5 The results are somewhat different from those on other datasets. Figure 5Among them, the best performance is still EA-SNNs using SE module. However, the CBAM module and SE module have almost the same performance, both of which have a maximum accuracy of 95.14%, but the SE module uses fewer training times to achieve the highest accuracy. The performance gap between the ECA module and the two modules is not large, and it also has an accuracy of 94.44%. The EA-SNNs method is also compared with other methods on the DVS Gesture128 dataset, and the results are shown in Table 1-2. At present, the most advanced DVS Gesture128 recognition algorithm is proposed by Yu et al. (2022) with an accuracy of 98.96%. The accuracy of other algorithms ranges from 95% to 98%. The accuracy of EA-SNNs on the DVS Gesture 128 dataset is 95.14%, which is still slightly lower than the most advanced algorithms, but comparable to the average level of other algorithms. In addition, EA-SNNs achieves this accuracy with a minimum time step of 16, which is the lowest among all algorithms.

[0069]

[0070]

[0071] Table 1-2 Comparison of EA-SNNs and other algorithms on the DVS Gesture 128 dataset

[0072] This example also evaluates the performance of the three attention modules in EA-SNNs on the CIFAR10-DVS dataset. The experimental results are shown in Figure 6 As shown. On the CIFAR10-DVS dataset, the SE module still performs best, with a maximum accuracy of 74.5%. The performance of the ECA module and the CBAM module is quite different from that of the SE module, with the maximum accuracy of the two modules being only 71.7% and 66.8%. This is similar to the results on other datasets. The results of EA-SNNs using the SE module on the CIFAR10-DVS dataset are compared with other methods in Tables 1-3. On the CIFAR10-DVS dataset, the EA-SNNs model shows strong performance, surpassing most algorithms or network models, with an accuracy of 74.45%. This result is only 0.35% lower than the method reported in Wu et al. (2021). PLIF neurons have better learning characteristics than LIF neurons, which is also the reason for this difference. This shows that when both use the LIF neuron model, EA-SNNs outperform other network architectures or algorithms, which proves the important potential of the EA-SNN model.

[0073]

[0074]

[0075] Table 1-3 Comparison of EA-SNN and other algorithms on the CIFAR10-DVS dataset A data recognition method based on a pulse neural network model is provided in the first embodiment of the present invention. By combining the pulse neural network model with the attention mechanism, the model can more easily capture the features in the data and better extract the pulse signal, thereby improving the model's recognition and processing efficiency of the data, and significantly improving the accuracy of the data features extracted by the model.

[0076] See also Figure 2 The second embodiment of the present invention further provides a training method for a pulse neural network model, which is applied to a data recognition method based on a pulse data network model in the first embodiment. The training method includes:

[0077] K1: Provide a pulse data network model in a data recognition method based on a neural network model in the first embodiment;

[0078] K2: Select time series data or event-driven data for training and input the training data into the spiking neural network model;

[0079] K3: The training data is encoded as a pulse signal in the spiking neural network model and then propagated layer by layer to obtain the output pulse of the spiking neural network model;

[0080] K4: Define the loss function and set the target pulse. The loss value is calculated by the difference between the output pulse and the target pulse.

[0081] K5: Calculate the gradient of the spiking neural network model according to the loss function, and adjust the parameters of the spiking neural network model according to the gradient;

[0082] K6: Provide the pulse neural network model with adjusted parameters to output new output pulses, update the loss function and calculate the gradient, adjust the parameters of the pulse neural network model according to the gradient, and iterate and optimize the pulse neural network model.

[0083] Understandably, since the output of the spiking neurons in the spiking neural network model is a discrete pulse signal, the traditional back-propagation algorithm cannot be directly applied to the training of the spiking neural network model. However, effective training can be achieved through pulse coding, a specific forward propagation mechanism, and suitable learning rules. Moreover, the practicality and performance of the spiking neural network model can be improved through training strategies suitable for specific tasks. For this purpose, the gradient substitution method is used in this method to train the spiking neural network.

[0084] Specifically, the loss function in the training process of the spiking neural network model is calculated using the mean square error function. During the training process, a loss function is usually required to evaluate the difference between the network output and the target output. Since the pulse signal in the model is discrete, the mean square error function can be used to convert the pulse signal into frequency or pulse firing rate, thereby calculating its mean square error and optimizing the training process of the model.

[0085] Optionally, the loss function in the training process of the spiking neural network model can also select the cross entropy loss function. The cross entropy measures the difference between two probability distributions. In machine learning, we usually have a true probability distribution and a probability distribution predicted by the model. The derivative calculation of the cross entropy loss function is relatively simple, which is very friendly to the implementation of optimization algorithms such as gradient descent. At the same time, the cross entropy loss function is well combined with many optimization algorithms (such as SGD, Adam), which can effectively guide the parameter update of the model.

[0086] Furthermore, in this method, it is first assumed that the time step and the number of classification categories of the network are T and N respectively. The output of the network O = [o t,i ] is a T×N dimensional tensor. For a given label l, it is hoped that the neurons representing class l will fire pulses earlier than neurons of other classes in the output layer of the network. Neurons representing other classes will not fire pulses or will fire pulses after neurons representing class l fire pulses. Suppose the target output of the network is defined as P = [p t,i ], when i = l, p t,i =1, otherwise, p t,i = 0. The network loss function is calculated by the mean square error function, which can be expressed by the following formula:

[0087]

[0088] Specifically, the sigmoid function is used to replace the step function in the training process of the pulse neural network model. In the pulse neural network model, the step function is usually used to generate pulses, and the output of the step function is used to calculate the loss function during the training process. However, the step function is prone to gradient vanishing or inability to perform effective gradient updates during training. Therefore, using a continuous and differentiable sigmoid function as a substitute for the step function can provide continuous output for the pulse neural network model, making the model easier to train and improving the convergence speed.

[0089] Furthermore, during network training, the membrane voltage of the neurons in the i-th layer before firing a pulse is defined as The membrane voltage after the firing pulse is defined as The threshold voltage is defined as The resting voltage is defined as The injected current for each neuron is Indicates that, and at the i-th layer of the network, The actual pulse output matrix of the network is indicates that, and when When , it means that the i-th neuron of the network emits a pulse. When , no pulse is emitted. According to the chain rule of reverse gradient substitution:

[0090]

[0091] Among them, according to the first-order linear differential equation of the neuron, we have:

[0092]

[0093] Among them, θ(x) is an alternative function to the step function Θ(x), and the sigmoid function is used in this article. Finally, according to the chain derivation rule, the weight update formula of the network is:

[0094]

[0095] Since the pulse signal in the pulse neural network model is discrete, the traditional gradient-based learning method (such as back propagation) is not directly applicable. This method approximates the gradient in the pulse neural network model through the gradient substitution method, converts the spike emission of the pulse signal into a continuous value, and uses a smooth activation function (such as the sigmoid function) to be able to approximately calculate the gradient, thereby training the pulse neural network model to achieve higher performance and application effects.

[0096] To solve the above technical problems, please refer to Figure 7 The third embodiment of the present invention further provides a computer device 10, which includes a memory, a processor and a computer program stored in the memory. The processor executes the computer program to implement the steps of the above-mentioned data recognition method based on the pulse neural network model, which has the same beneficial effects as the above-mentioned data recognition method based on the pulse neural network model, and will not be repeated here.

[0097] To solve the above technical problems, please refer to Figure 8 The fourth embodiment of the present invention further provides a computer-readable storage medium 20, on which a computer program is stored. When the computer program is executed by the processor, the steps of the data recognition method based on the pulse neural network model are implemented, and the steps have the same beneficial effects as the data recognition method based on the pulse neural network model, which will not be elaborated here.

[0098] To solve the above technical problems, please refer to Fig. 9 The fifth embodiment of the present invention further provides a computer program product 30, which includes a computer program / instruction. When the computer program / instruction is executed by a processor, the steps of the data recognition method based on the pulse neural network model are implemented, and the method has the same beneficial effects as the data recognition method based on the pulse neural network model, which will not be described in detail here.

[0099] It is understandable that, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU), the above functions defined in the method of the present application are executed. It should be noted that the computer-readable medium described in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. Computer-readable storage media, for example, include but are not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, an apparatus or a device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, an apparatus or a device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0100] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0101] In the embodiments provided by the present invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0102] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. Those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0103] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the above-mentioned processes does not mean the necessary order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0104] The flow chart and block diagram in the accompanying drawings of the present invention illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which is determined based on the functions involved. It should be particularly noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs a specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0105] The above is a detailed introduction to a data recognition method based on a pulse neural network model, a computer device, a readable storage medium and a program product disclosed in an embodiment of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea: At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention. Any modifications, equivalent substitutions and improvements made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A data recognition method based on a pulse neural network model, characterized in that: The following steps are involved: Provide a spiking neural network model including an input layer, a convolutional layer, a fully connected layer, and an output layer connected in sequence; Inputting external data information into the pulse neural network model, the pulse neural network model receives the external data information through an input layer, the input layer converts the external data information into first pulse information and outputs it to a convolution layer, the convolution layer extracts features from the first pulse information and outputs second pulse information according to a preset weight; The convolutional layer is also connected to an attention module, which is also connected to the fully connected layer. The attention module receives the second pulse information output by the convolutional layer and recalculates the weight of the feature, and then outputs the third pulse information to the fully connected layer. The fully connected layer performs weighted integration on the third pulse information and transmits it to the output layer, and the output layer generates the prediction results of the pulse neural network model about the input external data information.

2. The data recognition method based on the pulse neural network model according to claim 1 is characterized in that: The external information received by the input layer includes dynamic visual sensing data obtained through a dynamic visual sensor.

3. The data recognition method based on the pulse neural network model according to claim 1 is characterized in that: The spiking neural network model is trained using the gradient substitution method.

4. The data recognition method based on the pulse neural network model according to claim 1 is characterized in that: The loss function during the training process of the spiking neural network model is calculated using the mean square error function.

5. The data recognition method based on the pulse neural network model according to claim 3 is characterized in that: The sigmoid function is used instead of the step function during the training of the spiking neural network model.

6. The data recognition method based on the pulse neural network model according to claim 1, characterized in that: The attention module uses the SE module, ECA module or CBAM module.

7. The data recognition method based on the pulse neural network model according to claim 1 is characterized in that: LIF neurons, IF neurons or PLIF neurons are used in the spiking neural network model.

8. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of a data recognition method based on a pulse neural network model as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a data recognition method based on a pulse neural network model as described in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that: When the computer program / instructions are executed by a processor, the steps of a data recognition method based on a pulse neural network model as described in any one of claims 1 to 7 are implemented.

Images