Sparse data efficient processing method based on novel pulse neural network

Abstract

The invention discloses a sparse data efficient processing method based on a novel spiking neural network. The sparse data efficient processing method comprises the following steps: step 1, designing and realizing a novel spiking neuron architecture based on bilinear integration; 2, constructing a novel spiking neural network based on the spiking neuron model; 3, training set and verification set data are prepared; 4, preprocessing the original data; 5, converting the data into a sparse discrete pulse form; step 6, executing forward propagation, calculating a loss function, updating network parameters and the like for network training; 7, calculating the accuracy of the verification set after each round of training, and storing the optimal model; 8, according to the loss function value and the verification set accuracy, the training stop opportunity is judged; and step 9, deploying an optimal model for reasoning.

Description

Technical Field

[0001] The present invention relates to a sparse data processing method, in particular to an efficient sparse data processing method based on a novel pulse neural network. Background Art

[0002] With the rapid development of the internet and big data technologies, sparse data processing has become a key research area in artificial intelligence. In applications such as recommendation systems, natural language processing, and the recently emerging neuromorphic data processing, data often exhibits high dimensionality and low density. This sparsity leads to problems such as the curse of dimensionality, increased computing resource requirements, and reduced model generalization, posing challenges to traditional machine learning methods.

[0003] Existing approaches to processing sparse data include: feature embedding-based methods, such as word2vec; matrix decomposition-based methods, such as factorization machines; and regularization methods, such as L1 regularization. While some of these methods are based on traditional machine learning models, in recent years, an increasing number of studies have begun to adopt or combine neural network models, leveraging their automatic feature extraction capabilities, powerful expressiveness, and good scalability to address sparse data processing. However, in practical applications, traditional artificial neural networks still suffer from high inference energy consumption, limiting their application in energy-constrained scenarios such as edge computing.

[0004] Compared to artificial neural networks, biological neural networks offer advantages such as low energy consumption, fast response, and high stability. In recent years, biologically inspired spiking neural networks have become a hot topic of research due to their event-driven computing model and low energy consumption and low-latency inference. Due to their event-based computing nature, spiking neurons only respond when they receive a spike signal. This allows them to process sparse data while keeping most neurons in a resting state, significantly reducing energy consumption. This lower computational burden enables spiking neural networks to process sparse data more efficiently.

[0005] However, sparse data in real-world scenarios often contain some complex high-order features. For example, the Hadamard product of two features may form an important combination feature, and general neural networks may not be able to learn this well. With the deepening of research on biological neurons, researchers have found that biological neurons use a nonlinear input integration mechanism, which can enhance the nonlinear computing ability of neurons. For example, a single biological neuron can complete XOR calculations. This gives it stronger computing power within a limited network scale and can complete computing tasks more efficiently. Reference: GIDON A, ZOLNIK TA, FIDZINSKI P, et al. Dendritic action potentials and computation in human layer2 / 3 cortical neurons[J]. Science, 2020, 367(6473): 83-87.

[0006] Among the many characteristics of biological neuron integration mechanisms, bilinear integration characteristics have relatively high potential application value in sparse data processing scenarios. Studies have found that the membrane potential of biological neurons does not adopt the linear integration mode of general artificial neurons, but exhibits bilinear integration characteristics. This allows neurons to directly model second-order interaction features, significantly improving the computational ability of neurons for complex nonlinear features of sparse data, thereby improving the overall performance of the network. In addition, the spatiotemporal distribution of spike data usually has sparse characteristics. Even if the original data is not sparse, it will exhibit sparse characteristics after being converted into discrete spike sequences through spike coding. Therefore, enhancing the sparse data computing ability of spike neural networks is not only applicable to naturally sparse data, but can also effectively improve the processing ability of non-sparse data. References: LI S, LIU N, ZHANG X, et al. Bilinearity in spatiotemporal integration of synaptic inputs [J]. PLOS Computational Biology, 2014, 10(12).

[0007] Although the bilinear integration mode enhances the sparse data processing capability and nonlinear computing capability of the spiking neural network, its computational complexity is much higher than that of linear integration. The computational cost of a single n-input neuron is Rise to Therefore, it is rarely used in neural networks. When introducing the bilinear integration mechanism into spiking neural networks, it is also necessary to consider the computational complexity of the balanced network. Summary of the Invention

[0008] Purpose of the invention: The problem to be solved by the present invention is to design a new type of pulse neural network to efficiently process sparse data in view of the shortcomings of existing neural network models in processing sparse data.

[0009] In order to solve the above technical problems, the present invention discloses a novel design and implementation of a bilinear integrated spiking neural network, as well as an efficient sparse data processing method based on the spiking neural network, comprising the following steps:

[0010] Step 1: Based on the bilinear integration mechanism of biological neuron membrane potential and combined with the low-rank matrix decomposition technology of the factorization machine, we design and implement a bilinear integrated spiking neuron model that can efficiently capture second-order interaction characteristics and has stronger nonlinear computing capabilities;

[0011] Step 2: Based on the neuron model, a fully connected, convolutional, and other network topologies are implemented, and a bilinear integrated spiking neural network with a corresponding structure is constructed according to the specific task.

[0012] Step 3: Prepare the data set, including the training data set D train and validation dataset D val ;

[0013] Step 4: Preprocess the raw data, including normalization, data augmentation, and other operations. The specific operations depend on the type and distribution of the dataset.

[0014] Step 5: Convert the neuromorphic data or normal data into pulse form as input to the spiking neural network. The specific conversion method depends on the data type.

[0015] Step 6: Input the training samples into the spiking neural network model in batches, perform forward propagation, calculate and record the loss function value, and update the network parameters through the proxy gradient descent method;

[0016] Step 7: After each round of training, use the obtained model to calculate its accuracy on the validation set. If it is better than the historical highest accuracy, save the current model as the optimal model.

[0017] Step 8: Observe the changes in the loss function value and the validation set accuracy during training. Stop training when the loss stops decreasing and the validation set accuracy stops increasing, or when the maximum number of training rounds is reached.

[0018] Step 9: Deploy the model for inference tasks based on the saved best-performing model.

[0019] Step 1 of the present invention comprises:

[0020] Step 1-1: Implement the basic architecture of a general spiking neuron. Based on the LIF neuron model, which is the most commonly used spiking neural network, a spiking neuron that receives input in the form of discrete pulses can be described as follows:

[0021] u t =α(u t-1 -b)(1-o t-1 )+i t +b,

[0022] o t =σ(u t -u thr ),

[0023] Among them, t∈[0, T-1] is the time step, u t and u t-1 are the membrane potentials at time steps t and t-1, respectively, t and o t-1 are the outputs of time steps t and t-1 respectively, the constant α represents the attenuation coefficient, i t is the input integrated current at time step t, the bias b represents the resting membrane potential of the neuron, σ(·) is the unit step function, and u thr is the threshold;

[0024] Step 1-2, analyze the bilinear integration mechanism of biological neurons, and its integrated current (i bio ) t It can be expressed as follows:

[0025]

[0026] Among them, x t ∈{0, 1} n is the pulse input of time step t, w is the weight, is the shunting coefficient matrix, the diagonal elements of K are all 0, and the integrated current calculation cost of this neuron in a single time step is

[0027] Steps 1-3, based on the low-rank matrix decomposition technology of factorization machine (reference: RENDLE S. Factorization Machines [C] / / IEEE International Conference on Data Mining. (ICDM). 2010: 995-1000.) reconstruct the second-order interaction terms in the biological neuron current. When the matrix K is positive, there is a Cholesky decomposition in And k≥Rank(K). For the case where K is a sparse matrix, k<<n is usually true. If the matrix K is not positive definite, the above decomposition is an approximate representation. The second-order interaction term of the current in steps 1-2 can be reconstructed as:

[0028]

[0029] in, is a series of independent vectors;

[0030] Steps 1-4, remove the self-interaction terms. To satisfy the constraint that all diagonal elements of matrix K are 0, it is necessary to remove the terms of the product of all elements and themselves. Therefore, the approximate process of the second-order interaction terms of current can be modified as follows:

[0031]

[0032] in, and Respectively and In the present invention, the square of a vector and a matrix both represent the Hadamard product with itself;

[0033] In steps 1-5, an adjustable characteristic ratio parameter λ is added to dynamically adjust the ratio of the linear term and the second-order interaction term. The input integrated current at time step t can be expressed as:

[0034]

[0035] Where λ is a learnable parameter. The adjustable feature ratio can balance the contribution of the two features and the learning rate of the two feature weight parameters. The cost of the above input integration current is calculated as

[0036] Step 2 of the present invention comprises:

[0037] Step 2-1, embed the bilinear integrated pulse neuron model into the fully connected layer of the neural network, for the input vector x of t time step t ∈{0, 1} n , membrane potential The response is as follows:

[0038]

[0039] Among them, W, are the linear term weight matrix and the second-order interaction term weight matrix, respectively. To achieve learnable ratio parameters, each neuron in the fully connected layer is set with an independent feature ratio parameter, which can flexibly adjust the contribution ratio of the two features;

[0040] Step 2-2, embed the bilinear integrated pulse neuron model into the convolutional layer of the neural network, for the tensor input X of t time steps t ∈{0, 1} c×h×w , where c is the number of input channels and the neuron membrane potential is Where c′ is the number of output channels, the response of the membrane potential is as follows:

[0041]

[0042] Among them, W, are the linear term weight matrix and the second-order interaction term weight matrix, respectively. To learn the scale parameter, neurons in the same channel share a scale parameter;

[0043] Step 2-3, use the fully connected layer and convolutional layer to build a spiking neural network. The specific structure of the network depends on the specific task requirements. For the output of the last layer of the network The final output is calculated as follows:

[0044]

[0045] Where T represents the time window of the pulse neural network, that is, the length of the discrete pulse code, Each dimension of takes a value between [0, 1] and can be regarded as a probability estimate for each category.

[0046] In step 3 of the present invention, the specific division method of the data set depends on the data set itself. If the data set itself provides a training set and a validation set, the original training set is used as D train , the original validation set is used as D val ; If the dataset itself only provides a training set and no validation set, the original training set is randomly divided into training set D in a certain proportion train and validation set D val .

[0047] In step 4 of the present invention, different preprocessing schemes are used for different types of data sets: for neuromorphic data, no preprocessing is used; for image data, random cropping, random horizontal flipping, tensor transformation, and normalization operations are performed on the training set in sequence, and only tensor transformation and normalization operations are performed on the validation set.

[0048] In step 5 of the present invention, the raw data needs to be converted into a form that can be directly used by the spiking neural network. For computing tasks on general computing hardware (such as CPU and GPU), the spiking neural network needs to use pulse data in the form of a frame. Therefore, event-based neuromorphic data and ordinary real number data need to be converted into a pulse form:

[0049] Event-based neuromorphic data usually contains information such as timestamps, polarity (increase or decrease in brightness), pixel coordinates, etc. The SpikingJelly framework (reference: FANG W, CHEN Y, DING J, et al. SpikingJelly: an open-source machine learning infrastructure platform for spike-based intelligence [J]. CoRR, 2023, abs / 2310.16620.) simulates the accumulation process of spike events in time and space, and the data can be converted into (X o , X1,…,X T-1 )∈{0,1} T×c×h×w The form of, where T is the time window, the number of channels c = 2 corresponds to two polarities, h and w represent the height and width of the image;

[0050] For real data, the image tensor is usually In the form of pulse coding, it is necessary to convert it into (X0, X1, ..., X T-1 )∈{0,1} T×c×h×w The form of , the conversion method is Poisson coding, and the random variable generation method is as follows:

[0051] Pr(X t,i,j,k =1)=X i,j，k ,

[0052] where X i，j，k ∈[0, 1] corresponds to the expected pulse emission frequency. Although the real original data may not be sparse data, it will become a sparse pulse after pulse coding conversion.

[0053] In step 6 of the present invention, the mean square error is defined as the model loss function, and D train The data is input into the pulse neural network for forward propagation, and the loss function value is calculated and recorded according to the output and sample label. Then, the proxy gradient of the loss function for each parameter of the network is calculated, and the Adam optimization algorithm is used for back propagation to update the network parameters.

[0054] In step 7 of the present invention, after each round of training is completed, the model obtained by this round of learning is used to calculate its D val If the accuracy is higher than the historical highest accuracy, it is used as the new highest accuracy and the current model is saved as the optimal model.

[0055] In step 8 of the present invention, based on the recorded loss function value and verification set accuracy data, if the loss function value does not show a significant decrease for a certain number of consecutive rounds and the verification set accuracy data no longer increases, or when the set maximum number of training rounds is reached, the training is stopped.

[0056] In step 9 of the present invention, the saved optimal model is used as the final model, deployed and the reasoning task is performed.

[0057] Beneficial effects:

[0058] 1. This paper designs a bilinear integrated spiking neuron model with enhanced second-order interactive feature learning and nonlinear computational capabilities. This model improves the neuron's ability to process sparse data while maintaining linear computational complexity. By introducing an adjustable feature scaling parameter, the model can adapt to different task requirements, improve generalization across diverse data, and enhance its robustness in learning.

[0059] 2. The neuron model described in this paper can be easily integrated into various network topologies to construct a variety of novel spiking neural networks, enabling flexible processing of different data types. Even non-sparse data can be converted into sparse spike forms through spike coding, further expanding the application scope of the method and meeting the needs of more practical scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, and the above and other advantages of the present invention will become more apparent.

[0061] Figure 1 It is a schematic diagram of the process of the present invention.

[0062] Figure 2 Schematic diagram comparing the application of the neuron model proposed in this invention and the traditional factorization machine.

[0063] Figure 3 This is a schematic diagram of parameter settings during the experiment of the method proposed in the present invention.

[0064] Figure 4 This is a schematic diagram of the test results of the example verification of the method proposed in the present invention on various data sets and models. DETAILED DESCRIPTION

[0065] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0066] like Figure 1 The following is an efficient sparse data processing method based on a novel spiking neural network, which includes nine steps:

[0067] Step 1 includes:

[0068] Step 1-1, implement the basic architecture of a general spiking neuron. The LIF model is the most mainstream neuron model in spiking neural networks. Based on this, implement the following basic framework:

[0069] u t = α(u t-1 - b)(1 - o t-1 ) + i t + b,

[0070] o t = σ(u t - u thr ),

[0071] where t ∈ [0, T - 1] is the time step, u t and u t-1 are the membrane potentials at time steps t and t - 1 respectively, o t and o t-1 are the outputs at time steps t and t - 1 respectively, the constant α represents the decay coefficient, i t is the input integration current at time step t, the bias b represents the resting membrane potential of the neuron, σ(·) is the unit step function, and u thr is the threshold;

[0072] Step 1-^{}2, formally represent the bilinear integration mechanism of biological neurons. Its integration current (i bio ) t for the input spikes at time step t can be expressed as follows:

[0073]

[0074] where x t ∈ {0, 1} n is the spike input at time step t, w is the weight, is the shunting coefficient matrix, and the diagonal elements of K are all 0 to ensure the elimination of the self-interaction terms of the variables;

[0075] Step 1-^{}3, reconstruct the second-order interaction term in the current described in Step^{}1-^{}2 of biological neurons based on the low-rank matrix factorization technology of factorization machines. Use to approximate the matrix K, where and k ≥ Rank(K). In spiking neural networks, K is usually a sparse matrix, so k << n. The above process can be expressed in vector form as:

[0076]

[0077] where, are a series of independent vectors;

[0078] Steps 1-4 modify the matrix approximation method to satisfy the constraint that all diagonal elements of matrix K are 0, thereby eliminating the self-interaction term. The approximation process for the second-order interaction term of the current can be modified as follows:

[0079]

[0080] in, and Respectively and In the present invention, the square of a vector and a matrix both represent the Hadamard product with itself;

[0081] Steps 1-5 add the function of dynamically adjusting the proportion of the linear term and the second-order interaction term by introducing a learnable feature proportion parameter. The input integrated current of the modified model at time step t can be expressed as:

[0082]

[0083] Where λ is a learnable feature scale parameter.

[0084] Step 2 includes:

[0085] Step 2-1, embed the bilinear integrated pulse neuron model into the fully connected layer of the neural network, for the input vector x of t time step t ∈{0, 1} n , membrane potential The response is as follows:

[0086]

[0087] Among them, W, are the linear term weight matrix and the second-order interaction term weight matrix, respectively. To learn the scale parameter, each neuron in the fully connected layer is set with an independent feature scale parameter;

[0088] Step 2-2, embed the bilinear integrated pulse neuron model into the convolutional layer of the neural network, for the tensor input X of t time steps t ∈{0, 1} c×h×w , where c is the number of input channels and the neuron membrane potential is Where c′ is the number of output channels, the response of the membrane potential is as follows:

[0089]

[0090] Among them, W, are the linear term weight matrix and the second-order interaction term weight matrix, respectively. To learn the scale parameter, neurons in the same channel share a scale parameter;

[0091] Step 2-3, use the fully connected layer and convolution layer combined with the pooling layer, dropout layer and other structures to build a pulse neural network. The specific structure of the network depends on the specific task requirements. For the output of the last layer of the network The final output is calculated as follows:

[0092]

[0093] Where T represents the time window of the pulse neural network, that is, the length of the discrete pulse code, Each dimension of takes a value between [0, 1] as the probability estimate for each category.

[0094] In step 3, the dataset is divided according to its specific situation. If the dataset itself provides a training set and a validation set, the original training set is used as D train , the original validation set is used as D val ; If the dataset itself only provides a training set and no validation set, the original training set is randomly divided into training set D in a certain proportion train and validation set D val .

[0095] In step 4, different types of data are preprocessed using corresponding schemes: for neuromorphic data, no preprocessing is used; for image data, random cropping, random horizontal flipping, tensor transformation, and normalization operations are performed on the training set in sequence, and only tensor transformation and normalization operations are performed on the validation set.

[0096] In step 5, the raw data is converted into a form that can be directly used by the spiking neural network. The spiking neural network running on general computing hardware (such as CPU and GPU) requires the use of spike data in a frame format. Therefore, the event-based neuromorphic data and ordinary real number data need to be converted into the following form:

[0097] Event-based neuromorphic data usually contains information such as timestamp, polarity (indicating brightness increase or decrease), pixel coordinates, etc. The SpikingJelly framework simulates the accumulation process of spike events in time and space, and converts the data into (X0, X1, ..., X T-1 )∈{0,1} T×c×h×w The form of, where T is the time window, the number of channels c = 2 corresponds to two polarities, h and w represent the height and width of the image;

[0098] Real image tensor data is usually In the form of pulse coding, it is necessary to convert it into (X0, X1, ..., X T-1)∈{0,1} T×c×h×w The form of , the conversion method is Poisson coding, the process is as follows:

[0099] Pr(X t,i，j,k =1)=X i,j，k ,

[0100] where X i，j，k ∈[0, 1] corresponds to the expected pulse emission frequency. Although the real original data may not be sparse data, it will become a sparse pulse after pulse coding conversion.

[0101] In step 6, define the mean square error as the model loss function and divide D train The data is input into the pulse neural network for forward propagation, and the loss function value is calculated and recorded according to the output and sample label. Then, the proxy gradient of the loss function for each parameter of the network is calculated, and the Adam optimization algorithm is used for back propagation to update the network parameters.

[0102] In step 7, after each round of training is completed, the model learned in this round is used to calculate its val If the accuracy is higher than the historical highest accuracy, it is used as the new highest accuracy and the current model is saved as the optimal model.

[0103] In step 8, based on the recorded loss function value and validation set accuracy data, if the loss function value does not show a significant decrease for a certain number of consecutive rounds and the validation set accuracy data no longer increases, or when the set maximum number of training rounds is reached, training is stopped.

[0104] In step 9, the saved optimal model is used as the final model, deployed, and used for inference tasks.

[0105] Example

[0106] To verify the effectiveness of this invention, we conducted practical experiments on a variety of datasets and basic models, including the neuromorphic dataset N-MNIST, as well as the general image datasets MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100. The basic model uses STBP (reference: WU Y, DENG L, LI G, et al. Spatio-temporal backpropagation for training highperformance spiking neural networks [J]. Frontiers in Neuroscience, 2018, 12: 331.), PLIF (reference: FANG W, YU Z, CHEN Y, et al. Incorporating learnable membrane time constant to enhance learning of spiking neural networks [C] / / IEEE / CVF International Conference on Computer Vision. (ICCV). 2021: 2641-2651.) and MPBN (reference: GUO Y, ZHANG Y, CHEN Y, et al. Membrane potential batch normalization for spiking neural networks [C] / / IEEE / CVF International Conference on Computer Vision. (ICCV). 2023: 19363-19373.). This example takes the CIFAR-10 dataset and the PLIF model as an example and performs the test according to the following steps:

[0107] 1. Based on the bilinear integration mechanism of biological neurons and the low-rank matrix decomposition technology of factorization machine, an efficient bilinear integrated spiking neuron model is implemented, where the parameter k = 1 and the initial value of λ is 0.1.

[0108] 2. The spiking neuron model is embedded into the network topology of full connection and convolution to obtain the corresponding full connection layer and convolution layer implementation, and a deep spiking convolutional neural network is constructed based on this. The specific structure of the network based on PLIF is shown in Figure 3 Settings in.

[0109] 3. Since the CIFAR-10 dataset only has a training set and a test set, but no validation set, the original training set is randomly split into training datasets D trainand validation dataset D val , where D val It accounts for 15% of the original training set, and the original test set is used as D test Used in reasoning tasks.

[0110] 4. For training data D train Perform preprocessing operations such as random cropping, random horizontal flipping, tensor transformation, and normalization. val With D test Use tensor transformation and normalization operations.

[0111] 5. Through Poisson coding, the data is transformed from Transformed into (X0, X1, ..., X7) ∈ {0, 1} 8×3×32×32 Sparse pulses of the form.

[0112] 6. Input the training samples into the spiking neural network model in batches, with each batch containing 16 samples. Perform forward propagation, calculate and record the loss function value, and update the network parameters using the proxy gradient descent method. The proxy gradient function selects the derivative of the arctan function.

[0113] 7. After each round of training, use the obtained model to calculate its accuracy on the validation set. If it is better than the historical highest accuracy, save the current model as the optimal model.

[0114] 8. Observe the changes in the loss function value and the validation set accuracy during training. Stop training when the loss stops decreasing and the validation set accuracy stops increasing, or when 100 rounds are reached.

[0115] 9. Deploy the best performing model in D test Test on it, calculate and record its accuracy index.

[0116] Our method has been tested on the CIFAR-10 test dataset, improving classification accuracy from 89.95% to 91.08%. The experimental process for other datasets and models is similar to the above process. The experimental results show that our method has a certain improvement over existing methods, demonstrating the effectiveness of the proposed method for efficient sparse data processing based on a novel spiking neural network.

[0117] like Figure 2As shown, the comparison between the mainstream application mode of factor decomposition machine and the present invention is demonstrated. In the prior art, factor decomposition machine and deep neural network mostly adopt a parallel fusion mode, and this architecture has significant limitations: the factor decomposition machine only participates in joint training as an independent module, and its core advantage of capturing second-order interaction features fails to deeply penetrate into the internal structure of the neural network, resulting in its limited application scenarios. The present invention applies the low-rank matrix decomposition technology of the factor grading machine to the neuron level. This improvement can be applied to different network topologies, enabling it to process different types of data. For example, it is usually difficult for a factor decomposition machine to directly learn features in image data, but neurons based on the factor decomposition machine can be easily integrated into a convolutional neural network.

[0118] like Figure 3 Figure 2 shows the experimental network parameter settings. The experiment used the Adam optimizer to achieve faster convergence. Experimental data showed that the network reached convergence within 100 epochs, so we set the maximum number of training epochs to 100. Since the STBP-based model is suitable for processing simple data, it has fewer network layers and a simpler structure. The PLIF and MPBN-based methods are suitable for processing complex data, so they have more network layers and a more complex structure.

[0119] like Figure 4 The test results show that the bilinear integration mode proposed in this invention can effectively improve the processing capability of the spiking neural network for sparse spikes.

[0120] The present invention provides an efficient method for processing sparse data based on a spiking neural network. There are numerous methods and approaches for implementing this technical solution. The above is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art may make various improvements and modifications without departing from the principles of the present invention, and such improvements and modifications are also within the scope of protection of the present invention. Any components not specified in this embodiment may be implemented using existing technologies.

Claims

1. A method for efficient processing of sparse data based on a novel spiking neural network, characterized in that: The following steps are involved: Step 1: Based on the bilinear integration mechanism of biological neuron membrane potential and combined with the low-rank matrix decomposition technology of the factorization machine, we design and implement a bilinear integrated spiking neuron model that can efficiently capture second-order interaction characteristics and has stronger nonlinear computing capabilities; Step 2: Based on the neuron model, a fully connected, convolutional, and other network topologies are implemented, and a bilinear integrated spiking neural network with a corresponding structure is constructed according to the specific task. Step 3: Prepare the data set, including the training data set D train and validation dataset D val ; Step 4: Preprocess the raw data, including normalization, data augmentation, and other operations. The specific operations depend on the type and distribution of the dataset. Step 5: Convert the neuromorphic data or normal data into pulse form as input to the spiking neural network. The specific conversion method depends on the data type. Step 6: Input the training samples into the spiking neural network model in batches, perform forward propagation, calculate and record the loss function value, and update the network parameters through the proxy gradient descent method; Step 7: After each round of training, use the obtained model to calculate its accuracy on the validation set. If it is better than the historical highest accuracy, save the current model as the optimal model. Step 8: Observe the changes in the loss function value and the validation set accuracy during training. Stop training when the loss stops decreasing and the validation set accuracy stops increasing, or when the maximum number of training rounds is reached. Step 9: Deploy the model for inference tasks based on the saved best-performing model.

2. The method for efficiently processing sparse data based on a novel spiking neural network according to claim 1, characterized in that: Step 1 includes: Step 1-1: Implement the basic architecture of a general spiking neuron. Based on the LIF neuron model, which is the most commonly used spiking neural network, a spiking neuron that receives input in the form of discrete pulses can be described as follows: u t =α(u t-1 -b)(1-o t-1 )+i t +b, on t =σ(u t -in thr ), Among them, t∈[0, T-1] is the time step, u t and u t-1 are the membrane potentials at time steps t and t-1, respectively, t and o t-1 are the outputs of time steps t and t-1 respectively, the constant α represents the attenuation coefficient, i t is the input integrated current at time step t, the bias b represents the resting membrane potential of the neuron, σ(·) is the unit step function, and u thr is the threshold; Step 1-2, analyze the bilinear integration mechanism of biological neurons, and its integrated current (i bio ) t It can be expressed as follows: Among them, x t ∈{0, 1} n is the pulse input of time step t, w is the weight, is the shunting coefficient matrix, the diagonal elements of K are all 0, and the integrated current calculation cost of this neuron in a single time step is Steps 1-3, reconstruct the second-order interaction term in the biological neuron current based on the low-rank matrix factorization technology of the factorization machine. When the matrix K is positive definite, there exists a Cholesky decomposition where and k ≥ Rank(K). For the case where K is a sparse matrix, usually k << n. If the matrix K is not positive definite, the above decomposition is an approximate representation, and the second-order interaction term of the current described in Steps 1-2 can be reconstructed as: in, is a series of independent vectors; Steps 1-4, remove the self-interaction terms. To satisfy the constraint that all diagonal elements of matrix K are 0, it is necessary to remove the terms of the product of all elements and themselves. Therefore, the approximate process of the second-order interaction terms of current can be modified as follows: in, and Respectively and In the present invention, the square of a vector and a matrix represents the Hadamard product with itself; In steps 1-5, an adjustable characteristic ratio parameter λ is added to dynamically adjust the ratio of the linear term and the second-order interaction term. The input integrated current at time step t can be expressed as: Where λ is a learnable parameter. The adjustable feature ratio can balance the contribution of the two features and the learning rate of the two feature weight parameters. The cost of the above input integration current is calculated as 3. The method for efficiently processing sparse data based on a novel spiking neural network according to claim 2, wherein: Step 2 includes: Step 2-1, embed the bilinear integrated pulse neuron model into the fully connected layer of the neural network, for the input vector x of t time step t ∈{0, 1} n , membrane potential The response is as follows: Among them, W, are the linear term weight matrix and the second-order interaction term weight matrix, To achieve learnable ratio parameters, each neuron in the fully connected layer is set with an independent feature ratio parameter, which can flexibly adjust the contribution ratio of the two features; Step 2-2, embed the bilinear integrated pulse neuron model into the convolutional layer of the neural network, for the tensor input X of t time steps t ∈{0, 1} c×h×w , where c is the number of input channels and the neuron membrane potential is Where c′ is the number of output channels, the response of the membrane potential is as follows: Among them, W, are the linear term weight matrix and the second-order interaction term weight matrix, respectively. To learn the scale parameter, neurons in the same channel share a scale parameter; Step 2-3, use the fully connected layer and convolutional layer to build a spiking neural network. The specific structure of the network depends on the specific task requirements. For the output of the last layer of the network The final output is calculated as follows: Where T represents the time window of the pulse neural network, that is, the length of the discrete pulse code, Each dimension of takes a value between [0, 1] and can be regarded as a probability estimate for each category.

4. The method for efficiently processing sparse data based on a novel spiking neural network according to claim 3, wherein: In step 3, the specific division method of the data set depends on the data set itself. If the data set itself provides a training set and a validation set, the original training set is used as D train , the original validation set is used as D val If the dataset only provides a training set and no validation set, the original training set is randomly divided into training set D in a certain proportion. train and validation set D val .

5. The method for efficiently processing sparse data based on a novel spiking neural network according to claim 4, characterized in that: In step 4, different preprocessing schemes are used for different types of datasets: for neuromorphic data, no preprocessing is used; for image data, random cropping, random horizontal flipping, tensor transformation, and normalization operations are performed on the training set in sequence, and only tensor transformation and normalization operations are performed on the validation set.

6. The method for efficiently processing sparse data based on a novel spiking neural network according to claim 5, characterized in that: In step 5, the raw data needs to be converted into a form that can be directly used by the spiking neural network. For computing tasks on general computing hardware (such as CPU and GPU), the spiking neural network requires spike data in the form of frames. Therefore, event-based neuromorphic data and ordinary real number data need to be converted into spike form: Event-based neuromorphic data usually contains information such as timestamp, polarity (indicating brightness increase or decrease), pixel coordinates, etc. The SpikingJelly framework simulates the accumulation process of spike events in time and space, and the data can be converted into (X0, X1, ..., X T-1 )∈{0,1} T×c×h×w The form of, where T is the time window, the number of channels c = 2 corresponds to two polarities, h and w represent the height and width of the image; For real data, the image tensor is usually In the form of pulse coding, it is necessary to convert it into (X0, X1, ..., X T-1 )∈{0,1} T×c×h×w The form of , the conversion method is Poisson coding, and the random variable generation method is as follows: Pr(X t,i，j,k =1)=X i，j，k , where X i，j，k ∈[0, 1] corresponds to the expected pulse emission frequency. Although the real original data may not be sparse data, it will become a sparse pulse after pulse coding conversion.

7. The method for efficiently processing sparse data based on a novel spiking neural network according to claim 6, characterized in that: In step 6, define the mean square error as the model loss function and divide D train The data is input into the pulse neural network for forward propagation, and the loss function value is calculated and recorded according to the output and sample label. Then, the proxy gradient of the loss function for each parameter of the network is calculated, and the Adam optimization algorithm is used for back propagation to update the network parameters.

8. The method for efficiently processing sparse data based on a novel spiking neural network according to claim 7, characterized in that: In step 7, after each round of training is completed, the model learned in this round is used to calculate its val If the accuracy is higher than the historical highest accuracy, it is used as the new highest accuracy and the current model is saved as the optimal model.

9. The method for efficiently processing sparse data based on a novel spiking neural network according to claim 8, characterized in that: In step 8, based on the recorded loss function value and validation set accuracy data, if the loss function value does not show a significant decrease for a certain number of consecutive rounds and the validation set accuracy data no longer increases, or when the set maximum number of training rounds is reached, training is stopped.

10. The method for efficiently processing sparse data based on a novel spiking neural network according to claim 9, characterized in that: In step 9, the saved optimal model is used as the final model, deployed, and used for inference tasks.

Images