A multi-layer pulse hopfield network image classification method based on spectral norm regularization

Abstract

This invention discloses an image classification method based on spectral norm regularization using a multi-layer spiking Hopfield network, comprising the following steps: (1) replacing the neuron units in the multi-layer Hopfield network with spiking neurons to obtain a multi-layer spiking Hopfield network; (2) replacing the neuron state vector in the local gradient calculation formula of the multi-layer spiking Hopfield network with the firing frequency of the spiking neurons; (3) obtaining an image training set, and adding a Jacobian matrix spectral norm regularization term constraint during the training process of the multi-layer spiking Hopfield network; (4) inputting the image to be classified into the trained multi-layer spiking Hopfield network, performing image spiking processing on the first layer of the network, and classifying the image by counting the spiking of neurons in the last layer of the network. Using this invention, the image classification accuracy of the multi-layer spiking Hopfield network can be improved.

Description

Technical Field

[0001] This invention belongs to the field of neural network image classification, and in particular relates to an image classification method based on a multi-layer pulse Hopfield network with spectral norm regularization. Background Technology

[0002] Artificial Neural Networks (ANNs), as the second generation of artificial neural networks, have made groundbreaking progress driven by the development of deep learning. They are widely used in fields such as computer vision, speech recognition, and natural language processing, and perform exceptionally well in tasks such as image classification, object detection, and semantic segmentation. ANNs transmit and process information through continuous real-valued neuron activations, enabling them to efficiently fit complex nonlinear functional relationships. However, ANNs rely on large-scale matrix operations during model training and inference, resulting in computational intensity and high energy consumption. This drawback becomes increasingly apparent when dealing with massive amounts of data and ultra-large-scale network structures. In contrast, Spiking Neural Networks (SNNs), as an emerging third-generation neural network, are closer to the working mechanism of biological nervous systems. Their neurons transmit information along the time axis through discrete pulses, exhibiting event-driven and sparse computation characteristics. This mechanism not only reduces computational redundancy and energy consumption but also allows SNNs to better capture temporal information, demonstrating advantages in modeling dynamic scenes and spatiotemporal features. Furthermore, SNNs outperform ANNs in terms of biological interpretability, providing a new direction for brain-inspired computing research. SNNs have shown broad application prospects in fields such as image classification, speech processing, and neuromorphic computing, and have potential value, especially in low-power hardware and real-time perception tasks.

[0003] Hopfield networks, as a classic energy-based neural network structure, have been widely used in image processing, pattern recognition, and optimization computation due to their excellent associative memory and energy convergence properties. However, traditional Hopfield networks typically employ continuous-value neuron update rules during modeling, resulting in a significant difference between their neural dynamics and those of real biological neural systems. This discrepancy not only weakens the network's biological interpretability but also makes it difficult to directly transfer to emerging neuromorphic hardware platforms.

[0004] To bridge this gap, combining Hopfield networks with SNNs, i.e. replacing traditional Hopfield neurons with spiking neurons, not only helps improve the biological interpretability of Hopfield networks, but also makes it possible to deploy them in neuromorphic computing and low-power hardware.

[0005] However, spiking Hopfield networks face new challenges during training: on the one hand, spiking neurons are prone to over-excitation during iterative updates, leading to abnormally high membrane potentials and disrupting the network's balance and energy convergence properties; on the other hand, over-excitation also weakens the network's overall expressive power, causing its performance to decline in complex tasks. How to mitigate the over-excitation problem of spiking neurons while maintaining the advantages of the Hopfield network's energy framework has become an urgent problem to be solved.

[0006] In deep neural networks, existing research has shown that introducing spectral norm regularization can effectively constrain the spectral radius of the Jacobian matrix, enhancing the stability and robustness of model training. Transferring this idea to pulsed Hopfield networks holds promise for preventing overexcitation and improving overall convergence by constraining the propagation of inter-layer perturbations. However, currently, in the context of combining SNNs with the Hopfield energy framework, systematic research on the feasibility and application of spectral norm regularization is still lacking.

[0007] In practical image classification tasks, the classification boundaries of traditional Hopfield networks are often quite ambiguous. Since the energy function only ensures convergence to a local stable point, rather than a clear class discriminant surface, energy basins for different classes may overlap, leading to unstable classification results. Furthermore, although traditional Hopfield networks theoretically possess some noise recovery capabilities, their energy landscape is easily perturbed in complex image backgrounds or high-noise environments, thus converging to the wrong image class. Summary of the Invention

[0008] This invention provides an image classification method based on spectral norm regularization of multilayer spurious Hopfield networks. By constructing the Jacobian matrix between layers of the multilayer spurious Hopfield network and applying regularization penalty to its spectral norm, the overall neuron spur firing stability and training generalization ability of the multilayer spurious Hopfield network are effectively enhanced, thereby improving the image classification accuracy of the multilayer spurious Hopfield network.

[0009] A multi-layer pulse Hopfield network image classification method based on spectral norm regularization includes the following steps:

[0010] (1) Replace the neuron units in the multilayer Hopfield network with spiking neurons to obtain a multilayer spiking Hopfield network;

[0011] (2) The neuron state vector in the local gradient calculation formula of the multilayer spiking Hopfield network is replaced with the firing frequency of the spiking neuron;

[0012] (3) Obtain the image training set and add Jacobi matrix spectral norm regularization constraints during the training of the multilayer pulse Hopfield network;

[0013] (4) Input the image to be classified into the trained multi-layer spiking Hopfield network. The first layer of the network performs image spiking processing, and the last layer of the statistical network performs image classification by counting the pulse firings of neurons.

[0014] In step (1), the spiking neuron is either an integral firing neuron (IF) or a leaky integral firing neuron (LIF).

[0015] In step (1), the spiking neurons are given threshold, membrane potential, and pulse count attributes. In the multilayer Hopfield network, each layer of spiking neurons simultaneously receives pulse input from the layers before and after, and accumulates and sums the pulses on the membrane potential. After integrating the membrane potential, it is determined whether the threshold is reached at that time step. If it is greater than the threshold, the neuron fires a pulse at that time step and increments the pulse firing count by one; otherwise, the neuron does not fire a pulse.

[0016] In step (2), the firing frequency of the spiking neuron is obtained by dividing the firing count by the total time steps.

[0017] In step (3), the multi-layer pulse Hopfield network is trained. The specific process is as follows:

[0018] (3-1) Calculate and record the gradient of the weight matrix of each layer using the gradient calculation formula;

[0019] (3-2) Calculate the Jacobian matrix in the gradient propagation path for each sample, which is divided into the Jacobian matrix of the FC layer and the Jacobian matrix of the convolutional layer;

[0020] (3-3) After obtaining the Jacobian matrix, the spectral norm and left and right singular vectors are calculated by performing singular value decomposition on the Jacobian matrix. Then, the gradient of the spectral norm of the Jacobian matrix with respect to the weight matrix is ​​calculated through the left and right singular vectors. Finally, the gradient of the spectral norm regularization term of the Jacobian matrix is ​​calculated.

[0021] (3-4) The gradient of the spectral norm regularization term of the Jacobian matrix is ​​added to the gradient of the weight matrix, and the weight matrix distribution is optimized using the optimizer to achieve training of a multilayer spiking Hopfield network with spectral norm constraints. The weight matrix is ​​calculated as the difference between the outer products of the pulse firing frequencies of adjacent neurons in the two-stage equilibrium perturbation phase.

[0022] In step (3-2), the calculation process of the Jacobian matrix of the FC layer is as follows:

[0023] For each sample, construct the Jacobian matrix of the network's single-layer output versus input. ,in, , This is the weight matrix. The membrane potential of the input layer, The activation function is represented by the derivative of the activation function with respect to the membrane potential, which is constructed using the alternative gradient sigmoid function.

[0024] The calculation process of the Jacobian matrix of the convolutional layer is as follows:

[0025] The input features of the convolutional layer are processed by the convolutional kernel and a nonlinear activation function to obtain the activated output. The Jacobian matrix of the convolutional layer is obtained by approximation calculation using the power iteration method.

[0026] In step (3-3), the gradient of the spectral norm regularization term of the Jacobian matrix is ​​calculated using the following formula:

[0027] ;

[0028] In the formula, Here, W is the regularization term; W is the weight matrix. The derivative of the regularization term with respect to the weight matrix is ​​the gradient of the spectral norm regularization term. Represents the total number of samples. To control the overall amplitude using hyperparameters, The target spectral norm is set. For the first The spectral norm of each sample; The gradient of the spectral norm of the Jacobian matrix with respect to the weight matrix is ​​calculated using the left and right singular vectors.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. Improve the dynamic stability and controllability of the network: This invention suppresses excessive fluctuations in the singular values ​​of the Jacobian matrix between layers by regularizing the spectral norm, thereby avoiding overexcitation of the neuronal membrane potential during multilayer transmission, ensuring the dynamic stability of the pulsatile Hopfield network during energy convergence, and enabling the network to maintain equilibrium evolution during training.

[0031] 2. Enhance the separability and feature preservation of pulse signals: By embedding spectral norm constraints in the pulsed Hopfield structure, the information collapse phenomenon caused by neuron overactivation can be effectively prevented, and the pulse features can be preserved in multi-layer mapping, thereby improving the discriminability and classifiability of pulse signals and enhancing the network's ability to preserve spatiotemporal information.

[0032] 3. Improve the robustness and performance of image classification: This invention uses spectral norm regularization to weaken the network's sensitivity to input perturbations, enabling the model to maintain stable output even in the presence of noise, occlusion, or adversarial examples, effectively improving the robustness of image classification tasks. At the same time, the performance on various datasets proves the good performance of the proposed multilayer Hopfield network in image classification tasks.

[0033] 4. Balancing computational efficiency and biological interpretability: While maintaining the Hopfield energy structure, this invention uses spiking neurons (IF / LIF) to achieve information transmission and storage, which conforms to the bio-inspired mechanism of neuromorphic computing and has higher computational efficiency and interpretability. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a network structure diagram of a multilayer pulsed Hopfield network in an embodiment of the present invention.

[0036] Figure 2 This is a flowchart illustrating the training process of a multi-layer pulsed Hopfield network in an embodiment of the present invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0039] A multi-layer spiking Hopfield network image classification method based on spectral norm regularization is proposed. This method extends the Hopfield network by multiple layers, replacing the activation functions of its neurons with spiking models. Image spiking is performed in the first layer, and image classification is performed by statistically analyzing the spiking of neurons in the last layer. The weight training process of the multi-layer spiking Hopfield neural network is as follows: Figure 2 As shown. The specific description is as follows:

[0040] S1, image dataset constructed with multilayer spurious Hopfield network.

[0041] Image dataset construction: The original data is standardized and divided into training, validation and test sets according to a reasonable ratio. These sets are used for model parameter learning, hyperparameter tuning and final performance evaluation, respectively, to ensure the scientific validity and reliability of the experimental results.

[0042] Specifically, the embodiments of this invention mainly selected three representative datasets: MNIST: This dataset contains a large number of handwritten digit images with a resolution of 28×28, and is used for basic testing tasks of image classification and pattern recognition, effectively verifying the model's performance in processing low-dimensional grayscale images. CIFAR-10: This dataset consists of color images of 10 classes of natural objects, each image being 32×32 in size, reflecting the model's recognition ability when processing moderately complex color images. CIFAR-100: Compared to CIFAR-10, this dataset contains 100 more fine-grained categories, placing higher demands on the model's feature extraction ability, classification accuracy, and generalization ability, verifying the model's effectiveness in complex multi-class tasks.

[0043] Construction of multilayer pulsed Hopfield network: based on Figure 1 As shown, the extended multi-layer Hopfield network has six layers. The shape of the first layer changes according to the size of the dataset images. This layer receives the input from the image dataset and performs image spiking convolution processing. The size of the last layer varies depending on the number of classification categories in the dataset. For example, the last layer of the CIFAR10 dataset uses 10 neurons, while the last layer of the CIFAR100 dataset uses 100 neurons. The remaining layers are recurrent convolutional structures, each receiving input from both the convolutional and transposed convolutional layers simultaneously. Finally, the multi-layer Hopfield network neurons are replaced with spiking neurons, which are equipped with threshold, membrane potential, and pulse count attributes. The membrane potential receives input from the recurrent connections of adjacent layers. The threshold determines whether the neuron fires a pulse; if a pulse is fired, the pulse count is incremented.

[0044] This invention extends the traditional single-layer recurrent structure of the Hopfield network into a multi-layer structure, forming local recurrent structures between layers, and the neuron input changes as follows:

[0045] ;

[0046] in, and These represent the neuronal states of the front and rear layers, respectively. For convolution operations, , These represent the neural network weights and biases, respectively.

[0047] The continuous activation function in the traditional Hopfield network is replaced with the LIF (Leaky Integrate-and-Fire) or IF (Integrate-and-Fire) model to construct the membrane potential integral and threshold firing mechanism. At the same time, the input and output are also adapted to the pulsed network. Specifically, the input image is processed by the convolution of the first layer of spiking neurons in the Hopfield network, and the output calculates the firing frequency statistics of the last layer of spiking neurons.

[0048] S2, network input and cyclical updates of neurons in each layer.

[0049] Input sample Inputting a multilayer spiking Hopfield network, each spiking neuron in one layer simultaneously receives spiking inputs from the layers before and after it, and accumulates and sums these inputs on the membrane potential. After integrating the membrane potential, it is determined whether a threshold is reached at that time step. If the threshold is exceeded, the neuron fires a spiking signal at that time step; otherwise, the neuron does not fire a spiking signal. The spiking output of one layer of neurons is denoted as . At the same time, the pulse firing count of all neurons in that layer is updated to facilitate the subsequent calculation of the pulse firing frequency. In the next time step, the neural input of a certain layer is the pulse output generated by the neurons of the adjacent layers before and after the previous time step.

[0050] S3, Free Equilibrium Phase: The input continues for several time steps, iterating through each layer of neurons, updating the pulse firing rate until the network reaches a stable state. During this period, the time-averaged number of pulse firings across all layers of neurons is recorded to facilitate the calculation of the pulse firing frequency. .

[0051] S4, Perturbation Phase: While the input image persists for several time steps, a perturbation term is added to the input of the output layer neurons. The value of the perturbation term is calculated as the derivative of the output layer neuron's pulse firing frequency with the squared distance to the target label. This process continues for several time steps, iterating through each layer of neurons to update the pulse firing of all layers of the spiking Hopfield network until the network reaches a new stable state. The time-averaged pulse firing frequency of all layers of the spiking Hopfield network is recorded. .

[0052] S5, Gradient calculation of the weight matrix.

[0053] The gradient of the weight matrix of each layer is calculated and recorded using the gradient calculation formula. If there is no regularization constraint, the SGD optimizer can be used to optimize the weights based on the weight gradient. If the Jacobian matrix spectral norm regularization constraint is added, the gradient of the Jacobian matrix spectral norm regularization term also needs to be calculated.

[0054] Calculating the weight gradient of the FC layer in a multi-layer pulsed Hopfield network: For a pulsed Hopfield network, the neuron state needs to be redefined as the time-averaged frequency of pulse firing, where the gradient update formula for the FC layer becomes:

[0055] ;

[0056] in, and For the first and The pulse firing frequency of a layer, with the subscripts free or nudge indicating whether it is the pulse frequency of neurons in the free equilibrium phase or the pulse frequency of neurons in the perturbation phase.

[0057] Calculating the convolutional layer weight gradients of a multi-layer spiking Hopfield network: For a spiking Hopfield network, the formula for the convolutional layer gradient needs to be modified and adapted for spiking neurons, and it becomes:

[0058] ;

[0059] in, and For the first and The pulse firing frequency of a layer, with the subscripts "free" or "nudge" indicating whether it is the pulse frequency of neurons in the free equilibrium phase or the pulse frequency of neurons in the perturbation phase. This is for reverse pooling.

[0060] S6, construct the Jacobian matrix.

[0061] Calculate the Jacobian matrix in the gradient propagation path for each sample. It is divided into Jacobian matrices for FC layers and Jacobian matrices for convolutional layers, and its dimensions are constructed according to the shape of the preceding and following layers.

[0062] Constructing the Jacobian matrix of the FC layer: For each sample, construct the Jacobian matrix of the network's single-layer output against the input. ,in, , This is the weight matrix. The membrane potential of the input layer, The activation function is represented by the derivative of the activation function with respect to the membrane potential, which is constructed using the alternative gradient sigmoid function.

[0063] Constructing the Jacobian matrix of the convolutional layer: The input features of the convolutional layer are processed by the convolution kernel and a nonlinear activation function to obtain the activated output. Since the Jacobian matrix of the convolutional layer is too large when directly unfolded, the power iteration method is usually used for approximate calculation. The core process is as follows: initialize random vectors and normalize them. After the iteration converges, the spectral norm approximation is obtained, that is, the spectral norm of the Jacobian matrix of the convolutional layer.

[0064] S7, calculate the spectral norm normalized gradient to obtain the Jacobian matrix. Then, using the Jacobian matrix Spectral norm to Jacobian matrix The gradient is used to calculate the Jacobian matrix. The gradient of the spectral norm with respect to the weight matrix is ​​used to construct the gradient of the regularization term with respect to the weight matrix, where the convolutional Jacobian matrix is... The spectral norm can be calculated using methods such as Power iteration or truncated SVD to estimate the spectral norm, and finally the gradient of the regularization term is obtained.

[0065] Calculated for each sample Calculate its spectral norm Construct regular terms :

[0066] ;

[0067] set up The constrained spectral norm, the derivative of which is the regularization term with respect to the weight matrix, is:

[0068] ;

[0069] Among them, the Jacobian matrix is ​​calculated. Gradient of spectral norm with respect to the weight matrix The method can utilize the Jacobian matrix. Spectral norm to Jacobian matrix The gradient is calculated using a chain method, employing... Implement the gradient calculation of the derivative of the spectral norm with respect to the weights, where, Spectral norm The outer product of the corresponding left and right singular vectors , This represents the sample index. Finally, the gradient is calculated for each sample in a batch, and the average gradient of all samples is added to the loss gradient before being used together for training.

[0070] S8, optimize the training weight matrix.

[0071] The gradient of the regularization term with respect to the weight matrix is ​​added to the gradient of the weight matrix, and the optimizer is used together to optimize the distribution of the weight matrix, thereby achieving the training of a multi-layer spurious Hopfield network with spectral norm constraints.

[0072] S9 involves repeating the training steps for 5000 images per epoch, for a total of 100 epochs. After each epoch, the accuracy is tested on the test set. The image classification label is calculated by statistically analyzing the firing frequency of neurons in the last layer of the network. The final network accuracy is obtained by comparing it with the correctly classified image label. The classification accuracy of the three specific image classification datasets is shown in Table 1.

[0073] Table 1

[0074]

[0075] This invention presents an image classification method based on a multi-layer pulsed Hopfield network with spectral norm regularization. By introducing spectral norm constraint terms during the inter-layer iteration process of the multi-layer Hopfield network, the method achieves self-steady-state control of network dynamics and global energy optimization.

[0076] The embodiments described above provide a detailed explanation of the technical solutions and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-layer pulse Hopfield network image classification method based on spectral norm regularization, characterized in that, Includes the following steps: (1) Replace the neuron units in the multilayer Hopfield network with spiking neurons to obtain a multilayer spiking Hopfield network; (2) The neuron state vector in the local gradient calculation formula of the multilayer spiking Hopfield network is replaced with the firing frequency of the spiking neuron; (3) Obtain the image training set and add Jacobian matrix spectral norm regularization constraints during the training of the multilayer pulse Hopfield network; the specific process is as follows: (3-1) Calculate and record the gradient of the weight matrix of each layer using the gradient calculation formula; (3-2) Calculate the Jacobian matrix in the gradient propagation path for each sample, which is divided into the Jacobian matrix of the FC layer and the Jacobian matrix of the convolutional layer; (3-3) After obtaining the Jacobi matrix, the spectral norm and left and right singular vectors are calculated by performing singular value decomposition on the Jacobi matrix. Then, the gradient of the spectral norm of the Jacobi matrix with respect to the weight matrix is ​​calculated through the left and right singular vectors. Finally, the gradient of the spectral norm regularization term of the Jacobi matrix is ​​calculated. (3-4) Add the gradient of the spectral norm regularization term of the Jacobian matrix to the gradient of the weight matrix, and use the optimizer together to optimize the distribution of the weight matrix to achieve training of the multilayer spurious Hopfield network with spectral norm constraints. (4) Input the image to be classified into the trained multi-layer spiking Hopfield network. The first layer of the network performs image spiking processing, and the last layer of the statistical network performs image classification by counting the pulse firings of neurons.

2. The image classification method based on spectral norm regularization using a multilayer pulse Hopfield network according to claim 1, characterized in that, In step (1), the spiking neuron is either an integral firing neuron (IF) or a leaky integral firing neuron (LIF).

3. The image classification method based on spectral norm regularization using a multilayer pulse Hopfield network according to claim 1, characterized in that, In step (1), threshold, membrane potential, and pulse count attributes are added to the spiking neurons; wherein, the pulse output generated by a layer of neurons is denoted as... Simultaneously, the pulse firing count of all neurons in that layer is updated. In the next time step, the neural input of a certain layer is the pulse output generated by the neurons in the adjacent layers before and after the previous time step. In the multilayer Hopfield network, each layer of spiking neurons simultaneously receives pulse input from the layers before and after, and accumulates and sums them on the membrane potential. After integrating the membrane potential, it is determined whether the threshold is reached in that time step. If it is greater than the threshold, the neuron fires a pulse in that time step, and the pulse firing count is incremented by one; otherwise, the neuron does not fire a pulse.

4. The image classification method based on spectral norm regularization using a multilayer pulse Hopfield network according to claim 3, characterized in that, In step (2), the firing frequency of the spiking neuron is obtained by dividing the firing count by the total time steps.

5. The image classification method based on spectral norm regularization using a multilayer pulse Hopfield network according to claim 1, characterized in that, In step (3-2), the calculation process of the Jacobian matrix of the FC layer is as follows: For each sample, construct the Jacobian matrix of the network's single-layer output versus input. ,in, , This is the weight matrix. The membrane potential of the input layer, The activation function is represented by the derivative of the activation function with respect to the membrane potential, which is constructed using the alternative gradient sigmoid function.

6. The image classification method based on spectral norm regularization using a multilayer pulse Hopfield network according to claim 1, characterized in that, In step (3-2), the calculation process of the Jacobian matrix of the convolutional layer is as follows: The input features of the convolutional layer are processed by the convolutional kernel and a nonlinear activation function to obtain the activated output. The Jacobian matrix of the convolutional layer is obtained by approximation calculation using the power iteration method.

7. The image classification method based on spectral norm regularization using a multilayer pulse Hopfield network according to claim 1, characterized in that, In step (3-3), the gradient of the spectral norm regularization term of the Jacobian matrix is ​​calculated using the following formula: ； In the formula, Here, W is the regularization term; W is the weight matrix. The derivative of the regularization term with respect to the weight matrix is ​​the gradient of the spectral norm regularization term. Represents the total number of samples. To control the overall amplitude using hyperparameters, The target spectral norm is set. For the first The spectral norm of each sample; The gradient of the spectral norm of the Jacobian matrix with respect to the weight matrix is ​​calculated using the left and right singular vectors.

Images