Continuous learning method of neuromorphic classification network and neuromorphic classification method
By calculating the gradient information autocorrelation matrix of the neuromorphic classification network, the consistency of parameter update direction and task relevance are characterized, which solves the problems of catastrophic forgetting and insufficient generalization ability of neuromorphic classification models in dynamic environments, and realizes knowledge transfer and accurate classification across architectures and multiple scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-03
- Publication Date
- 2026-03-27
AI Technical Summary
Traditional deep learning models lack the ability to adapt continuously in dynamic environments and face the problem of catastrophic forgetting. Furthermore, existing neuromorphic classification methods have poor generalization ability across multiple scenarios and architectures.
By calculating the autocorrelation matrix of the gradient information of the historical parameter tensors of each network layer in the neuromorphic classification network, the consistency of the parameter update direction of the structural units and the relevance to the task are characterized. The autocorrelation matrix is used to regularize the parameter update and suppress the over-adjustment of the historical task-sensitive direction.
It effectively avoids catastrophic forgetting, enhances knowledge transfer and generalization capabilities, adapts to spiking neural networks of different structures and sizes, and maintains classification accuracy.
Smart Images
Figure CN121746771A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the field of image classification, and more particularly relates to a continuous learning method of a neuromorphic classification network and a neuromorphic classification method. BACKGROUND
[0002] With the rapid development of artificial intelligence and deep learning technology, neural networks have made significant progress in many fields. However, traditional deep learning models are usually trained in an offline static environment and optimized for specific tasks. Such models lack the ability to continuously adapt to dynamic environments or changes in task distribution. Once a new task arrives, the model often forgets previously learned knowledge, a phenomenon known as catastrophic forgetting. This problem severely limits the long-term autonomous learning and decision-making ability of intelligent agents in real complex environments.
[0003] To solve this problem, researchers have proposed a continuous learning (CL) mechanism. This method enables the model to gradually learn new knowledge while retaining old knowledge in a sequence of tasks. However, despite the theoretical ability of CL to significantly improve model adaptability, its practical application still faces the important challenge of catastrophic forgetting.
[0004] At the same time, spiking neural networks (SNN) as the third generation of brain-like neural networks, transmit information through discrete spikes, have an event-driven computing mechanism and efficient temporal information processing capability, and are therefore particularly suitable for processing sparse spatiotemporal data generated by neuromorphic sensors and time-dependent tasks.
[0005] However, combining SNN with CL to build an intelligent system with energy efficiency and adaptive learning ability still faces the serious challenge of catastrophic forgetting. Currently, there are three main solutions to this problem. The method based on neuron regulation adjusts the dynamic characteristics of neurons or synapses to enhance the adaptability and memory retention ability of the model. Although this method has strong biological rationality, it is usually complex in mechanism, difficult to implement, and has high computational overhead. The method based on parameter isolation maintains existing knowledge by structurally constraining parameter updates. This method performs well in the task of incremental learning, but often relies on additional classification head structures, and its scalability and efficiency are limited in more extensive learning paradigms. The hybrid method combines SNN with traditional artificial neural networks, which has certain advantages in performance, but usually relies on specific network structures and is difficult to fully exploit the unique characteristics of SNN. At the same time, most researches are still limited to static data sets, and the exploration of dynamic neuromorphic scenarios is relatively insufficient.
[0006] Therefore, designing a general continuous learning method that can maintain the low-power characteristics of SNN and effectively alleviate catastrophic forgetting has become a key research direction in the field of neuromorphic intelligence. In the context of neuromorphic classification tasks, in order to realize continuous learning and knowledge preservation, the following three technical challenges are mainly faced: First, the problem of catastrophic forgetting. When the model is trained on sequential tasks, the learning of new tasks often leads to the covering or forgetting of previous task knowledge, especially in neuromorphic data, the time dynamic characteristics and sparse coding further amplify the forgetting effect. How to effectively preserve existing knowledge while maintaining the ability to learn new tasks is a core problem in continuous learning based on spiking neural networks.
[0007] Second, the problem of knowledge preservation and generalization in multiple scenarios. In the actual application of neuromorphic datasets, task types, users and illumination condition domains often change over time. Traditional methods often lack uniformity in the task incremental and domain incremental scenarios, making it difficult to maintain consistent performance in different task settings. How to build a unified learning method compatible with task incremental and domain incremental, which can effectively suppress forgetting and achieve knowledge transfer in different scenarios, is the key to realizing the universalization of continuous learning.
[0008] Third, the algorithm adaptation and generalization problem across architectures. Most continuous learning methods rely on specific network structures, making it difficult to directly transfer in different spiking neural network architectures. How to design an architecture-independent regularization mechanism to enable continuous learning methods to maintain stability and scalability in different structures and different sizes of spiking neural networks, and to achieve excellent model learning for differentially distributed neuromorphic datasets, is an important challenge for cross-model universal learning. SUMMARY
[0009] In view of the above defects or improvement needs of the prior art, the present application provides a continuous learning method for a neuromorphic classification network and a neuromorphic classification method, thereby solving the technical problem that traditional methods cannot overcome the catastrophic forgetting problem, resulting in neuromorphic classification accuracy that cannot be guaranteed in different distributions and different tasks, and poor generalization ability in multiple scenarios and cross-architecture scenarios.
[0010] To achieve the above purpose, according to the first aspect of the present application, a continuous learning method for a neuromorphic classification network is provided, comprising: When a new classification task arrives, the historical parameter tensor of each network layer of the neuromorphic classification network after training for historical classification tasks and the gradient information of the historical parameter tensor are obtained; wherein the neuromorphic classification network is constructed based on a spiking neural network; For any network layer in the neuromorphic classification network, a self-correlation matrix of gradient information of a historical parameter tensor of the any network layer is calculated; wherein the self-correlation matrix contains the correlation of each structural unit in the any network layer in the historical classification task; In the training process for the new classification task, a current parameter tensor of each network layer in the neuromorphic classification network is obtained, a parameter change tensor between the current parameter tensor and the historical parameter tensor of each network layer is calculated, a structure consolidation loss is calculated based on the self-correlation matrix of gradient information of the historical parameter tensor of each network layer and the parameter change tensor between the current parameter tensor and the historical parameter tensor of each network layer, a classification loss of the neuromorphic classification network is obtained based on the sample neuromorphic image and its label, and the parameters of each network layer in the neuromorphic classification network are adjusted based on the structure consolidation loss and the classification loss.
[0011] According to the above continuous learning method of the neuromorphic classification network, the calculation of the self-correlation matrix of the gradient information of the historical parameter tensor of the any network layer specifically includes: The gradient information of the historical parameter tensor of the any network layer is flattened and combined with the first dimension of the gradient information of the historical parameter tensor of the any network layer to form a two-dimensional gradient matrix; Based on the two-dimensional gradient matrix and its transpose matrix, the self-correlation matrix of the gradient information of the historical parameter tensor of the any network layer is calculated.
[0012] According to the above continuous learning method of the neuromorphic classification network, the self-correlation matrix of the gradient information of the historical parameter tensor of the any network layer is calculated based on the following formula:
[0013] Wherein, is the self-correlation matrix, is a batch of training samples used when training for the historical classification task, is the two-dimensional gradient matrix corresponding to the historical parameter tensor of the any network layer obtained after training based on the batch of training samples.
[0014] According to the above continuous learning method of the neuromorphic classification network, the calculation of the structure consolidation loss based on the self-correlation matrix of the gradient information of the historical parameter tensor of each network layer and the parameter change tensor between the current parameter tensor and the historical parameter tensor of each network layer specifically includes: combining all dimensions except the first dimension of the parameter change tensor between the current parameter tensor and the historical parameter tensor of any network layer after being flattened with the first dimension of the parameter change tensor between the current parameter tensor and the historical parameter tensor of the any network layer into a two-dimensional parameter change matrix; calculating the structure consolidation loss based on the Frobenius norm between the autocorrelation matrix of the gradient information of the historical parameter tensor of each network layer and the parameter change matrix.
[0015] According to the continuous learning method of the neuromorphic classification network, the structure consolidation loss is calculated based on the following formula:
[0016] wherein, is the structure consolidation loss, is a regularization coefficient, is the autocorrelation matrix of the i-th network layer, is the parameter change matrix of the i-th network layer, denotes the Frobenius norm, and Tr denotes the trace of a matrix.
[0017] According to a second aspect of the present application, a continuous learning device for the neuromorphic classification network is provided, comprising: a gradient information acquisition unit configured to acquire the historical parameter tensor of each network layer of the neuromorphic classification network after being trained for historical classification tasks and the gradient information of the historical parameter tensor when a new classification task arrives, wherein the neuromorphic classification network is constructed based on a spiking neural network; a correlation determination unit configured to calculate the autocorrelation matrix of the gradient information of the historical parameter tensor of any network layer in the neuromorphic classification network, wherein the autocorrelation matrix contains the correlation of each structural unit in the any network layer in the historical classification tasks; a continuous learning unit configured to acquire the current parameter tensor of each network layer in the neuromorphic classification network during the training process for the new classification task, calculate the parameter change tensor between the current parameter tensor and the historical parameter tensor of each network layer, calculate the structure consolidation loss based on the autocorrelation matrix of the gradient information of the historical parameter tensor of each network layer and the parameter change tensor between the current parameter tensor and the historical parameter tensor of each network layer, acquire the classification loss of the neuromorphic classification network based on the sample neuromorphic image and its label, and adjust the parameters of each network layer in the neuromorphic classification network based on the structure consolidation loss and the classification loss.
[0018] According to a third aspect of the present application, a neuromorphic classification method is provided, comprising: obtain a current neuromorphic image to be classified; input the neuromorphic image to be classified into a neuromorphic classification network to obtain a classification result of the neuromorphic image to be classified output by the neuromorphic classification network; The neuromorphic classification network is trained based on the continuous learning method of the neuromorphic classification network according to the first aspect.
[0019] According to a fourth aspect of the present application, a neuromorphic classification apparatus is provided, comprising: an image obtaining unit configured to obtain a current neuromorphic image to be classified; a neuromorphic image classification unit configured to input the neuromorphic image to be classified into a neuromorphic classification network to obtain a classification result of the neuromorphic image to be classified output by the neuromorphic classification network; The neuromorphic classification network is trained based on the continuous learning method of the neuromorphic classification network according to the first aspect.
[0020] According to a fifth aspect of the present application, an electronic device is provided, comprising a computer readable storage medium and a processor. The computer readable storage medium is configured to store executable instructions. The processor is configured to read the executable instructions stored in the computer readable storage medium and execute the method according to the first aspect or the third aspect.
[0021] According to a sixth aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for causing a processor to execute the method according to the first aspect or the third aspect.
[0022] Overall, the above technical solutions conceived by the present application can achieve the following beneficial effects compared with the prior art: Aiming at the problems that in continuous learning (CL) of neuromorphic dataset, the model is easy to forget old knowledge when learning new tasks, it is difficult to balance stability and plasticity at the same time, and the generalization ability is poor in multiple scenes and cross-architecture situations, a parameter importance estimation and constraint method based on gradient statistical structure is provided, the autocorrelation matrix of gradient information is calculated, each network layer in the neuromorphic classification network is taken as the target, the consistency and task relevance of the parameter update direction of each structural unit in each network layer are described, and a fine-grained structural importance representation is obtained. When learning a new task, the continuous learning method provided by the application uses the autocorrelation matrix to regularize and constrain the parameter update, suppresses the excessive adjustment of the sensitive direction to the historical task, and effectively avoids catastrophic forgetting. At the same time, the continuous learning method evaluates the importance of each structural unit in the historical classification task by describing the consistency and task relevance of the parameter update direction of each structural unit in the same network layer, thereby constraining the parameter update direction and avoiding adjusting the parameters of the direction sensitive to the historical classification task, realizing structural independence and scene compatibility, being able to adapt to different structures and different sizes of pulse neural networks, and effectively suppressing forgetting and realizing knowledge transfer in different scenes. BRIEF DESCRIPTION OF DRAWINGS
[0023] Figure 1 A flowchart of the continuous learning method of the neuromorphic classification network provided by the embodiment of the application is shown. Figure 2 A schematic diagram of the sensitivity analysis method based on gradient provided by the embodiment of the application is shown, wherein (a) represents the memory mechanism in the human brain, (b) represents a two-dimensional parameter space, and (c) represents a continuous learning process. Figure 3 A schematic diagram of the CIFAR10-DVS task incremental learning performance comparison provided by the embodiment of the application is shown. Figure 4 A schematic diagram of the DVS128 Gesture task incremental learning performance comparison provided by the embodiment of the application is shown. Figure 5 A schematic diagram of the MNIST-DVS task incremental learning performance comparison provided by the embodiment of the application is shown. Figure 6 A schematic diagram of the domain incremental performance comparison of DVS128 Gesture is shown, wherein the left side corresponds to the continuous learning of two domains, and the right side corresponds to the continuous learning of three domains. Figure 7 A schematic diagram of the continuous learning performance comparison on two MNIST domains provided by the embodiment of the application is shown. Figure 8Figures showing the changes of AA and BWT under different λ values provided by the embodiments of the present application, wherein (a) is a figure showing the changes of AA and BWT under different λ values for the CIFAR10-DVS data set, and (b) is a figure showing the changes of AA and BWT under different λ values for the DVS128 Gesture data set. DETAILED DESCRIPTION
[0024] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.
[0025] The embodiments of the present application provide a continuous learning method of a neuromorphic classification network, as shown in Figure 1 The method comprises the following steps: In step 110, when a new classification task arrives, the historical parameter tensor of each network layer of a neuromorphic classification network after training for historical classification tasks and gradient information of the historical parameter tensor are obtained; wherein the neuromorphic classification network is constructed based on a spiking neural network; In step 120, for any network layer in the neuromorphic classification network, the autocorrelation matrix of the gradient information of the historical parameter tensor of the any network layer is calculated; wherein the autocorrelation matrix contains the correlation of each structural unit in the any network layer in the historical classification tasks; In step 130, during the training process for the new classification task, the current parameter tensor of each network layer in the neuromorphic classification network is obtained, the parameter change tensor between the current parameter tensor and the historical parameter tensor of each network layer is calculated, the structural consolidation loss is calculated based on the autocorrelation matrix of the gradient information of the parameter tensor of each network layer and the parameter change tensor between the current parameter tensor and the historical parameter tensor of each network layer, the classification loss of the neuromorphic classification network is obtained based on the sample neuromorphic image and its label, and the parameters of each network layer in the neuromorphic classification network are adjusted based on the structural consolidation loss and the classification loss.
[0026] Here, the synaptic consolidation concept in biology believes that synapses with persistent or high-intensity activity will be selectively stabilized to maintain memory. Empirical research shows that synaptic efficacy is closely related to the intensity and frequency of neuron activation, therefore, the embodiments of the present application will simulate this mechanism through gradient-based sensitivity analysis, as shown in Figure 2The problems that the model in the continuous learning CL of the neuromorphic dataset is easy to forget old knowledge when learning a new task, is difficult to balance stability and plasticity at the same time, and has poor generalization ability in multiple scenes and cross-architecture cases. Embodiments of the present application propose a parameter importance estimation and constraint method based on gradient statistical structure. By calculating the autocorrelation matrix of gradient information, the consistency and task relevance of the parameter update direction of each structural unit in each network layer in the neuromorphic classification network are described, so as to obtain a fine-grained structural importance representation. When learning a new task, the continuous learning method uses the autocorrelation matrix to regularize and constrain the parameter update, suppresses the excessive adjustment of the sensitive direction to the historical task, and effectively avoids catastrophic forgetting. At the same time, the continuous learning method evaluates the parameter importance of each structural unit in the historical classification task by describing the consistency and task relevance of the parameter update direction of each structural unit in the same network layer, thereby constraining the parameter update direction and avoiding adjusting the parameters of the direction sensitive to the historical classification task. The structural independence and scene compatibility are realized, which can adapt to different structures and different sizes of the spiking neural network, and can effectively suppress forgetting and realize knowledge transfer in different scenes.
[0027] Specifically, when a new classification task arrives, the historical parameter tensor of each network layer of the neuromorphic classification network after training for a historical classification task and the gradient information of the historical parameter tensor are obtained. The neuromorphic classification network is constructed based on a spiking neural network, but embodiments of the present application do not specifically limit the specific structure and size of the spiking neural network, because the continuous learning method provided by embodiments of the present application is suitable for different structures and different sizes of the spiking neural network. Assuming that the new classification task is T k+1 The new classification task can be a different classification task (i.e., different task labels, corresponding to task incremental learning scenarios) from the historical classification task, or a classification task (corresponding to domain incremental learning scenarios) with the same historical classification task but different distribution of input data to be classified, which is not specifically limited by embodiments of the present application.
[0028] When a new classification task arrives, the historical parameter tensor of each network layer of the neuromorphic classification network after training for a historical classification task can be obtained, for example, the historical parameter tensor of each network layer of the neuromorphic classification network after training for the latest historical classification task T kAfter training, the historical parameter tensor of each network layer. Among them, the historical parameter tensor of any network layer contains the parameter values of each structural unit in the network layer updated after this training, and the structural unit can be a neuron in the fully connected layer, a filter or a channel in the convolutional layer, etc. Based on the historical parameter tensor of each network layer, the gradient information of each historical parameter tensor can be calculated. The gradient information contains the gradient of each parameter corresponding to the structural unit in the training process for the historical classification task, and the gradient also encodes the learning direction information and sensitivity of each parameter with respect to the historical classification task. That is, the gradient information of the historical parameter tensor expresses that in the historical classification task, each parameter should be in what learning direction (for example, "increase" or "decrease") to most effectively reduce the error rate of the task, and also expresses the "sensitivity" of the parameter, that is, the influence degree of each parameter on the correct rate of the task under the historical classification task. Among them, given the batch training sample k from task T , the loss function is calculated based on the following formula (which can be calculated based on the batch training sample using the classification loss function) about gradient information :
[0029] Among them, is the historical parameter tensor of the i-th network layer, is the gradient operator.
[0030] Because the gradient information of the historical parameter tensor encodes the learning direction information and sensitivity of each parameter with respect to the historical classification task, in order to depict the dependency relationship and direction importance between structural units in the same network layer, and thus quantify the importance of the parameters corresponding to each structural unit to the historical classification task, for any network layer in the neuromorphic classification network, the autocorrelation matrix of the gradient information of the historical parameter tensor of the network layer can be calculated. Among them, the autocorrelation matrix contains the correlation of each structural unit in the network layer in the historical classification task. Specifically, the autocorrelation matrix evaluates the similarity and stability between the gradients of the parameters corresponding to two structural units in the network layer in the historical training. In the autocorrelation matrix, a high autocorrelation value indicates that the update mode (i.e., the error feedback mode it feels) of the parameters of the corresponding structural unit is highly correlated and consistent at different stages of the training process, and the importance of the parameters of the corresponding structural unit to the historical classification task is also higher. For structural units with higher importance, the parameter adjustment of the corresponding structural unit needs to be constrained in the training process of the new classification task, to avoid the corresponding structural unit's parameter deviating from the previous learning value, thereby avoiding the catastrophic forgetting problem.
[0031] In some embodiments, to calculate the autocorrelation matrix of the gradient information of the historical parameter tensor of any network layer, all dimensions other than the first dimension of the gradient information of the historical parameter tensor of the network layer can be flattened and combined with the first dimension of the gradient information of the historical parameter tensor of the network layer to form a two-dimensional gradient matrix. :
[0032] in, , Let be the length of the j-th dimension in the gradient information of the historical parameter tensor of this network layer.
[0033] Here, `reshape()` is a matrix reshaping operation used to convert an m-dimensional historical parameter tensor into a two-dimensional gradient matrix. Each row in the graph corresponds to the gradient of a structural unit in the corresponding network layer.
[0034] Subsequently, as Figure 2 As shown in (c), the autocorrelation matrix of the gradient information of the historical parameter tensor of the network layer can be calculated using the following formula based on the above two-dimensional gradient matrix and its transpose:
[0035] in, The above autocorrelation matrix is... These are the batch training samples used when training for historical classification tasks. This is the two-dimensional gradient matrix corresponding to the historical parameter tensor of the network layer obtained after training based on this batch of training samples.
[0036] It should be noted that in spiking neural networks, the state evolution of neurons is driven by discrete impulse events, and the gradient signal exhibits discontinuous and sparse characteristics, making it difficult to directly apply traditional second-order approximations or Fisher information estimation. However, the embodiments of this invention characterize the importance of structural units through the statistical structure of the autocorrelation matrix of gradient information, capturing task-related structural sensitivity without explicitly calculating higher-order derivatives. Therefore, it can better adapt to the discrete impulse-driven dynamics and sparse activity characteristics of spiking neural networks, and can serve as an approximation consistent with biological mechanisms to simulate synaptic plasticity based on impulse rate.
[0037] Therefore, for the new classification task T k+1In the training process, the current parameter tensor of each network layer in the neuromorphic classification network can be obtained, and the parameter change tensor between the current parameter tensor and the historical parameter tensor of each network layer can be calculated. The current parameter tensor of any network layer can be determined based on the classification loss calculated in the current training process. The parameter change tensor between the current parameter tensor and the historical parameter tensor of any network layer contains the change amount of the parameters of each structural unit of the network layer relative to the learning value in the historical training process in the current training process. The structural consolidation loss is calculated based on the autocorrelation matrix of the gradient information of the historical parameter tensor of each network layer and the parameter change tensor between the current parameter tensor and the historical parameter tensor of each network layer. The structural consolidation loss is used to punish the degree of deviation of the parameters of important structural units from the previous learning value. This punishment mechanism can guide the parameter update to avoid sensitive directions in the parameter space and avoid modifying the parameters of important structural units in the training process, thereby preserving existing knowledge.
[0038] In some embodiments, in order to calculate the structural consolidation loss, the parameter change tensor between the current parameter tensor and the historical parameter tensor of any network layer can be flattened in all dimensions except the first dimension and combined with the first dimension of the parameter change tensor between the current parameter tensor and the historical parameter tensor of the network layer to form a two-dimensional parameter change matrix. The two-dimensional conversion processing manner of the parameter change tensor is similar to the processing manner of the gradient information of the historical parameter tensor, which will not be described here. Subsequently, the structural consolidation loss is calculated based on the Frobenius norm between the autocorrelation matrix of the gradient information of the historical parameter tensor of each network layer and the parameter change matrix. For example, the structural consolidation loss can be calculated based on the following formula:
[0039] wherein, is the structural consolidation loss, is a regularization coefficient, is the autocorrelation matrix of the i-th network layer, is the parameter change matrix of the i-th network layer, denotes the Frobenius norm, and Tr denotes the trace of the matrix.
[0040] In addition, a classification loss of the neuromorphic classification network is also calculated based on the sample neuromorphic image and the label thereof (the classification loss can be calculated based on a loss function such as cross entropy, which is not specifically limited in the embodiments of the present application), which is used to punish classification errors for a new classification task. Based on the above structural consolidation loss and classification loss, a total loss for the new classification task can be calculated, so that the parameters of each network layer in the neuromorphic classification network are adjusted according to the total loss, so as to take into account the classification accuracy for the new classification task and the knowledge retention for the old classification task, and maintain the classification accuracy for different classification tasks in the process of continuous learning.
[0041] Figures 3-7 The experimental results of the above embodiments in four public data sets (CIFAR10-DVS, DVS128 Gesture, DVS128 Gesture and MNIST-DVS), three backbone networks (SNNBaseModel, Spikingformer and SEWResNet), and two experimental scenarios of domain increment and task increment are shown, and the optimal indicators in the experimental results are shown in bold. Among them, AA is average accuracy, and BWT is backward transfer. Since AA provides the main basis for evaluating the performance of the model, AA is used as the main standard in most continuous learning scenarios. BWT only provides supplementary insights into forgetting when the model reaches a similar accuracy level. Therefore, the analysis emphasizes AA, and BWT is used as an auxiliary indicator of stability.
[0042] From the results of different backbone networks and experimental scenarios, the continuous learning method provided by the above embodiments is better than the remaining baseline methods in all cases, which fully verifies its effectiveness in improving the classification accuracy and generalization ability of neuromorphic images.
[0043] Among them, for the task increment scenario, on the CIFAR10-DVS dataset, the average AA of the three backbone networks is improved by 1.68% compared with the suboptimal baseline; on the DVS128 Gesture dataset, the AA is improved by 8.84%; on the DVS128 Gesture dataset, the AA is improved by 5.59%. For the domain increment scenario, on the DVS128 Gesture dataset, the AA of the two backbone networks in the two experimental settings reaches the optimal effect, and the average is improved by 0.55%; in the domain increment experiment of the two MNIST datasets, the AA is improved by 0.09%. In the 15 experimental settings, the BWT indicator of the continuous learning method provided by the above embodiments achieves 7 optimal, 5 suboptimal, which shows its stable memory ability in the task training process.
[0044] Furthermore, to verify the robustness of the proposed continuous learning method to the regularization coefficient λ, parameter sensitivity analysis was performed using the CIFAR10-DVS and DVS128Gesture datasets under task increments. All experiments used the SEWResNet model to maintain evaluation consistency. Figure 8 As shown, the results indicate that AA and BWT exhibit only slight fluctuations under different λ values. This demonstrates that the proposed method is not very sensitive to the choice of λ and can maintain stable performance across a wide range of hyperparameter settings. This insensitivity to λ further highlights the robustness and practicality of the proposed method in real-world applications.
[0045] In summary, the continuous learning method provided in this invention addresses the problems of models in continuous learning CL (Continuous Learning Classification) of neuromorphic datasets, such as easy forgetting of old knowledge when learning new tasks, difficulty in simultaneously maintaining stability and plasticity, and poor generalization ability in multiple scenarios and cross-architecture situations. It offers a parameter importance estimation and constraint method based on gradient statistical structures. By calculating the autocorrelation matrix of gradient information, targeting each network layer in the neuromorphic classification network, it characterizes the consistency and task relevance of parameter update directions of each structural unit in each network layer, thereby obtaining a fine-grained representation of structural importance. When learning a new task, this continuous learning method uses this autocorrelation matrix to regularize parameter updates, suppressing over-adjustment to directions sensitive to historical tasks and effectively avoiding catastrophic forgetting. Simultaneously, by characterizing the consistency and task relevance of parameter update directions among structural units within the same network layer, this continuous learning method assesses the importance of each structural unit in historical classification tasks, thereby constraining parameter update directions and avoiding parameter adjustments to directions sensitive to historical classification tasks. This achieves structural independence and scenario compatibility, adapting to spiking neural networks of different structures and sizes, and effectively suppressing forgetting and achieving knowledge transfer in different scenarios.
[0046] Based on any of the above embodiments, this invention also provides a neuromorphic classification method, including: Obtain the current neural morphology image to be classified; The neuromorphic image to be classified is input into the neuromorphic classification network to obtain the classification result of the neuromorphic image to be classified output by the neuromorphic classification network. The neuromorphic classification network is trained based on the continuous learning method of the neuromorphic classification network provided in any of the above embodiments.
[0047] The continuous learning device for neuromorphic classification networks provided by the present invention will be described below. The continuous learning device for neuromorphic classification networks described below and the continuous learning method for neuromorphic classification networks described above can be referred to in correspondence.
[0048] The embodiment of the present application provides a continuous learning device of a neuromorphic classification network, comprising: a gradient information acquisition unit, configured to acquire a historical parameter tensor of each network layer of the neuromorphic classification network after the neuromorphic classification network is trained for a historical classification task and gradient information of the historical parameter tensor when a new classification task arrives; wherein the neuromorphic classification network is constructed based on a spiking neural network; a correlation determination unit, configured to calculate an autocorrelation matrix of the gradient information of the historical parameter tensor of any network layer in the neuromorphic classification network; wherein the autocorrelation matrix contains correlations of each structural unit in the any network layer in the historical classification task; a continuous learning unit, configured to acquire a current parameter tensor of each network layer in the neuromorphic classification network in a training process for the new classification task, calculate a parameter change tensor between the current parameter tensor and the historical parameter tensor of each network layer, calculate a structure consolidation loss based on the autocorrelation matrix of the gradient information of the historical parameter tensor of each network layer and the parameter change tensor between the current parameter tensor and the historical parameter tensor of each network layer, acquire a classification loss of the neuromorphic classification network based on a sample neuromorphic image and a label thereof, and adjust parameters of each network layer in the neuromorphic classification network based on the structure consolidation loss and the classification loss.
[0049] Based on any of the above embodiments, the autocorrelation matrix of the gradient information of the historical parameter tensor of the any network layer is calculated, and specifically comprises: combining all dimensions except the first dimension of the gradient information of the historical parameter tensor of the any network layer with the first dimension of the gradient information of the historical parameter tensor of the any network layer into a two-dimensional gradient matrix after flattening; calculating the autocorrelation matrix of the gradient information of the historical parameter tensor of the any network layer based on the two-dimensional gradient matrix and a transpose matrix thereof.
[0050] Based on any of the above embodiments, the autocorrelation matrix of the gradient information of the historical parameter tensor of the any network layer is calculated based on the following formula:
[0051] wherein, the autocorrelation matrix is, the batch training sample used when training for the historical classification task, the two-dimensional gradient matrix corresponding to the historical parameter tensor of the any network layer obtained after training based on the batch training sample.
[0052] Based on any of the above embodiments, the structure consolidation loss is calculated based on the autocorrelation matrix of the gradient information of the historical parameter tensor of each network layer and the parameter change tensor between the current parameter tensor and the historical parameter tensor of each network layer. The parameter change tensor between the current parameter tensor and the historical parameter tensor of any network layer is flattened in all dimensions except the first dimension, and then combined with the first dimension of the parameter change tensor between the current parameter tensor and the historical parameter tensor of the any network layer to form a two-dimensional parameter change matrix. The structure consolidation loss is calculated based on the Frobenius norm between the autocorrelation matrix of the gradient information of the historical parameter tensor of each network layer and the parameter change matrix.
[0053] Based on any of the above embodiments, the structure consolidation loss is calculated based on the following formula:
[0054] wherein, is the structure consolidation loss, is a regularization coefficient, is the autocorrelation matrix of the i-th network layer, is the parameter change matrix of the i-th network layer, denotes the Frobenius norm, and Tr denotes the trace of a matrix.
[0055] The continuous learning device provided by the embodiment of the present application solves the problems that the model in the continuous learning CL of the neuromorphic dataset is easy to forget old knowledge when learning a new task, is difficult to balance stability and plasticity at the same time, and has poor generalization ability in multiple scenarios and cross-architecture cases, and provides a parameter importance estimation and constraint method based on gradient statistical structure. By calculating the autocorrelation matrix of the gradient information, the consistency and task relevance of the parameter update direction of each structural unit in each network layer in the neuromorphic classification network are described, so as to obtain a fine-grained structural importance representation. When learning a new task, the continuous learning method uses the autocorrelation matrix to regularize and constrain the parameter update, suppresses the excessive adjustment of the sensitive direction to the historical task, and effectively avoids catastrophic forgetting. At the same time, the continuous learning method evaluates the importance of each structural unit in the historical classification task by describing the consistency and task relevance of the parameter update direction between each structural unit in the same network layer, thereby constraining the parameter update direction and avoiding adjusting the parameters of the direction sensitive to the historical classification task, achieving structural independence and scene compatibility, being able to adapt to different structures and different sizes of the pulse neural network, and effectively suppressing forgetting and realizing knowledge transfer in different scenarios.
[0056] The embodiment of the present application also provides a neuromorphic classification device, which comprises: An image acquisition unit is configured to acquire a current neuromorphic image to be classified; A neuromorphic image classification unit is configured to input the neuromorphic image to be classified into a neuromorphic classification network to obtain a classification result of the neuromorphic image to be classified output by the neuromorphic classification network. The neuromorphic classification network is trained based on the continuous learning method of the neuromorphic classification network provided in any of the above embodiments.
[0057] An electronic device is provided, which includes a computer readable storage medium and a processor. The computer readable storage medium is configured to store executable instructions. The processor is configured to read the executable instructions stored in the computer readable storage medium and execute the method provided in any of the above embodiments.
[0058] A computer readable storage medium is provided, which stores computer instructions, and the computer instructions are configured to cause a processor to execute the method provided in any of the above embodiments.
[0059] A computer program product is provided, which includes a computer program or instructions, and the computer program or instructions are executed by a processor to implement the method provided in any of the above embodiments.
[0060] Those skilled in the art can easily understand that the above description is only a preferred embodiment of the present application, and is not intended to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A method for continual learning of a neuromorphic classification network, the method comprising: The method comprises the following steps: When a new classification task arrives, obtain the historical parameter tensor of each network layer of a neuromorphic classification network after the neuromorphic classification network is trained for historical classification tasks and gradient information of the historical parameter tensor; wherein the neuromorphic classification network is constructed based on a spiking neural network; For any network layer in the neuromorphic classification network, calculate the autocorrelation matrix of the gradient information of the historical parameter tensor of the any network layer; wherein the autocorrelation matrix contains the correlation of each structural unit in the any network layer in the historical classification tasks; During the training process for the new classification task, obtain the current parameter tensor of each network layer in the neuromorphic classification network, calculate the parameter change tensor between the current parameter tensor and the historical parameter tensor of each network layer, calculate the structure consolidation loss based on the autocorrelation matrix of the gradient information of the historical parameter tensor of each network layer and the parameter change tensor between the current parameter tensor and the historical parameter tensor of each network layer, obtain the classification loss of the neuromorphic classification network based on the sample neuromorphic image and its label, and adjust the parameters of each network layer in the neuromorphic classification network based on the structure consolidation loss and the classification loss. 2.The method of Claim 1, wherein, The calculation of the autocorrelation matrix of the gradient information of the historical parameter tensor of the any network layer specifically comprises: After flattening all dimensions except the first dimension in the gradient information of the historical parameter tensor of the any network layer, combine the first dimension of the gradient information of the historical parameter tensor of the any network layer into a two-dimensional gradient matrix; Based on the two-dimensional gradient matrix and its transpose matrix, calculate the autocorrelation matrix of the gradient information of the historical parameter tensor of the any network layer. 3.The method of Claim 2, wherein, The autocorrelation matrix of the gradient information of the historical parameter tensor of the any network layer is calculated based on the following formula: wherein, is the auto-correlation matrix, is a batch training sample employed when training for a historical classification task, is the two-dimensional gradient matrix corresponding to the historical parameter tensor of the any network layer obtained after training based on the batch training sample. 4.The method of Claim 1, wherein, The calculation of the structure consolidation loss based on the autocorrelation matrix of the gradient information of the historical parameter tensor of each network layer and the parameter change tensor between the current parameter tensor and the historical parameter tensor of each network layer specifically comprises: After flattening all dimensions except the first dimension in the parameter change tensor between the current parameter tensor and the historical parameter tensor of the any network layer, combine the first dimension of the parameter change tensor between the current parameter tensor and the historical parameter tensor of the any network layer into a two-dimensional parameter change matrix; Based on the Frobenius norm between the autocorrelation matrix of the gradient information of the historical parameter tensor of each network layer and the parameter change matrix, calculate the structure consolidation loss. 5.The method of Claim 4, wherein, The structure consolidation loss is calculated based on the following formula: wherein, is the structure regularized loss, is a regularization coefficient, is the auto-correlation matrix of the i-th network layer, is the parameter change matrix of the i-th network layer, denotes the Frobenius norm and Tr denotes the trace of a matrix.
6. A device for continual learning of a neuromorphic classification network, characterized in that, The method comprises the following steps: A gradient information acquisition unit is configured to, when a new classification task arrives, obtain the historical parameter tensor of each network layer of a neuromorphic classification network after the neuromorphic classification network is trained for historical classification tasks and gradient information of the historical parameter tensor; wherein the neuromorphic classification network is constructed based on a spiking neural network; The correlation determination unit is configured to calculate, for any network layer in the neuromorphic classification network, an autocorrelation matrix of gradient information of a historical parameter tensor of the any network layer, wherein the autocorrelation matrix contains correlations of each structural unit in the any network layer in historical classification tasks. The continuous learning unit is configured to, in a training process for the new classification task, obtain a current parameter tensor of each network layer in the neuromorphic classification network, calculate a parameter change tensor between the current parameter tensor and a historical parameter tensor of each network layer, calculate a structure consolidation loss based on the autocorrelation matrix of the gradient information of the historical parameter tensor of each network layer and the parameter change tensor between the current parameter tensor and the historical parameter tensor of each network layer, obtain a classification loss of the neuromorphic classification network based on a sample neuromorphic image and a label thereof, and adjust parameters of each network layer in the neuromorphic classification network based on the structure consolidation loss and the classification loss.
7. A neuromorphic classification method, characterized in that, The method comprises: obtaining a current neuromorphic image to be classified; inputting the neuromorphic image to be classified into a neuromorphic classification network to obtain a classification result of the neuromorphic image to be classified output by the neuromorphic classification network; wherein the neuromorphic classification network is trained based on the continuous learning method of the neuromorphic classification network according to any one of claims 1 to 5.
8. A neuromorphic classification apparatus, characterized by, The method comprises: an image obtaining unit configured to obtain a current neuromorphic image to be classified; a neuromorphic image classification unit configured to input the neuromorphic image to be classified into a neuromorphic classification network to obtain a classification result of the neuromorphic image to be classified output by the neuromorphic classification network; wherein the neuromorphic classification network is trained based on the continuous learning method of the neuromorphic classification network according to any one of claims 1 to 5.
9. An electronic device, comprising: The method comprises: a computer readable storage medium and a processor; the computer readable storage medium is configured to store executable instructions; the processor is configured to read the executable instructions stored in the computer readable storage medium and execute the method according to any one of claims 1 to 5 and claim 7.
10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing the processor to execute the method according to any one of claims 1 to 5 and claim 7.