Rapid incremental learning technology based on pulse brain-like model

Abstract

The invention discloses a fast incremental learning technology based on a pulse brain-like model, and belongs to the technical field of computer vision. According to the invention, a Yolov3-tiny model is selected as a basic model of SpikingYolov3 of the spiking neural network, and the adaptability of the Yolov3-tiny model is improved; an IF neuron model is used as a neuron model of SpikingYolov3; carrying out incremental learning training on the Yolov3-tiny by adopting a method based on network expansion; the weight of the trained model is assigned to the newly constructed SpikingYolov3 through a method of converting ANN into SNN; according to the method, a test image is subjected to coding operation in a Poisson coding mode, so that the test image is converted into an asynchronous pulse sequence, the asynchronous pulse sequence is sent into a Spiking Yolov3 model to be tested and is subjected to comparative analysis with corresponding Yolov3-tiny, and a conclusion that the detection performance of the Spiking Yolov3 model is better is obtained. The SpikingYolov3 is applied to a battlefield environment, so that accurate detection of different types of targets is realized. The problem that a traditional network model needs to be retrained when encountering a new task is solved, and a new solution is provided for a detection task of a complex target.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and specifically relates to a fast incremental learning technology based on a pulse brain model. Background Art

[0002] With the continuous development of computer vision, the application of deep learning and neural networks has spread to all walks of life and achieved many landmark results. In this field, many excellent algorithm models have emerged, which have satisfactory performance in terms of detection speed, accuracy and lightweight. The common feature of these algorithm models is that before actual deployment, they will use a large amount of data to train the model in a targeted manner according to relevant work requirements so that the model can fully learn the characteristics of the target; then apply the trained model to the actual scene. However, when facing new learning tasks, they cannot use the knowledge they have learned to learn new tasks more efficiently. When learning new tasks, they are prone to "catastrophic forgetting", which makes it difficult to meet the uncertain battlefield environment and complex task requirements, and cannot effectively deal with emergencies in the battlefield environment. In response to similar situations, incremental learning technology came into being. The key to achieving incremental learning of models is to take into account the stability and plasticity of the model, and to use the learned knowledge to help the model learn new knowledge while retaining the knowledge that has been learned. In the actual environment, higher requirements are placed on the incremental learning ability of deep learning models. When the model needs to learn a new task, incremental learning only needs to use the data set of the new task for training, instead of starting training from scratch each time, which greatly reduces the learning cost of the old task data and brings huge economic benefits and practical value to systems that need to frequently update models.

[0003] When deploying the algorithm model to the actual device, one of the conditions that needs to be met is to make the model as lightweight as possible without affecting the performance of the model. Although the research on how to optimize the existing algorithm model to reduce the complexity and number of parameters of the model as much as possible has become a hot topic, and many landmark results have been achieved. However, after the model is lightweighted by the existing methods, the stability and plasticity of the model may be reduced, and the "catastrophic forgetting" when learning new tasks is more obvious. As the third generation of neural networks, the way of information processing of the spiking neural network is closer to the working method of the human brain. The spiking neural network achieves a higher level of biological neural simulation. In addition to the state of neurons and synapses, it also incorporates the concept of time into its operation. Compared with the existing traditional algorithm model, a significant advantage of the spiking neural network-based model is that it occupies less computer resources when processing the same information with little difference in performance. Therefore, the proposal of the spiking brain-like model solves the problem of lightweight algorithm models from the root, and provides a new solution for tasks such as small target detection in complex battlefield environments. . Summary of the invention

[0004] The purpose of the present invention is to provide a fast incremental learning technology based on a pulse brain model.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] A fast incremental learning technology based on a pulse brain model, specifically comprising the following steps:

[0007] Step 1: Use the Yolov3-tiny model as the basic model of SpikingYolov3 based on pulse neural network, and improve the Yolov3-tiny model;

[0008] Step 2: Select the IF neuron model as the neuron model of SpikingYolov3, so that the average postsynaptic potential of the IF neuron model matches the activation function value of the neuron in Yolov3-tiny, and realize the unbiased approximation of the activation function by the IF neuron model;

[0009] Step 3: In the model training stage, the network expansion-based method is used to perform incremental learning training on Yolov3-tiny;

[0010] Step 4: Assign the trained model weights to the newly constructed SpikingYolov3 through the ANN to SNN method;

[0011] Step 5: Use the new and old task test set samples to test the SpikingYolov3 model and the corresponding Yolov3-tiny respectively, obtain their respective target detection performance indicators, and compare and analyze the two to find that the SpikingYolov3 model has better detection performance;

[0012] Step 6: Use SpikingYolov3 to perform small target detection tasks. The model can accurately detect different types of targets and respond to emergencies through rapid incremental learning of small sample data.

[0013] Furthermore, the specific steps of step 1 are:

[0014] Step 1.1: Simplify the convolution operation and full connection operation of the lth layer of neurons in the target detection Yolov3-tiny model into a combination of linear operation and nonlinear activation function:

[0015] M l =f(W l M l-1 ), 1≤l≤L

[0016] Among them, the vector M l and vector M l-1 Respectively represent the outputs of all neurons in the lth layer and the l-1th layer in the ANN model, W l represents the connection weight between the neurons in the l-1th layer and the lth layer, and f(·) represents the nonlinear activation function;

[0017] Replace the Leaky ReLU activation function in Yolov3-tiny with a ReLU activation function that has a strong correlation with the pulse firing rate of the IF neuron in the SNN;

[0018] Step 1.2: Replace the maximum pooling layer with a stride of 2 in Yolov3-tiny with a convolutional layer with the same stride, and delete the convolutional layer with a stride of 1.

[0019] Step 1.3: Replace the upsampling module Sample in Yolov3-tiny with the deconvolution module DeConv;

[0020] Step 1.3: Cancel the batch normalization BN layer in Yolov3-tiny and merge the weight parameters of the BN layer with the convolution Conv layer.

[0021] Furthermore, the specific steps of step 2 are:

[0022] Step 2.1: The IF neuron dynamics equation is expressed by the following two formulas:

[0023]

[0024] in, represents the presynaptic membrane potential of the spiking neuron at the tth time step in the spiking neural network, V l (t) represents the postsynaptic membrane potential of the spiking neuron in the lth layer of the spiking neural network at the tth time step, x l-1 (t) indicates that the neurons in layer l receive input from layer l-1, r l (t) represents the pulse emitted by the neurons in layer l, V th represents the firing threshold of the neurons in layer l;

[0025] When the IF neurons in layer l receive weighted input signals and the postsynaptic membrane potential V l (t) When the excitation threshold is exceeded, a pulse signal with a value of 1 is emitted and the membrane potential is reset at the same time:

[0026]

[0027] Step 2.2: Use the average postsynaptic potential of the IF neuron model to approximate the activation function value in Yolov3-tiny, and combine the IF neuron dynamics equation in step 2.1 to obtain:

[0028] V l (t) = V l (t)-1)+W l x l-1 (t)-r l (t)V th

[0029] Accumulate the time steps of IF neurons from 1 to T:

[0030]

[0031] Where T represents the sum of the time steps of neuron simulation;

[0032] Step 2.3: The average postsynaptic potential of the IF neuron is:

[0033]

[0034] The recursive relationship between the average postsynaptic potential of IF neurons between any two adjacent layers in SNN is:

[0035]

[0036] When T approaches infinity, the pulse firing rate of the IF neuron model in the SNN is approximately the output of the ANN, and the error between the SNN and the ANN approaches zero.

[0037] Furthermore, the specific steps of step 3 are:

[0038] Step 3.1: When training Yolov3-tiny with the original task sample data, use L1 regularization to train the network so that the model produces sparse weights W t-1 , reducing the connections between neurons in each layer:

[0039]

[0040] Among them, l represents the lth layer of the network, W is the tensor, is the network parameter of layer l, is the loss function, D t is the training set; after training the original task samples, the loss threshold is set according to the training situation;

[0041] Step 3.2: When a sample type that has not been encountered before appears, first fit a sparse linear model to predict task t, using the top hidden unit of the neural network as follows:

[0042]

[0043] Get the connection between the output unit and the L-1 layer hidden unit, determine the units and weights in the model affected by the new task sample, fix the parts not connected to the output unit, and only train the selected sub-network S1, whose weight parameter is

[0044]

[0045] Step 3.3: Calculate the loss of the new model on the validation set after training and compare it with the loss threshold; if the new task is highly correlated with the old task, it is considered that the learned knowledge is sufficient to complete the new task, and selective retraining can meet the new task; if the loss value of the new model is higher than the loss threshold, it is considered that the characteristics of the new task samples are quite different from the characteristics of the samples that the model has learned, and network expansion is required; at this time, the weights in subnet S1 are restored to

[0046] Step 3.4: When the learned features cannot accurately express the new task, new neurons need to be introduced into the original network model; group sparse regularization is used to dynamically add neurons to adapt to the learning of different task features; when task t is input into the model, the lth layer of the model is added, the parameters of the lth layer at the previous moment are retained and N parameters are added to expand the weight matrix:

[0047]

[0048] Step 3.5: When training the new model, find a feasible region in the gradient space that does not affect the performance of the old task, so that the model can perform gradient updates in this region; at this time, the old task samples do not participate in the training of the new task, but constrain the direction of the model gradient descent:

[0049] sL task (θ t ;M t )≤L task (θ t-1 ;M t )

[0050] Among them, M t represents the set of old task samples, represents the gradient on the new task sample, Represents the gradient on the old task sample; when the angle between the gradient directions of the two is acute, use Applying gradient updates to the model will not affect the model's performance on old tasks.

[0051] Furthermore, the specific steps of step 4 are:

[0052] Step 4.1: Convert the trained Yolov3-tiny to SpikingYolov3. Each ANN unit corresponds to an SNN unit. Assuming that the ANN network model has L layers, the value of the continuous neuron i in the lth (l∈{1,2,…,L}) layer after passing through the activation function layer is:

[0053]

[0054] Among them, W l represents the weight matrix between two adjacent layers, b l is bias;

[0055] Step 4.2: Let the input after weight normalization be x, then a 0 = x; adopt Represents the membrane potential of the i-th SNN neuron in the l-th layer of the SNN corresponding to the ANN at time t, then the input potential at each time step is:

[0056]

[0057] Among them, V thr is the threshold potential of the spiking neuron, It is used to indicate whether a pulse is generated at time t, which is expressed as:

[0058]

[0059] Step 4.3: Let the time window be T, and the size of each time step be Δt∈R + , then when the spiking neuron fires a pulse at every time step, the pulse firing rate reaches a maximum of The firing rate of the i-th neuron in the l-th layer of the SNN is:

[0060]

[0061] That is, the spike firing rate of the spiking neurons in the SNN and the activation value of the ANN Positive correlation.

[0062] Furthermore, the specific steps of step 5 are:

[0063] Step 5.1: Before the test set enters the model, the test image is encoded using Poisson coding to convert it into an asynchronous pulse sequence; the input image is encoded into a pulse emission form that conforms to the Poisson distribution:

[0064]

[0065] Among them, k is the number of excitation pulses, and λ is the excitation frequency proportional to the pixel value; each pixel in the image is converted into an independent Poisson code, and the total number of time steps is set to T. In each time step, the pixel value of the unit pixel in the image is used as the basis for generating pulses. The larger the pixel value, the higher the probability of generating a pulse; thus, the pulse sequence corresponding to each image is obtained, and finally the pulse sequence is sent to the input layer of the SNN;

[0066] Step 5.2: After the pulse sequence is input into the SNN model, a module consisting of convolution batch normalization and IF neurons is used to extract feature information of the pulse sequence, and the small-size features are upsampled through the deconvolution module DeConv to achieve fusion of multi-scale features; finally, the target is detected through two detection heads of different scales;

[0067] Step 5.3: Use precision P, recall R and average precision mAP as the performance evaluation indicators of the Spiking Yolov3 model. The calculation formulas of these indicators are:

[0068]

[0069]

[0070] Among them, T P Indicates the number of positive samples predicted by the model as positive samples, F P Indicates the number of negative samples predicted by the model as positive samples, F NIt represents the number of positive samples predicted by the model as negative samples; P(R) represents the curve of precision and recall; i represents the i-th category in the data sample, and n represents the total number of categories in the data sample.

[0071] Furthermore, the SpikingYolov3 model is easily affected by the time step T. An ablation experiment is conducted on the time step T to test the detection effect of the model on the test set when T=64, 128, 256, 512, 1024 and 2048 respectively, and finally the optimal time step value is determined; and compared with the corresponding Yolov3-tiny model.

[0072] A computer device / equipment / system comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of a rapid incremental learning technology based on a pulse brain model.

[0073] A computer-readable storage medium stores a computer program / instruction, which, when executed by a processor, implements the steps of a rapid incremental learning technology based on a pulse brain-like model.

[0074] A computer program product includes a computer program / instruction, which, when executed by a processor, implements the steps of a fast incremental learning technology based on a pulse brain model.

[0075] The beneficial effects of the present invention are:

[0076] The invention discloses a fast incremental learning technology based on a pulse brain model, which applies incremental learning to a pulse neural network, so as to achieve a model that can learn new tasks well and avoid "catastrophic forgetting" while keeping the model lightweight. Since pulses are non-differentiable, the traditional back propagation algorithm cannot be applied to the training of pulse brain models. Therefore, the invention adopts the idea of ​​ANN to SNN, selects a relatively lightweight Yolov3-tiny network model, and makes corresponding modifications to the activation function, maximum pooling layer and other links therein to reduce the precision error after conversion. According to the input and output characteristics of the activation function in Yolov3-tiny, the IF neuron model unbiased approximate activation function is selected. When the model learns a new task, it first determines whether the performance of the existing model on the new task is qualified. If it is unqualified, the network is expanded, and the expanded model is trained by the gradient projection method. Finally, the trained model is converted into a pulse brain model. The training method of this model avoids directly training the SNN, and takes into account the precision error of ANN to SNN, so that the model has good plasticity and generalization.

[0077] The present invention can use the learned knowledge to learn new tasks, which solves the problem that traditional network models need to be retrained when encountering new tasks, greatly saves computing costs, and provides a new solution for complex target detection tasks in battlefield environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 This is an algorithm flow chart of a rapid incremental learning technology based on a pulse brain model according to the present invention;

[0079] Figure 2 is a structural diagram of the IF neuron model of the present invention;

[0080] Figure 3 This is the structural diagram of the converted SpikingYolov3 model;

[0081] Figure 4 This is a schematic diagram of the network expansion used in the model training stage according to the present invention. DETAILED DESCRIPTION

[0082] The present invention is further described below in conjunction with the accompanying drawings.

[0083] according to Figure 1 , a fast incremental learning technology based on a spiking brain model, comprising the following steps:

[0084] S1. The Yolov3-tiny model is used as the basic model of SpikingYolov3 based on pulse neural network, and the Yolov3-tiny model is improved accordingly according to the inevitable accuracy loss in the process of converting Yolov3-tiny into Spiking Yolov3.

[0085] Furthermore, the specific implementation method of step S1 includes the following steps:

[0086] S1.1. Simplify the convolution operation and full connection operation of the lth layer of neurons in the target detection Yolov3-tiny model into a combination of linear operation and nonlinear activation function:

[0087] M l =f(W l M l-1 ), 1≤l≤L

[0088] Among them, the vector M l and vector M l-1 Respectively represent the outputs of all neurons in the lth layer and the l-1th layer in the ANN model, W l represents the connection weight between the neurons in the l-1th layer and the lth layer, and f(·) represents the nonlinear activation function.

[0089] Replace the Leaky ReLU activation function in Yolov3-tiny with a ReLU activation function that has a strong correlation with the pulse firing rate of the IF neuron in the SNN;

[0090] Furthermore, since negative data will be lost during the conversion of ANN to SNN, the accuracy of the converted SNN will decrease. Since the pulse emission frequency and input-output characteristics of IF neurons are highly similar to the ReLU activation function and negative data will not appear, the use of ReLU activation function can improve the accuracy of the converted SNN.

[0091] S1.2. Replace the maximum pooling layer with a stride of 2 in Yolov3-tiny with a convolutional layer with the same stride, and delete the convolutional layer with a stride of 1.

[0092] S1.3. Replace the upsampling module Sample in Yolov3-tiny with the deconvolution module DeConv;

[0093] Furthermore, since there are no learnable parameters in the upsampling module, there is a problem that the model units cannot correspond one to one when converted to SNN. The present invention replaces the upsampling module with a deconvolution module that has a similar structure and function to the upsampling module, so that the accuracy error of the model conversion from ANN to SNN is smaller.

[0094] S1.4. Cancel the batch normalization (BN) layer in Yolov3-tiny and merge the weight parameters of the BN layer with the convolution Conv layer.

[0095] S2. Select the IF neuron model as the neuron model of SpikingYolov3, so that the average postsynaptic potential of the IF neuron model matches the activation function value of the neuron in Yolov3-tiny, and realize the unbiased approximation of the activation function by the IF neuron model;

[0096] Furthermore, the specific implementation method of step S2 includes the following steps:

[0097] S2.1 Schematic diagram of the IF neuron Figure 2 As shown, its kinetic equation is expressed by the following two formulas:

[0098]

[0099] in, represents the presynaptic membrane potential of the spiking neuron at the tth time step in the spiking neural network, V l (t) represents the postsynaptic membrane potential of the spiking neuron in the lth layer of the spiking neural network at the tth time step, xl-1 (t) indicates that the neurons in layer l receive input from layer l-1, r l (t) represents the pulse emitted by the neurons in layer l, V th Represents the firing threshold of the neurons in layer l.

[0100] When the IF neurons in layer l receive weighted input signals and the postsynaptic membrane potential V l (t) When the excitation threshold is exceeded, a pulse signal with a value of 1 is emitted and the membrane potential is reset at the same time:

[0101]

[0102] S2.2 uses the average postsynaptic potential of the IF neuron model to approximate the activation function value in Yolov3-tiny, and combines the IF neuron dynamics equation in S2.1 to obtain:

[0103] V l (t) = V l (t-1)+W l x l-1 (t)-r l (t)V th

[0104] Accumulate the time steps of IF neurons from 1 to T:

[0105]

[0106] Where T represents the total number of time steps of neuron simulation.

[0107] S2.3 The average postsynaptic potential of IF neurons is:

[0108]

[0109] The recursive relationship between the average postsynaptic potential of IF neurons between any two adjacent layers in SNN is:

[0110]

[0111] When T approaches infinity, the pulse firing rate of the IF neuron model in the SNN is approximately the output of the ANN, and the error between the SNN and the ANN approaches zero. The converted SpikingYolov3 model structure is as follows: Figure 3 shown.

[0112] S3. The pulse neural network model has the remarkable characteristic of being lightweight. Therefore, in the training stage of the model, the network expansion method is used to perform incremental learning training on Yolov3-tiny. The principle structure diagram of network expansion is shown in the figure. Figure 4 shown.

[0113] Furthermore, the specific implementation method of step S3 includes the following steps:

[0114] S3.1 When training Yolov3-tiny using the original task sample data, the network is trained using L1 regularization to make the model generate sparse weights W t-1 , reducing the connections between neurons in each layer:

[0115]

[0116] Among them, l represents the lth layer of the network, W is the tensor, is the network parameter of layer l, is the loss function, D t is the training set. After training the original task samples, the loss threshold is set according to the training situation.

[0117] S3.2 When a new task (a sample type that has not been encountered before) appears, first fit a sparse linear model to predict task t, using the top hidden unit of the neural network as follows:

[0118]

[0119] Get the connection between the output unit and the L-1 layer hidden unit, determine the units and weights in the model affected by the new task sample, fix the parts not connected to the output unit, and only train the selected sub-network S1, whose weight parameter is

[0120]

[0121] S3.3 calculates the loss of the new model on the validation set after training and compares it with the loss threshold. If the new task is highly related to the old task, or the learned knowledge is sufficient to complete the new task, then only selective retraining can meet the new task. If the loss value of the new model is higher than the loss threshold, it is considered that the characteristics of the new task sample are very different from the characteristics of the sample that the model has learned, and the existing network model is not sufficient to support the learning of the current task, and the network needs to be expanded. At this time, the weights in subnet S1 are restored to

[0122] S3.4 When the learned features cannot accurately express the new task, it is necessary to introduce new neurons into the original network model in order to better learn the features of the new task. Since the position and number of neurons added for different tasks are not necessarily the same, group sparse regularization is used to dynamically add neurons to adapt to the learning of different task features. When task t is input into the model, the lth layer of the model is added, the parameters of the lth layer at the previous moment are retained, and N parameters are added to expand the weight matrix:

[0123]

[0124] S3.5 When training a new model, find a feasible region in the gradient space that does not affect the performance of the old task, so that the model can perform gradient updates in this region. At this time, the old task samples do not participate in the training of the new task, but constrain the direction of the model gradient descent:

[0125] sL task (t; M t )≤L task (t -1 ;M t )

[0126] Among them, M t represents the set of old task samples, represents the gradient on the new task sample, Represents the gradient on the old task sample. When the angle between the gradient directions of the two is acute, use Applying gradient updates to the model will not affect the model's performance on old tasks.

[0127] S4. Assign the trained model weights to the newly constructed SpikingYolov3 through the ANN to SNN method.

[0128] Furthermore, the specific implementation method of step S4 includes the following steps:

[0129] S4.1 converts the trained ANN (Yolov3-tiny) into SNN (SpikingYolov3), and each ANN unit corresponds to an SNN unit. Assuming that the ANN network model has L layers, the value of the continuous neuron i in the lth (l∈{1,2,…,L}) layer after passing through the activation function layer is:

[0130]

[0131] Among them, W l represents the weight matrix between two adjacent layers, b l For bias.

[0132] S4.2 Let the weighted normalized input be x, then a 0 = x. Represents the membrane potential of the i-th SNN neuron in the l-th layer of the SNN corresponding to the ANN at time t, then the input potential at each time step is:

[0133]

[0134] Among them, Vthr is the threshold potential of the spiking neuron, It is used to indicate whether a pulse is generated at time t, which can be expressed as:

[0135]

[0136] S4.3 Let the time window be T, and the size of each time step be Δt∈R + , then when the spiking neuron fires a pulse at every time step, the pulse firing rate reaches a maximum of The firing rate of the i-th neuron in the l-th layer of the SNN is:

[0137]

[0138] That is, the spike firing rate of the spiking neurons in the SNN and the activation value of the ANN Positive correlation.

[0139] S5. Use the new and old task test set samples to test the SpikingYolov3 model and the corresponding Yolov3-tiny respectively, obtain their respective target detection performance indicators, and compare and analyze the two, and conclude that the Spiking Yolov3 model has better detection performance.

[0140] Furthermore, the specific implementation method of step S5 includes the following steps:

[0141] Before the S5.1 test set enters the model, the test image is encoded using Poisson coding to convert it into an asynchronous pulse sequence. The input image is encoded into a pulse emission form that conforms to the Poisson distribution:

[0142]

[0143] Among them, k is the number of excitation pulses, and λ is the excitation frequency proportional to the pixel value. Each pixel in the image is converted into an independent Poisson code, and the total number of time steps is set to T. In each time step, the pixel value of the unit pixel in the image is used as the basis for generating pulses. The larger the pixel value, the higher the probability of generating a pulse. Therefore, the pulse sequence corresponding to each image is obtained, and finally the pulse sequence is sent to the input layer of the SNN.

[0144] S5.2 Since Yolov3-tiny has been adapted to the characteristics of the spiking neural network before being converted to SNN, and the focus of the conversion process is to match the average output of the IF neuron with the activation value of the ReLU activation function. The Poisson encoding method enables SNN to satisfy the average input that is approximately equal to the image input in the ANN, so SpikingYolov3 and the corresponding Yolov3-tiny process most of the information of the pulse sequence and the input image in a similar way. After the pulse sequence is input into the SNN model, a module composed of convolution (batch normalization) and IF neurons is used to extract feature information from the pulse sequence, and the small-size features are upsampled by the deconvolution module DeConv, thereby realizing the fusion of multi-scale features. Finally, the target is detected by two detection heads of different scales.

[0145] S5.3 uses accuracy P (Precision), recall R (Recall) and average precision mAP (MeanAveragePrecision) as the performance evaluation indicators of the SpikingYolov3 model. The calculation formulas of these indicators are:

[0146]

[0147] Among them, T P Indicates the number of positive samples predicted by the model as positive samples, F P Indicates the number of negative samples predicted by the model as positive samples, F N It represents the number of positive samples predicted by the model as negative samples; P(R) represents the curve of precision and recall; i represents the i-th category in the data sample, and n represents the total number of categories in the data sample.

[0148] The performance of the Spiking Yolov3 model is easily affected by the time step T. We conducted an ablation experiment on the time step T, and tested the detection effect of the model on the test set when T = 64, 128, 256, 512, 1024 and 2048, and finally determined the optimal time step value. By comparing it with the corresponding Yolov3-tiny model, we concluded that the SpikingYolov3 model is superior to the corresponding Yolov3-tiny model in terms of computational complexity, detection accuracy, model size, etc.

[0149] S5.4 applies Spiking Yolov3 to the detection of small targets in uncertain environments. The model achieves accurate detection of different types of targets in the environment through rapid incremental learning of small sample data, thereby assisting in effective response to emergencies.

[0150] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A fast incremental learning technology based on a spiking brain model, characterized by: The specific steps include: Step 1: Use the Yolov3-tiny model as the basic model of SpikingYolov3 based on pulse neural network, and improve the Yolov3-tiny model; Step 2: Select the IF neuron model as the neuron model of SpikingYolov3, so that the average postsynaptic potential of the IF neuron model matches the activation function value of the neuron in Yolov3-tiny, and realize the unbiased approximation of the activation function by the IF neuron model; Step 3: In the model training stage, the network expansion-based method is used to perform incremental learning training on Yolov3-tiny; Step 4: Assign the trained model weights to the newly constructed SpikingYolov3 through the ANN to SNN method; Step 5: Use the new and old task test set samples to test the SpikingYolov3 model and the corresponding Yolov3-tiny respectively, obtain their respective target detection performance indicators, and compare and analyze the two to find that the SpikingYolov3 model has better detection performance; Step 6: Use SpikingYolov3 to perform small target detection tasks. The model can accurately detect different types of targets and respond to emergencies through rapid incremental learning of small sample data.

2. The rapid incremental learning technology based on the pulse brain model according to claim 1, characterized in that: The specific steps of step 1 are: Step 1.1: Simplify the convolution operation and full connection operation of the lth layer of neurons in the target detection Yolov3-tiny model into a combination of linear operation and nonlinear activation function: M l =f(W l M l-1 ),1≤l≤L Among them, the vector M l and vector M l-1 Respectively represent the outputs of all neurons in the lth layer and the l-1th layer in the ANN model, W l represents the connection weight between the neurons in the l-1th layer and the lth layer, and f(·) represents the nonlinear activation function; Replace the Leaky ReLU activation function in Yolov3-tiny with a ReLU activation function that has a strong correlation with the pulse firing rate of the IF neuron in the SNN; Step 1.2: Replace the maximum pooling layer with a stride of 2 in Yolov3-tiny with a convolutional layer with the same stride, and delete the convolutional layer with a stride of 1. Step 1.3: Replace the upsampling module Sample in Yolov3-tiny with the deconvolution module DeConv; Step 1.3: Cancel the batch normalization BN layer in Yolov3-tiny and merge the weight parameters of the BN layer with the convolution Conv layer.

3. The rapid incremental learning technology based on the pulse brain model according to claim 1, characterized in that: The specific steps of step 2 are: Step 2.1: According to the IF neuron dynamics equation, when the IF neuron in layer l receives the weighted input signal and the postsynaptic membrane potential V l (t) When the excitation threshold is exceeded, a pulse signal with a value of 1 is emitted and the membrane potential is reset at the same time: in, represents the presynaptic membrane potential of the spiking neuron at the tth time step in the spiking neural network, V l (t) represents the postsynaptic membrane potential of the spiking neuron in the lth layer of the spiking neural network at the tth time step, r l (t) represents the pulse emitted by the neurons in layer l, V th represents the firing threshold of the neurons in layer l; Step 2.2: Use the average postsynaptic potential of the IF neuron model to approximate the activation function value in Yolov3-tiny. According to the IF neuron dynamics equation in step 2.1, we get: V l (t)=V l (t-1)+W l x l-1 (t)-r l (t)V th Accumulate the time steps of IF neurons from 1 to T: Where T represents the sum of the time steps of neuron simulation; Step 2.3: The average postsynaptic potential of the IF neuron is: The recursive relationship between the average postsynaptic potential of IF neurons between any two adjacent layers in SNN is: When T approaches infinity, the pulse firing rate of the IF neuron model in the SNN is approximately the output of the ANN, and the error between the SNN and the ANN approaches zero.

4. The rapid incremental learning technology based on the pulse brain model according to claim 1, characterized in that: The specific steps of step 3 are: Step 3.1: When training Yolov3-tiny with the original task sample data, use L1 regularization to train the network so that the model produces sparse weights W t-1 , reducing the connections between neurons in each layer: Among them, l represents the lth layer of the network, W is the tensor, is the network parameter of layer l, is the loss function, D t is the training set; after training the original task samples, the loss threshold is set according to the training situation; Step 3.2: When a sample type that has not been encountered before appears, first fit a sparse linear model to predict task t, using the top hidden unit of the neural network as follows: Get the connection between the output unit and the L-1 layer hidden unit, determine the units and weights in the model affected by the new task sample, fix the parts not connected to the output unit, and only train the selected sub-network S1, whose weight parameter is Step 3.3: Calculate the loss of the new model on the validation set after training and compare it with the loss threshold; if the new task is highly correlated with the old task, it is considered that the learned knowledge is sufficient to complete the new task, and selective retraining can meet the new task; if the loss value of the new model is higher than the loss threshold, it is considered that the characteristics of the new task samples are quite different from the characteristics of the samples that the model has learned, and network expansion is required; at this time, the weights in subnet S1 are restored to Step 3.4: When the learned features cannot accurately express the new task, new neurons need to be introduced into the original network model; group sparse regularization is used to dynamically add neurons to adapt to the learning of different task features; when task t is input into the model, the lth layer of the model is added, the parameters of the lth layer at the previous moment are retained and N parameters are added to expand the weight matrix: Step 3.5: When training the new model, find a feasible region in the gradient space that does not affect the performance of the old task, so that the model can perform gradient updates in this region; at this time, the old task samples do not participate in the training of the new task, but constrain the direction of the model gradient descent: Among them, M t represents the set of old task samples, represents the gradient on the new task sample, Represents the gradient on the old task sample; when the angle between the gradient directions of the two is acute, use Applying gradient updates to the model will not affect the model's performance on old tasks.

5. The rapid incremental learning technology based on the pulse brain model according to claim 1, characterized in that: The specific steps of step 4 are: Step 4.1: Convert the trained Yolov3-tiny to SpikingYolov3. Each ANN unit corresponds to an SNN unit. Assuming that the ANN network model has L layers, the value of the continuous neuron i in the lth (l∈{1,2,…,L}) layer after passing through the activation function layer is: Among them, W l represents the weight matrix between two adjacent layers, b l is bias; Step 4.2: Let the input after weight normalization be x, then a 0 = x; adopt Represents the membrane potential of the i-th SNN neuron in the l-th layer of the SNN corresponding to the ANN at time t, then the input potential at each time step is: Among them, V thr is the threshold potential of the spiking neuron, It is used to indicate whether a pulse is generated at time t, which is expressed as: Step 4.3: Let the time window be T, and the size of each time step be Δt∈R + , then when the spiking neuron fires a pulse at every time step, the pulse firing rate reaches a maximum of The firing rate of the i-th neuron in the l-th layer of the SNN is: That is, the spike firing rate of the spiking neurons in the SNN and the activation value of the ANN Positive correlation.

6. The rapid incremental learning technology based on the pulse brain model according to claim 1, characterized in that: The specific steps of step 5 are: Step 5.1: Before the test set enters the model, the test image is encoded using Poisson coding to convert it into an asynchronous pulse sequence; the input image is encoded into a pulse emission form that conforms to the Poisson distribution; each pixel in the image is converted into an independent Poisson code, and the total number of time steps is set to T. In each time step, the pixel value of the unit pixel in the image is used as the basis for generating pulses. The larger the pixel value, the higher the probability of generating a pulse. Therefore, the pulse sequence corresponding to each picture is obtained, and finally the pulse sequence is sent to the input layer of SNN; Step 5.2: After the pulse sequence is input into the SNN model, a module consisting of convolution batch normalization and IF neurons is used to extract feature information of the pulse sequence, and the small-size features are upsampled through the deconvolution module DeConv to achieve fusion of multi-scale features; finally, the target is detected through two detection heads of different scales; Step 5.3: Use precision P, recall R and average precision mAP as the performance evaluation indicators of the Spiking Yolov3 model. The calculation formulas of these indicators are: Among them, T P Indicates the number of positive samples predicted by the model as positive samples, F P Indicates the number of negative samples predicted by the model as positive samples, F N It represents the number of positive samples predicted by the model as negative samples; P(R) represents the curve of precision and recall; i represents the i-th category in the data sample, and n represents the total number of categories in the data sample.

7. The rapid incremental learning technology based on the pulse brain model according to claim 6 is characterized by: The Spiking Yolov3 model is easily affected by the time step T. An ablation experiment is conducted on the time step T. The detection effect of the model on the test set is tested when T=64, 128, 256, 512, 1024 and 2048 respectively, and the optimal time step value is finally determined; and compared with the corresponding Yolov3-tiny model.

8. A computer device / equipment / system comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images