Neural network continuous learning online evaluation method based on multilayer feature difference

Abstract

The invention relates to a neural network continuous learning online evaluation method based on multilayer feature difference. Firstly, network parameters of each layer of a tested neural network in a continuous learning process are extracted, wherein the network parameters comprise a weight matrix and an offset vector; secondly, extracting network parameter similarity for each layer based on weights of adjacent learning tasks; then, a long-short memory recurrent neural model is constructed to serve as an evaluation model, the layer depth of the tested network model serves as time sequence data to train the recurrent network evaluation model, the input of the evaluation model is the parameter similarity of the tested neural network of each layer, and the final output of the evaluation model is whether the tested neural network is currently forgotten or not; and finally, evaluating the continuous learning state of the intelligent neural network model by using the trained evaluation model, extracting a parameter similarity parameter between a current state and a previous state, and inputting the parameter similarity parameter into a circulating network evaluation model to identify and evaluate the current continuous learning state.

Description

Technical Field

[0001] The invention relates to the field of testing, and in particular to an online evaluation method for continuous learning of a neural network. Background Art

[0002] Continuous learning is a research direction in machine learning. Its goal is to enable the model to retain and utilize previously learned knowledge while continuously learning new tasks or new knowledge without forgetting. The difference from traditional machine learning is that continuous learning focuses on training models on a dynamic data set, and needs to consider the continuity and change of data. As time goes by, new samples in the task sequence gradually arrive, and the knowledge learned by the model is constantly increasing. The model capabilities cannot interfere with each other, and it must not lead to catastrophic forgetting of information. The general process of continuous learning is to first learn a part of the data, acquire knowledge, and then train the next part of the data to obtain "new knowledge". The focus is on learning new knowledge better when the knowledge has been well acquired. Generally speaking, in the process of learning new knowledge, there are only previously trained network models and network weights. The new knowledge to be acquired may be related or irrelevant to the trained knowledge. When new knowledge is added to the original learning system, the system's performance on previously learned tasks will drop sharply, which is called forgetting.

[0003] The requirement for artificial intelligence algorithms is to be able to continuously learn new knowledge and improve their capabilities during use, so it is very important to evaluate the continuous learning of an intelligent algorithm / system. Continuous learning represents the "potential" of the algorithm and can reflect the trend of the intelligent algorithm's ability to grow in the future. Intelligent algorithms can generally achieve good results on new tasks after training with new data, so the forgetting rate indicator is often used to evaluate the continuous learning of an intelligent algorithm.

[0004] The forgetting rate indicator reflects the degree to which the model forgets the old tasks after learning new tasks. It is usually calculated by comparing the accuracy of each task when the training is completed with the accuracy at the final evaluation. This method requires the storage of a large amount of past task data for judgment. The computational complexity increases linearly with the continuous learning process. It is not suitable for the continuous learning ability evaluation of long-term intelligent algorithms in systems such as unmanned driving, intelligent robots, and smart cities. Summary of the invention

[0005] In view of this, the present invention proposes an online evaluation method for the continuous learning of a neural network based on multi-layer feature differences, which realizes an effective evaluation of the continuous learning state by evaluating the similarity of network parameters. Based on the theory of neural network parameter knowledge representation, the present invention evaluates the current continuous learning state of the neural network through multi-layer feature differences, effectively reducing the computational complexity and improving the efficiency of continuous learning state evaluation.

[0006] The intelligence of neural networks emerges from the network structure and weight parameters. If the structure does not change during training, then knowledge can be equated with the distribution of parameters. Therefore, the forgetting phenomenon can be assumed to be a huge change in the distribution of network parameters.

[0007] Based on the above important assumptions, the present invention first extracts the network parameters of each layer of the tested neural network during the continuous learning process, and the network parameters include a weight matrix and a bias vector; then, based on the weights of adjacent learning tasks, the network parameter similarity is extracted for each layer, and the similarity vector includes the cosine similarity, Euclidean distance, Manhattan distance, Chebyshev distance, Mahalanobis distance, Pearson correlation coefficient, Jaccard similarity coefficient, and KL divergence of the network parameters, totaling 8-dimensional similarity vectors; then, a long-short memory recurrent neural model is constructed as an evaluation model, and the recurrent network evaluation model is trained with the layer depth of the tested network model as time series data. The input of the evaluation model is the parameter similarity of each layer of the tested neural network, and the final output is whether the tested neural network is currently forgetting; finally, the trained evaluation model is used to evaluate the continuous learning state of the intelligent neural network model, and the parameter similarity parameters between the current state and the previous state are extracted and input into the recurrent network evaluation model to identify and evaluate the current continuous learning state.

[0008] Beneficial Effects

[0009] 1. The present invention can evaluate the continuous learning ability of neural network intelligent algorithms online, that is, the evaluation of the forgetting phenomenon of the tested neural network.

[0010] 2. The recognition and evaluation of the continuous learning state of the fully connected neural network in the present invention no longer relies on the performance calculation of the test data set, but is based on the similarity of network parameters, which effectively solves the problem of excessive data storage space caused by the continuous learning process of the traditional test set method.

[0011] 3. The present invention effectively reduces the computational complexity and improves the test efficiency of continuous learning state evaluation through feature modeling and encoding, continuous learning evaluation model design, and evaluation using the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 Multi-level feature difference representation diagram

[0013] Figure 2 Implementation Flowchart DETAILED DESCRIPTION

[0014] The first step is data collection and annotation: The neural network under test has a set of initial weights and supporting data sets. Based on the initial weights of the object under test and the supporting data sets, the original data of the continuous learning evaluation model is collected. The sample is the parameter similarity of each training task, and the annotation is the result value of the forgetting rate.

[0015] Data collection is to split the data sets of multiple tasks into training sets and test sets, use the training sets to train the neural network under test in random order, and calculate the parameter similarity of each layer of the neural network under test based on the network parameters of each round of tasks and the network parameters of the previous round of tasks, and save them as data sets.

[0016] The data annotation of the present invention is automatically generated. After each round of task training, the test sets of all previously trained tasks are tested. Continuous learning is a sequential task. Each training task before this has a corresponding test data set. The minimum value of the results on all test sets is recorded as the forgetting rate result. If the current forgetting rate decreases by more than a threshold value θ compared with the previous round of forgetting rate, it means that the model under test has forgotten. The category of the parameter similarity of this round is recorded as 1, that is, the model under test has forgotten. Otherwise, the category is recorded as 0.

[0017] The second step is to calculate the network parameters p of each layer of the neural network under test during the continuous learning process. i,j Extraction, where p i,j Represents the parameters of the j-th layer network in the i-th continuous learning task, and the network parameters include the weight matrix ω i,j and the bias vector b i,j , where ω i,j and b i,j The dimension of is determined by the number of network neurons. If the number of network neurons in the jth layer is m, and if the number of network neurons in the j-1th layer is n, then ω i,j is an m×n dimensional matrix, b i,j is an m×1 dimensional matrix. At this time, p i,j It is a matrix of m×(n+1) dimensions as shown below.

[0018] p i,j =[ω i,j b i,j ]

[0019] The second step is to extract the similarity of network parameters for each layer based on the weights of adjacent learning tasks. Neural network theory believes that network parameters are the main way to represent knowledge, and parameters are mainly used for learning and training during the training process. Therefore, the task of the present invention can identify and evaluate the learning state through the similarity of network parameters during continuous learning. The present invention proposes that parameter similarity includes cosine similarity of network parameters, Euclidean distance, Manhattan distance, Chebyshev distance, Mahalanobis distance, Pearson correlation coefficient, Jaccard similarity coefficient, and relative entropy, totaling 8-dimensional similarity vectors. For the i-th continuous learning task, its similarity vector x i,j is the weight parameter p of the current task i,j and the previous continuous learning task p i-1,j To obtain, in the process of calculation Represents the parameter value at position (k, l) in the parameter matrix of the j-th layer network in the i-th continuous learning task.

[0020] The first dimension network parameter similarity metric parameter Is the cosine similarity, which evaluates the similarity between two vectors by calculating the cosine of the angle between them, as shown in the following formula.

[0021]

[0022] The second dimension network parameter similarity measurement parameter is the Euclidean distance, which is the straight-line distance between two points in Euclidean space, as shown below.

[0023]

[0024] The third dimension network parameter similarity measurement parameter It is the Manhattan distance, which is the sum of the distances of the projections of the line segment formed by two points on the axis in the fixed rectangular coordinate system of Euclidean space, as shown in the following formula.

[0025]

[0026] The fourth dimension network parameter similarity measurement parameter is the Chebyshev distance, which is the maximum absolute value of the difference between its coordinate values, as shown in the following formula.

[0027]

[0028] Fifth dimension network parameter similarity measurement parameter is the Mahalanobis distance, which evaluates the similarity between two vectors by calculating the cosine of the angle between them, as shown in the following formula, where Σ′ represents the neural network parameter matrix p i,j ,p i-1,j The covariance matrix of .

[0029]

[0030] The sixth dimension network parameter similarity measurement parameter is the Pearson correlation coefficient, which measures the degree of linear correlation between the variables of two sets of data. It is the ratio of the covariance of the two variables to the product of their standard deviations, as shown in the following formula.

[0031]

[0032] in Represents the mean and variance of the k-th row vector in the parameter matrix of the j-th layer network in the i-th continuous learning task.

[0033] The seventh dimension network parameter similarity measurement parameter is the generalized Jaccard similarity coefficient, which compares the similarities and differences between finite sample sets, as shown in the following formula.

[0034]

[0035] The eighth dimension network parameter similarity measurement parameter is the relative entropy, which is a measure of the asymmetry between two probability distributions, as shown below.

[0036]

[0037] Based on the above network parameter similarity measurement parameters, a network parameter measurement similarity vector x is constructed. i,j , based on the multi-layer network similarity vector x i,j The continuous learning status of the network model can be effectively identified and evaluated.

[0038] The fourth step is to build the model. Since the definition of the network's forgetting phenomenon focuses on the final output task, but is also related to the parameter changes of the previous intermediate network layer, it is necessary to identify the continuous learning state through a recurrent network model. The recurrent network model is trained with the layer depth d of the tested network model as the time series data. The input of the evaluation model is the parameter similarity of the tested network model at each layer, and the final output is whether the current tested network model has forgotten.

[0039] The present invention adopts the long short-term memory recurrent neural model to achieve this. It is a recurrent neural network that can effectively avoid the gradient vanishing and gradient exploding problems when processing sequence data. It maintains long-term dependencies by introducing three gates: input gate, forget gate, and output gate, as well as an internal memory unit called "state". The specific calculation units are as follows:

[0040] ① Input gate: determines whether to store new features in the state. The calculation formula is as follows.

[0041] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0042]

[0043] Among them, W i is the weight matrix of the input gate; b i is the bias of the input gate; W c is the weight parameter matrix of the state; b c is the bias parameter of the state; tanh is the hyperbolic tangent activation function

[0044] ②Forget gate: decides to discard relevant long-term information from the state. The calculation formula is as follows.

[0045] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0046] Among them, h t-1 The hidden state of the previous time step; W f is the weight parameter matrix of the forget gate; b f is the bias parameter of the forget gate; x t is the input of the current time step; v is the sigmoid activation function.

[0047] ③Output gate: determines what information is output from the state to the hidden state. The calculation formula is as follows.

[0048] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0049] Among them, W o is the weight parameter matrix of the output gate; b o is the bias parameter of the output gate.

[0050] ④State unit: During the calculation process, the state is updated by combining the results of the forget gate and the input gate. The calculation formula is as follows.

[0051]

[0052] Where ° represents the element-by-element product; c t-1is the state at the previous time step. The hidden state is updated based on the combination of the output gate and the state.

[0053] h t =o t °tanh(c t )

[0054] The overall calculation process of the long short-term memory recurrent neural model of the present invention is to first store the weight of the i-1th learning task, and calculate the parameter similarity of each layer of the tested network model (a total of n layers) according to the training weight of the i-th task, where x i,n represents the parameter similarity of the nth layer of the i-th task; then the parameter similarity of the first layer x i,1 Input to the input gate, forget gate, and output gate. At this time, the state variable h 0 ,c 0 and weight W LSTM , bias parameter b LSTM (W LSTM 、b LSTM That is, the set of all W and b parameters defined above) is a random initialization vector of normal distribution, and f is calculated 1 , i 1 ,o 1 ; Then update the state quantity and change the second layer feature x i,2 Input to the input gate, forget gate, output gate, and loop calculation until feature x i,n Complete the calculation; finally based on the output layer o n , connect the fully connected layer to make probability prediction of the forgotten state.

[0055] The fifth step is model training. The weights W in the long-short memory recurrent neural model are trained based on the data and annotation information collected in the first step. LSTM , bias parameter b LSTM .

[0056] The learning order of the data set is changed multiple times to generate large quantities of training data, and based on the collected and automatically labeled data, the cross entropy loss function and the back propagation method are used to train the parameters of the long short-term memory recurrent neural model constructed by the present invention to obtain an online evaluation model.

[0057] The sixth step is to deploy and use the model, using the trained online evaluation model to evaluate the continuous learning state of the tested model with the same training architecture. During the use of the tested model, the parameters of the model are observed and recorded at fixed intervals, and the multi-level feature difference vector is calculated based on the model parameters at the current moment and the model parameters recorded at the previous moment. The model depth is used as the time series data, and the vector is used as the input. The long-short memory recurrent neural model trained in the previous step is used to predict the continuous learning state and output the corresponding results. The parameters saved at the previous moment are deleted, and the parameters at the current moment are saved.

Claims

1. A method for online evaluation of continuous learning of a neural network based on multi-layer feature differences, characterized by: In the process of continuous learning, the continuous learning ability of the tested neural network is evaluated by judging whether the distribution change of the network parameters of the tested neural network exceeds the threshold.

2. According to claim 1, a method for online evaluation of continuous learning of a neural network based on multi-layer feature differences is characterized by: The following steps are included: Step 1: extract the network parameters of each layer of the tested neural network corresponding to each learning task during the continuous learning process; Step 2: Based on the network parameters of adjacent learning tasks, extract the parameter similarity of each layer of the tested neural network; Step 3: construct a long short-term memory recurrent neural model as an evaluation model and train the evaluation model; Step 4: Use the trained evaluation model to evaluate the continuous learning state of the neural network under test.

3. The method for online evaluation of continuous learning of a neural network based on multi-layer feature differences according to claim 2, characterized in that: The network parameters of each layer of the neural network under test extracted during the continuous learning process include the weight matrix and the bias vector.

4. The method for online evaluation of continuous learning of a neural network based on multi-layer feature differences according to claim 2, characterized in that: The extraction of parameter similarity totals 8-dimensional similarity vectors, including cosine similarity, Euclidean distance, Manhattan distance, Chebyshev distance, Mahalanobis distance, Pearson correlation coefficient, Jaccard similarity coefficient, and KL divergence of network parameters.

5. According to any one of claims 2 to 4, the online evaluation method for continuous learning of a neural network based on multi-layer feature differences is characterized by: The input data of the evaluation model is: based on the layer depth of the neural network under test, the time series data composed of the parameter similarities of each layer of the neural network under test is used as input data; the output data of the evaluation model is: whether forgetting occurs when the neural network under test performs the current learning task.

Images