Lifelong learning method of neural network model based on game theory framework

Abstract

The present application proposes a lifelong learning method for a neural network model based on a game theory framework, including: obtaining a network driving factor through quantitative expression of environmental entropy and network entropy, promoting changes in network structure, and realizing continuous learning of the network, and maintaining effectiveness, transferability, and continuous evolution when the environment changes in space and time. At the same time, the cost, accuracy, and output of the network are used as feedback information to further help the network adjust its structure, realize data closure, and realize sustainable learning of the network as it evolves over time, thereby realizing stable and effective self-learning. As a result, the present application can continue to learn when the data set undergoes modal and temporal changes, and has strong practical application prospects.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a lifelong learning method of a neural network model based on a game theory framework. Background Art

[0002] With the development of information technology, artificial neural networks are currently facing problems such as insufficient data sets in specific fields, as well as the existence of multimodal data sets that change rapidly over time in some fields. For the training of specific data sets, a mature network shows a lag in the changes of data sets: the model that performs well on the training set will have fixed parameters, which is no longer applicable to other modalities and data that migrates over time. In addition, the current network has poor robustness to noise, poor portability to new scenarios and some special scenarios, and cannot adapt to new data sets. Summary of the invention

[0003] The purpose of this application is to solve one of the above-mentioned technical problems at least to a certain extent.

[0004] To this end, one purpose of this application is to propose a lifelong learning method for a neural network model based on a game theory framework. This application uses the realization process of entropy balance as a driving factor under the framework of game theory, and through modeling the growth and decline strategy of network nodes driven by entropy balance and analyzing the process of reaching game equilibrium in the network, to establish lifelong learning under the framework of game theory, thereby achieving sustainable learning of the network as it evolves over time, thereby achieving stable and effective self-learning.

[0005] To achieve the above objectives, an embodiment of the present application proposes a lifelong learning method of a neural network model based on a game theory framework, including:

[0006] Performing feature extraction on an input image to obtain a feature vector for the input image;

[0007] Inputting the feature vector of the input image into an environment entropy calculation network to perform environment entropy calculation, and obtaining the environment entropy output by the environment entropy calculation network;

[0008] Calculating network entropy for the subject network based on the network entropy calculation network, and mapping the network entropy to the same space as the environment entropy;

[0009] Generate a corresponding entropy balance driving factor according to the environment entropy and the network entropy;

[0010] Driving the change of the strategy and weight of the connection nodes in the main network according to the entropy balance driving factor, and obtaining the network cost and network error according to the changed main network;

[0011] The network cost, the network error and the output result of the main network are used as feedback information, and the feedback information and the feature vector of the input image are input into the environmental entropy calculation network to calculate the environmental entropy, and the step of obtaining the environmental entropy output by the environmental entropy calculation network is returned to execute.

[0012] Optionally, in some embodiments of the present application, the learning process of the environment entropy calculation network is as follows:

[0013]

[0014]

[0015] Among them, f e (),f d () are encoder and decoder respectively, is the weight of the encoder, b e is the encoder bias, is the weight of the decoder, b d is the decoder bias, z u is a hidden variable containing entropy information, y u is the entropy-containing latent variable z u The decoded features; the encoder maps the feature vector to a latent space containing entropy information by learning the feature vector; the decoder decodes the latent variables sampled from the latent space into features and verifies the reliability of the latent variables; wherein, during training, the following objective function needs to be satisfied:

[0016] lm u =||x u -y u || 2 ;

[0017] Among them, x u is the feature vector of the input image; and the trained encoder is used as the environment entropy calculation network.

[0018] Optionally, in some embodiments of the present application, the learning process of the network entropy calculation network is as follows:

[0019]

[0020]

[0021] Among them, g e (),g d () are encoder and decoder respectively, is the weight of the encoder, α e is the encoder bias, is the weight of the decoder, α d is the bias of the decoder; z n is a hidden variable containing entropy information, y n is the entropy-containing latent variable z n Reconstructed network entropy; during training, the following objective function needs to be satisfied:

[0022] lm n =||H graph -y n || 2

[0023] Among them, H graph is the graph entropy, in, For each node i and d in the main network i The connection strength between the nodes; the trained encoder is used as the network entropy calculation network.

[0024] Optionally, in some embodiments of the present application, generating a corresponding entropy balance driving factor according to the environment entropy and the network entropy includes:

[0025] Calculating the difference between the environment entropy and the network entropy;

[0026] An entropy balance driving factor is generated according to the difference and a preset proportionality coefficient.

[0027] Optionally, in the embodiment of the present application, generating an entropy balance driving factor according to the difference and a preset proportionality coefficient includes:

[0028] The difference is multiplied by the proportional coefficient, and the negation of the product obtained by the multiplication is taken, and the negation is used as the entropy balance driving factor.

[0029] Optionally, in some embodiments of the present application, the neural network model is constructed based on a game theory framework:

[0030] In the graph theory representation, for the node V in the subject network, i |i=1,2…k} is used to model the strategy adopted by each node. The strategies adopted by each node are “growth” and “decrease”, with probabilities p1 and p2 respectively. The strategy adopted by each node i for the jth node is s ij :

[0031]

[0032] S i =(s i1 ,s i2 ,...,s ik )

[0033] The strategy set of all nodes is:

[0034]

[0035] When the game equilibrium is reached, any specific decision s' ij All of these make the network cost U i (s i1 ,s i2 ,...,s' ij ,...,s ik )≥U i (s i1 ,s i2 ,...,s ij ,...,s ik ),for Established, at this time node i reaches game equilibrium.

[0036] Optionally, in the embodiment of the present application, the network cost is calculated by the following formula:

[0037]

[0038] Among them, U i,t is the node cost of node i after adopting the strategy at time t; assuming that the connection status of node i to other nodes is “the connection strength is Θ i , disconnected to 0", write it into the weight matrix M i =(Θ1,Θ2,…,Θ k ), then the node cost of node i after adopting the strategy is

[0039] Optionally, in some embodiments of the present application, the taking the network cost, the network error and the output result of the main network as feedback information includes:

[0040] Acquire a corresponding objective function according to the network cost and the network error;

[0041] The objective function and the output result of the main network are used as the feedback information.

[0042] Optionally, in the embodiment of the present application, the objective function is calculated by the following formula:

[0043]

[0044] Among them, λ g is the weight coefficient, which indicates the proportion of network complexity in the optimization process, ||X t -Y t ||2 is the network error; is the network cost.

[0045] According to the technical solution of the embodiment of the present application, by quantitatively expressing the environmental entropy and the network entropy, the network driving factor is obtained to promote the change of the network structure, realize the continuous learning of the network, and maintain effectiveness, migration and continuous evolution when the environment changes in space and time. At the same time, the cost, accuracy and output of the network are used as feedback information to further help the network adjust the structure, realize the data closed loop, and realize the sustainable learning of the network as it evolves over time, thereby realizing stable and effective self-learning. As a result, the present application can continue to learn when the data set undergoes modal and temporal changes, and has a strong practical application prospect.

[0046] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0048] Figure 1 A flowchart of a lifelong learning method for a neural network model based on a game theory framework provided in an embodiment of the present application;

[0049] Figure 2 A schematic diagram of the principle block diagram of the lifelong learning method of a neural network model constructed based on a game theory framework according to an embodiment of the present application;

[0050] Figure 3 A schematic diagram of an entropy calculation network according to an embodiment of the present application;

[0051] Figure 4 A schematic diagram of the structural change process of the main network of an embodiment of the present application. DETAILED DESCRIPTION

[0052] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0053] The following describes the lifelong learning method of a neural network model based on a game theory framework according to an embodiment of the present application with reference to the accompanying drawings.

[0054] Figure 1The following is a flow chart of a lifelong learning method of a neural network model based on a game theory framework provided in an embodiment of the present application. Figure 1 As shown, the lifelong learning method of the neural network model constructed based on the game theory framework can include the following steps.

[0055] In step 101, feature extraction is performed on an input image to obtain a feature vector for the input image.

[0056] In order to better describe the amount of image information, the input image needs to be converted into a vector form, that is, feature extraction is performed. Optionally, an existing image feature extraction tool is used to extract one or more feature vectors from the input image and combine them into an overall vector x = (x1, x2, x3, ..., x n ), where x i represents the i-th eigenvector,

[0057] In step 102, the feature vector of the input image is input into an environment entropy calculation network to perform environment entropy calculation, and the environment entropy output by the environment entropy calculation network is obtained.

[0058] That is to say, after obtaining the feature vector of the input image, the feature vector x=(x1,x2,x3…x n ) Input the environmental entropy calculation network to calculate the environmental entropy.

[0059] The specific calculation method is as follows:

[0060] H u =f(x)

[0061] Among them, f() is a neural network. We only need to use a simple multilayer perceptron (MLP) to directly map the feature vector input to the high-dimensional space and obtain the environmental entropy H. u The multilayer perceptron is a feedforward artificial neural network model that can map multiple input data sets to a single output data set.

[0062] It should be noted that the environmental entropy calculation network is essentially an autoencoder. An autoencoder is a type of neural network, which can be considered to consist of two parts: an encoder function and a decoder that generates reconstruction. Traditionally, autoencoders are used for dimensionality reduction or feature learning. Unlike general autoencoders, this application uses the encoding part. In some embodiments, the learning process of the environmental entropy calculation network is as follows:

[0063]

[0064]

[0065] Among them, f e (),f d () are encoder and decoder respectively, is the weight of the encoder, b e is the encoder bias, is the weight of the decoder, b d is the decoder bias, z u is a hidden variable containing entropy information, y u is the entropy-containing latent variable z u The decoded features; the encoder maps the feature vector to a latent space containing entropy information by learning the feature vector; the decoder decodes the latent variables sampled from the latent space into features to verify the reliability of the latent variables; during training, the following objective function needs to be met:

[0066] lm u =||x u -y u || 2 ;

[0067] Among them, x u is the feature vector of the input image; the trained encoder is used as the environment entropy calculation network. From the objective function, it can be seen that the generated features are as close as possible to the input features, indicating that the hidden variable z u It is a reliable quantity that can represent the environmental entropy in this application. In the application process, the environmental entropy is calculated according to the following formula:

[0068]

[0069] In step 103, the network entropy is calculated for the subject network based on the network entropy calculation network, and the network entropy is mapped to the same space as the environment entropy.

[0070] It can be understood that the main network is actually a neural network. When calculating the network entropy of the main network, the main network can be regarded as a graph G = (V, E). The node set V = {v i |i=1,2,...,k}, each node i and d i The nodes are connected, and the connection strength is

[0071]

[0072]

[0073] H n =g(H graph )

[0074] In the above formula, H graph is the graph entropy, which directly calculates the network connection complexity under the graph theory representation. g() is also a multi-layer perceptron that can convert the graph entropy H graph Mapped to the high-dimensional feature space, we get the network entropy H n .

[0075] In some embodiments of the present application, the learning process of the network entropy calculation network is as follows:

[0076]

[0077]

[0078] Among them, g e (),g d () are encoder and decoder respectively, is the weight of the encoder, α e is the encoder bias, is the weight of the decoder, α d is the decoder bias; z n is a hidden variable containing entropy information, y n is the entropy-containing latent variable z n Reconstructed network entropy; during training, the following objective function needs to be satisfied:

[0079] lm n =||H graph -y n || 2

[0080] Among them, H graph is the graph entropy, in, For each node i and d in the main network i The connection strength between the nodes is calculated; the trained encoder is used as the network entropy calculation network.

[0081] When generating network entropy, it only needs to go through the encoding network and a nonlinear mapping:

[0082]

[0083] In step 104, a corresponding entropy balance driving factor is generated according to the environment entropy and the network entropy.

[0084] It should be noted that both the environmental entropy calculation network and the network entropy calculation network are autoencoders that can encode features into high-dimensional space. However, unlike general autoencoders, this application retains the encoding part in time applications to obtain high-dimensional latent variables containing entropy meanings. The information entropy (i.e., environmental entropy) after the entropy calculation network has the same physical meaning as the network entropy, so they can be subtracted to drive changes in the network structure.

[0085] Optionally, in some embodiments of the present application, the difference between the environment entropy and the network entropy is calculated, and the entropy balance driving factor is generated according to the difference and a preset proportional coefficient. As an example, the difference and the proportional coefficient can be multiplied, and the product obtained by the multiplication is taken as the inverse, and the inverse is used as the entropy balance driving factor.

[0086] For example, the entropy balance driver can be calculated as follows:

[0087] Q=-λ(H u -H n )

[0088] Among them, Q is the entropy balance driving factor, representing the information difference between the environment and the network, and λ is the proportionality coefficient.

[0089] In step 105, the strategies and weights of connecting nodes in the main network are driven to change according to the entropy balance driving factor, and the network cost and network error are obtained according to the changed main network.

[0090] It should be noted that in the embodiment of the present application, a neural network model is constructed based on the game theory framework:

[0091] In the graph theory representation, for the nodes V in the main network, i |i=1,2…k} is used to model the strategy adopted by each node. The strategies adopted by each node are “growth” and “decrease”, with probabilities p1 and p2 respectively. The strategy adopted by each node i for the jth node is s ij :

[0092]

[0093] S i =(s i1 ,s i2 ,...,s ik )

[0094] The strategy set of all nodes is:

[0095]

[0096] When the game equilibrium is reached, any specific decision s' ij All of these make the network cost Ui (s i1 ,s i2 ,...,s' ij ,...,s ik )≥U i (s i1 ,s i2 ,...,s ij ,...,s ik ),for Established, at this time node i reaches game equilibrium.

[0097] Optionally, in the embodiment of the present application, the network cost is calculated by the following formula:

[0098]

[0099] Among them, U i,t is the node cost of node i after adopting the strategy at time t; assuming that the connection status of node i to other nodes is “the connection strength is Θ i , disconnected to 0", write it into the weight matrix M i =(Θ1,Θ2,…,Θ k ), then the node cost of node i after adopting the strategy is

[0100] This application focuses on the problem of adjusting the network structure, which is a dynamic process, so it is meaningful to calculate the loss function only at a specific time t. In this case, this application needs to calculate the cost of the entire network, that is:

[0101]

[0102] The above formula means the sum of the node costs of each node after adopting the strategy, that is, the total cost of the network. In the specific task, this application also cares about the accuracy of the network (set network error), that is:

[0103] L 1,t =||X t -Y t || 2

[0104] Where X t is the input image feature, Y t is the output of the network, which measures the accuracy of the network.

[0105] In step 106, the network cost, network error and output result of the main network are used as feedback information, and the feedback information and the feature vector of the input image are input into the environment entropy calculation network for environment entropy calculation, and the step of obtaining the environment entropy output by the environment entropy calculation network is returned.

[0106] In some embodiments of the present application, the corresponding objective function is obtained based on the network cost and network error, and the objective function and the output result of the main network are used as feedback information to further help the network adjust its structure and realize data closure, thereby achieving stable and effective self-learning.

[0107] Optionally, the objective function is calculated by the following formula:

[0108]

[0109] Among them, λ g is the weight coefficient, which indicates the proportion of network complexity in the optimization process, ||X t -Y t || 2 is the network error; For network costs.

[0110] It should be noted that the lifelong learning method of the neural network model proposed in this application based on the game theory framework is as follows: Figure 2 As shown. By learning two entropy calculation networks, the environment entropy and network entropy are mapped to the same space, making them have the same physical meaning. The difference between the two entropies is calculated using the entropy difference network to drive the change of the connected nodes in the main network. During use, the fixed entropy calculation network provides the entropy driving factor, while the structure of the main network is constantly adjusted with the changes in input and output.

[0111] In this embodiment, the process of entropy calculation is as follows: Figure 3 As shown. Figure 3 The left side of the figure shows the learning process of the entropy calculation network. Figure 3 The right side of the figure shows the use process of the entropy calculation network. In the learning process of the entropy calculation network, the feature input, network complexity and output accuracy are comprehensively considered, mapped to a high-dimensional latent space, sampled from the latent space, reconstructed the feature vector, and the entropy encoder is adjusted according to the difference between the reconstructed result and the original feature vector. During use, only the entropy encoder is retained, and the entropy-containing latent variable is calculated according to the input feature. After mapping through a nonlinear function, the environmental entropy or network entropy is calculated.

[0112] In this embodiment, if Figure 4The figure shows the schematic diagram of the structural change process of the main network. In the main network, each node is modeled. The decision space and cost function of the node are specified. Driven by the entropy balance, the network nodes and cost functions change, changing the growth and decline strategy of the network, thereby changing the network structure and achieving self-adaptation with time and space. At the same time, the comprehensive consideration of decision cost and network accuracy will enable the network to operate efficiently while maintaining accuracy, and change its network structure at any time for inputs of different complexity.

[0113] From the above description, it can be seen that the present application can obtain the network driving factor by quantitatively expressing the environmental entropy and the network entropy, thereby promoting the change of the network structure. At the same time, the cost, accuracy and output of the network are used as feedback information to further help the network adjust the structure. A data closed loop is achieved to achieve stable and effective self-learning. Compared with traditional learning methods, the lifelong learning method of the neural network model proposed in this application based on the game theory framework can continue to learn when the data set undergoes modal and temporal changes, and has a strong practical application prospect.

[0114] In order to facilitate those skilled in the art to understand the present application more clearly, Figures 2 to 4 For example, the lifelong learning method of the neural network model proposed in this application based on the game theory framework is implemented as follows:

[0115] (1) Extracting image features

[0116] In order to better describe the amount of image information, it is necessary to convert the input image into a vector form, that is, to perform feature extraction. To this end, the existing image feature extraction tools are used to extract one or more feature vectors from the image and combine them into an overall vector x = (x1, x2, x3, ..., x n ), where x i represents the i-th eigenvector,

[0117]

[0118] (2) Calculate the information entropy of the feature vector and the network entropy of the neural network

[0119] like Figure 2 As shown, after obtaining the eigenvector, the eigenvector x=(x1,x2,x3,...,x n ) Input the environmental entropy calculation network to calculate the environmental entropy. The specific calculation method is as follows:

[0120] H u =f(x)

[0121] Among them, f() is a neural network. We only need to use a simple multilayer perceptron (MLP) to directly map the feature vector input to the high-dimensional space and obtain the environmental entropy H. u The multilayer perceptron is a feedforward artificial neural network model that can map multiple input data sets to a single output data set.

[0122] To calculate the network entropy of the main network, the main network can be regarded as a graph G = (V, E). The node set V = {v i |i=1,2,...,k}, each node i and d i The nodes are connected, and the connection strength is

[0123]

[0124]

[0125] H n =g(H graph )

[0126] In the above formula, H graph is the graph entropy, which directly calculates the network connection complexity under the graph theory representation. g() is also a multi-layer perceptron that can convert the graph entropy H graph Mapped to the high-dimensional feature space, we get the network entropy H n .

[0127] (3) Design of entropy calculation network

[0128] This application needs to calculate the difference in entropy, so it is necessary to map the two entropies to the same space so that they have the same physical meaning, so it is necessary to design entropy calculation networks f() and g() for the two entropies respectively.

[0129] The entropy calculation network is essentially an autoencoder. An autoencoder is a type of neural network that can be considered to consist of two parts: an encoder function and a decoder that generates reconstructions. Traditionally, autoencoders are used for dimensionality reduction or feature learning. Unlike general autoencoders, this application uses the encoding part.

[0130] The learning process of the environmental entropy calculation network is as follows:

[0131]

[0132]

[0133] Among them, f e (),f d() are encoding and decoding networks respectively, which are simple multi-layer perceptrons. is the weight of the encoder, b e is the encoder bias, is the weight of the decoder, b d is the decoder bias. u is a hidden variable containing entropy information, y u is the entropy-containing latent variable z u Decoded features. The encoder maps the feature vector to a latent space containing entropy information by learning it; the decoder decodes the latent variables sampled from the latent space into features, verifies the reliability of the latent variables, and indicates that they contain sufficient feature information. During training, the following objective function needs to be met:

[0134] lm u =||x u -y u || 2

[0135] That is, the generated features are as close as possible to the input features, indicating that the latent variable z u It is a reliable quantity that can represent the environmental entropy in this application. In the application process, the environmental entropy is calculated according to the following formula:

[0136]

[0137] The network entropy calculation network learning process is as follows:

[0138]

[0139]

[0140] Among them, g e (),g d () are encoding and decoding networks respectively. With the same environmental entropy, a multi-layer perceptron can complete this task. The difference is that g e The input of (·) is a single channel, not a multi-dimensional feature vector. is the weight of the encoder, α e is the encoder bias, is the weight of the decoder, α d is the decoder bias. n is a hidden variable containing entropy information, y n is the entropy-containing latent variable z n Reconstructed network entropy. During training, the following objective function needs to be met:

[0141] lm n =||H graph -yn || 2

[0142] When generating network entropy, it only needs to go through the encoding network and a nonlinear mapping:

[0143]

[0144] Both networks are autoencoders that can encode features into high-dimensional space. However, unlike general autoencoders, this application retains the encoding part in time applications to obtain high-dimensional hidden variables containing entropy meanings. The information entropy (i.e., environmental entropy) after the entropy calculation network has the same physical meaning as the network entropy, so they can be subtracted to drive changes in the network structure.

[0145] Q=-λ(H u -H n )

[0146] Among them, Q is the entropy balance driving factor, representing the information difference between the environment and the network, and λ is the proportionality coefficient.

[0147] (4) Game theory modeling of network nodes

[0148] Under the graph theory representation, the network nodes are modeled, V = {v i |i=1,2,...,k}, the strategies that each node can adopt are "growth" and "decrease", with probabilities p1 and p2 respectively. The strategy that each node i makes after adopting the strategy for the jth node is s ij :

[0149]

[0150] S i =(s i1 ,s i2 ,...,s ik )

[0151] The strategy set of all nodes is:

[0152]

[0153] When the game equilibrium is reached, any specific decision s' ij All of these make the network cost U i (s i1 ,s i2 ,...,s' ij ,...,s ik )≥U i (s i1 ,s i2 ,...,s ij ,...,sik ),for Established, at this time node i reaches game equilibrium.

[0154] (5) Cost function in the game theory framework

[0155] In order to better describe the effect of the entire network after the network nodes adopt strategies in the game theory framework, it is necessary to calculate the network cost U according to the strategy selection. i Calculate and assume that the connection strength of node i to other nodes is "θ i , disconnected to 0", write it into the weight matrix M i =(Θ1,Θ2,...,Θ k ):

[0156]

[0157] Weight matrix M i It is a variable quantity. By adjusting the weight matrix, the network cost under the current connection strategy is measured to guide further adjustment of the network structure.

[0158] (6) Establishment of loss function

[0159] This application focuses on the problem of adjusting the network structure, which is a dynamic process, so it is meaningful to calculate the loss function only at a specific time t. In this case, this application needs to calculate the cost of the entire network, that is:

[0160]

[0161] The above formula means the sum of the node costs of each node after adopting the strategy, that is, the total cost of the network. In the specific task, this application also cares about the accuracy of the network, that is:

[0162] L 1,t =||X t -Y t || 2

[0163] Where X t is the input image feature, Y t is the output of the network, which measures the accuracy of the network.

[0164] In summary, the optimization objectives obtained in this application are as follows:

[0165]

[0166] Among them, λ gIs a weight coefficient, which indicates the proportion of network complexity in the optimization process. When the network is working, back propagation is no longer performed, but the accuracy of the network can still change with the change of the network structure. Therefore, by inputting the loss at time t-1 into the network together with the input at the next moment, the dual adjustment of network accuracy and connection structure is achieved.

[0167] (7) Introduction of entropy balance driving factors

[0168] In the previous step, we obtained the overall objective function. In this step, we will introduce the entropy driving factor in detail. t The impact of network structure adjustment, at time t, U i,t =U i,t (S i,t ,Q t ) is the node v i In the mixed strategy S i,t The cost function under. Calculate the weight matrix M of the network cost i,t (Q t ) is as follows:

[0169] M i,t =M i,t-1 +f M (Q t )

[0170] Among them, f M () refers to the guiding polynomial that guides the change of the weight matrix. Expand the above formula, that is:

[0171] (Θ i1 ,Θ i2 ,...,Θ ik ) t =(Θ i1 ,Θ i2 ,...,Θ ik ) t-1 +f M (Q t )

[0172] The connection strength between node i and node j is written in component form as follows, where f M (Q t )| i [j] represents f M (Q t )| i The jth component of

[0173] Θ ij | t =Θ ij | t-1 +f M (Qt )| i [j]

[0174] in, It is fitted by a k-order polynomial:

[0175]

[0176] Among them, γ i are the coefficients of the polynomial and are fixed quantities.

[0177] The strategy set S of the network nodes i,t The evolution of the entropy driving factor is as follows, where g s (·) is a polynomial function that guides the change of connection strategy:

[0178] S i,t =S i,t -1+g s (Q t )

[0179] Expanded into component form, it is:

[0180] (s i1 ,s i2 ,…,s ik ) t =(s i1 ,s i2 ,…,s ik ) t-1 +g s (Q t )| i

[0181] The connection strategy between node i and node j is written in component form as follows, where g s (Q t )| i [j] represents g s (Q t )| i The jth component of

[0182]

[0183] In order to ensure It makes sense for g S (Q t )| i [j] The following constraints need to be made:

[0184]

[0185] So g S (Q t )|i Written in the following form:

[0186]

[0187]

[0188] is a polynomial fitting function. After the above processing, it can be seen that the nodes with strong connection strength will be suppressed if they continue to strengthen their connection, which ensures that the network structure will not tend to solidify and is flexible and adjustable. By introducing the entropy driving factor, the network can adjust the network structure as the environment changes and re-assign the network structure.

[0189] (8) Implementation of Continuous Learning

[0190] The above steps take the output of the previous moment and the network cost as input, build a feedback system and input them into the network, and obtain a closed loop structure that changes over time. The above steps need to be repeated continuously, and the network can continue to work in a closed loop and continuously adjust the network structure over time to achieve continuous learning.

[0191] According to the technical solution of the embodiment of the present application, by quantitatively expressing the environmental entropy and the network entropy, the network driving factor is obtained to promote the change of the network structure, realize the continuous learning of the network, and maintain effectiveness, migration and continuous evolution when the environment changes in space and time. At the same time, the cost, accuracy and output of the network are used as feedback information to further help the network adjust the structure, realize the data closed loop, and realize the sustainable learning of the network as it evolves over time, thereby realizing stable and effective self-learning. As a result, the present application can continue to learn when the data set undergoes modal and temporal changes, and has a strong practical application prospect.

[0192] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless otherwise clearly and specifically defined, without contradiction. In addition, the term "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0193] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0194] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0195] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0196] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0197] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0198] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A lifelong learning method for a neural network model based on a game theory framework, characterized in that: include: Performing feature extraction on an input image to obtain a feature vector for the input image; Inputting the feature vector of the input image into an environment entropy calculation network to perform environment entropy calculation, and obtaining the environment entropy output by the environment entropy calculation network; Calculating network entropy for the subject network based on the network entropy calculation network, and mapping the network entropy to the same space as the environment entropy; Generate a corresponding entropy balance driving factor according to the environment entropy and the network entropy; The strategy and weight of connecting nodes in the main network are driven according to the entropy balance driving factor, and the network cost and network error are obtained according to the changed main network; wherein the network cost is calculated by the following formula: Among them, U i,t is the node cost of node i after adopting the strategy at time t; assuming that the connection status of node i to other nodes is "the connection strength is Θ i , disconnected to 0", write it into the weight matrix M i =(Θ1,Θ2,…,Θ k ), then the node cost of node i after adopting the strategy is The network cost, the network error and the output result of the main network are used as feedback information, and the feedback information and the feature vector of the input image are input into the environment entropy calculation network to calculate the environment entropy, and the step of obtaining the environment entropy output by the environment entropy calculation network is returned to be executed; The learning process of the environment entropy calculation network is as follows: Among them, f e (),f d () are encoder and decoder respectively, is the weight of the encoder, b e is the encoder bias, is the weight of the decoder, b d is the decoder bias, z u is a hidden variable containing entropy information, y u is the entropy hidden variable z u The decoded features; the encoder maps the feature vector to a latent space containing entropy information by learning the feature vector; the decoder decodes the latent variables sampled from the latent space into features and verifies the reliability of the latent variables; wherein, during training, the following objective function needs to be satisfied: min L u =||x u -y u || 2 ; Among them, x u is the feature vector of the input image; and the trained encoder is used as the environment entropy calculation network; The learning process of the network entropy calculation network is as follows: Among them, g e (),g d () are encoder and decoder respectively, is the weight of the encoder, α e is the encoder bias, is the weight of the decoder, α d is the bias of the decoder; z n is a hidden variable containing entropy information, y n is the entropy hidden variable z n Reconstructed network entropy; during training, the following objective function needs to be satisfied: min L n =||H graph -y n || 2 Among them, H graph is the graph entropy, in, For each node i and d in the main network i The connection strength between the nodes; the trained encoder is used as the network entropy calculation network.

2. The method according to claim 1, characterized in that The generating a corresponding entropy balance driving factor according to the environment entropy and the network entropy includes: Calculating the difference between the environment entropy and the network entropy; An entropy balance driving factor is generated according to the difference and a preset proportionality coefficient.

3. The method according to claim 2, characterized in that The step of generating an entropy balance driving factor according to the difference and a preset proportionality coefficient includes: The difference is multiplied by the proportional coefficient, and the negation of the product obtained by the multiplication is taken, and the negation is used as the entropy balance driving factor.

4. The method according to claim 1, characterized in that: The neural network model is constructed based on the game theory framework: In the graph theory representation, for the node V in the subject network, i |i=1,2…k} is used to model the strategy adopted by each node. The strategies adopted by each node are "growth" and "decrease", with probabilities p1 and p2 respectively. The strategy adopted by each node i for the jth node is s ij : S i =(s i1 ,s i2 ,...,s ik ) The strategy set of all nodes is: When the game equilibrium is reached, any specific decision s i ' j All of these make the network cost U i (s i1 ,s i2 ,...,s i ' j ,...,s ik )≥U i (s i1 ,s i2 ,...,s ij ,...,s ik ),for Established, at this time node i reaches game equilibrium.

5. The method according to any one of claims 1 to 4, characterized in that The using the network cost, the network error and the output result of the main network as feedback information includes: Acquire a corresponding objective function according to the network cost and the network error; The objective function and the output result of the main network are used as the feedback information.

6. The method according to claim 5, characterized in that The objective function is calculated by the following formula: Among them, λ g is the weight coefficient, which indicates the proportion of network complexity in the optimization process, ||X t -Y t || 2 is the network error; is the network cost.

Images