A geodesic metric-based interpretability approach for manifold network modeling

Abstract

The present invention discloses an interpretability method for manifold network modeling based on geodesic metrics. The present invention first uses Riemann manifolds to model the manifold features of the original image, so that the extracted feature subspace is used as the input of the deep learning network; then, based on the geodesic distance and back propagation model of the feature subspace, the gradient model of the deep learning network on the manifold is derived, the manifold curvature index is defined, and the geodesic distance between the feature layers of the deep network is calculated to finally output the manifold curvature index of each layer of the feature space. The present invention effectively utilizes the geometric structure of the data to model the feature space of the deep feature, analyzes the effectiveness principle of deep learning from the geometric perspective of the manifold space, and provides an interpretability method for the deep learning network by calculating the geodesic distance of the feature space of the deep learning network.

Description

Technical Field

[0001] The present invention relates to the field of deep learning and artificial intelligence technology, and in particular provides an interpretability method for manifold network modeling based on geodesic metrics. Background Art

[0002] In recent years, the breakthroughs made by artificial intelligence algorithms represented by deep learning in research directions such as target recognition and target tracking have set off a new round of development boom in artificial intelligence. Deep learning has been widely used in fields such as autonomous driving, medical treatment, and drone rescue. The key reason for the success of deep learning in many artificial intelligence application fields lies in the powerful learning ability and end-to-end learning method of deep network models. However, due to the high complexity of the deep learning method, it is often difficult for people to understand the decision results of the model, resulting in the difficulty of interpreting the deep learning model. For many years, the deep learning method, which optimizes parameters by annotating a large amount of data for error back propagation, has been regarded as a black box model. Most conclusions are confirmed by relying on a lot of engineering experience rather than theory. When the network model makes a wrong decision, it is difficult for researchers to trace the cause and make timely adjustments to the model. This problem also means that it is difficult to improve the algorithm according to task requirements in actual engineering applications, which hinders the development of the field of artificial intelligence. Therefore, researchers have been committed to a more thorough understanding of the complex network structure and learning process inside the deep learning model, so as to achieve the purpose of further optimizing the model. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a manifold network modeling interpretability method based on geodesic metrics to solve the problem of deep network interpretability.

[0004] The technical solution provided by the present invention is: a method for interpretability of manifold network modeling based on geodesic metrics, comprising the following steps:

[0005] 1) Preprocess the sample image to obtain the image features of the corresponding sample and establish the Riemann manifold feature space of the image;

[0006] 2) The Riemannian manifold feature space matrix is ​​used as the input of the deep learning network, trained through the convolution layer, and the feature space corresponding to each convolution layer is obtained;

[0007] 3) In the forward propagation phase, the gradient model of the Riemannian manifold feature space is constructed and the parameters of the deep learning network are transferred;

[0008] 4) In the back-propagation phase, the back-propagation algorithm based on the matrix chain rule is used to update the deep learning network parameters;

[0009] 5) Based on the geodesic distance of the Riemann manifold feature space, the curvature of the manifold space of each layer of the feature space is defined. Through the calculation results of the manifold curvature index, an interpretable analysis of the effectiveness principle of the deep learning network model is made.

[0010] The step of establishing the Riemannian manifold feature space of the image comprises the following steps:

[0011] For deep learning networks, the image set in each training phase contains n training samples, that is, the training image set can be expressed as I set = {I 1 ,I 2 ,…,I n}, extract each image I in the image set i The corresponding features constitute the feature vector v i The d-dimensional image feature vector representing each image;

[0012] Each image set I set The image set feature matrix X can be expressed as X = [v 1 ,v 2 ,…,v n ], the feature space can be represented as a Grassmann manifold matrix of d×r dimensions: where f QR (X) is the QR orthogonal decomposition function of X, and the output is Q 0 is the orthogonal matrix obtained after decomposition of X.

[0013] The feature space matrix with Grassmann manifold structure is input into the deep learning network for training, which includes the following steps:

[0014] Grassmann manifold matrix Q 0 As input, Q 0 is the input high-dimensional manifold, and Y is the learned new manifold as output. For the manifold learning layer, we have: Y = f(Q 0 )=W T Q 0 , where W to be learned is the transformation matrix;

[0015] The output Y of the manifold learning layer is used as the input of the manifold transformation layer, and the output of the manifold transformation layer is Q, that is, f QR (Y)=YR -1 =Q, where f QR (Y) is the QR orthogonal decomposition function of Y, and the output Q is the orthogonal matrix obtained after the decomposition of Y;

[0016] Q is input to the Euclidean mapping layer to obtain M, then M = f pro (Q)=QQT , where f pro (Q) is the mapping function;

[0017] M is used as the input of the fully connected layer and then passed to the cost function through the softmax layer.

[0018] In the forward propagation stage, the gradient model of the Riemann feature space is constructed and the parameters of the deep learning network are transferred, including the following steps:

[0019]

[0020] W t+1 =Γ(W t -λgrad R L(W)

[0021] Among them, W t+1 Represents the weight parameter for the next update, W t is the weight parameter of the current iteration t, Γ(·) represents the pullback mapping operator, λ represents the learning rate, and grad R L(W) represents the Riemann gradient, is the Euclidean gradient of the cost function L(W) of the manifold learning layer with respect to W The regular component of ;

[0022] The corresponding Euclidean gradient is obtained by using the matrix chain rule The expression is:

[0023]

[0024] Where L(Y) represents the cost function of the manifold transformation layer;

[0025] W t+1 Substitute the learning transformation matrix W in step 2) to update the learning transformation matrix.

[0026] In the back propagation phase, the back propagation algorithm based on the matrix chain rule is used to update the parameters of the deep learning network, including the following steps:

[0027] For the manifold learning layer, the parameter update formula is:

[0028]

[0029] In the above formula, Q 0 is the input of the manifold learning layer, and Y is the output of the manifold learning layer; L(W) represents the cost function of the manifold learning layer, and L(Y) represents the cost function of the manifold transformation layer;

[0030] Will After being superimposed with the transformation matrix W in step 2), it is used as the new W to replace the W of the previous iteration to update the learning transformation matrix;

[0031] For the manifold transformation layer, the parameter update formula is:

[0032]

[0033] In the above formula, A btril =A tril -(A) T tril , A tril Indicates that all upper triangular elements of matrix A are set to 0; L pro is the cost function of the manifold mapping layer; E is the identity matrix;

[0034] Will It is superimposed with the output Y of the manifold learning layer in step 2) as the new Y, which is used to replace the Y of the previous iteration to achieve the output update of the manifold learning layer.

[0035] Obtaining the geodesic distance of the feature space matrix of each layer in the deep network includes the following steps:

[0036] The geodesic metric of a Grassmann manifold is calculated as:

[0037] The feature space of each layer in the deep network is represented as a linear subspace, and each point on the Grassmann manifold is a linear subspace, represented as span(X); the linear subspace is composed of an orthogonal basis of the d×r-dimensional matrix X; on the Grassmann manifold, that is, the manifold learning layer outputs the new Y, the geodesic distance between any two points is calculated by the following formula:

[0038] d g (p,q)=||Θ|| 2

[0039] Where, d g represents the geodesic distance, p, q represent any two points on the Grassmann manifold, Θ=(θ 1 ,θ 2 ,…,θ k ) represents the main angle vector between the two points, and k represents the number of groups that the two points form.

[0040] The method of defining the manifold curvature index of the feature space and calculating the manifold curvature index of each layer of the deep network during the training process includes the following steps:

[0041] For each layer of the deep network, the feature space matrix X i With X j, the manifold curvature index c is defined as:

[0042]

[0043] In the above formula, D R (i,j)=d geo (X i ,X j ) represents the Riemannian manifold feature space X i With X j The geodesic distance between E (i,j)=||X i -X j || F Denotes the feature space X i With X j is the Euclidean distance between them, and N is the number of all points on the Grassmann manifold.

[0044] The calculation results of the manifold curvature index are used to make an interpretable analysis of the effectiveness principle of the deep network model, as follows:

[0045] The smaller the manifold curvature index is, the flatter the structure of the feature subspace is;

[0046] If the manifold curvature index is within the set threshold, it means that the learning process of the deep network is within the set reasonable range; otherwise, it means that the learning process of the deep network is not easy to converge.

[0047] Optimize the weight parameter W in the deep network based on the interpretability analysis results.

[0048] The present invention provides an interpretable method for manifold network modeling based on geodesic metrics, which has the following beneficial effects: combining differential geometry theory with deep learning theory, based on the geodesic distance of feature subspace and the back propagation model, a gradient model is derived for the deep learning network on the manifold, and a manifold curvature index based on the manifold geodesic distance is proposed. This invention effectively utilizes the geometric structure of data, performs manifold modeling on the feature space of deep features, analyzes the effectiveness principle of deep learning from the geometric perspective of manifold space, calculates the geodesic distance of the feature space of the deep learning network, and finally outputs the manifold curvature index of each layer of the feature space, providing an interpretable method for deep learning networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments:

[0050] Figure 1 A schematic diagram of a flow chart of an interpretability method for manifold network modeling based on geodesic metrics provided by the present invention;

[0051] Figure 2 A sample graph of a training image set in an interpretable method for manifold network modeling based on geodesic metrics provided by the present invention. DETAILED DESCRIPTION

[0052] The present invention will be further explained below in conjunction with specific implementation schemes, but the present invention is not limited thereto.

[0053] A method for interpretability of manifold network modeling based on geodesic metrics, which models the feature space of each layer of the network through input images, and explains the learning process of image data in the deep network by calculating the manifold curvature index, so as to optimize or trace the interpretability and effectiveness of the deep network, including:

[0054] Step 1: Input the original image.

[0055] Step 2: Extract features from the input original image, establish a feature space model of the image, and realize manifold feature modeling.

[0056] Step three: design a deep learning network model and train the deep network.

[0057] Step 4: Derive the gradient model of the deep learning network on the manifold.

[0058] Step 5: Update the network parameters using the back-propagation algorithm based on the matrix chain rule.

[0059] Step 6: Calculate the geodesic distance of the corresponding feature space in each network layer.

[0060] Step 7: Define and calculate the manifold curvature index.

[0061] The second step extracts features from the input original image and establishes a feature space model of the image. The manifold feature modeling includes the following steps:

[0062] Step 2-1: Extract the features corresponding to each image and construct a feature vector v i Represents the d-dimensional image feature vector for each image.

[0063] Step 2-2, construct the feature vector matrix of all images in each image set.

[0064] Step 2-3, construct the feature space model of each image set and calculate the Grassmann manifold feature subspace matrix.

[0065] The step three includes designing a deep learning network based on the manifold geometric structure of the feature space. According to the manifold structure of each layer of the feature space in the deep network, various operation layers of the deep learning network are designed to realize functions such as manifold learning and Euclidean mapping.

[0066] The step 4 includes deriving a gradient model of the deep learning network on the manifold. In the manifold learning layer, in order to calculate the feature space learned by each layer of the network, it is necessary to maintain the geometric structure of the feature space during the optimization process. When optimizing parameters, the Riemann optimization method is used to solve in the manifold space. First, the gradient model on the Riemann manifold is derived, and then the Riemann gradient descent method is used to search for the optimal solution of the parameters to be sought in the parameter space.

[0067] The step 5 includes updating the network parameters using a back propagation algorithm based on the matrix chain rule. First, in the forward propagation phase of the deep network, the parameter propagation formula of the model is calculated; in the back propagation phase of the network, the network parameters are updated using the matrix chain rule and the matrix back propagation algorithm.

[0068] The step six includes calculating the geodesic distance of the feature space using a geodesic metric based on Grassmann manifold.

[0069] The step seven includes defining a manifold curvature index of the feature space based on the geodesic metric of the Grassmann manifold and the Euclidean distance in the Euclidean space, and calculating the manifold curvature index of each layer of the deep network during the training process.

[0070] like Figure 1 As shown, the present invention provides a method for interpretability of manifold network modeling based on geodesic metrics, comprising the following steps:

[0071] Step 1: Input an image and preprocess it. In practice, in order to reduce the influence of illumination in the image, the image is usually normalized.

[0072] Step 2: Establish a feature space model of the image set. For each training stage, the training image set I contains n training samples. set = {I 1 ,I 2 ,…,I n}, extract each image I in the image set i The corresponding features constitute the feature vector v i Represents the d-dimensional image feature vector of each image, then each image set I set The image set feature matrix X can be expressed as X = [v 1 ,v 2 ,…,v n], the feature space can be represented as a Grassmann manifold matrix of d×r dimensions:

[0073]

[0074] where f QR (X) is the QR orthogonal decomposition function of X, and the output is Q 0 is the orthogonal matrix obtained after decomposition of X, R 0 represents the orthogonal decomposition matrix.

[0075] Step 3: Based on the manifold geometry of the feature space of each layer of the deep network, a deep learning network on the manifold is designed. First, a manifold learning layer is proposed. This layer uses the Grassmann manifold matrix Q calculated in step 2 0 As input, Q 0 is the input high-dimensional manifold, and Y is the learned new manifold as output. For the manifold learning layer, we have: Y = f(Q) 0 =W T Q 0 , where W to be learned is the transformation matrix. After that, the manifold transformation layer is proposed, which is immediately after the manifold learning layer. For the manifold transformation layer, after passing through several manifold learning layers, the output Y of the upper layer is used as the input of this layer through the manifold transformation layer. At this time, the output of this layer is Q, that is, f QR (Y)=YR -1 =Q, where f QR (X) is the QR orthogonal decomposition function of Y, the output Q is the orthogonal matrix obtained after Y is decomposed, and R represents the orthogonal decomposition matrix. Next, the Euclidean mapping layer is proposed, which is immediately after the manifold transformation layer. For the Euclidean mapping layer, the input at this time is Q and the output is M, then M = f pro (Q)=QQ T , where f pro (Q) is the mapping function of this layer. Finally, M is used as the input of the fully connected layer and then passed to the cost function through the softmax layer.

[0076] Step 4: Based on the Riemann optimization method, the gradient model of the deep learning network on the manifold is derived. When the input data is a Grassmann manifold matrix, the Riemann gradient descent method is used to update the parameters in the transformation layer. Specifically, for the cost function L(W) of the manifold learning layer, its Riemann gradient grad R The calculation of L(W) can be obtained by the following formula:

[0077]

[0078] in L(W) is the Euclidean gradient with respect to W, L(W) is the cost function of the softmax layer combined to the cost function of the manifold learning layer, and E is the identity matrix.

[0079] At this time, the specific expression of L(W) is

[0080]

[0081] Where L is the cost function of the softmax layer and f represents f(X).

[0082] The weight update formula of this layer on the Riemann manifold is:

[0083]

[0084] W t+1 =Γ(W t -λgrad R L(W)

[0085] Where W t The current weight parameter, Γ(·) represents the pullback mapping operator, λ represents the learning rate, is the Euclidean gradient of the objective function The regular component of .

[0086] According to the matrix chain rule, the corresponding Euclidean gradient can be obtained. The expression is:

[0087]

[0088] Among them, L(Y) is the cost function of the manifold transformation layer, and its specific expression is:

[0089]

[0090] Step 5: In the back-propagation phase, the network parameters are updated using a back-propagation algorithm based on the matrix chain rule.

[0091] For the manifold learning layer, the parameter update formula is:

[0092]

[0093] In the above formula, Q 0 is the input of the manifold learning layer, and Y is the output of the manifold learning layer; L(W) represents the cost function of the manifold learning layer, and L(Y) represents the cost function of the manifold transformation layer.

[0094] For the manifold transformation layer, the parameter update formula is:

[0095]

[0096] In the above formula, A btril =A tril -(A) T tril , A tril Indicates that all upper triangular elements of matrix A are set to 0. Among them, L pro Represents the cost function of the manifold mapping layer, and its specific expression is:

[0097]

[0098] Step 6: For each layer of feature space matrix in the deep network, it has a Grassmann manifold structure, so the geodesic distance on the manifold is calculated based on the geodesic metric of the Grassmann manifold.

[0099] On a Grassmann manifold, the geodesic distance between any two points can be calculated using the following formula:

[0100] d g (p,q)=||Θ|| 2

[0101] Where p and q represent any two points on the Grassmann manifold, Θ = (θ 1 ,θ 2 ,…,θ k ) represents the principal angle vector between the two points.

[0102] Step 7: Calculate the manifold curvature index of each layer of the deep network during training. For the feature space matrix X of each layer in the deep network i With X j , the manifold curvature index c is defined as:

[0103]

[0104] In the formula, D R (i,j)=d geo (X i ,X j ) represents the Riemannian manifold feature space X i With X j The geodesic distance between E (i,j)=||X i -X j || F Denotes the feature space X i With X j The Euclidean distance between .

[0105] The sample library diagram used by the manifold network modeling interpretability method based on geodesic metric proposed in this invention is as follows: Figure 2As shown in Table 1, the manifold curvature indexes between different feature layers in the deep learning network are shown.

[0106] Table 1

[0107]

[0108]

[0109] The manifold curvature index c is defined as: In the above formula, D R (i,j)=d geo (X i ,X j ) represents the Riemannian manifold feature space X i With X j The geodesic distance between E (i,j)=||X i -X j || F Denotes the feature space X i With X j The Euclidean distance between .

[0110] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A geodesic metric-based interpretability method for manifold network modeling, characterized by The following steps are involved: 1) Preprocess the sample image to obtain the image features of the corresponding sample and establish the Riemann manifold feature space of the image; 2) The Riemannian manifold feature space matrix is ​​used as the input of the deep learning network, trained through the convolution layer, and the feature space corresponding to each convolution layer is obtained; 3) In the forward propagation phase, the gradient model of the Riemannian manifold feature space is constructed and the parameters of the deep learning network are transferred; 4) In the back-propagation phase, the back-propagation algorithm based on the matrix chain rule is used to update the deep learning network parameters; 5) Based on the geodesic distance of the Riemann manifold feature space, the curvature of the manifold space of each layer of the feature space is defined. Through the calculation results of the manifold curvature index, an interpretable analysis of the effectiveness principle of the deep learning network model is made.

2. The method for interpretability of manifold network modeling based on geodesic metrics according to claim 1, characterized in that: The step of establishing the Riemannian manifold feature space of the image comprises the following steps: For deep learning networks, the image set in each training phase contains n training samples, that is, the training image set can be expressed as I set ={I1,I2,…,I n }, extract each image I in the image set i The corresponding features constitute the feature vector i=1,2,…,n,v i The d-dimensional image feature vector representing each image; Each image set I set The image set feature matrix X can be expressed as X = [v1, v2, ..., v n ], the feature space can be represented as a Grassmann manifold matrix of d×r dimensions: where f QR (X) is the QR orthogonal decomposition function of X, and the output Q0 is the orthogonal matrix obtained after X is decomposed.

3. The method for interpretability of manifold network modeling based on geodesic metrics according to claim 1, characterized in that: The feature space matrix with Grassmann manifold structure is input into the deep learning network for training, which includes the following steps: The Grassmann manifold matrix Q0 is used as input, that is, Q0 is the input high-dimensional manifold, and Y is the learned new manifold as output. For the manifold learning layer, we have: Y = f(Q0) = W T Q0, where W to be learned is the transformation matrix; The output Y of the manifold learning layer is used as the input of the manifold transformation layer, and the output of the manifold transformation layer is Q, that is, f QR (Y)=YR -1 =Q, where f QR (Y) is the QR orthogonal decomposition function of Y, and the output Q is the orthogonal matrix obtained after the decomposition of Y; Q is input to the Euclidean mapping layer to obtain M, then M = f pro (Q)=QQ T , where f pro (Q) is the mapping function; M is used as the input of the fully connected layer and then passed to the cost function through the softmax layer.

4. The method for interpretability of manifold network modeling based on geodesic metrics according to claim 1, characterized in that: In the forward propagation stage, the gradient model of the Riemann feature space is constructed and the parameters of the deep learning network are transferred, including the following steps: IN t+1 =Γ(W t -λgrad R L(W)) Among them, W t+1 Represents the weight parameter for the next update, W t is the weight parameter of the current iteration t, Γ(·) represents the pullback mapping operator, λ represents the learning rate, and grad R L(W) represents the Riemann gradient, is the Euclidean gradient of the cost function L(W) of the manifold learning layer with respect to W The regular component of ; The corresponding Euclidean gradient is obtained by using the matrix chain rule The expression is: Where L(Y) represents the cost function of the manifold transformation layer; W t+1 Substitute the learning transformation matrix W in step 2) to update the learning transformation matrix.

5. The method for interpretability of manifold network modeling based on geodesic metrics according to claim 1, characterized in that: In the back propagation phase, the back propagation algorithm based on the matrix chain rule is used to update the parameters of the deep learning network, including the following steps: For the manifold learning layer, the parameter update formula is: In the above formula, Q0 is the input of the manifold learning layer, Y is the output of the manifold learning layer; L(W) represents the cost function of the manifold learning layer, and L(Y) represents the cost function of the manifold transformation layer; Will After being superimposed with the transformation matrix W in step 2), it is used as the new W to replace the W of the previous iteration to update the learning transformation matrix; For the manifold transformation layer, the parameter update formula is: In the above formula, A btril =A tril -(A) T tril , A tril Indicates that all upper triangular elements of matrix A are set to 0; L pro is the cost function of the manifold mapping layer; E is the identity matrix; Will It is superimposed with the output Y of the manifold learning layer in step 2) as the new Y, which is used to replace the Y of the previous iteration to achieve the output update of the manifold learning layer.

6. The method for interpretability of manifold network modeling based on geodesic metrics according to claim 1, characterized in that: Obtaining the geodesic distance of the feature space matrix of each layer in the deep network includes the following steps: The geodesic metric of a Grassmann manifold is calculated as: The feature space of each layer in the deep network is represented as a linear subspace, and each point on the Grassmann manifold is a linear subspace, represented as span(X); the linear subspace is composed of an orthogonal basis of the d×r-dimensional matrix X; on the Grassmann manifold, that is, the manifold learning layer outputs the new Y, the geodesic distance between any two points is calculated by the following formula: d g (p,q)=||Θ||2 Where, d g represents the geodesic distance, p, q represent any two points on the Grassmann manifold, Θ=(θ1,θ2,…,θ k ) represents the main angle vector between the two points, and k represents the number of groups that the two points form.

7. The method for interpretability of manifold network modeling based on geodesic metrics according to claim 1, characterized in that: The method of defining the manifold curvature index of the feature space and calculating the manifold curvature index of each layer of the deep network during the training process includes the following steps: For each layer of the deep network, the feature space matrix X i With X j , the manifold curvature index c is defined as: In the above formula, D R (i,j)=d geo (X i ,X j ) represents the Riemannian manifold feature space X i With X j The geodesic distance between E (i,j)=||X i -X j || F Denotes the feature space X i With X j is the Euclidean distance between them, and N is the number of all points on the Grassmann manifold.

8. The method for interpretability of manifold network modeling based on geodesic metrics according to claim 1, characterized in that: The calculation results of the manifold curvature index are used to make an interpretable analysis of the effectiveness principle of the deep network model, as follows: The smaller the manifold curvature index is, the flatter the structure of the feature subspace is; If the manifold curvature index is within the set threshold, it means that the learning process of the deep network is within the set reasonable range; otherwise, it means that the learning process of the deep network is not easy to converge.

9. The method for interpretability of manifold network modeling based on geodesic metrics according to claim 1, characterized in that: Optimize the weight parameter W in the deep network based on the interpretability analysis results.

Images