A method for quantifying uncertainty in material composition and property mapping

Abstract

The present application relates to a kind of material composition and performance mapping uncertainty quantification method, belong to material science and technology field.First, material composition data is collected from Materials Project database;Then, the atomic number and fraction in data are encoded into high-dimensional vector using fully connected neural network;Then, these high-dimensional vectors are combined into element derived matrix, and input into Transformer module to learn chemical structure information;Finally, the chemical structure information features learned are processed by spectral normalization residual MLP, and Gaussian process modeling is applied in output layer to quantify uncertainty.The present application is applied in material property prediction, not only can give the predicted value of material performance, but also can quantify the uncertainty of prediction, provide the distribution of prediction result.

Description

TECHNICAL FIELD

[0001] The present application relates to a material composition and performance mapping uncertainty quantification method, belonging to the cross technical field of material science and artificial intelligence. BACKGROUND

[0002] The performance of materials is influenced by its composition, microstructure, processing techniques, and other factors. These factors interact with each other, making it extremely complex to predict material performance. Traditional material development relies on laboratory experiments and trial-and-error methods, which not only consume time and resources, but also may not cover all possible material combinations. Machine learning provides a data-driven approach to predict material performance, which is expected to accelerate the process of material design. However, existing machine learning models usually lack quantification of prediction uncertainty, which limits their reliability and credibility in high-risk applications. Gaussian process is a powerful non-parametric Bayesian method that can provide uncertainty estimates about predictions, which is very suitable for dealing with complex systems with uncertainty. Laplace approximation is a mathematical method used to simplify the calculation of posterior distribution, which makes Gaussian process more feasible in practical applications.

[0003] At the same time, deep learning models are prone to gradient explosion during training. Spectral normalization helps control the size of the gradient by limiting the spectral norm of the network layer weights. Lipschitz constraint ensures that the output of the model will not fluctuate dramatically due to small changes in the input, improving the stability and generalization ability of the model. Residual network effectively extracts deep features of material composition through its residual learning framework. The Transformer model captures the global dependency of sequence data through self-attention mechanism, providing a new perspective for material performance prediction.

[0004] In summary, through the analysis and evaluation of prediction uncertainty, we can more accurately understand the credibility of model prediction, thereby guiding experimental design and material discovery projects, avoiding waste of resources and consumption of time. The present application introduces a material composition and performance mapping uncertainty quantification method, which combines Gaussian process with deep learning framework to explore the complex relationship between material composition and its characteristics through detailed data analysis. The uncertainty is quantified using Gaussian process, thereby providing higher reliability and accuracy in material performance prediction. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a material composition and performance mapping uncertainty quantification method, which can quantify the uncertainty in the prediction process, thereby solving the above problems.

[0006] The technical scheme of the present application is: a material composition and performance mapping uncertainty quantification method, first, collect material composition data from the Materials Project database; then, use a fully connected neural network to encode the atomic number and fraction in the data into a high-dimensional vector; then, combine these high-dimensional vectors into an element-derived matrix and input them into a Transformer module to learn chemical structure information, obtaining chemical structure information features. Finally, the chemical structure information features are processed through spectral normalization residual MLP, and a Gaussian process modeling is applied in the output layer to quantify the uncertainty.

[0007] The specific steps are:

[0008] Step 1: Collect material composition data from the Materials Project database, the composition data including the atomic number of each element and the fraction in the material;

[0009] Step 2: Use a fully connected neural network to encode the atomic number and fraction into a high-dimensional vector, obtaining an element-derived matrix that contains the relationship information between elements;

[0010] Step 3: Build a Transformer module, input the element-derived matrix into the Transformer module, and learn the interaction of each element in the material and the chemical structure information of each element through the self-attention mechanism in the Transformer module, obtaining chemical structure information features;

[0011] Step 4: Input the chemical structure information features into the residual multilayer perceptron module, design spectral normalization in the residual multilayer perceptron module to constrain the mapping of the weight matrix in the residual multilayer perceptron module, prevent overfitting, and improve the generalization ability of the model;

[0012] Step 5: Build a Gaussian model, optimize the parameters of the Gaussian model through the log posterior likelihood and Hessian matrix, then input the chemical structure information features in the residual multilayer perceptron module into the optimized Gaussian model for Gaussian process processing, realizing the quantification of the uncertainty of the mapping between material composition and performance.

[0013] The Step 2 is specifically:

[0014] Step 2.1: Stratified random sampling is performed on the data set that has not been divided, the class proportion in different sets is fixed, and finally the training set, validation set and test set required for the experiment are extracted;

[0015] Step 2.2: The atomic number of each element is generated through a fully connected neural network layer to generate a corresponding high-dimensional vector representation, obtaining an element embedding matrix;

[0016] Step2.3: The score of each element is converted into a corresponding high-dimensional vector representation through score encoding to obtain a score embedding matrix;

[0017] Step2.4: The element embedding matrix and the score embedding matrix are combined to form an element derived matrix, and the preprocessing of the data set is completed.

[0018] The Transformer module is composed of multiple layers, each layer including a self-attention mechanism and a feedforward neural network;

[0019] Two inter-layer enhancement mechanisms are introduced in the Transformer module: residual connection and layer normalization; the residual connection directly adds the input to the processed output;

[0020] The layer normalization is a standardization process on the output of each layer, which normalizes the mean and variance of the features, so that different batches of data can be trained on similar scales.

[0021] The Transformer module includes 3 encoder layers, each encoder layer containing 4 attention heads, which are used to capture the context information in the input data;

[0022] The enhancement mechanism between the layers of the encoder is that each sub-layer contains a residual connection and layer normalization:

[0023]

[0024] In the formula, X is the feature vector input to this layer, with a dimension of R n×d , Z(X) is the output of the input x after processing by the previous layer, Z(X)+X represents the residual connection, LayerNorm represents the layer normalization, FFN is the feedforward neural network, F l is the final output encoding feature, which has the same dimension as X.

[0025] The Step4 is specifically:

[0026] Step4.1: The residual multi-layer perceptron module is composed of different residual blocks, represented as:

[0027] h=h L-1 ·...·h2·h1 (1)

[0028] h l (F l )=F l +g l (F l ) (2)

[0029] g l (x) = a(W l x + b l ) (3)

[0030] Formula (1) represents the forward propagation process of the entire network, which is composed of multiple layers, i.e., from h1 to h L-1 ;

[0031] In formula (2), F l is the input into the Lth layer, g l (F l ) is the transformation of the input F l in the Lth layer, representing the new features after passing through this layer;

[0032] In formula (3), W l represents the weight matrix of the Lth layer, which is used for linear transformation of the input feature x, a is the activation function, b l is the bias term, and g l (x) represents the nonlinear transformation of the Lth layer on the input x;

[0033] Step 4.2: Design spectral normalization to perform spectral normalization on the weight matrix of each layer in the MLP to realize the Lipschitz constraint on g l ;

[0034] The spectral normalization is to estimate the maximum eigenvalue of the weight matrix by power iteration The maximum eigenvalue is used to normalize the weight;

[0035]

[0036] In the formula, c is a hyperparameter used to adjust the upper bound of the spectral norm of the weight matrix, so that ||W l ||2≤ c, W l is the weight matrix of the current Lth layer, and λ is a threshold value representing a scalar value used for comparison under certain conditions.

[0037] The Gaussian process processing is specifically:

[0038] The prior distribution of the Gaussian model equipped with the RBF kernel is a multivariate normal distribution:

[0039]

[0040] In the formula, σ 2 and l are hyperparameters, g N×1 is a random vector of length N, representing the output of the Gaussian process, i.e., the predicted value, MVN is a multivariate normal distribution, 0 N×1is a zero vector of length N, indicating that the mean of the multivariate normal distribution is zero, K N×N is an N x N covariance matrix generated by the RBF kernel function, indicating the correlation between any two observation points, K i,j represents the element in the i-th row and j-th column of the covariance matrix, which describes the similarity between the i-th and j-th input points, represents the square of the Euclidean distance between the i-th and j-th input points, L is the length scale of the RBF kernel Using random feature expansion to approximate the prior, define the kernel matrix K = ΦΦ T , the random feature Gaussian process is:

[0041]

[0042] where g N×1 is a random vector of length N, representing the output of the Gaussian process, i.e. the predicted value; MVN is a multivariate normal distribution; 0 N×1 is a zero vector of length N, indicating that the mean of the multivariate normal distribution is zero; is calculated from the random feature matrix Φ, where Φ is an N x D L matrix representing the feature mapping generated by the random feature method; Φ i represents the last layer of the residual network, with a dimension of D L , W L is a fixed weight matrix, sampled independently and identically distributed from N (0，1) , b L is a fixed bias term;

[0043] Using random feature expansion with Gaussian prior to write a neural network layer composed of fixed hidden weights W L and learnable output weights β:

[0044]

[0045] where F o is the input feature vector, g(F o ) is the output of the neural network layer, calculated based on the input feature F o ; is a scaling factor, cos(-W L F o +b L ) is the nonlinear mapping of the input feature F o after linear transformation, b L is the bias vector, β conforms to the multivariate normal distribution with mean zero and variance τ, is a D L x D L identity matrix.

[0046] The log posterior likelihood is specifically:

[0047]

[0048] In the formula, logp(beta|x) represents the log posterior probability of the parameter beta given the input data x, is a term related to the likelihood function, which measures the difference between the model prediction value g i and the true value y i , is a regularization term, called L2 regularization.

[0049] The Hessian matrix is specifically

[0050]

[0051] In the formula, is an estimate of the D L ×D L dimensional Hessian matrix, Phi i is an eigenvector generated in the random feature expansion or neural network layer; This term represents the outer product of the eigenvector Phi i , generating a D L ×D L matrix, is a D L ×D L identity matrix.

[0052] The beneficial effects of the present application are:

[0053] 1. Compared with existing uncertainty quantification methods, the present application provides more accurate material performance prediction while generating more reliable uncertainty estimates.

[0054] 2. The present application does not require maintaining a large number of model parameters, simplifying the complexity of the model. Even in small data sets, the model of the present application shows competitive performance, and under the condition of resource allowance, combined with deep integration method, more accurate material performance prediction and high-quality uncertainty estimation can be produced. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 is the overall structure diagram of the network of the present application combined with Transformer and Gaussian process;

[0056] Figure 2 is a structural diagram of the Transformer encoding module of the present application;

[0057] Figure 3is a different model uncertainty ranking and error change visualization chart on a single atom formation energy attribute data set in an embodiment of the application. DETAILED DESCRIPTION

[0058] The application will be further described below in conjunction with the drawings and specific embodiments.

[0059] Embodiment 1: An uncertainty quantification method for material composition and performance mapping, the specific steps are as follows:

[0060] Step 1: Collect the composition data of the material from the Materials Project (MP) database, which includes the atomic number of each element and the fraction in the material.

[0061] Step 2: Use a fully connected neural network to encode the atomic number and fraction into a high-dimensional vector to obtain an element-derived matrix, which contains the relationship information between elements, specifically:

[0062] Step 2.1: Stratified random sampling is performed on the unpartitioned data set, and the class proportion in different sets is fixed, and finally the training set, validation set and test set required for the experiment are extracted;

[0063] Step 2.2: The atomic number of each element is passed through a fully connected neural network layer to generate a corresponding high-dimensional vector representation, obtaining an element embedding matrix;

[0064] Step 2.3: The fraction of each element is encoded to generate a corresponding high-dimensional vector representation, obtaining a fraction embedding matrix;

[0065] Step 2.4: Combine the element embedding matrix and the fraction embedding matrix to form an element-derived matrix, and complete the preprocessing of the data set.

[0066] The element-derived matrix integrates the properties of each element and its proportion in the material, and is used to effectively represent the material composition information.

[0067] Step 3: Construct a Transformer module, input the element-derived matrix into the Transformer module, and learn the interaction of each element in the material and the chemical structure information of each element through the self-attention mechanism in the Transformer module, to obtain the chemical structure information feature.

[0068] The Transformer module is composed of multiple layers, each layer including a self-attention mechanism and a feedforward neural network;

[0069] In order to enhance the stability and effect of model training, two inter-layer enhancement mechanisms are introduced in the Transformer module: residual connection and layer normalization, which are designed to ensure that the network can still be effectively trained under deep structure.

[0070] The residual connection directly adds the input to the processed output, avoiding the loss or weakening of information when passing between layers.

[0071] The layer normalization is a standardization process for the output of each layer, which normalizes the mean and variance of the features, so that different batches of data can be trained on similar scales.

[0072] The Transformer module includes 3 layers of encoders, each of which contains 4 attention heads;

[0073] The attention head is used to capture the context information in the input data;

[0074] The enhancement mechanism between the layers of the encoder is that each sub-layer (self-attention mechanism and feedforward neural network) contains residual connection and layer normalization:

[0075]

[0076] In the formula, X is the feature vector input to this layer, with a dimension of R n×d , Z(X) is the output of the input x after processing by the previous layer (or part of the current layer), Z(X)+X represents the residual connection, LayerNorm represents the layer normalization, FFN is the feedforward neural network, F l is the encoded feature after multiple layers, which has the same dimension as X.

[0077] Step4: input the chemical structure information feature into the residual multi-layer perceptron (MLP) module, design spectral normalization in the residual multi-layer perceptron module to constrain the mapping of the weight matrix in the residual multi-layer perceptron module, improve its stability, and prevent the collapse of hidden space features, specifically:

[0078] Step4.1: the residual multi-layer perceptron module is composed of different residual blocks, denoted as:

[0079] h=h L-1 ·...·h2·h1 (1)

[0080] h l (F l )=F l +g l (F l ) (2)

[0081] g l (x) = a(W l x + b l ) (3)

[0082] Equation (1) represents the forward propagation process of the entire network is composed of multiple layers, that is, from h1 to h L-1 ;

[0083] In equation (2), F l is the input into the Lth layer, which can also be regarded as the feature representation passed from the previous layer (or several previous layers). g l (F l ) is the transformation of the input F l in the Lth layer, representing the new features after passing through this layer;

[0084] In equation (3), W l represents the weight matrix of the Lth layer, which is used to linearly transform the input feature x, a is the activation function, b l is the bias term, and g l (x) represents the nonlinear transformation of the Lth layer on the input x.

[0085] Step 4.2: Design spectral normalization, and perform spectral normalization on the weight matrix of each layer in the MLP to achieve the Lipschitz constraint on g l .

[0086] Specifically, the maximum eigenvalue of the weight matrix is estimated by power iteration , and the weight is normalized using the maximum eigenvalue.

[0087]

[0088] In the equation, c is a hyperparameter used to adjust the upper bound of the spectral norm of the weight matrix, so that ||W l ||2≤c, W l is the weight matrix of the current Lth layer, and λ is a threshold value representing a scalar value used for comparison under certain conditions.

[0089] Step 5: Construct a Gaussian model, optimize the parameters of the Gaussian model through the log posterior likelihood and Hessian matrix, then input the chemical structure information features in the residual multilayer perceptron module into the optimized Gaussian model for Gaussian process processing, and realize the quantification of the uncertainty of the mapping between material composition and performance.

[0090] The Gaussian process processing is specifically:

[0091] The prior distribution of the Gaussian model equipped with RBF kernel is a multivariate normal distribution:

[0092]

[0093] where σ 2 and l are hyperparameters, g N×1 is a random vector of length N representing the output of the Gaussian process, i.e., the predicted values, MVN is a multivariate normal distribution, 0 N×1 is a zero vector of length N indicating that the mean of the multivariate normal distribution is zero, K N×N is an N x N covariance matrix generated by the RBF kernel function, indicating the correlation between any two observation points, K i,j represents the element in the i-th row and j-th column of the covariance matrix, describing the similarity between the i-th and j-th input points, represents the square of the Euclidean distance between the i-th and j-th input points, L is the length scale of the RBF kernel Using random feature expansion to approximate the prior, define the kernel matrix K = ΦΦ T , the random feature Gaussian process is:

[0094]

[0095] where g N×1 is a random vector of length N representing the output of the Gaussian process, i.e., the predicted values; MVN is a multivariate normal distribution; 0 N×1 is a zero vector of length N indicating that the mean of the multivariate normal distribution is zero; is calculated by the random feature matrix Φ, where Φ is an N x D L matrix representing the feature mapping generated by the random feature method; Φ i represents the last layer of the residual network, with a dimension of D L , W L is a fixed weight matrix, sampled independently and identically distributed from N (0，1) , and b L is a fixed bias term;

[0096] Using random feature expansion with Gaussian prior to write a neural network layer composed of fixed hidden weights W L and learnable output weights β:

[0097]

[0098] where F o is the input feature vector, g(F o ) is the output of the neural network layer, calculated based on the input feature F o ; is a scaling factor, often used to normalize random features to ensure that the computed results are within a reasonable numerical range. L F o +b L is the input feature F o after linear transformation. The non-linear mapping uses a cosine function for non-linear expansion. b L is the bias vector, used to adjust the position in the feature space before non-linear mapping. β conforms to a multivariate normal distribution with mean zero and variance τ (where τ is a hyperparameter), is a D L × D L identity matrix.

[0099] The log posterior likelihood is specifically:

[0100]

[0101] In the formula, logp(β|x) represents the log posterior probability of parameter β given the input data x. This part is the term of the likelihood function, which measures the difference between the predicted value g i of the model and the true value y i . is the regularization term, called L2 regularization;

[0102] The Hessian matrix is specifically

[0103]

[0104] In the formula, is an estimate of the D L × D L Hessian matrix, Φ i is a feature vector, which is generated in the random feature expansion or neural network layer; This term represents the outer product of the feature vector Φ i , generating a D L × D L matrix, is a D L × D L identity matrix.

[0105] In order to verify the effect of the model of the present application on quantifying the uncertainty between the predicted material composition and the refractive index, the method proposed by the present application is compared with several existing uncertainty quantification methods, such as Representation Of Organics, Inorganics, and Synthesized Targets (Roost), Compositionally Restricted Attention-Based Neural Network (CrabNet), and the like. The specific steps are as follows:

[0106] First, the material composition data is collected from the Materials Project database; then, the atomic number and fraction in the data are encoded into a high-dimensional vector using a fully connected neural network; then, these high-dimensional vectors are combined into an element-derived matrix and input into a Transformer module to learn the chemical structure information, to obtain the chemical structure information features. Finally, the chemical structure information features are processed by spectral normalization residual MLP, and a Gaussian process modeling is applied in the output layer to quantify the uncertainty.

[0107] Meanwhile, in order to verify the effectiveness of the algorithm, the method proposed by the present application is also compared with the existing uncertainty quantification methods such as Roost, CrabNet, and TUPNet (single), and the like. The results show that the method TUPNet (ensemble) proposed by the present application has lower error and better uncertainty quantification performance. The comparison effect is shown in Tables 1 and 2 and the accompanying Figure 3 The single refers to using only one single model in the training and prediction process. The ensemble represents an ensemble model, that is, using a combination of multiple models in the training and prediction process, combining the prediction results of these models, for example, taking the average of the predicted values as the final prediction result.

[0108] Table 1: Prediction accuracy and uncertainty quantification quality evaluation on the refractive index attribute dataset

[0109]

[0110] Table 2: Prediction accuracy and uncertainty quantification quality evaluation on the single atom formation energy attribute dataset

[0111]

[0112] The specific embodiments of the present application are described in detail above in combination with the accompanying drawings, but the present application is not limited to the above-mentioned embodiments, and various changes can be made within the knowledge possessed by those skilled in the art without departing from the purpose of the present application.

Claims

1. A method of uncertainty quantification of material composition and property mapping, comprising: The method specifically comprises: Step1: collect the composition data of the material from the Materials Project database, the composition data including the atomic number of each element and the fraction in the material; Step2: encode the atomic number and the fraction into a high-dimensional vector using a fully connected neural network to obtain an element-derived matrix, the element-derived matrix containing the relationship information between elements; Step3: build a Transformer module, input the element-derived matrix into the Transformer module, learn the interaction of each element in the material and the chemical structure information of each element through the self-attention mechanism in the Transformer module, and obtain the chemical structure information feature; Step4: input the chemical structure information feature into a residual multilayer perceptron module, design spectral normalization in the residual multilayer perceptron module to constrain the mapping of the weight matrix in the residual multilayer perceptron module, prevent overfitting, and improve the generalization ability of the model; Step5: build a Gaussian model, optimize the parameters of the Gaussian model through the log posterior likelihood and the Hessian matrix, then input the chemical structure information feature in the residual multilayer perceptron module into the optimized Gaussian model for Gaussian process processing to realize the quantification of the uncertainty of the mapping between the composition and the performance of the material; The log posterior likelihood is specifically: ； Where, Representation parameters The logarithmic posterior probability given the input data x, It is a term about the likelihood function, which is used to measure the model prediction value and the true value The difference between is a regularization term, called L2 regularization; The Hessian matrix is specifically ； wherein is an estimate of the Hessian matrix of order is an estimate of the Hessian matrix of order is an eigenvector generated in a random feature expansion or a neural network layer; This term represents the outer product of the eigenvector and generates a matrix is an identity matrix of order is an identity matrix of order is an identity matrix of order 2. The method of uncertainty quantification of material composition and property map of claim 1, wherein, The Step2 is specifically: Step2.1: stratified random sampling is performed on the data set without division, the class proportion in different sets is fixed, and finally the training set, the validation set and the test set required for the experiment are extracted; Step2.2: the atomic number of each element is generated through a fully connected neural network layer to generate a corresponding high-dimensional vector representation to obtain an element embedding matrix; Step2.3: the fraction of each element is generated through fraction encoding to generate a corresponding high-dimensional vector representation to obtain a fraction embedding matrix; Step2.4: the element embedding matrix and the fraction embedding matrix are combined to form an element-derived matrix, and the preprocessing of the data set is completed.

3. The method of uncertainty quantification of material composition and property maps of claim 1, wherein: The Transformer module is composed of multiple layers, each layer including a self-attention mechanism and a feedforward neural network; Two inter-layer enhancement mechanisms are introduced in the Transformer module: residual connection and layer normalization; the residual connection directly adds the input to the processed output; The layer normalization is a standardization processing of the output of each layer, which makes different batches of data trained on similar scales by standardizing the mean and variance of the features.

4. The method of uncertainty quantification of material composition and property maps of claim 1, wherein: The Step4 is specifically: Step4.1: the residual multilayer perceptron module is composed of different residual blocks, represented as: （1）； （2）； （3）； Formula (1) shows that the forward propagation process of the entire network is composed of multiple layers, namely arrive combination of; In formula (2), is the input into the Lth layer, is the transformation performed on the input in the Lth layer, which represents the new features after this layer. In formula (3), represents a weight matrix of the Lth layer for linear transformation of the input feature x, a is an activation function, is a bias term, (x) represents a non-linear transformation of the Lth layer on the input x; Step 4.2: Design spectral normalization to impose a Lipschitz constraint on the weight matrices of each layer in the MLP Step 4.2: Design spectral normalization to impose a Lipschitz constraint on the weight matrices of each layer in the MLP Step 4.2: Design spectral normalization to impose a Lipschitz constraint on the weight matrices of each layer in the MLP The spectrum normalization is to estimate the largest eigenvalue of the weight matrix by power iteration , and normalize the weight by the largest eigenvalue. ； where c is a hyper-parameter to adjust the upper bound of the spectral norm of the weight matrix such that , is the weight matrix of the current L-th layer, is a threshold value representing a scalar value used for comparison under certain conditions.

5. The method of uncertainty quantification of material composition and property maps of claim 3, wherein, The Transformer module includes 3 encoder layers, each encoder layer containing 4 attention heads, which are used to capture the context information in the input data; The enhancement mechanism between the layers of the encoder is specifically that each sub-layer contains a residual connection and a layer normalization: ； where X is the input feature vector to this layer with dimension , is the output of the input x after processing by the previous layer, denotes the residual connection, denotes layer normalization, is a feedforward neural network, is the final output encoded feature with the same dimension as X.

6. The method of uncertainty quantification of material composition and property maps of claim 1, wherein, The Gaussian process processing is specifically: The prior distribution of the Gaussian model equipped with RBF kernel is a multivariate normal distribution: ； where, and are hyperparameters, is a random vector of length N representing the output of the Gaussian process, i.e., the predicted values, MVN is a multivariate normal distribution, is a zero vector of length N representing the mean of the multivariate normal distribution is zero, is an N x N covariance matrix generated by the RBF kernel function, representing the correlation between any two observation points, represents the element in the ith row and jth column of the covariance matrix, describing the similarity between the ith and jth input points, represents the square of the Euclidean distance between the ith and jth input points, l is the length scale of the RBF kernel, and the kernel matrix is defined as The random feature Gaussian process is: = ； where, is a random vector of length N representing the output of the Gaussian process, i.e., the predicted values; MVN is a multivariate normal distribution; is a zero vector of length N representing the mean of the multivariate normal distribution being zero; is computed from the random feature matrix where is an N by N matrix representing the feature map generated by the random feature method; represents the last layer of the residual network with dimension , is a fixed weight matrix sampled independently and identically distributed from , is a fixed bias term; Neural network layers composed of fixed hidden weights and learnable output weights using random feature expansion with Gaussian priors: ； Where, is the input feature vector, is the output of the neural network layer, based on the input features Calculated; is a scaling factor, is the input feature After the nonlinear mapping of linear transformation, is a fixed bias term, The mean is zero and the variance is The multivariate normal distribution of Is the size of The identity matrix of .

Images