Learning device, learning method, and program

Abstract

A learning device according to one embodiment of the present disclosure is for learning a neural network having a parameter containing a continuous function on a compact Lie group as an element thereof. The learning device includes: an input unit for inputting learning data for learning the neural network; a first approximation unit for approximating input data included in the learning data by linear combination of a finite number of matrix elements on the compact Lie group; a second approximation unit for approximating each element included in the parameter by linear combination of the matrix elements; a third approximation unit for approximating a loss function value when the input data is inputted to the neural network by linear combination of the matrix elements; and an update unit for updating the parameter so as to minimize the norm of a loss function value approximated by linear combination of the matrix elements.

Description

Learning device, learning method, and program

[0001] The present disclosure relates to a learning device, a learning method, and a program.

[0002] In recent years, various anomaly detection methods using deep learning have been proposed (Non-Patent Document 1). Deep learning is a fundamental tool of artificial intelligence, and generally uses a neural network consisting of multiple intermediate layers.

[0003] In recent years, research has been conducted on the continuation of neural networks, and as a framework for generalizing existing neural networks, the parameters of neural networks are ＊ Ring-extended C ＊ A ring network has been proposed (Non-Patent Document 2). ＊ The rings used are mainly those consisting of continuous functions on compact sets.

[0004] Ryoichi Kawahara, Keishiro Watanabe, Kaoru Harada, and Takehiro Kawada, "AI Application to Network Operations," Communications Society Magazine No. 45, Summer 2018, pp. 29-38. Yuka Hashimoto, Zhao Wang, and Tomoko Matsui, "C^*-algebra Net: A New Approach Generalizing Neural Network Parameters to C^*-algebra." International Conference on Machine Learning, Baltimore, Maryland, USA, PMLR 162, 2022.

[0005] C ＊ Since ring networks can take on an infinite number of element values ​​on a compact set, it is necessary to approximate the loss function value when computing it. However, this approximation may not be appropriate, resulting in problems such as C ＊ The accuracy of anomaly detection using a ring network may decrease.

[0006] The present disclosure has been made in view of the above points, ＊The purpose is to improve the performance of the ring network.

[0007] A learning device according to one aspect of the present disclosure is a learning device that trains a neural network having parameters whose elements are continuous functions on a compact Lie group, and includes: an input unit that inputs learning data for training the neural network; a first approximation unit that approximates input data included in the learning data with a linear combination of a finite number of matrix elements on the compact Lie group; a second approximation unit that approximates each element included in the parameters with a linear combination of the matrix elements; a third approximation unit that approximates a loss function value when the input data is input to the neural network with the linear combination of the matrix elements; and an update unit that updates the parameters so as to minimize the norm of the loss function value approximated by the linear combination of the matrix elements.

[0008] C ＊ The performance of the ring network can be improved.

[0009] FIG. 1 is a diagram illustrating an example of the hardware configuration of a learning device according to the present embodiment; FIG. 2 is a diagram illustrating an example of the functional configuration of a learning device according to the present embodiment; FIG. 3 is a flowchart illustrating an example of a model learning process according to the present embodiment; FIG. 4 is a diagram illustrating an example of the functional configuration of an anomaly detection device according to an application example; FIG. 5 is a flowchart illustrating an example of an anomaly detection process according to an application example; FIG. 6 is a diagram illustrating loss function values ​​in Experimental Example 1; FIG. 7 is a diagram illustrating the accuracy rate in Experimental Example 1; and FIG. 8 is a diagram illustrating loss function values ​​in Experimental Example 2.

[0010] An embodiment of the present invention will be described in detail below with reference to the drawings. ＊ A learning device 10 that can improve the performance of a ring network will be described. ＊ The performance of the ring network is ＊ The term "learning" refers to the accuracy achieved when optimizing a network through learning (i.e., how accurately it can solve a target task such as anomaly detection or classification problem). Note that learning may also be called "training," for example.

[0011] <Theoretical structure> Let x∈R be a finite-dimensional vector.Ｄ_１ Here, R is the set of all real numbers. Also, D_1 is "D １ " and is a predetermined natural number. In this case, f θ_１ (x) = σ(Wx + b) to obtain the function f θ_１ where W is a linear mapping between finite-dimensional vector spaces, b is a vector called bias, and σ is an activation function (for example, ReLU function). Also, "θ_1" is defined as "θ １ " and θ １ is a collection of W and b (i.e., a set of elements of W and elements of b).

[0012] f θ_１ Similarly, f θ_２ , ..., f θ_Ｎ In this case, a deep learning model is generally defined as a function f θ It is expressed by:

[0013] Note that θ is θ １ , ..., θ Ｎ (i.e., θ = {θ １ , ..., θ Ｎ}) Hereinafter, the parameter θ is represented by a D-dimensional vector composed of all elements of W and b.

[0014] The function f shown in the above formula (1) θ The parameter θ can be found by solving minimization problems of various loss functions (for example, mean square error, cross entropy, log-likelihood function, etc.).

[0015] The function f shown in the above formula (1) θ As an extension of the deep learning model represented by ＊ There is a ring network. ＊ Let the ring be A. In this case, C ＊ The ring network is a network in which the input vector x is Ｄ_１ Furthermore, the elements of each linear mapping W and the elements of each bias b are expressed as C ＊ It is an extension to the ring A. The parameter θ is C ＊ By extending to the ring A, the function f θIt has succeeded in increasing the expressive power of C. ＊ As the ring A, a ring C(Z) consisting of continuous functions on a compact set Z is mainly used.

[0016] Below, C ＊ The ring network is assumed to consist of many intermediate layers, and C ＊ In the following, a deep learning model is a neural network consisting of many hidden layers, and its parameters are set as C ＊ This refers to the model extended to a ring. ＊ Let A be a ring C(Z) consisting of continuous functions on a compact set Z.

[0017] C ＊ In a ring network, the loss function is generally L:C(Z) Ｄ →C(Z), but L(θ) is often parameterized by z∈Z. That is, a function L':R Ｄ × Z → R exists, and C(Z) ∋ L(θ): Z → R; L(θ)(z) = L'(θ(z), z). In this case, to add interactions for each z∈Z, regularization is added to the loss function, and the following formula (2) or the following formula (3) may be used as the loss function.

[0018]

[0019] Here, in the above equation (2), 1 is the identity element of C(Z), and λ is the regularization parameter.

[0020] However, C ＊ Since the ring network can have an infinite number of z∈Z, when calculating the integrals in the above formula (2) and the integrals in the above formula (3) on a computer, we select some points z from the compact set Z. １ , ..., z Ｋ is sampled and substituted by the following equation (4).

[0021] That is, the integral of L(θ)(z) with respect to zεZ is approximated by the above equation (4).

[0022] Note that, for example, a density estimation problem is an example of a case where L(θ) is parameterized over z∈Z (Non-Patent Document 2). In the density estimation problem, L(θ) is parameterized by the following equation (5) using a negative log-likelihood function for each z∈Z.

[0023] Here, x ｍ is a data point, p θ，ｚ (x ｍ ) is the parameter θ∈C(Z) of the deep learning model Ｄ is the density estimated value at point z∈Z. θ，ｚ The design method varies depending on the problem being considered.

[0024] As described above, when calculating the integral of L(θ)(z) with respect to z∈Z on a computer, it is approximated by the above formula (4). On the other hand, the value of the above formula (4) is obtained by the sampling point z １ , ..., z Ｋ Therefore, the above formula (4) may not be suitable as an approximation of the integral of L(θ)(z) with respect to z∈Z, and the deep learning model f θ may exhibit behavior not intended by the designer. ＊ This creates a problem in that the accuracy of anomaly detection using a ring network may decrease.

[0025] Therefore, we propose a method that does not require sampling as a method for solving the above problem. Note that, in the following, it is assumed that the integral of L(θ)(z) with respect to z∈Z (i.e., the above formula (3)) is used as the loss function.

[0026] Let Z be a compact Lie group, and C ＊ The ring A is assumed to be a ring C(Z) consisting of continuous functions on a compact Lie group Z. In the following, the loss function L is assumed to be parametrized by z∈Z.

[0027] By the Peter-Weyl theorem, any f∈C(Z) is a matrix element {φ ｊ It is known that it can be uniformly approximated by a linear combination of V = span {φ －Ｊ, ..., φ Ｊ} where J is a predetermined natural number.

[0028] In this case, the proposed method is composed of the following steps 1 to 4.

[0029] Step 1: First, let the input vector x be x∈V. Ｄ_１ That is, x = (x １ , ..., x Ｄ_１ ), for each d∈{1,...,D １}, x ｄ is expressed by the following equation (6).

[0030] Here, a ｄ ｊ are the coefficients of the linear combination and are the parameters to be learned.

[0031] Step 2: Next, C ＊ Each parameter of the ring network (i.e., the elements of W and b) is also expressed as a linear combination of elements of V, and the coefficients of each basis function are used as the parameters to be learned. That is, since the elements of W and b are elements of C(Z), the matrix element {φ ｊ}, a uniform approximation is possible. Therefore, each element of W and b is expressed as {φ －Ｊ , ..., φ Ｊ}, and the coefficients of the linear combination are used as the parameters to be learned.

[0032] Step 3: Next, the loss function L(θ)εC(Z) is calculated in the same way as in the prior art, and is expressed by the following equation (7).

[0033] Here, a ｊ (θ) is the coefficient of the linear combination.

[0034] Step 4: Finally, a new loss function is calculated using the loss function L shown in the following equation (8): ｎｅｗ This loss function L ｎｅｗ The parameter θ is updated to minimize

[0035] That is, the loss function L shown in the above equation (8) ｎｅｗ Let (θ) be the approximation of equation (3) above.

[0036] As a result, the loss function L(θ) can be approximated without using sampling. In this case, increasing the value of J improves the approximation accuracy of the loss function L(θ), making it possible to build a more accurate deep learning model.

[0037] However, increasing the value of J increases the computer's memory consumption and calculation time, so the value of J must be determined depending on the desired accuracy of the deep learning model (e.g., depending on the desired accuracy of anomaly detection).

[0038] <Example of Hardware Configuration of Learning Device 10> An example of the hardware configuration of the learning device 10 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the hardware configuration of the learning device 10 according to this embodiment.

[0039] 1, the learning device 10 according to this embodiment is realized by the hardware configuration of a general computer, and includes an input device 101, a display device 102, an external I / F 103, a communication I / F 104, a RAM (Random Access Memory) 105, a ROM (Read Only Memory) 106, an auxiliary storage device 107, and a processor 108. Each of these pieces of hardware is connected to each other via a bus 109 so as to be able to communicate with each other.

[0040] The input device 101 is, for example, a keyboard, a mouse, a touch panel, physical buttons, etc. The display device 102 is, for example, a display, a display panel, etc. Note that the learning device 10 does not necessarily have to have at least one of the input device 101 and the display device 102, for example.

[0041] The external I / F 103 is an interface with an external device such as a recording medium 103a. Examples of the recording medium 103a include a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), and a USB (Universal Serial Bus) memory card.

[0042] The communication I / F 104 is an interface for connecting to a communication network. The RAM 105 is a volatile semiconductor memory (storage device) that temporarily stores programs and data. The ROM 106 is a non-volatile semiconductor memory (storage device) that can store programs and data even when the power is turned off. The auxiliary storage device 107 is a non-volatile storage device such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. The processor 108 is a variety of arithmetic devices such as a central processing unit (CPU) or a graphic processing unit (GPU).

[0043] 1 is an example, and the hardware configuration of the learning device 10 is not limited to this. For example, the learning device 10 may have multiple auxiliary storage devices 107 or multiple processors 108, may not have some of the hardware shown in the figure, or may have various hardware other than the hardware shown in the figure.

[0044] <Example of Functional Configuration of Learning Device 10> An example of the functional configuration of the learning device 10 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the functional configuration of the learning device 10 according to this embodiment.

[0045] 2, the learning device 10 according to this embodiment includes a learning dataset input unit 201 and a model learning unit 202. These units are realized, for example, by processing in which one or more programs installed in the learning device 10 are executed by the processor 108 or the like. The learning device 10 according to this embodiment also includes a learning dataset storage unit 203 and a model storage unit 204. These storage units are realized, for example, by storage areas in the auxiliary storage device 107 or the like.

[0046] The training dataset input unit 201 inputs a training dataset stored in the training dataset storage unit 203. The training dataset is a collection of training data used for training a deep learning model stored in the model storage unit 204. When training a deep learning model by unsupervised learning, the training dataset is {x （ｉ） ∈R Ｄ_１ |i=1,...,I}. On the other hand, when training a deep learning model using supervised learning, the training dataset is expressed as {(x （ｉ） , y （ｉ） ) |i=1,...,I}, where I is the number of training data. （ｉ） Ha x （ｉ） is the correct label (teaching data) corresponding to

[0047] When learning a deep learning model using unsupervised learning, the learning data x （ｉ） is the input vector to the deep learning model. On the other hand, when learning a deep learning model using supervised learning, the learning data (x （ｉ） , y （ｉ） ) x included in （ｉ） is the input vector input to the deep learning model.

[0048] An example of a case where a deep learning model is learned by unsupervised learning is when the deep learning model is applied to anomaly detection, etc. In this case, each input vector x （ｉ） For example, traffic data in the normal state of the system to be detected for anomalies is used as the input vector x. On the other hand, when a deep learning model is trained by supervised learning, for example, a deep learning model is applied to an image classification problem. In this case, each input vector x （ｉ） The image is represented by a vector, y ｉ However, traffic data is just one example, and any data (e.g., metrics data) obtained from the system that is the target of anomaly detection can be used.

[0049] The model learning unit 202 uses the learning dataset input by the learning dataset input unit 201 to learn the deep learning model stored in the model storage unit 204. That is, the model learning unit 202 updates the parameter θ of the deep learning model by steps 1 to 4 of the proposed method.

[0050] In addition, when a deep learning model is learned by unsupervised learning, the deep learning model is, for example, a model in which each parameter is an element of C(Z) and the dimensions (widths) of the input layer and the output layer are both D １ , and the dimension of the hidden layer is D １ C smaller than ＊ A ring network (also called an "autoencoder") is used. The loss function L(θ) is, for example, （ｉ） and its input vector x （ｉ） The mean square error between the input vector and the output vector when inputting the vector into a deep learning model (also called the "reconstruction error") is used.

[0051] On the other hand, when a deep learning model is trained by supervised learning, the deep learning model is, for example, a C ＊ Ring network (e.g., C using a fully connected layer in the middle layer and a softmax function in the output layer activation function) ＊ A loss function L(θ) may be, for example, a cross entropy.

[0052] The learning dataset storage unit 203 stores a learning dataset. The model storage unit 204 stores a deep learning model (that is, a function f θ ) is memorized.

[0053] <Model Learning Process> The model learning process according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of the model learning process according to this embodiment.

[0054] The learning dataset input unit 201 inputs a learning dataset stored in the learning dataset storage unit 203 (step S101).

[0055] The model learning unit 202 uses the learning data set input in step S101 above to learn the deep learning model stored in the model storage unit 204 (step S102).

[0056] For example, when a deep learning model is trained by online training, the model training unit 202 uses the input vector x （ｉ） For each step, the parameter θ of the deep learning model is updated by the above steps 1 to 4. Alternatively, for example, when the deep learning model is trained by mini-batch training, the model training unit 202 updates the parameter θ of each input vector x included in the mini-batch. （ｉ） After performing the above procedure 1 on x, the parameter θ of the deep learning model is updated by the above procedures 2 to 4. Alternatively, for example, when training a deep learning model by batch training, the model training unit 202 performs the following steps on each input vector x included in the training data set: （ｉ） After performing the above procedure 1 on , the parameter θ of the deep learning model is updated by the above procedures 2 to 4. These parameter updates are performed when a predetermined termination condition is satisfied (for example, the number of updates exceeds a predetermined number, the parameters converge, or the loss function L ｎｅｗ This is repeated until the value of x is less than a predetermined threshold. （ｉ） The above procedure 1 may be executed in advance for the

[0057] [Application Example] As an application example, an anomaly detection device 20 that realizes anomaly detection using a deep learning model (hereinafter also referred to as a "trained deep learning model") trained by the learning device 10 according to the above embodiment will be described below. Note that the trained deep learning model in this application example is an autoencoder that has been trained so as to minimize the reconstruction error by unsupervised learning.

[0058] <Example of functional configuration of anomaly detection device 20> An example of the functional configuration of an anomaly detection device 20 according to one application example will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the functional configuration of an anomaly detection device 20 according to one application example. Note that the anomaly detection device 20 according to one application example receives an input vector x∈A, such as traffic data acquired from a system that is the target of anomaly detection. Ｄ_１ is given as the anomaly detection target data x. This makes it possible to detect an anomaly such as a communication anomaly in the anomaly detection target system.

[0059] 4 , the anomaly detection device 20 according to one application example includes an anomaly detection target data input unit 301, an anomaly detection unit 302, and an anomaly detection result output unit 303. These units are realized, for example, by a processor such as a CPU or a GPU executing one or more programs installed in the anomaly detection device 20. The anomaly detection device 20 according to one application example also includes a trained model storage unit 304. The trained model storage unit 304 is realized, for example, by a storage area of ​​an auxiliary storage device such as an HDD, SSD, or flash memory.

[0060] The abnormality detection target data input unit 301 inputs the abnormality detection target data x every time the abnormality detection target data x is given.

[0061] The anomaly detection unit 302 performs anomaly detection using the anomaly detection target data x input by the anomaly detection target data input unit 301, with the trained deep learning model stored in the trained model storage unit 304. That is, the anomaly detection unit 302 calculates the reconstruction error when the anomaly detection target data x is input to the trained deep learning model, and determines that there is an anomaly if the reconstruction error is greater than a predetermined threshold, and that there is no anomaly if not.

[0062] The anomaly detection result output unit 303 outputs the anomaly detection result obtained by the anomaly detection unit 302 to a predetermined output destination. Examples of the output destination include a display device such as a display, a storage device such as an auxiliary storage device, and a device or apparatus connected to the anomaly detection device 20 so as to be able to communicate with the anomaly detection device 20 (e.g., a control device that controls the operation of the system in accordance with the anomaly detection result).

[0063] The trained model storage unit 304 stores a trained deep learning model used for anomaly detection. Note that the trained deep learning model used for anomaly detection may be called, for example, an "anomaly detection model."

[0064] <Abnormality Detection Processing> The abnormality detection processing according to one application example will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of the abnormality detection processing according to one application example.

[0065] The abnormality detection target data input unit 301 inputs given abnormality detection target data x (step S201).

[0066] The anomaly detection unit 302 uses the anomaly detection target data x input in step S201 above to perform anomaly detection using the trained deep learning model stored in the trained model storage unit 304 (step S202).

[0067] The abnormality detection result output unit 303 outputs the abnormality detection result obtained in step S202 to a predetermined output destination (step S203).

[0068] [Experimental Examples] Below, experimental examples 1 and 2 using MNIST data will be described as experimental examples of the learning device 10 according to the above embodiment. Experimental example 1 describes a case where a fully connected network is used as a deep learning model to solve a number classification problem, while experimental example 2 describes a case where an autoencoder is used as a deep learning model to reconstruct an image. In both experimental examples 1 and 2, J = 5. Note that MNIST data refers to image data of handwritten black and white numbers from 0 to 9.

[0069] Experimental Example 1 In this experimental example, a deep learning model was constructed that uses MINIST data as input and predicts the number written in an image represented by the MINIST data. Specifically, the model was a fully connected network with an input layer dimension of 28 x 28, intermediate layer dimensions of 28 x 28, 20 x 20, and 10 x 10, and an output layer dimension of 10, and each element of the intermediate layer was {φ －５ , φ －４ , ..., φ ４ , φ ５}, and the activation functions of the intermediate layer are all ReLU functions, and the activation function of the output layer is softmax function. ＊ The ring network was constructed as a deep learning model. The output is a 10-dimensional vector, and the value of each element of the vector represents a score or probability that indicates the predicted result of a number from 0 to 9.

[0070] The loss function used was the cross entropy between the vector output by the deep learning model and the one-hot vector represented by the correct label. Adam was used as the optimization algorithm.

[0071] The results of training a deep learning model using steps 1 to 4 above are shown in Figures 6 and 7. Figure 6 is a graph showing the loss function values ​​during training and testing for each epoch. Figure 7 is a graph showing the accuracy rate during training and testing for each epoch.

[0072] As shown in Figures 6 and 7, the loss function value decreases and the accuracy rate improves as the epochs progress, indicating that the deep learning model is progressing well.

[0073] Experimental Example 2 In this experimental example, an autoencoder that uses MINIST data as input and reconstructs the MINIST data was configured as a deep learning model. Specifically, it was a fully connected network with an input layer dimension of "28 x 28", intermediate layer dimensions of "28 x 28", "10 x 10", and "5 x 5", and an output layer dimension of "2", and each element of the intermediate layer was {φ －５ , φ －４ , ..., φ ４ , φ ５} and all activation functions are ReLU functions. ＊ The ring network is an encoder, the input layer dimension is "2", the intermediate layer dimension is "5x5", "10x10", "28x28", and the output layer dimension is "28x28". The elements of the intermediate layer are {φ －５ , φ －４ , ..., φ ４ , φ ５} and all activation functions are ReLU functions. ＊ A ring network was used as the decoder, and an autoencoder consisting of an encoder and decoder was constructed as the deep learning model.

[0074] The loss function used was the mean square error between the MINIST data and the vector output when the MINIST data was input to the deep learning model. Adam was used as the optimization algorithm.

[0075] The results of training a deep learning model using the above steps 1 to 4 are shown in Figure 8. Figure 8 is a graph showing the loss function values ​​during training for each epoch.

[0076] As shown in Figure 8, the loss function value decreases as the epoch progresses, indicating that the deep learning model is progressing well.

[0077] [Summary] As described above, the learning device 10 according to this embodiment ＊ When training a deep learning model realized by a ring network, it is possible to approximate the loss function L(θ) without sampling points in the compact set Z. Moreover, by increasing the value of J, it is possible to improve the approximation accuracy of the loss function L(θ), making it possible to build a highly accurate deep learning model. Note that in the above proposed method, points in the compact set Z are not sampled, but a finite number of matrix elements {φ ｊ It should be noted that this method can be implemented on a computer as a program because it is a method for uniformly approximating any f∈C(Z) by a linear combination of {}.

[0078] The present invention is not limited to the above-described specifically disclosed embodiments, and various modifications, changes, and combinations with known technologies are possible without departing from the scope of the claims.

[0079] 10 Learning device 20 Anomaly detection device 101 Input device 102 Display device 103 External I / F 103a Recording medium 104 Communication I / F 105 RAM 106 ROM 107 Auxiliary storage device 108 Processor 109 Bus 201 Learning dataset input unit 202 Model learning unit 203 Learning dataset storage unit 204 Model storage unit 301 Anomaly detection target data input unit 302 Anomaly detection unit 303 Anomaly detection result output unit 304 Trained model storage unit

Claims

1. A learning device for training a neural network having parameters whose elements are continuous functions on a compact Lie group, comprising: an input unit that inputs training data for training the neural network; a first approximation unit that approximates input data included in the training data with a linear combination of a finite number of matrix elements on the compact Lie group; a second approximation unit that approximates each element included in the parameters with a linear combination of the matrix elements; a third approximation unit that approximates a loss function value when the input data is input to the neural network with the linear combination of the matrix elements; and an update unit that updates the parameters so as to minimize the norm of the loss function value approximated by the linear combination of the matrix elements.

2. The learning device according to claim 1, wherein the loss function is a function that is parametrized by elements on the compact Lie group.

3. A learning method in which a computer that trains a neural network having parameters whose elements are continuous functions on a compact Lie group executes the following steps: an input procedure for inputting training data for training the neural network; a first approximation procedure for approximating the input data included in the training data with a linear combination of a finite number of matrix elements on the compact Lie group; a second approximation procedure for approximating each element included in the parameters with a linear combination of the matrix elements; a third approximation procedure for approximating a loss function value when the input data is input to the neural network with a linear combination of the matrix elements; and an update procedure for updating the parameters so as to minimize the norm of the loss function value approximated by the linear combination of the matrix elements.

4. A program that causes a computer that trains a neural network having parameters whose elements are continuous functions on a compact Lie group to execute the following steps: an input procedure for inputting training data for training the neural network; a first approximation procedure for approximating input data included in the training data with a linear combination of a finite number of matrix elements on the compact Lie group; a second approximation procedure for approximating each element included in the parameters with a linear combination of the matrix elements; a third approximation procedure for approximating a loss function value when the input data is input to the neural network with a linear combination of the matrix elements; and an update procedure for updating the parameters so as to minimize the norm of the loss function value approximated by the linear combination of the matrix elements.

Images