Information processing device, training method, and program

Abstract

This information processing device for training a model using a neural network includes: a perturbation generation unit for generating perturbations to a parameter of the model from a distribution designed so that an output perturbation appearing in an output from the model becomes isotropic; and a training unit for updating a parameter of the model by using an approximate gradient calculated by using the perturbation.

Description

Information processing device, training method, and program

[0001] The present invention relates to techniques for training neural networks.

[0002] In recent years, optical neural networks (ONNs) using programmable integrated circuits have been attracting attention as neural networks capable of high-speed processing with low energy consumption.

[0003] The ONN is a hardware circuit, and there are manufacturing variations that cannot be observed. The ONN also has a repeating structure such as a Mach-Zehnder interferometer array (MZI array).

[0004] Due to the above-described characteristics of ONNs, conventional techniques that use backpropagation to calculate gradients have been unable to properly train ONNs. As an alternative approach when backpropagation cannot accurately calculate gradients, the use of zeroth-order (ZO) optimization, a type of black-box optimization (e.g., Non-Patent Document 1), has been proposed. ZO optimization is considered to be the mainstream method for training ONNs.

[0005] In conventional ZO optimization, training is performed by applying perturbations sampled from a normal distribution with a unit matrix as the covariance matrix to the ONN parameters. Note that perturbations are small changes that generally have random values. In ZO optimization, perturbations are applied to the parameters to search for better parameter values ​​during training.

[0006] S. Liu, P.-Y. Chen, B. Kailkhura, G. Zhang, AO Hero III, and PK Varshney. 2020. A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications. IEEE Signal Processing Magazine 37, 5 (2020), 43- 54.

[0007] However, in an ONN with layered modules, the parameters are interrelated, so conventional ZO optimization cannot perform optimal training. Note that this problem is not limited to neural networks with hardware circuits such as ONNs, but can occur in neural networks in general.

[0008] The present invention has been made in view of the above points, and has an object to provide a technique for more efficiently training a neural network.

[0009] According to the disclosed technology, there is provided an information processing device that trains a model using a neural network, comprising: a perturbation generation unit that generates perturbations for parameters of the model from a distribution designed so that output perturbations appearing in the output from the model are isotropic; and a training unit that updates the parameters of the model using an approximate gradient calculated using the perturbations.

[0010] The disclosed technology provides a technique for more efficiently training neural networks.

[0011] 1 is a diagram illustrating an example of the structure of an ONN. A diagram summarizing the meanings of characters. A diagram illustrating an example of a module used in a neural network. A diagram for explaining an example of parameter perturbation. A diagram for explaining an example of parameter perturbation. A diagram for explaining an example of parameter perturbation. A diagram illustrating Algorithm 1. A diagram illustrating an example of the configuration of an optimization device 100. A flowchart for explaining the operation of the optimization device 100. A diagram illustrating a detailed configuration of a perturbation generation unit 110. A diagram illustrating a detailed configuration of a smoothed FIM calculation unit 114. A diagram illustrating default settings of hyperparameters. A diagram illustrating settings of an image classification task in an experiment. A diagram illustrating experimental results. A diagram illustrating experimental results. A diagram illustrating experimental results. A diagram illustrating experimental results. A diagram illustrating an example of the configuration of an information processing device 200. A diagram illustrating an example of the configuration of an information processing device 300. A diagram illustrating an example of the configuration of an information processing device 400. A diagram illustrating an example of the hardware configuration of an apparatus.

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.

[0013] In this embodiment, an ONN is taken as the subject of training (which may also be called "learning"), but this is merely one example. For example, the subject of training may be an analog circuit other than an ONN, in which internal manufacturing variations cannot be observed, or a circuit other than an ONN, which has a repetitive structure such as an MZI array. Furthermore, the subject of training is not limited to a hardware circuit such as an ONN, but may also be a general neural network that operates as software on a computer.

[0014] (Problems and Overview of Technology According to the Present Embodiment) First, problems related to the technology according to the present embodiment will be described in more detail, and then an overview of the technology according to the present embodiment will be described.

[0015] As mentioned above, ONNs, which are programmable integrated circuits, are attracting attention as a promising system because they are capable of high-speed processing with low energy consumption.

[0016] Figure 1 shows an example of the structure of an ONN consisting of a linear module and a nonlinear module. As shown in Figure 1, the linear module (module u) of the ONN consists of an array of Mach-Zehnder interferometers (MZIs) connected by a waveguide. The MZIs have two sets of phase shifters (PSs) with a phase parameter θ and a beam splitter (BS). Figure 1 also shows that the nonlinear module (module u') also has parameters. The programmability (or trainability) of the ONN is achieved by adjusting these parameters. Hereinafter, a device with an ONN may be referred to as a "chip."

[0017] Typically, the primary approach to training neural networks (NNs) is to use backpropagation to calculate the gradient of the loss function with respect to the parameters. However, when training ONNs, backpropagation cannot accurately calculate the gradient. This is because each ONN circuit implemented on a chip has manufacturing variations. As shown in the bottom part of Figure 1, we model such variations using errors from ideal component characteristics (e.g., split angle error γ∈R, damping phase error ξ∈C).

[0018] Because it is impossible to accurately observe the complete information inside the chip, black-box optimization methods have been proposed as training methods for ONNs. Known black-box optimization methods include genetic algorithms, particle swarm optimization, bacterial foraging optimization, and zero-order (ZO) optimization. Among these methods, ZO optimization has become the dominant approach because experiments have shown that it is clearly superior to particle swarm optimization.

[0019] In ZO optimization, parameters are randomly perturbed to search for optimal parameters. The random parameter perturbations are typically sampled from a normal distribution N(0,I) with unit covariance matrix I. This means that perturbations are applied independently to each parameter, whether for the entire neural network or for each module. This is reasonable when each parameter is responsible for each input-output pair independently, as in the case of a linear module in a typical neural network.

[0020] However, the parameters of the linear modules in an ONN are not independent but are interrelated through a layered structure as shown in the top part of Figure 1. "Interrelated" means that a change in a parameter can be approximated by a change in its upstream / downstream parameters.

[0021] That is, when parameters in a neural network are correlated, parameter perturbation from a normal distribution with a unit matrix as its covariance matrix is ​​not necessarily optimal.

[0022] In this embodiment, we apply a new concept called "layered parameter perturbation" that is suitable for the parameters of ONN. Specifically, we apply the parameters of a module u with a layered structure to the covariance matrix Σ calculated by the new method in this embodiment. ｕ Normal distribution N(0,Σ ｕ ) The covariance matrix (i.e., normal distribution) of the output vector y ｉｕ (Fig. 1) is designed to be as isotropic as possible. As a result, we can uniformly search for better values ​​at the output of the linear module, which allows us to better train the entire ONN in a black-box setting with limited information. Note that the output vector y ｉｕ As long as the perturbation of becomes isotropic, the distribution for perturbing the parameters is not limited to the normal distribution, and may be a distribution other than the normal distribution.

[0023] In this embodiment, an example will be described in which the layered parameter perturbation technique is applied to each module u, but this technique can also be extended to the entire ONN.

[0024] In order to facilitate understanding of the technology according to this embodiment, we will first explain NN training, ZO optimization, and the Jacobian matrix, and then explain the method according to this embodiment in detail.

[0025] Figure 2 shows a summary of the meanings of the letters used in the following explanation. In the text of this specification, for convenience of description, bold letters are not used for vectors or matrices, but it is clear from the context that they indicate vectors or matrices. Also, normal typeface letters are used for letters representing sets. Furthermore, "_" is used as a subscript for subscripts (or superscripts). "C Ｍ＿ｕ " is an example. Also, the symbol T is used to represent transposition.

[0026] (ONN) As shown in Figure 1, ONN has a linear module and a nonlinear module. ｕ real-valued parameter θ of dimension ｕ Overall, ONN has N = Σ ｕ N ｕ Parameters θ = [θ １ , ..., θ Ｎ ]∈R Ｎ The ONN implements a function f and has an input vector x ｉ ∈C Ｋ complex-valued output vector y ｉ = f(x ｉ , θ)∈C Ｍ Note that ONN is an example of a model using a neural network. For module u, the function f ｕ Implements the input vector x ｉｕ ∈C Ｋ＿ｕ For output vector y ｉｕ = f ｕ (x ｉｕ , θ ｕ ) ∈ C Ｍ＿ｕ Generate.

[0027] Outside the ONN, the target vector t ｉ Using the loss function l(y ｉ , t ｉ)∈R is defined and this loss function is used to train the ONN.

[0028] The linear module is usually an array of MZIs. Each MZI consists of two PS-BS pairs. One PS-BS pair functions as a transfer matrix shown in the following equation (1), which is programmable by a phase parameter θ.

[0029] where j is the imaginary unit and j ２ = -1. Here, we assume an ideal situation where there is no error, that is, ξ = 1 and γ = 0. In this case, the MZI satisfies UU Ｈ = I. Ｈ is the conjugate transpose operator).

[0030] The nonlinear module is modReLU having a bias parameter θ, and the function for each element is given by the following equation (2).

[0031] Figure 3 shows an example of a module used in a NN. The Clement's mesh shown in Figure 3 is the most common ONN module. ｕ The case where θ = 8 is shown as Clements(8,8). This realizes an arbitrary unitary matrix with the diagonal matrix shown as PSdiag(8). The diagonal elements of the diagonal matrix are the diagonal elements of e with the phase parameter θ. ｊθ It has the shape of

[0032] In the above case, the connected modules "Clements(8,8) + PSdiag(8)" have a total of 7 × 2 × 4 + 8 = 56 + 8 = 64 parameters, which corresponds to the necessary and sufficient number of parameters for any 8-dimensional unitary matrix. The circuit size can also be reduced by using a truncated version of the Clements mesh, and the four-layer case is shown in Figure 3 as Clements(8,4). In this case, the number of parameters is reduced to 7 × 2 × 2 = 28, and the degrees of freedom for realizing the unitary matrix are also reduced.

[0033] The Clement mesh and its truncated counterparts are layered arrays of MZIs. Therefore, the phase parameters within a module are correlated. An example of this will be given later. On the other hand, the parameters of PSdiag and modReLU are uncorrelated when viewed as a single module. The linear modules of a typical neural network have weight parameters w that are similarly uncorrelated.

[0034] (Training a NN with Zeroth-Order (ZO) Optimization) A training dataset of size D, D={x ｉ , t ｉ} Ｄ ｉ＝１ Given θ, the NN is trained by optimizing the parameter θ to minimize the loss function shown in equation (3) below.

[0035] A standard method for optimizing the parameter θ is mini-batch stochastic gradient descent (SGD), which iteratively updates the parameter as shown in Equations (4) and (5) below.

[0036]

[0037] Here, B = {x ｉ , t ｉ} Ｄ ｉ＝１ ⊂D is a mini-batch dataset drawn from D, and ∇ θ is the gradient with respect to θ, usually computed by backpropagation, and η>0 is the learning rate.

[0038] In ONN training, the gradient cannot be accurately calculated using backpropagation due to manufacturing variations. Therefore, the gradient is approximately calculated using ZO optimization as a black-box optimization method, as shown in Equation (6) below. This gradient is called the approximate gradient. In other words, the approximate gradient is an approximation of the gradient calculated in ZO optimization.

[0039] where λ>0 is a scale hyperparameter and δθ ｑ, q=1,...,Q are random perturbation vectors. The perturbation vectors may also be called random direction vectors. ｑ is a difference quotient obtained by the finite difference method using a smoothing hyperparameter μ>0, as shown in the following equation (7).

[0040] In the prior art, the perturbation vector δθ ｑ is a zero-mean normal distribution N(0,I) whose covariance matrix is ​​the identity matrix of size N. Ｎ ) is sampled from δθ ｑ ~N(0,I Ｎ ) which is expressed as the parameter θ ｕ is perturbed by a vector generated from the distribution shown in equation (8) below.

[0041] However, due to the layered structure, the parameter θ ｕ For a module u with which the Ｎ＿ｕ is not an optimal covariance matrix.

[0042] Therefore, in this embodiment, as shown in the following equation (9), the parameters for the layered modules are expressed as the covariance matrix Σ ｕ Perturb using

[0043] The above sampling (δθ ｕｑ Although the specific method for generating the identity covariance matrix (E[rr Ｔ ] = I( Ｔ The above sampling is realized using a subroutine that generates random vectors r to N(0,I) using the transposition operator).

[0044] First, the covariance matrix Σ ｕ Cholesky decomposition of Σ ｕ =LL Ｔ where L is a lower triangular matrix. ｕｑ = δθ as Lr ｕｑ This fits into the following equation:

[0045] (Jacobian matrix) The Jacobian matrix is ​​a matrix of the partial differentiation of a multivariate function whose output is a vector. ｉｕ , the parameter θ in module u ｕ to the output vector y ｉｕ To model the propagation of perturbations to , we introduce the Jacobian matrix:

[0046] Next, the parameter perturbation δθ ｕｑ The output perturbation δy caused by ｉｕｑ can be expressed as a Jacobian-vector product as shown in the following equation (11). Note that the output perturbation refers to the amount by which the output value changes slightly as a result of a perturbation given to a parameter.

[0047] This can be calculated by automatic differentiation (AD) in the forward propagation mode. Conversely, the output perturbation δy ｉｕｐ (here we use the index p to distinguish it from q) （ｉ） ｕｐ can be expressed as a vector-Jacobian product as shown in the following formula (12). Note that in the format of the following formula (12), the transpose is performed on the column vector. Ｔ This is being done.

[0048] This can be calculated by backpropagation, or in this context, AD in backpropagation mode. Since AD ​​in forward and backpropagation modes requires accurate information inside module u, we are forced to assume an ideal error-free situation, i.e., ξ = 1 and γ = 0.

[0049] Motivating Examples Below we provide some examples of what happens when we use equation (8) for parameter perturbations, and how the situation improves by using the layered parameter perturbations of equation (9) instead. That is, we explain the motivation for using equation (9) instead of equation (8). We refer to (a)-(e) shown in Figures 4-6.

[0050] 4 to 6 show a simple two-dimensional case where the situation can be examined in detail. ｕ = [π / 4, -π / 4, 0, 0] Ｔ , where u is a linear module that represents a two-dimensional unitary matrix with

[0051] Input vector x ｉｕ = [e －ｊπ／４ , e ｊπ／２ ] Ｔ For this module, the output is y ｉｕ = [-1, j] Ｔ 4(b) and 6(e) respectively show the two-dimensional complex space of the output, where the real part Re([y ｉｕ ] １ ), Re([y ｉｕ ] ２ ), and the imaginary part Im([y ｉｕ ] １ ), Im([y ｉｕ ] ２ ) plot.

[0052] In each of Fig. 4(b) and Fig. 6(e), the square near the center represents the output value without perturbation, i.e., the current parameter θ ｕ Output value when using [-1, j] Ｔ ☆ represents the output value when the target optimal parameters are used. In FIG. 4(b), the set of black circles drawn around the square represents the four-dimensional unit matrix I ４ distribution N(0,I ４ ) typical parameter perturbations δθ for Q=1000 sampled from ｕｑ , q=1,...,Q. In this case, the output perturbation δyｉｕｑ is not isotropic, but is strong in certain directions and weak in others.

[0053] The reason is that the parameters are interrelated through the layered structure shown in FIG. 4(a). One way to measure the degree of interrelatedness is to use the parameter θ ｕ In this embodiment, the Fisher information matrix is ​​calculated as a covariance matrix as shown in the following equation (13).

[0054] The above covariance matrix is ​​expressed as the output y ｉｕ This is caused by newly generated isotropic perturbations in

[0055] Here, δθ （ｉ） ｕｐ is the parameter θ ｕ is the perturbation induced in ｉｕｐ can be calculated by backpropagation from (Equation (12)).

[0056] FIG. 5(c) shows the random input vector x ｉｕ is R ｉｎ = 1000, output perturbation is R ｏｕｔ = 10, the average of the matrix shown below is shown.

[0057] This is because the light beam is not connected by an optical path (θ ３ｕ , θ ４ｕ ), all parameter pairs are interrelated by non-negligible matrix elements.

[0058] In this embodiment, the covariance matrix Σ calculated by the method described later is ｕ distribution N(0,Σ ｕ ) in this case. ｕ is shown in Fig. 5(d), and it is clear that I ４The set of circles around the square in FIG. 6(e) is different from N(0,Σ ｕ ) parameter perturbations δθ sampled from ｕｑ The output perturbation δy caused by ｉｕｑ , q = 1, ..., Q. It can be seen that the perturbation in Fig. 6(e) is more isotropic than the perturbation in Fig. 4(b). Since the output from the ZO optimization aims for the optimal parameters that produce a star, the output perturbation in Fig. 6(e) can more easily approach a star than that in Fig. 4(b). One way to gauge the quality of the output perturbation is to examine the output covariance matrix shown in Equation (15) below.

[0059] In particular, the eigenvalues ​​of the output covariance matrix indicate the degree of isotropy. The more similar the eigenvalues ​​are, the more isotropic the output perturbation is. The eigenvalues ​​for this example are shown in the bottom of Figure 4(b) and Figure 6(e).

[0060] In particular, when performing black-box optimization in a situation where there is no information to rely on, it is better to search isotropically, so the perturbation in FIG. 6( e) can be said to be better than the perturbation in FIG. 4( b).

[0061] Note that by perturbing the parameters using perturbations generated from a normal distribution with a covariance matrix according to this embodiment, isotropic output perturbations can be obtained even in configurations larger than the configuration shown in FIG. 4( a) (e.g., Clements(8,8) and Clements(8,4)).

[0062] (Proposed Method) The method according to the present embodiment improves the ZO optimization for training ONNs. The key to the improvement is to improve the ZO optimization for a module u with layered parameters (such as the module shown in the top of Figure 3) by using the identity matrix I Ｎ＿ｕ Instead of equation (8) using the covariance matrix Σ ｕ The parameter θ of the module u is calculated by the equation (9) using ｕ θ ｕ +μ・δθ ｕｑ The idea is to perturb it as follows:

[0063] <Covariance matrix Σ for perturbations ｕ> The covariance matrix Σ in Equation (9) is calculated so as to minimize the cost function shown in Equation (16) below. ｕ Design.

[0064] Here, L Ｍ（←Ｍ＿ｕ，Ｎ＿ｕ） is the LogDet (or Burg matrix or multi-channel Itakura-Saito) divergence for a matrix of size M×M, and is shown below:

[0065] The LogDet divergence measures how different two covariance matrices are and is zero if the two matrices are the same. The first term in equation (16) is the variance of the various input vectors x ｉｕ ,i=1,...,R ｉｎ , the output covariance matrix C ｙ＿ｉｕ This shows that the eigenvalues ​​of the output perturbation are approximated to the identity matrix. As a result, the output perturbation becomes as isotropic as possible, as shown in Fig. 6(e). The second term with hyperparameter ρ>0 is the covariance matrix Σ ｕ is the identity matrix I Ｎ＿ｕ By not moving too far from Σ ｕ This is to prevent Σ from becoming a singular matrix. ｕ I Ｎ＿ｕ Since the objective is to replace equation (8) with equation (9), the hyperparameter ρ is not set to be too large. As a result, the covariance matrix Σ ｕ is obtained.

[0066] In equation (18), the Fisher information matrix F （ｉ） ｕ is R ｉｎ Random input vectors x ｉｕ The method for deriving equation (18) from equation (16) will be described later. （ｉ） ｕ As already explained when introducing F （ｉ） ｕ is first the output perturbation δy ｉｕｐ～ N(0,I Ｍ＿ｕ ), p=1,.... R ｏｕｔLet I be the unit matrix I Ｍ＿ｕ (Equation (14)), and then sample them with parameters δθ （ｉ） ｕｐ (12) and finally take their cross product as p = 1, .... R ｏｕｔ is calculated by averaging over (equation (13)).

[0067] (Overall Procedure of ZO Optimization) Algorithm 1 shown in FIG. 7 shows the overall procedure of ZO optimization for generating layered parameter perturbations. Lines 2, 3, and 14 to 17 relate to the basic procedure of ZO optimization. Lines 4 to 10 are specific to the technology according to this embodiment. Line 9 is an important operation in the technology according to this embodiment, and generates the perturbation vector δθ of the module u with layered parameters. ｕｑ For a module u with non-layered parameters, the perturbation vector δθ is sampled by Equation (8) as in the normal ZO optimization. ｕｑ (line 12). Note that if lines 4 to 10 are skipped and line 12 is applied to all modules u, the procedure becomes normal.

[0068] In the iterations of Algorithm 1 (lines 1 to 18), exponential smoothing is used to provide robust estimates of several statistics. In our particular case, we modify equation (18) as follows:

[0069]

[0070] Here, in formula (20), F ｕ is initially set as an identity matrix. 0≦α≦1 is a smoothing hyperparameter. Furthermore, it has been empirically confirmed that reducing the update frequency of these statistics does not significantly degrade the performance of the ZO optimization. Therefore, to reduce the computational load, another hyperparameter T ｕｄ (line 5) is introduced.

[0071] Algorithm 1 corresponds to the operation of the optimization device 100, which will be described later. Algorithm 1 will be described more specifically as the operation of the optimization device 100.

[0072] The optimization device 100 uses a training data set D={x ｉ , t ｉ} ｉ＝１ Ｄ The optimization device 100 repeatedly executes the processes in lines 2 to 17 until the training converges. The number of iterations is represented by τ.

[0073] In the second line, the optimization device 100 extracts some data from the training dataset D into a mini-batch B={x ｉ , t ｉ} ｉ＝１ Ｂ The subsequent training process from line 3 to line 17 is performed using this mini-batch. In line 3, the optimization device 100 calculates the loss l Ｂ Calculate (θ).

[0074] As shown in the fourth line, the optimization device 100 executes the processing in the fifth to tenth lines for each module u having a layer parameter.

[0075] As shown in the fifth line, the optimization device 100 calculates "τ mod T ｕｄ =0”, the optimization device 100 performs the processing of the sixth and seventh lines. ｕ In the seventh line, Σ ｕ Update.

[0076] In the ninth line, the optimization device 100 calculates the perturbation vector δθ by equation (9). ｕｑ , q=1,...,Q are sampled.

[0077] As shown in line 11, the optimization device 100 executes the process in line 12 for each module u having a non-layered parameter. In line 12, the optimization device 100 calculates the perturbation vector δθ ｕｑ , q=1,...,Q are sampled.

[0078] In the 14th line, the optimization device 100 calculates the perturbation vector δθ for each module u obtained by the processing in the 4th to 13th lines. ｕｑ By concatenating these, the perturbation vector δθ of the entire ONN is ｑ , q=1,..., Q is calculated.

[0079] On the 15th line, the optimization device 100 calculates the difference quotient δl using equation (7). ｑ , q=1, ..., Q. On line 16, the optimization device 100 calculates the approximate gradient using equation (6). On line 17, the optimization device 100 updates the parameter θ using equation (4).

[0080] (Derivation of (18)) A method for deriving equation (18) will be explained below. First, the two equations mentioned above will be restated. The first equation (13) can be rewritten as follows:

[0081] where: ＊ is the element-wise conjugate operator. Here, we use equation (12) and ｉｎｐ ~N(0,I Ｍ＿ｕ ), so E ｐ [δy ｉｎｐ δy Ｈ ｉｎｐ ]=I Ｍ＿ｕ The second equation (15) can be rewritten as follows:

[0082] Here, we use equation (11) and δθ ｕｑ ~N(0,Σ ｕ ), so E ｑ [δθ ｕｑ δθ Ｔ ｕｑ ]=Σ ｕ This becomes:

[0083] According to the definition of LogDet divergence (Equation (17)) and the new form of the output covariance matrix (Equation (22)), the cost function ignoring the constant term (Equation (16)) can be rewritten as follows:

[0084] Σ ｕ The complex gradient matrix for is given by:

[0085]

[0086] where we use equation (21): Setting equation (25) to a zero matrix, the covariance matrix (18) is obtained as the minimization solution of equation (16).

[0087] (Device Configuration Example) Fig. 8 shows a configuration example of an optimization device 100 according to this embodiment. As shown in Fig. 8, the optimization device 100 includes a perturbation generation unit 110, a difference quotient calculation unit 120, an approximate gradient calculation unit 130, and a parameter update unit 140. The optimization device 100 may also be referred to as an information processing device. In the following description, the neural network targeted for parameter optimization is an ONN. Using an ONN as the target for parameter optimization is just one example, and the target for parameter optimization may also be a neural network other than an ONN.

[0088] The optimization device 100 does not include an ONN that is the target of parameter optimization. However, this is not limited to this, and the optimization device 100 may include an ONN. The operation of each unit is as follows.

[0089] The perturbation generator 110 generates a perturbation vector δθ ｑ (q=1, . . . , Q). The configuration and operation of the perturbation generator 110 will be described in detail later.

[0090] The difference quotient calculation unit 120 calculates the δθ ｑ Using the above, the difference quotient δl shown in Equation (7) is ｑ (q = 1, ..., Q). The loss l in Eq. (7) Ｂ (·) is calculated by equation (5). That is, the difference quotient calculation unit 120 calculates the input x ｉ The output f(x ｉ , θ) is obtained, the loss is calculated using equation (5), and the difference quotient δl is calculated using equation (7). ｑ Calculate the output f(x ｉ , θ) may be provided outside the quotient difference calculation unit 120.

[0091] The approximate gradient calculation unit 130 calculates δθ ｑ and δl ｑ By calculating the product of θ l Ｂ Calculate (θ).

[0092] The parameter update unit 140 updates the ONN parameter θ by θ l Ｂ (θ)”.

[0093] (Operation of the Optimization Device 100) The operation of the optimization device 100 will be described with reference to the flowchart shown in Fig. 9. The flowchart in Fig. 5 basically corresponds to the processing procedure in Algorithm 1 in Fig. 7. As a premise, the optimization device 100 performs the optimization of a training data set D={x ｉ , t ｉ} ｉ＝１ Ｄ Assume that we hold

[0094] In S1 (step 1), the optimization device 100 extracts some data from the training dataset D into a mini-batch B={x ｉ , t ｉ} ｉ＝１ Ｂ The subsequent training processes from S2 to S7 are performed using this mini-batch.

[0095] In S2, the difference quotient calculation unit 120 in the optimization device 100 calculates the loss l by using the formula (5). Ｂ Calculate (θ).

[0096] In S3, the perturbation generator 110 generates a perturbation vector δθ for each module u. ｕｑ (q=1,...,Q) and calculate the perturbation vector δθ for each module u. ｕｑ By concatenating these, the perturbation vector δθ of the entire ONN is ｑ , q=1,..., Q is calculated.

[0097] In S4, the difference quotient calculation unit 120 calculates the difference quotient δl using equation (7). ｑ, q=1, ..., Q. In S5, the approximate gradient calculation unit 130 calculates the approximate gradient using equation (6). In S6, the parameter update unit 140 updates the parameter θ using equation (4).

[0098] Steps S1 to S6 are repeated until the training converges. Once the training converges, in step S8, the optimization device 100 (for example, the parameter update unit 140) outputs the parameter θ.

[0099] When the above procedure is applied to a neural network of a hardware circuit such as an ONN, a trained ONN is obtained when the training converges. The trained ONN can be used to perform an inference process that outputs an inference result for input data.

[0100] (Details of the perturbation generator 110) Fig. 10 shows the detailed configuration of the perturbation generator 110. As shown in Fig. 10, the perturbation generator 110 has a layered module determination unit 111, a normal distribution sampling unit 112, a normal distribution sampling unit 113, a smoothed FIM calculation unit 114, and a covariance matrix calculation unit 115. Note that FIM is an abbreviation for Fisher Information Matrix. The operation of each unit is as follows. The row numbers used in the following explanation are the row numbers of Algorithm 1.

[0101] The layer module determination unit 111 performs the processing of lines 4 and 11 in Algorithm 1. That is, for each module u constituting the ONN targeted for parameter optimization, the layer module determination unit 111 determines whether or not it is a layer module.

[0102] When the module u is a layered module, the normal distribution sampling unit 112 calculates the perturbation vector δθ using the equation (9). ｕｑ The covariance matrix calculation unit 115 obtains the covariance matrix Σ used in sampling in the normal distribution sampling unit 112. ｕ is calculated by equation (19) (line 7 of algorithm 1). In addition, the smoothed FIM calculation unit 114 calculates F ｕis calculated using equations (20), (13), (12), and (14) (line 6 of Algorithm 1).

[0103] If the module u is not a layered module, the normal distribution sampling unit 113 uses an identity matrix as a covariance matrix and calculates the perturbation vector δθ ｕｑ (line 12 of Algorithm 1).

[0104] When a covariance matrix other than the identity matrix is ​​used, the normal distribution sampling unit 112 calculates the perturbation vector δθ ｕｑ Generate.

[0105] (Details of the smoothed FIM calculation unit 114) Fig. 11 shows a detailed configuration of the smoothed FIM calculation unit 114. As shown in Fig. 11, the smoothed FIM calculation unit 114 has an output perturbation generation unit 1141, a VJP calculation unit 1142, an FIM calculation unit 1143, and an exponential smoothing unit 1144. Note that VJP is an abbreviation for Vector Jacobian Product. The operation of each unit is as follows.

[0106] The output perturbation generator 1141 generates the output perturbation δy ｉｕｐ The VJP calculation unit 1142 generates δθ, which is the product of the output perturbation and the Jacobian, using equation (12). （ｉ） ｕｐ Calculate.

[0107] The FIM calculation unit 1143 calculates the Fisher information matrix F using equation (13). （ｉ） ｕ The exponential smoothing unit 1144 calculates F by exponential smoothing using equation (20). ｕ Smooth the

[0108] (Experiments) In order to demonstrate that the proposed method using equation (9) is more effective than the conventional method using equation (8), experiments were conducted on an image classification task.

[0109] <Experimental Setup> First, the experimental setup will be explained. In this experiment, an ONN circuit with manufacturing variations was simulated using a conventional computer. For this purpose, the split angle error γ and the damping phase error ξ (see Figure 1) were randomly set as follows:

[0110] Here, r ０ is a random number sampled from the normal distribution N(0,1), and r １ and ２ is a random number sampled from the uniform distribution [0, 1). Experiments were conducted with the following error settings:

[0111] where β is a parameter that controls the amount of error. Setting β=1 corresponded to the estimation for the actual calibrated Clements mesh on silicon photonics.

[0112] PyTorch was used to simulate and train the ONN. A customized CUDA kernel was constructed to accelerate the computation, and the program was run on an NVIDIA RTX A6000 GPU. The parameters of the layered module Clements(K,L) were randomly initialized, and the parameters of non-layered modules, such as PSdiag(K) and modReLU(K), were initialized to zero.

[0113] The training procedure consisted of two stages. The first stage used backpropagation under the error-free assumption (i.e., ξ = 1 and γ = 0) and was performed for a small number of epochs (10 epochs). The purpose of the first stage was to quickly and roughly minimize the loss function from the initial parameters, even if the calculated gradients were inaccurate due to incorrect error assumptions. The second stage considered the actual error situation and performed ZO optimization for a sufficient number of epochs (200 epochs).

[0114] To achieve better convergence of the ZO optimization algorithm, we used the Adam optimizer instead of vanilla SGD (Equation (4)). The learning rate η of Adam was optimized using Optuna for each task, dimension (K = 16, 32, 64), and method (conventional or proposed). Figure 12 shows the default settings of other hyperparameters. The top four settings in Figure 12 are common to both the conventional and proposed methods, while the bottom five settings are specific to the proposed method.

[0115] <Image Classification Task> Fig. 13 shows the settings for the image classification task. As shown in Fig. 13, the ONN includes multiple Clement meshes, and the output of the ONN is obtained from the final stage Clement mesh. MNIST and FashionMNIST were used as image datasets. First, a discrete Fourier transform was applied to an image of 784 = 28 x 28 pixels, and K low-frequency components up to the (K+1)th frequency, excluding 0 Hz, were extracted and used as the input vector x ｉ Next, x ｉ is input to ONN, and the output vector y ｉ Then, the central 10 dimensions were extracted. Finally, the power of the extracted elements was calculated to generate a 10-dimensional real-valued vector representing the image classification result. β = 1 was used as the error setting. In Figure 13, K is the number of input dimensions of the ONN, and L is the number of layers in the truncated version of Clement's method.

[0116] The box plots shown in Figures 14 to 16 show the distribution of training loss resulting from eight independent trials for each combination of dataset, method, dimension K, and number of layers L in Clements(K, L). ｕ ) is the conventional method (I Ｎ＿ｕ), and this advantage increases as K increases. Particularly noteworthy is that the training results of the proposed method using a configuration where L is smaller than K, e.g., Clement's (K = 64, L = 48), are better than the training results of the conventional technique using a configuration where L = K, e.g., Clement's (K = 64, L = 64). This indicates that the circuit size of the ONN can be reduced without sacrificing performance, and the proposed method is highly industrially practical.

[0117] Figure 17 shows how the training loss (vertical axis) decreases over time (horizontal axis) until 200 epochs are completed. Although the proposed method requires more computation time than the conventional method, it is highly effective in reducing training loss, so it can be observed that a smaller training loss is achieved over the same time period.

[0118] Figure 18 shows how robust the proposed method is to the hyperparameter α in equation (20) and ρ in equation (19) for the MNIST task in Clement's (64, 64). The vertical bars and shading represent one standard deviation over eight independent runs. The leftmost α = 0 indicates that F in equation (20) ｕ Since is initially set as an identity matrix, it corresponds to the conventional method. The proposed method is not very sensitive to these hyperparameters and is observed to be very effective around α = 0.1 and ρ = 0.1. It also shows sufficiently good performance around these values, and can be said to be robust.

[0119] (Other Configuration Examples) In this embodiment, the device configuration for training a neural network is not limited to the configuration in FIG. 8. For example, an information processing device 200 shown in FIG. 19 may train a neural network. This "neural network" includes both a neural network using a circuit such as an ONN, and a neural network on a general computer. The "neural network" may also be called a "model."

[0120] An information processing device 200 shown in Fig. 19 includes a perturbation generation unit 210 and a training unit 220. The perturbation generation unit 210 corresponds to the perturbation generation unit 110 in Fig. 8, and the training unit 220 corresponds to the "divisor difference calculation unit 120, approximate gradient calculation unit 130, and parameter update unit 140" in Fig. 8.

[0121] The perturbation generator 210 generates perturbations for the parameters of a model using a neural network from a distribution designed so that output perturbations appearing in the output from the model are isotropic.

[0122] The "output from a model using a neural network" may be an output from a linear module portion of an entire neural network having a linear module and a non-linear module. Also, the "output from a model using a neural network" may be an output from a layered module portion of an entire neural network having a layered module and a non-layered module.

[0123] The training unit 220 updates the parameters of the model using the approximate gradient calculated using the perturbation.

[0124] The information processing device 200 may include a model to be trained, or the model to be trained may be provided outside the information processing device 200.

[0125] FIG. 20 shows the configuration of an information processing device 300 that includes a model to be trained and performs inference using the model.

[0126] As shown in Fig. 20, the information processing device 300 includes a perturbation generation unit 210, a training unit 220, and an inference unit 230. The perturbation generation unit 210 and the training unit 220 are the same as those shown in Fig. 19. The inference unit 230 includes a neural network model 231.

[0127] The training unit 220 trains the model 231 and optimizes the parameters of the model 231. The inference unit 230 uses the trained model 231 to output an inference result for input data. That is, the model 231 outputs an inference result for input data. For example, similar to the task in the above-mentioned experiment, the model 231 inputs an image and outputs a classification result for the image.

[0128] In the configuration of Fig. 20, the perturbation generation unit 210 and the training unit 220 may be located outside the device that performs inference. A configuration in this case is shown in Fig. 21. The information processing device 400 shown in Fig. 21 includes an inference unit 230. The inference unit 230 includes a model 231 that outputs an inference result for input data.

[0129] The model 231 is trained by the training unit 220 in the information processing device 200 shown in Fig. 19. That is, the parameters of the model 231 are parameters generated from a certain distribution and updated by an approximate gradient calculated using perturbations to the parameters, and the distribution is a distribution designed so that the output perturbations appearing in the output from the model 231 are isotropic.

[0130] (Hardware Configuration Example) Any of the devices described in this embodiment (optimization device 100, information processing devices 200 to 400) can be realized, for example, by causing a computer to execute a program. This computer may be a physical computer or a virtual machine on the cloud.

[0131] That is, the device can be realized by executing a program corresponding to the processing performed by the device using hardware resources such as a CPU and memory built into a computer. The program can be recorded on a computer-readable recording medium (such as a portable memory) and stored or distributed. The program can also be provided via a network such as the Internet or email.

[0132] Fig. 22 is a diagram showing an example of the hardware configuration of the computer. The computer in Fig. 22 includes a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, and the like, all of which are interconnected via a bus B. The computer may further include a GPU.

[0133] The program that realizes the processing on the computer is provided by a recording medium 1001, such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.

[0134] The memory device 1003 reads and stores a program from the auxiliary storage device 1002 when an instruction to start the program is received. The CPU 1004 realizes functions related to the device in accordance with the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network, etc. The display device 1006 displays a GUI (Graphical User Interface) or the like according to the program. The input device 1007 is composed of a keyboard, mouse, buttons, a touch panel, etc., and is used to input various operation instructions. The output device 1008 outputs the results of calculations.

[0135] (Summary of the embodiment, effects, etc.) As described above, the technology described in this embodiment makes it possible to generate better perturbations for training when parameters are correlated, such as in the layered modules of an ONN, compared to conventional technology (perturbations using the normal distribution of a unit matrix), and as a result, it is possible to perform appropriate training.

[0136] The following additional notes are provided regarding the above-described embodiments.

[0137] <Additional Notes> (Additional Item 1) An information processing device that trains a model using a neural network, comprising: a memory; and at least one processor connected to the memory, wherein the processor generates perturbations for parameters of the model from a distribution designed so that output perturbations appearing in the output from the model are isotropic, and updates the parameters of the model using an approximate gradient calculated using the perturbation. (Additional Item 2) An information processing device that performs inference using a model using a neural network, comprising: a memory; and at least one processor connected to the memory, wherein the processor performs inference using the model that outputs an inference result for input data, wherein the parameters of the model are parameters updated by an approximate gradient calculated using perturbations for the parameters generated from a certain distribution, and the distribution is a distribution designed so that output perturbations appearing in the output from the model are isotropic. (Additional Item 3) The information processing device according to Additional Item 1, wherein the distribution is a normal distribution having a covariance matrix other than an identity matrix. (Supplementary Item 4) The information processing device according to Supplementary Item 3, wherein the processor calculates the covariance matrix using a Fisher information matrix smoothed by exponential smoothing. (Supplementary Item 5) The information processing device according to Supplementary Item 1, wherein the processor generates perturbations for the parameters from the distribution for layered modules in the neural network, and generates perturbations for the parameters from a normal distribution having an identity matrix as a covariance matrix for non-layered modules in the neural network. (Supplementary Item 6) The information processing device according to Supplementary Item 1, wherein the neural network is an optical neural network. (Supplementary Item 7) A training method executed by an information processing device that trains a model using a neural network, comprising: generating perturbations for the parameters of the model from a distribution designed so that output perturbations appearing in an output from the model are isotropic; and updating the parameters of the model using an approximate gradient calculated using the perturbations.(Supplementary Item 8) A non-transitory storage medium storing a program for causing a computer that functions as an information processing device for training a model using a neural network to function as: a perturbation generation unit that generates perturbations for the parameters of the model from a distribution designed so that output perturbations appearing in the output from the model are isotropic; and a training unit that updates the parameters of the model using approximate gradients calculated using the perturbations.

[0138] Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims.

[0139] 100 Optimization device 110 Perturbation generation unit 111 Layered module determination unit 112 Normal distribution sampling unit 113 Normal distribution sampling unit 114 Smoothed FIM calculation unit 1141 Output perturbation generation unit 1142 VJP calculation unit 1143 FIM calculation unit 1144 Exponential smoothing unit 115 Covariance matrix calculation unit 120 Divided difference calculation unit 130 Approximation gradient calculation unit 140 Parameter update unit 200-400 Information processing device 210 Perturbation generation unit 220 Training unit 230 Inference unit 231 Model 1000 Drive device 1001 Recording medium 1002 Auxiliary storage device 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input device 1008 Output device

Claims

1. An information processing device that trains a model using a neural network, comprising: a perturbation generation unit that generates perturbations for the parameters of the model from a distribution designed so that output perturbations appearing in the output from the model are isotropic; and a training unit that updates the parameters of the model using an approximate gradient calculated using the perturbations.

2. An information processing device that performs inference using a model using a neural network, comprising an inference unit having the model that outputs inference results for input data, wherein the parameters of the model are parameters that are generated from a certain distribution and updated by an approximate gradient calculated using perturbations to the parameters, and the distribution is a distribution designed so that output perturbations appearing in the output from the model are isotropic.

3. The information processing device according to claim 1, wherein the distribution is a normal distribution having a covariance matrix other than a unit matrix.

4. The information processing device according to claim 3, wherein the perturbation generation unit calculates the covariance matrix using a Fisher information matrix smoothed by exponential smoothing.

5. The information processing device according to claim 1, wherein the perturbation generation unit generates perturbations for the parameters from the distribution for layered modules in the neural network, and generates perturbations for the parameters from a normal distribution having a unit matrix as a covariance matrix for non-layered modules in the neural network.

6. The information processing device according to claim 1, wherein the neural network is an optical neural network.

7. A training method executed by an information processing device that trains a model using a neural network, comprising: a step of generating perturbations for parameters of the model from a distribution designed so that output perturbations appearing in the output from the model are isotropic; and a step of updating the parameters of the model using an approximate gradient calculated using the perturbations.

8. A program for causing a computer that functions as an information processing device for training a model using a neural network to function as a perturbation generation unit that generates perturbations to the parameters of the model from a distribution designed so that the output perturbations appearing in the output from the model are isotropic, and a training unit that updates the parameters of the model using the approximate gradient calculated using the perturbations.

Images