Optical neural network training method and device, medium and program product

Abstract

The invention provides an optical neural network training method and device, a medium and a program product, which can be applied to the technical field of computers. The optical neural network comprises at least one optical interference unit, and the optical interference unit has a phase parameter; the method comprises the following steps: iteratively executing the following operations to an iteration stop condition: inputting a training optical signal corresponding to a training sample into an optical neural network, and outputting first output data for the training sample; performing parameter offset processing on the current phase parameter of the at least one optical interference unit by using the parameter offset to obtain an optical neural network after parameter offset; inputting the training optical signal into the optical neural network after parameter offset, and outputting second output data; determining a current parameter gradient for the current phase parameter according to the first output data, the second output data and a preset loss function; and optimizing the current phase parameter of the optical neural network by using the current parameter gradient to obtain an optimized optical neural network.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a training method, device, medium and program product for an optical neural network. Background Art

[0002] Optical neural networks (ONNs) are a new computing architecture that uses photons as information carriers to realize neural network functions. It uses the volatility and interference effects of photons to perform computing tasks. It has the advantages of high speed, low energy consumption, parallel processing and three-dimensional integration. It is used in image recognition, speech recognition, big data processing, complex system simulation, optical communications and biomedicine.

[0003] In the related art, the finite difference method and the adjoint variable method are generally used to determine the parameter gradient during the training process of the optical neural network. However, the calculation speed and accuracy of the parameter gradient in the related art are low, which affects the training efficiency of the optical neural network. Summary of the invention

[0004] In view of the problem, the present invention provides a method, apparatus, device, medium and program product for training an optical neural network.

[0005] According to a first aspect of the present invention, a training method for an optical neural network is provided, wherein the optical neural network comprises at least one optical interference unit, wherein the optical interference unit has at least one phase parameter; the training method comprises: iteratively performing the following operations until an iteration stop condition is reached:

[0006] Inputting a training light signal corresponding to a training sample into an optical neural network, and outputting first output data for the training sample;

[0007] Performing parameter shift processing on a current phase parameter of at least one optical interference unit using the parameter shift amount to obtain an optical neural network after parameter shift;

[0008] Inputting the training light signal corresponding to the training sample into the optical neural network after the parameter shift, and outputting second output data for the training sample;

[0009] Determine a current parameter gradient for a current phase parameter according to the first output data, the second output data and a preset loss function;

[0010] The current phase parameter of the optical neural network is optimized using the current parameter gradient to obtain an optimized optical neural network.

[0011] According to an embodiment of the present invention, the optical neural network after parameter shift includes an optical neural network after positive shift and an optical neural network after negative shift;

[0012] Performing parameter offset processing on the current phase parameter of at least one optical interference unit by using the parameter offset to obtain an optical neural network after the parameter offset includes:

[0013] Add the current phase parameter to the parameter offset to obtain the optical neural network after forward offset;

[0014] Subtract the current phase parameter from the parameter offset to obtain the optical neural network after negative offset.

[0015] According to an embodiment of the present invention, the second output data includes positive offset output data and negative offset output data;

[0016] Inputting the training light signal corresponding to the training sample into the optical neural network after the parameter shift, outputting the second output data for the training sample comprises:

[0017] Inputting a training optical signal corresponding to the training sample into the optical neural network after forward shift, and outputting forward shift output data;

[0018] The training light signal corresponding to the training sample is input into the negatively offset optical neural network, and the negatively offset output data is output.

[0019] According to an embodiment of the present invention, the output of the optical neural network is represented by an objective function, and the objective function is a function of a phase parameter; and determining a current parameter gradient for a current phase parameter according to the first output data, the second output data, and a preset loss function comprises:

[0020] Processing the second output data using a preset parameter offset formula to obtain a first gradient of the objective function relative to the current phase parameter;

[0021] determining a second gradient of the loss function with respect to the first output data;

[0022] A current parameter gradient for a current phase parameter is determined based on the first gradient and the second gradient.

[0023] According to an embodiment of the present invention, the second output data includes positive offset output data and negative offset output data;

[0024] Processing the second output data using a preset parameter offset formula to obtain a first gradient of the objective function relative to the current phase parameter includes:

[0025] The difference between the positive offset output data and the negative offset output data is multiplied by a preset value to obtain a first gradient.

[0026] According to an embodiment of the present invention, the loss function is a function of the objective function;

[0027] Determining a second gradient of the loss function with respect to the first output data includes:

[0028] Determine the derivative of the loss function with respect to the objective function;

[0029] The first output data is processed using the derivative to obtain a second gradient.

[0030] According to an embodiment of the present invention, determining a current parameter gradient for a current phase parameter according to the first gradient and the second gradient includes:

[0031] Multiply the first gradient and the second gradient to get the current parameter gradient.

[0032] According to an embodiment of the present invention, the optical interference unit includes at least one phase shifter, and the phase shifter is represented by a transfer matrix; the parameter offset is determined by the following steps:

[0033] The transfer matrix is ​​processed using the offset calculation formula to determine the parameter offset.

[0034] According to an embodiment of the present invention, the offset calculation formula includes matrix transformation parameters, and the matrix transformation parameters are expressed by a matrix transformation formula;

[0035] Processing the transfer matrix using the offset calculation formula includes:

[0036] Converting the transfer matrix into an exponential form of a preset format, wherein the exponential form includes a Hermitian matrix;

[0037] Use the matrix transformation formula to process the Hermitian matrix and determine the parameter values ​​of the matrix transformation parameters;

[0038] The parameter offset is determined based on the parameter value and the offset calculation formula.

[0039] According to an embodiment of the present invention, the Hermitian matrix is ​​processed using a matrix transformation formula to determine parameter values ​​of the matrix transformation parameters, including:

[0040] Determine two eigenvalues ​​for a Hermitian matrix;

[0041] The two eigenvalues ​​are processed using a matrix transformation formula to obtain parameter values ​​of the matrix transformation parameters.

[0042] According to an embodiment of the present invention, optimizing the current phase parameter of the optical neural network using the current parameter gradient to obtain the optimized optical neural network includes:

[0043] The current parameter gradient and the current phase parameter are processed using the preset gradient descent formula to obtain the optimized phase parameter;

[0044] The current phase parameter of at least one optical interference unit is adjusted according to the optimized phase parameter to obtain an optimized optical neural network.

[0045] According to an embodiment of the present invention, adjusting the current phase parameter of at least one optical interference unit according to the optimized phase parameter includes:

[0046] For each optical interference unit, according to the optimized phase parameter for the optical interference unit, the current phase parameter of the optical interference unit is adjusted by adjusting the voltage and / or temperature of the optical interference unit.

[0047] A second aspect of the present invention provides a training device for an optical neural network, wherein the optical neural network has at least one optical interference unit, and the optical interference unit includes at least one phase parameter; the device comprises:

[0048] A first input-output module, used to input a training light signal corresponding to a training sample into the optical neural network, and output first output data for the training sample;

[0049] A parameter shift module, used to perform parameter shift processing on a current phase parameter of at least one optical interference unit using a parameter shift amount to obtain an optical neural network after parameter shift;

[0050] A second input-output module is used to input a training light signal corresponding to the training sample into the optical neural network after the parameter is shifted, and output second output data for the training sample;

[0051] A determination module, configured to determine a current parameter gradient for a current phase parameter according to the first output data, the second output data and a preset loss function;

[0052] The optimization module is used to optimize the current phase parameters of the optical neural network using the current parameter gradient to obtain the optimized optical neural network.

[0053] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0054] The fourth aspect of the present invention further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.

[0055] The fifth aspect of the present invention also provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0057] Figure 1 2 is an application scenario diagram of an optical neural network training method according to an embodiment of the present invention.

[0058] Figure 2 4 is a flow chart of a method for training an optical neural network according to an embodiment of the present invention.

[0059] Figure 3 is a schematic diagram of a training method for an optical neural network according to an embodiment of the present invention.

[0060] Figure 4 4 is a schematic diagram of a method for determining a parameter offset according to an embodiment of the present invention.

[0061] Figure 5 is a schematic diagram of determining the current parameter gradient according to an embodiment of the present invention.

[0062] Figure 6 4 is a structural block diagram of an optical neural network training device according to an embodiment of the present invention.

[0063] Figure 7 is a block diagram of an electronic device suitable for implementing the above method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0065] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0066] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0067] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0068] With the development of big data resources, advanced algorithms, and high-performance computing hardware, artificial intelligence has played a major role in various industries. However, neural network computing has gradually reached a physical bottleneck, and the demand for new types of computing is increasing. Optical computing, as a new computing model, has developed rapidly and can provide more efficient computing power and development potential in processing some computing processes. For example, silicon-based optoelectronics provide an attractive platform for optical computing due to its high-speed signal processing capabilities and compatibility with existing microelectronics manufacturing processes. Integrating optical components on silicon chips allows the creation of complex optoelectronic circuits that can perform neural network operations at the speed of light.

[0069] In the application of optical neural networks, the most important process is the training of the model. Common training methods are gradient descent methods, such as the more basic finite difference method and the adjoint variable method. The finite difference method is to calculate the gradient of each target parameter. , by adding a small perturbation to the parameter The change of the loss function L is calculated The adjoint variable method obtains an analytical solution for the gradient value calculation by converting the gradient calculation formula into an expression of physical quantities (such as the dielectric constant in the waveguide) in the optical calculation process. However, the related technology has low speed and accuracy in parameter gradient calculation, which affects the training efficiency of the optical neural network.

[0070] An embodiment of the present invention provides a training method for an optical neural network, wherein the optical neural network has at least one optical interference unit, and the optical interference unit includes at least one phase parameter; the method includes: iteratively performing the following operations until an iteration stop condition is reached: inputting a training light signal corresponding to a training sample into the optical neural network, and outputting first output data for the training sample; performing parameter offset processing on the current phase parameter of at least one optical interference unit using a parameter offset to obtain an optical neural network after parameter offset; inputting a training light signal corresponding to the training sample into the optical neural network after parameter offset, and outputting second output data for the training sample; determining a current parameter gradient for the current phase parameter based on the first output data, the second output data and a preset loss function; and optimizing the current phase parameter of the optical neural network using the current parameter gradient to obtain an optimized optical neural network.

[0071] Figure 1 2 is an application scenario diagram of an optical neural network training method according to an embodiment of the present invention.

[0072] like Figure 1 As shown, the application scenario 100 according to this embodiment may include an optical neural network 101 composed of at least one optical interference unit 110, a first terminal device 102, a second terminal device 103, a server 104, and a network 105. The network 105 is used to provide a medium for a communication link between the optical neural network 101 and the first terminal device 102, the second terminal device 103, and the server 104. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc. It should be noted that Figure 1 At least one of the optical interference units 110 is connected in a rectangular network, and the other optical interference units 110 are connected in a triangular network in the same manner.

[0073] The user can input the training light signal into the optical neural network 101 to obtain the output data of the optical neural network 101; then send the output data of the optical neural network 101 to the first terminal device 102, the second terminal device 103 or the server 104 through the network 104. The first terminal device 102, the second terminal device 103 or the server 104 can determine the current parameter gradient based on the output data and the preset loss function, so as to adjust the current phase parameter of the optical neural network 101 according to the current parameter gradient.

[0074] Various communication client applications may be installed on the first terminal device 102 and the second terminal device 103 , such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (for example only).

[0075] The first terminal device 102 and the second terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0076] Server 104 can be a server that provides various services. For example, the server can perform data analysis and other processing on the output data of the optical neural network, and feed back the processing results to the optical neural network 101 to adjust the phase parameters of the optical neural network and realize the training of the optical neural network.

[0077] It should be noted that Figure 1 At least one of the optical interference units 110 is connected in a rectangular network, and the other optical interference units 110 are connected in a triangular network in the same manner.

[0078] It should be understood that Figure 1The number of optical neural networks, terminal devices, networks and servers in the embodiment is only for illustration. Any number of optical neural networks, terminal devices, networks and servers may be provided according to the implementation requirements.

[0079] The following will be based on Figure 1 The scene described by Figure 2~Figure 5 The training method of the optical neural network of the disclosed embodiment is described in detail.

[0080] Figure 2 4 is a flow chart of a method for training an optical neural network according to an embodiment of the present invention.

[0081] According to an embodiment of the present invention, the optical neural network has at least one optical interference unit, and the optical interference unit includes at least one phase parameter. For example, the optical interference unit can be a Mach-Zehnder interferometer (MZI), which generally consists of two beam splitters and two phase shifters, wherein the adjustable phase parameter is in the phase shifter. The transfer matrix of the phase shifter can be expressed as ,in is the phase parameter, which is also the parameter that needs to be trained in the optical neural network.

[0082] like Figure 2 As shown, the optical neural network training method 200 of this embodiment can iteratively perform operations S210 to S250 until an iteration stop condition is reached, and the optical neural network training method can be executed by a server.

[0083] The iteration stop condition can be pre-set for the training rounds, and the training is completed when the number of iterations reaches the set training rounds. For example, if the training rounds are 5000, the training is completed when the number of network iterations reaches 5000.

[0084] The iteration stopping condition may also be that the loss function value is less than a certain value. For example, the training is completed when the loss function value is less than a preset value.

[0085] It should be noted that the present invention does not limit the iteration stop condition, and it can be flexibly configured according to actual application needs.

[0086] In operation S210, a training optical signal corresponding to a training sample is input into an optical neural network, and first output data for the training sample is output.

[0087] Before executing operation S210 , the step may further include obtaining a training set including a plurality of training samples and a sample label of each training sample.

[0088] In operation S220, parameter offset processing is performed on the current phase parameter of at least one optical interference unit using the parameter offset to obtain an optical neural network after parameter offset.

[0089] In operation S230, the training optical signal corresponding to the training sample is input to the optical neural network after the parameter shift, and the second output data for the training sample is output.

[0090] In operation S240, a current parameter gradient for a current phase parameter is determined according to the first output data, the second output data, and a preset loss function.

[0091] In operation S250, the current phase parameter of the optical neural network is optimized using the current parameter gradient to obtain an optimized optical neural network.

[0092] The training sample may be an image sample, a voice sample, etc. The training optical signal corresponding to the training sample may be a training optical signal formed by encoding the training sample into an optical signal. One training sample corresponds to a group of training optical signals, and a group of training optical signals may include multiple training optical signals.

[0093] One method of obtaining the above training set is to collect historical image samples or historical voice samples from different periods by the user and perform corresponding manual annotation. Another method is to obtain the image samples with sample labels actually required by selecting data samples from a public image sample database provided by a third party, or to obtain the voice samples with sample labels actually required by selecting data samples from a public voice sample database provided by a third party.

[0094] After the training set is obtained by the above method, the training samples are encoded into training light signals for training the optical neural network.

[0095] It can be understood that the above optical neural network is an optical neural network that has not been optimized in this embodiment.

[0096] The output of the optical neural network can be used as an objective function related to the phase parameter In one embodiment, the objective function It can be expressed by the following formula (1).

[0097] (1);

[0098] in, is the current phase parameter, , is the vector composed of the input training light signal, is the transfer matrix of the optical neural network, for The transpose of .

[0099] In some other embodiments, the objective function can be expressed as ,in, , is the vector composed of the input training light signal, is the transfer matrix of the optical neural network, for O is the transpose of the output selection matrix. The elements on the diagonal of the output selection matrix are 1 or 0. An element of 1 indicates that the light in the waveguide is selected as output, and 0 indicates that it is not selected.

[0100] For example, if the light in the first waveguide of the two waveguides is selected as the output, the output selection , for example, if the light in the second waveguide of the two waveguides is selected as the output, then the matrix , for example, if all the light from the two waveguides is selected as output, then the matrix It should be noted that the same applies to the case of multiple waveguides.

[0101] The parameter offset can be determined according to a preset method and pre-configured, for example, the parameter offset can be .

[0102] The parameter offset processing may include adding the current phase parameter to the parameter offset and subtracting the current phase parameter from the parameter offset.

[0103] For example, using a parameter offset to perform parameter offset processing on the current phase parameter of at least one optical interference unit to obtain an optical neural network after parameter offset may include: adding the current phase parameter to the parameter offset to obtain an optical neural network after a positive offset; subtracting the current phase parameter from the parameter offset to obtain an optical neural network after a negative offset.

[0104] Optical neural network after forward migration It can be expressed by the following formula (2).

[0105] (2);

[0106] in, is the parameter offset.

[0107] Optical neural network after negative migration It can be expressed by the following formula (3).

[0108] (3);

[0109] in, is the parameter offset.

[0110] Specifically, for example, the optical neural network includes an optical interference unit A and an optical interference unit B, the current phase parameter of the optical interference unit A and the current phase parameter of the optical interference unit B are respectively added to the parameter offset to obtain the optical neural network after positive offset; the current phase parameter of the optical interference unit A and the current phase parameter of the optical interference unit B are respectively subtracted from the parameter offset to obtain the optical neural network after negative offset.

[0111] The second output data may include positively offset output data output by the optical neural network after positive offset and negatively offset output data output by the optical neural network after negative offset.

[0112] For example, inputting the training light signal corresponding to the training sample into the optical neural network after parameter offset may include inputting the training light signal corresponding to the training sample into the optical neural network after positive offset, and outputting positive offset output data; inputting the training light signal corresponding to the training sample into the optical neural network after negative offset, and outputting negative offset output data.

[0113] By using the parameter offset to perform parameter offset processing on the optical neural network, an optical neural network after positive offset and an optical neural network after negative offset are obtained, so as to obtain positive offset output data and negative offset output data, so that the current parameter formula can be determined using the parameter offset formula.

[0114] The preset loss function can be a mean square error loss function, a logarithmic loss function, a cross entropy loss function, etc.

[0115] Optimizing the current phase parameter of the optical neural network using the current parameter gradient may include processing the current parameter gradient and the current phase parameter using a preset gradient descent formula to obtain an optimized phase parameter; and adjusting the current phase parameter of at least one optical interference unit according to the optimized phase parameter.

[0116] The gradient descent formula can be expressed as follows (4).

[0117] (4);

[0118] in, is the current phase parameter, is the current parameter gradient, is the optimized phase parameter, For the learning rate, the learning rate can be pre-configured.

[0119] The current phase parameter of the optical neural network can be optimized by using the current parameter gradient. The current phase parameter can also be optimized by using an optimizer method. For example, the current phase parameter can be optimized by using an Adam optimization algorithm.

[0120] According to an embodiment of the present invention, adjusting the current phase parameter of at least one optical interference unit according to the optimized phase parameter includes: for each optical interference unit, adjusting the current phase parameter of the optical interference unit by adjusting the voltage and / or temperature of the optical interference unit according to the optimized phase parameter of the optical interference unit.

[0121] According to an embodiment of the present invention, a training light signal corresponding to a training sample is input into an optical neural network, and first output data processed by the optical neural network is output. Then, parameter offset processing is performed on the current phase parameter of the optical neural network using a parameter offset. The training light signal corresponding to the training sample is input into the optical neural network after the parameter offset, and second output data is output. Thereafter, the first output data, the second output data and a preset loss function are used to calculate the current parameter gradient, thereby achieving efficient and accurate calculation of the parameter gradient, which in turn helps to improve the training efficiency of the optical neural network.

[0122] Figure 3 is a schematic diagram of a training method for an optical neural network according to an embodiment of the present invention.

[0123] like Figure 3 As shown, the optical neural network 320 of this embodiment includes the current phase parameter and the current phase parameters ; Set the current phase parameter and the current phase parameters Subtract the parameter offset to obtain the negatively offset optical neural network 330, wherein the negatively offset optical neural network 330 includes the phase parameter after the negative offset And the phase parameters after negative offset ; Set the current phase parameter and the current phase parameters The optical neural network 340 after the forward shift is obtained by adding the parameter offset, wherein the optical neural network 340 after the forward shift includes the phase parameter after the forward shift and the phase parameter after the positive shift ; Then, the training light signal 310 is respectively input into the optical neural network 320, the neural network 330 after negative offset and the neural network 340 after positive offset, and the first output data 321, the negative offset output data 331 and the positive offset output data 341 are output; then the current parameter gradient 350 is determined according to the first output data 321, the negative offset output data 331, the positive offset output data 341 and the preset loss function; then the current parameter gradient 350 is used to optimize the current phase parameter of the optical neural network 320 to achieve one iteration of the optical neural network, and then the above operation is performed repeatedly until the iteration stop condition is reached.

[0124] According to an embodiment of the present invention, the optical interference unit includes at least one phase shifter, and the phase shifter is represented by a transfer matrix; the parameter offset is determined by the following steps: using an offset calculation formula to process the transfer matrix to determine the parameter offset.

[0125] According to an embodiment of the present invention, the offset calculation formula includes matrix transformation parameters, and the matrix transformation parameters are expressed by a matrix transformation formula.

[0126] The offset calculation formula can be expressed as follows:

[0127] (5);

[0128] in, represents the parameter offset, and r represents the matrix transformation parameter.

[0129] Processing the transfer matrix using the offset calculation formula, determining the parameter offset may include:

[0130] First, the transfer matrix is ​​converted into an exponential form of a preset format, wherein the exponential form includes a Hermitian matrix.

[0131] For example, the transfer matrix Convert to exponential form , so that , then solve , we can conclude that a=-1 is a real number, is a Hermitian matrix.

[0132] Then, the Hermitian matrix is ​​processed using the matrix transformation formula to determine the parameter values ​​of the matrix transformation parameters.

[0133] The matrix transformation formula can be as follows (6).

[0134] (6);

[0135] Where e1 and e0 are the two eigenvalues ​​of the Hermitian matrix G, and a is -1, which is a real constant.

[0136] According to an embodiment of the present invention, using a matrix transformation formula to process a Hermitian matrix and determining parameter values ​​of matrix transformation parameters includes: determining two eigenvalues ​​for the Hermitian matrix; and using a matrix transformation formula to process the two eigenvalues ​​to obtain parameter values ​​of the matrix transformation parameters.

[0137] according to , we can get e1=1, e0=0, and thus determine the matrix transformation parameters according to formula (6): .

[0138] Afterwards, the parameter offset is determined according to the parameter value and the offset calculation formula.

[0139] For example, the parameter values ​​of the matrix transformation parameters Substituting into the above formula (5), we get the parameter offset .

[0140] The Hermitian matrix is ​​determined by changing the transfer matrix into an exponential form, and then the parameter offset is determined according to the Hermitian matrix and the offset calculation formula, so that the determination of the parameter offset is closely related to the phase shifter, so that the parameter gradient can be calculated efficiently and accurately.

[0141] Figure 4 4 is a schematic diagram of a method for determining a parameter offset according to an embodiment of the present invention.

[0142] like Figure 4 As shown, the transfer matrix 410 converted to exponential form 420, we get the equation , and then solve to obtain the Hermitian matrix G430 and parameter a440, where a=-1 is a real number, ; Then determine the two eigenvalues ​​of the Hermitian matrix G to obtain the eigenvalue e1450 and the eigenvalue e0460; Then, substitute the eigenvalue e1450, the eigenvalue e0460 and the parameter a440 into the matrix transformation formula The parameter value 470 of the matrix transformation parameter r is obtained; then the parameter value 470 of the matrix transformation parameter r is substituted into the offset calculation formula Determine the parameter offset in 480.

[0143] According to an embodiment of the present invention, the output of the optical neural network is represented by an objective function, and the objective function is a function of a phase parameter. According to the first output data, the second output data and a preset loss function, determining a current parameter gradient for a current phase parameter comprises:

[0144] The second output data is processed using a preset parameter offset formula to obtain a first gradient of the objective function relative to the current phase parameter; a second gradient of the loss function relative to the first output data is determined; and a current parameter gradient for the current phase parameter is determined based on the first gradient and the second gradient.

[0145] For the current phase parameters The current parameter gradient It can be expressed as the following formula (7).

[0146] (7);

[0147] in, represents the second gradient, Represents the first gradient.

[0148] According to an embodiment of the present invention, the second output data may include positive offset output data and negative offset output data, wherein processing the second output data using a preset parameter offset formula to obtain a first gradient of the objective function relative to the current phase parameter may include: multiplying the difference between the positive offset output data and the negative offset output data by a preset value to obtain a first gradient.

[0149] The parameter offset formula can be expressed as follows:

[0150] (8);

[0151] in, represents the first gradient, and is the second output data, where Can represent positive offset output data, It can represent negative offset output data; r is the matrix transformation parameter, , is the parameter offset, and .

[0152] The first gradient can be obtained by inputting the positive offset output data and the negative offset output data into the above formula (8).

[0153] According to an embodiment of the present invention, the loss function is a function of the objective function. Determining the second gradient of the loss function relative to the first output data includes: determining the derivative of the loss function with respect to the objective function; and processing the first output data using the derivative to obtain the second gradient.

[0154] In one embodiment, the loss function can be obtained by using the MSE loss function as shown in the following formula (9).

[0155] (9);

[0156] in, Represents the objective function, and label represents the sample label.

[0157] At this time, the derivative of the loss function with respect to the objective function can be expressed as follows:

[0158] (10).

[0159] Processing the first output data using the derivative to obtain the second gradient may include converting the first output data Entering into formula (10), the second gradient can be obtained.

[0160] According to an embodiment of the present invention, determining a current parameter gradient for a current phase parameter according to the first gradient and the second gradient includes: multiplying the first gradient and the second gradient to obtain the current parameter gradient.

[0161] Figure 5 is a schematic diagram of determining the current parameter gradient according to an embodiment of the present invention.

[0162] like Figure 5 As shown, the method for determining the current parameter gradient in this embodiment includes determining a first gradient and determining a second gradient. Determining the first gradient includes: obtaining positive offset output data 510 output by the optical neural network after positive offset and negative offset output data 520 output by the optical neural network after negative offset, wherein the positive offset output data 510 can be expressed as , the negative offset output data 520 can be expressed as ; Then, the positive offset output data 510 , negative offset output data 520 Input parameter offset formula using formula In, get This is the first gradient 530 .

[0163] Determining the second gradient includes: determining a derivative 560 of the loss function 540 with respect to the objective function, wherein the loss function 540 can be expressed as , As the objective function, the derivative 560 can be expressed as ; Then the first output data 550 is input into the derivative 560 , the second gradient 570 is obtained; then the first gradient 530 and the second gradient 570 are multiplied to obtain the current parameter gradient 580.

[0164] The present invention provides a method for training using gradient descent of an optical neural network model based on the characteristics of MZI for optical computing and a parameter offset method. The method obtains the output of the optical neural network model after adding or subtracting the parameter offset from the current phase parameter, such as positive offset output data and negative offset output data, and further calculates the output to achieve efficient and accurate calculation of the parameter gradient, thereby improving the model training efficiency.

[0165] Based on the above optical neural network training method, the present invention also provides an optical neural network training device. Figure 6 The device is described in detail.

[0166] Figure 6 4 is a structural block diagram of an optical neural network training device according to an embodiment of the present invention.

[0167] like Figure 6 As shown, the optical neural network of this embodiment has at least one optical interference unit, and the optical interference unit includes at least one phase parameter. The training device 600 of the optical neural network includes a first input-output module 610, a parameter offset module 620, a second input-output module 630, a determination module 640 and an optimization module 650.

[0168] The first input-output module 610 is used to input the training light signal corresponding to the training sample into the optical neural network, and output the first output data for the training sample.

[0169] The parameter shift module 620 is used to perform parameter shift processing on the current phase parameter of at least one optical interference unit using the parameter shift amount to obtain the optical neural network after the parameter shift.

[0170] The second input-output module 630 is used to input the training light signal corresponding to the training sample into the optical neural network after the parameter is shifted, and output second output data for the training sample.

[0171] The determination module 640 is used to determine a current parameter gradient for a current phase parameter according to the first output data, the second output data and a preset loss function.

[0172] The optimization module 650 is used to optimize the current phase parameter of the optical neural network using the current parameter gradient to obtain the optimized optical neural network.

[0173] According to an embodiment of the present invention, the optical neural network after parameter shift includes an optical neural network after positive shift and an optical neural network after negative shift.

[0174] According to an embodiment of the present invention, the parameter offset module includes: a first offset submodule and a second offset submodule.

[0175] The first offset submodule is used to add the current phase parameter to the parameter offset to obtain the optical neural network after forward offset.

[0176] The second offset submodule is used to subtract the current phase parameter from the parameter offset to obtain the optical neural network after negative offset.

[0177] According to an embodiment of the present invention, the second output data includes positive-shift output data and negative-shift output data.

[0178] According to an embodiment of the present invention, the second input-output module includes: a first input-output submodule and a second input-output submodule.

[0179] The first input-output submodule is used to input the training optical signal corresponding to the training sample into the optical neural network after forward shift, and output forward shift output data.

[0180] The second input-output submodule is used to input the training light signal corresponding to the training sample into the optical neural network after negative shift, and output negative shift output data.

[0181] According to an embodiment of the present invention, the output of the optical neural network is represented by an objective function, which is a function of a phase parameter.

[0182] According to an embodiment of the present invention, the determination module includes: a first processing submodule, a first determination submodule and a second determination submodule.

[0183] The first processing submodule is used to process the second output data using a preset parameter offset formula to obtain a first gradient of the objective function relative to the current phase parameter.

[0184] The first determination submodule is used to determine a second gradient of the loss function relative to the first output data.

[0185] The second determining submodule is used to determine a current parameter gradient for a current phase parameter according to the first gradient and the second gradient.

[0186] According to an embodiment of the present invention, the second output data includes positive-shift output data and negative-shift output data.

[0187] According to an embodiment of the present invention, the first processing submodule includes: a difference multiplication unit.

[0188] The difference multiplication unit is used to multiply the difference between the positive offset output data and the negative offset output data by a preset value to obtain a first gradient.

[0189] According to an embodiment of the present invention, the loss function is a function related to the objective function.

[0190] According to an embodiment of the present invention, the first determining submodule includes: a first determining unit and a first processing unit.

[0191] The first determining unit is used to determine the derivative of the loss function with respect to the objective function.

[0192] The first processing unit is used to process the first output data using the derivative to obtain a second gradient.

[0193] According to an embodiment of the present invention, the second determining submodule includes:

[0194] The multiplication unit is used to multiply the first gradient and the second gradient to obtain the current parameter gradient.

[0195] According to an embodiment of the present invention, the optical interference unit includes at least one phase shifter, and the phase shifter is represented by a transfer matrix; the parameter offset is determined by the following module:

[0196] The processing module is used to process the transfer matrix using the offset calculation formula to determine the parameter offset.

[0197] According to an embodiment of the present invention, the offset calculation formula includes matrix transformation parameters, and the matrix transformation parameters are expressed by a matrix transformation formula.

[0198] According to an embodiment of the present invention, the processing module includes: a conversion submodule, a second processing submodule and a third determination submodule.

[0199] The conversion submodule is used to convert the transfer matrix into an exponential form of a preset format, wherein the exponential form includes a Hermitian matrix.

[0200] The second processing submodule is used to process the Hermitian matrix using the matrix transformation formula to determine the parameter values ​​of the matrix transformation parameters.

[0201] The third determination submodule is used to determine the parameter offset according to the parameter value and the offset calculation formula.

[0202] According to an embodiment of the present invention, the second processing submodule includes: a second determining unit and a second processing unit.

[0203] The second determining unit is used to determine two eigenvalues ​​for the Hermitian matrix.

[0204] The second processing unit is used to process the two eigenvalues ​​using a matrix transformation formula to obtain parameter values ​​of the matrix transformation parameters.

[0205] According to an embodiment of the present invention, the optimization module includes: a third processing submodule and an adjustment submodule.

[0206] The third processing submodule is used to process the current parameter gradient and the current phase parameter using a preset gradient descent formula to obtain an optimized phase parameter.

[0207] The adjustment submodule is used to adjust the current phase parameter of at least one optical interference unit according to the optimized phase parameter to obtain the optimized optical neural network.

[0208] According to an embodiment of the present invention, the adjustment submodule includes: an adjustment unit.

[0209] The adjustment unit is used to adjust the current phase parameter of each optical interference unit by adjusting the voltage and / or temperature of the optical interference unit according to the optimized phase parameter of the optical interference unit.

[0210] According to an embodiment of the present invention, any multiple modules among the first input-output module 610, the parameter offset module 620, the second input-output module 630, the determination module 640 and the optimization module 650 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the first input-output module 610, the parameter offset module 620, the second input-output module 630, the determination module 640 and the optimization module 650 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in a proper combination of any of them. Alternatively, at least one of the first input-output module 610, the parameter offset module 620, the second input-output module 630, the determination module 640 and the optimization module 650 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0211] Figure 7 is a block diagram of an electronic device suitable for implementing the above method according to an embodiment of the present invention.

[0212] like Figure 7As shown, the electronic device 700 according to an embodiment of the present invention includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 to a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0213] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 704. Processor 701 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in the one or more memories.

[0214] According to an embodiment of the present invention, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the input / output (I / O) interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.

[0215] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0216] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 702 and / or RAM 703 described above and / or one or more memories other than ROM 702 and RAM 703.

[0217] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present invention.

[0218] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 701. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0219] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 709, and / or installed from the removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0220] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0221] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0222] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0223] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0224] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for training an optical neural network, characterized in that: The optical neural network includes at least one optical interference unit, and the optical interference unit has at least one phase parameter; the training method includes: Iteratively perform the following operations until the iteration stop condition is reached: Inputting a training light signal corresponding to a training sample into the optical neural network, and outputting first output data for the training sample; Performing parameter offset processing on the current phase parameter of the at least one optical interference unit using the parameter offset to obtain an optical neural network after parameter offset; Inputting a training light signal corresponding to the training sample into the optical neural network after parameter shift, and outputting second output data for the training sample; Determining a current parameter gradient for the current phase parameter according to the first output data, the second output data and a preset loss function; and The current parameter gradient is used to optimize the current phase parameter of the optical neural network to obtain an optimized optical neural network.

2. The training method according to claim 1, characterized in that: The optical neural network after parameter shift includes an optical neural network after positive shift and an optical neural network after negative shift; The step of performing parameter offset processing on the current phase parameter of the at least one optical interference unit by using the parameter offset to obtain the optical neural network after the parameter offset comprises: Adding the current phase parameter to the parameter offset to obtain the optical neural network after the forward offset; The current phase parameter is subtracted from the parameter offset to obtain the optical neural network after the negative offset.

3. The training method according to claim 2, characterized in that: The second output data includes positive offset output data and negative offset output data; Inputting the training light signal corresponding to the training sample into the optical neural network after parameter shift, and outputting second output data for the training sample comprises: Inputting a training optical signal corresponding to the training sample into the optical neural network after forward shift, and outputting the forward shift output data; The training light signal corresponding to the training sample is input into the negatively offset optical neural network, and the negatively offset output data is output.

4. The training method according to claim 1, characterized in that: The output of the optical neural network is represented by an objective function, and the objective function is a function of the phase parameter; and determining a current parameter gradient for the current phase parameter according to the first output data, the second output data, and a preset loss function comprises: Processing the second output data using a preset parameter offset formula to obtain a first gradient of the objective function relative to the current phase parameter; determining a second gradient of the loss function with respect to the first output data; A current parameter gradient for the current phase parameter is determined according to the first gradient and the second gradient.

5. The training method according to claim 4, characterized in that: The second output data includes positive offset output data and negative offset output data; The step of processing the second output data by using a preset parameter offset formula to obtain a first gradient of the objective function relative to the current phase parameter comprises: The first gradient is obtained by multiplying a difference between the positive offset output data and the negative offset output data by a preset value.

6. The training method according to claim 4, characterized in that: The loss function is a function of the objective function; Determining a second gradient of the loss function relative to the first output data comprises: Determining a derivative of the loss function with respect to the objective function; The first output data is processed using the derivative to obtain the second gradient.

7. The training method according to any one of claims 4 to 6, characterized in that: Determining a current parameter gradient for the current phase parameter according to the first gradient and the second gradient includes: The first gradient and the second gradient are multiplied to obtain the current parameter gradient.

8. The training method according to claim 1, characterized in that: The optical interference unit includes at least one phase shifter, and the phase shifter is represented by a transfer matrix; the parameter offset is determined by the following steps: The transfer matrix is ​​processed using an offset calculation formula to determine the parameter offset.

9. The training method according to claim 8, characterized in that: The offset calculation formula includes matrix transformation parameters, and the matrix transformation parameters are expressed by a matrix transformation formula; The processing of the transfer matrix by using the offset calculation formula comprises: Converting the transfer matrix into an exponential form of a preset format, wherein the exponential form includes a Hermitian matrix; Processing the Hermitian matrix using the matrix transformation formula to determine parameter values ​​of the matrix transformation parameters; The parameter offset is determined according to the parameter value and the offset calculation formula.

10. The training method according to claim 9, characterized in that: The using the matrix transformation formula to process the Hermitian matrix and determine the parameter value of the matrix transformation parameter comprises: determining two eigenvalues ​​for the Hermitian matrix; The two eigenvalues ​​are processed using the matrix transformation formula to obtain parameter values ​​of the matrix transformation parameters.

11. The training method according to claim 5, characterized in that: The step of optimizing the current phase parameter of the optical neural network by using the current parameter gradient to obtain an optimized optical neural network comprises: Processing the current parameter gradient and the current phase parameter using a preset gradient descent formula to obtain an optimized phase parameter; The current phase parameter of the at least one optical interference unit is adjusted according to the optimized phase parameter to obtain the optimized optical neural network.

12. The training method according to claim 11, characterized in that: The step of adjusting the current phase parameter of the at least one optical interference unit according to the optimized phase parameter comprises: For each of the optical interference units, according to the optimized phase parameter for the optical interference unit, the current phase parameter of the optical interference unit is adjusted by adjusting the voltage and / or temperature of the optical interference unit.

13. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the training method according to any one of claims 1 to 12.

14. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the training method according to any one of claims 1 to 12 are implemented.

15. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the training method according to any one of claims 1 to 12 are implemented.

Images