Optical neural network chip construction method and system with reduced number of optical components

Abstract

The present invention proposes a method and system for constructing an optical neural network chip that reduces the number of optical devices. The method uses a method that coordinates the chip structure and the training method to compress the number of redundant optical devices in the optical chip through low-rank approximation, and at the same time, based on a number of optimization methods, the recognition accuracy of the neural network is guaranteed. The method for constructing an optical neural network chip that reduces the number of optical devices proposed in the present invention includes weight matrix preprocessing, unitary preservation training, device pruning, and functional optical device network construction. Therefore, the present invention greatly reduces the number of optical devices used while keeping the recognition rate basically unchanged through the coordination of structure and algorithm.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and photonic computing, and in particular to a method and system for constructing an optical neural network chip that reduces the number of optical devices. Background Art

[0002] In recent years, artificial neural networks have achieved results far exceeding traditional algorithms in many fields including image recognition, speech recognition, and behavioral decision-making, and have therefore been widely used in real life. However, the deep structure and large number of neurons require neural networks to consume a lot of computing resources. The industry's solution to this is to use highly parallel GPUs or even dedicated acceleration hardware to carry out neural network operations to improve computing speed and energy efficiency. However, as the characteristic size of integrated circuits approaches physical limits, Moore's Law gradually becomes invalid. The electrical domain neural network accelerator based on CMOS technology cannot meet the computing power and energy efficiency requirements of several applications, becoming a major problem facing the information field in the next few decades.

[0003] Compared with electrical signals, optical signals have great advantages in frequency, energy efficiency and anti-interference. Optical computing technology based on optical signals has the advantages of extremely high natural parallelism, extremely fast computing speed and extremely low energy consumption. As a substitute for electrical computing technology, it has broad prospects. In particular, neural network accelerators based on optical computing technology use optical devices to construct vector matrix multiplication and addition structures to realize the matrix multiplication and addition operations with the largest amount of computation in artificial neural networks. A large-scale matrix calculation can be completed in one clock cycle, which has great speed and energy efficiency improvements. Although neural network accelerators based on optical devices have great advantages in speed and energy efficiency, their construction method directly uses SVD decomposition to realize the weight matrix of the neural network on the optical device network, which requires the use of square-order optical devices. This increases the area and energy consumption of the optical chip on the one hand, and also amplifies the random errors in the manufacture of each optical device on the other hand, which reduces the fault tolerance of the entire network and increases the difficulty of control, becoming an important obstacle to the practical application of photonic computing. Summary of the invention

[0004] In order to overcome the problems of the above-mentioned prior art that large area is occupied, many devices are consumed, and the operation is complex when mapping large-scale neural networks onto optical structures, the present invention proposes a method for constructing an optical neural network chip that can greatly reduce the number of optical devices used. Through low-rank approximation of neural network weights and a training method that matches the characteristics of optical devices, the number of devices required for the optical neural network chip is greatly reduced while the recognition rate remains basically unchanged, thereby reducing the chip area and the complexity of operation.

[0005] Specifically, the present invention proposes a method for constructing an optical neural network chip that reduces the number of optical components, including:

[0006] Step 1: Obtain an original neural network to be mapped to an optical device, and convert the original neural network into a matrix sequence consisting of diagonal matrices and unitary matrices by decomposing each weight matrix in the neural network into a diagonal matrix and a unitary matrix.

[0007] Step 2: Input the samples in the training set into the matrix sequence, and obtain the loss value Loss by comparing the last output in the matrix sequence with the label of the training set;

[0008] Step 3: Through the regularization function, the diagonal matrix and unitary matrix of each weight matrix are obtained to constrain the regular terms of the parameter optimization direction. All regular terms are added to the loss value Loss to obtain the optimization objective function FullLoss.

[0009] Step 4: Use the optimization objective function FullLoss to derive the diagonal matrix and unitary matrix of each weight matrix, and based on the obtained partial derivatives, update the diagonal matrix and unitary matrix of each weight matrix through back propagation and gradient descent;

[0010] Step 5, compare the diagonal matrix and the unitary matrix after and before the update to determine whether the difference between the two is lower than the threshold. If so, execute step 7, otherwise execute step 6;

[0011] Step 6: Prune the updated diagonal matrix and unitary matrix and then execute steps 2 to 5 again;

[0012] Step 7: Map the updated diagonal matrix and unitary matrix to the optical device to generate an optical neural network chip corresponding to the original neural network.

[0013] The method for constructing an optical neural network chip with reduced number of optical components, wherein step 1 comprises:

[0014] The weight matrix W in the original neural network is decomposed to obtain a semi-unitary matrix U, a non-negative diagonal matrix Σ and a semi-unitary matrix V;

[0015] This step 3 includes:

[0016] For the matrices U, Σ and V of each layer in the original neural network, two regularization terms g(U, V, Σ) and h(U, V, Σ) are calculated respectively. The calculated values ​​of the regularization terms of all matrices and the Loss are added together to obtain the complete optimization objective function FullLoss.

[0017] In the method for constructing an optical neural network chip that reduces the number of optical components, the calculation method of the regularization term g(U, V, Σ) is:

[0018]

[0019] The diagonal values ​​of Σ are σ1,σ2,...,σ r , the vector formed by the diagonal values ​​is s = [σ1,σ2,…,σ r ] T ,use and denote the 1-norm and 2-norm of s respectively;

[0020] The calculation method of the regularization term h(U, V, Σ) is:

[0021]

[0022] where Δ u =I r -Q u , Δ v =I r -Q v , I r represents the identity matrix of size r×r, Q u and Q v is a strictly semiunitary matrix with the same dimensions as the matrices U and V respectively.

[0023] The method for constructing an optical neural network chip with reduced number of optical components, wherein step 6 comprises:

[0024] The pruning method is used to reduce the ranks of the semi-unitary matrix U, the non-negative diagonal matrix Σ and the semi-unitary matrix V.

[0025] The method for constructing an optical neural network chip with reduced number of optical components, wherein step 7 comprises:

[0026] The updated semi-unitary matrix V and semi-unitary matrix U are decomposed into the product of the diagonal matrix and the unitary matrix that only works on n channels, respectively. According to the transformation matrix in the product and combined The optical devices are connected in series in the order of to obtain the updated mapping of the diagonal matrix and the unitary matrix to the optical devices.

[0027] The present invention also proposes an optical neural network chip construction system that reduces the number of optical components, including:

[0028] Module 1 is used to obtain the original neural network to be mapped to the optical device, and convert the original neural network into a matrix sequence consisting of diagonal matrices and unitary matrices by decomposing each weight matrix in the neural network into a diagonal matrix and a unitary matrix.

[0029] Module 2 is used to input the samples in the training set into the matrix sequence, and obtain the loss value Loss by comparing the last output in the matrix sequence with the label of the training set;

[0030] Module 3 is used to obtain the diagonal matrix and unitary matrix of each weight matrix through the regularization function to constrain the regular terms of the parameter optimization direction, and add all regular terms to the loss value Loss to obtain the optimization objective function FullLoss;

[0031] Module 4 is used to use the optimization objective function FullLoss to derive the diagonal matrix and unitary matrix of each weight matrix, and based on the obtained partial derivatives, update the diagonal matrix and unitary matrix of each weight matrix through back propagation and gradient descent;

[0032] Module 5 is used to compare the diagonal matrix and the unitary matrix after and before the update to determine whether the difference between the two is lower than a threshold. If so, execute module 7, otherwise execute module 6;

[0033] Module 6, used for pruning the updated diagonal matrix and unitary matrix and then executing modules 2 to 5 again;

[0034] Module 7 is used to map the updated diagonal matrix and unitary matrix to the optical device to generate an optical neural network chip corresponding to the original neural network.

[0035] The optical neural network chip construction system with reduced number of optical components, wherein the module 1 comprises:

[0036] The weight matrix W in the original neural network is decomposed to obtain a semi-unitary matrix U, a non-negative diagonal matrix Σ and a semi-unitary matrix V;

[0037] This module 3 includes:

[0038] For the matrices U, Σ and V of each layer in the original neural network, two regularization terms g(U, V, Σ) and h(U, V, Σ) are calculated respectively. The calculated values ​​of the regularization terms of all matrices and the Loss are added together to obtain the complete optimization objective function FullLoss.

[0039] In the optical neural network chip construction system for reducing the number of optical devices, the calculation process of the regularization term g(U, V, Σ) is:

[0040]

[0041] The diagonal values ​​of Σ are σ1, σ2, …, σ r , the vector formed by the diagonal values ​​is s = [σ1,σ2,…,σ r ] T ,use and denote the 1-norm and 2-norm of s respectively;

[0042] The calculation process of the regularization term h(U, V, Σ) is:

[0043]

[0044] where Δ u =I r -Q u , Δ v =I r -Q v , I r represents the identity matrix of size r×r, Q u and Q v is a strictly semiunitary matrix with the same dimensions as the matrices U and V respectively.

[0045] The optical neural network chip construction system with reduced number of optical components, wherein the module 6 comprises:

[0046] Pruning is used to reduce the ranks of the semi-unitary matrix U, the non-negative diagonal matrix Σ, and the semi-unitary matrix V.

[0047] The optical neural network chip construction system with reduced number of optical devices, wherein the module 7 comprises:

[0048] The updated semi-unitary matrix V and semi-unitary matrix U are decomposed into the product of the diagonal matrix and the unitary matrix that only works on n channels, respectively. According to the transformation matrix in the product and combined The optical devices are connected in series in the order of to obtain the updated mapping of the diagonal matrix and the unitary matrix to the optical devices.

[0049] It can be seen from the above scheme that the advantages of the present invention are:

[0050] A neural network processing chip is constructed using highly parallel, low-energy and anti-interference optical signals and devices, which can greatly improve the speed and energy efficiency of neural network computing and processing; at the same time, the invention can greatly reduce the use of optical devices (greater than 80%), and can use fewer optical devices and optical chip areas to achieve the same functions, which not only reduces the area of ​​the chip, but also reduces the difficulty of operating the optical devices, greatly reduces the cost, and is of great value to the application and deployment of optical neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of the three-layer neural network structure;

[0052] Figure 2 is a training flow chart of the optical neural network of the present invention;

[0053] Figure 3 A single iteration flow chart of the optical neural network of the present invention;

[0054] Figure 4The schematic diagram of pruning the network from rank 3 to rank 2, where the gray part is the removed component;

[0055] Figure 5 This is a low-rank (rank 2) optical neural network structure diagram of the first layer of the network in the implementation example. DETAILED DESCRIPTION

[0056] After investigation and experimentation, the inventor found that the number of redundant optical devices in the optical chip can be compressed by low-rank approximation through the coordination of chip structure and training method, and the recognition accuracy of the neural network can be guaranteed based on a number of optimization methods. Therefore, the present invention proposes a design method for an optical neural network chip that reduces the number of optical devices. Through the coordination of structure and algorithm, the number of optical devices used can be greatly reduced while the recognition rate remains basically unchanged.

[0057] The optical neural network chip construction method for reducing the number of optical devices proposed in the present invention includes four main processes and sub-methods, namely, a weight matrix preprocessing method, a unitary preservation training method, a device pruning method, and a functional optical device network construction method. Among them:

[0058] The weight matrix preprocessing method refers to processing the weight matrix connecting two levels of the neural network, decomposing the weight matrix into a diagonal matrix and a unitary matrix using the matrix decomposition method, and then training and processing the multiple matrices obtained separately, rather than processing them as one matrix.

[0059] The unitary preserving training method refers to the process of constraining the optimization direction of the parameters by controlling the regularization function in the objective function during the training process, ensuring that the unitary matrix after the weight matrix processing remains an approximate semi-unitary matrix. This method can ensure that the U and V matrices do not deviate too far from the semi-unitary matrix, so that there will not be too much loss in the mapping operation after the training is completed.

[0060] The device pruning method refers to inducing the weight matrix to update in the direction of rank reduction through the control of the regularization function during the training process; at the same time, after one or more steps of training, the components corresponding to the smaller singular values ​​are pruned, thereby reducing the number of parameters and further reducing the number of optical devices used; moreover, the setting of the singular value threshold can be dynamically set according to user needs between the recognition rate and the number of devices.

[0061] The method for constructing a functional optical device network refers to mapping a number of trained matrices (including diagonal matrices, unitary matrices, etc.) to devices such as optical couplers, attenuators, phase modulators and Mach-Zehnder interferometers, and obtaining a functional optical device network corresponding to the matrix parameters by adjusting the parameters.

[0062] The optical neural network chip construction method that can greatly reduce the number of optical devices used refers to processing layer by layer through matrix preprocessing methods and device pruning methods according to the logical level of the neural network, and overall processing through unitary retention training methods and functional optical device retention methods. The matrix preprocessing method and the functional device retention method only need to be run once in the whole process, while the unitary retention training method and the device pruning method need to be iterated several times.

[0063] The optical computing chip constructed can be used in any system that requires neural network processing, such as autonomous driving and video surveillance.

[0064] In order to make the above features and effects of the present invention more clearly understood, embodiments are given below and described in detail with reference to the accompanying drawings.

[0065] The training and mapping method proposed in the present invention can solve the mapping problem of any neural network to an optical device, and is independent of the number of layers, connection relationships, and number of neurons of the neural network, wherein the neural network can be, for example, a neural network composed of a fully connected layer and a convolutional layer (which can be converted into a fully connected layer), such as a multilayer perceptron, LeNet, AlexNet, VGG, etc. In order to more clearly demonstrate the concept of the present invention, a three-layer neural network is used as an example for explanation, wherein the three-layer neural network includes an input layer, a hidden layer, and an output layer. Without loss of generality, it is assumed that the input layer includes 5 neurons, the hidden layer includes 4 neurons, and the output layer includes 2 neurons, and the layers are connected by full connection. The neural network structure used in this implementation example is as follows: Figure 1 shown.

[0066] The optical neural network chip construction method for reducing the number of optical devices proposed in the present invention includes four main processes and sub-methods, namely, a weight matrix preprocessing method, a unitary preservation training method, a device pruning method and a functional optical device network construction method.

[0067] In one embodiment, before training begins, a weight matrix preprocessing method is used. The so-called weight matrix processing method refers to processing the weight matrix connecting two levels of the neural network, decomposing the weight matrix into a diagonal matrix and a unitary matrix using a matrix decomposition method, and then training and processing the obtained multiple matrices separately, rather than processing them as one matrix.

[0068] Step S110: Decompose the connection matrix (4×5 weight matrix) W of the first layer using the SVD method;

[0069] Step S120: the obtained 4×4 semi-unitary matrix U, 4×4 non-negative diagonal matrix Σ, and 5×4 semi-unitary matrix V are stored separately and processed separately;

[0070] Step S130: Decompose the other layers in the neural network in accordance with step S110;

[0071] Step S140: The neural network will become a series of three matrices in sequence. For example, the original neural network is regarded as a sequence, and each item in the sequence is the weight matrix between layers (if it is a convolution layer, it is the matrix formed by the rearrangement of the convolution kernel). After step S140, the neural network is still a sequence, and each item in the sequence is decomposed into three matrices. These three matrices have a characteristic that when they are multiplied together, they are equal to the corresponding matrices in the original sequence. Suppose the original neural network contains the matrix sequence [W1, W2], and the SVD decomposition gives Then after step S140, the matrix sequence of the neural network becomes in Represents the conjugate transpose of a matrix.

[0072] In one embodiment, the three matrices U, Σ and V obtained in steps S110, S120 and S130 are used as initial parameters, and training is performed using a unitary preserving algorithm and a device pruning method. The so-called unitary preserving training method refers to the process of constraining the optimization direction of the parameters by controlling the regularization function in the objective function during the training process, to ensure that the unitary matrix after weight matrix processing remains a unitary matrix and does not change its properties during the training process. The so-called device pruning method refers to pruning the components corresponding to smaller singular values ​​after one or several steps of training, thereby reducing the number of parameters and further reducing the number of optical devices used; moreover, the setting of the singular value threshold can be dynamically set between the recognition rate and the number of devices according to user needs. The overall process is as follows: Figure 2 shown.

[0073] In this embodiment, several iterations are required to stabilize the number of parameters. For each iteration:

[0074] Step S210 inputs samples of a training set selected from a public data set, such as MNIST, FashionMNIST, ImageNet, etc., into the first matrix of a series of matrices obtained in S140, and compares the last output of the series of matrices with the label of the training set to calculate the loss value Loss. The matrix sequence of the neural network is Then when the input is vector x, the output The label records the correct output of the sample;

[0075] Step S220 calculates two regularization terms g(U, V, Σ) and h(U, V, Σ) for the parameter matrices U, Σ and V of each layer respectively. The regularization terms play the role of constraining the optimization direction of the parameters. The calculated values ​​of the regularization terms of all matrices are added to the Loss to obtain the complete optimization objective function FullLoss.

[0076] Step S230 uses FullLoss to derive each parameter matrix, and uses the calculated partial derivatives to update the parameter matrix with an appropriate learning rate γ, i.e., the back propagation and gradient descent process;

[0077] Step S240 compares the parameter matrix before and after the update. If the difference between the two is small enough, it is considered that the parameter matrix has converged and the training is stopped. Otherwise, proceed to the next step.

[0078] Step S250 applies device pruning to the parameter matrix according to the value of Σ, and cuts the size of the parameter matrix;

[0079] Step S260 goes to S210 for a new round of iteration and continues training with the pruned U, Σ and V.

[0080] Steps S210-S260 together constitute the key process in one iteration of the embodiment, namely, the training and updating of parameters, wherein the detailed process is as follows: Figure 3 shown.

[0081] In step S220, one of the regularization terms g(U, V, Σ) is one of the components of the device pruning method, which is used to measure the rank of the parameter matrix, so that U, V and Σ can be updated in the direction that can reduce the rank of the parameter matrix in the subsequent gradient descent. Note that any function with the above-mentioned rank measurement characteristics can be used as a regularization term in this method. The following regularization term formula is only one example, and it does not mean that only the following regularization term formula can be used. One of the regularization terms can be shown in Formula 1, where the diagonal values ​​of Σ are σ1, σ2, …, σ r , the vector formed by the diagonal values ​​is s = [σ1,σ2,…,σ r ] T ,use and denote the 1-norm and 2-norm of s respectively. Then the calculation method of the regularization term g(U,V,Σ) is

[0082]

[0083] In step S250, the components with low importance in the high-rank parameter matrix are reduced according to the value of Σ to turn it into a low-rank matrix. This method can configure the pruning threshold by setting the parameter β, so as to strike a balance between the low rank and recognition rate of the neural network. For example, one method is to calculate the sum of all diagonal elements in Σ, set the pruning threshold to 1 / β of this sum, and if the diagonal element located in the i-th row and i-th column is less than the pruning threshold, both the i-th row and i-th column of Σ are deleted, and the i-th column of U is deleted, and the i-th column of V is deleted. The size change of the matrix before and after pruning is as follows: Figure 4 It should be noted that any method that can reduce unimportant components in a high-rank matrix can be used as a means of implementing the device pruning method, and is not limited to the methods mentioned above.

[0084] In step S220, one of the regularization terms h(U, V, Σ) plays a unitary preservation role, and its function is to measure the distance between U and V in the parameter matrix and the unitary matrix, so that U and V can be updated in the direction of maintaining the unitary matrix in the subsequent gradient descent. Any function with the above characteristics of measuring the distance of the unitary matrix can be used as a regularization term in this method. Without loss of generality, a specific method for calculating h(U, V, Σ) is given below. Applying QR decomposition to U of size m×r and V of size n×r respectively, they can be expressed as U=Q u R u and V = Q v R v In the form of u and Q v is a semi-unitary matrix with the same dimensions as U and V; R u and R v are all square matrices of size r×r. r represents the identity matrix of size r×r, 1 1×r represents a row vector of length r and all elements are 1, let Δ u =I r -Q u , Δ v =I r -Q v , using the ° operator to represent the Hadamard product of the matrix, then the regularization term h(U,V,Σ) is calculated as

[0085]

[0086] After the iterative training is completed, a series of matrices U, Σ and V are mapped to the optical device using the optical device network construction sub-method to form a photonic chip structure with neural network functions. Different from the existing technology that can only map square unitary matrices to optical devices, the improved method proposed in the present invention can map rectangular semi-unitary matrices to optical devices.

[0087] The method for constructing a functional optical device network refers to mapping several trained matrices (including diagonal matrices, unitary matrices, etc.) to devices such as optical couplers, attenuators, phase modulators, and Mach-Zehnder interferometers, and obtaining a functional optical device network corresponding to the matrix parameters by adjusting the parameters. Assuming that the rank of the final weight matrix is ​​2, that is, the sizes of U, Σ, and V are 4×2, 2×2, and 5×2, respectively, U and V can be decomposed into the product of a diagonal matrix and a series of unitary matrices that only work on two channels:

[0088] U=T 3，4 T 2，3 T 1，2 T 3，4 T 2,3 D u

[0089] V=D v T 2,3 T 3,4 T 4,5 T 1,2 T 2,3 T 3,4 T 4,5

[0090] Where T j,j+1 represents the transformation matrix of the optical signals of the jth and j+1th channels through the Mach-Zehnder interferometer, and the diagonal matrix Σ, D u and D v Simply insert a separate phase modulator and attenuator into each channel. After connecting the optical devices in series in the order of , the mapping of the low-rank neural network on the optical device is obtained. In the implementation example, the structure of the first layer of the network (rank 2) mapped to the optical device is as follows Figure 5 As shown. u ,D v Like Σ, it is a diagonal matrix, D u is the unitary matrix after diagonalization of U, D v is V after diagonalization using a unitary matrix.

[0091] Through the combination of the above methods, through the low-rank approximation of the neural network weights and the training method matching the characteristics of the optical device, the number of devices required for the optical neural network chip is greatly reduced while the recognition rate remains basically unchanged, and the chip area and the complexity of operation are reduced.

[0092] Based on the above method, photonic computing chips and systems can be constructed for autonomous driving, video surveillance, and any other applications that require neural network processing. The input of the optical chip can be image data collected by image sensors or obstacle information detected by lidar.

[0093] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment. In order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiment.

[0094] The present invention also proposes an optical neural network chip construction system that reduces the number of optical components, including:

[0095] Module 1 is used to obtain the original neural network to be mapped to the optical device, and convert the original neural network into a matrix sequence consisting of diagonal matrices and unitary matrices by decomposing each weight matrix in the neural network into a diagonal matrix and a unitary matrix.

[0096] Module 2 is used to input the samples in the training set into the matrix sequence, and obtain the loss value Loss by comparing the last output in the matrix sequence with the label of the training set;

[0097] Module 3 is used to obtain the diagonal matrix and unitary matrix of each weight matrix through the regularization function to constrain the regular terms of the parameter optimization direction, and add all regular terms to the loss value Loss to obtain the optimization objective function FullLoss;

[0098] Module 4 is used to use the optimization objective function FullLoss to derive the diagonal matrix and unitary matrix of each weight matrix, and based on the obtained partial derivatives, update the diagonal matrix and unitary matrix of each weight matrix through back propagation and gradient descent;

[0099] Module 5 is used to compare the diagonal matrix and the unitary matrix after and before the update to determine whether the difference between the two is lower than a threshold. If so, execute module 7, otherwise execute module 6;

[0100] Module 6, used for pruning the updated diagonal matrix and unitary matrix and then executing modules 2 to 5 again;

[0101] Module 7 is used to map the updated diagonal matrix and unitary matrix to the optical device to generate an optical neural network chip corresponding to the original neural network.

[0102] The optical neural network chip construction system with reduced number of optical components, wherein the module 1 comprises:

[0103] The weight matrix W in the original neural network is decomposed to obtain a semi-unitary matrix U, a non-negative diagonal matrix Σ and a semi-unitary matrix V;

[0104] This module 3 includes:

[0105] For the matrices U, Σ and V of each layer in the original neural network, two regularization terms g(U, V, Σ) and h(U, V, Σ) are calculated respectively. The calculated values ​​of the regularization terms of all matrices and the Loss are added together to obtain the complete optimization objective function FullLoss.

[0106] In the optical neural network chip construction system for reducing the number of optical devices, the calculation process of the regularization term g(U, V, Σ) is:

[0107]

[0108] The diagonal values ​​of Σ are σ1, σ2, …, σ r , the vector formed by the diagonal values ​​is s = [σ1,σ2,…,σ r ] T ,use and denote the 1-norm and 2-norm of s respectively;

[0109] The calculation process of the regularization term h(U, V, Σ) is:

[0110]

[0111] where Δ u =I r -Q u , Δ v =I r -Q v , I r represents the identity matrix of size r×r, Q u and Q v is a strictly semiunitary matrix with the same dimensions as the matrices U and V respectively.

[0112] The optical neural network chip construction system with reduced number of optical components, wherein the module 6 comprises:

[0113] Pruning is used to reduce the ranks of the semi-unitary matrix U, the non-negative diagonal matrix Σ, and the semi-unitary matrix V.

[0114] The optical neural network chip construction system with reduced number of optical components, wherein the module 7 comprises:

[0115] The updated semi-unitary matrix V and semi-unitary matrix U are decomposed into the product of the diagonal matrix and the unitary matrix that only works on n channels, respectively. According to the transformation matrix in the product and combined The optical devices are connected in series in the order of to obtain the updated mapping of the diagonal matrix and the unitary matrix to the optical devices.

Claims

1. A method for constructing an optical neural network chip with reduced number of optical components, characterized in that: include: Step 1: obtaining an original neural network to be mapped to an optical device, and converting the original neural network into a matrix sequence consisting of a diagonal matrix and a unitary matrix by decomposing each weight matrix in the neural network into a diagonal matrix and a unitary matrix; Step 2: Input the samples in the training set into the matrix sequence, and obtain the loss value Loss by comparing the last output in the matrix sequence with the label of the training set; Step 3: Through the regularization function, the diagonal matrix and unitary matrix of each weight matrix are obtained to constrain the regular terms of the parameter optimization direction. All regular terms are added to the loss value Loss to obtain the optimization objective function FullLoss. Step 4: Use the optimization objective function FullLoss to derive the diagonal matrix and unitary matrix of each weight matrix, and based on the obtained partial derivatives, update the diagonal matrix and unitary matrix of each weight matrix through back propagation and gradient descent; Step 5, compare the diagonal matrix and the unitary matrix after and before the update to determine whether the difference between the two is lower than the threshold. If so, execute step 7, otherwise execute step 6; Step 6: Prune the updated diagonal matrix and unitary matrix and then execute steps 2 to 5 again; Step 7: Map the updated diagonal matrix and unitary matrix to the optical device to generate an optical neural network chip corresponding to the original neural network.

2. The method for constructing an optical neural network chip with reduced number of optical components as claimed in claim 1, characterized in that: This step 1 includes: The weight matrix W in the original neural network is decomposed to obtain a semi-unitary matrix U, a non-negative diagonal matrix Σ and a semi-unitary matrix V; This step 3 includes: For the matrices U, Σ and V of each layer in the original neural network, two regularization terms g(U, V, Σ) and h(U, V, Σ) are calculated respectively. The calculated values ​​of the regularization terms of all matrices and the Loss are added together to obtain the complete optimization objective function FullLoss.

3. The method for constructing an optical neural network chip with reduced number of optical components as claimed in claim 2, characterized in that: The calculation method of the regularization term g(U,V,Σ) is: The diagonal values ​​of Σ are σ1, σ2, …, σ r , the vector formed by the diagonal values ​​is s = [σ1,σ2,…,σ r ] T ,use and denote the 1-norm and 2-norm of s respectively; The calculation method of the regularization term h(U,V,Σ) is: Where Δ u =I r -Q u , Δ v =I r -Q v , I r represents the identity matrix of size r×r, Q u and Q v is a strictly semiunitary matrix with the same dimensions as the matrices U and V respectively.

4. The method for constructing an optical neural network chip with reduced number of optical components as claimed in claim 2, characterized in that: This step 6 includes: The pruning method is used to reduce the ranks of the semi-unitary matrix U, the non-negative diagonal matrix Σ and the semi-unitary matrix V.

5. The method for constructing an optical neural network chip with reduced number of optical components as claimed in claim 3, characterized in that: This step 7 includes: The updated semi-unitary matrix V and semi-unitary matrix U are decomposed into the product of the diagonal matrix and the unitary matrix that only works on n channels, respectively. According to the transformation matrix in the product and combined The optical devices are connected in series in the order of to obtain the updated mapping of the diagonal matrix and the unitary matrix to the optical devices.

6. An optical neural network chip construction system that reduces the number of optical components, characterized in that: include: Module 1 is used to obtain an original neural network to be mapped to an optical device, and to convert the original neural network into a matrix sequence consisting of a diagonal matrix and a unitary matrix by decomposing each weight matrix in the neural network into a diagonal matrix and a unitary matrix; Module 2 is used to input the samples in the training set into the matrix sequence, and obtain the loss value Loss by comparing the last output in the matrix sequence with the label of the training set; Module 3 is used to obtain the diagonal matrix and unitary matrix of each weight matrix through the regularization function to constrain the regular terms of the parameter optimization direction, and add all regular terms to the loss value Loss to obtain the optimization objective function FullLoss; Module 4 is used to use the optimization objective function FullLoss to derive the diagonal matrix and unitary matrix of each weight matrix, and based on the obtained partial derivatives, update the diagonal matrix and unitary matrix of each weight matrix through back propagation and gradient descent; Module 5 is used to compare the diagonal matrix and the unitary matrix after and before the update to determine whether the difference between the two is lower than a threshold. If so, execute module 7, otherwise execute module 6; Module 6, used for pruning the updated diagonal matrix and unitary matrix and then executing modules 2 to 5 again; Module 7 is used to map the updated diagonal matrix and unitary matrix to the optical device to generate an optical neural network chip corresponding to the original neural network.

7. The optical neural network chip construction system for reducing the number of optical components as claimed in claim 6, characterized in that: This module 1 includes: The weight matrix W in the original neural network is decomposed to obtain a semi-unitary matrix U, a non-negative diagonal matrix Σ and a semi-unitary matrix V; This module 3 includes: For the matrices U, Σ and V of each layer in the original neural network, two regularization terms g(U, V, Σ) and h(U, V, Σ) are calculated respectively. The calculated values ​​of the regularization terms of all matrices and the Loss are added together to obtain the complete optimization objective function FullLoss.

8. The optical neural network chip construction system for reducing the number of optical components as claimed in claim 7, characterized in that: The calculation process of the regularization term g(U,V,Σ) is: The diagonal values ​​of Σ are σ1, σ2, …, σ r , the vector formed by the diagonal values ​​is s = [σ1,σ2,…,σ r ] T ,use and denote the 1-norm and 2-norm of s respectively; The calculation process of the regularization term h(U,V,Σ) is: where Δ u =I r -Q u , Δ v =I r -Q v , I r represents the identity matrix of size r×r, Q u and Q v is a strictly semiunitary matrix with the same dimensions as the matrices U and V respectively.

9. The optical neural network chip construction system for reducing the number of optical components as claimed in claim 7, characterized in that: This module 6 includes: Pruning is used to reduce the ranks of the semi-unitary matrix U, the non-negative diagonal matrix Σ, and the semi-unitary matrix V.

10. The optical neural network chip construction system for reducing the number of optical components as claimed in claim 8, characterized in that: This module 7 includes: The updated semi-unitary matrix V and semi-unitary matrix U are decomposed into the product of the diagonal matrix and the unitary matrix that only works on n channels, respectively. According to the transformation matrix in the product and combined The optical devices are connected in series in the order of to obtain the updated mapping of the diagonal matrix and the unitary matrix to the optical devices.

Images