An optical matrix-vector multiplier based on unitary-diagonal matrix decomposition

Abstract

The application discloses an optical matrix vector multiplier based on a unitary matrix-diagonal matrix decomposition. The multiplier comprises a Mach-Zehnder interferometer array for realizing a synthetic unitary matrix Omega and a modulation unit for realizing a corresponding diagonal matrix Sigma, wherein a target weight matrix M satisfies a decomposition form of M=ΩΣ. The method fuses two unitary matrices required for mapping in a traditional singular value decomposition into a single synthetic unitary matrix Omega, so that a unitary transformation can be completed by only one interferometer array of optical hardware, and a complete matrix multiplication is realized in cooperation with the modulation unit. Through algorithm-hardware collaborative design, the number of basic optical interference units required is significantly reduced, the hardware complexity, manufacturing cost and system power consumption of the optical neural network are greatly reduced while the computing performance is maintained, and an effective solution is provided for efficient optical computing acceleration of artificial intelligence.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence hardware acceleration and optical computing technology, specifically to an optical matrix-vector multiplication computing device based on matrix decomposition. Background Technology

[0002] Optical neural networks are a product of the deep integration of optical information processing technology and neural network theory. Their core lies in utilizing the interaction between light and matter to map the computational process of a neural network onto an optical platform to execute specific computational tasks. Leveraging the inherent characteristics of photons, such as high bandwidth, low power consumption, strong parallelism, low crosstalk, and resistance to electromagnetic interference, optical neural networks offer a promising technological path to overcome the computing power and energy efficiency bottlenecks faced in the post-Moore's Law era.

[0003] Matrix-vector multiplication is the core of optical neural networks. Although this operation accounts for the majority of the computational cost in neural network training, it can be efficiently implemented on optical platforms by leveraging the coherence and superposition properties of light. Silicon photonics provides an important pathway for realizing optical matrix-vector multiplication, and its devices are compatible with standard complementary metal-oxide-semiconductor (CMOS) processes, possessing excellent integration potential.

[0004] Optical matrix-vector multipliers utilize the physical properties of light, such as interference and refraction, to construct an architectural paradigm of "transmission as computation, structure as function," enabling computational tasks to be completed with fewer functional units, thus significantly saving hardware resources and improving system fault tolerance. However, existing optical matrix-vector multipliers suffer from limitations such as fixed computational architecture and single-function operators. Once the system hardware architecture and optical path configuration are fixed, the system typically only performs specific types of computations (such as matrix multiplication or convolution operations), lacking the ability to flexibly reconstruct multifunctional operators, resulting in insufficient computational versatility. Furthermore, current parameter mapping mainly relies on singular value decomposition methods, which decompose any matrix into the product of two unitary matrices on the left and right and a diagonal matrix. This requires constructing a symmetric unitary matrix structure in the optical implementation, further increasing system cost and complexity. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a solution that optimizes the mapping mechanism and physical implementation device of the existing optical matrix-vector multiplier, thereby significantly reducing the cost and complexity of the computing system.

[0006] The technical solution adopted by this invention to solve the above-mentioned technical problems is an optical matrix-vector multiplier based on unitary matrix-diagonal matrix decomposition, comprising:

[0007] A Mach-Zehnder interferometer array is used to perform the operation of synthesizing the unitary matrix Ω;

[0008] A modulation unit, connected to the optical path of the Mach-Zehnder interferometer array, is used to perform operations on the corresponding diagonal matrix Σ;

[0009] The synthesized unitary matrix Ω and the diagonal matrix Σ are obtained by decomposing the target weight matrix M, and satisfy M=ΩΣ. The synthesized unitary matrix Ω is obtained by multiplying the left unitary matrix U and the right unitary matrix V by the transpose of the target weight matrix M through singular value decomposition.

[0010] Specifically, the Mach-Zehnder interferometer array is composed of several Mach-Zehnder interferometer units connected in a triangular or rectangular topology.

[0011] Meanwhile, a method for configuring the aforementioned optical matrix-vector multiplier is provided, comprising the following steps:

[0012] S1. Obtain the target weight matrix M;

[0013] S2. Decompose the target weight matrix M into the product of the composite unitary matrix Ω and the diagonal matrix Σ, i.e., M = ΩΣ;

[0014] S3. Determine the control parameters of the Mach-Zehnder interferometer array based on the synthesized unitary matrix Ω;

[0015] S4. Determine the control parameters of the modulation unit based on the diagonal matrix Σ.

[0016] The beneficial effects of this invention are:

[0017] (1) While maintaining the performance of the neural network, by introducing appropriate constraints during the training and inference process, the matrix of arbitrary size can be decomposed into the form of multiplication of unitary matrix and diagonal matrix, thus providing a simpler matrix operation path for optical computing.

[0018] (2) The number of basic unitary matrix interference units required by the proposed optical matrix vector multiplier structure is significantly reduced compared to existing schemes (by about 50%), which significantly reduces manufacturing costs and computational complexity while improving system integration. The relative advantages of this invention become more apparent as the matrix size increases. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the training and inference process of an optical neural network under hardware constraints.

[0020] Figure 2 The optical matrix-vector multiplier structure is based on traditional singular value decomposition, where a) is a triangular architecture and b) is a rectangular architecture.

[0021] Figure 3 The optical matrix-vector multiplier structure proposed in this invention is shown, where a is a triangular structure and b is a rectangular structure.

[0022] Figure 4 This is a schematic diagram of a two-layer fully connected neural network structure;

[0023] Figure 5 Fitting curves for recognition accuracy on the MNIST dataset. Detailed Implementation

[0024] To map the trained weight matrix to the optical hardware platform in a high-fidelity and highly stable manner to realize an optical matrix-vector multiplier, the following technical solution is adopted:

[0025] 1. Integration of neural network training and inference

[0026] The high performance of neural networks depends on an effective training process; their performance is directly determined by the training results. The proposed integrated training and inference process for neural networks is as follows: Figure 1 As shown, the specific steps are as follows: First, a network model adapted to the target hardware system is constructed to simulate the computation process of an optical neural network; second, singular value decomposition is used to decompose the trained weight matrix into a product of two unitary matrices and a diagonal matrix, and new weight coefficients are generated by reconstructing the decomposition structure using an algorithm; subsequently, combined with optimization methods such as backpropagation and stochastic gradient descent, the trainable parameters (mainly weight coefficients) in the network are iteratively optimized through computer simulation; finally, based on the photoelectric conversion effect, the trained parameters are mapped to the corresponding optical hardware platform.

[0027] The core of this process lies in the reconstruction of the weight matrix. Although matrix multiplication does not satisfy the commutative law, considering that neural networks are essentially function approximation models, their training process has certain black-box characteristics. Therefore, this invention imposes constraints on the weight matrix during the training phase, making it decomposeable into the product of a unitary matrix and a diagonal matrix. The specific steps are as follows: First, singular value decomposition is performed on the weight matrix to obtain a left unitary matrix. Right unitary matrix and a diagonal matrix ; for The transpose, and then, during training... and Direct multiplication yields a new matrix Since the product of unitary matrices is still a unitary matrix, This is called the composite unitary matrix; finally, and Direct multiplication yields the reconstructed weight matrix. Through the above process, an arbitrary-dimensional matrix M can be decomposed into the form of the product of a composite unitary matrix and a diagonal matrix: ;

[0028] in It is m m-dimensional composite unitary matrix It is m an n-dimensional diagonal matrix It is m An arbitrary n-dimensional matrix. During the hardware deployment phase, the synthesized unitary matrix obtained from training and decomposition will be... and diagonal matrix The mathematical parameters are converted into control signals for the corresponding optical functional units. Specifically, according to... Configure the phase modulation of each element in the Mach-Zehnder interferometer array, according to Configure the amplitude modulation coefficients of each unit in the independent modulator array. In this way, the computational task is embedded into the optical hardware structure, thereby achieving low-power, highly parallel optical domain matrix-vector multiplication acceleration.

[0029] 2. Topology of Optical Matrix-Vector Multiplier

[0030] The core of optical matrix computation lies in mapping computational parameters to an optical hardware platform, that is, transforming the coefficients of the corresponding unitary and diagonal matrices into an optical system implementation. Neural networks contain numerous matrix-vector multiplications and multiply-accumulate operations. If matrix-vector multiplication could be performed at the speed of light in the optical domain, it would greatly accelerate the intensive matrix operations and significantly reduce energy consumption and time overhead. Theoretically, any matrix can be decomposed into the product of a unitary matrix and a diagonal matrix. Furthermore, a unitary matrix of any dimension can be realized through a topological cascade of second-order unitary matrices. The Mach-Zehnder interferometer, as a basic optical interference structure (with two input and output ports), is essentially a natural unit for performing minimal unitary matrix operations, capable of directly performing 2×2 unitary matrix operations. Therefore, by topologically cascading multiple Mach-Zehnder interferometer arrays, an optical array for high-order unitary matrix operations can be constructed.

[0031] Existing methods for constructing an Nth-order Mach-Zehnder interferometer array require N² second-order interferometer units combined in a specific topology. The construction process is as follows: First, multiple basic units are arranged in parallel at the input end to form a multi-channel input array; then, every two adjacent optical channels are coupled through a Mach-Zehnder interferometer unit, and the desired optical field distribution and transmission matrix can be achieved by adjusting the phase of this unit; finally, the output array is constructed in a mirror-symmetric manner, thus completing the encapsulation of the entire rectangular topology. Figure 2 A schematic diagram of a fourth-order triangular and rectangular topological interference array structure constructed based on this method is shown. Figure 2The α-triangle architecture in the model employs a symmetrical structure of input array + intermediate interference layer + mirror output array. It requires mirror design to match the matrix decomposition logic of traditional SVD. The MZI arrays numbered 1-6 and 11-16 together constitute two symmetrical unitary matrix optical arrays: the left unitary matrix U optical array and the transpose V of the right unitary matrix. T Optical array. MZI units numbered 1-6 are concentrated on the optical path from the input to the intermediate layer, handling low-order matrix operations of the U matrix. MZI units numbered 11-16 are concentrated on the optical path from the intermediate layer to the output, handling V... T Low-order matrix operations. MZIs numbered 7, 8, 9, and 10 are distributed along the core coupling path of the intermediate layer. Through collaborative work with MZIs of symmetric numbers (e.g., 10 corresponds to 16, 9 corresponds to 15, etc.), they undertake two unitary matrices (U and V). T The high-order matrix operations ensure the complete mapping of the biunitary matrix, satisfying the requirements of traditional decomposition for symmetric optical paths. Figure 2 The rectangular architecture in the model also follows a symmetrical topology, with the optical channels at the input and output ends arranged in a completely mirror image, ensuring that the hardware mapping requirements for the biunitary matrix in traditional decomposition are met. Both are composed of 16 Mach-Zehnder interferometer units.

[0032] The optical matrix-vector multiplier topology proposed in this invention does not require a mirror-symmetric output array; it can be implemented using only a synthesized unitary matrix topology array and a diagonal interference array. For example... Figure 3 As shown, the 4th-order triangular and rectangular topological interference array constructed based on this method requires only 10 Mach-Zehnder interferometers (MZIs). MZIs numbered 1-6 are concentrated on the optical path from the input to the intermediate layer, undertaking low-order matrix operations for synthesizing the unitary matrix. MZIs numbered 7, 8, 9, and 10 are used to connect the leftmost MZIs (1-6) with the diagonal matrix, achieving single-unitary matrix cascading and undertaking high-order matrix operations for synthesizing the unitary matrix. Compared to... Figure 2 The existing structure significantly reduces the number of units.

[0033] As the matrix size increases, the optical matrix-vector multiplier topology proposed in this invention exhibits significant advantages over existing methods in terms of hardware cost, integration, and debugging complexity. Taking 256-order matrix multiplication as an example, existing methods require approximately 65,536 second-order interference units, while this invention requires only 32,640 units to achieve the same computation, reducing the number of units by approximately 50%, demonstrating significant advantages in terms of resource consumption and system complexity.

[0034] 3. Combining unitary matrices Implementation: Mach-Zehnder interferometer array

[0035] Arbitrary m-order unitary matrix This can be achieved by cascading multiple 2×2 unitary matrices (i.e., basic unitary transformation units).

[0036] A single Mach-Zehnder interferometer (MZI) comprises two 3dB directional couplers (or beam splitters) and two tunable phase modulators (typically based on thermo-optic or electro-optic effects). To achieve an m-order synthesized unitary matrix... This requires connecting multiple MZI basic units according to a specific topology. The present invention preferably uses a triangular or rectangular topology.

[0037] Connection method: m parallel input / output optical waveguides are arranged on the optical waveguide plane. The network is constructed by sequentially embedding MZI units between adjacent waveguides.

[0038] Parameter mapping process: For a given composite unitary matrix Using known decomposition algorithms, a unique set of phase parameters can be calculated { Let i = 1, 2, ..., m, where m is the total number of MZI required. Let i be the phase modulation angle of the i-th MZI. Let be the coupling phase angle of the i-th MZI, through { The control voltage or current signal drives the phase modulator in the corresponding MZI unit. By setting these control signals, the overall transfer matrix of the entire MZI array can approximate the synthesized matrix. The total number of MZI required by this invention is far less than that of traditional schemes that require the implementation of two unitary matrices.

[0039] 3. Diagonal matrix Implementation: Intensity modulator array

[0040] To achieve Diagonal matrix The amplitude scaling function represents the introduction of an independently controllable modulator array in the optical path. This array consists of m modulators (corresponding to...). The array is composed of independent optical intensity modulators (number of rows), such as electroabsorption modulators or Mach-Zehnder intensity modulators. In terms of physical optical path layout, this array is directly cascaded in a matrix used to achieve the synthesized unitary matrix. The output of the Mach-Zehnder interferometer array. Each modulator individually controls the output light intensity of one optical channel. The function of this modulator array is to perform independent, programmable amplitude weighting on each optical channel after unitary transformation, thereby accurately completing the diagonal matrix. Multiplication operations. Compared to unitary matrix implementations that require complex interferometric networks, diagonal matrix implementations only require simple, decoupled, channel-by-channel control.

[0041] Parameter mapping relationship: Diagonal matrix obtained after training and decomposition The i-th element on the diagonal is directly mapped to the driving signal (such as bias voltage or current) of the i-th intensity modulator. By precisely controlling this driving signal, the ratio of the output optical power to the input optical power (i.e., transmittance) of the optical signal after passing through the modulator is proportional to the corresponding element. The specific proportional relationship depends on the working curve of the intensity modulator.

[0042] 4. Overall System Workflow

[0043] The target weight matrix to be multiplied is M. The workflow of the entire optical matrix-vector multiplier is as follows:

[0044] Electro-optic conversion: The input electrical vector signal x is converted into a multi-channel coherent optical signal carrying the input information by a combination of a digital-to-analog converter and a laser / modulator.

[0045] Unitary transform: The multi-channel optical signal is input to the Mach-Zehnder interferometer array. The array is based on a preset { The parameters interfere and mix the light field to synthesize a unitary matrix. The linear transformation it represents;

[0046] Diagonal weighting: The optical signal after unitary transformation enters the intensity modulator array. The light intensity of each channel is independently scaled (attenuated) according to the corresponding singular value.

[0047] Output: The weighted multi-channel optical signal can be directly output for subsequent optical processing, or converted into an electrical signal by a photodetector array to obtain the matrix-vector multiplication result y= x= x.

[0048] 5. Demonstration of typical applications of feedforward neural networks

[0049] To verify The effectiveness of the decomposition method was demonstrated by constructing a fully connected neural network model using a feedforward neural network, and testing it on an image classification task, i.e., applying the decomposition method to the hardware implementation of the feedforward neural network. The training of the feedforward neural network aims to obtain the target weight matrix M for image classification. Subsequently, using the method proposed in this invention, the target weight matrix M is... Decomposed into Form, and further unitary matrix The parameters are mapped / converted into a set of tunable phase parameters for a Mach-Zehnder interferometer array. Finally, these phase parameters are used to configure the optical hardware to enable forward inference computation of the network.

[0050] Feedforward neural networks are trained on a dataset containing input image pixels x and output classification labels y to learn an optimal weight matrix M (and parameters such as biases), enabling the network to perform specific tasks (such as image classification). Feedforward neural networks are the most basic deep learning model. In this architecture, signals are transmitted unidirectionally from the input layer through intermediate hidden layers to the output layer; there are no feedback loops or inter-layer connections. During this process, neurons use trainable parameters (i.e., weights and biases) to process the input data layer by layer. Because each neuron in a given layer is directly connected to all neurons in the next layer, it is also called a fully connected neural network. The general approximation theorem states that a fully connected neural network with enough hidden layers can approximate continuous functions with arbitrary precision, providing a theoretical basis for handling complex tasks.

[0051] The experiment used the standard MNIST dataset, which is widely used in neural networks, for training and evaluation. This dataset contains 70,000 pure handwritten digit images (60,000 training images and 10,000 training images). Each image was center-aligned and size-normalized to a grayscale image of 28×28 pixels and associated with labels of 10 categories.

[0052] The constructed two-layer fully connected neural network is as follows: Figure 4 As shown, the input layer contains 784 neurons corresponding to the pixel unfolding of the MNIST data. The hidden layer and output layer contain 256 and 10 neurons respectively, corresponding to matrix connections of 256×784 and 10×256 dimensions. Here, the number of neurons in the output layer represents the 10 different output categories from 0 to 9. During the classification process, the input data is forward-propagated through the hidden layer to the output layer to obtain the category probability distribution. The final classification result is determined by the position of the neuron corresponding to the maximum probability, i.e., it is classified as the corresponding digit category.

[0053] During network optimization training, the cross-entropy error function is used as the cost function to quantify the error between the actual output and the expected label. Simultaneously, classic backpropagation and stochastic gradient descent algorithms are used to optimize the trainable parameters of the network model to achieve optimal recognition performance. After 100 iterations of training, the model achieved a classification accuracy of 94.45% on the MNIST test set, as shown in the fitted curve. Figure 5 As shown, the feasibility of the decomposition algorithm is confirmed. It is worth noting that... Figure 4 The network structure shown can still be further optimized to improve performance. Furthermore, the MNIST classification task is only a typical application demonstration of the proposed algorithm; it is also applicable to other deep learning models and tasks.

Claims

1. An optical matrix-vector multiplier based on unitary matrix-diagonal matrix decomposition, characterized in that, include: A Mach-Zehnder interferometer array is used to perform the operation of synthesizing the unitary matrix Ω; A modulation unit, connected to the optical path of the Mach-Zehnder interferometer array, is used to perform operations on the corresponding diagonal matrix Σ; The synthesized unitary matrix Ω and the diagonal matrix Σ are obtained by decomposing the target weight matrix M and satisfying M = ΩΣ. The synthesized unitary matrix Ω is obtained by multiplying the left unitary matrix U and the right unitary matrix V by the transpose of the target weight matrix M through singular value decomposition.

2. The optical matrix-vector multiplier as described in claim 1, characterized in that, The Mach-Zehnder interferometer array is composed of several Mach-Zehnder interferometer units connected in a triangular or rectangular topology.

3. The optical matrix-vector multiplier as described in claim 1 or 2, characterized in that, The modulation unit includes an intensity modulator array for independently amplitude-weighting each optical channel.

4. The optical matrix-vector multiplier as described in claim 1, characterized in that, Each Mach-Zehnder interferometer unit in the Mach-Zehnder interferometer array has an adjustable phase modulator, the control parameters of which are determined according to the synthesized unitary matrix Ω.

5. A method for configuring an optical matrix-vector multiplier as described in any one of claims 1, characterized in that, Includes the following steps: S1. Obtain the target weight matrix M; S2. Decompose the target weight matrix M into the product of the composite unitary matrix Ω and the diagonal matrix Σ, i.e., M = ΩΣ; S3. Determine the control parameters of the Mach-Zehnder interferometer array based on the synthesized unitary matrix Ω; S4. Determine the control parameters of the modulation unit based on the diagonal matrix Σ.

6. The method according to claim 5, characterized in that, Step S2 includes: Singular value decomposition is performed on the target weight matrix M to obtain a left unitary matrix U, a right unitary matrix V, and a diagonal matrix Σ; Multiplying the left unitary matrix U by the transpose of the right unitary matrix V yields the composite unitary matrix Ω, i.e., Ω = UV. T .

Images