A Flexible High-Dimensional Complex Matrix Operation Unit Based on Light Scattering

Abstract

The present invention discloses a flexible high-dimensional complex matrix operation structure based on light scattering, which includes an optical splitter, an electro-optical switch, 4 CWDM wavelength division multiplexing units, 4 OSU matrix operation units and a photodetector; an electrical signal is used to control the incident light to enter different CWDM wavelength division multiplexing units at different times, and the CWDM wavelength division multiplexing unit separates the incident light according to wavelengths and outputs it from different output waveguides; then, these output lights will be sent to the corresponding wavelength waveguides of the OSU matrix operation unit, and after being scattered by the unit, they are output from the corresponding waveguides according to a certain ratio; the photodetection unit performs simulation of matrix operation by detecting the normalized power of the output light of each waveguide within a certain period of time. The electro-optical hybrid matrix multiplication of the present invention has the advantages of high bandwidth, low loss, small required space, etc., and at the same time has good scalability and can realize flexible matrix operation simulation.

Description

Technical Field

[0001] The present invention relates to a flexible high-dimensional complex matrix operation unit based on light scattering, so as to realize electro-optical hybrid on-chip matrix operation, and designs multiple links to control and flexibly expand the matrix operation. Background Art

[0002] In recent years, with the development of machine learning technology, deep neural networks have shown revolutionary performance improvements in various emerging applications. In particular, deep convolutional neural networks (CNNs) have had a profound impact in fields such as computer vision, image processing, speech processing, medical diagnosis, gaming, and signal processing, and have become the cornerstone of modern artificial intelligence. Although deep neural networks have advanced performance, their complex structures and large numbers of parameters consume a large amount of computing resources during training and inference. Therefore, there is an urgent need for high-speed, low-power neural network accelerators.

[0003] Optical methods are potential methods for next-generation neural network accelerators because optical components and technologies have the characteristics of ultra-wide bandwidth and low power consumption. The feasibility of optical neural networks (ONNs) has been demonstrated using optical technologies such as spatial light diffraction, on-chip coherent interference, and wavelength division multiplexing. Its high-speed and low-power performance can be well inferred from numerical and experimental results. In these pioneer works on optical neural networks, fully connected neural networks were mainly considered, so these architectures were designed as vector matrix multipliers. When it comes to convolutional neural networks (CNNs), these architectures may face huge challenges because converting the convolutional layer into a vector-matrix multiplication requires a huge optical circuit. If the size of the input image is N×N, the number of embedded parameters of this optical circuit is N 4 。A feasible way to overcome this obstacle is to convert the convolutional layer into a matrix-matrix multiplication by reusing optical hardware. Therefore, the number of embedded parameters is significantly reduced, about dozens, and the complete calculation is completed within N 2 time periods. Summary of the Invention

[0004] According to the concept of hardware reuse, the present invention proposes a flexible high-dimensional complex matrix operation unit based on light scattering, and this structure can perform complex matrix operations with certain characteristics in one unit. This module has a certain degree of flexibility, and different matrix operations and weighting coefficients can be controlled from the composition of the incident light and the time ratio of light incidence; at the same time, only by further designing the matrix operation unit therein, the simulation of matrix operations with different characteristics can be realized. Repeatedly using this unit can realize a series of consecutive matrix multiplications and the simulation of more complex matrix operations.

[0005] A flexible high-dimensional complex matrix operation unit based on light scattering according to the present invention is used to implement a 4x4 matrix operation, including an optical splitter, an electro-optic switch, four CWDM wavelength division multiplexing units, and four OSU matrix operation units connected in sequence; the optical splitter is used to divide the mixed light into four paths, and the ratio of the light intensities of each wavelength in the mixed light is α = [α1, α2, α3, α4], and the wavelengths included in the mixed light are 1270nm, 1290nm, 1310nm, and 1330nm;

[0006] The four CWDM wavelength division multiplexing units are sorted in the stacking order, and are the first CWDM wavelength division multiplexing unit, the second CWDM wavelength division multiplexing unit, the third CWDM wavelength division multiplexing unit, and the fourth CWDM wavelength division multiplexing unit from bottom to top in sequence; the four paths of mixed light evenly divided by the optical splitter are respectively incident on different CWDM wavelength division multiplexing units, and the CWDM wavelength division multiplexing unit is used to separate the mixed light with different wavelengths;

[0007] The electro-optic switch is located between the optical splitter and the four CWDM wavelength division multiplexing units, and is used to control the individual on-off of the incident light of each CWDM wavelength division multiplexing unit, so that only one CWDM wavelength division multiplexing unit receives the incident light at the same time. The electro-optic switch converts the electrical signal represented by the matrix into the relative duration β = [β1, β2, β3, β4] of the incident light received by each CWDM wavelength division multiplexing unit; the simulation of the following input matrix is realized:

[0008] X = [x1, x2, x3, x4] = [β1α, β2α, β3α, β4α];

[0009] The four OSU matrix operation units are stacked horizontally, and the four wavelengths emitted by each CWDM are respectively incident on the four OSU matrix operation units.

[0010] Further, the mixed light is composed of light in the 1320nm - 1270nm band.

[0011] The emitted wavelength bands of each CWDM wavelength division multiplexing unit correspond to the emitted wavelengths of the 4 waveguides in the sequence of the CWDM unit in turn as follows:

[0012] The wavelengths emitted by the first CWDM wavelength division multiplexing unit are 1270nm, 1290nm, 1310nm, and 1330nm;

[0013] The wavelengths emitted by the second CWDM wavelength division multiplexing unit are 1290nm, 1310nm, 1330nm, and 1270nm;

[0014] The wavelengths emitted by the third CWDM wavelength division multiplexing unit are 1310nm, 1330nm, 1270nm, and 1290nm;

[0015] The wavelengths emitted by the fourth CWDM wavelength division multiplexing unit are 1330nm, 1270nm, 1290nm, and 1310nm.

[0016] Furthermore, the CWDM wavelength division multiplexing unit is obtained by reverse optimization design using the L-BFGS algorithm.

[0017] Beneficial effects brought by the above technical solutions:

[0018] (1) Using the reverse optimization algorithm for the structural optimization design, the designed chip structure is more compact and has better performance.

[0019] (2) The present invention adopts two feasible electro-optical signal conversion methods, namely, the modulation of the composition ratio of the optical wavelengths and the control of the length of the optical input time for each channel. The optical signal conversion method for the input electrical signal is easy to implement.

[0020] (3) The front-end structure of the optical matrix calculation unit used in the present invention performs linear optical splitting operation on the optical signal, with low power consumption and an operation speed much higher than that of pure circuit operation.

[0021] (4) The present invention designs two controllable input matrices, which improves the freedom of matrix simulation and provides a feasible expansion for implementing designs similar to high-dimensional matrix operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is the overall structural schematic diagram of the implementation of the present invention;

[0023] Figure 2 is the spatial structural schematic diagram of the wavelength division multiplexing unit and the matrix operation unit;

[0024] Figure 3 is the connection mode diagram of the overall structure;

[0025] Figure 4 is the topological structure schematic diagram and transmission effect diagram of the four-way optical splitter;

[0026] Figure 5 is the topological structure schematic diagram of the CWDM wavelength division multiplexing unit;

[0027] Figure 6 is the topological structure schematic diagram of the OSU matrix operation unit. DETAILED DESCRIPTION OF THE INVENTION

[0028] The technical solutions of the present invention will be described in detail below with reference to the accompanying drawings.

[0029] As Figure 1As shown in the figure, a flexible high-dimensional complex matrix operation unit based on light scattering includes an optical splitter, an electro-optic switch, four CWDM wavelength division multiplexing units, and four OSU matrix operation units.

[0030] The optical splitter, the electro-optic switch, the four CWDM wavelength division multiplexing units, and the four OSU matrix operation units are connected in sequence;

[0031] The optical splitter is used to evenly divide the incident mixed light into four paths. The proportion of the light intensities of each wavelength in the mixed light is α = [α1, α2, α3, α4]. The mixed light uses the light wavelengths in the mixed wavelength band near 1320nm to 1270nm, specifically including these several determined wavelengths: 1270nm, 1290nm, 1310nm, and 1330nm.

[0032] The four CWDM wavelength division multiplexing units are arranged longitudinally and sorted in the stacking order. From bottom to top, they are the first CWDM wavelength division multiplexing unit, the second CWDM wavelength division multiplexing unit, the third CWDM wavelength division multiplexing unit, and the fourth CWDM wavelength division multiplexing unit. The four paths of light after the mixed light is evenly divided are respectively incident on different CWDM wavelength division multiplexing units;

[0033] The electro-optic switch is used to control the individual on-off of the incident light of each CWDM wavelength division multiplexing unit, so that only one CWDM wavelength division multiplexing unit can receive the incident light at the same time. The electro-optic switch converts the electrical signal into the relative duration β = [β1, β2, β3, β4] of the incident light received by each CWDM wavelength division multiplexing unit;

[0034] For example, β1 corresponds to the incident time of the first CWDM wavelength division multiplexing unit, β2 corresponds to the incident time of the second CWDM wavelength division multiplexing unit, β3 corresponds to the incident time of the third CWDM wavelength division multiplexing unit, and β4 corresponds to the incident time of the fourth CWDM wavelength division multiplexing unit;

[0035] The input matrix X can be expressed by the above variables as:

[0036] X = [x1, x2, x3, x4] = [β1α, β2α, β3α, β4α]

[0037] Each CWDM wavelength division multiplexing unit is used to separate the received mixed light into different wavelengths. Each CWDM wavelength division multiplexing unit has four output ports. After the mixed light passes through the CWDM wavelength division multiplexing unit, the light of different wavelengths exits from the four output ports according to the light intensity ratio of α = [α1, α2, α3, α4];

[0038] The output wavelength bands of each CWDM wavelength division multiplexing unit corresponding to the 4 waveguides in the sequence of the CWDM unit are as follows:

[0039] The wavelengths emitted by the first CWDM wavelength division multiplexing unit are 1270 nm, 1290 nm, 1310 nm, and 1330 nm;

[0040] The wavelengths emitted by the second CWDM wavelength division multiplexing unit are 1290 nm, 1310 nm, 1330 nm, and 1270 nm;

[0041] The wavelengths emitted by the third CWDM wavelength division multiplexing unit are 1310 nm, 1330 nm, 1270 nm, and 1290 nm;

[0042] The wavelengths emitted by the fourth CWDM wavelength division multiplexing unit are 1330 nm, 1270 nm, 1290 nm, and 1310 nm;

[0043] The span of each wavelength band is 10 nm. For example, the incident wavelength range of the 1270 nm band is 1265 - 1275 nm.

[0044] The incident wavelength band of the OSU matrix operation unit corresponds to the output wavelength of the CWDM, and the waveguides corresponding to the wavelength relationship are directly connected. Among them, the OSU matrix operation units are stacked horizontally. As Figure 3 shown, the four wavelengths emitted by each CWDM are respectively incident on four OSU matrix operation units.

[0045] Specifically, the digital signal is converted into an optical signal by a pre - placed electro - optic modulation unit. The intensity correspondence of the optical wavelength input to this electro - optic modulation unit is α = [α1, α2, α3, α4], and the ratio corresponds to the wavelengths of 1270 nm, 1290 nm, 1310 nm, and 1330 nm in sequence.

[0046] The optical signal after electro - optic conversion is incident on the flexible high - order matrix operation unit of the present invention for operation. First, it passes through a four - way optical splitter, which evenly divides the incident mixed light into four beams and then emits them. The four - way optical splitter is designed using an optimization method, and the topological optimization structure is as Figure 4 shown in (a) therein, Figure 4 and (b) therein is the transmission effect diagram of the four - way optical splitter. After being scattered by the four - way optical splitter, the four beams of mixed light are incident on the electro - optic switch. The electro - optic switch converts the electrical signal into the relative duration β = [β1, β2, β3, β4] of each path of light passing through the CWDM wavelength division multiplexing unit. Then the light intensity input to the i - th CWDM wavelength division multiplexing unit is expressed as x i = β i α, where i = 1, 2, 3, or 4.

[0047] The CWDM wavelength division multiplexer separates the light of different wavelengths according to the light intensity ratio α = [α1, α2, α3, α4] of the respective wavelength components in the mixed light and emits it from the corresponding waveguides, and then it is incident on the OSU matrix operation module to complete the matrix operation, such as Figure 5 is the topological structure of the 4 CWDM wavelength division multiplexers included in this structure, and their corresponding relationship is: the wavelengths of the 4 waveguides from bottom to top are 1270nm, 1290nm, 1310nm, 1330nm; 1290nm, 1310nm, 1330nm, 1270nm; 1310nm, 1330nm, 1270nm, 1290nm; and 1330nm, 1270nm, 1290nm, 1310nm; the CWDM wavelength division multiplexing units in four cases.

[0048] After being processed by the wavelength division multiplexing unit, the emitted light is incident on the relevant waveguides of the OSU matrix operation unit for matrix operation simulation. Figure 6 is the topological structure of the 4 OSU matrix operation units designed by the present invention, and the corresponding matrix I1, I2, I3, I4 operation results are realized as follows:

[0049]

[0050] The actual emission of the CWDM wavelength division multiplexing unit that separates the incident light into four output waveguides according to the composition ratio of the light wavelengths is x i = [β i α1, β i α2, β i α3, β i α4], and the total output of the 4 CWDM wavelength division multiplexing units is expressed as:

[0051]

[0052] Finally, the OSU matrix operation unit emits the output light into the photoelectric conversion unit, and the photoelectric detection unit detects the power ratio of the output light of each waveguide within one period and converts it into the electrical signal corresponding to the corresponding matrix operation result to complete the simulation of the matrix operation.

[0053] After matrix operation, the finally obtained matrix is:

[0054]

[0055] It can be abbreviated as [β1, β2, β3, β4] · [α1, α2, α3, α4] T · [I1, I2, I3, I4] T = X · [I1, I2, I3, I4] T

[0056] Using this matrix unit, the operation simulation of some matrices can be realized. By controlling the wavelength composition of the light incident on the matrix operation unit, the composition of matrix α can be controlled, and by controlling the relative time length of the input corresponding to the CWDM matrix, the composition of matrix β can be controlled; matrix operations and matrix simulations with other characteristics can be constructed by replacing the implementation function of the base matrix corresponding to I.

[0057] In another embodiment, a matrix operation unit based on light scattering of the present invention is used to perform linear operation simulation on the input optical signal. Suppose the 4D optical signal input to the matrix operation unit of light scattering of the present invention is The output 4D optical signal is Where:

[0058]

[0059]

[0060]

[0061]

[0062] The coefficients a output by 4 OSU matrix operation units mn have different values, respectively simulating the multiplication operations for I1, I2, I3, and I4, and matrix simulation is realized by superimposing the outputs of 4 OSU matrix operation units.

[0063] The photoelectric detection unit performs linear superposition operation on the input optical signal to distribute the optical power, specifically:

[0064] Suppose the four-dimensional optical signal input to the photoelectric detection unit is The output 4D electrical signal is Where:

[0065]

[0066] After being controlled by the electro-optical switch, each OSU matrix operation module has only a single wavelength output at the same time. By detecting the total power of each channel after each CWDM wavelength division multiplexing unit is incident once, the above relationship can be obtained.

[0067] If the electro-optical switch is not applicable, the light input in each matrix operation actually has different wavelengths, and the corresponding powers can also be linearly directly superimposed.

[0068] The present invention utilizes the L-BFGS algorithm in combination with Lumerical FDTD Solutions for the optimized design of devices, obtaining an optical computing device with a non-intuitive topological structure. Through continuous iterative optimization, the corresponding functions of each part of the device are realized, and the operation simulation of the matrix is completed.

Claims

1. A flexible high-dimensional complex matrix operation unit based on light scattering, characterized in that, It includes an optical splitter, an electro-optical switch, four CWDM wavelength division multiplexing units, and four OSU matrix operation units; the optical splitter, the electro-optical switch, the four CWDM wavelength division multiplexing units, and the four OSU matrix operation units are connected in sequence; the optical splitter is used to divide the mixed light into four paths, and the ratio of the light intensities of each wavelength in the mixed light is α = [α1, α2, α3, α4]. The four CWDM wavelength division multiplexing units are sorted in the stacking order, and are successively the first CWDM wavelength division multiplexing unit, the second CWDM wavelength division multiplexing unit, the third CWDM wavelength division multiplexing unit, and the fourth CWDM wavelength division multiplexing unit from bottom to top; the four paths of mixed light evenly divided by the optical splitter are respectively incident on different CWDM wavelength division multiplexing units, and the CWDM wavelength division multiplexing unit is used to separate the mixed light with different wavelengths. The electro-optical switch is located between the optical splitter and the four CWDM wavelength division multiplexing units, and is used to control the individual on-off of the incident light of each CWDM wavelength division multiplexing unit, so that only one CWDM wavelength division multiplexing unit receives the incident light at the same time. The electro-optical switch converts the electrical signal represented by the matrix into the relative duration β = [β1, β2, β3, β4] of the incident light received by each CWDM wavelength division multiplexing unit; the simulation of the following input matrix is realized. X = [x1, x2, x3, x4] = [β1α, β2α, β3α, β4α].[[]END]] The four OSU matrix operation units are stacked horizontally, and the four wavelengths emitted by each CWDM are respectively incident on the four OSU matrix operation units.

2. The flexible high-dimensional complex matrix operation unit based on light scattering according to claim 1, characterized in that The mixed light is composed of light in the 1320nm - 1270nm band.

3. The flexible high-dimensional complex matrix operation unit based on light scattering according to claim 2, characterized in that The wavelengths emitted by each CWDM wavelength division multiplexing unit corresponding to the 4 waveguides in the sequence of the CWDM unit are as follows:[[]END]] The wavelengths emitted by the first CWDM wavelength division multiplexing unit are 1270nm, 1290nm, 1310nm, and 1330nm. The wavelengths emitted by the second CWDM wavelength division multiplexing unit are 1290nm, 1310nm, 1330nm, and 1270nm. The wavelengths emitted by the third CWDM wavelength division multiplexing unit are 1310nm, 1330nm, 1270nm, and 1290nm. The wavelengths emitted by the fourth CWDM wavelength division multiplexing unit are 1330nm, 1270nm, 1290nm, and 1310nm.

4. The flexible high-dimensional complex matrix operation unit based on light scattering according to claim 2, wherein The CWDM wavelength division multiplexing unit is obtained by reverse optimization design using the L-BFGS algorithm.

Images