Optical diffraction neural network, optical computing chip and matrix multiplication calculation method

Abstract

The application discloses an optical diffraction neural network, an optical computing chip and a matrix product calculation method. The optical diffraction neural network comprises: an addition calculation unit configured to perform addition modulation on light signals corresponding to a first vector and light signals corresponding to a second vector and output a first light signal; the first vector is generated by copying any element in a first matrix, and the second vector is a row vector corresponding to the column ordinal number of any element in the first matrix in a second matrix; a subtraction calculation unit has the same input light signal as the addition calculation unit, and the subtraction calculation unit is configured to perform subtraction modulation on the light signals corresponding to the first vector and the light signals corresponding to the second vector and output a second light signal; and a differential photoelectric detector is used for converting the first light signal and the second light signal into electrical signals and performing differential processing on the electrical signals. By dynamically changing the first matrix and the second matrix, the optical diffraction neural network can be used for calculating the product of any two matrices.

Description

Technical Field

[0001] This application relates to the field of optical neural networks, and in particular to an optical diffraction neural network, an optical computing chip, and a matrix multiplication calculation method. Background Technology

[0002] Optical computing has the advantages of high parallelism, low power consumption, and ultra-high speed. Large-scale matrix multiplication can give full play to the core advantages of optical computing.

[0003] In existing technologies, optical diffraction neural networks (ODNs) calculate matrix multiplication by encoding the input matrix into the input optical signal. The ODN is trained to perform matrix multiplication, and the product of the input matrix and the optical mapping matrix of the ODN is obtained by photoelectric conversion of the output optical signal. Calculating the multiplication of any two matrices in an ODN typically requires the use of a phase-tunable metasurface. Specifically, the modulation layer in the ODN is a phase-tunable metasurface. By adjusting the phase of the phase-tunable metasurface, the optical mapping matrix of the ODN is altered, thereby reconstructing the optical mapping matrix and ultimately calculating the matrix product of the input matrix and the optical mapping matrix.

[0004] However, the architecture of optical diffraction neural networks, including phase-tunable metasurfaces, is quite complex and difficult to fabricate. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides an optical diffraction neural network, an optical computing chip, and a matrix multiplication calculation method, aiming to provide an optical diffraction neural network with a simple architecture, low manufacturing difficulty, and usability for matrix multiplication calculation.

[0006] According to one aspect of the embodiments of this application, an optical diffraction neural network for matrix multiplication calculation is disclosed, which is used to calculate the matrix product of a first matrix and a second matrix, including:

[0007] The addition calculation unit is configured to perform addition modulation on the optical signal corresponding to the first vector and the optical signal corresponding to the second vector and output the first optical signal; the first vector is generated by copying any element in the first matrix, and the second vector is a row vector in the second matrix corresponding to the column number of any element in the first matrix.

[0008] The subtraction calculation unit has the same input optical signal as the addition calculation unit. The subtraction calculation unit is configured to perform subtraction modulation on the optical signal corresponding to the first vector and the optical signal corresponding to the second vector and output a second optical signal.

[0009] A differential photodetector is used to convert the first optical signal and the second optical signal into electrical signals and perform differential processing on them.

[0010] In some embodiments, the optical diffraction neural network further includes a first light source array, which includes a plurality of first sub-light sources, the number of which is equal to the number of elements in the first vector to achieve element replication.

[0011] In some embodiments, the optical parameters of the beam emitted by each of the first sub-light sources are encoded as corresponding elements in the first vector;

[0012] The optical diffraction neural network further includes a second light source array, which includes multiple second sub-light sources. The number of the second sub-light sources is configured to be equal to the number of elements in the second vector, and the optical parameters of the beam emitted from each second sub-light source are encoded as the corresponding elements in the second vector.

[0013] In some embodiments, the optical diffraction neural network further includes a light source and a modulator, the light source being used to emit a light beam and the modulator being used to encode the optical parameters of the light beam.

[0014] In some embodiments, the optical diffraction neural network further includes a light field distribution interface configured to split the light signal and propagate it to the addition calculation unit and the subtraction calculation unit respectively, so that the addition calculation unit and the subtraction calculation unit have the same input signal.

[0015] In some embodiments, the total number of beams emitted from the light field distribution interface is twice the total number of elements of the first vector and the second vector.

[0016] In some embodiments, the optical signal corresponding to the first vector and the optical signal corresponding to the second vector are separated from each other before being input to the addition calculation unit;

[0017] The optical signals corresponding to the first vector and the second vector are separated before being input to the subtraction calculation unit.

[0018] In some embodiments, the differential photodetector includes a first photoelectric conversion unit, a second photoelectric conversion unit, and a differential processing unit; the first photoelectric conversion unit is used to convert the first optical signal into a first electrical signal, the second photoelectric conversion unit is used to convert the second optical signal into a second electrical signal, and the differential processing unit is used to perform differential processing on the first electrical signal and the second electrical signal.

[0019] A second aspect of this application provides an optical computing chip including at least two optical diffraction neural networks as described in any one of the preceding claims, wherein each optical diffraction neural network is arranged in a planar array and / or stacked along the height direction.

[0020] A third aspect of this application provides a matrix product calculation method based on an optical diffraction neural network as described in any of the preceding claims. The method is used to calculate the matrix product of a first matrix and a second matrix, and includes:

[0021] The first vector is obtained by copying any element of the first matrix as the target element, and the corresponding row vector in the second matrix is ​​selected as the second vector based on the column number of the target element.

[0022] The optical signal corresponding to the first vector and the optical signal corresponding to the second vector are modulated by addition using the addition calculation unit to obtain a first optical signal. The optical signal corresponding to the first vector and the optical signal corresponding to the second vector are modulated by subtraction using the subtraction calculation unit to obtain a second optical signal. The first optical signal and the second optical signal are differentially processed using the differential photodetector to obtain a target output vector. The row number of the target output vector is equal to the row number of the target element.

[0023] Sort and merge all the target output vectors according to their row numbers to obtain the matrix product of the first matrix and the second matrix.

[0024] In some embodiments, sorting and merging all the target output vectors according to their row numbers to obtain the matrix product of the first matrix and the second matrix includes:

[0025] The target output vectors obtained by using the same second vector are arranged according to row order to obtain an intermediate matrix;

[0026] The output matrix is ​​obtained by adding all the intermediate matrices corresponding to all the second vectors. The output matrix is ​​the matrix product of the first matrix and the second matrix.

[0027] In some embodiments, the rank of the intermediate matrix is ​​1.

[0028] In some embodiments, sorting and merging all the target output vectors according to their row numbers to obtain the matrix product of the first matrix and the second matrix includes:

[0029] The intermediate output vector is obtained by adding the target output vectors obtained by the target elements located in the same row, and the row number of the intermediate output vector is equal to the row number of the target elements that participated in obtaining the intermediate output vector.

[0030] The intermediate output vectors are sorted based on their row order to obtain an output matrix, which is the matrix product of the first matrix and the second matrix.

[0031] In some embodiments, the optical diffraction neural network further includes a light source, which is used to input optical signals to the addition calculation unit and the subtraction calculation unit;

[0032] The first vector is obtained by copying any element of the first matrix as the target element, and the second vector is selected from the corresponding row vector in the second matrix based on the column index of the target element. This includes:

[0033] The first vector is obtained by taking each element in each row of the first matrix as the target element in turn.

[0034] The first vector and the corresponding second vector are jointly encoded into the time domain of the light source output signal.

[0035] The optical diffraction neural network provided in this application is used to calculate the product of a first matrix and a second matrix. The optical diffraction neural network includes: an addition unit configured to perform additive modulation on the optical signals corresponding to a first vector and a second vector, and output a first optical signal; the first vector is generated by copying any element in the first matrix, and the second vector is a row vector in the second matrix corresponding to the column ordinal number of any element in the first matrix; a subtraction unit has the same input optical signal as the addition unit, and is configured to perform subtraction modulation on the optical signals corresponding to the first vector and the second vector, and output a second optical signal; a differential photodetector is used to convert the first and second optical signals into electrical signals and perform differential processing on them. The optical diffraction neural network provided in this application can be used to calculate the product of a first matrix and a second matrix. By dynamically changing the first and second matrices, the product of any two matrices can be calculated. The optical diffraction neural network provided in this application has the advantages of simple architecture, low processing difficulty, and low production cost. Attached Figure Description

[0036] The above and other objectives, features and advantages of this application will become more apparent from a detailed description of exemplary embodiments thereof with reference to the accompanying drawings.

[0037] Figure 1 A schematic diagram of the architecture layout of an optical diffraction neural network in one embodiment of this application is shown.

[0038] Figure 2 The diagram shows the outer product of the column vectors of the first matrix and the row vectors of the second matrix.

[0039] Figure 3 An embodiment of this application based on element x is shown. j,i The copied row vector and row vector w i The Hadama pile.

[0040] Figure 4 A schematic diagram of the architecture layout of the addition calculation unit in one embodiment of this application is shown.

[0041] Figure 5 A schematic diagram of the architecture layout of the subtraction calculation unit in one embodiment of this application is shown.

[0042] Figure 6 The diagram illustrates the product of a first vector obtained by copying elements from the same row and the corresponding second vector in one embodiment of this application.

[0043] Figure 7 A schematic diagram of the architecture layout of an optical computing chip in one embodiment of this application is shown.

[0044] Figure 8 A schematic diagram of the architecture layout of an optical computing chip in one embodiment of this application is shown.

[0045] Figure 9 A schematic diagram of the architecture layout of an optical computing chip in one embodiment of this application is shown.

[0046] Figure 10 A flowchart of a method for matrix multiplication calculation based on an optical diffraction neural network in one embodiment of this application is shown.

[0047] Figure 11 A flowchart of step S30 in one embodiment of this application is shown.

[0048] Figure 12 A flowchart of step S30 in one embodiment of this application is shown.

[0049] Figure 13 A flowchart of step S10 in one embodiment of this application is shown.

[0050] Figure Labels

[0051] 100, Optical diffraction neural network; 10, Addition calculation unit; 110, First waveguide; 120, First modulation layer; 20, Subtraction calculation unit; 210, Second waveguide; 220, Second modulation layer; 30, Differential photodetector; 310, First photoelectric conversion unit; 320, Second photoelectric conversion unit; 330, Differential processing unit; 40, Light field distribution interface; 1000, Optical computing chip. Detailed Implementation

[0052] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided to make the description of this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The drawings are merely illustrative of this application and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0053] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more exemplary embodiments. Numerous specific details are provided in the following description to give a full understanding of exemplary embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced with one or more specific details omitted, or other modules, components, etc., can be employed. In other instances, well-known structures, methods, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0054] The optical diffraction neural network 100 provided in this application is used to calculate the first matrix. Second matrix The product of the first matrix X and the second matrix W is matrix Y, i.e. In this application, any row in any matrix is ​​called a row vector; therefore, a row vector can also be regarded as a row matrix. Any column in any matrix is ​​called a column vector; therefore, a column vector can also be regarded as a column matrix. Represents the row and column indices of a matrix, for example, x. i Let w represent the i-th (0≤i≤K) column of the first matrix X. i This represents the i-th (0≤i≤K) row in the second matrix W. It should be noted that neither the first matrix X nor the second matrix W represents a specific matrix; both can be dynamically adjusted. Therefore, the optical diffraction neural network 100 provided in this application is used to calculate the matrix product of any two matrices.

[0055] Please see Figure 1 , Figure 1 An optical diffraction neural network 100 for matrix multiplication calculation is shown. The optical diffraction neural network 100 includes an addition calculation unit 10, a subtraction calculation unit 20, and a differential photodetector 30.

[0056] The addition calculation unit 10 is configured to perform additive modulation on the optical signals corresponding to the first vector and the second vector, and output a first optical signal. The information carried by the first optical signal is the result of adding the corresponding first and second vectors. For example, when the optical signals input to the addition calculation unit 10 are the optical signals corresponding to the first vector A and the second vector B, the information carried by the first optical signal is vector A + B (element-by-element addition). The addition calculation unit 10 can perform additive modulation on the optical signals through interference superposition.

[0057] The first vector is generated by copying any element from the first matrix, and the second vector is a row vector in the second matrix corresponding to the column number of any element in the first matrix. That is, the row number of the second vector in the second matrix is ​​the same as the column number of the aforementioned element in the first matrix. For example, if the first vector is obtained by copying the element in the 1st column and 2nd row of the first matrix, since the copied element is located in the 1st column of the first matrix, the second vector will be in the 1st row of the second matrix. Similarly, if the first vector is obtained by copying the element in the 1st column and 3rd row of the first matrix, since the copied element is located in the 1st column of the first matrix, the second vector will be in the 1st row of the second matrix. And if the first vector is obtained by copying the element in the 2nd column and 3rd row of the first matrix, since the copied element is located in the 2nd column of the first matrix, the second vector will be in the 2nd row of the second matrix.

[0058] The subtraction calculation unit 20 and the addition calculation unit 10 are independent of each other, and they share the same input optical signal. The subtraction calculation unit 20 is configured to perform subtraction modulation on the optical signal corresponding to the first vector and the optical signal corresponding to the second vector, and output a second optical signal. The subtraction modulation of the optical signal by the subtraction calculation unit 20 can be achieved through interference superposition. The information carried by the second optical signal is the result of subtracting the corresponding first and second vectors. For example, when the optical signals input to the subtraction calculation unit 20 are the optical signals corresponding to the first vector A and the second vector B, the information carried by the second optical signal is vector AB (element-by-element subtraction).

[0059] The differential photodetector 30 is used to convert the first optical signal and the second optical signal into electrical signals and perform differential processing on them. Since the electrical signal response output by the differential photodetector 30 is proportional to the square difference of the incident coherent photoelectric field intensity, for example, when the optical signals input to the addition calculation unit 10 and the subtraction calculation unit 20 are both the optical signal corresponding to vector A and the optical signal corresponding to vector B, the electrical signal output by the differential photodetector 30 is proportional to... Proportional, output in the current domain Vectors can be obtained (The result of element-wise multiplication, i.e., the Hadamard product). Therefore, by inputting the first optical signal and the second optical signal into the differential photodetector 30, the Hadamard product of the two vectors input to the addition calculation unit 10 and the subtraction calculation unit 20 can be obtained.

[0060] Since the product of any two matrices can be decomposed into the sum of a series of matrices of rank 1, matrix Y can be decomposed into the sum of a series of matrices of rank at most 1. See also... Figure 2 , Figure 2 The column vectors x of the first matrix X are shown. i The row vector w corresponding to the second matrix W i The outer product calculation result, where the column vector x i The elements in are row vector w i The elements in are , , Represents the outer product of vectors. According to Figure 2 It can be seen that matrix Y i The first row can be represented as Matrix Y i The second line can be represented as ... matrix Y i The Mth row can be represented as Therefore, matrix Y i The maximal linearly independent set contains 1 vector, i.e., the matrix Y... i The rank of Y is 1. i Since the rank of matrix Y is 1, matrix Y can be obtained through the column vectors x of the first matrix X. i The row vector w corresponding to the second matrix W i This is achieved by summing the outer products, and the process can be expressed as formula (1):

[0061] (1).

[0062] like Figure 2 As shown, due to Matrix Y i Each row can be represented as x i The elements in the corresponding row and w i Scalar multiplication. See also Scalar multiplication. Figure 3 , column vector x i element x in j,i Copy (row j, column i) of X to expand its dimensions. Based on element x j,i Each element in the copied row vector is x. j,i Based on element x j,i The dimension of the copied row vector is the same as w.i The dimensions are consistent, based on x j,i The copied row vector and row vector w i The Hadamard product with element x j,i and row vector w i The scalar products are equal, both being matrices Y. i The j-th row. Therefore, element x can be... j,i and row vector w i scalar product transformed into x-based j,i The copied row vector and row vector w i The Hadamarda. See also Figure 2 Column vector xi and row vector w i The outer product of the first and second matrices is a two-dimensional matrix. If the first and second matrices are large matrices, then their matrix product is k two-dimensional matrices Y. i Element-by-element addition requires a large-scale detector array for fully parallel reading, which presents challenges due to the large device area occupied by the detectors and the problem of optical path intersections. This application, however, uses column vector xi and row vector w... i The outer product is transformed into x-based j,i The copied row vector and row vector w i The result of the Hadamard product is still a row vector. The differential photodetector 30 does not require a large-scale detector array, which can reduce the device area occupied by the differential photodetector 30, and there is no problem of optical path crossing.

[0063] Since the optical diffraction neural network 100 provided in this application can calculate the Hadamard product of the first vector and the second vector, please refer to... Figure 3 , will be based on x j,i The row vector obtained after copying and w i By inputting the optical signals corresponding to the first vector and the second vector, respectively, into the optical diffraction neural network 100, vector Y can be obtained. i,j Vector Y i,j For matrix Y i The j-th row. Column vector x... i By repeating the above process for each element in the matrix Y, the optical diffraction neural network 100 provided in this application can be used to sequentially obtain matrix Y in the time domain. i Each row then yields the complete matrix Y. i Using the optical diffraction neural network 100 provided in this application, matrices Y1, Y2, Y3, ..., Y4 are obtained sequentially. K Then, by summing the results, we can obtain matrix Y, and thus obtain the matrix product of the first matrix X and the second matrix W.

[0064] The optical diffraction neural network 100 provided in this application can be used to calculate the matrix product of a first matrix and a second matrix. By dynamically changing the first matrix and the second matrix, it can calculate the matrix product of any two matrices. Compared with existing optical diffraction neural networks that can only calculate the product of an input matrix with a specific fixed matrix, the optical diffraction neural network 100 provided in this application can realize arbitrary dynamic matrix multiplication, and has a wider range of applications. Compared with existing optical diffraction neural networks that form a modulation layer using a phase-tunable metasurface, the optical diffraction neural network 100 provided in this application has the advantages of simple architecture, low processing difficulty, and low production cost.

[0065] Please refer to it again. Figure 1 In some embodiments, the differential photodetector 30 includes a first photoelectric conversion unit 310, a second photoelectric conversion unit 320, and a differential processing unit 330; the first photoelectric conversion unit 310 is used to convert a first optical signal into a first electrical signal, the second photoelectric conversion unit 320 is used to convert a second optical signal into a second electrical signal, and the differential processing unit 330 is used to perform differential processing on the first electrical signal and the second electrical signal, and outputs the Hadamard product of the first vector and the second vector by outputting one-quarter of the square difference of the first electrical signal and the square difference of the second electrical signal in the current domain.

[0066] Please see Figure 1 In some embodiments, the optical diffraction neural network 100 further includes a light field distribution interface 40, which is disposed at the input terminals of the addition calculation unit 10 and the subtraction calculation unit 20. The light field distribution interface 40 is configured to split the optical signal, with a portion of the split beam propagating to the addition calculation unit 10 and another portion of the split optical signal propagating to the subtraction calculation unit 20, so that the addition calculation unit 10 and the subtraction calculation unit 20 have the same input signal.

[0067] The replication factor of the optical field distribution interface 40 when splitting beams can be configured according to requirements. For example, if the replication factor of the optical field distribution interface 40 is configured to 4, when there are a total of K incident beams, each of the K beams is replicated into 4 beams. At this time, the beam emitted by the optical field distribution interface 40 has a total of 4K beams. 2K beams of the 4K beams propagate to the addition calculation unit 10, and 2K beams of the 4K beams propagate to the subtraction calculation unit 20. The information carried by the 2K beams propagating to the addition calculation unit 10 is exactly the same as the information carried by the 2K beams propagating to the subtraction calculation unit 20.

[0068] Furthermore, in some embodiments, the total number of beams emitted from the optical field distribution interface 40 is twice the total number of elements in the first and second vectors, ensuring that each element in both vectors can be encoded by the corresponding beam. The first vector has a dimension of N, and since the dimensions of the first and second vectors are equal, the total dimension of the first and second vectors is 2N. The total number of beams generated by the optical field distribution interface 40 is 4N. 2N beams from the 4N beams emitted from the optical field distribution interface 40 propagate to the addition calculation unit 10, and each of these 2N beams loads an element in both the first and second vectors. Similarly, 2N beams from the 4N beams emitted from the optical field distribution interface 40 propagate to the subtraction calculation unit 20, and each of these 2N beams loads an element in both the first and second vectors.

[0069] In some embodiments, the optical signals corresponding to the first vector and the second vector are separated before being input to the addition calculation unit 10, so as to avoid interference between the optical signals corresponding to the two vectors before being input to the addition calculation unit 10.

[0070] The optical signals corresponding to the first vector and the second vector are separated before being input to the subtraction calculation unit 20, so as to avoid interference between the two optical signals before they are input to the subtraction calculation unit 20.

[0071] Please see Figure 4 In some embodiments, the addition calculation unit 10 includes a first waveguide 110 and a first modulation layer 120. The first waveguide 110 propagates the optical signal in a quasi-free diffraction manner. The first modulation layer 120 applies target modulation to the input optical signal. The first modulation layer 120 is embedded in the first waveguide 110 to divide the first waveguide 110 into multiple segments. At least two first modulation layers 120 are provided, and the two first modulation layers 120 are spaced apart. The first modulation layer 120 includes, but is not limited to, a metasurface. A metasurface is a two-dimensional artificial material / interface with a thickness much smaller than the working wavelength, composed of subwavelength-scale nanostructure units arranged according to a specific pattern, which can precisely and flexibly control the phase, amplitude, polarization, propagation mode, and other characteristics of electromagnetic waves.

[0072] Please see Figure 5 In some embodiments, the subtraction calculation unit 20 includes a second waveguide 210 and a second modulation layer 220. The second waveguide 210 propagates the optical signal via quasi-free diffraction. The second modulation layer 220 applies target modulation to the input optical signal and is embedded in the second waveguide 210 to divide the second waveguide 210 into multiple segments. At least two second modulation layers 220 are provided, and the two second modulation layers 220 are spaced apart. The second modulation layer 220 includes, but is not limited to, a metasurface.

[0073] In some embodiments, the optical diffraction neural network 100 further includes a first light source array (not shown), which comprises a plurality of first sub-light sources (not shown). The light signal emitted from each first sub-light source is used to load information of the corresponding element in the vector. The number of first sub-light sources is equal to the number of elements in the first vector to realize element x j,i The copying. In this application, the dimension of a vector refers to the number of elements it contains. For example, when element x... j,i When it is [2], we want to change element x j,i Copy the vector to [2, 2, 2] to obtain the first vector. Then, load the first vector using the light signals emitted from the three first sub-light sources to achieve the desired effect on element x. j,i The replication. In other words, this application achieves element x by configuring the number of first sub-light sources according to the first vector. j,i Copying.

[0074] Furthermore, in some embodiments, the optical parameters of the beams emitted from each first sub-light source are encoded as corresponding elements in a first vector, thereby enabling the loading of the first vector by the beams emitted from the first sub-light sources. That is, the optical parameters of the beams emitted from the first sub-light sources carry information about each element of the first vector, thereby generating an optical signal corresponding to the first vector. The optical parameters include, but are not limited to, amplitude and phase. For example, taking the loading of a first vector containing three elements by the amplitude of the beams emitted from the first light source array as an example, in this case, the first light source array contains three first sub-light sources. The amplitude of the beam emitted from the first first sub-light source is loaded with the first element of the first vector, the amplitude of the beam emitted from the second first sub-light source is loaded with the second element of the first vector, and the amplitude of the beam emitted from the third first sub-light source is loaded with the third element of the first vector.

[0075] Furthermore, in some embodiments, by configuring each first sub-light source to control the optical parameters of the beam emitted by each first sub-light source, the optical parameters of the beam emitted by each first sub-light source are encoded as corresponding elements in a first vector.

[0076] Furthermore, in some embodiments, the optical diffraction neural network 100 further includes a first modulator (not shown) for encoding the beams emitted from each of the first sub-source lights, thereby enabling the optical parameters of the beams emitted from each of the first sub-source lights to be encoded as corresponding elements in a first vector.

[0077] In some embodiments, the optical diffraction neural network 100 further includes a second light source array (not shown), which includes a plurality of second sub-light sources (not shown). The number of second sub-light sources is configured to be equal to the number of elements in the second vector. The optical parameters of the beams emitted from each second sub-light source are encoded as corresponding elements in the second vector, thereby enabling the beams emitted from the second sub-light sources to load the second vector. That is, the optical parameters of the beams emitted from the second sub-light sources carry information about each element of the second vector, thereby generating an optical signal corresponding to the second vector. The optical parameters include, but are not limited to, amplitude and phase.

[0078] Furthermore, in some embodiments, by configuring each second sub-light source to control the optical parameters of the beam emitted by each second sub-light source, the optical parameters of the beam emitted by each second sub-light source are encoded as corresponding elements in a second vector.

[0079] Furthermore, in some embodiments, the optical diffraction neural network 100 also includes a second modulator (not shown) for encoding the beams emitted from each of the second sub-sources, thereby enabling the optical parameters of the beams emitted from each of the second sub-sources to be encoded as corresponding elements in a second vector.

[0080] In some embodiments, the optical diffraction neural network 100 further includes a light source (not shown) and a modulator (not shown). The light source is used to emit a light beam, and the modulator is used to encode the optical parameters of the light beam emitted by the light source, thereby loading a first vector and a second vector into the light beam, and generating an optical signal corresponding to the first vector and an optical signal corresponding to the second vector.

[0081] The number of light sources is not limited; for example, there can be one, two, or three light sources. When the number of light sources is less than the total number of elements in the first and second vectors, the beams emitted from the light sources can be split to achieve beam replication. The total number of beams obtained by splitting is equal to the total number of elements in the first and second vectors, enabling element replication and allowing each element in the first and second vectors to be loaded into the beam.

[0082] Since the optical signals corresponding to the first vector and the second vector are input sequentially, only matrix Y can be calculated at a time. i A row, matrix Y i It has M rows, and the matrix Y i There are K, therefore it needs to go through It takes several calculations to obtain matrix Y. Therefore, please refer to [link / reference needed]. Figure 6 In some embodiments, the optical signals corresponding to the first vector and the second vector are sequentially input in the time domain to generate the first row of matrix Y1, the first row of matrix Y2, ..., matrix Y... KThe first row represents the integration mode pair of the differential photodetector 30, specifically the first row of matrix Y1, the first row of matrix Y2, and so on, for matrix Y. K The first row of matrix Y can be obtained by summing the first row. Since matrix Y has M rows, the matrix product of the first matrix X and the second matrix W—matrix Y—can be obtained in M ​​calculations, which can significantly shorten the calculation time.

[0083] Please see Figures 7 to 9 This application also provides an optical computing chip 1000, which includes at least two of the above-mentioned optical diffraction neural networks 100, each of which is independent of the others.

[0084] Please see Figure 7 In some embodiments, the optical diffraction neural networks 100 are arranged in a planar array, and each optical diffraction neural network 100 is configured to calculate the product of a first vector generated by different elements in the first matrix and a corresponding second vector. In other words, different optical diffraction neural networks 100 are configured to calculate the same or different matrices Y. i The rows are processed in parallel by multiple optical diffraction neural networks 100, thereby significantly reducing the computation time. For example, as mentioned above, one optical diffraction neural network can obtain the output matrix Y after M calculations. Therefore, when the optical computing chip 1000 contains M optical diffraction neural networks 100, it is possible to obtain the output matrix Y in a single calculation.

[0085] Please see Figure 8 In some embodiments, the optical diffraction neural networks 100 are stacked along the height direction, and each optical diffraction neural network 100 is configured to calculate the product of a first vector generated by different elements in the first matrix and a corresponding second vector. In other words, different optical diffraction neural networks 100 are configured to calculate the same or different matrices Y. i The rows are processed in parallel by multiple optical diffraction neural networks 100, thereby significantly reducing the computation time. For example, as mentioned above, one optical diffraction neural network can obtain the output matrix Y after M calculations. Therefore, when the optical computing chip 1000 contains M optical diffraction neural networks 100, it is possible to obtain the output matrix Y in a single calculation.

[0086] Please see Figure 9 In some embodiments, the optical computing chip 1000 has multiple layers stacked together, and each layer has an array of multiple optical diffraction neural networks 100. Each optical diffraction neural network 100 is configured to calculate the product of a first vector generated by different elements in a first matrix and a corresponding second vector; in other words, different optical diffraction neural networks 100 are configured to calculate the same or different matrices Y. iThe rows are processed in parallel by multiple optical diffraction neural networks 100, thereby significantly reducing the computation time. For example, as mentioned above, one optical diffraction neural network can obtain the output matrix Y after M calculations. Therefore, when the optical computing chip 1000 contains M optical diffraction neural networks 100, it is possible to obtain the output matrix Y in a single calculation.

[0087] This application also provides a matrix product calculation method based on the above-mentioned optical diffraction neural network 100. The optical diffraction neural network 100 includes an addition calculation unit 10, a subtraction calculation unit 20 and a differential photodetector 30. The specific functions and structures of the addition calculation unit 10, the subtraction calculation unit 20 and the differential photodetector 30 can be referred to above, and will not be repeated here.

[0088] Please see Figure 10 The matrix product calculation method provided in this application is used to calculate the matrix product of a first matrix X and a second matrix W, where the product of the first matrix X and the second matrix W is matrix Y, i.e. The matrix multiplication calculation method provided in this application includes steps S10 to S30.

[0089] Step S10: Copy any element of the first matrix as the target element to obtain the first vector, and select the corresponding row in the second matrix as the second vector based on the column number of the target element.

[0090] In the calculation of the product of the first matrix X and the second matrix W, after obtaining the first vector, to ensure the correctness of the calculation result, the corresponding row in the second matrix needs to be selected as the second vector based on the column number of the target element. For example, if the first vector is obtained by copying the element in the 1st column and 2nd row of the first matrix as the target element, and the column number of the target element is 1, then the 1st row of the second matrix is ​​selected as the second vector. Similarly, if the first vector is obtained by copying the element in the 1st column and 3rd row of the first matrix as the target element, and the column number of the target element is 1, then the 1st row of the second matrix is ​​selected as the second vector. Finally, if the first vector is obtained by copying the element in the 2nd column and 3rd row of the first matrix as the target element, and the column number of the target element is 2, then the 2nd row of the second matrix is ​​selected as the second vector.

[0091] Furthermore, although the first vector is obtained by copying the elements in the column vector, it can be transformed into a row vector by transposing it.

[0092] Step S20: Based on the addition calculation unit 10, the optical signal corresponding to the first vector and the optical signal corresponding to the second vector are added and modulated to obtain the first optical signal; based on the subtraction calculation unit 20, the optical signal corresponding to the first vector and the optical signal corresponding to the second vector are subtracted and modulated to obtain the second optical signal; based on the differential photodetector 30, the first optical signal and the second optical signal are differentially subtracted to obtain the target output vector, and the row number of the target output vector is equal to the row number of the target element.

[0093] In step S20, the target output vector is obtained, which is the Hadamard product of the first and second vectors. The row number of the target output vector is equal to the row number of the target element, and this row number serves as the index for subsequent permutations and combinations. For example, when the target element is x... 2,3 When, it means that the target element is located in the 2nd row and 3rd column of the first matrix, based on the target element x 2,3 The first vector is copied and generated, and the third row in the second matrix is ​​selected as the second vector. The optical signals corresponding to the first vector and the second vector are input to the addition calculation unit 10 and the subtraction calculation unit 20, respectively. The target output vector is output by the differential photodetector 30, and the row number of the target output vector is 2.

[0094] Step S30: Sort and merge all target output vectors according to their row numbers to obtain the matrix product of the first matrix and the second matrix.

[0095] Please see Figure 11 In some embodiments, step S30: sorting and merging all target output vectors according to the row number of the target output vector to obtain the matrix product of the first matrix and the second matrix includes:

[0096] Step S310: Arrange the target output vectors obtained by the same second vector according to the row number to obtain the intermediate matrix;

[0097] The intermediate matrix is ​​matrix Y. i The rank of the intermediate matrix is ​​1.

[0098] Step S320: Add all the intermediate matrices corresponding to all the second vectors to obtain the output matrix. The output matrix is ​​the matrix product of the first matrix and the second matrix.

[0099] Among them, all the intermediate matrices are matrix Y1, matrix Y2... matrix Y K For matrices Y1, Y2, ..., Y... K The matrix Y can be obtained by summing each element.

[0100] Please see Figure 12In some embodiments, step S30: sorting and merging all target output vectors according to the row number of the target output vector to obtain the matrix product of the first matrix and the second matrix includes:

[0101] Step S350: Add the target output vectors obtained by the target elements in the same row to obtain the intermediate output vector. The row number of the intermediate output vector is equal to the row number of the target elements that participated in obtaining the intermediate output vector.

[0102] Since the first vector is obtained by copying the target element, the target output vector obtained by copying the first vector based on a specific target element, and then using the optical signal corresponding to the first vector and the optical signal corresponding to the corresponding second vector, is the target output vector obtained based on that target element. For example, x in the first matrix X 2,3 The first vector is obtained by copying the element x as the target element. The target output vector is then obtained based on the optical signal corresponding to the first vector and the optical signal corresponding to the corresponding second vector. This is the target output vector based on the target element x. 2,3 The obtained target output vector.

[0103] The intermediate output vector is obtained by adding the target output vectors obtained from the target elements located in the same row. For example, the elements in the first row of the first matrix X are as follows: Please see Figure 6 Based on the target element x 1,1 The obtained target output vector Y 1,1 Based on target element x 1,2 The obtained target output vector Y 2,1 ... based on target element x 1,1 The obtained target output vector Y m,1 Target output vector Y i,j The subscript i in the matrix represents the matrix index, and the target output vector Y i,j The subscript j in the vector indicates the row number it belongs to. For example, the target output vector Y 2,1 This indicates that the vector is located in the first row of matrix Y2.

[0104] The target output vector Y 1,1 Target output vector Y 2,1 ..., target output vector Y m,1 The intermediate output vector Y1 is obtained by summing the results. Similarly, intermediate output vectors Y2, Y3, ..., Y can be obtained. m .

[0105] The row number of the intermediate output vector is equal to the row number of the target element involved in obtaining the intermediate output vector. The row number of the intermediate output vector is used to determine its row number in the output matrix Y. The output matrix is ​​the matrix product of the first matrix and the second matrix. Please refer to [link / reference] again. Figure 6 The intermediate output vector Y1 is derived from the target output vector Y. 1,1 Target output vector Y 2,1 ..., target output vector Y m,1 The summation yields the target output vector Y. 1,1 Target output vector Y 2,1 ..., target output vector Y m,1 Since the intermediate output vector Y1 is obtained from the elements in the first row of the first matrix X, its row number is equal to the row number of the target element used to obtain the intermediate output vector (1), meaning the row number of the intermediate output vector Y1 is 1. Similarly, the row numbers of intermediate output vectors Y2 (2), Y3 (3), ..., Y..., are obtained. m The row number is m.

[0106] Step S360: Merge and sort the intermediate output vectors based on the row order to obtain the output matrix, which is the matrix product of the first matrix and the second matrix.

[0107] In this method, the row number of the intermediate output vector is used to determine its row number in the output matrix Y. By sorting the intermediate output vectors according to their row numbers, the output matrix Y can be obtained. The output matrix Y is the matrix product of the first and second matrices. This method directly outputs each row of the output matrix Y, without needing to output matrices Y1, Y2, ..., Y... K .

[0108] Please see Figure 13 In some embodiments, the optical diffraction neural network 100 further includes a light source for inputting optical signals to the addition calculation unit 10 and the subtraction calculation unit 20.

[0109] Step S10: Copy any element of the first matrix as the target element to obtain the first vector, and select the corresponding row vector in the second matrix as the second vector based on the column index of the target element, including:

[0110] Step S110: Based on each element in each row of the first matrix, take them as target elements in turn to obtain the first vector.

[0111] Among them, selecting target elements by column can be achieved by taking each element in the same row of the first matrix as the target element in turn.

[0112] Step S120: Encode the first vector and the corresponding second vector together into the time domain of the light source output light signal.

[0113] Encoding the first vector and the corresponding second vector together into the time domain of the light source output light signal can load the first vector and the corresponding second vector into the light signal, thereby enabling the input of the light signal corresponding to the first vector and the light signal corresponding to the second vector to the addition calculation unit 10 and the subtraction calculation unit 20 in the time domain.

[0114] Please see Figure 6 Taking the first vector as an example, based on the elements in the first row of the first matrix as the target elements, at the initial moment, the information carried by the input optical signal is x from the first matrix. 1,1 The first vector obtained by copying the element as the target element ( Figure 6 The first vector (left side of the first row) and the second vector corresponding to the first vector—row vector w1; at the next moment, the information carried by the input optical signal is the information from x in the first matrix. 1,2 The first vector obtained by copying the element as the target element ( Figure 6 (The second row on the left) and the second vector corresponding to the first vector—row vector w2; at the last moment, the information carried by the input optical signal is the information from x in the first matrix. 1,m The first vector obtained by copying the element as the target element ( Figure 6 The last row (left side) and the second vector corresponding to the first vector—row vector w m During this process, the differential photodetector 30 is in integration mode. The differential photodetector 30 accumulates the results generated at each time step and outputs the first row of the output matrix Y. Traversing all rows in the first matrix, the differential photodetector 30 can sequentially output each row of the output matrix Y. Since the output matrix Y is... Therefore, by loading specific first and second vectors into the input optical signal in this way, the output matrix can be obtained through M calculations. When the first and second matrices are large matrices, the calculation time can be significantly shortened.

[0115] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the appended claims.

Claims

1. An optical diffraction neural network for matrix multiplication calculation, used to calculate the matrix product of a first matrix and a second matrix, characterized in that, include: The addition calculation unit is configured to perform addition modulation on the optical signal corresponding to the first vector and the optical signal corresponding to the second vector and output the first optical signal; The first vector is generated by copying any element in the first matrix, and the second vector is a row vector in the second matrix corresponding to the column ordinal number of any element in the first matrix; The subtraction calculation unit has the same input optical signal as the addition calculation unit. The subtraction calculation unit is configured to perform subtraction modulation on the optical signal corresponding to the first vector and the optical signal corresponding to the second vector and output a second optical signal. A differential photodetector is used to convert the first optical signal and the second optical signal into electrical signals and perform differential processing on them.

2. The optical diffraction neural network according to claim 1, characterized in that, The optical diffraction neural network further includes a first light source array, which includes multiple first sub-light sources. The number of first sub-light sources is equal to the number of elements in the first vector to achieve element replication.

3. The optical diffraction neural network according to claim 2, characterized in that, The optical parameters of the beam emitted from each of the first sub-light sources are encoded as the corresponding elements in the first vector; The optical diffraction neural network further includes a second light source array, which includes multiple second sub-light sources. The number of the second sub-light sources is configured to be equal to the number of elements in the second vector, and the optical parameters of the beam emitted from each second sub-light source are encoded as the corresponding elements in the second vector.

4. The optical diffraction neural network according to claim 1, characterized in that, The optical diffraction neural network also includes a light source and a modulator, wherein the light source is used to emit a light beam and the modulator is used to encode the optical parameters of the light beam.

5. The optical diffraction neural network according to claim 1, characterized in that, The optical diffraction neural network further includes a light field distribution interface, which is configured to split the light signal and propagate it to the addition calculation unit and the subtraction calculation unit respectively, so that the addition calculation unit and the subtraction calculation unit have the same input signal.

6. The optical diffraction neural network according to claim 5, characterized in that, The total number of beams emitted from the light field distribution interface is twice the total number of elements of the first vector and the second vector.

7. The optical diffraction neural network according to claim 1, characterized in that, The optical signals corresponding to the first vector and the second vector are separated before being input to the addition calculation unit; The optical signals corresponding to the first vector and the second vector are separated before being input to the subtraction calculation unit.

8. The optical diffraction neural network according to any one of claims 1-7, characterized in that, The differential photodetector includes a first photoelectric conversion unit, a second photoelectric conversion unit, and a differential processing unit; the first photoelectric conversion unit is used to convert the first optical signal into a first electrical signal, the second photoelectric conversion unit is used to convert the second optical signal into a second electrical signal, and the differential processing unit is used to perform differential processing on the first electrical signal and the second electrical signal.

9. An optical computing chip, characterized in that, It includes at least two optical diffraction neural networks as described in any one of claims 1-8, wherein each of the optical diffraction neural networks is arranged in a planar array and / or stacked along the height direction.

10. A matrix product calculation method based on an optical diffraction neural network as described in any one of claims 1-9, the method being used to calculate the matrix product of a first matrix and a second matrix, characterized in that, The method includes: The first vector is obtained by copying any element of the first matrix as the target element, and the corresponding row vector in the second matrix is ​​selected as the second vector based on the column number of the target element. The optical signal corresponding to the first vector and the optical signal corresponding to the second vector are modulated by addition using the addition calculation unit to obtain a first optical signal. The optical signal corresponding to the first vector and the optical signal corresponding to the second vector are modulated by subtraction using the subtraction calculation unit to obtain a second optical signal. The first optical signal and the second optical signal are differentially processed using the differential photodetector to obtain a target output vector. The row number of the target output vector is equal to the row number of the target element. Sort and merge all the target output vectors according to their row numbers to obtain the matrix product of the first matrix and the second matrix.

11. The method for calculating matrix multiplication according to claim 10, characterized in that, Sort and merge all the target output vectors according to their row numbers to obtain the matrix product of the first matrix and the second matrix, including: The target output vectors obtained by using the same second vector are arranged according to row order to obtain an intermediate matrix; The output matrix is ​​obtained by adding all the intermediate matrices corresponding to all the second vectors. The output matrix is ​​the matrix product of the first matrix and the second matrix.

12. The method for calculating matrix multiplication according to claim 11, characterized in that, The rank of the intermediate matrix is ​​1.

13. The method for calculating matrix multiplication according to claim 10, characterized in that, Sort and merge all the target output vectors according to their row numbers to obtain the matrix product of the first matrix and the second matrix, including: The intermediate output vector is obtained by adding the target output vectors obtained by the target elements located in the same row, and the row number of the intermediate output vector is equal to the row number of the target elements that participated in obtaining the intermediate output vector. The intermediate output vectors are sorted based on their row order to obtain an output matrix, which is the matrix product of the first matrix and the second matrix.

14. The method for calculating matrix multiplication according to claim 10, characterized in that, The optical diffraction neural network also includes a light source, which is used to input optical signals to the addition calculation unit and the subtraction calculation unit; The first vector is obtained by copying any element of the first matrix as the target element, and the second vector is selected from the corresponding row vector in the second matrix based on the column index of the target element. This includes: The first vector is obtained by taking each element in each row of the first matrix as the target element in turn. The first vector and the corresponding second vector are jointly encoded into the time domain of the light source output signal.

Images