Block type optical neural network, matrix calculation method, chip and electronic equipment

Abstract

The invention discloses a block type optical neural network, a matrix calculation method, a chip and electronic equipment, and the block type optical neural network comprises at least two input modules which are arranged side by side at intervals; the output modules are arranged side by side at intervals; the at least two sub-processing modules are arranged in an array, the sub-processing modules located in the same row are in coupled connection with the same input module, and the sub-processing modules located in the same column are in coupled connection with the same output module; wherein the sub-processing modules coupled and connected with the same input module have the same input optical signal, and each output module is configured to superpose the optical field output by each sub-processing module coupled and connected with the output module. Compared with an optical neural network in the prior art, when the partitioned optical neural network provided by the invention processes a calculation task, each sub-processing module performs parallel processing, so that the error accumulation is remarkably reduced, and the calculation precision can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of optical systems, in particular to a block type optical neural network, a matrix calculation method, a chip and an electronic device. BACKGROUND

[0002] In the prior art, when a single optical neural network is used to process large-scale matrix calculation, the optical neural network needs to be configured with a large number of modulation layers. Since there may be processing errors and coupling errors in each modulation layer, once an error occurs in a modulation layer, the error will be transmitted to the lower layer, and the accumulated error will gradually be eliminated by the method, resulting in a serious distortion of the final output result. SUMMARY

[0003] To solve the problem of low calculation accuracy of the optical neural network in the prior art, the embodiments of the present application provide a block type optical neural network, a matrix calculation method, a chip and an electronic device.

[0004] According to an aspect of the embodiments of the present application, a block type optical neural network is disclosed, comprising:

[0005] At least two input modules, each of the input modules is arranged side by side and spaced apart;

[0006] At least two output modules, each of the output modules is arranged side by side and spaced apart;

[0007] At least two sub-processing modules, the at least two sub-processing modules are arranged in an array, the sub-processing modules located in the same row are coupled to the same input module, and the sub-processing modules located in the same column are coupled to the same output module;

[0008] Among them, each of the sub-processing modules coupled to the same input module has the same input light signal, and each of the output modules is configured to superimpose the light field output by each of the sub-processing modules coupled thereto.

[0009] In some embodiments, the combined matrix composed of the optical mapping matrices of all the sub-processing modules is the same as the optical mapping weight matrix of the block type optical neural network.

[0010] In some embodiments, the optical mapping matrices of different sub-processing modules are located at different positions of the optical mapping weight matrix of the block type optical neural network and do not overlap.

[0011] In some embodiments, the number of rows of the optical mapping matrix of each of the sub-processing modules is greater than or equal to 2, and the number of columns is greater than or equal to 2.

[0012] In some embodiments, the optical mapping matrix of each of the sub-processing modules is either a row matrix or a column matrix.

[0013] In some embodiments, each of the sub-processing modules includes at least one input port and at least one output port; each input module includes an input waveguide and an input coupler, wherein there is at least one input waveguide, and the input waveguides are arranged in an array; the input coupler, the input waveguide, and the input port are arranged in a one-to-one correspondence, one end of the input coupler is coupled to the corresponding input waveguide, and the other end of the input coupler is coupled to the corresponding input port; each of the output modules includes an output waveguide and an output coupler, wherein there is at least one output waveguide, and the output waveguides are arranged in an array; the output coupler, the output waveguide, and the output port are arranged in a one-to-one correspondence, one end of the output coupler is coupled to the corresponding output waveguide, and the other end of the output coupler is coupled to the corresponding output port.

[0014] In some embodiments, the center distance between any two adjacent input couplers is greater than 2.5 micrometers; and / or the center distance between any two adjacent output couplers is greater than 2.5 micrometers; and / or the center distance between any two adjacent input waveguides is greater than 2.5 micrometers; and / or the center distance between any two adjacent output waveguides is greater than 2.5 micrometers.

[0015] In some embodiments, each of the sub-processing modules includes a waveguide and at least one diffractive optical element; the diffractive optical element is embedded in the waveguide and is used to modulate the input optical signal; the waveguide is used to propagate the input optical signal in a quasi-free diffraction manner.

[0016] In some embodiments, the diffractive optical element is a phase-fixed metasurface or a phase-tunable metasurface.

[0017] A second aspect of this application provides a matrix calculation method based on a block-based optical neural network as described in any of the preceding claims, comprising:

[0018] The input matrix is ​​divided into blocks based on the number of rows and columns of the sub-processing module to generate at least two sub-input matrices; the optical signals corresponding to the sub-input matrices located in the same row or column are sequentially input to different input modules;

[0019] The output light field is obtained by superimposing the light fields output by each of the sub-processing modules coupled to the same output module;

[0020] Photoelectric conversion is performed based on each of the output light fields to obtain the calculation results of the weight matrix and the input matrix of the optical mapping of the segmented optical neural network.

[0021] In some embodiments, the matrix calculation method further includes:

[0022] Configure the modulation parameters of the segmented optical neural network and the segmentation method of the weight matrix and the input matrix so that the segmented optical neural network can switch between matrix addition, matrix subtraction, matrix multiplication, matrix division and matrix convolution operations.

[0023] A third aspect of this application provides a modular optical neural network chip, comprising: at least one modular optical neural network as described in any of the preceding claims, wherein each of the modular optical neural networks is stacked.

[0024] In some embodiments, the input modules of two adjacent segmented optical neural networks are misaligned; and / or, the output modules of two adjacent segmented optical neural networks are misaligned.

[0025] A third aspect of this application provides an electronic device including a modular optical neural network chip as described in any of the preceding claims.

[0026] The modular optical neural network provided in this application includes: at least two input modules, each arranged side-by-side with spacing; at least two output modules, each arranged side-by-side with spacing; and at least two sub-processing modules, arranged in an array. Sub-processing modules located in the same row are coupled to the same input module, and sub-processing modules located in the same column are coupled to the same output module. Each sub-processing module coupled to the same input module has the same input optical signal, and each output module is configured to superimpose the optical fields output by each of its coupled sub-processing modules. Compared to existing optical neural networks, the modular optical neural network provided in this application allows for parallel processing of sub-processing modules during computational tasks, significantly reducing error accumulation and improving computational accuracy. Furthermore, the multiplexing of input optical signals reduces the number of electro-optical conversion interfaces and also reduces the complexity of related wiring and electrical design. Each output module is configured to superimpose the optical fields output by its coupled sub-processing modules to obtain the output optical field before performing photoelectric conversion, further reducing the number of photoelectric conversion interfaces and reducing the complexity of related wiring and electrical design. Attached Figure Description

[0027] The above and other objectives, features and advantages of this application will become more apparent from a detailed description of exemplary embodiments thereof with reference to the accompanying drawings.

[0028] Figure 1 A schematic diagram of the architecture layout of a segmented optical neural network in one embodiment of this application is shown.

[0029] Figure 2 This invention illustrates the correlation matrix during matrix calculation performed by a block-based optical neural network in one embodiment of the present application, wherein... Figure 2 (a) is the input matrix. Figure 2 (b) is the weight matrix. Figure 2 (c) is the output matrix.

[0030] Figure 3 The working principle of a segmented optical neural network 100 in one embodiment of this application is shown.

[0031] Figure 4 This invention illustrates the correlation matrix during matrix calculation performed by a block-based optical neural network in one embodiment of the present application, wherein... Figure 4 (a) is the input matrix. Figure 4 (b) is the weight matrix. Figure 4 (c) is the output matrix.

[0032] Figure 5 The working principle of a segmented optical neural network 100 in one embodiment of this application is shown.

[0033] Figure 6 The working principle of a segmented optical neural network 100 in one embodiment of this application is shown.

[0034] Figure 7 A cross-sectional schematic diagram of a segmented optical neural network according to an embodiment of this application is shown.

[0035] Figure 8 A cross-sectional schematic diagram of a segmented optical neural network according to an embodiment of this application is shown.

[0036] Figure 9 The diagram shows a schematic representation of the input and output modules of a segmented optical neural network according to an embodiment of this application.

[0037] Figure 10 This invention illustrates the input and output information of a block-based optical neural network performing matrix operations in one embodiment of the present application; wherein, Figure 10 (a) is the weight matrix of the block-based optical neural network 100. Figure 10 (b) is the input matrix. Figure 10 (c) is the output matrix. Figure 10 (d) shows the output results of the comparison.

[0038] Figure 11 This invention illustrates the input and output information of a block-based optical neural network performing matrix operations in one embodiment of the present application; wherein,Figure 11 (a) is the weight matrix of the block-based optical neural network 100. Figure 11 (b) is the input matrix. Figure 11 (c) is the output matrix. Figure 11 (d) shows the output results of the comparison.

[0039] Figure 12 A flowchart of a matrix calculation method in one embodiment of this application is shown.

[0040] Figure 13 A cross-sectional view of a segmented optical neural network chip according to an embodiment of this application is shown.

[0041] Figure 14 A cross-sectional view of a segmented optical neural network chip according to an embodiment of this application is shown.

[0042] Figure Labels

[0043] 100, Segmented optical neural network; 10, Input module; 110, Input waveguide; 120, Input coupler; 20, Subprocessing module; 210, Waveguide; 220, Diffractive optical element; 30, Output module; 310, Output waveguide; 320, Output coupler. Detailed Implementation

[0044] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided to make the description of this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The drawings are merely illustrative of this application and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0045] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more exemplary embodiments. Numerous specific details are provided in the following description to give a full understanding of exemplary embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced with one or more specific details omitted, or other modules, components, etc., can be employed. In other instances, well-known structures, methods, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0046] Please see Figure 1 , Figure 1A schematic diagram of the architecture of the segmented optical neural network 100 provided in this application is shown. The segmented optical neural network 100 includes at least two input modules 10, at least two output modules 30, and at least two sub-processing modules 20.

[0047] Input module 10 is used to input optical signals. Since there are at least two input modules 10, they are arranged side by side with intervals to avoid optical crosstalk. It should be noted that "side by side" here does not mean that the input modules 10 need to be strictly parallel. That is, the input modules 10 can be arranged in parallel or non-parallel, as long as they are independent of each other.

[0048] Output module 30 is used to output optical signals. Since there are at least two output modules 30, they are arranged side by side with intervals to avoid optical crosstalk. It should be noted that "side by side" here does not mean that the output modules 30 need to be strictly parallel. That is, the output modules 30 can be arranged in parallel or non-parallel, as long as they are independent of each other.

[0049] Each sub-processing module 20 has the same physical structure and is arranged in an array. It is worth mentioning that the "array arrangement" here does not necessarily require that each sub-processing module 20 be set in strict parallel and equally spaced order. That is, each sub-processing module 20 can be arranged in a strict array or an approximate array arrangement.

[0050] Sub-processing modules 20 located in the same row are coupled to the same input module 10, while sub-processing modules 20 located in different rows are coupled to different input modules 10. Sub-processing modules 20 located in the same column are coupled to the same output module 30, while sub-processing modules 20 located in different columns are coupled to different output modules 30.

[0051] The input information of each sub-processing module 20 satisfies the following: each sub-processing module 20 coupled to the same input module 10 has the same input optical signal, which enables the multiplexing of input optical signals, reduces the number of electro-optical conversion interfaces, and also reduces the complexity of related wiring and electrical design. Each sub-processing module 20 processes computational tasks in parallel. Each output module 30 is configured to superimpose the light fields output by each sub-processing module 20 coupled to it to obtain an output light field. Photoelectric conversion of the output light field yields the output result of the block-type optical neural network 100. Because photoelectric conversion is performed only after the output light field is superimposed, rather than performing photoelectric conversion and summing on the light fields output by each sub-processing module 20 separately, the number of photoelectric conversion interfaces can be reduced, and the complexity of related wiring and electrical design can also be reduced. Compared to optical neural networks in the prior art, the block-type optical neural network 100 provided in this application, with each sub-processing module 20 processing in parallel during computational tasks, significantly reduces error accumulation and improves computational accuracy.

[0052] By training the segmented optical neural network 100, it can achieve a target response to the optical signal corresponding to the input matrix. During optical computation, the segmented optical neural network 100 is optically mapped to a weight matrix. Therefore, the output information of the segmented optical neural network 100 is the result of matrix computation performed on the weight matrix and the input matrix. By configuring the modulation coefficients of the segmented optical neural network 100, it can be optically mapped to different weight matrices and / or configured to perform different types of matrix operations during optical computation. For example, by changing the modulation coefficient of one segmented optical neural network 100, the optically mapped weight matrix can be changed from one matrix to another. Similarly, by changing the modulation coefficient of one segmented optical neural network 100, it can be changed from performing matrix multiplication to performing matrix addition. Since matrix convolution and division operations can be directly mapped to matrix multiplication in mathematics, the block-based optical neural network 100 provided in this application can perform matrix calculations including but not limited to the four arithmetic operations (matrix addition, matrix subtraction, matrix multiplication, and matrix division) and convolution operations.

[0053] Furthermore, since the block-type optical neural network 100 provided in this application generates multiple sub-processing modules 20 through weight decomposition, the modulation units of each modulation layer in each sub-processing module 20 are reduced, and the modulation unit density of the optical neural network satisfies formula (1):

[0054] (1);

[0055] Where dx is the period of the modulation unit, and N LLet N be the number of modulation units in a single modulation layer, and α be the linear coefficient between the modulation layer spacing and the modulation layer height. According to formula (1), the number of modulation units N... L Reducing the number of modulation units allows for a significant increase in the density of the segmented optical neural network 100 provided in this application.

[0056] Please see Figure 1 , Figure 1 The architecture of the sub-processing module 20 is shown. Since the physical structure of each sub-processing module 20 is the same, therefore, Figure 1 Only the detailed structure of the sub-processing module 20 located in the upper left corner is labeled. The physical structures of all sub-processing modules 20 are identical. Each sub-processing module 20 includes a waveguide 210 and at least one diffractive optical element 220. Each sub-processing module 20 can be equipped with 1 to 4 diffractive optical elements 220 to improve computational accuracy. The diffractive optical elements 220 are embedded in the waveguide 210, such that when a sub-processing module 20 includes p (p≥1, p∈N*) diffractive optical elements 220, the sub-processing module 20 has p+1 waveguides 210. The diffractive optical elements 220 are used to modulate the input optical signal; one diffractive optical element 220 constitutes a modulation layer. Each waveguide 210 is used to propagate the input optical signal in a quasi-free diffraction manner.

[0057] In some embodiments, each sub-processing module 20 further includes at least one input port (not shown) and at least one output port (not shown). The number of input ports is greater than or equal to the number of elements in the sub-input matrix corresponding to the optical signal input by the input module 10, and each input port is used to transmit one matrix element from the sub-input matrix corresponding to the input optical signal. The number of output ports is greater than or equal to the number of elements in the output matrix corresponding to the optical signal output by the sub-processing module 20, and each output port is used to transmit one matrix element from the output matrix corresponding to the optical signal output by the sub-processing module 20. For example, when the sub-input matrix corresponding to the optical signal input by the input module 10 is a 3×4 matrix, the corresponding sub-processing module 20 has at least 12 input ports, and each input port is used to transmit the optical signal corresponding to one matrix element. For example, when the matrix corresponding to the optical signal output by a certain sub-processing module 20 is a 5×4 matrix, the corresponding sub-processing module 20 has at least 20 output ports, and each output port is used to transmit the optical signal corresponding to one matrix element.

[0058] In some embodiments, the number of rows and columns of the optical mapping matrix of each sub-processing module 20 in the segmented optical neural network 100 are greater than or equal to 2, which can reduce the number of input ports and output ports of each sub-processing module 20, thereby simplifying the physical structure of each sub-processing module 20 and reducing the processing difficulty and processing cost of the segmented optical neural network 100.

[0059] In some embodiments, when the number of rows and columns of the optical mapping matrix of each sub-processing module 20 is greater than or equal to 2, each sub-processing module 20 of the block optical neural network 100 has 3 to 4 diffractive optical elements 220, that is, each sub-processing module 20 has 3 to 4 modulation layers, and each modulation layer has several modulation units. Due to the existence of processing errors, the modulation units are coupled to each other. Each modulation layer introduces new errors and passes the errors of the previous layer to the next layer. Therefore, the more modulation layers there are, the greater the error will be. On the other hand, if the number of modulation layers is too small, the function fitting ability of the block optical neural network 100 will decrease, for example, the fitting matrix multiplication effect will be extremely poor. Therefore, setting 3 to 4 modulation layers in each sub-processing module 20 can enable the block optical neural network 100 to have both high computational accuracy and strong function fitting ability.

[0060] The following text combines Figure 2 and Figure 3 Taking matrix multiplication as an example, this paper details the working principle of matrix multiplication in the block-based optical neural network 100 when the number of rows and columns of the optical mapping matrix of each sub-processing module 20 is greater than or equal to 2. Figure 2 This application shows the input matrix X, the optical mapping weight matrix W, and the output matrix Y of the block optical neural network 100 when performing matrix calculations. Figure 3 The working principle of the segmented optical neural network 100 provided in this application is illustrated.

[0061] Please see Figure 2 (a) Input matrix Given any element x in the input matrix X i,j (1≤i≤4, 1≤j≤4, i, j∈N*) The first subscript i indicates the row it belongs to, and the second subscript j indicates the column it belongs to. For example, x 2,3 This represents the element located in the 2nd row and 3rd column of the input matrix X. For ease of demonstration, the input matrix X is divided into sub-input matrices X1, X2, X3, and X4, each of which is a 2×2 matrix.

[0062] In the field of optical computing, optical neural networks are optically mapped to weight matrices during computation. (See also...) Figure 2 (b) The segmented optical neural network 100 provided in this application is optically mapped to a weight matrix W. Any element w in the weight matrix W m,n (1≤m≤4, 1≤n≤4, m, n∈N*) The first subscript m indicates its row, and the second subscript n indicates its column. For example, w2,3 This represents the element located in the 2nd row and 3rd column of the input matrix W. The block-based optical neural network 100 is optically mapped to a weight matrix W. The weight matrix W is then divided into blocks to generate sub-weight matrices W1, W2, W3, and W4, each of which is a 2×2 matrix. For example, sub-weight matrix W1 corresponds to the element located in the 2nd row and 3rd column of the input matrix W. 1,1 w 1,2 w 2,1 w 2,2 The block-based optical neural network 100 consists of four sub-processing modules 20, the composition of which is not listed here. Each sub-weight matrix corresponds to a sub-processing module 20, meaning the block-based optical neural network 100 includes four sub-processing modules 20. The upper input module 10 is coupled to the sub-processing modules 20 corresponding to sub-weight matrices W1 and W3, and has the same input information as the sub-processing modules 20 corresponding to sub-weight matrices W1 and W3. The lower input module 10 is coupled to the sub-processing modules 20 corresponding to sub-weight matrices W2 and W4, and has the same input information as the sub-processing modules 20 corresponding to sub-weight matrices W2 and W4. The left-side output module 30 is coupled to the sub-processing modules 20 corresponding to sub-weight matrices W1 and W2, and the right-side output module 30 is coupled to the sub-processing modules 20 corresponding to sub-weight matrices W3 and W4.

[0063] Please see Figure 2 (c) and combined Figure 3 The block-based optical neural network 100 calculates the weight matrix W and the input matrix X, and then performs a multiplication operation to output the matrix Y = W × X. First, the optical signal corresponding to sub-input matrix X1 is input to the input module 10 in the first row, and the optical signal corresponding to sub-input matrix X3 is input to the input module 10 in the second row. The sub-output matrix Y1 = W1∙X1 + W2∙X3 corresponds to the optical signal output by the output module 30 on the left, and the sub-output matrix Y3 = W3∙X1 + W4∙X3 corresponds to the optical signal output by the output module 30 on the right. Finally, the sub-input matrix X2 is input to the input module 10 in the first row, and the sub-input matrix X4 is input to the input module 10 in the second row. The sub-output matrix Y2 = W1∙X2 + W2∙X4 corresponds to the optical signal output by the output module 30 on the left, and the sub-output matrix Y4 = W3∙X2 + W4∙X4 corresponds to the optical signal output by the output module 30 on the right. Sub-output matrices Y1, Y2, Y3, and Y4 together form the output matrix Y.

[0064] Of course, this process can also be based on Figure 3The block-type optical neural network 100 shown in the diagram spatially replicates another identical and completely independent block-type optical neural network 100. The first block-type optical neural network 100 is input with the optical signals corresponding to sub-input matrices X1 and X3, while the second block-type optical neural network 100 is input with sub-input matrices X2 and X4. The first block-type optical neural network 100 outputs sub-output matrices Y1 and Y3, and the second block-type optical neural network 100 outputs sub-output matrices Y2 and Y4. The output matrix Y is obtained by combining the outputs of the two block-type optical neural networks 100. This process involves the two block-type optical neural networks 100 operating in parallel over time, which improves computational speed.

[0065] Combination Figure 2 (c) and Figure 3 Substituting the sub-weight matrices W1, W2, W3, W4, and sub-input matrices X1, X2, X3, and X4 into the output matrix Y yields the same result as directly calculating the product of the weight matrix W and the input matrix X: Y = W × X. This demonstrates that the block-based optical neural network 100 provided in this application can be used for matrix calculations. It should be noted that this explanation is only intended to illustrate the working principle of the block-based optical neural network 100 and should not be construed as limiting the number of its 20 sub-processing modules or their functions. For matrix addition, subtraction, division, and convolution operations, the input and output methods remain consistent with matrix multiplication, but the block-based optical neural network 100 is configured to execute the corresponding matrix operation rules on the input matrix.

[0066] In some embodiments, the optical mapping matrix of each sub-processing module 20 is either a row matrix or a column matrix; that is, either the number of rows or the number of columns in the optical mapping matrix of each sub-processing module 20 is 1, and the other is greater than or equal to 2. See also... Figure 5 When the optical mapping matrices of each sub-processing module 20 are row matrices or column matrices, the expressive power requirements of each sub-processing module 20 are reduced. Each sub-processing module 20 only needs to set a smaller number (e.g., 1, 2, or 3) of diffractive optical elements 220, which can significantly reduce the requirements on the sub-processing module 20, thereby reducing the structural complexity and manufacturing cost of the sub-processing module 20. At the same time, since each sub-processing module 20 only needs to set a smaller number of diffractive optical elements 220, matrix calculation can be completed through fewer quasi-free diffractions, which can reduce the interlayer accumulation error of each sub-processing module 20 and thus improve the calculation accuracy.

[0067] Furthermore, when the optical mapping matrix of each sub-processing module 20 is a row matrix or column matrix, the matrix input to the output module 30 is also a row matrix or column matrix. In this case, each output module 30 is coupled to only one sub-processing module 20. The information carried by the optical field output by each sub-processing module 20 is the element in the output matrix. That is, each output module 30 is configured to superimpose the optical field output by only the sub-processing module coupled to it, which reduces the length of the output module 30 and simplifies its structure. Since each output module 30 is configured to superimpose the optical field output by only the sub-processing module coupled to it, please refer to... Figure 6 The block-type optical neural network 100 can also omit the output module 30 and directly perform photoelectric conversion on the light field output by each sub-processing module 20 to obtain the output result of the block-type optical neural network 100.

[0068] The following text combines Figure 4 and Figure 5 Taking matrix multiplication as an example, this paper details the working principle of matrix multiplication in the block-based optical neural network 100 when the optical mapping matrices of each sub-processing module 20 are row matrices or column matrices. Figure 4 This application shows the input matrix X, the optical mapping weight matrix W, and the output matrix Y of the block optical neural network 100 when performing matrix calculations. Figure 5 The working principle of the segmented optical neural network 100 provided in this application is illustrated.

[0069] Please see Figure 4 (a) Input matrix Given any element x in the input matrix X i,j (1≤i≤4, 1≤j≤4, i, j∈N*) The first subscript i indicates the row it belongs to, and the second subscript j indicates the column it belongs to. For example, x 1,3 This represents the element located in the 1st row and 3rd column of the input matrix X. The input matrix X is divided into blocks to generate sub-input matrices X1, X2, X3, and X4, each of which is a 4×1 column matrix.

[0070] Please see Figure 4 (b) The segmented optical neural network 100 provided in this application is optically mapped to a weight matrix W. Any element w in the weight matrix W m,n (1≤m≤4, 1≤n≤4, m, n∈N*) The first subscript m indicates its row, and the second subscript n indicates its column. For example, w 1,4This represents the element located in the 1st row and 4th column of the input matrix W. The block-based optical neural network 100 is optically mapped to a weight matrix W. The weight matrix W is then divided into blocks to generate sub-weight matrices W1, W2, W3, and W4, each of which is a 1×4 row matrix. Each sub-weight matrix corresponds to a sub-processing module 20. That is, the block-based optical neural network 100 includes four sub-processing modules 20. The matrices corresponding to the optical signals output by the four sub-processing modules 20 together form the output matrix Y.

[0071] Please see Figure 4 (c) and combined Figure 5 The block-based optical neural network 100 calculates the weight matrix W and the input matrix X, and then performs a multiplication operation to output the matrix Y = W × X. Output the elements Y in matrix Y i,j =Wi∙Xj. First, the light signal corresponding to the sub-input matrix X1 is input to the input modules 10 located in the first and second rows. The light signal output by the output module 30 located on the left side of the first row corresponds to the element Y in the output matrix Y. 1,1 Y 1,1 =W1∙X1, the optical signal output by output module 30 located on the right side of the first row corresponds to element Y in the output matrix Y. 2,1 Y 2,1 =W2∙X1, the optical signal output by output module 30 located on the left side of the second row corresponds to element Y in the output matrix Y. 3,1 Y 3,1 =W3∙X1, the optical signal output by output module 30 on the right side of the second row corresponds to element Y in the output matrix Y. 4,1 Y 4,1 =W4∙X1. Next, the light signals corresponding to the sub-input matrix X2 are input to the input modules 10 located in the first and second rows. The light signals output by the four output modules 30 correspond to elements Y in the output matrix Y, respectively. 1,2 Y 2,2 Y 3,2 Y 4,2 Next, the light signals corresponding to the sub-input matrix X3 are input to the input modules 10 located in the first and second rows, and the light signals output by the four output modules 30 correspond to the elements Y in the output matrix Y, respectively. 1,3 Y 2,3 Y 3,3 Y 4,3 Finally, the light signals corresponding to the sub-input matrix X4 are input to the input modules 10 located in the first and second rows, and the light signals output by the four output modules 30 correspond to the elements Y in the output matrix Y, respectively. 1,4 Y 2,4 Y 3,4 Y4,4 Y 1,1 Y 2,1 Y 3,1 Y 4,1 Y 1,2 Y 2,2 Y 3,2 Y 4,2 Y 1,3 Y 2,3 Y 3,3 Y 4,3 Y 1,4 Y 2,4 Y 3,4 Y 4,4 Together they form the output matrix Y.

[0072] Combination Figure 4 (c) and Figure 5 The output matrix Y is consistent with the direct calculation of the product of the weight matrix W and the input matrix X, both being Y = W × X, indicating that the block-based optical neural network 100 provided in this application can be used for matrix calculations. It should be noted that this is only used to explain the working principle of the block-based optical neural network 100 and should not be construed as limiting the number of its 20 sub-processing modules or their functions. For matrix addition, matrix subtraction, matrix division, and matrix convolution operations, the input and output methods remain consistent with matrix multiplication, but the block-based optical neural network 100 is configured to perform the corresponding matrix operation rules on the input matrix.

[0073] In some embodiments, the segmented optical neural network 100 further includes photodetectors (not shown), each corresponding to an output module. The photodetectors are used to perform photoelectric conversion on the output light field of the output module 30.

[0074] Furthermore, in some embodiments, the photodetector, input module 10, subprocessing module 20, and output module 30 are integrated together to make the segmented optical neural network 100 an on-chip structure, thereby reducing the size of the segmented optical neural network 100.

[0075] Furthermore, in some embodiments, the photodetector is not integrated with the input module 10, the sub-processing module 20, and the output module 30. In this case, the input module 10, the sub-processing module 20, and the output module 30 are integrated, and the photodetector is independent of the integrated structure of the input module 10, the sub-processing module 20, and the output module 30. The output light field of the output module 30 is projected into space and then converted into photoelectric value by the photodetector.

[0076] Please refer to the following: Figure 2 and Figure 3In some embodiments, the merged matrix composed of the optical mapping matrices of all subprocessing modules 20 is the same as the optical mapping weight matrix W of the block optical neural network 100. Figure 3 Taking the segmented optical neural network 100 shown as an example, it includes four sub-processing modules 20. The optical mapping matrices of the four sub-processing modules 20 are sub-weight matrix W1, sub-weight matrix W2, sub-weight matrix W3, and sub-weight matrix W4, respectively. Combined with... Figure 2 (b) The matrix formed by sub-weight matrices W1, W2, W3, and W4 is the same as the optical mapping weight matrix W of the block-based optical neural network 100. In other words, the sub-matrices generated after dividing the optical mapping weight matrix W of the block-based optical neural network 100 into blocks ( Figure 2 (b) Sub-weight matrices W1, W2, W3, and W4 all have corresponding sub-processing modules 20, and each sub-matrix (sub-weight matrix) after the optical mapping weight matrix W of the block-type optical neural network 100 is the same as the optical mapping matrix of the corresponding sub-processing module 20. The optical mapping matrices of all sub-processing modules 20 together form the optical mapping weight matrix W of the block-type optical neural network 100.

[0077] Furthermore, the optical mapping matrices of different sub-processing modules 20 are located at different positions in the optical mapping weight matrix W of the block optical neural network 100, and the positions of the optical mapping matrices of different sub-processing modules 20 in the optical mapping weight matrix W of the block optical neural network 100 do not overlap. Figure 3 The optical mapping matrices of the four sub-processing modules 20 in the block optical neural network 100 are sub-weight matrices W1, W2, W3, and W4, respectively. Sub-weight matrix W1 is located in the upper left corner of the optical mapping weight matrix W of the block optical neural network 100, sub-weight matrix W2 is located in the upper right corner of the optical mapping weight matrix W of the block optical neural network 100, sub-weight matrix W3 is located in the lower left corner of the optical mapping weight matrix W of the block optical neural network 100, and sub-weight matrix W4 is located in the lower right corner of the optical mapping weight matrix W of the block optical neural network 100. The positions of sub-weight matrices W1, W2, W3, and W4 in the optical mapping matrix W of the block optical neural network 100 do not overlap.

[0078] Please see Figure 3In some embodiments, each input module 10 has independent input information. Different blocks of the optical mapping matrix W of the block-based optical neural network 100 correspond to different sub-processing modules 20, enabling each sub-processing module 20 to process different matrix operations in parallel. This allows each sub-processing module 20 to process different sub-tasks in parallel. The output module 30 superimposes and sums the light fields output by each sub-processing module 20 coupled to it. By combining the information output by the output module 30, the output result of performing a preset type of matrix operation (matrix addition, matrix subtraction, matrix multiplication, matrix division, convolution, etc.) on the optical mapping weight matrix W of the block-based optical neural network 100 and the input matrix X corresponding to the input light signal can be obtained. Each sub-processing module 20 processes different sub-tasks in parallel, enabling task decomposition and input information reuse. For example, in Figure 3 In the diagram, although the two sub-processing modules 20 in the first row have the same input information (the optical signal corresponding to the sub-input matrix X1), because the optical mapping matrices of the two sub-processing modules 20 are different, the sub-processing module 20 located in the upper left handles the sub-task W1∙X1, and the sub-processing module 20 located in the upper right handles the sub-task W3∙X1, allowing the sub-processing modules 20 in the same row to process different matrix operations in parallel; Figure 3 In this application, sub-processing modules 20 located in different rows have different input information, and each input module 10 is optically mapped to a different weight matrix, enabling the sub-processing modules 20 located in different rows to process different matrix operations in parallel. That is, by combining the input modules 10 located in different rows, the block-based optical neural network 100 provided by this application can realize parallel processing of different matrix operations by each sub-processing module 20. Since the sub-processing modules 20 are in parallel relationship, there is no information transfer between them, which avoids the problem of error accumulation and amplification, and can significantly improve computational accuracy. Furthermore, since the sub-processing modules 20 located in the same row have the same input information, for example, Figure 3 The two sub-processing modules 20 located in the first row have the same input information—the optical signal corresponding to the sub-input matrix X1. Figure 3 The two sub-processing modules 20 located in the second row have the same input information—the optical signal corresponding to the sub-input matrix X3. Therefore, the sub-processing modules 20 located in the same row have the same input information, which can realize the multiplexing of the input optical signal, reduce the number of input electro-optic conversion interfaces, and reduce the complexity of related wiring and electrical design.

[0079] It is worth mentioning that "each input module 10 has independent input information" means that the input information of each input module 10 is independent of each other, and the sub-input matrices input by different input modules 10 can be the same or different. For example, in Figure 3 In the middle, the sub-input matrix X1 is The sub-input matrix X3 is The sub-input matrices X1 and X3 are different, causing the input information of the sub-processing module 20 located in the first row to be different from that of the sub-processing module 20 located in the second row. For example, the sub-input matrix X1 is... The sub-input matrix X3 is Sub-input matrices X1 and X3 are the same, so that the input information of sub-processing module 20 located in the first row is the same as that of sub-processing module 20 located in the second row.

[0080] Since the input module 10 and the output module 30 can be located on the same plane or on different planes in the height direction, there are no requirements for the relative position between the input module 10 and the output module 30.

[0081] In some embodiments, the input module 10 and the output module 30 are located on the same plane along the height direction, which allows the segmented optical neural network 100 to have a smaller interlayer thickness and thus a higher degree of integration.

[0082] Please see Figure 7 and Figure 8 , Figure 7 A cross-sectional view of the block-type optical neural network 100 along the arrangement direction of the input module 10 is shown. Figure 8 A cross-sectional view of the segmented optical neural network 100 along the arrangement direction of the output module 30 is shown. In some embodiments, the input module 10 and the output module 30 are located on different planes along the thickness direction, which can reduce crosstalk and loss between the input module 10 and the output module 30 to improve computational accuracy, while also reducing the complexity of the system.

[0083] Please see Figure 9 , Figure 9 The specific structure of the input module 10 in the modular optical neural network 100 is shown. In some embodiments, the input module 10 includes an input waveguide 110 and an input coupler 120, wherein the input waveguide 110 has at least one member. When there are two or more input waveguides 110, they are arranged in an array. The input coupler 120, the input waveguide 110, and the input port are configured in a one-to-one correspondence. One end of the input coupler 120 is coupled to the corresponding input waveguide 110, and the other end of the input coupler 120 is coupled to the input port of the corresponding sub-processing module 20.

[0084] Please see Figure 1 and Figure 9 , Figure 9The specific structure of the output module 30 in the modular optical neural network 100 is shown. The output module 30 includes an output waveguide 310 and an output coupler 320, with at least one output waveguide 310. When there are two or more output waveguides 310, they are arranged in an array. The output coupler 320, the output waveguide 310, and the output port are configured in a one-to-one correspondence. One end of the output coupler 320 is coupled to the corresponding output waveguide 310, and the other end of the output coupler 320 is coupled to the output port of the corresponding sub-processing module 20.

[0085] When the input module 10 and the output module 30 are located on the same plane along the height direction, the input waveguide 110 and the output waveguide 310 intersect each other. A cross waveguide structure can be used at the intersection of the input waveguide 110 and the output waveguide 310 to reduce crosstalk and loss between the input waveguide 110 and the output waveguide 310.

[0086] In some embodiments, the center distance between any two adjacent input couplers 120 is greater than 2.5 micrometers to avoid crosstalk between any two adjacent input couplers 120.

[0087] In some embodiments, the center distance between any two adjacent output couplers 320 is greater than 2.5 micrometers to avoid crosstalk between any two adjacent output couplers 320.

[0088] In some embodiments, the center distance between any two adjacent input waveguides 110 is greater than 2.5 micrometers to avoid crosstalk between any two adjacent input waveguides 110.

[0089] In some embodiments, the center distance between any two adjacent output waveguides 310 is greater than 2.5 micrometers to avoid crosstalk between any two adjacent output waveguides 310.

[0090] Please see Figure 1 In some embodiments, for each sub-processing module 20, there is no coupling between the input module 10 and the sub-processing module 20 except for the coupler (including the input coupler 120 and the output coupler 320).

[0091] In some embodiments, the segmented optical neural network 100 satisfies the following relationship during neural network task training:

[0092] (2);

[0093] (3);

[0094] (4);

[0095] in, For the m-th wavelength, the wavefront before the L-th modulation layer; For the m-th wavelength, the wavefront after the L-th modulation layer; Let m be the wavelength, and L be the modulation coefficient of the modulation layer. Z represents the Gaussian noise of the m-th wavelength, L-th modulation layer, and B-th sample in the training batch; FFT and IFFT are the Discrete Fourier Transform and its inverse transform, respectively; f is the spatial frequency corresponding to the Fourier transform; Z m This represents the distance between the (m-1)th modulation layer and the mth modulation layer. is the equivalent refractive index at the operating wavelength of waveguide 210; j is the imaginary unit. According to the relations (2), (3) and (4), the distance between two adjacent modulation layers in each sub-processing module 20 and the modulation coefficients corresponding to all modulation units in the diffractive optical element 220 can be determined during the training of the neural network task. Based on the distance between two adjacent modulation layers in each sub-processing module 20 and the modulation coefficients corresponding to all modulation units in the diffractive optical element 220, a block-type optical neural network 100 can be constructed.

[0096] Please see Figure 1 and Figure 3 In some embodiments, the diffractive optical element 220 is a phase-fixed metasurface, which is a metasurface with a fixed and non-tunable phase. The phase-fixed metasurface includes micro- and nano-structures, which are modulation units. The fabrication process of the metasurface is compatible with existing CMOS (complementary metal-oxide-semiconductor) fabrication processes, making it easy to fabricate and integrate the modular optical neural network 100.

[0097] In some embodiments, the diffractive optical element 220 is a phase-tunable metasurface, which includes micro / nano structures made of phase-change materials. Applying external excitation (including but not limited to electrical, thermal, and optical excitation) to the phase-tunable metasurface can change its phase, thereby enabling changes to the matrix of the optical mapping of the sub-processing module 20 and / or the type of matrix calculation performed by the sub-processing module 20 on the input matrix corresponding to the input optical signal. When the matrix of the optical mapping of the sub-processing module 20 changes, its reconstruction can be achieved, thereby improving the expressive power of the sub-processing module 20. When the type of matrix calculation performed by the sub-processing module 20 on the input matrix corresponding to the input optical signal changes, switching between different matrix calculation types can be achieved, improving system compatibility. For example, before the phase-tunable metasurface undergoes a phase transition, the weighted moment of the optical mapping of the sub-processing module 20 containing the phase-tunable metasurface is A; after applying excitation to the phase-tunable metasurface to change its phase, the weighted moment of the optical mapping of the sub-processing module 20 becomes B. For example, before the phase-tunable metasurface undergoes a phase transition, sub-processing module 20 performs matrix multiplication on the input matrix corresponding to the input optical signal. After applying an excitation to the phase-tunable metasurface to change its phase, sub-processing module 20 performs matrix addition on the input matrix corresponding to the input optical signal. For example, before the phase-tunable metasurface undergoes a phase transition, the weighted moments of the optical mapping of sub-processing module 20 containing the phase-tunable metasurface are C, and sub-processing module 20 performs matrix division on the input matrix corresponding to the input optical signal. After applying an excitation to the phase-tunable metasurface to change its phase, the weighted moments of the optical mapping of sub-processing module 20 become D, and sub-processing module 20 performs matrix multiplication on the input matrix corresponding to the input optical signal. The above examples are for illustrative purposes only and should not be construed as limitations.

[0098] The computational performance of the block-based optical neural network 100 provided in this application is illustrated below with two specific embodiments.

[0099] Example 1

[0100] The modular optical neural network 100 includes four sub-processing modules 20, arranged in a 2x2 array. Each sub-processing module 20 has the same physical structure, comprising 32×32 inputs, 32×32 outputs, and four spaced metasurfaces. Each metasurface includes 32×32 modulation units (micro / nano structures), meaning each sub-processing module 20 contains a total of 32×32×4 modulation units. The modular optical neural network 100 operates at a wavelength of 1550 nm. The waveguide is a silicon substrate with a silicon dioxide cladding structure at its outer edge. The waveguide 210 has a thickness of 220 nm and an equivalent refractive index n. eff=2.84. The modulation unit has a period of 0.5 μm and a length that varies from 0 to 3 μm, covering a 2π phase.

[0101] See Figure 10 (a), Figure 10 (a) is the weight matrix W of the 100 optical mapping of the block-based optical neural network. The weight matrix W is a 64×64 matrix, that is, the total number of weights is 64×64. The large-scale weight matrix W is divided into 2×2 sub-weight matrices according to the matrix partitioning rules. That is, after the weight matrix W is partitioned, it is a new matrix with 2 rows and 2 columns. Each sub-weight matrix includes 32×32 inputs.

[0102] The segmented optical neural network 100 performs matrix multiplication, meaning each sub-processing module 20 performs matrix multiplication operations. See also Figure 10 (b) Figure 10 (b) is the input matrix X, and the block partitioning method for the input matrix X is the same as that for the weight matrix W. The task of each sub-weighted optical mapping is assigned to four sub-processing modules 20 for execution. The modulation unit can selectively add a Gaussian phase error with a mean of 0 and a variance of 0.02, ultimately yielding the following result: Figure 10 (c) shows the output error matrix Y. The result is shown as a percentage error. The percentage error is calculated by dividing the absolute value of the output light intensity difference of the test set with and without Gaussian phase error by the average value of the output light intensity without Gaussian phase error.

[0103] Comparing the results with the same input matrix, the same weight matrix, the same number of modulation units per modulation layer (ensuring the same modulation unit density), and the same total number of modulation units, a comparative example using a single optical neural network with 16 modulation layers shows the following output error matrix: Figure 10 As shown in (d), the results are presented as percentage errors. The percentage error is calculated by dividing the absolute value of the difference in output light intensity of the test set with and without Gaussian phase error by the average value of the output light intensity without Gaussian phase error.

[0104] The results of matrix multiplication fitting are presented in the form of error. The average light intensity error is defined as the sum of the absolute values ​​of the output light intensity differences of the test set with and without Gaussian phase error, divided by the sum of the output light intensity without Gaussian phase error. An error of 0 indicates that phase noise has no effect on the output. The average light intensity error of the output in this embodiment is only 3.85%, while the average light intensity error of the output of the comparative single optical neural network is 7.12%. It can be seen that, under the same modulation unit density, i.e., the same number of modulation units in a single modulation layer, the comparative single optical neural network requires 16 modulation layers, while the sub-processing module 20 of this embodiment only requires 4 modulation layers. Therefore, the average light intensity error of this embodiment is lower.

[0105] Based on the above conclusions, it is easy to conclude that, while maintaining the same error, the modulation unit density of this embodiment is higher than that of the comparative single optical neural network.

[0106] Example 2

[0107] The modular optical neural network 100 comprises 16 sub-processing modules 20, arranged in a 4x4 array. Each sub-processing module 20 has the same physical structure, including 32×32 inputs, 32×32 outputs, and 4 spaced metasurfaces. Each metasurface includes 32×32 modulation units (micro / nano structures), meaning each sub-processing module 20 comprises a total of 32×32×4 modulation units. The modular optical neural network 100 operates at a wavelength of 1550 nm. The waveguide is a silicon substrate with a silicon dioxide cladding structure at its outer edge. The waveguide 210 has a height of 220 nm and an equivalent refractive index n. eff =2.84. The modulation unit has a period of 0.5 μm and a length that varies from 0 to 3 μm, covering a 2π phase.

[0108] See Figure 11 (a), Figure 11 (a) is the weight matrix W of the 100 optical mapping in the block-based optical neural network. The weight matrix W is a 128×128 matrix, that is, the total number of weights is 128×128. The large-scale weight matrix W is divided into 4×4 sub-weight matrices according to the matrix partitioning rules. That is, the weight matrix W after being partitioned is a new matrix with 4 rows and 4 columns. Each sub-weight matrix includes 32×32 inputs.

[0109] The segmented optical neural network 100 performs matrix multiplication, meaning each sub-processing module 20 performs matrix multiplication operations. See also Figure 11 (b) Figure 11 (b) is the input matrix X, and the block partitioning method for the input matrix X is the same as that for the weight matrix W. The task of each sub-weight mapping is assigned to 16 sub-processing modules 20 for execution. The modulation unit can selectively add a Gaussian phase error with a mean of 0 and a variance of 0.02, ultimately yielding the following result: Figure 11 (c) shows the output error matrix Y. The result is shown as a percentage error. The percentage error is calculated by dividing the absolute value of the output light intensity difference of the test set with and without Gaussian phase error by the average value of the output light intensity without Gaussian phase error.

[0110] Comparing the results with the same input matrix, the same weight matrix, the same number of modulation units per modulation layer (ensuring the same modulation unit density), and the same total number of modulation units, a comparative example using a single optical neural network with 64 modulation layers shows the following output error matrix: Figure 11As shown in (d), the results are presented as percentage errors. The percentage error is calculated by dividing the absolute value of the difference in output light intensity of the test set with and without Gaussian phase error by the average value of the output light intensity without Gaussian phase error.

[0111] The results of matrix multiplication fitting are presented in the form of error. The average light intensity error is defined as the sum of the absolute values ​​of the output light intensity differences of the test set with and without Gaussian phase error, divided by the sum of the output light intensity without Gaussian phase error. An error of 0 indicates that phase noise has no effect on the output. In this embodiment, the average light intensity error is only 3.91%, while the average light intensity error of the comparative single optical neural network is 17.75%. It can be seen that, under the same modulation unit density, i.e., the same number of modulation units in a single modulation layer, the comparative single optical neural network requires 64 modulation layers, while the sub-processing module 20 of this embodiment only requires 4 modulation layers. Therefore, the average light intensity error of this embodiment is lower.

[0112] Based on the above conclusions, it is easy to conclude that, while maintaining the same error, the modulation unit density of this embodiment is higher than that of the comparative single optical neural network.

[0113] This application also provides a matrix calculation method based on the aforementioned block-based optical neural network 100, wherein the block-based optical neural network 100 is optically mapped to a weight matrix W, and matrix calculations according to corresponding rules can be performed on the input matrix X. The optically mapped matrices of all sub-processing modules 20 in the block-based optical neural network 100 together constitute the optical mapping matrix W of the block-based optical neural network 100. Please refer to... Figure 12 The matrix calculation method of the segmented optical neural network 100 includes steps S10 to S30.

[0114] Step S10: Divide the input matrix into blocks based on the number of rows and columns of the sub-processing module 20 to generate at least two sub-input matrices; sequentially input the optical signals corresponding to the sub-input matrices located in the same row or column to different input modules 10.

[0115] Please see Figure 2 and Figure 3 Because the sub-matrix in the weight matrix W of the optical mapping between the subprocessing module 20 and the block optical neural network 100 (e.g. Figure 2 (b) Sub-weight matrices W1, W2, W3, and W4 correspond one-to-one with sub-processing module 20. Therefore, the number of rows and columns of the sub-matrices in the optical mapping matrix W of the block optical neural network 100 can be known from the number of rows and columns of sub-processing module 20. In other words, the block division method of the block optical neural network 100 can be known from the number of rows and columns of sub-processing module 20.

[0116] The input matrix X is divided into blocks based on the number of rows and columns of the sub-processing module 20. That is, the input matrix X is divided into blocks according to the block-based optical mapping matrix W of the block-based optical neural network 100. This ensures that the sub-weight matrices of the optical mapping in the sub-processing module 20 and the sub-input matrices generated after dividing the input matrix X (e.g., sub-input matrices X1, X2, X3, and X4) can perform corresponding matrix calculations. For example, the sub-weight matrices and the sub-input matrices are multiplicative.

[0117] Since transposing the input matrix X can transform both rows and columns, for example, by transforming the rows of the input matrix X and the transpose of the input matrix X, we can achieve the same transformation. T Since the columns are the same, by using the matrix transpose operation, inputting sub-input matrices located in the same row to different input modules 10 in sequence, and inputting sub-input matrices located in the same column to different input modules 10 in sequence, will produce the same output result. Please combine this with... Figure 2 and Figure 3 The optical signals corresponding to sub-input matrices X1 and X3 located in the same column can be input into two different input modules 10 respectively, and then the optical signals corresponding to sub-input matrices X2 and X4 located in the same column can be input into two different input modules 10 respectively.

[0118] Step S20: Superimpose the output light fields of each sub-processing module coupled to the same output module to obtain the output light field.

[0119] Please combine Figure 3 For each output module 30, the output light field can be obtained by superimposing the light fields output by each sub-processing module 20 coupled to itself.

[0120] Step S30: Perform photoelectric conversion based on each output light field to obtain the calculation results of the weight matrix and input matrix of the optical mapping of the block optical neural network.

[0121] By combining the information obtained from the photoelectric conversion of the output light fields of different output modules 30, the calculation results of the weight matrix and input matrix of the optical mapping of the block optical neural network can be obtained. For example, when the matrix calculation method of the block optical neural network 100 is configured to perform matrix multiplication calculation, the output matrix Y, Y=W×X, is obtained by photoelectric conversion based on the output light fields of each output module 30.

[0122] In some embodiments, the matrix calculation method further includes configuring the modulation parameters of the block-based optical neural network 100 and the block-based method of the weight matrix W and the input matrix X, so that the block-based optical neural network 100 can switch between matrix addition, matrix subtraction, matrix multiplication, matrix division, and matrix convolution operations. The modulation parameters of the block-based optical neural network 100 can be obtained through training. Before use, training the block-based optical neural network 100 can make its optical mapping weight matrix W have a target response to the optical signal corresponding to the input matrix X. The target response includes, but is not limited to, performing matrix addition, matrix subtraction, matrix multiplication, matrix division, and matrix convolution operations.

[0123] Furthermore, in some embodiments, the block-based optical neural network 100 is configured to perform matrix addition or matrix subtraction. Provided that the weight matrix W and the input matrix X satisfy the requirement that matrix addition or matrix subtraction operations can be performed, the row and column blocks of the weight matrix W and the input matrix X should be completely consistent.

[0124] Furthermore, in some embodiments, the block-based optical neural network 100 is configured to perform matrix multiplication and matrix division, provided that the weight matrix W and the input matrix X satisfy the requirement that matrix multiplication can be performed, and the column blocks of the weight matrix W and the row blocks of the input matrix X should be completely consistent.

[0125] This application also provides a modular optical neural network chip; please refer to [link / reference]. Figure 13 and Figure 14 , Figure 13 and Figure 14 Cross-sectional views of the modular optical neural network chip in different directions are shown. The modular optical neural network chip includes at least one of the aforementioned modular optical neural networks 100. The specific architecture of the modular optical neural network 100 can be referred to above and will not be repeated here. When the modular optical neural network chip includes two or more modular optical neural networks 100, each modular optical neural network 100 forms a layer structure, and the modular optical neural networks 100 are stacked.

[0126] In some embodiments, each segmented optical neural network 100 is configured to process different tasks so that the segmented optical neural network chip can process multiple tasks in parallel.

[0127] In some embodiments, each segmented optical neural network 100 is configured to process the same task, which can reduce the time it takes for the segmented optical neural network chip to process a single task.

[0128] In some embodiments, the input module 10 and output module 30 of each segmented optical neural network 100 constituting the segmented optical neural network chip are located on the same plane and intersect, and the segmented optical neural networks 100 constituting the segmented optical neural network chip are stacked.

[0129] Please see Figure 13 , Figure 13 A cross-sectional view of a modular optical neural network chip is shown, where the dividing line S1 is the boundary between two adjacent modular optical neural networks 100. In some embodiments, the input module 10 and output module 30 of each modular optical neural network 100 constituting the modular optical neural network chip are interleaved, and the input module 10 and output module 30 are located on different planes along the thickness direction. The input modules 10 of two adjacent modular optical neural networks 100 are staggered so that the input modules 10 at the same position of two adjacent modular optical neural networks 100 are not stacked in the thickness direction. This allows the interface shapes of two adjacent modular optical neural networks 100 to be complementary, making full use of space and thus enabling the modular optical neural network chip to have a high modulation unit density.

[0130] Please see Figure 14 , Figure 14 A cross-sectional view of a modular optical neural network chip is shown, where the dividing line S2 is the boundary between two adjacent modular optical neural networks 100. In some embodiments, the input module 10 and output module 30 of each modular optical neural network 100 constituting the modular optical neural network chip are interleaved, and the input module 10 and output module 30 are located on different planes along the thickness direction. The output modules 30 of two adjacent modular optical neural networks 100 are staggered, so that the output modules 30 at the same position of two adjacent modular optical neural networks 100 are not stacked in the thickness direction. This allows the interface shapes of two adjacent modular optical neural networks 100 to be complementary, making full use of space and thus enabling the modular optical neural network chip to have a high modulation unit density.

[0131] This application also provides an electronic device (not shown), which includes the above-described segmented optical neural network chip to enable the electronic device to have high computational accuracy.

[0132] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the appended claims.

Claims

1. A segmented optical neural network, characterized in that, include: At least two input modules are provided, with each input module arranged side-by-side at intervals. At least two output modules are provided, with each output module arranged side-by-side at intervals. At least two sub-processing modules are arranged in an array, with sub-processing modules located in the same row coupled to the same input module, and sub-processing modules located in the same column coupled to the same output module. Each of the sub-processing modules coupled to the same input module has the same input optical signal, and each of the output modules is configured to superimpose the optical fields output by each of the sub-processing modules coupled to it.

2. The segmented optical neural network according to claim 1, characterized in that, The merged matrix composed of the optical mapping matrices of all the subprocessing modules is the same as the optical mapping weight matrix of the block-based optical neural network.

3. The segmented optical neural network according to claim 2, characterized in that, The optical mapping matrices of different subprocessing modules are located at different positions in the optical mapping weight matrix of the segmented optical neural network and do not overlap.

4. The segmented optical neural network according to claim 1, characterized in that, The optical mapping matrix of each of the sub-processing modules has a number of rows greater than or equal to 2 and a number of columns greater than or equal to 2.

5. The segmented optical neural network according to claim 1, characterized in that, The optical mapping matrix of each of the sub-processing modules is either a row matrix or a column matrix.

6. The segmented optical neural network according to claim 1, characterized in that, Each of the sub-processing modules includes at least one input port and at least one output port; Each input module includes an input waveguide and an input coupler. The input waveguide has at least one component, and the input waveguides are arranged in an array. The input coupler, the input waveguide, and the input port are arranged in a one-to-one correspondence. One end of the input coupler is coupled to the corresponding input waveguide, and the other end of the input coupler is coupled to the corresponding input port. Each output module includes an output waveguide and an output coupler. There is at least one output waveguide, and the output waveguides are arranged in an array. The output coupler, the output waveguide, and the output port are arranged in a one-to-one correspondence. One end of the output coupler is coupled to the corresponding output waveguide, and the other end of the output coupler is coupled to the corresponding output port.

7. The segmented optical neural network according to claim 6, characterized in that, The center distance between any two adjacent input couplers is greater than 2.5 micrometers; and / or; The center distance between any two adjacent output couplers is greater than 2.5 micrometers; and / or; The center distance between any two adjacent input waveguides is greater than 2.5 micrometers; and / or; The center distance between any two adjacent output waveguides is greater than 2.5 micrometers.

8. The segmented optical neural network according to any one of claims 1-7, characterized in that, Each of the sub-processing modules includes a waveguide and at least one diffractive optical element; the diffractive optical element is embedded in the waveguide and is used to modulate the input optical signal; the waveguide is used to propagate the input optical signal in a quasi-free diffraction manner.

9. The segmented optical neural network according to claim 8, characterized in that, The diffractive optical element is a phase-fixed metasurface or a phase-tunable metasurface.

10. A matrix calculation method based on a block-based optical neural network as described in any one of claims 1-9, characterized in that, include: The input matrix is ​​divided into blocks based on the number of rows and columns of the sub-processing module to generate at least two sub-input matrices; The optical signals corresponding to the sub-input matrices located in the same row or column are sequentially input to different input modules; The output light field is obtained by superimposing the light fields output by each of the sub-processing modules coupled to the same output module; Photoelectric conversion is performed based on each of the output light fields to obtain the calculation results of the weight matrix and the input matrix of the optical mapping of the segmented optical neural network.

11. The matrix calculation method according to claim 10, characterized in that, The matrix calculation method also includes: Configure the modulation parameters of the segmented optical neural network and the segmentation method of the weight matrix and the input matrix so that the segmented optical neural network can switch between matrix addition, matrix subtraction, matrix multiplication, matrix division and matrix convolution operations.

12. A modular optical neural network chip, characterized in that, include: At least one segmented optical neural network as described in any one of claims 1-9, wherein each of the segmented optical neural networks is stacked.

13. The segmented optical neural network chip according to claim 12, characterized in that, The input modules of two adjacent segmented optical neural networks are misaligned; and / or, the output modules of two adjacent segmented optical neural networks are misaligned.

14. An electronic device, characterized in that, Including the segmented optical neural network chip as described in any one of claims 12-13.

Images