Integrated diffraction neural network for matrix calculation and processor
By integrating the input module, diffractive optical transformation module, and output module of the diffractive neural network, the problems of large integration area and high energy consumption of interferometric optical neural networks in large matrix calculations are solved, achieving efficient and low-energy matrix calculations.
Patent Information
- Application Number
- CN202511980277.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-25
- Publication Date
- 2026-03-06
AI Technical Summary
Existing interferometric optical neural networks suffer from problems such as large chip integration area and high energy consumption in large matrix calculations.
An integrated diffractive neural network is used to encode the row or column vectors of the input matrix into input light signals of different wavelengths through the input module, perform optical mapping using a diffractive optical transformation module, and separate and detect the output light signals through the output module to achieve matrix calculation.
It reduces the integration area and energy consumption of matrix calculations, improves the operation speed, has large-scale parallel computing capabilities and high bandwidth, and has a fast operation speed and large information transmission capacity.
Smart Images
Figure CN121615701A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of optical computing, and more particularly to an integrated diffractive neural network and an integrated optical neural network processor for matrix computation. Background Technology
[0002] Unlike electronic computing, optical computing uses light as an information carrier, and achieves data transmission, processing and computation through the physical properties of light and the interaction between light and matter.
[0003] In the field of artificial intelligence, the model training and inference stages of deep learning require a large number of matrix calculations. Current technologies typically use interferometric optical neural networks (IONs) for matrix calculations. However, INNs require extensive use of Mach-Zehnder interferometers (MZIs) and photoelectric detectors (PDs), which leads to problems such as large chip integration area and high power consumption for large-scale matrix calculations. Therefore, there is an urgent need to provide a new optical neural network for matrix calculations. Summary of the Invention
[0004] To address the aforementioned technical problems, embodiments of this application provide an integrated diffractive neural network and an integrated optical neural network processor for matrix calculations, aiming to provide an integrated optical neural network with small area and low power consumption.
[0005] According to one aspect of the embodiments of this application, an integrated diffraction neural network for matrix calculation is disclosed, comprising:
[0006] The input module is configured to encode the row vectors or column vectors of the input matrix into input optical signals, wherein different row vectors or column vectors are encoded into input optical signals with different wavelengths.
[0007] A diffractive optical transformation module is configured as an optical mapping of a transformation operator matrix. The diffractive optical transformation module is used to receive an input optical signal and apply the same optical transformation to the input optical signals of different wavelengths and convert them into output optical signals.
[0008] The output module is used to receive and separate the output optical signals of different wavelengths and guide them to the detector used for signal detection.
[0009] In some embodiments, the diffractive optical transformation module includes a quasi-free diffraction region and at least one diffractive optical element; the diffractive optical element is embedded in the quasi-free diffraction region to divide the diffraction region into at least two sub-quasi-free diffraction regions, the diffractive optical element is used to apply wavefront modulation to the input optical signal; the sub-quasi-free diffraction regions are used to propagate the input optical signal in a quasi-free diffraction manner.
[0010] In some embodiments, the diffractive optical transformation module further includes a nonlinear transformation layer, which is used to nonlinearly control the light wavefront. The nonlinear transformation layer is embedded in the quasi-free diffraction region and is disposed corresponding to at least one of the diffractive optical elements.
[0011] In some embodiments, the diffractive optical element is a metasurface, which is configured as an achromatic metasurface.
[0012] In some embodiments, the metasurface includes micro / nano structures, the total number of which is greater than or equal to the total number of elements in the mathematical form of the transformation operator matrix.
[0013] In some embodiments, the metasurface is further configured as a refractive index-tunable metasurface to reconstruct the transformation operator matrix.
[0014] In some embodiments, the metasurface includes micro / nanostructures made of non-volatile phase change materials to configure the metasurface as the refractive index tunable metasurface.
[0015] In some embodiments, the input optical signals of different wavelengths pass through the diffractive optical conversion module in parallel at the same time.
[0016] In some embodiments, the integrated diffraction neural network for matrix computation is used to perform convolution, matrix addition, matrix subtraction, matrix multiplication, or matrix division operations on the input matrix and the transformation operator matrix.
[0017] A second aspect of this application provides an integrated optical neural network processor, which includes the integrated diffraction neural network for matrix calculation as described in any of the preceding claims.
[0018] The integrated diffractive neural network for matrix computation provided in this application includes an input module configured to encode the row vectors or column vectors of an input matrix into input optical signals, wherein different row vectors or column vectors are encoded into input optical signals with different wavelengths; a diffractive optical transformation module configured to perform optical mapping of the transformation operator matrix, which receives the input optical signals and applies the same optical transformation to the input optical signals of different wavelengths, converting them into output optical signals; and an output module for receiving and separating output optical signals of different wavelengths and guiding them to a detector for signal detection. Because the input module, diffractive optical transformation module, and output module in the integrated diffractive neural network for matrix computation provided in this application are easily integrated, it is beneficial to reduce the minimum integration area of the integrated diffractive neural network for matrix computation. The matrix computation process is passive, which allows the integrated diffractive neural network for matrix computation provided in this application to significantly reduce the system energy consumption of large-scale matrix computation. Furthermore, the computation process is characterized by pure optical behavior, proceeding at the speed of light and not limited by the frequency of electronic devices. This gives the integrated diffraction neural network used for matrix computation the advantages of fast computation speed, large bandwidth and information transmission capacity, and large-scale parallel computing. Attached Figure Description
[0019] The above and other objectives, features and advantages of this application will become more apparent from a detailed description of exemplary embodiments thereof with reference to the accompanying drawings.
[0020] Figure 1 A schematic diagram of the architecture layout of an integrated diffraction neural network for matrix computation in one embodiment of this application is shown.
[0021] Figure 2 A schematic diagram illustrating the computational principle of an integrated diffraction neural network for matrix calculation in one embodiment of this application is shown.
[0022] Figure 3 A schematic diagram of the computational principle of an integrated diffractive neural network for matrix calculation in one embodiment of this application is shown, wherein the diffractive optical element of the integrated diffractive neural network for matrix calculation is a metasurface with tunable refractive index.
[0023] Figure 4 A schematic diagram of the architecture layout of an integrated diffraction neural network for matrix computation in one embodiment of this application is shown.
[0024] Figure 5 A cross-sectional schematic diagram of an integrated diffraction neural network for matrix calculation in one embodiment of this application is shown.
[0025] Figure 6 A schematic diagram of the architecture of a diffractive optical transformation module of an integrated diffractive neural network for matrix calculation is shown in one embodiment of this application.
[0026] Figure 7 The diagram illustrates computational data and results of an integrated diffraction neural network used for matrix calculations in one embodiment of this application.
[0027] Figure 8 A schematic diagram of the architecture of a diffractive optical transformation module of an integrated diffractive neural network for matrix calculation is shown in one embodiment of this application.
[0028] Figure 9 The diagram illustrates computational data and results of an integrated diffraction neural network used for matrix calculations in one embodiment of this application.
[0029] Figure 10 A schematic diagram of the architecture of a diffractive optical transformation module of an integrated diffractive neural network for matrix calculation is shown in one embodiment of this application.
[0030] Figure 11 The diagram illustrates computational data and results of an integrated diffraction neural network used for matrix calculations in one embodiment of this application.
[0031] Figure 12 A schematic diagram of the architecture of a diffractive optical transformation module of an integrated diffractive neural network for matrix calculation is shown in one embodiment of this application.
[0032] Figure 13 The diagram illustrates computational data and results of an integrated diffraction neural network used for matrix calculations in one embodiment of this application.
[0033] Figure Labels
[0034] 100. Integrated diffraction neural networks for matrix computation;
[0035] 10. Input module;
[0036] 20. Diffractive optical transformation module; 210. Quasi-free diffraction region; 2110. Sub-quasi-free diffraction region; 220. Diffractive optical element; 220a. Micro / nano structure; 230. Nonlinear transformation layer; 240. Cover plate; 250. Insulating layer; 260. Substrate;
[0037] 30. Output module;
[0038] S1, Input optical signal; S2, Input optical signal. Detailed Implementation
[0039] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided to make the description of this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The drawings are merely illustrative of this application and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.
[0040] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more exemplary embodiments. Numerous specific details are provided in the following description to give a full understanding of exemplary embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced with one or more specific details omitted, or other modules, components, etc., can be employed. In other instances, well-known structures, methods, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.
[0041] This application provides an integrated diffraction neural network 100 for matrix computation, which performs corresponding matrix calculations on an input matrix. For ease of distinction, the input matrix will be hereinafter referred to as input matrix X. Please refer to... Figure 1 , Figure 1 A schematic diagram of the architecture of an integrated diffractive neural network 100 for matrix calculation is shown. The integrated diffractive neural network 100 for matrix calculation includes an input module 10, a diffractive optical transformation module 20, and an output module 30. The input module 10 is used to input an optical signal S1, the diffractive optical transformation module 20 is used to process the optical signal, and the output module 30 is used to output the optical signal processed by the diffractive optical transformation module 20.
[0042] Input module 10 is configured to input the row vector or column vector X of the input matrix X. i Encoded as input light, different row vectors or column vectors X of the input matrix X. i The encoded input light has different wavelengths. See also... Figure 2 , Figure 2 This illustrates the process of encoding the row vectors of an input matrix X (4 rows and N columns, where N is greater than or equal to 1 and is an integer) into the input signal light. It's important to note that since matrices can be transposed, the transposed matrix X is obtained by transposing the input matrix X. T Such that taking row vectors of the input matrix X and taking the transpose of the matrix X TTaking column vectors makes no essential difference. Therefore, there is no difference between encoding input module 10 by row vectors and encoding by column vectors. For example, according to the rules of matrix multiplication, when calculating the product of input matrix X and transformation operator matrix W, it is necessary to multiply the row vectors of input matrix X with the corresponding column vectors of transformation operator matrix W. However, even if the input matrix X is transposed first and then the column vectors are multiplied with the corresponding column vectors of transformation operator matrix W, the calculated result is still correct.
[0043] Specifically, in combination Figure 2 The process by which input module 10 encodes the row vectors of input matrix X into input light signals is as follows: the row vector X1 corresponding to the first row of input matrix X is encoded into an input light signal S1 with wavelength λ1; the row vector X2 corresponding to the second row of input matrix X is encoded into an input light signal S1 with wavelength λ2; the row vector X3 corresponding to the third row of input matrix X is encoded into an input light signal S1 with wavelength λ3; and the row vector X4 corresponding to the third row of input matrix X is encoded into an input light signal S1 with wavelength λ4, where λ1 ≠ λ2 ≠ λ3 ≠ λ4. It should be noted that... Figure 2 This is for illustrative purposes only and should not be construed as a limitation on the dimensions of the input matrix X.
[0044] The diffractive optical transformation module 20 is a diffractive device, which makes it easy to integrate. The diffractive optical transformation module 20 is configured as an optical mapping of a two-dimensional transformation operator matrix, denoted as the transformation operator matrix W. The diffractive optical transformation module 20 receives the input optical signal S1, which propagates quasi-free diffraction within it. The module applies the same optical transformation to the input optical signal S1. "Applying the same optical transformation" means that the module executes completely consistent physical rules and mathematical mapping relationships for input optical signals S1 of different wavelengths. The optical transformation effect does not change with the wavelength of the input optical signal S1; that is, the response of the module to input optical signals S1 of different wavelengths is consistent, achieving achromatic aberration. After the input optical signal S1 is optically transformed by the diffractive optical transformation module 20, it is converted into the output signal S2. Assume the input matrix X∈C. M×N Transformation operator matrix W∈C N×K Input matrix X, either the row vector or the column vector X. m =[x m,1 ,x m,2 ,……,x m,N (Where m = 1, ..., M) Please refer to... Figure 2 Take any row or column of the input matrix X. i =[x i,1 ,x i,2 ,……,xi,N The input optical signal S1 is encoded and then modulated by the diffractive optical transformation module 20. The output optical field is the output signal S2.
[0045] Since the diffraction-based optical transformation module 20 applies the same optical transformation to input optical signals S1 of different wavelengths, the transformation operator matrix W is consistent for input optical signals S1 of different wavelengths within a certain bandwidth. The row vector or column vector X of the input matrix X... i Input signal light of different wavelengths can be processed by a modulation unit to obtain the matrix calculation result of input matrix X and transformation operator matrix W. The calculated output matrix is denoted as output matrix Y, which is the result of the corresponding matrix calculation performed by input matrix X and transformation operator matrix W. For example, when the integrated diffraction neural network 100 for matrix calculation is configured to calculate matrix multiplication, Y = X × W. For example, when the integrated diffraction neural network 100 for matrix calculation is configured to calculate matrix addition, Y = X + W. Since matrix operations have requirements on the dimension matching and element type compatibility of the matrices involved in the calculation, the diffraction neural network 100 for matrix calculation provided in this application is based on the premise that the matrices involved in the calculation can perform the corresponding matrix calculations. For example, when the integrated diffraction neural network 100 for matrix calculation provided in this application is configured to calculate matrix multiplication, the number of columns of input matrix X is equal to the number of rows of transformation operator matrix W. For example, when the integrated diffraction neural network 100 for matrix computation provided in this application is configured to compute matrix addition or subtraction, the number of columns of the input matrix X is equal to the number of rows and columns of the transformation operator matrix W.
[0046] The output module 30 is used to receive and separate output signals S2 of different wavelengths and guide them to a detector (not shown) for detecting signals. The detector detects the light intensity of the output signal S2 and outputs an electrical signal through photoelectric conversion. The information carried by the output electrical signal is equal to the matrix calculation result of the input matrix X and the transformation operator matrix W.
[0047] Because the input module 10, diffractive optical transformation module 20, and output module 30 of the integrated diffractive neural network 100 for matrix computation provided in this application are easily integrated, it helps to reduce the minimum integration area of the integrated diffractive neural network 100 for matrix computation. The matrix computation process is a passive process, which significantly reduces the system energy consumption for large-scale matrix computation provided in this application. Furthermore, the computation process is purely optical, occurring at the speed of light, and is not limited by the frequency of electronic devices, giving the integrated diffractive neural network 100 for matrix computation the advantages of high computation speed, large bandwidth and information transmission capacity, and large-scale parallel computing capability.
[0048] It is worth mentioning that the detector can be integrated with the input module 10, the diffractive optical conversion module 20 and the output module 30, or the detector can be not integrated with the input module 10, the diffractive optical conversion module 20 and the output module 30.
[0049] Since matrix convolution and division operations can be directly mapped to matrix multiplication in mathematics, the integrated diffractive neural network 100 for matrix calculations provided in this application can perform matrix calculations including, but not limited to, the four arithmetic operations (matrix addition, matrix subtraction, matrix multiplication, and matrix division) and convolution operations. Therefore, by configuring the modulation parameters of the diffractive optical transformation module 20, the integrated diffractive neural network 100 for matrix calculations can be used to perform convolution, matrix addition, matrix subtraction, matrix multiplication, or matrix division operations on the input matrix X and the transformation operator matrix W. For example, when the integrated diffractive neural network 100 for matrix calculations provided in this application is configured to calculate matrix multiplication, the modulation parameters of the diffractive optical transformation module 20 can be configured so that the modulation effect of the diffractive optical transformation module 20 on the input matrix X is equivalent to multiplication, thereby achieving the output matrix Y = X × W. For example, when the diffractive neural network 100 for matrix calculation provided in this application is configured to calculate matrix addition, the modulation parameters of the diffractive optical transformation module 20 can be configured so that the modulation effect of the diffractive optical transformation module 20 on the input matrix X is equivalent to addition, thereby realizing the output matrix Y = X + W.
[0050] In some embodiments, input optical signals S1 of different wavelengths pass through the diffractive optical transformation module 20 in parallel at once, enabling simultaneous operations on the row or column vectors of multiple input matrices X and the transformation operator matrix W, thereby completing matrix calculations of the input matrix X and the transformation operator matrix W. Passing input optical signals S1 of different wavelengths through the diffractive optical transformation module 20 in parallel at once significantly improves the computational speed.
[0051] In some embodiments, input optical signals S1 of different wavelengths sequentially pass through the diffraction optical transformation module 20, which can also complete the matrix calculation of the input matrix X and the transformation operator matrix W. For a large two-dimensional input matrix X, the row vectors or column vectors X of the matrix X can be input one by one in the time dimension. i The corresponding output row vector or column vector Yi is obtained through calculation. After M calculations, the two-dimensional matrix calculation can be completed. Although M calculations are required, each calculation is performed at the speed of light, so the overall calculation time is still extremely short.
[0052] Please see Figure 2In some embodiments, the diffractive optical transformation module 20 includes a quasi-free diffraction region 210 and at least one diffractive optical element 220, which are integrated into an on-chip structure. The diffractive optical element 220 is embedded in the quasi-free diffraction region 210 to divide the quasi-free diffraction region 210 into at least two sub-quasi-free diffraction regions 2110. The diffractive optical elements 220 are spaced apart. Therefore, when the diffractive optical transformation module 20 includes n (n≥1 and is a positive number) diffractive optical elements 220, the diffractive optical transformation module 20 includes n+1 quasi-free diffraction regions 210. The diffractive optical element 220 is used to apply wavefront modulation to the input optical signal S1, and the sub-quasi-free diffraction region 2110 is used to propagate the input optical signal S1 in the form of quasi-free diffraction in the on-chip waveguide structure, so as to realize the optical process of wavefront modulation of nth diffractive optical element 220 and quasi-free diffraction of n+1th order as transformation operator matrix W.
[0053] The complex transmission coefficient T(r) of the diffractive optical element 220 is a function of the spatial position r, and the complex transmission coefficient T(r) modulates the wavefront at r. According to the Rayleigh-Sommerfeld scalar diffraction theory, the light field distribution U on the output plane... out (r′) can be described as:
[0054] U out (r′)=∫∫ S U in (r)*T(r)*h(r',r)d 2 r
[0055] Where h(r',r) is the impulse response function for free space propagation. r and r′ are spatial position vectors, L is spatial propagation distance, i is the imaginary unit, k is the wave vector, λ is the working wavelength, and S is the aperture region of the diffractive optical element 220.
[0056] Diffractive optical elements 220 include, but are not limited to, metasurfaces, diffractive lenses, binary optical elements (BOE), diffraction gratings, holographic optical elements (HOE), etc.
[0057] In some embodiments, the diffractive optical element 220 is a metasurface, which is an on-chip integrated structure. A metasurface is a two-dimensional artificial electromagnetic material capable of precisely controlling the amplitude, phase, polarization, and other characteristics of electromagnetic waves. The metasurface is configured as an achromatic on-chip integrated metasurface, thereby ensuring that the transformation operator matrix W remains consistent for input signal light of different wavelengths within a certain bandwidth. The method for achromatic transformation using a metasurface can be found in existing technologies and will not be elaborated upon here.
[0058] Because the metasurface is configured as an achromatic metasurface, it can support multi-wavelength parallel transmission and wavelength multiplexing, fully unleashing the potential of optical computing in terms of bandwidth and speed. At the same time, the metasurface also has the advantages of small size and easy integration, which enables the integrated diffraction neural network 100 for matrix computation provided in this application to have extremely high integration density and can be designed as a system-on-a-chip.
[0059] The four arithmetic operations and convolution operations of matrices can be implemented using the architecture of the integrated diffractive neural network 100 for matrix computation provided in this application. The following section, in conjunction with metasurfaces, explains the principle of matrix multiplication calculated by vector encoding using the integrated diffractive neural network 100 for matrix computation provided in this application.
[0060] Assume the input matrix X∈C M×N Transformation operator matrix W∈C N×K During matrix multiplication, row vectors or column vectors are encoded into different wavelengths to construct a spectral coding mechanism. This involves encoding the row vectors or column vectors of the input matrix X into different wavelengths. m =[x m,1 ,x m,2 ,……,x m,N (where m = 1, ..., M) are mapped to M orthogonal discrete wavelengths λ. m Above. On the input plane, the complex amplitude distribution U of the total optical field composed of input optical signals S1 of different wavelengths. in (r,λ) can be represented as a linear superposition of different wavelength components:
[0061]
[0062] Among them, P n (r) is the spatial aperture function of the nth input node, δ(λ-λ m ) is the Dirac function, which ensures a strict correspondence between the row vector and the wavelength.
[0063] The complex transmission coefficient T(r,λ) of the metasurface under multiple wavelengths after the light field propagates through the achromatic metasurface. m The approximation is a function of spatial position r, independent of wavelength. This allows for different wavelengths λ... m When light waves pass through the same physical structure, they undergo approximately uniform wavefront modulation. The light field distribution U on the output plane... out (r′,λ) can be described as:
[0064] U out (r',λ m )=∫∫ S U in (r,λ m )*T(r,λ m )*h(r',r;λ m)d 2 r
[0065] Impulse response function h(r', r; λ) m The matrix W is designed as a mapping transformation operator. Due to the achromatic effect of the metasurface, the system is sensitive to each wavelength λ. m It exhibits consistent optical modulation. In the detection plane where the detector is located, the spectrum is spatially separated by a beam-splitting grating or narrowband filter array, and the wavelength λ is extracted at the output port k. m The light intensity, i.e., the element y of the output matrix. m,k :
[0066]
[0067] In the above formula, ∫∫p n (r)*T(r,λ m )*h(r′ k ,r;λ m )d 2 r is any element W of the transformation operator matrix of the physical implementation. n,k Through the above process, the output light field naturally reorganizes all row vector calculation results of X in the spectral dimension:
[0068]
[0069] Ultimately, within one optical propagation time, by utilizing the parallelism of the wavelength dimension, the multiplication operations of the M vectors of the input matrix and the transformation operator matrix W were completed in one go, realizing all-optical two-dimensional matrix multiplication.
[0070] Please see Figures 1 to 3 In some embodiments, the metasurface includes micro / nanostructures 220a, each of which scatters light propagating to its location and propagates it to the next metasurface. The accompanying drawings of this application only show the light modulation effect of one micro / nanostructure 220a. Vertically arranged ellipses in the drawings indicate that other micro / nanostructures 220a have similar light modulation effects but are not shown.
[0071] In the diffractive optical transformation module 20, the total number of micro / nano structures 220a on all metasurfaces is greater than or equal to the total number of elements in the mathematical form of the transformation operator matrix W. The total number of micro / nano structures 220a on all metasurfaces in the integrated diffractive neural network 100 used for matrix computation is the total number of parameters of the integrated diffractive neural network 100 used for matrix computation. The total number of elements in the mathematical form of the transformation operator matrix W refers to the mathematical size of the transformation operator matrix W; for example, if the transformation operator matrix W ∈ C... N×KThe total number of elements in the transformation operator matrix W is N×K. The total number of parameters in the integrated diffraction neural network 100 used for matrix calculations is greater than or equal to the mathematical size of the transformation operator matrix W. The modulation of the redundant micro / nano structure 220a, combined with the transverse optical field coupling of multilayer quasi-free diffraction, enables the integrated diffraction neural network 100 used for matrix calculations to overcome the sparsity limitation of single-layer optical systems and construct a dense matrix mapping with full-rank representation capability. Furthermore, when faced with processing errors or optical path alignment deviations, the integrated diffraction neural network 100 used for matrix calculations can compensate for these errors through redundant parameters, thereby ensuring the high accuracy and robustness of all-optical calculations.
[0072] In some embodiments, the metasurface is further configured as a refractive index-tunable metasurface. When the metasurface in the diffractive optical transformation module 20 is a refractive index-tunable metasurface, the diffractive optical transformation module 20 can be mapped to an arbitrary, real-time reconfigurable transformation operator matrix, that is, the diffractive optical transformation module 20 can reconstruct the transformation operator matrix W into an arbitrary matrix. When the optical properties of the diffractive optical element 220 in the diffractive optical transformation module 20 are not adjustable, the integrated diffractive neural network 100 used for matrix calculation can only perform matrix calculations between the input matrix X and the fixed operator transformation matrix W. When the metasurface in the diffractive optical transformation module 20 is a tunable metasurface, by reconstructing the transformation operator matrix W, the limitation of traditional optical computing networks being only for specific tasks is broken, and any two matrices can perform operations including but not limited to arithmetic operations and convolution operations, and the entire process is completed within the time of a single optical propagation, realizing a high-throughput, low-latency, and universal matrix computing architecture.
[0073] The reconstruction of the transformation operator matrix W can be achieved by using at least one diffractive optical element 220 in the diffractive optical transformation module 20 as a metasurface with adjustable refractive index.
[0074] In some embodiments, a portion of the diffractive optical elements 220 in the diffractive optical transformation module 20 are refractive index tunable metasurfaces to reduce system complexity.
[0075] In some embodiments, all diffractive optical elements 220 in the diffractive optical transformation module 20 are refractive index tunable metasurfaces to improve the system's controllability.
[0076] In some embodiments, the on-chip integrated metasurface is further configured as a refractive index tunable metasurface, and the total number of micro / nano structures 220a of all on-chip integrated metasurfaces in the diffractive optical transformation module 20 is greater than or equal to the total number of elements in the mathematical form of the transformation operator matrix W. Since the integrated diffractive neural network 100 used for matrix calculation has parameter redundancy characteristics, the dynamic mapping of the tunable metasurface enables the diffractive optical transformation module 20 to construct arbitrary transformation operator matrices. The redundancy characteristics can ensure that the target matrix is approximated with extremely high accuracy, thereby meeting the needs of different computing tasks while improving computing accuracy.
[0077] Please see Figure 3 In some embodiments, the diffractive optical element 220 of the diffractive optical transformation module 20 is a metasurface, and the material of the micro / nano structure 220a of the metasurface is a non-volatile phase-change material (PCMs) to configure the metasurface as a refractive index tunable metasurface. Specifically, for any metasurface in the diffractive optical transformation module 20, at least a portion of the micro / nano structure 220a can be configured as a refractive index tunable metasurface by using a non-volatile phase-change material.
[0078] Non-volatile phase change materials can be germanium-antimony-tellurium (Ge₂Sb₂Te₅), antimony triselenate (Sb₂Se₃), etc. When the material of the micro / nano structure 220a is a non-volatile phase change material, the degree of crystallinity of the non-volatile phase change material can be precisely controlled by applying external energy excitation to the micro / nano structure 220a. Figure 3 The different colored micro / nano structures 220a represent the degree of crystallinity of non-volatile phase change materials.
[0079] The phase transition process is accompanied by a significant change in the complex refractive index of the material, enabling continuous or discrete multi-level control of the phase of each micro / nano structure 220a on the metasurface. This endows the metasurface with programmable dynamic control capabilities and allows for real-time reconstruction of the optical transfer function T(r,λ) of the diffractive optical transformation module 20. m This enables real-time reconstruction of the transformation operator matrix W.
[0080] Furthermore, in some embodiments, the external energy excitation applied to the micro / nano structure 220a is an electrical excitation, including but not limited to excitation by an external electric field to control the degree of crystallization of the non-volatile phase change material.
[0081] Furthermore, in some embodiments, the external energy excitation applied to the micro / nano structure 220a is optical excitation, including but not limited to laser pulse writing to control the degree of crystallization of the non-volatile phase change material.
[0082] Please see Figure 4In some embodiments, the diffractive optical transformation module 20 further includes a nonlinear transformation layer 230, which is an on-chip structure, allowing the nonlinear transformation layer 230, the quasi-free diffraction region 210, and the diffractive optical element 220 to be integrated into an on-chip structure. The nonlinear transformation layer 230 is used for nonlinearly controlling the light wavefront. The on-chip integrated nonlinear transformation layer 230 is embedded in the quasi-free diffraction region 210 and is disposed corresponding to at least one diffractive optical element 220. The nonlinear transformation layer 230 is disposed on the incident light side and / or the exit light side of the diffractive optical element 220.
[0083] In some embodiments, the on-chip integrated nonlinear transformation layer 230 is implemented using optoelectronic nonlinear devices, including but not limited to photodetectors, electro-optic modulators, etc.
[0084] In some embodiments, the on-chip integrated nonlinear transformation layer 230 is implemented using all-optical nonlinear materials, including but not limited to heterojunctions, phase change materials, quantum dots, two-dimensional materials, the Kerr effect of the material itself, two-photon absorption, free carrier dispersion or absorption nonlinearity, etc.
[0085] Please Figure 4 Based on the above Figure 5 , Figure 5 This is a partial cross-sectional view of the diffractive optical transformation module 20. (Combined with...) Figure 5In some embodiments, the diffractive optical conversion module 20 further includes a cover plate 240, an insulating layer 250, and a substrate 260. Each sub-quasi-free diffraction region 2110 collectively constitutes the quasi-free diffraction region 210 in the on-chip waveguide structure. The cover plate 240 protects the quasi-free diffraction region 210 and the diffractive optical element 220. The material of the cover plate 240 includes, but is not limited to, quartz glass. The insulating layer 250 restricts light propagation within the quasi-free diffraction region 2110 and the diffractive optical element 220, preventing leakage to the substrate 260. The material of the insulating layer 250 includes, but is not limited to, quartz glass. Each sub-quasi-free diffraction region 2110 collectively constitutes the quasi-free diffraction region 210 in the on-chip waveguide structure. The diffraction region 210 is sandwiched between the cover plate 240 and the insulating layer 250 along the thickness direction. The material of the quasi-free diffraction region 210 includes, but is not limited to, silicon waveguides. Substrate 260 is disposed on the side of insulating layer 250 opposite to cover plate 240. Substrate 260 provides support, and the material of substrate 260 includes, but is not limited to, silicon and quartz glass. The architecture of diffractive optical transformation module 20 is silicon-on-insulator (SOI) architecture. Silicon waveguide devices are compatible with mature complementary metal-oxide-semiconductor (CMOS) manufacturing processes and can be integrated with electronic control units to achieve high-speed data transmission. Currently, matrix multiplication is the most computationally intensive part of artificial intelligence models and large-scale numerical simulations. The integrated diffractive neural network 100 for matrix calculation provided in this application can replace and accelerate high-load matrix operation tasks in traditional electronic processors, significantly reducing electronic system power consumption and improving computational throughput.
[0086] This application provides four integrated diffraction neural networks 100 for matrix calculation in four exemplary embodiments. The integrated diffraction neural networks 100 for matrix calculation provided in each embodiment of this application will be described in detail below.
[0087] Example 1
[0088] The integrated diffraction neural network 100 for matrix calculations provided in Example 1 is an on-chip integrated structure. Please refer to... Figure 6 , Figure 6 The architecture of the diffractive optical transformation module 20 of the integrated diffractive neural network 100 for matrix calculation provided in Embodiment 1 is shown. The diffractive optical transformation module 20 includes four diffractive optical elements 220 and five sub-quasi-free diffraction regions 2110, wherein the diffractive optical elements 220 are metasurfaces. The total characteristic length of each metasurface along the direction perpendicular to the propagation direction of the input optical signal S1 is 400 μm, the thickness of each metasurface along the perpendicular waveguide plane is 220 nm, and the distance between two adjacent metasurfaces is 200 μm.
[0089] Based on the standard silicon-on-insulator (SOI) platform, the integrated diffraction neural network 100 for matrix calculation provided in Example 1 was numerically simulated and implemented in the Python environment. The propagation of the input signal between metasurface layers was modeled and calculated using scalar diffraction theory. Figure 7 (a) is an input matrix X with 20 rows and 20 columns. Figure 7 (b) Given a 20x20 transformation operator matrix W, use the integrated diffraction neural network 100 for matrix computation provided in Example 1 to calculate the product of the input matrix X and the transformation operator matrix W. Figure 7 (c) The output result Y of the integrated diffraction neural network 100 for matrix calculation provided in Example 1. o , Figure 7 (d) is the output Y of the computer calculating the product of the input matrix X and the transformation operator matrix W. e The transformation operator matrix W and the input matrix X are normalized. The output result Y is then compared. o and output result Y e The absolute error is 0.0096 and the root mean square error is 0.0122, indicating that the calculation results of the integrated diffraction neural network 100 for matrix calculation provided in Example 1 have extremely small errors.
[0090] Example 2
[0091] The integrated diffraction neural network 100 for matrix calculations provided in Example 2 is an on-chip integrated structure. Please refer to... Figure 8 , Figure 8 The architecture of the diffractive optical transformation module 20 of the integrated diffractive neural network 100 for matrix calculation provided in Embodiment 2 is illustrated. The diffractive optical transformation module 20 includes five diffractive optical elements 220 and six sub-quasi-free diffraction regions 2110, wherein the diffractive optical elements 220 are metasurfaces. The total characteristic length of each metasurface along the direction perpendicular to the propagation of the input optical signal S1 is 1000 μm, the thickness of each metasurface along the direction perpendicular to the waveguide plane is 220 nm, and the distance between two adjacent metasurfaces is 600 μm.
[0092] Based on the standard silicon-on-insulator (SOI) platform, the integrated diffraction neural network 100 for matrix calculation provided in Example 2 was numerically simulated and implemented in the Python environment. The propagation of the input signal between the layers of the metasurface was modeled and calculated using scalar diffraction theory. Figure 9 (a) is an input matrix X with 32 rows and 32 columns. Figure 9 (b) Given a 32x32 transformation operator matrix W, use the integrated diffraction neural network 100 for matrix computation provided in Example 2 to calculate the product of the input matrix X and the transformation operator matrix W. Figure 9(c) The output result Y of the integrated diffraction neural network 100 for matrix calculation provided in Example 2. o , Figure 9 (d) is the output Y of the computer calculating the product of the input matrix X and the transformation operator matrix W. e The transformation operator matrix W and the input matrix X are normalized. The output result Y is then compared. o and output result Y e The absolute error is 0.0147 and the root mean square error is 0.0185, indicating that the calculation results of the integrated diffraction neural network 100 for matrix calculation provided in Example 2 have extremely small errors.
[0093] Example 3
[0094] The integrated diffraction neural network 100 for matrix calculations provided in Example 3 is an on-chip integrated structure. Please refer to... Figure 10 , Figure 10 The architecture of the diffractive optical transformation module 20 of the integrated diffractive neural network 100 for matrix calculation provided in Embodiment 3 is illustrated. The diffractive optical transformation module 20 includes four diffractive optical elements 220 and five sub-quasi-free diffraction regions 2110, wherein the diffractive optical elements 220 are tunable metasurfaces. The total characteristic length of each tunable metasurface along the direction perpendicular to the propagation direction of the input optical signal S1 is 400 μm, the thickness of each tunable metasurface along the perpendicular waveguide plane is 220 nm, and the distance between two adjacent tunable metasurfaces is 200 μm.
[0095] Based on the standard silicon-on-insulator (SOI) platform, the integrated diffraction neural network 100 for matrix calculation provided in Example 3 was numerically simulated and implemented in the Python environment. The propagation of the input signal between the layers of the tunable metasurface was modeled and calculated using scalar diffraction theory. Figure 11 (a1) is the input matrix X1 with 20 rows and 20 columns. Figure 11 (b1) is a 20x20 transformation operator matrix W1. The product of the input matrix X1 and the transformation operator matrix W1 is calculated using the integrated diffraction neural network 100 for matrix computation provided in Example 3. Figure 11 (c1) is the output result Y of the integrated diffraction neural network 100 for matrix calculation provided in Example 3. o1 , Figure 11 (d1) represents the output Y of the computer calculating the product of the input matrix X1 and the transformation operator matrix W1. e1 The transformation operator matrix W and the input matrix X are normalized. The output result Y is then compared. o1 and output result Y e1The absolute error is 0.0103 and the root mean square error is 0.0149, indicating that the calculation results of the integrated diffraction neural network 100 for matrix calculation provided in Example 3 have extremely small errors.
[0096] The parameters of the diffractive optical transformation module 20 are dynamically controlled by the phase transition process of the phase change material, thereby enabling the construction of arbitrary transformation operator matrices. During the simulation, the phase transition process is simulated by changing the refractive index of the phase change material. By applying external energy excitation to the tunable metasurface, the transformation operator matrix W1 is reconstructed into the transformation operator matrix W2. Figure 11 (a2) is the input matrix X2 with 20 rows and 20 columns. Figure 11 (b2) is a 20x20 transformation operator matrix W2. The product of the input matrix X2 and the transformation operator matrix W2 is calculated using the integrated diffraction neural network 100 for matrix computation provided in Example 3. Figure 11 (c2) is the output result Y of the integrated diffraction neural network 100 for matrix calculation provided in Example 3. o2 , Figure 11 (d2) represents the output Y of the computer calculating the product of the input matrix X2 and the transformation operator matrix W2. e2 The transformation operator matrix W2 and the input matrix X2 are normalized. The output result Y is then compared. o2 and output result Y e2 The absolute error is 0.0129 and the root mean square error is 0.0177, indicating that the calculation results of the integrated diffraction neural network 100 for matrix calculation provided in Example 3 have extremely small errors.
[0097] Example 4
[0098] The integrated diffraction neural network 100 for matrix calculations provided in Example 4 is an on-chip integrated structure. Please refer to... Figure 12 , Figure 12 The architecture of the diffractive optical transformation module 20 of the integrated diffractive neural network 100 for matrix computation provided in Embodiment 4 is illustrated. The diffractive optical transformation module 20 includes five diffractive optical elements 220 and six sub-quasi-free diffraction regions 2110, wherein the diffractive optical elements 220 are tunable metasurfaces. The total characteristic length of each tunable metasurface along the direction perpendicular to the propagation of the input optical signal S1 is 1000 μm, the thickness of each on-chip integrated tunable metasurface along the perpendicular waveguide plane is 220 nm, and the distance between two adjacent tunable metasurfaces is 600 μm.
[0099] Based on the standard silicon-on-insulator (SOI) platform, the integrated diffraction neural network 100 for matrix calculation provided in Example 4 was numerically simulated and implemented in the Python environment. The propagation of the input signal between the layers of the tunable metasurface was modeled and calculated using scalar diffraction theory. Figure 13 (a1) is the input matrix X1 with 32 rows and 32 columns. Figure 13 (b1) is a 32-row, 32-column transformation operator matrix W1. The product of the input matrix X1 and the transformation operator matrix W1 is calculated using the integrated diffraction neural network 100 for matrix computation provided in Example 4. Figure 13 (c1) is the output result Y of the integrated diffraction neural network 100 for matrix calculation provided in Example 4. o1 , Figure 13 (d1) represents the output Y of the computer calculating the product of the input matrix X1 and the transformation operator matrix W1. e1 The transformation operator matrix W and the input matrix X are normalized. The output result Y is then compared. o1 and output result Y e1 The absolute error is 0.0131 and the root mean square error is 0.0152, indicating that the calculation results of the integrated diffraction neural network 100 for matrix calculation provided in Example 4 have extremely small errors.
[0100] The parameters of the diffractive optical transformation module 20 are dynamically controlled by the phase transition process of the phase change material, thereby enabling the construction of arbitrary transformation operator matrices. During the simulation, the phase transition process is simulated by changing the refractive index of the phase change material. By applying external energy excitation to the tunable metasurface, the transformation operator matrix W1 is reconstructed into the transformation operator matrix W2. Figure 13 (a2) is the input matrix X2 with 32 rows and 32 columns. Figure 13 (b2) is a 32x32 transformation operator matrix W2. The product of the input matrix X2 and the transformation operator matrix W2 is calculated using the integrated diffraction neural network 100 for matrix computation provided in Example 4. Figure 13 (c2) is the output result Y of the integrated diffraction neural network 100 for matrix calculation provided in Example 4. o2 , Figure 13 (d2) represents the output Y of the computer calculating the product of the input matrix X2 and the transformation operator matrix W2. e2 The transformation operator matrix W2 and the input matrix X2 are normalized. The output result Y is then compared. o2 and output result Y e2 The absolute error is 0.0169 and the root mean square error is 0.0189, indicating that the calculation results of the integrated diffraction neural network 100 for matrix calculation provided in Example 4 have extremely small errors.
[0101] This application also provides an integrated optical neural network processor, which includes the integrated diffraction neural network 100 for matrix calculation described above. The architecture of the integrated diffraction neural network 100 for matrix calculation is described above and will not be repeated here.
[0102] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the appended claims.
Claims
1. An integrated diffractive neural network for matrix computation, characterized in that, The input module is configured to encode row vectors or column vectors of an input matrix into input light signals, different row vectors or column vectors of the input matrix are encoded into input light signals with different wavelengths; The diffractive optical transformation module is configured to transform an optical mapping of a transformation operator matrix, the diffractive optical transformation module is used to receive input light signals, and the same optical transformation is applied to the input light signals with different wavelengths and is converted into output light signals; The output module is used to receive and separate the output light signals with different wavelengths, and guide them to a detector for detecting signals. The diffractive optical transformation module includes a quasi-free diffraction region and at least one diffractive optical element; the diffractive optical element is embedded in the quasi-free diffraction region to separate the diffraction region into at least two sub quasi-free diffraction regions, and the diffractive optical element is used to apply wavefront modulation to the input light signals; 2. The integrated diffractive neural network for matrix computation of claim 1, wherein, The sub quasi-free diffraction region is used to propagate the input light signals in a quasi-free diffraction form. The diffractive optical transformation module further includes a nonlinear transformation layer for nonlinearly regulating an optical wavefront, and the nonlinear transformation layer is embedded in the quasi-free diffraction region and corresponds to at least one diffractive optical element.
3. The integrated diffractive neural network for matrix computation of claim 2, wherein, The diffractive optical element is a metasurface, and the metasurface is configured as an achromatic metasurface.
4. The integrated diffractive neural network for matrix computation of claim 2, wherein, The metasurface includes micro-nano structures, and the total number of the micro-nano structures is greater than or equal to the total number of elements in a mathematical form of the transformation operator matrix.
5. The integrated diffractive neural network for matrix computation of claim 4, wherein, The metasurface is further configured as a refractive index adjustable metasurface to reconstruct the transformation operator matrix.
6. The integrated diffractive neural network for matrix computation of claim 4, wherein, The metasurface includes micro-nano structures, and the material of the micro-nano structures is a non-volatile phase change material to configure the metasurface as the refractive index adjustable metasurface.
7. The integrated diffractive neural network for matrix computation of claim 6, wherein, The input light signals with different wavelengths pass through the diffractive optical transformation module in parallel at one time.
8. The integrated diffractive neural network for matrix computation according to any one of claims 1-7, wherein, The integrated diffractive neural network for matrix calculation is used to perform convolution operation, matrix addition operation, matrix subtraction operation, matrix multiplication operation or matrix division operation on the input matrix and the transformation operator matrix.
9. The integrated diffractive neural network for matrix computation according to any one of claims 1-7, wherein, The integrated diffractive neural network for matrix calculation in any one of claims 1-9 is included.
10. An integrated optical neural network processor, characterized by