An optical deep neural network chip
By designing a time-domain-based optical deep neural network chip, high-throughput computing of multi-layer optical deep neural networks was achieved, solving the problem that traditional optical computing is difficult to scale and adapt to complex applications, and possessing real-time training and error correction capabilities.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUAZHONG UNIV OF SCI & TECH
- Filing Date
- 2024-06-28
- Publication Date
- 2026-06-26
AI Technical Summary
Existing optical computing chips are difficult to implement multi-layer deep neural networks, and traditional optical computing is difficult to scale up and adapt to complex neural network application scenarios.
Design a time-domain-based optical deep neural network chip to realize the computation of a multi-layer optical neural network by cascading an optical signal input area, an input data modulator area, a convolutional weight modulator area, a nonlinear unit area, a pooling modulator area, a fully connected weight modulator area, a photodetector area, and a time-domain integrator area.
It achieves high-throughput computing of multi-layer optical deep neural networks, supports ultra-large-scale data input, adapts to complex neural network applications, and solves the bottleneck and error problems of traditional optical computing through real-time weight updates.
Smart Images
Figure CN118551818B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of optical computing technology, and more specifically, relates to an optical deep neural network chip. Background Technology
[0002] Optical computing, as a cutting-edge field at the intersection of optics and computational science, has attracted much attention, utilizing the multidimensional properties of light to achieve complex information processing. With the development of technologies such as artificial intelligence, large-scale AI models, and medical imaging, the demand for high-speed, high-dimensional computing is urgent. The parallel processing capabilities of optical computing accelerate the training of large-scale neural networks, improving the processing speed of large-scale AI models and neural networks. The field of optical computing not only drives the advancement of computer technology but also provides enormous potential for scientific research and application innovation, contributing to social progress.
[0003] Most current mainstream optical computing methods only implement the functions of simple single-layer neural networks. Although optical computing based on free space diffraction has achieved a large scale, the development trend of high-speed computing is integration, miniaturization and high energy efficiency. Moreover, optical computing based on free space diffraction is difficult to align between layers in engineering, making it difficult to realize a complete neural network architecture with deep multi-layer networks.
[0004] Current chip-integrated optical neural networks (ONNs) suffer from overly complex weight control circuits, making it difficult to achieve large-scale deep NNNs. While time-domain-based NNNs expand the dimensions of input information and computational networks in the time dimension, enabling large-scale, high-throughput computation, multi-layered deep NNNs are still lacking. Therefore, increasing the number of layers in optical deep NNNs is of significant research importance. Summary of the Invention
[0005] In view of the shortcomings of related technologies, the present invention aims to provide an optical deep neural network chip, which aims to solve the problems of poor scalability and small depth of multiple network layers in optical neural networks.
[0006] To achieve the above objectives, the present invention provides a time-domain-based optical deep neural network chip, comprising:
[0007] Optical signal input area, used to generate optical signals;
[0008] The input data modulator region, serving as the input layer of the optical neural network, is used to modulate and load the computational information from the source data onto the optical signal input from the optical signal input region.
[0009] The convolution weight modulator region, as a convolutional layer of the optical neural network, is used to multiply the optical signals output from the input data modulator region by weight allocation to obtain convolutional data.
[0010] The first optical nonlinear unit region is used to perform nonlinear processing on the convolutional data and nonlinearly activate the optical neural network.
[0011] The pooling modulator region, as a pooling layer of the optical neural network, is used to pool the optical signal with loaded convolution weights output from the first optical nonlinear unit region.
[0012] The fully connected weighted modulator region, as a fully connected layer of the optical neural network, is used to multiply the pooled optical signals by weight allocation.
[0013] The second optical nonlinear unit region is used to perform nonlinear processing on the fully connected optical signal and nonlinearly activate the optical neural network.
[0014] The high-speed photodetector region is used to convert the optical signal output from the optical nonlinear unit region into an electrical signal.
[0015] The time-domain integrator region is used to sum and output the electrical signals output by the high-speed photodetector region at different times in the time domain.
[0016] Optionally, the optical signal input area includes a laser or laser array.
[0017] Optionally, the input data modulator region includes a first modulator, which is used to modulate optical information through interference, resonance, or absorption;
[0018] The convolution weight modulator region includes a second modulator, which is used to assign weights and multiply them by interference, resonance or absorption;
[0019] The pooling modulator region includes a third modulator, which is used to perform pooling through interference, resonance or absorption; the pooling method includes setting a preset weight to filter fixed elements or average pooling, wherein the preset weight is 0, 1 or 1 / N, and N represents the N elements to be pooled.
[0020] The fully connected weighted modulator region includes a fourth modulator, which is used to distribute weights and multiply them by interference, resonance, or absorption.
[0021] Optionally, the first modulator, the second modulator, the third modulator, and the fourth modulator are all the same.
[0022] Optionally, both the first and second optical nonlinear unit regions employ saturation absorption or anti-saturation absorption to nonlinearly activate the optical neural network.
[0023] Optionally, the high-speed photodetector region employs a germanium-silicon photodetector with germanium epitaxial growth on silicon.
[0024] Optionally, the time-domain integrator region is a time-domain integrator based on an analog amplifier. The integration circuit of the analog amplifier is used to sample the current output by the high-speed photodetector region. The capacitor of the analog amplifier is used to accumulate charge for charging. The process from the completion of charging to the completion of discharging of the capacitor is one integration process.
[0025] Optionally, the RC parameters of the integrator circuit of the analog amplifier are used to adjust the integration time.
[0026] Compared with the prior art, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:
[0027] 1. This invention provides a time-domain-based optical deep neural network chip. It constructs a multi-layered optical neural network through modulator cascading, achieving large-scale, high-throughput optical computation via the time dimension. Multiple modulators and optical nonlinear unit regions are cascaded to create a multi-layered deep optical neural network containing convolutional layers, pooling layers, and fully connected layers. Each convolutional and fully connected layer is activated through the optical nonlinear unit region. Compared to existing optical computing chips, this optical deep neural network chip achieves more complete neural network functionality, enabling multi-layered optical deep neural network computation.
[0028] 2. This invention provides a time-domain-based optical deep neural network chip. Compared to traditional optical computing frameworks, which can only handle brief information inputs and small weight matrices, this chip can achieve ultra-large-scale data input through time-domain optical computing. Traditional optical computing uses wavelength or spatial domain accumulation, which is limited by wavelength and space. This application uses time-domain accumulation, with an infinite time dimension, enabling ultra-large-scale data input. The chip has up to millions of hidden layer neurons, making it more adaptable to various complex neural network application scenarios. Furthermore, because the modulator data can be updated in real time, this chip solves the core problem of traditional optical computing, which can only handle single, simple calculations. The chip can also achieve online training, completing weight iteration through real-time weight data writing, overcoming the bottleneck of traditional optical computing, which is difficult to reconstruct and can only perform inference functions. Simultaneously, real-time weight iteration solves the problem of computational errors introduced by process errors in traditional optical computing. Attached Figure Description
[0029] Figure 1 A schematic diagram of the architecture of a time-domain-based optical deep neural network chip provided in an embodiment of the present invention;
[0030] Figure 2 A schematic diagram of a deep neural network architecture provided in an embodiment of the present invention;
[0031] Figure 3A schematic diagram illustrating the multiplication principle of a time-domain-based optical deep neural network chip provided in an embodiment of the present invention;
[0032] Figure 4 A schematic diagram illustrating the computational principle of time-domain-based optical deep neural network chip accumulation provided in an embodiment of the present invention;
[0033] Figure 5 This is a schematic diagram illustrating the computational principle of a time-domain-based optical deep neural network chip provided in an embodiment of the present invention.
[0034] In all the accompanying drawings, the same reference numerals are used to denote the same elements or structures, wherein:
[0035] 1 is the optical signal input area, 2 is the input data modulator area, 3 is the convolution weight modulator area, 4 is the optical nonlinear unit area, 5 is the pooling modulator area, 6 is the fully connected weight modulator area, 7 is the optical nonlinear unit area, 8 is the high-speed photodetector area, and 9 is the time-domain integrator area. Detailed Implementation
[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.
[0037] The following description, in conjunction with a preferred embodiment, illustrates the content involved in the above embodiments.
[0038] A time-domain-based optical deep neural network chip includes the following components connected in sequence:
[0039] Optical signal input area 1 is used to generate optical signals;
[0040] Input data modulator area 2 serves as the input layer of the optical neural network, used to modulate and load the source data computation information onto the optical signal input to optical signal input area 1;
[0041] The convolution weight modulator region 3, as a convolutional layer of the optical neural network, is used to multiply the optical signal output from the input data modulator region 2 by weight allocation to obtain the convolutional data.
[0042] The first optical nonlinear unit region 4 is used to perform nonlinear processing on the convolutional data and nonlinearly activate the optical neural network.
[0043] Pooling modulator region 5, as a pooling layer of optical neural network, is used to pool the optical signal with loaded convolution weights output from the first optical nonlinear unit region 4.
[0044] Fully connected weighted modulator region 6, as a fully connected layer of the optical neural network, is used to multiply the pooled optical signals by weight allocation;
[0045] The second optical nonlinear unit region 7 is used to perform nonlinear processing on the fully connected optical signal and nonlinearly activate the optical neural network.
[0046] The high-speed photodetector region 8 is used to convert the optical signal output from the optical nonlinear unit region 7 into an electrical signal.
[0047] The time-domain integrator region 9 is used to sum and output the electrical signals output by the high-speed photodetector region 8 at different times in the time domain.
[0048] Optionally, the optical signal input area 1 includes a laser or laser array.
[0049] Optionally, the input data modulator region 2 includes a first modulator, which is used to modulate optical information through interference, resonance, or absorption;
[0050] The convolution weight modulator region 3 includes a second modulator, which is used to distribute weights and multiply them by interference, resonance or absorption;
[0051] The pooling modulator region 5 includes a third modulator, which is used to perform pooling through interference, resonance or absorption; the pooling method includes setting a preset weight to filter fixed elements or average pooling, wherein the preset weight is 0, 1 or 1 / N, and N represents the N elements to be pooled.
[0052] The fully connected weighted modulator region 6 includes a fourth modulator for distributing weights and multiplying them through interference, resonance, or absorption.
[0053] The first modulator, second modulator, third modulator, and fourth modulator are all the same and are all used to achieve interference, resonance, or absorption through carrier effect, Pockels effect, or Kerr effect.
[0054] Optionally, both the first optical nonlinear unit region 4 and the second optical nonlinear unit region 7 employ saturation absorption or anti-saturation absorption to nonlinearly activate the optical neural network.
[0055] Optionally, the high-speed photodetector region 8 uses a germanium-silicon photodetector with epitaxial germanium on silicon to convert the optical signal output from the second optical nonlinear unit region 7 into an electrical signal.
[0056] Optionally, the time-domain integrator region 9 is a time-domain integrator based on an analog amplifier. The integration circuit of the analog amplifier is used to sample the current output by the high-speed photodetector region. The capacitor of the analog amplifier is used to accumulate charge for charging. The process from the completion of charging to the completion of discharging of the capacitor is one integration process.
[0057] Furthermore, the RC parameters of the integrator circuit of the analog amplifier are used to adjust the integration time.
[0058] like Figure 1 The diagram shown is a schematic of the architecture of a time-domain-based optical deep neural network chip provided in an embodiment of the present invention. A schematic of the architecture of a multi-layer deep optical neural network is shown below. Figure 2 As shown, the time-domain-based optical deep neural network chip specifically includes a laser, four modulators, two nonlinear units, a photodetector, and a time-domain integrator. The laser outputs laser light, and the optical signal is transmitted to the information loading modulator. External image, voice, and other information are loaded onto the optical signal through intensity modulation. The optical signal loaded with input information enters the convolution weight modulator, which further modulates the intensity of the optical signal, performing multiplication operations on the light intensity. The optical signal continues to propagate forward into the pooling modulator, where pooling is also implemented through intensity modulation. A modulation intensity of 0 indicates that the data is discarded, and a modulation intensity of 1 indicates that the data is retained. If there are N elements in the pooling, setting the pooling weight to 1 / N achieves average pooling. The pooled optical signal continues to propagate forward into the fully connected weight modulator, where the weights of the fully connected layer are also loaded using intensity modulation. The modulated optical signal enters the photodetector and is converted into an electrical signal. The electrical signal enters the time-domain integrator, which charges the capacitor. As time increases, the electrical signal strength at different moments is integrated and accumulated. When the RC parameter setting is met, the signal is released to complete one accumulation cycle. Finally, the image label is determined based on the magnitude of the integrated pulse strength.
[0059] The principle and process of multiplication calculation in time-domain optical deep neural network chips are as follows: Figure 3 As shown, the intensity modulation of the electro-optic modulator is the adjustment of the light intensity of the optical signal. Different voltages correspond to different light intensities. When the light signal from the laser is 1, the first modulator modulates the light signal intensity to 0.2, and the second modulator modulates the already modulated light signal intensity of 0.2 back to the current 0.5, thus completing a multiplication of 0.2 × 0.5 = 0.1. Similarly, when the light signal from the laser is 1, the first modulator modulates the light signal intensity to 0.7, and the second modulator modulates the already modulated light signal intensity of 0.7 back to the current 0.4, thus completing a multiplication of 0.7 × 0.4 = 0.28.
[0060] The principle and process of addition calculation based on time-domain optical deep neural network chips are as follows: Figure 4 As shown, the photodetector performs photoelectric signal conversion. In the time-domain integrator circuit, the electrical signal enters the time-domain integrator, which charges the capacitor. As time increases, the electrical signal strength at different times is integrated and accumulated. When the RC parameter setting is met, the signal is released to complete one accumulation cycle, thus achieving accumulation in the time domain. For example, if the strengths at times t0-t3 are [0.1, 0.1, 0.18, 0.28], after time-domain integration, the strength at times t0-t3 gradually increases, and the accumulated signal is released at the end of time t3.
[0061] The computational principle and process of a time-domain optical deep neural network chip are as follows: Figure 5As shown, the laser outputs laser light, and the optical signal is transmitted to the information loading modulator. The grayscale information of the external image is loaded onto the optical signal through intensity modulation. The intensity of the optical signal from t0 to t3 becomes [0.2, 0.5, 0.3, 0.7], etc. The optical signal loaded with input information enters the convolution weight modulator, which further modulates the intensity of the optical signal. The weight values from t0 to t3 are [0.5, 0.2, 0.6, 0.4]. Multiplication is performed on the optical intensity, and the optical intensity becomes [0.1, 0.1, 0.18, 0.28]. The optical signal continues to propagate forward into the pooling modulator. In order to achieve the pooling function, the time is extended further, and the elements of multiple times [0.1, 0.1, 0.18, 0.28], [0. Inputting values [1, 0.1, 0.18, 0.28], [0.1, 0.1, 0.18, 0.28], and [0.1, 0.1, 0.18, 0.28] into the pooling modulator, the pooling function is also implemented through intensity modulation. The pooling weight is [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1]. A modulation intensity of 0 indicates that the data is discarded, and a modulation intensity of 1 indicates that the data is retained. If there are N elements to be pooled, the pooling weight is set to 1 / N, i.e., [1 / 9, ... The pooled optical signal continues to propagate into the fully connected weighted modulator, where intensity modulation is also applied to the weights of the fully connected layer, resulting in the optical signal becoming [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.1, 0.1, 0.18, 0.28]. The modulated optical signal then enters the photodetector and is converted into an electrical signal. This electrical signal enters the time-domain integrator, where a capacitor is charged. As time progresses, the electrical signal intensities at different times are integrated and accumulated. When the RC parameter setting is met, the signal is released, completing one accumulation cycle and outputting 0.76. Finally, the image label is determined based on the magnitude of the integrated pulse intensity.
[0062] This invention utilizes cascaded modulators (optical nonlinear unit regions) to realize a multi-layer deep optical neural network containing convolutional layers, pooling layers, and fully connected layers. Each convolutional and fully connected layer is activated through the optical nonlinear unit region. Large-scale, high-throughput optical computation is achieved through the time dimension. In the time-domain integration circuit, the charge transferred from the photodetector's current accumulates on the capacitor plates over time, releasing at a certain point, thus achieving accumulation in the time domain. This solves the problems of poor scalability and limited depth in multi-layer optical neural networks. The optical deep neural network chip provided in this application embodiment possesses more complete neural network functionality, enabling multi-layer optical deep neural network computation, and is applicable to optical computing fields such as large-scale artificial intelligence model training, semantic segmentation, image recognition, and medical instrument modeling and imaging.
[0063] The optical deep neural network chip provided in this application, compared to traditional optical computing frameworks which can only handle brief information inputs and small weight matrices, can achieve ultra-large-scale data inputs and up to millions of hidden layer neurons, making it more adaptable to various complex neural network application scenarios. Furthermore, because the modulator data can be updated in real time, this chip solves the core problem of traditional optical computing being limited to single, simple calculations. This chip can also achieve online training, completing weight iterations through real-time weight data writing, overcoming the bottleneck of traditional optical computing's inability to reconstruct and its limitation to inference functions. Simultaneously, real-time weight iterations also address the computational errors introduced by process variations in traditional optical computing.
[0064] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A time-domain-based optical deep neural network chip, characterized in that, Including those connected sequentially: Optical signal input area, used to generate optical signals; The input data modulator region, serving as the input layer of the optical neural network, is used to modulate and load the computational information from the source data onto the optical signal input from the optical signal input region. The convolution weight modulator region, as a convolutional layer of the optical neural network, is used to multiply the optical signals output from the input data modulator region by weight allocation to obtain convolutional data. The first optical nonlinear unit region is used to perform nonlinear processing on the convolutional data and nonlinearly activate the optical neural network. The pooling modulator region, as a pooling layer of the optical neural network, is used to pool the optical signal with loaded convolution weights output from the first optical nonlinear unit region. The fully connected weighted modulator region, as a fully connected layer of the optical neural network, is used to multiply the pooled optical signals by weight allocation. The second optical nonlinear unit region is used to perform nonlinear processing on the fully connected optical signal and nonlinearly activate the optical neural network. The high-speed photodetector region is used to convert the optical signal output from the optical nonlinear unit region into an electrical signal. The time-domain integrator region is used to sum and output the electrical signals output by the high-speed photodetector region at different times in the time domain.
2. The optical deep neural network chip as described in claim 1, characterized in that, The optical signal input area includes a laser or laser array.
3. The optical deep neural network chip as described in claim 1, characterized in that, The input data modulator region includes a first modulator, which is used to modulate optical information through interference, resonance, or absorption. The convolution weight modulator region includes a second modulator, which is used to assign weights and multiply them by interference, resonance or absorption; The pooling modulator region includes a third modulator, which is used to perform pooling through interference, resonance or absorption; the pooling method includes setting a preset weight to filter fixed elements or average pooling, wherein the preset weight is 0, 1 or 1 / N, and N represents the N elements to be pooled. The fully connected weighted modulator region includes a fourth modulator, which is used to distribute weights and multiply them by interference, resonance, or absorption.
4. The optical deep neural network chip as described in claim 3, characterized in that, The first modulator, the second modulator, the third modulator, and the fourth modulator are all the same.
5. The optical deep neural network chip as described in claim 1, characterized in that, Both the first and second optical nonlinear unit regions employ saturated absorption or anti-saturated absorption to nonlinearly activate the optical neural network.
6. The optical deep neural network chip as described in claim 1, characterized in that, The high-speed photodetector region employs a germanium-silicon photodetector with germanium epitaxial growth on silicon.
7. The optical deep neural network chip as described in claim 1, characterized in that, The time-domain integrator region is a time-domain integrator based on an analog amplifier. The integration circuit of the analog amplifier is used to sample the current output by the high-speed photodetector region. The capacitor of the analog amplifier is used to accumulate charge for charging. The process from the completion of charging to the completion of discharging of the capacitor is one integration process.
8. The optical deep neural network chip as described in claim 7, characterized in that, The RC parameters of the integrator circuit of the analog amplifier are used to adjust the integration time.