Silicon-based integrated all-optical deep neural network chip, training method thereof and intelligent device
By designing a silicon-based integrated all-optical deep neural network chip, and employing a combination of beam splitters, microcavity arrays, and nonlinear gain units, the problems of superposition loss and nonlinear activation functions in multi-layer cascades were solved, achieving efficient multi-layer cascaded optical matrix calculation and improving network depth and complexity.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUAZHONG UNIV OF SCI & TECH
- Filing Date
- 2024-04-24
- Publication Date
- 2026-06-09
AI Technical Summary
Existing optical neural network chips face the problem of superposition loss when multiple layers are cascaded and cannot implement nonlinear activation functions, which limits the network depth and complexity.
A silicon-based integrated all-optical deep neural network chip was designed, comprising an input layer, an output layer, and multiple fully connected layers. Each layer contains a beam splitter, a microcavity array, and a nonlinear gain unit. The signal is evenly distributed by the beam splitter, weights are loaded by the microcavity array, and optical amplification calculations are performed by the nonlinear gain unit to realize the nonlinear activation function of the multi-layer cascade.
The problem of superposition loss in multi-layer cascades was solved, and the nonlinear activation function of multi-layer cascades was realized, thereby improving signal quality and network depth.
Smart Images
Figure CN118428430B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of optical computing technology, and more specifically, relates to a silicon-based integrated all-optical deep neural network chip and its training method and intelligent device. Background Technology
[0002] Optical computing, as a cutting-edge field at the intersection of optics and computational science, possesses significant scientific and technological background. Its core lies in fully utilizing the multidimensional properties of light to achieve more complex and massive information processing. Neural networks, as a core artificial intelligence technology derived from bionics principles, have profound background significance based on simulating the neural network of the human brain, driving major breakthroughs in fields such as image recognition and speech processing. With the development of applications requiring neural network technology, such as large-scale AI models, autonomous driving, financial trend prediction, weather forecasting, geological disaster prevention, and medical imaging, there is an urgent need for highly parallel, ultra-high-speed, and low-energy-consumption computing platforms. All-optical neural network chips are a cutting-edge artificial intelligence technology that integrates the advantages of optics and neural network computing, bringing revolutionary changes to the fields of computer science and artificial intelligence. The background significance of this technology lies not only in its powerful computing capabilities and efficient energy consumption, but also in its innovation of existing computing models and its guidance for the future development of artificial intelligence.
[0003] Current mainstream optical neural network chips have made significant progress in combining optics and neural network computing. However, they face some limitations in realizing multi-layered cascaded all-optical deep neural networks. On the one hand, when performing multi-layered cascading, the cumulative loss makes it difficult to maintain sufficient signal quality when realizing deep structures, limiting the depth and complexity of the network. On the other hand, since silicon has almost no nonlinear effects, it cannot realize nonlinear activation functions in neural networks, thus it cannot support multi-layered neural network propagation. Therefore, single-layer network structures have become the mainstream choice, and realizing multi-layered cascaded all-optical deep neural networks has become an urgent problem to be solved. Summary of the Invention
[0004] In view of the above-mentioned defects or improvement needs of the existing technology, this application provides a silicon-based integrated all-optical deep neural network chip and its training method and intelligent device, the purpose of which is to solve the superposition loss problem in multi-layer cascading and realize the function of nonlinear activation function.
[0005] To achieve the above objectives, according to one aspect of this application, a silicon-based integrated all-optical deep neural network chip is provided, comprising: an input layer, an output layer, and a plurality of fully connected layers sequentially connected between the input layer and the output layer;
[0006] The input layer includes a polarization controller and an electro-optic modulator array connected in sequence: the polarization controller is used to receive N lasers of different wavelengths and adjust the lasers to a transverse electric mode bias state; the electro-optic modulator array is used to load the N data sources one by one onto different wavelengths and transmit them to the next layer;
[0007] Each fully connected layer and output layer includes a beam splitter, a microcavity array, and a nonlinear gain unit. Within the same layer: the beam splitter divides the wavelength signal output from the previous layer into M equal parts, each part containing N different wavelengths; different rows of the microcavity array are used to weight the different parts of the weighted signal, with the N different wavelengths corresponding to N columns, and each column containing at least N different wavelengths; the nonlinear gain unit receives the weighted signals from each column and performs nonlinear optical amplification calculations on each column to generate regenerated signals of N different wavelengths.
[0008] Each of the fully connected layers also includes a wavelength division multiplexing unit, which in the same layer is used to multiplex all the regenerated signals of the layer onto a waveguide and transmit them to the next layer;
[0009] The output layer also includes a photodetector array, which is used to perform photoelectric conversion on all the regenerated signals of the layer to obtain N inference results.
[0010] The weights of the signal loaded by the microcavity array can be trained, and N and M are both positive integers.
[0011] In one embodiment, it is used in conjunction with a laser for emitting N different wavelengths of laser light onto the silicon-based integrated all-optical deep neural network chip.
[0012] In one embodiment, the electro-optic modulator array includes multiple modulators, each of which is used to load different data sources onto different wavelengths; the modulators are Mach-Zehnder modulators or micro-ring modulators.
[0013] In one embodiment, the microcavity array includes microcavities arranged in an array, wherein the microcavities are microrings or microdisks, and the signal loading weight of each microcavity is adjusted by thermal regulation or electro-optic regulation.
[0014] In one embodiment, the nonlinear gain unit includes multiple heterogeneously integrated semiconductor amplifiers or injection lasers, each of which is used to receive the equally distributed signals loaded in different columns and perform nonlinear optical amplification calculations to generate N regenerated signals of different wavelengths.
[0015] In one embodiment, the wavelength division multiplexing unit is a upload / download type microring resonator.
[0016] In one embodiment, the photodetector array has multiple photodetectors, each photodetector being used to receive the regenerated signals from each column and perform photoelectric conversion. The photodetectors are germanium-silicon photodetectors implemented by epitaxial germanium on a silicon-based wafer.
[0017] According to another aspect of this application, a training method for a silicon-based integrated all-optical deep neural network chip is provided, comprising:
[0018] N different wavelengths of laser light are emitted from a laser onto the silicon-based integrated all-optical deep neural network chip described above;
[0019] The N data sources in the training set are input into the silicon-based integrated all-optical deep neural network chip. Based on the output of the silicon-based integrated all-optical deep neural network chip, the loading weights of each equally distributed signal in each microcavity array in the silicon-based integrated all-optical deep neural network chip are adjusted in reverse. The training is repeated until the training converges.
[0020] In one embodiment, the loading weights of each equally distributed signal in each microcavity array of the silicon-based integrated all-optical deep neural network chip are adjusted in reverse based on the output of the chip, including:
[0021] The output is converted from analog to digital using an analog-to-digital converter and then input into the FPGA.
[0022] After calculating the loss of the output results and labels using FPGA, the weight adjustment value is output.
[0023] The weight adjustment values are converted from digital to analog using a digital-to-analog converter and then input into each microcavity array to adjust the loading weight of each equally distributed signal.
[0024] According to another aspect of this application, an intelligent device is provided for information processing, which includes a silicon-based integrated all-optical deep neural network chip as described in any of the preceding claims.
[0025] In summary, compared with the prior art, the silicon-based integrated all-optical deep neural network chip provided by this application has the following advantages:
[0026] 1. The silicon-based integrated all-optical deep neural network chip provided in this application has an input layer, an output layer, and multiple fully connected layers sequentially connected between the input and output layers. Each fully connected layer and output layer is designed with a beam splitter, a microcavity array, and a nonlinear gain unit. The beam splitter divides light of each wavelength into M equal parts, each part containing N different wavelengths. After being input into the microcavity array, the M*N equally divided optical signals are weighted by the M*N microcavities of the microcavity array and transmitted to the nonlinear gain unit. The nonlinear gain unit performs nonlinear optical amplification calculations on each column of loaded light to generate N regenerated signals of different wavelengths. In the fully connected layer, the N regenerated signals are multiplexed onto a waveguide and transmitted to the next layer. In the output layer, the N regenerated signals are photoelectrically converted to obtain N derivation results. Based on the above structure, multi-layer high-dimensional optical matrix calculations can be realized. Furthermore, a nonlinear gain unit is introduced to perform nonlinear optical amplification calculations on the loaded signal of each column. On the one hand, amplifying the optical signal solves the problem of gradual energy attenuation in multi-layer cascaded structures; on the other hand, the nonlinear calculation also realizes the nonlinear activation function functionality of multi-layer cascaded structures. Therefore, using the chip designed in this application, a multi-level cascaded all-optical deep neural network with nonlinear activation function functionality can be realized, and the superposition loss problem of multi-level cascaded structures is solved.
[0027] 2. The training method for the silicon-based integrated all-optical deep neural network chip provided in this application, by inputting the source data in the training set into the silicon-based integrated all-optical deep neural network chip and training it, can adjust the weight of the signal loaded by the microcavity array, so that the deep neural network can learn the correlation between data, thereby obtaining a trained chip for application to signal classification or recognition. Attached Figure Description
[0028] Figure 1 This is a schematic diagram of the structure of a silicon-based integrated all-optical deep neural network chip according to an embodiment of this application;
[0029] Figure 2 This is a structural diagram of a training platform for a silicon-based integrated all-optical deep neural network chip according to one embodiment of this application. Detailed Implementation
[0030] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. Furthermore, the technical features involved in the various embodiments described below can be combined with each other as long as they do not conflict with each other.
[0031] Example 1
[0032] like Figure 1 The diagram shown is a schematic of the structure of a silicon-based integrated all-optical deep neural network chip in one embodiment of this application. It mainly includes an input layer, an output layer, and multiple fully connected layers connected sequentially between the input layer and the output layer.
[0033] The input layer includes a polarization controller 2 and an electro-optic modulator array 3 connected in sequence. The polarization controller receives N lasers of different wavelengths and adjusts them to a transverse electrical mode bias state for subsequent signal manipulation processing; the N lasers of different wavelengths serve as carriers carrying the data sources to be computed. The electro-optic modulator array loads the N data sources one-to-one onto different wavelengths and transmits them to the next layer; the data sources are image data or voice data for feature extraction by the input network.
[0034] In one embodiment, the electro-optic modulator array includes multiple modulators, each used to load a different data source onto a different wavelength. That is, N different data sources correspond one-to-one with N modulators, and each modulator loads its corresponding data source onto a specific wavelength. Specifically, the modulators can be Mach-Zehnder modulators or micro-ring modulators.
[0035] In one embodiment, the laser source is an off-chip light source, a hybrid integrated light source, a heterogeneous integrated light source, or a monolithic integrated light source. For example, the chip is used in conjunction with the laser, which is used to emit N different wavelengths of laser light to the silicon-based integrated all-optical deep neural network chip as a carrier to carry the data source being computed.
[0036] Each fully connected layer includes a beamsplitter, a microcavity array, a nonlinear gain unit, and a wavelength division multiplexing unit connected in sequence. The output of the previous fully connected layer serves as the input of the next pre-connected layer, and the input of the first fully connected layer serves as the output of the input layer. In each layer: the beamsplitter divides the wavelength signal output from the previous layer into M equal parts, each of which contains N different wavelengths, resulting in a total of M*N equal-divided signals; different rows of the microcavity array are used to weight different parts of the equal-divided signals, with the N different wavelengths respectively loaded into N columns, and each column containing at least N different wavelengths, forming an M*N loading array, with each row containing N different wavelengths, and each class containing M signals, with N different wavelengths among the M signals; the nonlinear gain unit receives the weighted equal-divided signals from each column and performs nonlinear optical amplification calculations on each column to generate regenerated signals of N different wavelengths; the wavelength division multiplexing unit multiplexes all the regenerated signals onto a single waveguide and transmits them to the next layer.
[0037] by Figure 1For example, it shows the specific structure of two fully connected layers. One fully connected layer includes a first beam splitter 4, a first microcavity array 5, a first nonlinear gain unit 6, and a first wavelength division multiplexing unit 7. The other fully connected layer includes a second beam splitter 8, a second microcavity array 9, a second nonlinear gain unit 10, and a second wavelength division multiplexing unit 11.
[0038] The first beam splitter 4 divides the multi-wavelength optical signal modulated by the electro-optic modulator array 3 into M parts, which facilitates wavelength accumulation.
[0039] The first microcavity array 5 applies weights to M modulated multi-wavelength optical signals respectively. Different weights are achieved by adjusting the resonant wavelength of the microrings. Each row of microcavities corresponds to N wavelengths, and each column of microcavities also corresponds to N wavelengths.
[0040] The first nonlinear gain unit 6 accumulates N wavelengths of 1 / M and outputs new N wavelength signals, realizing accumulation, signal regeneration, and nonlinear activation function by utilizing optical nonlinear effects.
[0041] The first wavelength division multiplexing unit 7 multiplexes the newly regenerated N wavelengths onto a bus waveguide for the next layer network to perform calculations.
[0042] The second beam splitter 8 divides the multiplexed multi-wavelength optical signal into M parts, which facilitates the multiplication and wavelength accumulation of the weight matrix of this layer.
[0043] The second microcavity array 9 applies weights to the M multiplexed multi-wavelength optical signals respectively. Different weights are achieved by adjusting the resonant wavelength of the microrings. Each row of microcavities corresponds to N wavelengths, and each column of microcavities corresponds to N wavelengths.
[0044] The second nonlinear gain unit 10 accumulates N regenerated wavelengths of 1 / M and outputs new N wavelength signals again, realizing accumulation, signal regeneration and nonlinear activation function;
[0045] The second wavelength division multiplexing unit 11 multiplexes the newly regenerated N wavelengths onto a bus waveguide for the next layer of the network to perform calculations, thereby completing the hidden layer of the two fully connected neural network layers, and can be further extended to more layers.
[0046] Specifically, the wavelength division multiplexing unit can be a upload / download type microring resonator.
[0047] The output layer comprises a beam splitter, a microcavity array, a nonlinear gain unit, and a photodetector array connected in sequence. The beam splitter divides the wavelength signals output from the last fully connected layer into M equal parts, each of which contains N different wavelengths. Different rows of the microcavity array are used to weight the different equal parts of the signals. The N different wavelengths are respectively loaded into N columns, and each column has at least N different wavelengths. The nonlinear gain unit receives the weighted signals from each column and performs nonlinear optical amplification calculations on each column to generate regenerated signals of N different wavelengths. The photodetector array performs photoelectric conversion on all the regenerated signals of the layer to obtain N inference results.
[0048] Continue with Figure 1 For example, the output layer includes a third beam splitter 12, a third microcavity array 13, a third nonlinear gain unit 14, and a third photodetector array 15.
[0049] The third beam splitter 12 divides the multiplexed N wavelength multi-wavelength optical signal into M parts, which facilitates the multiplication of the output layer weight matrix.
[0050] The third microcavity array 13 applies weights to the M multiplexed multi-wavelength optical signals respectively. Different weights are achieved by adjusting the resonant wavelength of the micro-rings. Each row of microcavities corresponds to N wavelengths, and each column of microcavities corresponds to N wavelengths.
[0051] The third nonlinear gain unit 14 accumulates N regenerated wavelengths of 1 / M and outputs new N wavelength signals again, realizing accumulation, signal regeneration and nonlinear activation function.
[0052] The third photodetector array 15 performs photoelectric conversion on the accumulated signal output by the third nonlinear gain unit 14, and outputs the computational inference result of the neural network. In one embodiment, the photodetector array has multiple photodetectors, each photodetector being used to receive and perform photoelectric conversion on the signals regenerated in each column. Specifically, the photodetector is a germanium-silicon photodetector implemented by epitaxial germanium on a silicon-based wafer.
[0053] In one embodiment, the above microcavity array includes microcavities distributed in an array, wherein the microcavities are microrings or microdisks, and the signal loading weight of each microcavity is adjusted by thermal regulation or electro-optic regulation.
[0054] In one embodiment, the above nonlinear gain unit includes multiple heterogeneously integrated semiconductor optical amplifiers (SOAs) or injection lasers. Each semiconductor optical amplifier or injection laser is used to receive the equally distributed signals loaded in different columns and perform nonlinear optical amplification calculations to generate N regenerated signals of different wavelengths.
[0055] Specifically, the nonlinear calculation formula can be:
[0056]
[0057] Where α is the linewidth enhancement factor, α int Let Γ be the internal loss of the SOA, Γ be the mode field constraint factor, and v be the internal loss of the SOA. g Let A(z,t) be the group velocity, and let A(z,t) be the slowly varying complex amplitude envelope of the electric field, where z represents the position, t represents the time, and g(N) be the material gain of the SOA.
[0058] In one embodiment, the wavelength division multiplexing unit can be a upload / download type microring resonator.
[0059] Example 2
[0060] This embodiment relates to a training method for a silicon-based integrated all-optical deep neural network chip, including:
[0061] N different wavelengths of laser light are emitted from a laser into the silicon-based integrated all-optical deep neural network chip of Example 1;
[0062] The N data sources in the training set are input into the silicon-based integrated all-optical deep neural network chip. Based on the output of the silicon-based integrated all-optical deep neural network chip, the loading weights of each equally distributed signal in each microcavity array in the silicon-based integrated all-optical deep neural network chip are adjusted in reverse. The training is repeated until the training converges.
[0063] like Figure 2 The diagram shows a training platform structure of a silicon-based integrated all-optical deep neural network chip according to an embodiment of this application. It includes a silicon-based integrated all-optical deep neural network chip to be trained, an ADC (Analog-to-Digital Converter), a DAC (Digital-to-Analog Converter), and an FPGA (FPGA-based Programmable Logic Device) to assist training. The ADC converts the analog signal output from the photodetector array into a digital signal to represent the computational results of the neural network. This part can be implemented using a discrete ADC, a monolithic optoelectronic integration, or a co-packaged optoelectronic solution. During inference, the FPGA performs functions such as tag judgment based on the digital signal output from the ADC; during training, it updates weights based on the digital signal output from the ADC. The DAC converts the digital signal of weight updates output from the FPGA into an analog signal to control weight updates, thereby achieving neural network training. This part can also be implemented using a discrete DAC, a monolithic optoelectronic integration, or a co-packaged optoelectronic solution.
[0064] Example 3
[0065] This embodiment also relates to the application of a silicon-based integrated all-optical deep neural network chip, specifically a smart device for information processing, such as image recognition or voice processing. This smart device includes the silicon-based integrated all-optical deep neural network chip as described in Embodiment 1. This chip can be a pre-trained chip as described in Embodiment 2, or it can be an initialized chip that can be trained by the user as described in Embodiment 2. In some specific embodiments, this smart device can be used in various fields such as autonomous driving, weather forecasting, geological disaster prevention, and medical imaging.
[0066] Overall, the silicon-based integrated all-optical deep neural network chip proposed in this application represents a significant innovation in the field of optical neural network chips. By realizing a multi-layered cascaded all-optical deep neural network, it not only solves the current technological challenges but also opens up new possibilities for the development of artificial intelligence. This innovation will not only drive technological progress but also influence the breadth and depth of artificial intelligence applications, paving the way for the arrival of the future intelligent era.
[0067] The technical features of the embodiments described above can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. It should be noted that the terms "in one embodiment," "for example," and "as in another example" in this application are intended to illustrate the application and are not intended to limit the application.
[0068] The embodiments described above are merely examples of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application.
Claims
1. A silicon-based integrated all-optical deep neural network chip, characterized in that: include: An input layer, an output layer, and multiple fully connected layers sequentially connected between the input layer and the output layer; The input layer includes a polarization controller and an electro-optic modulator array connected in sequence: the polarization controller is used to receive N lasers of different wavelengths and adjust the lasers to the bias state of the transverse electric mode; The electro-optic modulator array is used to load N data sources onto different wavelengths in a one-to-one correspondence and transmit them to the next layer; Each fully connected layer and output layer includes a beam splitter, a microcavity array, and a nonlinear gain unit. In the same layer: the beam splitter is used to divide the wavelength signal output from the previous layer into M parts, each part of the signal has N different wavelengths; different rows of the microcavity array are used to weight the different parts of the signal; the N different wavelengths are respectively loaded into N columns, and each column has at least N different wavelengths. The nonlinear gain unit is used to receive the equally distributed signals loaded in each column and perform nonlinear optical amplification calculations on each column to generate N regenerated signals of different wavelengths. Each of the fully connected layers also includes a wavelength division multiplexing unit, which in the same layer is used to multiplex all the regenerated signals of the layer onto a waveguide and transmit them to the next layer; The output layer also includes a photodetector array, which is used to perform photoelectric conversion on all the regenerated signals of the layer to obtain N inference results. The weights of the signal loaded by the microcavity array are trainable, and N and M are both positive integers.
2. The silicon-based integrated all-optical deep neural network chip as described in claim 1, characterized in that: For use in conjunction with a laser, which is used to emit N different wavelengths of laser light to the silicon-based integrated all-optical deep neural network chip.
3. The silicon-based integrated all-optical deep neural network chip as described in claim 1, characterized in that: The electro-optic modulator array includes multiple modulators, each of which is used to load different data sources onto different wavelengths; The modulator is a Mach-Zehnder modulator or a micro-ring modulator.
4. The silicon-based integrated all-optical deep neural network chip as described in claim 1, characterized in that: The microcavity array includes microcavities arranged in an array, wherein the microcavities are microrings or microdisks, and the signal loading weight of each microcavity is adjusted by thermal regulation or electro-optic regulation.
5. The silicon-based integrated all-optical deep neural network chip as described in claim 1, characterized in that: The nonlinear gain unit includes multiple heterogeneously integrated semiconductor amplifiers or injection lasers. Each semiconductor amplifier or injection laser is used to receive the equally distributed signals loaded in different columns and perform nonlinear optical amplification calculations to generate N regenerated signals of different wavelengths.
6. The silicon-based integrated all-optical deep neural network chip as described in claim 1, characterized in that: The wavelength division multiplexing unit is a upload / download type microring resonator.
7. The silicon-based integrated all-optical deep neural network chip as described in claim 1, characterized in that: The photodetector array has multiple photodetectors, each of which is used to receive the regenerated signals from each column and perform photoelectric conversion. The photodetector is a germanium-silicon photodetector implemented by epitaxial germanium on a silicon-based wafer.
8. A training method for a silicon-based integrated all-optical deep neural network chip, characterized in that, include: Using a laser to emit N different wavelengths of laser light onto the silicon-based integrated all-optical deep neural network chip as described in any one of claims 1 to 7; The N data sources in the training set are input into the silicon-based integrated all-optical deep neural network chip. Based on the output of the silicon-based integrated all-optical deep neural network chip, the loading weights of each equally distributed signal in each microcavity array in the silicon-based integrated all-optical deep neural network chip are adjusted in reverse. The training is repeated until the training converges.
9. The training method as described in claim 8, characterized in that, Based on the output of the silicon-based integrated all-optical deep neural network chip, the loading weights of each equally distributed signal in each microcavity array of the silicon-based integrated all-optical deep neural network chip are adjusted in reverse, including: The output is converted from analog to digital using an analog-to-digital converter and then input into the FPGA. After calculating the loss of the output results and labels using FPGA, the weight adjustment value is output. The weight adjustment values are converted from digital to analog using a digital-to-analog converter and then input into each microcavity array to adjust the loading weight of each equally distributed signal.
10. A smart device for information processing, characterized in that, Including the silicon-based integrated all-optical deep neural network chip as described in any one of claims 1 to 7.