An optical convolution processor based on weight reuse

Abstract

The present invention belongs to the technical field of optical signal processing, and specifically relates to an optical convolution processor based on weight reuse, comprising: an optical convolution component, wherein the optical convolution component comprises an input module, a convolution module and an output module connected in sequence; the input module comprises a laser array, a wavelength division multiplexer and a splitter; the convolution module comprises an optical operation layer; the output module comprises a standard single-mode optical fiber and a high-speed photodetector. The present invention proposes an optical convolution processor based on weight reuse, which introduces a non-volatile phase change material GST and embeds it into an MRR. By utilizing the non-volatile characteristics of GST and the high contrast characteristics between crystals and amorphous materials, the weight information is effectively mapped to the resonant wavelength of the microring resonator, and a high computing throughput is achieved through the weight reuse technology. At the same time, the parallel computing scale of the processor is increased without increasing the matrix scale, thereby effectively improving the throughput.

Description

Technical Field

[0001] The invention belongs to the technical field of optical signal processing, and in particular relates to an optical convolution processor based on weight reuse. Background Art

[0002] With the rapid development of information technology, digital applications have become part of our daily lives, such as autonomous driving and image recognition. However, the widespread use of these applications has brought new challenges to hardware data processing. In the past few decades, microelectronics technology and the electronic chip industry have been developing continuously in accordance with Moore's Law, but as traditional process technology gradually approaches its limits, electronic chips face some difficult-to-solve bottlenecks in further improving computing speed and reducing power consumption. In convolutional neural networks, a large number of complex convolution calculations account for more than 80% of the computing power of the entire neural network. In order to meet these challenges and improve computing efficiency, photonic neural network processors with optical multiplication as the core are particularly eye-catching. Compared with traditional electrical-based processing methods, optical computing has higher efficiency and speed, and can reduce energy consumption. Therefore, optical convolution processors based on optical multiplication operations are expected to become an important development direction for future data processing hardware, bringing faster and more efficient data processing capabilities to more fields.

[0003] In most neural networks, once the training phase is completed, there is no need to frequently update the trained weight matrix in the inference phase. Therefore, storing the weights in non-volatile phase change materials and performing calculations directly can achieve the integrated optical storage and computing function. This method can significantly reduce the efficiency loss and energy loss caused by the transfer of data between storage and computing.

[0004] In addition, large-scale parallel computing is the key to realizing high-performance optical convolution processors. In traditional convolution calculations, each weight needs to be stored and calculated separately, which leads to a lot of computational redundancy and storage requirements. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides an optical convolution processor based on weight reuse, comprising:

[0006] An optical convolution component, the optical convolution component comprising an input module, a convolution module and an output module connected in sequence;

[0007] The input module includes a laser array, a wavelength division multiplexer and an optical splitter, wherein the laser array is used to generate a plurality of continuous optical signals of different wavelengths, the wavelength division multiplexer is used to multiplex the plurality of continuous optical signals of different wavelengths, and the optical splitter is used to perform optical splitting processing on the optical signal multiplexed by the wavelength division multiplexer;

[0008] The convolution module includes an optical operation layer, which includes an optical calculation unit array, each column of optical calculation units operates on the same multiple resonant wavelengths, and is used to calculate the corresponding optical signals after the splitter is split by the spectrometer, and each optical calculation unit processes multiple resonant wavelengths at the same time to realize the multiplication operation of a weight value and multiple input signals;

[0009] The output module includes a standard single-mode optical fiber and a high-speed photodetector, which can make the relative time displacement between adjacent weighted wavelength copies have a certain progressive delay, matching the data symbol duration, and adding multiple wavelengths at different times to achieve complete convolution calculation.

[0010] Preferably, the optical computing layer is composed of 3×3 optical computing units based on non-volatile phase-change materials GST and MRR, that is, the optical computing units of GST and MRR contain GST materials, and the optical computing units of GST and MRR are passive.

[0011] Preferably, the laser array includes multiple continuous wave laser generators, which generate continuous optical signals of different wavelengths, which are modulated and input into corresponding multiple waveguides respectively, and the modulated signals in the multiple waveguides are multiplexed into the same waveguide through a wavelength division multiplexer, and the multiplexed signals in the same waveguide are used to generate multi-path split signals through a splitter.

[0012] Preferably, the optical operation layer selects the wavelength of each optical signal of the optical splitter through the MRR of each column of the optical calculation unit, and uniformly weights the multiple optical signals at the MRR resonance wavelength.

[0013] Preferably, the output module interleaves the time and wavelength of all inputs, which are detected by a high-speed photoelectric detector to obtain a convolution operation result.

[0014] Preferably, the wavelength division multiplexer processes the continuous optical signal generated by the laser array as follows:

[0015] After being modulated, continuous optical signals of different wavelengths are synthesized into a beam of light through a wavelength division multiplexer, coupled to the first waveguide, coupled to the second waveguide through the first on-chip delay line, and then coupled to the third waveguide through the second on-chip delay line.

[0016] Preferably, the first on-chip delay line and the second on-chip delay line are completely identical, and the delay time is fixed.

[0017] Preferably, the optical operation layer couples the optical signal after wavelength division multiplexing, separates a part of the light into the first waveguide, performs a convolution operation, and after the remaining light passes through the first delay line, couples a part of the light into the second waveguide, performs a convolution operation, and after passing through the second delay line, couples all the remaining light into the third waveguide for a convolution operation.

[0018] Beneficial effects of the present invention:

[0019] The present invention proposes an optical convolution processor based on weight reuse, introduces a non-volatile phase change material GST, and embeds it into the MRR. By utilizing the non-volatile characteristics of GST and the high contrast characteristics between crystals and amorphous materials, the weight information is effectively mapped to the resonant wavelength of the microring resonator. Considering the non-volatile characteristics exhibited by GST, the weight information can be maintained even in the absence of power. Combined with the characteristics of wavelength screening and multi-wavelength resonance of the microring resonator, effective weight reuse is achieved. High computing throughput is achieved through weight reuse technology, and the parallel computing scale of the processor is increased without increasing the matrix scale, thereby effectively improving the throughput. The result obtained by weight reuse generates dispersion through the optical fiber, so that the time and wavelength of all inputs are interleaved, thereby obtaining the convolution calculation result. The present invention is expected to meet the challenges of various data-intensive neural network applications, thereby realizing large-scale integration and low power consumption of optical neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Attached Figure 1 This is a basic principle diagram of an optical convolution processor based on weight reuse according to the present invention;

[0021] Attached Figure 2 A schematic diagram of an optical convolution processor based on weight reuse according to the present invention;

[0022] Attached Figure 3 Schematic diagram of convolution calculation corresponding to the present invention;

[0023] Attached Figure 4 It is a schematic diagram of the transmission curve of the optical computing unit and MRR of the present invention;

[0024] Attached Figure 5 It is a schematic diagram of the coupling splitting area of ​​the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] Various embodiments of the present invention may be presented in the form of a range. It should be understood that the description in the form of a range is only for convenience and simplicity and should not be understood as a rigid limitation on the scope of the present invention; therefore, it should be considered that the range description has specifically disclosed all possible sub-ranges and single values ​​within the range. For example, it should be considered that the range description from 1 to 6 has specifically disclosed sub-ranges, such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6, etc., as well as single numbers within the range, such as 1, 2, 3, 4, 5 and 6, regardless of the range. In addition, whenever a numerical range is indicated in the present invention, it is intended to include any quoted numbers (fractions or integers) within the indicated range. Unless otherwise specified, the various raw materials, reagents, instruments and equipment used in the present invention can be purchased from the market or can be prepared by existing equipment.

[0027] In the present invention, in the absence of any contrary description, the directional words used, such as "upper" and "lower", are specifically the directions of the drawings in the accompanying drawings. In addition, in the present invention, the terms "include", "comprise", etc. refer to "including but not limited to". In the present invention, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. In the present invention, "and / or" describes the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. Wherein A and B can be singular or plural. In the present invention, "at least one" refers to one or more, and "plural" refers to two or more. "At least one", "at least one of the following" or similar expressions refer to any combination of these items, including any combination of singular items or plural items. For example, "at least one of a, b, or c" or "at least one of a, b and c" can both mean: a, b, c, ab, i.e. a and b, ac, bc or abc, where a, b, c can be single or plural, respectively.

[0028] like Figure 1-Figure 5As shown, an embodiment of the present invention provides an optical convolution processor based on weight reuse, including:

[0029] An optical convolution component, the optical convolution component comprising an input module, a convolution module and an output module connected in sequence;

[0030] The input module includes a laser array, a wavelength division multiplexer and an optical splitter, wherein the laser array is used to generate a plurality of continuous optical signals of different wavelengths, the wavelength division multiplexer is used to multiplex the plurality of continuous optical signals of different wavelengths, and the optical splitter is used to perform optical splitting processing on the optical signal multiplexed by the wavelength division multiplexer;

[0031] The convolution module includes an optical operation layer, which includes an optical calculation unit array, each column of optical calculation units operates on the same multiple resonant wavelengths, and is used to calculate the corresponding optical signals after the splitter is split by the spectrometer, and each optical calculation unit processes multiple resonant wavelengths at the same time to realize the multiplication operation of a weight value and multiple input signals;

[0032] The output module includes a standard single-mode optical fiber and a high-speed photodetector, which can make the relative time displacement between adjacent weighted wavelength copies have a certain progressive delay, matching the data symbol duration, and adding multiple wavelengths at different times to achieve complete convolution calculation.

[0033] Specifically: Figure 2 The input module shown in (a) contains a laser array consisting of 9 continuous wave laser generators, where the lasers generate continuous optical signals of different wavelengths, which are input into 9 waveguides respectively. The signals are multiplexed into the same waveguide by a 9×1 wavelength multiplexer to realize wavelength division multiplexing signal transmission, and are evenly coupled into three waveguides through three coupling parts for convolution operation.

[0034] As mentioned above, in Figure 2 In the convolution module shown in (a), the wavelength division multiplexing optical signal in the waveguide is selected for multi-resonance wavelengths through the through-type MRR, and then the multi-resonance wavelength signal of the non-volatile phase change material is weighted to realize the optical point multiplication operation and the weighted multiplication of the multi-resonance wavelengths. Since both the MRR and the GST are passive, the point multiplication operation process does not consume any energy; in addition, the weighted optical signal is output to the same waveguide through the throught port, thus completing the optical convolution operation;

[0035] As described above, the optical signals passing through the operation layer are combined into one path and then reach the photodetector through the optical fiber. The photodetector is used to detect the optical signal after the operation of the optical operation layer, and finally realize the convolution operation of 3×3 scale. Through weight reuse, it is possible to input twice and obtain the convolution result of four window sliding, realizing efficient and energy-saving optical convolution operation.

[0036] like Figure 2 As shown: the optical computing layer is composed of 3×3 optical computing units based on non-volatile phase-change materials GST and MRR, that is, the optical computing units of MRR contain GST materials, and the optical computing units of non-volatile phase-change materials GST and MRR are both passive.

[0037] Specifically, the size of the laser array is 9, that is, it contains a laser array composed of 9 continuous wave laser generators, and the lasers generate continuous optical signals of different wavelengths. Therefore, the laser array can generate 9 continuous optical signals of different wavelengths, which are represented as λ1, λ2, λ3...λ9 in sequence; the generated 9 continuous optical signals of different wavelengths are modulated, synthesized into a beam of light through a wavelength division multiplexer, and then coupled to the first waveguide through the first on-chip delay line, coupled to the second waveguide, and then coupled to the third waveguide through the second on-chip delay line; the wavelength division multiplexer can perform wavelength division multiplexing on 9 continuous optical signals of different wavelengths, so the modulated 9 optical signals pass through a 9×1 wavelength multiplexer to multiplex the 9 optical signals of different wavelengths into the same waveguide to achieve wavelength division multiplexing.

[0038] like Figure 2 As shown: the laser array includes multiple continuous wave laser generators, which generate continuous optical signals of different wavelengths. After modulation, they are input into corresponding multiple waveguides respectively, and the modulated signals in the multiple waveguides are multiplexed into the same waveguide through a wavelength division multiplexer, and the multiplexed signals in the same waveguide are generated into multi-path split signals through a splitter.

[0039] like Figure 2 As shown: in the optical operation layer, the MRR of each column of the optical calculation unit selects the wavelength of each optical signal of the splitter respectively, and uniformly weights the multiple optical signals at the MRR resonance wavelength.

[0040] Specifically: Figure 2 As shown in (a), the optical computing unit consists of a ring waveguide embedded with GST phase change material and a straight waveguide, which is placed below the ring waveguide. The straight waveguide is used to transmit 9 wavelengths of wavelength division multiplexed optical signals. When the ring waveguide embedded with GST meets the resonance condition with 3 of them, as shown in Figure 2As shown in (b), the horizontal axis represents the wavelength, and the vertical axis represents the transmittance of the through end of the straight-through microring resonator. The optical signals of the three resonant wavelengths will be coupled into the ring waveguide, and after the loss of the ring waveguide and GST, they will be weighted until they are output from the through port. The optical signals that do not meet the resonant wavelength conditions will be directly output from the through port without passing through the ring waveguide, that is, no weighting operation is performed. Based on this principle, the MRR embedded with the GST phase change material can be used to select the wavelength of the wavelength division multiplexing signal and weighted multiplex the multi-resonance wavelengths.

[0041] like Figure 1 and Figure 2 As shown: the output module makes all input times and wavelengths staggered, which are detected by a high-speed photoelectric detector to obtain the convolution operation result.

[0042] like Figure 2 As shown: The method for the wavelength division multiplexer to process the continuous optical signal generated by the laser array is as follows:

[0043] After being modulated, continuous optical signals of different wavelengths are synthesized into a beam of light through a wavelength division multiplexer, coupled to the first waveguide, coupled to the second waveguide through the first on-chip delay line, and then coupled to the third waveguide through the second on-chip delay line.

[0044] Specifically, the wavelength division multiplexing optical signal is coupled, a part of the light is separated into the first waveguide, and a convolution operation is performed. After the remaining light passes through the first delay line, a part of the light is coupled into the second waveguide, and a convolution operation is performed. After passing through the second delay line, all the remaining light is coupled into the third waveguide, and a convolution operation is performed. The weights corresponding to the three convolution operations are different. Figure 2 As shown in (b), each optical computing unit operates at three different resonant wavelengths for storage calculations. Each row of optical computing units operates at different nine wavelengths, which are used to couple the corresponding optical signals into the waveguide. Therefore, a 3×3 optical computing unit array can perform 3 (3×3) multiplication operations simultaneously. By utilizing the characteristics of MRR multi-wavelength resonance, three wavelengths are weighted on one MRR at the same time, increasing the throughput of the operation by three times.

[0045] The optical computing layer is composed of 3×3 optical computing units based on non-volatile phase change materials GST and MRR, and the optical computing unit is composed of MRR containing GST material; the non-volatile phase change material GST can change its crystallization degree by a section of incident laser pulse, thereby changing the transmittance of the MRR to achieve weight storage, and finally, the three optical signals at the resonant wavelength of the MRR pass through the MRR to achieve multiplexing weighted operation of the optical signal. In the optical computing unit, each optical computing unit includes an MRR containing GST material; the MRR containing GST material is mainly used for the selection and multiplexing of the resonant wavelength, and the multiple resonant wavelength signals at the MRR are uniformly weighted to achieve weighted multiplexing of multiple resonant optical signals. Among them, MRR and GST are both passive, so the corresponding point multiplication operation process can consume no energy.

[0046] The first on-chip delay line and the second on-chip delay line are completely identical, and the delay time is fixed.

[0047] like Figure 3 As shown: the optical operation layer couples the optical signal after wavelength division multiplexing, separates a part of the light into the first waveguide, performs a convolution operation, and after the remaining light passes through the first delay line, couples a part of the light into the second waveguide, performs a convolution operation, and after passing through the second delay line, couples all the remaining light into the third waveguide for a convolution operation.

[0048] Specifically: Convolution calculation: The first input is a1, a2, a3....a9, corresponding to wavelengths λ1, λ2, λ3....λ9, and the convolution kernel is W 11 , W 12 , W 13 ....W 19 , the input and the corresponding convolution kernel are multiplied to obtain the corresponding convolution result. The input passes through the delay line, which can make the input at different times output at the same time. The convolution result will produce dispersion through the optical fiber, so that the relative time displacement between adjacent weighted wavelengths has the same asymptotic delay, which matches the delay time of the delay line. A high-speed photodetector is used for detection at a specific time. At time T1, the first detection result is a1W 11 +a4W 21 +a7W 31 At time T2, the detection result is a2W 12 +a5W 22 +a8W 32 At time T3, the detection result is a3W 13 +a5W 23 +a9W 33 , the detection results at three moments are added to get a complete convolution result. The second input is a 10 , a11 , a 12 ....a 18 , the corresponding wavelengths are λ1, λ2, λ3....λ9. After the same process as above, the detection results at the three moments are a4W respectively. 1l +a7W 21 +a 10 W 31 ,a5W 12 +a8W 22 +a 11 W 32 and a6W 13 +a9W 23 +a 12 W 33 , which corresponds to the convolution result of the second sliding window, that is, the input is a4, a5, a6....a 12 , the corresponding convolution result. Continue with photoelectric detection, and the three results obtained are a7W 11 +a 10 W 21 +a 13 W 31 ,a8W 12 +a 11 W 22 +a 14 W 32 and a9W 13 +

[0049] a 12 W 23 +a 15 W 33 It corresponds to the convolution result of the third sliding window, that is, the input is a7, a8, a9....a 15 The corresponding convolution result. Then the convolution result of the fourth sliding window will be generated, that is, the input is a 10 , a 11 , a 12 ....a 18 In summary, after weight reuse, it is possible to input twice and obtain the convolution result of four window sliding, which fully utilizes the high speed and parallel processing capability of optical components and realizes efficient and energy-saving optical convolution operation.

[0050] Figure 5 This is a schematic diagram of the coupling and splitting area proposed in the present invention. The light passing through the wavelength division multiplexer is on a straight waveguide, and after three coupling and splitting, it is evenly divided into three waveguides. Figure 5 As shown in (a), the splitting region consists of two parallel straight waveguides. The length of the parallel part is the coupling length, and the splitting ratio is controlled by the coupling length, as shown in Figure 5As shown in (b), different coupling lengths correspond to different splitting ratios, which can be calculated based on Figure 5 (b) Select a fixed splitting ratio.

[0051] The specific principle of using the MRR embedded with GST phase change material for weight reuse as described above is: from the resonance condition of the ring resonator, it can be seen that the output spectrum of the through-end of the straight-through ring resonator is a notch spectrum with a wavelength period. Therefore, there are many resonant wavelengths in one MRR. The above-mentioned non-volatile phase change material GST is used for weight storage and the characteristics of MRR multi-resonance wavelengths, and the multi-resonance wavelengths are multiplied with the weights to realize weight reuse.

[0052] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined in the present invention can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown in the present invention, but should conform to the widest range consistent with the principles and novel features of the present invention.

Claims

1. An optical convolution processor based on weight reuse, characterized in that: include: An optical convolution component, the optical convolution component comprising an input module, a convolution module and an output module connected in sequence; The input module includes a laser array, a wavelength division multiplexer and an optical splitter, wherein the laser array is used to generate a plurality of continuous optical signals of different wavelengths, the wavelength division multiplexer is used to multiplex the plurality of continuous optical signals of different wavelengths, and the optical splitter is used to perform optical splitting processing on the optical signal multiplexed by the wavelength division multiplexer; The convolution module includes an optical operation layer, which includes an optical calculation unit array, each column of optical calculation units operates on the same multiple resonant wavelengths, and is used to calculate the corresponding optical signals after the splitter is split by the spectrometer, and each optical calculation unit processes multiple resonant wavelengths at the same time to realize the multiplication operation of a weight value and multiple input signals; The optical computing layer is composed of 3×3 optical computing units based on non-volatile phase-change materials GST and MRR, that is, the optical computing unit of MRR contains GST material, and the optical computing units of non-volatile phase-change materials GST and MRR are both passive; the optical computing layer selects the wavelength of each optical signal of the splitter through the MRR of each column of optical computing units, and uniformly weights multiple optical signals at the resonance wavelength of this MRR; The output module includes a standard single-mode optical fiber and a high-speed photodetector, which enable the relative time displacement between adjacent weighted wavelength copies to have a certain progressive delay, matching the duration of the data symbol, and adding multiple wavelengths at different times to achieve complete convolution calculation.

2. The optical convolution processor based on weight reuse according to claim 1, characterized in that: The optical computing layer is composed of 3×3 optical computing units based on non-volatile phase-change materials GST and MRR, that is, the optical computing units of GST and MRR contain GST materials, and the optical computing units of GST and MRR are both passive.

3. The optical convolution processor based on weight reuse according to claim 1, characterized in that: The laser array includes multiple continuous wave laser generators, which generate continuous optical signals of different wavelengths. After modulation, the continuous wave laser generators are input into corresponding multiple waveguides respectively. The modulated signals in the multiple waveguides are multiplexed into the same waveguide through a wavelength division multiplexer, and the multiplexed signals in the same waveguide are used to generate multi-path split signals through a splitter.

4. The optical convolution processor based on weight reuse according to claim 1, characterized in that: The optical operation layer selects the wavelength of each optical signal of the optical splitter through the MRR of each column of the optical calculation unit, and uniformly weights multiple optical signals at the MRR resonance wavelength.

5. The optical convolution processor based on weight reuse according to claim 1, characterized in that: The output module interleaves all input times and wavelengths, which are detected by a high-speed photodetector to obtain a convolution operation result.

6. The optical convolution processor based on weight reuse according to claim 1, characterized in that: The method for the wavelength division multiplexer to process the continuous optical signal generated by the laser array is as follows: After being modulated, continuous optical signals of different wavelengths are synthesized into a beam of light through a wavelength division multiplexer, coupled to the first waveguide, coupled to the second waveguide through the first on-chip delay line, and then coupled to the third waveguide through the second on-chip delay line.

7. The optical convolution processor based on weight reuse according to claim 6, characterized in that: The first on-chip delay line and the second on-chip delay line are completely identical, and the delay time is fixed.

8. The optical convolution processor based on weight reuse according to claim 1, characterized in that: The optical operation layer couples the wavelength division multiplexed optical signal, separates a part of the light into the first waveguide, performs a convolution operation, and after the remaining light passes through the first delay line, couples a part of the light into the second waveguide, performs a convolution operation, and after passing through the second delay line, couples all the remaining light into the third waveguide, performs a convolution operation.

Images