Integrated diffraction optical neural network transfer learning chip and its parameter training method

Abstract

The present invention provides an integrated diffraction optical neural network transfer learning chip and parameter training method thereof, comprising an input silicon waveguide, an integrated diffraction optical neural network, a tapered silicon waveguide, an output silicon waveguide, and a small-scale transfer neural network arranged in a waveguide layer; the integrated diffraction optical neural network has a silicon slot array embedded therein; signal light is transmitted to the integrated diffraction optical neural network via the input silicon waveguide, the signal light is phase-delayed and optically diffracted by the silicon slot array in the integrated diffraction optical neural network, and then input into the tapered silicon waveguide; the tapered silicon waveguide performs mode selection on the input signal light, and then inputs into the small-scale transfer neural network via the output silicon waveguide; the small-scale transfer neural network modulates the signal and outputs it. The present invention can modulate the light field while changing its phase and can efficiently process multiple tasks, thereby improving the resource utilization and versatility of the chip.

Description

Technical Field

[0001] The present invention relates to the technical field of integrated optical neural networks, and in particular to an integrated diffraction optical neural network transfer learning chip and a parameter training method thereof. Background Art

[0002] Transfer learning is a machine learning technique that uses the knowledge learned in one task to improve the learning efficiency and effectiveness of other related tasks. It currently plays an important role in computer vision, natural language processing, and generative large models.

[0003] Traditional neural network architectures suffer from high computational energy consumption and near-limited computational speeds. Optical neural network architectures have been proposed to enable neural network computations to run at low energy consumption and high speed. Silicon-based integrated diffractive optical networks, with their high integration density, low energy consumption, and high speed, have become a representative architecture for optical neural networks. However, once the network structure of current integrated diffractive optical neural networks is fixed, it is difficult to change. Typically, a single chip can only handle one task, significantly limiting its application scope and resulting in low resource utilization.

[0004] How to enable silicon-based integrated diffraction optical neural networks to process information in a multi-tasking and efficient manner, and improve the resource utilization and versatility of chips has become a problem that needs to be solved at this stage. Summary of the Invention

[0005] In response to the defects of the existing technology, the present invention provides an integrated diffraction optical neural network transfer learning chip and a parameter training method thereof.

[0006] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] On the one hand, the present invention provides an integrated diffractive optical neural network transfer learning chip, comprising an input silicon waveguide arranged in a waveguide layer, an integrated diffractive optical neural network, a tapered silicon waveguide, an output silicon waveguide, and a small-scale transfer neural network;

[0008] A silicon groove array is embedded in the integrated diffractive optical neural network, and the silicon groove array is filled with a low refractive index optical material;

[0009] The signal light is transmitted to the integrated diffraction optical neural network through the input silicon waveguide. The signal light generates phase delay and optical diffraction through the silicon groove array in the integrated diffraction optical neural network, and then inputs the tapered silicon waveguide. The tapered silicon waveguide selects the mode of the input signal light and then inputs the small-scale migration neural network through the output silicon waveguide. The small-scale migration neural network modulates the signal and outputs it.

[0010] Furthermore, the phase delay is calculated according to the following formula:

[0011] ;

[0012] in, The first The length of the silicon trough unit; For the Phase delay caused by silicon trough unit; is the effective refractive index of the silicon trench unit area; is the effective refractive index of the wide waveguide where the silicon groove array is located; is the wave number of light, , is the wavelength of light.

[0013] Furthermore, the small-scale migration neural network includes at least one of a silicon-based integrated diffraction optical neural network, a computer fully connected neural network, an optical neural network based on a waveguide attenuation modulator, and an on-chip interference neural network based on a Mach-Zehnder interferometer.

[0014] Furthermore, the input silicon waveguide is a multi-input silicon waveguide array composed of multiple silicon waveguides.

[0015] Furthermore, the number of the tapered silicon waveguides and the number of the output silicon waveguides are the same.

[0016] Furthermore, the low refractive index optical material includes silicon dioxide, silicon nitride, and antimony selenide.

[0017] Furthermore, it also includes a photodetector, which is arranged between the output silicon waveguide and the small-scale migration neural network and is used to convert the optical signal into an electrical signal.

[0018] Furthermore, a grating coupler is included, which is arranged before the input silicon waveguide and is used for vertically coupling the optical signal.

[0019] Furthermore, it also includes a signal receiving device, which is arranged after the small-scale migration neural network and is used to receive the output result.

[0020] On the other hand, the present invention also provides a parameter training method for training the above-mentioned integrated diffraction optical neural network transfer learning chip, comprising the following steps:

[0021] S1. Build an integrated diffractive optical neural network transfer learning chip based on the source domain dataset test requirements;

[0022] S2. Inputting the source domain data set into the integrated diffraction optical neural network transfer learning chip for training to determine the optical diffraction parameters and phase delay parameters;

[0023] S3. Build an integrated diffractive optical neural network transfer learning chip based on the target domain dataset test requirements;

[0024] S4. Keep the optical diffraction parameters and phase delay parameters consistent with S2, input the target domain data set into the integrated diffraction optical neural network transfer learning chip to train the small-scale transfer neural network parameters, obtain the trained small-scale transfer neural network parameters, and complete the parameter training of the integrated diffraction optical neural network transfer learning chip.

[0025] Compared with the prior art, the beneficial technical effects of the present invention are:

[0026] The present invention provides an integrated diffraction optical neural network transfer learning chip. The present invention produces different degrees of phase delay on the input light through a silicon groove array embedded in the integrated diffraction optical neural network, and simultaneously causes optical diffraction, thereby modulating the light field while changing the phase; the signal is then selected through a tapered silicon waveguide mode, and after the light of other modes is almost completely lost, it is input into a small-scale transfer neural network for modulation processing; at the same time, the small-scale transfer neural network includes at least one neural network, so that the small-scale transfer neural network part has scalability and adjustability, thereby enhancing the model performance of the small-scale transfer neural network in complex situations. It only needs to modulate the small-scale transfer neural network to adapt to the environment and the influence of errors on the results, and can efficiently process multiple tasks with only a small cost and computing resources consumed, thereby improving the resource utilization and versatility of the chip.

[0027] On the other hand, the present invention also provides a parameter training method for training the parameters of the above-mentioned integrated diffraction optical neural network transfer learning chip. By using a source domain dataset to train the integrated diffraction optical neural network transfer learning chip, the optical diffraction parameters and phase delay parameters obtained from the training are fixed. Then, based on the fixed optical diffraction parameters and phase delay parameters, the small-scale transfer neural network parameters are trained using a target domain dataset to obtain the trained small-scale transfer neural network parameters, thereby completing the parameter training of the integrated diffraction optical neural network transfer learning chip. Through this parameter training, transfer learning is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0029] Figure 1 A schematic diagram of the structure of an integrated diffraction optical neural network transfer learning chip provided by one embodiment;

[0030] Figure 2 A partially enlarged view of a silicon groove array in an integrated diffraction optical neural network transfer learning chip provided in one embodiment;

[0031] Figure 3 A flow chart of a parameter training method for an integrated diffraction optical neural network transfer learning chip provided in one embodiment.

[0032] Figure annotation:

[0033] 1. Input silicon waveguide; 2. Integrated diffraction optical neural network; 3. Silicon groove array; 4. Tapered silicon waveguide; 5. Output silicon waveguide; 6. Small-scale migration neural network. DETAILED DESCRIPTION

[0034] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0035] Reference Figure 1 One embodiment provides an integrated diffractive optical neural network transfer learning chip, comprising an input silicon waveguide 1 arranged in a waveguide layer, an integrated diffractive optical neural network 2, a tapered silicon waveguide 4, an output silicon waveguide 5, and a small-scale transfer neural network 6;

[0036] A silicon groove array 3 is embedded in the integrated diffractive optical neural network 2, and the silicon groove array 3 is filled with a low refractive index optical material;

[0037] The signal light is transmitted to the integrated diffraction optical neural network 2 through the input silicon waveguide 1. The signal light generates phase delay and optical diffraction through the silicon groove array 3 in the integrated diffraction optical neural network 2 and then inputs the tapered silicon waveguide 4. The tapered silicon waveguide 4 selects the mode of the input signal light and then inputs the small-scale migration neural network 6 through the output silicon waveguide 5. The small-scale migration neural network 6 modulates the signal and outputs it.

[0038] Reference Figure 2 By embedding the silicon groove array 3 in the integrated diffraction optical neural network 2, a network structure in the integrated diffraction optical neural network 2 is formed. The optical diffraction process between the silicon groove arrays 3 can be regarded as the connection method between the layers of the neural network; the silicon groove array 3 can be regarded as the hidden layer of the neural network, and the length of the silicon groove in the silicon groove array 3 represents the training parameter of the neural network.

[0039] The phase delay is calculated according to the following formula:

[0040] ;

[0041] in, The first The length of the silicon trough unit; For the Phase delay caused by silicon trough unit; is the effective refractive index of the silicon trench unit area; is the effective refractive index of the wide waveguide where the silicon groove array is located; is the wave number of light, , is the wavelength of light.

[0042] The small-scale transfer neural network 6 includes at least one of a silicon-based integrated diffraction optical neural network, a computer-connected fully connected neural network, an optical neural network based on a waveguide attenuation modulator, and an on-chip interferometric neural network based on a Mach-Zehnder interferometer. By configuring multiple neural networks, multiple tasks can be processed efficiently, improving chip resource utilization and versatility.

[0043] The low-refractive-index optical material includes silicon dioxide, silicon nitride, antimony selenide, and other materials having a refractive index lower than that of silicon and supporting light propagation.

[0044] In a preferred embodiment, the input silicon waveguide 1 is a multi-input silicon waveguide array, consisting of multiple silicon waveguides corresponding to task requirements. The multi-input silicon waveguide array enables the integrated diffractive optical neural network 2 to handle tasks with different feature counts.

[0045] In a preferred embodiment, the number of tapered silicon waveguides 4 and output silicon waveguides 5 is the same. The tapered silicon waveguides 4 and output silicon waveguides 5 constitute a tapered silicon waveguide array and an output silicon waveguide array. The number of tapered silicon waveguides 4 and output silicon waveguides 5 corresponds to the number of features input into the small-scale transfer neural network 6. Depending on the difficulty of the task being processed and the distribution of task features, different numbers of tapered silicon waveguides 4 and output silicon waveguides 5 can be set. A larger number corresponds to a greater number of features input into the small-scale transfer neural network 6, and a stronger transfer learning capability.

[0046] In a preferred embodiment, the integrated diffraction optical neural network transfer learning chip further includes a grating coupler, which is arranged before the input silicon waveguide and is used to vertically couple the optical signal so that the input optical signal is coupled into the silicon waveguide.

[0047] In one embodiment, an integrated diffractive optical neural network transfer learning chip includes an input silicon waveguide 1 arranged in a waveguide layer, an integrated diffractive optical neural network 2, a tapered silicon waveguide 4, an output silicon waveguide 5, a small-scale transfer neural network 6, and a photodetector;

[0048] A silicon groove array 3 is embedded in the integrated diffractive optical neural network 2, and the silicon groove array 3 is filled with a low refractive index optical material;

[0049] The network type of the small-scale transfer neural network 6 is a computer fully connected neural network;

[0050] The signal light is transmitted to the integrated diffraction optical neural network 2 through the input silicon waveguide 1. The signal light generates phase delay and optical diffraction through the silicon groove array 3 in the integrated diffraction optical neural network 2 and then inputs the tapered silicon waveguide 4. The tapered silicon waveguide 4 selects the mode of the input signal light and then inputs the photodetector through the output silicon waveguide 5. The photodetector converts the optical signal into an electrical signal and then inputs it into the computer fully connected neural network. The computer fully connected neural network modulates and processes the electrical signal and then outputs it.

[0051] By setting up a photodetector between the output silicon waveguide 5 and the small-scale migration neural network 6, the optical signal is converted into an electrical signal, thereby adapting to the computer's fully connected neural network.

[0052] In a preferred embodiment, the integrated diffraction optical neural network transfer learning chip further includes a signal receiving device, which is arranged after the small-scale transfer neural network 6 and is used to receive output results.

[0053] Reference Figure 3 In one embodiment, a parameter training method is provided for training the above-mentioned integrated diffraction optical neural network transfer learning chip, comprising the following steps:

[0054] S1. Build an integrated diffractive optical neural network transfer learning chip based on the source domain dataset test requirements;

[0055] S2. Inputting the source domain data set into the integrated diffraction optical neural network transfer learning chip for training to determine the optical diffraction parameters and phase delay parameters;

[0056] S3. Build an integrated diffractive optical neural network transfer learning chip based on the target domain dataset test requirements;

[0057] S4. Keep the optical diffraction parameters and phase delay parameters consistent with S2, input the target domain data set into the integrated diffraction optical neural network transfer learning chip to train the small-scale transfer neural network parameters, obtain the trained small-scale transfer neural network parameters, and complete the parameter training of the integrated diffraction optical neural network transfer learning chip.

[0058] By training an integrated diffraction optical neural network transfer learning chip using a source domain dataset, fixing the optical diffraction parameters and phase delay parameters obtained through training, and then using a target domain dataset to train the small-scale transfer neural network parameters based on these fixed optical diffraction parameters and phase delay parameters, the parameter training of the integrated diffraction optical neural network transfer learning chip is completed. After parameter training is completed, the integrated diffraction optical neural network transfer learning chip can be used to process the target domain dataset.

[0059] Among them, the calculation of the parameters of the small-scale migration neural network is determined by the computing paradigm adopted by the type of its neural network. The silicon-based integrated diffraction optical neural network uses the modified Huygens-Fresnel formula under a certain waveguide structure to perform the forward propagation calculation of the network; the optical neural network based on the waveguide attenuation modulator uses the attenuation coefficient of the attenuator on each waveguide to realize the weight configuration of the neural network, and finally the weighted summation is used to realize the forward propagation; the on-chip interference neural network based on the Mach-Zehnder interferometer uses the interference principle of light to perform forward propagation calculation; the computer fully connected neural network performs forward propagation calculation through weighted summation and activation output.

[0060] Matters not covered by the present invention are known technologies.

[0061] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0062] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and such modifications and improvements are intended to fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

[0063] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. Integrated diffraction optical neural network transfer learning chip, characterized by: It includes an input silicon waveguide, an integrated diffraction optical neural network, a tapered silicon waveguide, an output silicon waveguide, and a small-scale migration neural network arranged in a waveguide layer; The input silicon waveguide is a multi-input silicon waveguide array, consisting of a plurality of silicon waveguides; A silicon groove array is embedded in the integrated diffractive optical neural network, and the silicon groove array is filled with a low refractive index optical material; The signal light is transmitted to the integrated diffraction optical neural network through the input silicon waveguide. The signal light generates phase delay and optical diffraction through the silicon groove array in the integrated diffraction optical neural network, and then inputs the tapered silicon waveguide. The tapered silicon waveguide selects the mode of the input signal light and then inputs the small-scale migration neural network through the output silicon waveguide. The small-scale migration neural network modulates the signal and outputs it.

2. The integrated diffraction optical neural network transfer learning chip according to claim 1, characterized in that: The phase delay is calculated according to the following formula: in, The first i The length of the silicon trough unit; For the i Phase delay caused by silicon trough unit; is the effective refractive index of the silicon trench unit area; is the effective refractive index of the wide waveguide where the silicon groove array is located; is the wave number of light, , is the wavelength of light.

3. The integrated diffraction optical neural network transfer learning chip according to claim 1, characterized in that: The small-scale migration neural network includes at least one of a silicon-based integrated diffraction optical neural network, a computer fully connected neural network, an optical neural network based on a waveguide attenuation modulator, and an on-chip interference neural network based on a Mach-Zehnder interferometer.

4. The integrated diffraction optical neural network transfer learning chip according to claim 1, characterized in that: The number of the tapered silicon waveguides and the number of the output silicon waveguides are the same.

5. The integrated diffraction optical neural network transfer learning chip according to claim 1, characterized in that: The low refractive index optical material includes silicon dioxide, silicon nitride, and antimony selenide.

6. The integrated diffraction optical neural network transfer learning chip according to claim 1, characterized in that: It also includes a photodetector, which is arranged between the output silicon waveguide and the small-scale migration neural network and is used to convert the optical signal into an electrical signal.

7. The integrated diffraction optical neural network transfer learning chip according to claim 1, characterized in that: The invention also includes a grating coupler, which is arranged before the input silicon waveguide and is used for vertically coupling the optical signal.

8. The integrated diffraction optical neural network transfer learning chip according to claim 1, characterized in that: It also includes a signal receiving device, which is arranged after the small-scale migration neural network and is used to receive the output result.

9. A parameter training method, characterized in that: The method for training the integrated diffraction optical neural network transfer learning chip according to any one of claims 1 to 8 comprises the following steps: S1. Build an integrated diffractive optical neural network transfer learning chip based on the source domain dataset test requirements; S2. Inputting the source domain data set into the integrated diffraction optical neural network transfer learning chip for training to determine the optical diffraction parameters and phase delay parameters; S3. Build an integrated diffractive optical neural network transfer learning chip based on the target domain dataset test requirements; S4. Keep the optical diffraction parameters and phase delay parameters consistent with S2, input the target domain data set into the integrated diffraction optical neural network transfer learning chip to train the small-scale transfer neural network parameters, obtain the trained small-scale transfer neural network parameters, and complete the parameter training of the integrated diffraction optical neural network transfer learning chip.

Images