Neural network device based on multimode interference and method of operation thereof

Abstract

The present disclosure provides a neural network device based on multi-mode interference, comprising: at least one convolution operation module, configured to perform convolution operation on a to-be-identified object based on a multi-mode interference coupler to obtain a convolution operation result; a data conversion module, configured to convert the convolution operation result into one-dimensional data; at least one fully connected operation module, configured to perform fully connected operation on the one-dimensional data to obtain a fully connected operation result; and a logic operation module, configured to perform logic operation on the fully connected operation result to obtain an identification result. The present disclosure also provides an operation method.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of big data technology and microwave photonics, in particular to a neural network device based on multi-mode interference and an operation method thereof. BACKGROUND

[0002] As a big data operation model that can be widely applied in the fields of signal processing, physics, image processing, artificial intelligence and the like, the neural network needs to rely on high-performance computers to improve the efficiency of data processing. However, the traditional computer based on electricity is facing the bottleneck of bandwidth, delay, energy consumption and the like. Therefore, the fact that the traditional computer has gradually been unable to meet the efficient processing of big data has become a problem to be solved. SUMMARY

[0003] Therefore, in order to overcome at least one aspect of the above problems, the present disclosure provides a neural network device based on multi-mode interference, comprising: at least one convolution operation module, configured to perform convolution operation on a to-be-identified object based on a multi-mode interference coupler to obtain a convolution operation result; a data conversion module, configured to convert the convolution operation result into one-dimensional data; at least one fully connected operation module, configured to perform fully connected operation on the one-dimensional data to obtain a fully connected operation result; and a logic operation module, configured to perform logic operation on the fully connected operation result to obtain an identification result.

[0004] According to the embodiment of the present disclosure, the convolution operation module comprises: a first laser generating module configured to generate first laser; a first data input module configured to modulate the to-be-identified object onto the first laser; a first delay module configured to delay and split the modulated first laser to obtain N beams of second laser; at least one multi-mode interference coupler configured to simultaneously perform multi-mode interference on the N beams of second laser according to a preset convolution matrix to output M groups of optical signals; and a first detector array comprising M photodetectors, the M photodetectors are configured to respectively receive the M groups of optical signals and detect the intensity of the M groups of optical signals to obtain the convolution operation result; wherein M and N are positive integers.

[0005] According to the embodiment of the present disclosure, the preset convolution matrix comprises M convolution kernels; the multi-mode interference coupler is configured to adjust the intensity of the N beams of second laser according to the M convolution kernels respectively to obtain M groups of optical signals; wherein the M convolution kernels correspond to the M groups of optical signals one by one.

[0006] According to an embodiment of the present disclosure, the multi-mode interference coupler comprises N input ports and M output ports; the multi-mode interference coupler is configured to receive the N beams of second laser through the N input ports respectively, and split the N beams of second laser to the M output ports according to N groups of preset splitting ratios respectively to obtain M groups of optical signals; wherein the N groups of preset splitting ratios correspond to the N beams of second laser one by one.

[0007] According to an embodiment of the present disclosure, in the case that the at least one multi-mode interference coupler comprises a plurality of multi-mode interference couplers, the plurality of multi-mode interference couplers are connected in parallel; the multi-mode interference coupler comprises N input waveguides, M output waveguides and a multi-mode interference region; the multi-mode interference region comprises a plurality of multi-mode interference units and a plurality of control units, and the plurality of control units are respectively configured to adjust the refractive index of the plurality of multi-mode interference units.

[0008] According to an embodiment of the present disclosure, the full connection operation module comprises: a second laser generation module configured to generate third laser, the third laser comprising P laser components; a second data input module configured to modulate the convolution operation result onto the third laser; a second delay module configured to delay the P laser components after modulation to obtain fourth laser; a waveform shaper configured to receive the fourth laser, adjust the intensity of the fourth laser according to Q preset judgments, and split the fourth laser after adjustment to obtain Q groups of optical signals; and a second detector array comprising Q balanced photodetectors, the Q balanced photodetectors being configured to detect the intensity of the Q groups of optical signals respectively to obtain the full connection operation result; wherein the P laser components have different wavelengths, the wavelength interval between any two adjacent laser components is equal, and P and Q are positive integers.

[0009] According to the embodiment of the present disclosure, the waveform shaper comprises Q groups of output ports, each group of output ports comprising a first output port and a second output port; the waveform shaper is configured to split the fourth laser to obtain Q groups of optical signals, each group of optical signals comprising P laser components, and adjust the intensities of the P laser components in the Q groups of optical signals according to Q preset decisions respectively to obtain Q groups of intensity-adjusted optical signals; the waveform shaper is further configured to output a first optical signal of one group of optical signals from the first output port of one group of output ports of the Q groups of output ports respectively, and output a second optical signal of the one group of optical signals from the second output port of the one group of output ports of the Q groups of output ports respectively; wherein the one group of adjusted optical signals comprises the first optical signal and the second optical signal, the first optical signal is an optical signal corresponding to a laser component obtained by adjusting according to a negative weight value in a preset decision, and the second optical signal is an optical signal corresponding to a laser component obtained by adjusting according to a non-negative weight value in the preset decision, and the Q preset decisions correspond to the Q groups of optical signals one by one.

[0010] According to the embodiment of the present disclosure, the balanced photodetector comprises two input ports; the balanced photodetector is configured to receive a first and a second optical signal of one group of optical signals through the two input ports respectively; the balanced photodetector is further configured to convert the first and the second optical signal into a first and a second electrical signal respectively, and perform a difference operation on the first and the second electrical signal to obtain a full connection operation result.

[0011] According to the embodiment of the present disclosure, the neural network device further comprises: an electrical amplifier configured to amplify the power of an electrical signal in an electrical link of the neural network device; an optical amplifier configured to amplify the power of a laser in an optical link of the neural network device; and a polarization controller configured to adjust the polarization state of the laser in the optical link of the neural network device.

[0012] The present disclosure provides an operation method, comprising: based on the multi-mode interference-based neural network device in any one of the above, the method comprises: performing convolution operation on a to-be-identified object by at least one convolution operation module based on a multi-mode interference coupler to obtain a convolution operation result; converting the convolution operation result into one-dimensional data by a data conversion module; performing full connection operation on the one-dimensional data by at least one full connection operation module to obtain a full connection operation result; and performing logical operation on the full connection operation result by a logical operation module to obtain an identification result.

[0013] Compared with the prior art, the present disclosure has the following beneficial effects:

[0014] 1. Utilize the advantages of large bandwidth, high speed and low delay of light, design the transmission time of light and the wavelength of light, and perform efficient large data processing through optical transmission.

[0015] 2. Relying on multiple lasers to realize parallel operation of multiple convolution layers and multiple fully connected layers, not only improves the input rate of data, but also makes the data processing process faster.

[0016] 3. The change of laser intensity represents the operation process, and the operation of multiple convolution layers and multiple fully connected layers can be realized by adjusting the number and intensity of lasers, which makes the device have high reconfigurability and scalability. At the same time, multiple operation modules can be cascaded to realize deeper neural network structure, which has strong expandability. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more completely understand the present disclosure and its advantages, the following description will now be made in conjunction with the accompanying drawings, in which:

[0018] Figure 1 A schematic diagram of a neural network device based on multi-mode interference according to an embodiment of the present disclosure is schematically shown;

[0019] Figure 2 A schematic diagram of a convolution operation module according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 3 A schematic diagram of laser delay according to an embodiment of the present disclosure is schematically shown;

[0021] Figure 4 A structural schematic diagram of a multi-mode interference coupler of an embodiment of the present disclosure is schematically shown;

[0022] Figure 5 A structural schematic diagram of a multi-mode interference coupler of another embodiment of the present disclosure is schematically shown;

[0023] Figure 6 A schematic diagram of a convolution operation module according to another embodiment of the present disclosure is schematically shown;

[0024] Figure 7 A schematic diagram of a fully connected operation unit according to an embodiment of the present disclosure is schematically shown; and

[0025] Figure 8 A flowchart of an operation method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the description is only exemplary but not intended to limit the scope of the present disclosure. For those skilled in the field, other drawings can also be obtained from these drawings without creative effort. In the following detailed description, many specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood as the common meanings understood by those skilled in the art.

[0027] The terms used herein are only intended to describe specific embodiments and are not intended to limit the present disclosure. The terms "include", "contain" and the like used herein indicate the existence of the described features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0028] Figure 1 A schematic diagram of a multi-mode interference-based neural network device according to an embodiment of the present disclosure is schematically shown. As shown in the figure, the present disclosure provides a multi-mode interference-based neural network device 100, comprising at least one convolution operation module 1, a data conversion module 2, at least one fully connected operation module 3 and a logical operation module 4. Figure 1

[0029] The convolution operation module 1, the data conversion module 2, the fully connected operation module 3 and the logical operation module 4 are connected in sequence by a cable.

[0030] In the case of implementing more complex operations based on the multi-mode interference-based neural network device 100, the multi-mode interference-based neural network device 100 comprises a plurality of convolution operation modules 1 and a plurality of fully connected operation modules 3 in cascade. The plurality of convolution operation modules 1 and the plurality of fully connected operation modules 3 form a more deep neural network structure.

[0031] The convolution operation module 1 is configured to perform convolution operation on the object to be identified based on a multi-mode interference coupler, to obtain a convolution operation result.

[0032] The data conversion module 2 is configured to convert the convolution operation result into one-dimensional data.

[0033] The convolution operation module 1 obtains a plurality of convolution operation results after performing convolution operation according to a plurality of convolution kernels. The data conversion module 2 first pools the plurality of convolution operation results to reduce the amount of data and reduce the complexity of subsequent fully connected operation; and then converts the pooled convolution operation result into one-dimensional data.

[0034] ​Exemplarily, the convolution operation result is usually an analog signal, and the data conversion module 2 converts the plurality of convolution operation results in parallel from the analog signals into digital signals, and then converts the plurality of digital signals into one-dimensional data, so as to realize the conversion of the data format from parallel to serial, thereby facilitating the full connection operation module 3 to perform the full connection operation.

[0035] The full connection operation module 3 is configured to perform a full connection operation on the one-dimensional data to obtain a full connection operation result.

[0036] The logic operation module 4 is configured to perform a logic operation on the full connection operation result to obtain a recognition result.

[0037] The full connection operation result is usually an analog signal, and the logic operation module 4 converts the full connection operation result from the analog signal into a digital signal, and compares the plurality of full connection results in the form of the digital signal, thereby obtaining the recognition result.

[0038] The object to be recognized includes but is not limited to images, audio, video and text. For example, the object to be recognized is an image, and after the convolution operation and the full connection operation are performed on the complex image, the text and the image in the image are recognized.

[0039] Figure 2 A schematic diagram of the convolution operation module according to an embodiment of the present disclosure is schematically shown. As shown in the figure, Figure 2 The convolution operation module 1 includes a first laser generation module 11, a first data input module 12, a first delay module 13, at least one multimode interference coupler 14 and a first detector array 15.

[0040] The first laser generation module 11, the first data input module 12, the first delay module 13, the at least one multimode interference coupler 14 and the first detector array 15 are sequentially connected through optical fiber jumpers. The convolution operation module 1 further includes an optical amplifier, which can be located at any position in the optical link and is used to amplify the power of the laser in the optical link.

[0041] The first laser generation module 11 is configured to generate a first laser.

[0042] In order to realize the convolution operation on a plurality of convolution kernels, the first laser can include N laser components, and N is a positive integer.

[0043] Exemplarily, the first laser generation module 11 includes N lasers and a beam combiner. The N lasers respectively generate N lasers with different wavelengths, and the beam combiner combines the N lasers with different wavelengths into the first laser. The beam combiner includes but is not limited to an optical coupler, a wavelength division multiplexer, a dense wavelength division multiplexer and an arrayed waveguide grating.

[0044] Exemplarily, the first laser generating module 11 can be a multi-wavelength laser. The multi-wavelength laser generates a laser beam including N laser components with different wavelengths.

[0045] The first data input module 12 is configured to modulate the to-be-recognized object onto the first laser.

[0046] The first data input module 12 includes an arbitrary waveform generator and an electro-optical modulator.

[0047] The first laser generating module 11, the electro-optical modulator and the first delay module 13 are sequentially connected through optical fiber jumpers. The arbitrary waveform generator and the electro-optical modulator are connected through an electric cable.

[0048] The first data input module 12 further includes an electric amplifier. The electric amplifier can be located between the arbitrary waveform generator and the electro-optical modulator, and the electric amplifier is connected to the arbitrary waveform generator and the electro-optical modulator through an electric cable at two ends, respectively, for receiving and amplifying the power of the to-be-processed signal and sending the amplified to-be-processed signal to the electro-optical modulator.

[0049] The arbitrary waveform generator is configured to convert the to-be-recognized object into a to-be-processed signal. The to-be-recognized object includes but is not limited to images, audio, video and text. The arbitrary waveform generator converts the to-be-recognized images, audio, text and the like into to-be-recognized electrical signals.

[0050] Exemplarily, the to-be-recognized object can be converted into a to-be-recognized signal through a programmable pulse generator (PPG), or through a combination of a field programmable gate array (FPGA), a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC) and the like logical operation unit and a digital-to-analog converter.

[0051] The electro-optical modulator is configured to receive the to-be-processed signal and the first laser, and modulate the to-be-processed signal onto the first laser.

[0052] The to-be-processed electrical signal is modulated onto the first laser through the electro-optical modulator as a modulation signal. Understandably, the to-be-recognized object is loaded onto the intensity of the first laser through the electro-optical modulator.

[0053] It should be noted that the intensity of the modulated first laser changes. The intensity of the N laser components included in the modulated first laser all changes, but the intensity of each laser component changes by the same proportion. The relative intensity relationship between the modulated laser components is consistent with the relative intensity relationship between the laser components before modulation.

[0054] The first delay module 13 is configured to delay and split the modulated first laser to obtain N second lasers.

[0055] Since N-bit data to be convolved needs to be input to the multimode interference coupler 14 at the same time when performing convolution operation, and the object to be identified is a serial modulation to the first laser, different degrees of delay need to be performed on the N laser components of the modulated first laser, so that N-bit data to be convolved can be input to the multimode interference coupler 14 at the same time.

[0056] The first delay module 13 includes a beam splitter and at least N-1 optical delay lines. The object to be identified includes N-bit data input continuously, and each bit of data lasts for one input period.

[0057] The beam splitter is used to divide the modulated first laser into N beams to obtain N laser components. The beam splitter divides the modulated first laser into N beams according to wavelength, each wavelength is divided into one beam, and each beam forms a laser component. The beam splitter includes but is not limited to a wavelength division demultiplexer, a dense wavelength division demultiplexer, an arrayed waveguide grating, and a waveform shaper.

[0058] The at least N-1 optical delay lines delay N-1 laser components of the N laser components after beam splitting to obtain N second lasers. Among them, N-1 second lasers of the N second lasers are sequentially delayed for one input period, so that N-bit data is input to the multimode interference coupler 14 at the same time.

[0059] Figure 3 The schematic diagram of laser delay according to the embodiment of the present disclosure is schematically shown.

[0060] As shown in Figure 3 As shown in (A), the first laser generating module 11 generates 4 laser beams Laser1~Laser4. The object to be identified includes data X1~X4, and the input frequency of the object to be identified is f. After passing through the first data input module 12, the laser Laser1~Laser4 all carries the data X1~X4. Since the input frequency of the data is f, the input time of each data is different by one input period T=1 / f.

[0061] If the 4 laser beams are input to the multimode interference coupler 14 at the same time, the multimode interference coupler 14 can only receive data X1 at the same time in the first data input period, and can only receive data X2 at the same time in the second data input period.

[0062] As shown in Figure 3 (B), the first delay module 13 delays the 4 laser beams respectively, so that the multimode interference coupler 14 can receive data X1~X4 at the same time at a certain time. The delay operation includes delaying the laser Laser1 for 3 input periods, delaying the laser Laser2 for 2 input periods, delaying the laser Laser3 for 1 input period, and not delaying the laser Laser4.

[0063] The multi-mode interference coupler 14 is configured to simultaneously perform multi-mode interference on the N beams of the second laser according to a preset convolution matrix, and output M groups of optical signals. The first detector array 15 includes M photodetectors 151, which are configured to respectively receive the M groups of optical signals and detect the intensities of the M groups of optical signals to obtain the convolution operation result.

[0064] The photodetectors 151 convert the received optical signals into electrical signals, so as to convert the convolution operation result to the electrical domain.

[0065] The multi-mode interference coupler 14 includes N input ports and M output ports. The multi-mode interference coupler 14 receives the N beams of the second laser through the N input ports, respectively, and splits the N beams of the second laser to the M output ports according to N groups of preset splitting ratios, respectively, and outputs M groups of optical signals through the M output ports.

[0066] The N groups of preset splitting ratios correspond to the N beams of the second laser one by one. Each beam of the second laser is split to M output ports according to the corresponding splitting ratio, and each output port outputs laser split from the N beams of the second laser.

[0067] For example, the multi-mode interference coupler 14 includes 4 input ports and 5 output ports, and the multi-mode interference coupler 14 receives 4 beams of laser Laser’1-Laser’4 through the 4 input ports. According to 4 groups of preset splitting ratios S 11 , S 15 , S 21 , S 25 , S 31 , S 35 , S 41 , S 45 , the 4 beams of laser Laser’1-Laser’4 are split.

[0068] According to the preset splitting ratios S 11 , S 15 , the laser Laser’1 is split to the 5 output ports according to the preset splitting ratios S 21 , S 25 , the laser Laser’2 is split to the 5 output ports according to the preset splitting ratios S 31 , S 35 , the laser Laser’3 is split to the 5 output ports according to the preset splitting ratios S 41 , S 45 , and the laser Laser’4 is split to the 5 output ports. The 5 output ports output 5 groups of optical signals, and the optical signal output by the first output port includes laser Laser’1-Laser’4 split according to the preset splitting ratios S 11 , S 21 , S 31 , respectively.41 The laser beams split to the first output port, the optical signals output by the second output port include the laser beams Laser'1~Laser'4 respectively according to preset splitting ratios S 12 , S 22 , S 32 and S 42 The laser beams split to the second output port, the optical signals output by the third output port include the laser beams Laser'1~Laser'4 respectively according to preset splitting ratios S 13 , S 23 , S 33 and S 43 The laser beams split to the third output port, the optical signals output by the fourth output port include the laser beams Laser'1~Laser'4 respectively according to preset splitting ratios S 14 , S 24 , S 34 and S 44 The laser beams split to the fourth output port, and the optical signals output by the fifth output port include the laser beams Laser'1~Laser'4 respectively according to preset splitting ratios S 15 , S 25 , S 35 and S 45 The laser beams split to the fifth output port.

[0069] The splitting of the N laser beams according to the preset splitting ratios is essentially a convolution operation of the data to be processed carried by the laser beams according to a preset convolution matrix. The preset convolution matrix includes M convolution kernels. The multimode interference coupler adjusts the intensity of the N second laser beams according to one of the M convolution kernels, respectively, to obtain M groups of optical signals. The M convolution kernels and the M groups of optical signals correspond to each other in one-to-one correspondence.

[0070] For example, the preset convolution matrix is , the convolution kernels are , the data to be processed includes X1~X N , and the output convolution operation result is Y1~Y M .

[0071] The convolution operation process can be represented as formula (1):

[0072] (1)

[0073] The convolution operation result .

[0074] For example, the preset convolution matrix includes 5 convolution kernels C1~C5. The multimode interference coupler 14 adjusts the intensity of the 4 laser beams Laser'1~Laser'4 according to the 5 convolution kernels C1~C5, respectively, to obtain 5 groups of optical signals.

[0075] The intensity of the four lasers Laser'1-Laser'4 is adjusted according to the convolution kernel C1, and the convolution operation is performed on the data to be processed loaded on the lasers Laser'1-Laser'4, and a first group of optical signals Laser''1 is output. Similarly, optical signals Laser''2-Laser''5 are output. Among them, the five convolution kernels correspond to the five groups of optical signals one by one, and each convolution kernel includes four elements.

[0076] It should be noted that the number of elements included in each convolution kernel is the same as the number of input ports of the multimode interference coupler. The convolution kernel has multiple forms. In the above embodiment, the convolution kernel is represented in the form of a one-dimensional transverse vector, and the convolution kernel can also be represented in the form of a multi-dimensional matrix. However, no matter what form the convolution kernel is represented in, the convolution operation process can be represented by the above formula (1). In the case of a multi-dimensional matrix convolution kernel, the multi-dimensional matrix can be converted into a one-dimensional transverse vector.

[0077] Figure 4 The structural schematic diagram of the multimode interference coupler in the embodiment of the present disclosure is schematically shown.

[0078] As shown in Figure 4 , the multimode interference coupler 14 includes N input waveguides 141, M output waveguides 142, and a multimode interference region 143. The N input waveguides 141 and the M output waveguides 142 are respectively installed on both sides of the multimode interference region 143 at a preset interval. Changing the installation position of the input waveguide 141 and the output waveguide 142 can realize the adjustment of the preset convolution matrix. The length L, the width W, and the height H of the multimode interference region 143 can be designed to realize the adjustment of the preset convolution matrix.

[0079] The multimode interference coupler 14 further includes N polarization controllers on the basis of the N input waveguides 141, the M output waveguides 142, and the multimode interference region 143.

[0080] Before the optical signal is input into the input waveguide 141, the optical signal is first input into the polarization controller. The polarization controller controls the polarization state of the laser respectively, so as to realize the adjustment of the convolution matrix.

[0081] The polarization controller can also be located at any position of the optical link, for adjusting the polarization state of the laser in the optical link of the neural network device.

[0082] Figure 5 The structural schematic diagram of the multimode interference coupler in another embodiment of the present disclosure is schematically shown.

[0083] As shown in Figure 5 , the multimode interference region 143 includes a plurality of multimode interference units and a plurality of control elements, and the plurality of control elements are respectively used for adjusting the refractive index of the plurality of multimode interference units. The plurality of multimode interference units and the plurality of control elements correspond to each other respectively.

[0084] In the embodiments of the present disclosure, the method for regulating the multi-mode interference unit includes multiple methods. The electrodes can be made in each multi-mode interference unit, different voltages are applied on the electrodes, and the adjustment of the refractive index is realized by means of carrier injection; the multi-mode interference regions can also be made of phase change materials or ferroelectric materials to realize independent regulation of the refractive index; the thickness, doping concentration and the like of each multi-mode interference unit can also be independently designed to realize independent adjustment of the refractive index of each multi-mode interference unit.

[0085] The present disclosure also provides another embodiment, the first delay module 13 can also be a dispersive medium and a beam splitter. In the case of the first delay module 13 being a dispersive medium and a beam splitter, the first laser generating module 11 generates laser with comb spectrum, which includes N laser components. The wavelengths of each laser component are different, and the wavelength interval between any two laser components adjacent in wavelength is equal.

[0086] For example, the laser generating module 1 generates four laser components Laser1~Laser4, and the wavelengths of the four laser components Laser1~Laser4 are λ1, λ2, λ3 and λ4 respectively. Among them, the wavelength interval between any two laser components adjacent in wavelength is the same. Understandably, the wavelength relationship between λ1, λ2, λ3 and λ4 satisfies Δλ = λ1-λ2 = λ2-λ3 = λ3-λ4.

[0087] The first laser generating module 11 can be a multi-wavelength laser, also can be N lasers, also can be an actively mode-locked laser, an optical frequency comb, a high-speed direct modulation laser, and a combination of a laser and an electro-optic modulator.

[0088] The dispersive medium includes but is not limited to at least one of a dispersion compensation fiber, a chirped fiber grating, a common single-mode fiber and a multi-mode fiber. The dispersive medium delays the N laser components to different degrees.

[0089] The present disclosure provides an exemplary method for realizing the delay of one data input period between any two laser components adjacent in wavelength. But the present disclosure does not limit the specific laser delay method.

[0090] For example, the laser components and the dispersive medium can be arranged to satisfy:

[0091]

[0092] Among them, Δλ is the wavelength interval between any two laser components adjacent in wavelength, f is the input frequency of the data to be processed, D is the dispersion coefficient of the dispersive medium, and is the length of the dispersive medium. Understandably, the delay between any two laser components adjacent in wavelength is .

[0093] It should be noted that in the case of the first delay module 13 being a beam splitter and N-1 optical delay lines, the laser needs to be split into N beams according to the wavelength first, and then delayed by the optical delay lines, and the N beams of delayed laser are input to the multimode interference coupler 14. In the case of the first delay module 13 being a dispersive medium and a beam splitter, the modulated laser is directly subjected to group velocity dispersion, the dispersed laser is split into N beams, and the N beams of laser are input to the multimode interference coupler 14.

[0094] The present disclosure also provides another embodiment, the first laser generation module 11 can be a single-wavelength laser, which generates a single-wavelength laser. The first delay module 13 is a plurality of optical delay lines. Before the optical delay lines delay the laser, the modulated single-wavelength laser needs to be split into N beams by an optical coupler, and the N beams of laser are respectively delayed by the optical delay lines.

[0095] Figure 6 The schematic diagram of the convolution operation module according to another embodiment of the present disclosure is schematically shown.

[0096] As Figure 6 shown, the convolution operation module 1 includes a first laser generation module 11, a first data input module 12, a first delay module 13, and N optical couplers 16, R multimode interference couplers 14, and R first detector arrays 15, and the R multimode interference couplers 14 are connected in parallel.

[0097] The first laser generation module 11, the first data input module 12, the first delay module 13, and the first detector array 15 are described above in the embodiment of the convolution operation module, and will not be repeated here.

[0098] The first delay module 13 delays the N laser components to obtain N beams of second laser, and the N optical couplers 16 split the N beams of second laser into R beams respectively. The R beams of laser split by each optical coupler 16 are input to the R multimode interference couplers 14 respectively, to realize parallel convolution calculation of multiple convolution kernels.

[0099] By the embodiment of the present disclosure, the data is convoluted by laser as a carrier, and the advantages of large bandwidth, high speed and low delay of light are used to improve the operation efficiency. The parallel convolution operation of multiple convolution kernels is performed by using multiple laser components with different wavelengths, which not only improves the input rate of data, but also makes the data processing process faster. In addition, without changing the hardware facilities of the device, only by adjusting the number of laser components and the intensity of the laser components, the number and dimension of the convolution kernel participating in the operation can be adjusted, thereby realizing the operation ability of different number and different dimension convolution. This makes the device have high reconfigurability and scalability.

[0100] Figure 7 The schematic diagram of the full connection operation unit according to the embodiment of the present disclosure is schematically shown. As shown in Figure 7 The full connection operation module 3 includes a second laser generating module 31, a second data input module 32, a second delay module 33, a waveform shaper 34 and a second detector array 35.

[0101] The second laser generating module 31, the second data input module 32, the second delay module 33, the waveform shaper 34 and the second detector array 35 are connected in sequence through the optical fiber jumper. The full connection operation module 3 further includes an optical amplifier, which can be located at any position in the optical link for amplifying the power of the laser in the optical link.

[0102] The second laser generating module 31 is configured to generate a third laser, and the third laser includes P laser components.

[0103] The second laser generating module 31 generates laser with comb spectrum, and the laser includes P laser components. The wavelengths of each laser component are different, and the wavelength interval between any two adjacent wavelength laser components is equal.

[0104] For example, the second laser generating module 31 generates four laser components Laser1-Laser4, and the wavelengths of the four laser components Laser1-Laser4 are λ1, λ2, λ3 and λ4 respectively. Among them, the wavelength interval between any two adjacent wavelength laser components in λ1, λ2, λ3 and λ4 is the same. Understandably, the wavelength relationship between λ1, λ2, λ3 and λ4 satisfies Δλ = λ1- λ2 = λ2- λ3 = λ3- λ4.

[0105] The second laser generating module 31 can be a multi-wavelength laser, or P lasers, or an actively mode-locked laser, an optical frequency comb, a high-speed direct modulation laser, or a combination of a laser and an electro-optical modulator.

[0106] The second data input module 32 is configured to modulate the convolution operation result onto the third laser.

[0107] For example, the second data input module 32 is an electro-optical modulator, and the second data input module 32 is connected to the data conversion module 2 through a cable.

[0108] The data conversion module 2 further comprises a digital-to-analog converter. The one-dimensional data is a digital signal, and the one-dimensional data needs to be converted from a digital signal to an analog signal through the digital-to-analog converter. The electro-optical modulator modulates the convolution data onto the third laser.

[0109] The second data input module 32 further comprises an electrical amplifier. The electrical amplifier can be located between the digital-to-analog converter and the electro-optical modulator, and the electrical amplifier is connected to the digital-to-analog converter and the electro-optical modulator through cables at two ends, respectively, for receiving and amplifying the power of the to-be-processed signal, and sending the amplified to-be-processed signal to the electro-optical modulator.

[0110] The second delay module 33 is configured to delay the modulated third laser to obtain a fourth laser. The second delay module 33 is a dispersive medium.

[0111] The electro-optical modulator of the embodiment of the present disclosure has the same specific implementation details and technical effects as the electro-optical modulator included in the first data input module 12, and thus the embodiment of the present disclosure will not be described in detail.

[0112] The waveform shaper 34 is configured to receive the fourth laser, adjust the intensity of the fourth laser according to Q preset decisions, and split the adjusted fourth laser to obtain Q groups of optical signals. The Q groups of optical signals correspond to the Q preset decisions one by one, respectively. Wherein, Q is a positive integer.

[0113] The waveform shaper 34 can also be replaced by a wavelength selection switch. The waveform shaper 34 attenuates the intensities of different laser components, so that the relative intensity relationship of the attenuated laser components is the same as the relative relationship of the preset decision values.

[0114] The waveform shaper 34 is configured to split the fourth laser to obtain Q groups of optical signals, each group of optical signals comprising P laser components, and adjust the intensities of the P laser components in the Q groups of optical signals according to Q preset decisions, respectively, to obtain the Q groups of optical signals with adjusted intensities.

[0115] For example, the waveform shaper 34 splits the fourth laser into Q groups of optical signals, each group containing P laser components, and each preset decision corresponds to one of the Q groups of optical signals.

[0116] Each preset decision includes G full connection weights, and the intensity of G laser components in each group of optical signals P laser components is adjusted according to the G full connection weights in each preset decision respectively, so as to obtain Q groups of optical signals after intensity adjustment, and each group of adjusted optical signals includes G laser components. Wherein, the G laser components included in each group of optical signals are G laser components with continuous adjacent wavelengths in the original P laser components of the group of optical signals, G≤P, G, P and Q are positive integers.

[0117] The waveform shaper 34 includes Q groups of output ports, and each group of output ports includes a first output port and a second output port.

[0118] The waveform shaper 34 outputs the first optical signal of each group of optical signals from the first output port of the corresponding group of output ports of the Q groups of output ports respectively, and outputs the second optical signal of each group of optical signals from the second output port of the corresponding group of output ports of the Q groups of output ports respectively.

[0119] The adjusted group of optical signals includes a first optical signal and a second optical signal. The first optical signal is an optical signal corresponding to a laser component adjusted according to a negative weight value in the preset decision in the group of optical signals, and the second optical signal is an optical signal corresponding to a laser component adjusted according to a non-negative weight value in the preset decision in the group of optical signals.

[0120] The second detector array 35 includes Q balanced photodetectors 351, and the Q balanced photodetectors are used to detect the intensity of the Q groups of optical signals respectively to obtain full connection operation results.

[0121] The balanced photodetector 351 includes two input ports. The first and second optical signals of each group of optical signals are received through the two input ports respectively. The balanced photodetector 351 converts the first and second optical signals into first and second electrical signals respectively, and performs a difference operation on the first and second electrical signals to obtain a full connection operation result. Q full connection results can be obtained through Q balanced photodetectors.

[0122] The present disclosure provides a detailed operation method suitable for the above-mentioned neural network device based on multimode interference, Figure 8 A flowchart of the operation method according to an embodiment of the present disclosure is schematically shown.

[0123] As Figure 8 shown, the object recognition method at least includes the following steps:

[0124] S1, performing convolution operation on the object to be recognized based on the multimode interference coupler through at least one convolution operation module to obtain a convolution operation result.

[0125] S2, convert the convolution operation result into one-dimensional data by a data conversion module.

[0126] S3, perform a full connection operation on the one-dimensional data by at least one full connection operation module to obtain a full connection operation result.

[0127] S4, perform a logical operation on the full connection operation result by a logical operation module to obtain a recognition result.

[0128] It should be noted that the operation method in the embodiments of the present disclosure corresponds to the neural network device based on multi-mode interference in the embodiments of the present disclosure, and the description of the operation method is specifically referred to the neural network device based on multi-mode interference, which will not be repeated here.

[0129] For the embodiments of the present disclosure, it should also be noted that the features in the embodiments of the present disclosure and the embodiments can be combined with each other to obtain new embodiments without conflict.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure rather than limit the present disclosure, and although the present disclosure has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present disclosure can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present disclosure.

Claims

1. A multi-mode interference based neural network device, comprising: The application relates to a method for identifying an object, comprising the following steps: at least one convolution operation module is used to perform convolution operation on the object to be identified based on a multimode interference coupler, so as to obtain a convolution operation result; a data conversion module is used to convert the convolution operation result into one-dimensional data; at least one full connection operation module is used to perform full connection operation on the one-dimensional data, so as to obtain a full connection operation result; and a logic operation module is used to perform logic operation on the full connection operation result, so as to obtain an identification result. The convolution operation module comprises: a first laser generation module is used to generate first laser; a first data input module is used to modulate the object to be identified onto the first laser; a first delay module is used to delay and split the modulated first laser, so as to obtain N second lasers; at least one multimode interference coupler is used to simultaneously perform multimode interference on the N second lasers according to a preset convolution matrix, so as to output M groups of optical signals; and a first detector array comprises M photodetectors, which are used to respectively receive the M groups of optical signals and detect the intensity of the M groups of optical signals, so as to obtain the convolution operation result; wherein M and N are positive integers.

2. The neural network apparatus of claim 1, wherein, The preset convolution matrix comprises M convolution kernels; the multimode interference coupler is used to adjust the intensity of the N second lasers according to the M convolution kernels, so as to obtain M groups of optical signals; wherein the M convolution kernels correspond to the M groups of optical signals one by one.

3. The neural network device of claim 1, wherein, The multimode interference coupler comprises N input ports and M output ports; the multimode interference coupler is used to receive the N second lasers through the N input ports and split the N second lasers to the M output ports according to N groups of preset light splitting ratios, so as to obtain M groups of optical signals; wherein the N groups of preset light splitting ratios correspond to the N second lasers one by one.

4. The neural network device of claim 1, wherein, In the case that the at least one multimode interference coupler comprises a plurality of multimode interference couplers, the plurality of multimode interference couplers are connected in parallel. The multimode interference coupler comprises N input waveguides, M output waveguides and a multimode interference region; the multimode interference region comprises a plurality of multimode interference units and a plurality of control units, and the plurality of control units are respectively used to adjust the refractive index of the plurality of multimode interference units.

5. The neural network device of claim 1, wherein, The full connection operation module comprises: a second laser generation module is used to generate third laser, and the third laser comprises P laser components; a second data input module is used to modulate the convolution operation result onto the third laser; a second delay module is used to delay the modulated P laser components, so as to obtain fourth laser; a waveform shaper is used to receive the fourth laser, adjust the intensity of the fourth laser according to Q preset judgments, and split the adjusted fourth laser, so as to obtain Q groups of optical signals; a second detector array comprises Q balanced photodetectors, which are used to respectively detect the intensity of the Q groups of optical signals, so as to obtain the full connection operation result; wherein the P laser components have different wavelengths, the wavelength interval between any two adjacent laser components is equal, and P and Q are positive integers.

6. The neural network device of claim 5, wherein, The waveform shaper comprises Q groups of output ports, each group of output ports comprising a first output port and a second output port; The waveform shaper is configured to split the fourth laser to obtain Q groups of optical signals, each group of optical signals comprising P laser components, and adjust the intensities of the P laser components in the Q groups of optical signals according to Q preset decisions respectively to obtain Q groups of intensity-adjusted optical signals; The waveform shaper is further configured to output a first optical signal of one group of optical signals from a first output port of a corresponding one group of output ports of the Q groups of output ports and output a second optical signal of the one group of optical signals from a second output port of the corresponding one group of output ports of the Q groups of output ports respectively. The one group of adjusted optical signals comprises the first optical signal and the second optical signal, the first optical signal is an optical signal corresponding to a laser component adjusted according to a negative weight value in the preset decision, and the second optical signal is an optical signal corresponding to a laser component adjusted according to a non-negative weight value in the preset decision, the Q preset decisions correspond to the Q groups of optical signals one by one.

7. The neural network device of claim 6, wherein, The balanced photodetector comprises two input ports; The balanced photodetector is configured to receive a first optical signal and a second optical signal of one group of optical signals through the two input ports respectively; The balanced photodetector is further configured to convert the first optical signal and the second optical signal into first and second electrical signals respectively, and perform a difference operation on the first and second electrical signals to obtain a full connection operation result.

8. The neural network device of claim 1, wherein, The neural network device further comprises: An electrical amplifier configured to amplify the power of an electrical signal in an electrical link of the neural network device; An optical amplifier configured to amplify the power of a laser in an optical link of the neural network device; A polarization controller configured to adjust the polarization state of the laser in the optical link of the neural network device.

9. An arithmetic method characterized by comprising: The multi-mode interference-based neural network device according to any one of claims 1-8, the method comprising: performing convolution operation on the object to be identified based on the multi-mode interference coupler through at least one convolution operation module to obtain a convolution operation result; converting the convolution operation result into one-dimensional data through a data conversion module; performing full connection operation on the one-dimensional data through at least one full connection operation module to obtain a full connection operation result; and performing logical operation on the full connection operation result through a logical operation module to obtain a recognition result.

Images