Opto-electric hybrid tensor convolution operation system and method

Abstract

The application discloses a kind of photoelectric hybrid tensor convolution operation system and method, the system uses same frequency double optical frequency comb, combines wave division multiplexing and demultiplexing technology, combines IQ modulator array, utilizes optical mixer to mix, utilizes photoelectric balance detector to carry out photoelectric conversion, finally utilizes oscilloscope and the like data acquisition device output tensor convolution operation result.The application, data packet is modulated to different optical frequency comb channel, the total signal spectrum spliced in frequency domain is wide, simultaneously reduce the bandwidth requirement of single channel to modulated electrical signal, with higher signal-to-noise ratio and greater bandwidth;In addition, data packet is modulated simultaneously in different channel, complete convolution result can be obtained by once operation via mixer and balance detector, and the application has the advantages of large throughput and adjustable configuration, simple operation, convenient and flexible adjustment;System uses all-fiber device, can be integrated, suitable for subsequent on-chip integration.

Description

Technical Field

[0001] This invention relates to the field of optical computing technology, specifically to a photoelectric hybrid tensor convolution operation system and method. Background Technology

[0002] Artificial neural networks are collections of nodes with weighted connections. These nodes can modify the network weights through appropriate feedback, enabling them to "learn" and perform complex operations such as facial recognition, speech translation, and medical diagnosis.

[0003] However, both traditional fully connected feedforward networks and the currently popular convolutional neural networks and recurrent neural networks require a large number of tensor convolution operations. These operations account for 55% to 90% of the total computation. Therefore, accelerating tensor convolution operations is an important way to improve the computational power of neural networks.

[0004] To enhance the computational power of neural networks and accelerate tensor convolution operations, it is essential to improve the computational capabilities of the underlying neuromorphic hardware. Optical neural networks (ONNs), with their potential to overcome the bandwidth bottleneck of electronic artificial neural networks and their high-speed, low-power capabilities, represent one of the development trends for next-generation neuromorphic computing. Currently, there are three main methods for implementing optical tensor convolution operations:

[0005] The first method involves loading the data of the weight tensor onto the amplitude of the optical frequency comb, then using an electro-optic modulator to modulate the data of the input tensor onto the optical frequency comb, and finally using optical fiber to introduce dispersion to complete the tensor convolution operation.

[0006] The second method maps the data of the weight tensor onto the phase change material array, while the data of the input tensor is loaded onto the amplitude of the optical frequency comb. The comb light completes the tensor calculation after passing through the phase change material array.

[0007] The third method uses a Mach-Zehnder interferometer array to represent the weight tensor. The input tensor data is loaded onto a coherent or incoherent light source, and the convolution operation is completed after passing through the array. The coherent light source can perform complex domain operations, while the incoherent light source can perform real domain operations.

[0008] In January 2020, Xu X et al. from Australia implemented an optical convolution accelerator with a computational speed of 11 TOPS based on an optical frequency comb, using a micro-ring resonator, waveform shaper, electro-optic modulator, and single-mode fiber. Due to implementation requirements, the tensor size and number are difficult to increase, and the need to utilize fiber dispersion effects requires a large physical space; therefore, the effective computational efficiency needs improvement (Xu X, Tan M, Corcoran B, et al. 11 TOPS photonicconvolutional accelerator for optical neural networks[J]. Nature, 2021, 589(7840):44–51.). Meanwhile, Feldmann J et al. from Germany implemented in-memory computation of an optical frequency comb with a TOPS-level computation speed using an on-chip integrated phase change material array, micro-ring resonator, and electro-optic modulator. Due to limitations in the scale of phase change material arrays, the tensor size and the number of tensor kernels need to be increased (Feldmann J, Youngblood N, Karpov M, et al. Parallel convolutional processing using an integrated photonic tensor core[J]. Nature, 2021, 589(7840): 52–58.). In 2018, Hengmen Bagherian in the United States proposed a tensor convolutional computation architecture based on a Mach-Zehnder interferometer array. This architecture can achieve millions of operations per second and consumes only 2 mJ / ops. However, as the tensor size increases, silicon photonic devices require increasing tensor size and number of kernels (N0, N ... 2 As the scale increases, the integration size becomes limited, making it difficult to meet the requirements for large-scale tensor convolution operations (Bagherian H, Skirlo S, Shen Y, et al. On-chip optical convolutional neural networks[J]. arXiv preprint arXiv:1808.03303,2018.).

[0009] Therefore, it is necessary to improve existing tensor convolution operation systems and methods to meet the requirements of large-scale tensor convolution operations. Summary of the Invention

[0010] To address the aforementioned shortcomings, the technical problem to be solved by this invention is to provide a photoelectric hybrid tensor convolution operation system and method, so as to solve the problem that current tensor convolution operation systems are unable to meet the requirements of large-scale tensor convolution operations.

[0011] Therefore, the present invention provides an optoelectronic hybrid tensor convolution operation system, comprising:

[0012] A phase-locked optical frequency comb device provides a first and second optical frequency comb for phase-locked optical frequency;

[0013] An electro-optic modulation array includes a first demultiplexer and a second demultiplexer disposed at the input end, and a first wavelength division multiplexer and a second wavelength division multiplexer disposed at the output end. A first modulation array is disposed between the first demultiplexer and the first wavelength division multiplexer, and a second modulation array is disposed between the second demultiplexer and the second wavelength division multiplexer. The first modulation array includes a first arbitrary waveform generator, a first modulation driver, a first 90° bridge, and a first IQ modulator corresponding to the number of tensor data packets to be convolved. The tensor data packets to be convolved are respectively modulated by each of the first modulation arrays onto each tooth of the first optical frequency comb, and then combined to generate a first modulation signal. The second modulation array includes a second arbitrary waveform generator, a second modulation driver, a second 90° bridge, and a second IQ modulator corresponding to the number of weight tensor data packets to be convolved. The weight tensor data packets to be convolved are respectively modulated by each of the second modulation arrays onto each tooth of the second optical frequency comb, and then combined to generate a second modulation signal.

[0014] An optical mixer mixes the first modulation signal and the second modulation signal to generate two optical outputs, which are then converted into electrical signals by a photoelectric balance detector.

[0015] The data acquisition device acquires the electrical signals and obtains the tensor convolution operation results.

[0016] In the above system, preferably, the optical frequency comb laser source provides a coherent phase-locked optical frequency comb;

[0017] A first polarization-maintaining erbium-doped fiber amplifier amplifies the optical frequency comb;

[0018] A 1:1 beam splitter outputs the optical frequency comb equally as the first optical frequency comb and the second optical frequency comb.

[0019] The first adjustable optical delay line is adjusted so that the optical path of the second optical frequency comb is equal to that of the first optical frequency comb.

[0020] In the above system, preferably, the first demultiplexer separates the different wavelength teeth of the first optical frequency comb to form a plurality of first teeth corresponding to the number of tensor data groups to be convolved; the second demultiplexer separates the different wavelength teeth of the second optical frequency comb to form a plurality of second teeth corresponding to the number of weight tensor data groups to be convolved.

[0021] In the above system, preferably,

[0022] The first comb tooth is input to the first IQ modulator. The first arbitrary waveform generator generates a corresponding first electrical signal according to the tensor data to be convolved, and after being amplified by the first modulator driver, it is input to the first 90° bridge. The first 90° bridge generates first I and Q electrical signals with a 90-degree phase difference, which are respectively input to the first IQ modulator. The tensor data to be convolved is modulated onto each of the first comb teeth, and then combined by the first wavelength division multiplexer to generate the first modulated signal.

[0023] The second comb tooth is input to the second IQ modulator. The second arbitrary waveform generator generates a corresponding second electrical signal according to the weight tensor data to be convolved, and after being driven and amplified by the second modulator, it is input to the second 90° bridge. The second 90° bridge generates second I and Q electrical signals with a 90-degree phase difference, which are respectively input to the second IQ modulator to modulate the weight tensor data to be convolved onto each of the second comb teeth, and then combined by the second wavelength division multiplexer to generate the second modulated signal.

[0024] In the above system, preferably, the optical path of the second modulation signal is adjusted by the second adjustable optical delay line so that it has the same optical path as the first modulation signal.

[0025] In the above system, preferably, the control system controls the tensor data to be convolved and the weight tensor data input to the first and second arbitrary waveform generators, controls the optical path length of the first and second adjustable optical delay lines, controls the bias voltage of the first and second IQ modulators, and controls the acquisition parameters of the data acquisition device.

[0026] This invention also provides a method for photoelectric hybrid tensor convolution, comprising the following steps:

[0027] A first and second optical frequency comb with the same frequency phase lock are provided, and a dewavelength division multiplexer is used to separate the optical comb teeth of different channels of the first and second optical frequency combs;

[0028] The tensor data to be convolved and the weight tensor data to be convolved are modulated onto each tooth of the first and second optical frequency combs, respectively, and then combined to generate the first modulation signal and the second modulation signal.

[0029] The first modulation signal and the second modulation signal are mixed by an optical mixer to generate two optical outputs, which are then converted into electrical signals by a photoelectric balance detector.

[0030] The electrical signals are acquired by a data acquisition device to obtain the tensor convolution operation results.

[0031] In the above method, preferably, the optical path of the second optical frequency comb is adjusted by using a first adjustable optical delay line so that it has the same optical path as the first optical frequency comb.

[0032] In the above method, preferably, the optical path of the second modulation signal is adjusted by a second adjustable optical delay line so that it has the same optical path as the first modulation signal.

[0033] In the above method, preferably, the control system controls the tensor data to be convolved and the weight tensor data to be convolved input to the first and second arbitrary waveform generators, controls the optical path length of the first and second adjustable optical delay lines, controls the bias voltage of the first and second IQ modulators, and controls the parameters of the data acquisition device.

[0034] As can be seen from the above technical solution, the photoelectric hybrid tensor convolution operation system and method provided by the present invention solves the problem that existing technologies cannot meet the requirements of large-scale tensor convolution operations. Compared with the prior art, the present invention has the following beneficial effects:

[0035] The system employs dual optical frequency combs at the same frequency, combined with wavelength division multiplexing and demultiplexing techniques, and an IQ modulator array. It utilizes an optical mixer for frequency mixing, a photoelectric balanced detector for photoelectric conversion, and finally outputs the tensor convolution operation results using data acquisition devices such as an oscilloscope. The system has advantages such as high signal-to-noise ratio, large bandwidth, high throughput, and easy on-chip integration.

[0036] The beat frequency in the optoelectronic fusion system is based on the modulation of orthogonal frequency signals with dual optical frequency combs of the same frequency. The configuration is adjustable and easy to use. Attached Figure Description

[0037] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the embodiments of the present invention or the prior art will be briefly introduced and explained below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a system block diagram of the optoelectronic hybrid tensor convolution operation system provided by the present invention;

[0039] Figure 2 This is a schematic diagram of the structure of the optoelectronic hybrid tensor convolution operation system provided by the present invention;

[0040] Figure 3The flowchart of the photoelectric hybrid tensor convolution operation method provided by the present invention is shown. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] It should be noted that the directional terms such as "inner" and "outer", "front" and "back" and "left" and "right" in this article are based on the product's usage status. Obviously, the use of these directional terms does not limit the scope of protection of this solution.

[0043] Please see Figure 1 , Figure 1 The system block diagram of the optoelectronic hybrid tensor convolution operation system provided by the present invention.

[0044] like Figure 1 As shown, the present invention provides a photoelectric hybrid tensor convolution operation system, including an optical frequency comb laser source 1, an optical amplification, splitting and delay device 2, an electro-optic modulation array 3, an optical delay and amplification device 4, an optical mixing device 5, an optical detection device 6, a data acquisition device 7 and a control device 8.

[0045] The optical frequency comb laser source 1 and the optical amplification, splitting, and delay device 2 constitute a co-frequency phase-locked optical frequency comb device, providing a first and a second optical frequency comb for co-frequency phase locking. Specifically, the optical frequency comb laser source 1 provides the system with a coherent phase-locked optical frequency comb. The optical amplification, splitting, and delay device 2 amplifies the optical frequency comb, then splits it and adjusts the two optical paths to be consistent, providing the system with two co-frequency phase-locked optical frequency combs: the first and the second.

[0046] The electro-optic modulation array 3 uses first and second demultiplexers to separate the different wavelength comb teeth of the first and second optical frequency combs, respectively, to obtain multiple first comb teeth corresponding to the number of tensor data groups to be convolved, and multiple second comb teeth corresponding to the number of weight tensor data groups to be convolved. Then, the tensor data groups to be convolved and the weight tensor data groups to be convolved are modulated onto each of the first comb teeth and each of the second comb teeth by different arbitrary waveform generators. Finally, they are combined to generate the first modulation signal and the second modulation signal for output.

[0047] The optical delay and amplification device 4 is used to adjust the optical path between the second modulation signal and the first modulation signal to be consistent and to amplify them.

[0048] The optical mixer 5 mixes the first modulation signal and the second modulation signal, then converts them into electrical signals by the photoelectric balance detector 6, and then acquires them through the data acquisition device 7 to obtain the tensor convolution operation result.

[0049] The control device 8 can control the input data of the arbitrary waveform generator (tensor data to be convolved and weight tensor data to be convolved), control and adjust the optical path of the adjustable optical delay line, control and adjust the bias voltage of the IQ modulator and the parameters of the data acquisition device 7, thereby realizing the flexible and adjustable optical computing system.

[0050] Specifically, please see Figure 2 The diagram shows the structure of a hybrid optoelectronic tensor convolution operation system. The optical amplification, splitting, and delay device 2 includes a first polarization-maintaining erbium-doped fiber amplifier 201, a 1:1 beam splitter 202, and a first tunable optical delay line 203. The first polarization-maintaining erbium-doped fiber amplifier 201 is used to amplify the power of the light source and compensate for optical losses during system operation. The electro-optic modulation array 3 includes a first dewavelength division multiplexer 301 and a second dewavelength division multiplexer 302 disposed at its input end, and a first wavelength division multiplexer 319 and a second wavelength division multiplexer 320 disposed at its output end. A first modulation array is provided between the first dewavelength division multiplexer 301 and the first wavelength division multiplexer 319, and the second dewavelength division multiplexer 302... A second modulation array is provided between the second wavelength division multiplexer 320 and the first modulation array. The first modulation array includes multiple sets of first arbitrary waveform generators 303, first modulation drivers 307, first 90° bridges 311 and first IQ modulators 315 corresponding to the number of tensor data packets to be convolved. The tensor data packets to be convolved are respectively modulated by the first modulation array to generate first modulation signals. The second modulation array includes multiple sets of second arbitrary waveform generators 304, second modulation drivers 308, second 90° bridges 312 and second IQ modulators 316 corresponding to the number of weight tensor data packets to be convolved. The weight tensor data packets to be convolved are respectively modulated by the second modulation array to generate second modulation signals.

[0051] The optical frequency comb laser source 1 outputs an optical frequency comb to a high-power erbium-doped fiber amplifier 201. After amplification by the fiber amplifier 201, the output is sent to a 1:1 beam splitter 202, which then equally outputs two optical frequency combs: a first optical frequency comb and a second optical frequency comb. The first optical frequency comb outputs to a first dewavelength division multiplexer 301, while the second optical frequency comb outputs to an adjustable optical delay line 203. After being delayed by the first adjustable optical delay line 203, the output is sent to the second dewavelength division multiplexer 302. The first adjustable optical delay line 203 is controlled by a control device 7 to adjust the optical path of the second optical frequency comb, ensuring that it is in phase with the first optical frequency comb.

[0052] The first wavelength division multiplexer 301 separates the different wavelength teeth of the first optical frequency comb, forming multiple first comb teeth with the same number of groups as the tensor data to be convolved, and inputs them into the corresponding first IQ modulators 315. Simultaneously, each first arbitrary waveform generator 303 generates a corresponding electrical signal based on the input tensor data to be convolved. This electrical signal is amplified by the first modulator driver 307 and input to the first 90° bridge 311. The first 90° bridge 311 generates I and Q electrical signals with a 90-degree phase difference, which are input to the I and Q electrical input ports of the first IQ modulator 315, modulating the input tensor data to be convolved onto each of the first optical frequency combs. Finally, the signals are combined by the first wavelength division multiplexer 319 to generate the first modulated signal. The first modulation driver 307 generates a bias voltage that can adjust the first IQ modulator, and the first 90° bridge generates two electrical drive signals with a 90° phase difference.

[0053] Similarly, the second wavelength division multiplexer 302 separates the different wavelength comb teeth of the second optical frequency comb to form multiple second comb teeth with the same number of groups as the weight tensor data to be convolved, and inputs them into the corresponding second IQ modulator 316. At the same time, each second arbitrary waveform generator 304 generates a corresponding electrical signal according to the input weight tensor data. The electrical signal is amplified by the second modulator driver 308 and input to the second 90° bridge 312. The second 90° bridge 312 generates I and Q electrical signals with a 90-degree phase difference, which are input to the I and Q electrical input ports of the second IQ modulator 316, respectively, to modulate the input weight tensor data to be convolved onto each second optical frequency comb. Finally, the signals are combined by the second wavelength division multiplexer 320 to generate the second modulated signal.

[0054] Tensor data A is convolved with weight tensor data B. Data A contains N tensor data points to be convolved, divided into M groups. The first modulation array includes M first arbitrary waveform generators 303, M first modulation drivers 307, M first 90° bridges 311, and M first IQ modulators 315. The second modulation array is similar to the first, with the same number of second arbitrary waveform generators, second modulation drivers, second 90° bridges, and second IQ modulators as the number of groups of the weight tensor data B to be convolved.

[0055] The optical delay and amplification device 4 includes a second optical delay line 401, a second amplifier 402, and a third amplifier 403. The first and second modulation signals output from the electro-optic modulation array 3 are respectively sent to the second optical delay line 401 and the second amplifier 402. The second modulation signal is delayed by the second optical delay line 401 to ensure that it is in phase with the first modulation signal. Then, the first modulation signal is amplified by the second amplifier 402, and the second modulation signal is amplified by the third amplifier 403 before being output to the optical mixer 5.

[0056] The optical mixer 5 mixes the first and second modulation signals to generate two optical outputs, which are then converted into electrical signals by the photoelectric balance detector 6. The electrical signals are then acquired by the data acquisition device 7 to obtain the tensor convolution operation results.

[0057] The system employs dual optical frequency combs at the same frequency, combined with wavelength division multiplexing and demultiplexing techniques, and an IQ modulator array. It utilizes an optical mixer for frequency mixing, a photoelectric balanced detector for photoelectric conversion, and finally outputs the tensor convolution operation results using data acquisition devices such as an oscilloscope. The system has advantages such as high signal-to-noise ratio, large bandwidth, high throughput, and easy on-chip integration.

[0058] Based on the aforementioned optoelectronic hybrid tensor convolution operation system, this invention also provides an optoelectronic hybrid tensor convolution operation method. Please refer to [link to relevant documentation]. Figure 3 , Figure 3 A flowchart of a photoelectric hybrid tensor convolution operation method provided by the present invention.

[0059] like Figure 3 As shown, the method includes the following steps:

[0060] Step 110: The optical frequency comb laser source 1 is made to provide the same multiple channel optical comb teeth for the upper and lower paths through a 1:1 optical beam splitter, namely the first optical frequency comb and the second optical frequency comb.

[0061] For example, the center wavelength of the optical frequency comb laser source 1 is planned to be 193.1 THz, and the repetition frequency is 12.5 GHz. For simplicity, only the optical paths of two channels in each of the upper and lower optical comb teeth are shown in the figure. The wavelengths of the upper and lower optical comb teeth are 193.1 THz and 193.1 THz + 12.5 GHz, respectively. The number of optical comb teeth can be increased according to actual application needs.

[0062] It is important to note that the wavelengths of the upper and lower optical combs should be as close as possible to prevent insufficient detector bandwidth from failing to reproduce the convolution result.

[0063] Step 120: Adjust the optical path of the second optical frequency comb using the first adjustable optical delay line 202 to ensure that the upper and lower optical comb teeth have equal optical path.

[0064] Step 130: Using the center wavelengths of the filters in the dewavelength division multiplexer, 193.1THz and 193.1THz+12.5GHz, the optical combs of the two different channels are separated.

[0065] Step 140: The control system 8 inputs the tensor data group to be convolved and the weight tensor data group to the corresponding arbitrary waveform generators. Each arbitrary waveform generator generates a corresponding electrical signal according to the input data. The electrical signals are amplified by the modulator and then I and Q electrical signals are generated by the 90° bridge. These signals are input to the electrical modulation input ports of the two IQ modulators, respectively, and the input tensor data and weight tensor data are modulated onto the comb teeth of the first and second optical frequency combs.

[0066] Step 150: The signals carried by the first and second optical frequency combs are combined by the first wavelength division multiplexer 319 and the second wavelength division multiplexer 320 respectively to generate the first modulation signal and the second modulation signal.

[0067] Step 160: Adjust the optical path of the second modulation signal through the second adjustable optical delay line 401 to ensure that the first modulation signal and the second modulation signal are in phase, and amplify them through the first and second erbium-doped fiber amplifiers respectively.

[0068] Step 170: Optical mixing signals with a phase difference of 0° and 180° are generated by a 180° optical bridge, and the first modulation signal and the second modulation signal are mixed.

[0069] Step 180: The optical mixing signal is detected by the photoelectric balance detector 6 and converted into an electrical signal, which is then acquired by the data acquisition device 7 to obtain the tensor convolution operation result.

[0070] In this invention, the mathematical principle of the operation method based on orthogonal frequency optoelectronic hybrid tensor convolution is as follows: The following description only takes the number of groups (number of comb teeth) equal to 2 as an example. In actual design, this number can be greater than or equal to 2.

[0071] Let the tensor data A to be convolved and the weight tensor data B to be convolved be respectively... and N1 is the number of the first group of tensor data to be convolved, N2 is the number of the second group of tensor data to be convolved, M1 is the number of the first group of weight tensor data, and M2 is the number of the second group of tensor data to be convolved.

[0072] The tensor data to be convolved is grouped and input into the system by the first arbitrary waveform generator 303, respectively. in The initial frequency of the orthogonal frequency electrical signal for each group of tensor data to be convolved in group A can be defined by the user, and t is time; the weighted tensor data, after being grouped, is input into the system by the second arbitrary waveform generator 305, respectively. in The initial frequency of the orthogonal frequency electrical signal for each group of data in the tensor to be weighted can be defined by the user.

[0073] It needs to satisfy M1ω0=Δω, N1ω0=Δω; or And ω0, That is, for the tensor data and weight data to be convolved, they are grouped sequentially, and the bandwidth occupied by each group is equal to or less than the optical comb frequency interval Δω. The signals of each group modulated onto the optical carrier do not overlap in the spectrum, and the bandwidth of the orthogonal signals used is much smaller than the optical carrier bandwidth ω. c .

[0074] Let the optical wavelengths of the two optical frequency comb channels used be expressed as Ccosω. c t、Ccos(ω c +Δω)t, where C is the amplitude of the light source, ω c Let be the frequency of the center wavelength of the optical frequency comb, and Δω be the repetition frequency of the optical frequency comb. The output signals of the two optical frequency combs after passing through the first and second IQ modulators 315 and 316 can be expressed as follows:

[0075]

[0076]

[0077]

[0078]

[0079] The above four signals are combined by a wavelength division multiplexer.

[0080] M1(t)=M s1 (t)+M s2 (t) (5)

[0081] M2(t)=M s3 (t)+M s4 (t) (6)

[0082] Since M1ω0=Δω and N1ω0=Δω, the signal after beam combining by the wavelength division multiplexer is:

[0083]

[0084]

[0085] After being received by the input terminals ① and ② of the 180° optical bridge 5 respectively, optical mixing signals E1(t) and E2(t) with a phase difference of 0° and 180° are generated at the output ports ③ and ④ respectively.

[0086] E1(t)=M s (t)+M s (t) (9)

[0087] E2(t)=M s (t)-M s (t) (10)

[0088] Subsequently, the optical mixing signals E1(t) and E2(t) generated by the output ports ③ and ④ of the 180° optical bridge 5 are received by the input ports ① and ② of the balanced detector, respectively, for detection. The output electrical signals of the balanced detector are I1(t) and I2(t), respectively, where α is the photoelectric conversion efficiency, which is determined by the performance of the photodetector.

[0089]

[0090] Due to the limited operating bandwidth of the detector, the actual current output is,

[0091]

[0092] A bandpass filter can be used to filter frequencies of 10 ... The signal is filtered out, and the amplitude is read to obtain the signal.

[0093] It is important to note that If z∈Z, then the mirror frequencies will be superimposed, that is, The superposition of the amplitudes of the frequency waveforms renders the results invalid.

[0094] The electrical signal is acquired by the data acquisition device 7 and output to the control device 8.

[0095] Thus, the convolution result can be given by formula (12).

[0096] Among them, the input tensor The length N1+N2 and the weight tensor The lengths M1 and M2 are both adjustable. Therefore, arbitrary convolution operations can be performed in the control device (N1, N2, M1, M2 are constrained by the orthogonal frequency interval of the electrical signal and the operating bandwidth of the optical and electrical devices), achieving adjustable configuration.

[0097] Based on the above description of specific embodiments, the photoelectric hybrid tensor convolution operation system and method provided by the present invention have the following advantages compared with the prior art:

[0098] First, in this invention, the data is grouped and modulated onto different optical frequency comb channels, resulting in a wider total signal spectrum when spliced ​​together in the frequency domain. This also reduces the bandwidth requirement of a single channel for the modulated electrical signal. Therefore, the system has a higher signal-to-noise ratio and a larger bandwidth.

[0099] Secondly, in this invention, after data is grouped, it is simultaneously modulated in different channels. The complete convolution result can be obtained in one operation through a mixer and a balanced detector. Therefore, the system has a large throughput and is flexible.

[0100] Third, in this invention, the system uses all-fiber devices, which are integrable and suitable for subsequent on-chip integration.

[0101] Fourth, in this invention, the parameters of each adjustable device can be modified through a computer control system, making operation simple and adjustment convenient.

[0102] Finally, it should be noted that the terms "comprising," "including," or any other variations thereof as used herein are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also any other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one…" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0103] This invention is not limited to the above-described preferred embodiments. Anyone should know that any structural changes made under the guidance of this invention, and any technical solutions that are the same as or similar to this invention, fall within the protection scope of this invention.

Claims

1. A photoelectric hybrid tensor convolution operation system, characterized in that, include: A phase-locked optical frequency comb device provides a first and second optical frequency comb for phase-locked optical frequency; The same-frequency phase-locked optical frequency comb device includes: Optical frequency comb laser source, providing coherent phase-locked optical frequency comb; A first polarization-maintaining erbium-doped fiber amplifier amplifies the optical frequency comb; A 1:1 beam splitter outputs the optical frequency comb equally as the first optical frequency comb and the second optical frequency comb. The first adjustable optical delay line is made to have the same optical path length as the first optical frequency comb by adjusting the optical path length of the second optical frequency comb. An electro-optic modulation array includes a first demultiplexer and a second demultiplexer disposed at the input end, and a first wavelength division multiplexer and a second wavelength division multiplexer disposed at the output end. A first modulation array is disposed between the first demultiplexer and the first wavelength division multiplexer, and a second modulation array is disposed between the second demultiplexer and the second wavelength division multiplexer. The first modulation array includes a first arbitrary waveform generator, a first modulation driver, a first 90° bridge, and a first IQ modulator corresponding to the number of tensor data packets to be convolved. The tensor data packets to be convolved are respectively modulated by each of the first modulation arrays onto each tooth of the first optical frequency comb, and then combined to generate a first modulation signal. The second modulation array includes a second arbitrary waveform generator, a second modulation driver, a second 90° bridge, and a second IQ modulator corresponding to the number of weight tensor data packets to be convolved. The weight tensor data packets to be convolved are respectively modulated by each of the second modulation arrays onto each tooth of the second optical frequency comb, and then combined to generate a second modulation signal. An optical mixer mixes the first modulation signal and the second modulation signal to generate two optical outputs, which are then converted into electrical signals by a photoelectric balance detector. The data acquisition device acquires the electrical signals and obtains the tensor convolution operation results.

2. The system according to claim 1, characterized in that, The first demultiplexer separates the different wavelength teeth of the first optical frequency comb to form multiple first teeth corresponding to the number of tensor data groups to be convolved; the second demultiplexer separates the different wavelength teeth of the second optical frequency comb to form multiple second teeth corresponding to the number of weight tensor data groups to be convolved.

3. The system according to claim 2, characterized in that, The first comb tooth is input to the first IQ modulator. The first arbitrary waveform generator generates a corresponding first electrical signal according to the tensor data to be convolved, and after being amplified by the first modulation driver, it is input to the first 90° bridge. The first 90° bridge generates first I and Q electrical signals with a phase difference of 90 degrees, which are respectively input to the first IQ modulator. The tensor data to be convolved is modulated onto each of the first comb teeth, and then combined by the first wavelength division multiplexer to generate the first modulated signal. The second comb tooth is input to the second IQ modulator. The second arbitrary waveform generator generates a corresponding second electrical signal according to the weight tensor data to be convolved, and after being amplified by the second modulation driver, it is input to the second 90° bridge. The second 90° bridge generates second I and Q electrical signals with a 90-degree phase difference, which are respectively input to the second IQ modulator to modulate the weight tensor data to be convolved onto each of the second comb teeth, and then combined by the second wavelength division multiplexer to generate the second modulated signal.

4. The system according to claim 3, characterized in that, The optical path length of the second modulation signal is adjusted by the second tunable optical delay line so that it has the same optical path length as the first modulation signal.

5. The system according to claim 1, characterized in that, The control system controls the tensor data to be convolved and the weight tensor data input to the first and second arbitrary waveform generators, controls the optical path length of the first and second adjustable optical delay lines, controls the bias voltage of the first and second IQ modulators, and controls the acquisition parameters of the data acquisition device.

6. A method for photoelectric hybrid tensor convolution operation, based on the photoelectric hybrid tensor convolution operation system as described in any one of claims 1-5, characterized in that, Includes the following steps: A first and second optical frequency comb with the same frequency phase lock are provided, and a dewavelength division multiplexer is used to separate the optical comb teeth of different channels of the first and second optical frequency combs; The tensor data to be convolved and the weight tensor data to be convolved are modulated onto each tooth of the first and second optical frequency combs, respectively, and then combined to generate the first modulation signal and the second modulation signal. The first modulation signal and the second modulation signal are mixed by an optical mixer to generate two optical outputs, which are then converted into electrical signals by a photoelectric balance detector. The electrical signal is acquired by a data acquisition device to obtain the result of tensor convolution operation.

7. The method according to claim 6, characterized in that, The optical path length of the second optical frequency comb is adjusted by using the first adjustable optical delay line so that it is equal to the optical path length of the first optical frequency comb.

8. The method according to claim 6, characterized in that, The optical path length of the second modulation signal is adjusted by using a second adjustable optical delay line so that it is equal to the optical path length of the first modulation signal.

9. The method according to claim 6, characterized in that, The control system controls the input tensor data to be convolved and the weight tensor data to be convolved to the first and second arbitrary waveform generators, controls the optical path length of the first and second adjustable optical delay lines, controls the bias voltage of the first and second IQ modulators, and controls the parameters of the data acquisition device.

Images