An all-optical convolutional device based on dual-combs

Abstract

The present invention belongs to the field of optics, and particularly relates to an all-optical convolutional device based on dual-combs. The present invention realizes the convolutional operation of two matrices based on dual optical frequency combs, and has excellent parallel processing capabilities. Due to the phase and repetition frequency locking of the soliton frequency comb, signals can be loaded in multiple paths to achieve parallel computing. The convolution kernel and input information are respectively loaded on the two optical frequency combs to achieve time delay; and they are simultaneously coupled into a single-mode optical fiber. Through the beat frequency characteristics of the photodetector, the product calculation of multiple signals can be performed simultaneously, and the multiplication is achieved through the coherent interference effect of light for beat frequency; therefore, through the method of loading signals, and then through the photodetector, the addition can be realized, and finally the convolution of two matrices is achieved. The entire system structure of the present invention is simple, has stable performance, generates less heat, and at the same time, this device is an all-optical convolution, consumes less time, has less computing consumption, and can be directly integrated into an optical neural network to realize an artificial intelligence system for an all-optical network.

Description

Technical Field

[0001] The present invention belongs to the field of optics, and particularly relates to an all-optical convolutional device based on dual-combs, which can perform high-speed convolutional calculations on input data. Background Art

[0002] With the popularization of ultra-high-speed mobile networks and Internet-connected devices, as well as the rise of artificial intelligence (AI), the world is generating exponentially growing data. In a series of applications, including autonomous driving, robotic vision, smart homes, remote sensing, microscopes, monitoring, defense, and the recording and processes of Internet imaging systems, an unprecedented amount of data is being processed not by humans but by artificial intelligence (AI) established through algorithms. Therefore, highly parallel, fast, and scalable hardware has become increasingly important. In these applications, deep neural networks (DNNs) are rapidly becoming the standard method for visual data processing. The powerful processing capabilities and parallelism of modern graphics processing units (GPUs), as well as the availability of large amounts of visual datasets, have driven the latest breakthroughs in the field of deep learning, which enable DNNs to be effectively trained using supervised machine learning strategies.

[0003] Currently, deep neural networks are mainly electrical neural networks. Electrical neural networks generally use methods for treating electrical signals to implement the processing, learning, and recognition of information such as images and sounds. Although these electrical neural networks are relatively mature and reliable, when running on high-end GPUs for increasingly complex neural networks, as well as other accelerators with growing power and bandwidth requirements, they require a large amount of processing time and face more complex form factors. These limitations make it challenging to adopt electrical DNNs in edge devices such as cameras, autonomous vehicles, robots, or Internet of Things peripherals. Despite half a century of research, general optical computing systems have not matured into a practical technology.

[0004] The dual-comb based all-optical convolutional processor can basically solve the above-mentioned drawbacks of deep neural networks. Compared with traditional electrical convolutional processors and the latest optical convolutional processors, it has the advantages of parallel processing, low power consumption, and fast response speed. Light is an important means of modern information transmission and processing. This way of converting various information into optical signals that can be widely transmitted, quickly analyzed, and massively processed, and transmitting them through optical fibers plays an important role in information systems. Optical computing systems can achieve a large number of parallel operations in cooperation with small devices, and can achieve extremely low or even zero power consumption in some cases. In fact, optical interconnection that uses light to achieve communication in computing systems has been widely used in today's data centers, and the increasing use of optical interconnection inside computing systems may be a necessary condition for continuous expansion. Different from electronic interconnection technology, as we delve deeper into integrated optics, optoelectronics, and electronics technologies, optical interconnection offers the potential for an order-of-magnitude improvement in bandwidth density and energy per bit in communication.

[0005] In January 2021, Feldmann et al. from the University of Münster in Germany proposed a parallel convolutional computing scheme based on an optical frequency comb and a two-dimensional matrix of phase change materials. This scheme describes an example of a photonic tensor core. Combining the state-of-the-art photonic integrated microcomb light source, it realizes parallel convolutional operations in one device, demonstrating synchronous data transmission and computing speed comparable to fiber optic networks. However, the preparation of phase change materials is too complex, and its scalability is poor.

[0006] At the same time, Xu et al. from Swinburne University of Technology proposed a photonic convolutional accelerator for optical neural networks. Using an optical frequency comb as the carrier of the signal, the signal is loaded through a modulator, and the dispersion of the optical fiber itself is used to achieve shift and convolution operations. The convolution result is detected by a photodetector. A photonic convolutional accelerator for optical neural networks is demonstrated, which realizes large-scale image convolution processing of 250,000 pixels. However, its information matrix uses an electrical signal as a single channel for input, and at the same time requires more than ten kilometers of optical fiber as a component, resulting in poor integration and high power consumption.

[0007] Therefore, simpler and higher-performance optical logic devices and optical computing chips still require further innovation and breakthroughs. Summary of the Invention

[0008] Aiming at the above existing problems or deficiencies, to solve the problem that high integration, low power consumption, and high scalability cannot be achieved simultaneously in existing optical convolutional processors for deep learning, the present invention provides a dual-comb based all-optical convolutional processor, which realizes optical convolutional computing through dual-comb beat frequency.

[0009] The present invention adopts the following technical solutions:

[0010] An all-optical convolutional device based on dual combs, comprising a dual optical frequency comb light source, an optical filter, an optical modulator, a photodetector, and an electrical filter.

[0011] The dual optical frequency comb light source includes two lasers, two optical amplifiers, and two on-chip microresonators. One laser, one optical amplifier, and one on-chip microresonator form one path, and there are two paths in total.

[0012] The wavelength range of the lasers is the entire C band. The laser light emitted by the lasers is amplified by the optical amplifiers and then incident into the two on-chip microresonators to generate soliton optical frequency combs; two optical frequency combs A and B are generated respectively in the two paths; after the two optical frequency combs A and B are output, they are respectively connected in series with the two optical filters one by one.

[0013] The central wavelengths of the optical frequency combs A and B are both in the range of 1549 - 1551 nm, the repetition frequency (wavelength difference between adjacent optical frequency combs) is greater than 50 GHz, the repetition frequency difference f △rep (the difference between the repetition frequencies of the two optical frequency combs) is less than 1 GHz, and the intensity of a single wavelength is greater than -25 dBm. Both the optical frequency combs A and B have N 2 (M - N + 1) 2 (N > 3, M > 5, M > N) optical frequency comb teeth to implement the convolution operation of an M×M (M > N) matrix with an N×N matrix.

[0014] There are two optical filters (DWDM), and their filtering bandwidths are both 0.5 - 1.5 nm. After filtering the optical frequency combs A and B with wavelengths covering the entire C band, they are respectively divided into K lights with different wavelengths for output, and the adjacent wavelength intervals are greater than 0.4 nm, K > N 2 (M - N + 1) 2 . The frequency difference ω of the first-path output of the two optical filters △1 is less than 1 GHz, and the frequency difference of the n-th path output is ω △1 +(n - 1)f △rep , n ∈ N 2 (M - N + 1) 2 . After the two optical filters are output, they are connected in series with the two optical modulators.

[0015] The trigger threshold of the optical modulator is -25 dBm, the modulation depth is greater than 3 dB, and the modulation speed is greater than 40 GHz. It intensity-modulates the two optical frequency combs A and B after passing through the optical filters. The intensity modulation method is as follows:

[0016] Flatten the matrix of size N×N by rows first and then by columns and repeat it (M - N + 1) 2 times to obtain a matrix with N 2 (M - N + 1) 2The vector A` of M data is modulated to N frequencies of the optical frequency comb A. 2 (M - N + 1) 2 frequencies.

[0017] The matrix of size M×M is split into (M - N + 1) matrices of size N in a row-by-row and then column-by-column manner with horizontal and vertical step sizes of 1, and arranged in a row-by-row and then column-by-column manner to form a vector B` with N 2 data, and the data of the vector B` is modulated to N 2 frequencies of the optical frequency comb B. 2 (M - N + 1) 2 After the optical frequency combs A and B are output by two optical modulators, the two groups of light are simultaneously coupled into a photodetector. 2 (M - N + 1) 2 frequencies.

[0018] After the optical frequency combs A and B are output by two optical modulators, the two groups of light are simultaneously coupled into a photodetector.

[0019] The photodetector performs optical beat frequency on the two groups of light output by the two optical modulators. The specific beat frequency process is as follows:

[0020] The frequency of the nth tooth of the optical frequency comb A is ω An , and the intensity is E An ; the frequency of the nth tooth of the optical frequency comb B is ω Bn , and the intensity is E Bn . When the teeth with frequencies ω An and ω Bn enter the photodetector simultaneously, a signal with a frequency of ω An - ω Bn and an amplitude of is observed in the frequency spectrum. Therefore, when the modulated optical frequency combs A and B enter the photodetector together, N 2 (M - N + 1) 2 electrical signals with different frequencies are generated in the frequency domain. The intensity of each electrical signal is the product of the intensities of the two corresponding wavelength lights. The signals output by the photodetector are connected in series to an electrical filter.

[0021] The electrical filter filters and sums all the electrical signals received in groups of N according to the calculation requirements of frequency kernel convolution, and finally obtains (M - N + 1) 2 electrical signals. The (M - N + 1) 2 electrical signals are rearranged to obtain the convolution result of the two matrices. 2 The (M - N + 1) electrical signals are rearranged to obtain the convolution result of the two matrices.

[0022] The double-comb assignment of the present invention is as Figure 2, an optical frequency comb A and an optical frequency comb B with an initial center frequency difference of less than 2 nm (center wavelength in the range of 1549 - 1551 nm) and a repetition frequency difference of less than 1 GHz are adopted. The light of the optical frequency comb A is subjected to multi-wavelength filtering through an optical filtering device A and output in K (such as 81) channels, with wavelengths being λ A1 ~λ A81 (the wavelengths can cover the C band), and then its intensity is modulated according to the amplitude of the above vector A` through an optical modulation device A. The light of the optical frequency comb B is subjected to multi-wavelength filtering through an optical filtering device B and output in K (such as 81) channels, with wavelengths being λ B1 ~λ B81 , and the wavelength difference between the kth (k ∈ 1 - 81) tooth of the optical frequency comb B and the corresponding tooth of the optical frequency comb A is

[0023] λ Ak -λ Bk =(k - 1)f △rep +λ A1 -λ B1

[0024] Then its intensity is modulated according to the amplitude of the above vector B` through an optical modulation device B. The two modulated K (such as 81) - channel outputs are passed through an optical detection device together for beat frequency. Due to the beat frequency principle, 81 electrical signals with different frequencies from ω1 to ω 81 are obtained. After summing every 9 electrical signals, 9 values are obtained, and then they are rearranged in a row - first and column - later arrangement to form a 3×3 matrix C, thus obtaining the convolution result of matrices A and B.

[0025] An optical frequency comb uses mode - locked laser to generate ultrashort optical pulses, and the characteristic is that the time interval between adjacent pulse waves is exactly the same. An optical frequency comb is like a ruler or timer with precise scales. However, general instruments use millimeters and milliseconds as units, while an optical frequency comb is more precise than nanometers in length measurement and femtoseconds, even attoseconds in time measurement. The advantage of the dual - comb is that it can calculate the data of the two matrices for convolution through light, which is superior to optoelectronic calculation and electronic calculation, greatly improving the convolution operation speed, reducing calculation loss, and realizing calculation through the beat frequency characteristics of light with different frequencies.

[0026] The present invention adopts a mature and stable optical principle, combines advanced processes in the fields of optics, materials science, and micro-nano processing, and realizes the convolution operation of two matrices based on dual optical frequency combs. This optical convolution structure has excellent parallel processing capabilities. Due to the phase and repetition frequency locking of the soliton frequency comb, signals can be loaded in multiple paths to achieve parallel computing. The convolution kernel and input information are respectively loaded on the two optical frequency combs to achieve time delay; and they are simultaneously coupled into a single-mode optical fiber. Through the beat frequency characteristics of the photodetector, the product calculation of multiple signals can be performed simultaneously, and multiplication is achieved through the coherent interference effect of light; therefore, by loading signals and then through the photodetector, addition can be realized, and finally the convolution of two matrices is achieved.

[0027] In summary, the entire system structure of the present invention is simple, has stable performance, generates less heat, and is suitable for building a large-scale optical neural network structure; at the same time, this device is an all-optical convolution, consumes less time, has low computing consumption, and can be directly integrated into an optical neural network to realize an artificial intelligence system for an all-optical network. Description of the Drawings

[0028] Figure 1 is a schematic diagram of the overall structure of the present invention;

[0029] Figure 2 is the optical frequency comb spectrum and filtering modulation beat frequency diagram of the embodiment. Detailed Embodiment

[0030] The following further describes the present invention in detail with reference to the drawings and embodiments.

[0031] The convolution size in the present invention is the convolution calculation of a matrix A of size N×N and a matrix B of size M×M to obtain a result matrix C of size (M - N + 1) 2 ;

[0032]

[0033] where i, j ∈ M - N + 1.

[0034] In this embodiment, taking the convolution of a 2×2 matrix with a 3×3 matrix as an example, a 2×2 matrix will be obtained, where c 22 = a 11 b 22 + a 12 b 23 + a 21 b 32 + a 22 b 33 , and each matrix has N 2 data participating in a single calculation.

[0035] Among them, the data of matrix A can be assigned to the teeth of the first optical frequency comb through the optical modulator 1, and then the data of matrix B can be assigned to the teeth of the second optical frequency comb through the optical modulator 2.

[0036] For the nth tooth of comb A and the nth tooth of comb B, after beating, a signal with a frequency of ω An -ω Bn , and an intensity of can be observed in the frequency spectrum. Therefore, when every N 2 teeth of optical frequency comb A and optical frequency comb B enter the photodetector simultaneously, there will be N 2 pairs of teeth that can produce observable beating results, and the beating results can be summed to output a single electrical signal c ij .

[0037] Taking a 3×3 convolution kernel to convolve a 5×5 matrix processed into a grayscale image as an example, in the total calculation, first flatten A row by row and column by column and repeat it (5 - 3 + 1)^2 = 9 times to obtain a vector A` with 81 data. Split matrix B into 9 3×3 matrices row by row and column by column with horizontal and vertical step sizes of 1, and arrange them row by row and column by column into a vector B` with 81 data.

[0038] The dual - comb assignment is as Figure 2 . In this embodiment, combs A and B with initial central frequencies of 1550.0000 nm and 1550.0008 nm, repetition frequencies of 100 GHz and 100.1 GHz, and a repetition frequency difference of 100 MHz are used. The light of comb A is filtered by the optical filtering device A for multi - wavelength filtering and output in 81 channels, with wavelengths of λ Ak = 1550.0008 + 0.8000k (m ≠ n) (k is an integer between 1 and 81), and then its intensity is modulated according to the amplitude of the above - mentioned vector A` through the optical modulation device 1. The light of comb B is filtered by the optical filtering device B for multi - wavelength filtering and output in 81 channels, with wavelengths of λ Bk = 1550.0008 + 0.8008k (nm), and the wavelength difference from the kth tooth of comb A corresponding to it is λ Ak - λ Bk = 0.0008k + 0.0008. Then its intensity is modulated according to the amplitude of the above - mentioned vector B` through the optical modulation device 2. The two modulated 81 - channel outputs are passed through an optical detection device together for beating, and frequencies of ω1 to ω 81 are respectively ω k81 electrical signals with different frequencies of 0.1(k + 1)(GHz). After summing every 9 electrical signals, 9 values are obtained, namely 36, 32, 36, 32, 28, 32, 36, 32, 36. Reorganize them into a 3×3 matrix C in the arrangement order of first row then column, and the convolution result of matrix A and matrix B is obtained.

[0039] As can be seen from the above embodiments, the present invention takes into account both the data capacity and the calculation speed of the convolutional device. At the same time, it has a small size, a simple structure, good thermal stability, and less consumables. Its response time is only one-thousandth of that of an electrical convolutional device, specifically reaching more than 40 GHz, and it can be directly integrated into an all-optical system to realize high-speed information processing of an all-optical neural network. The present invention loads the convolution kernel and the input information on two optical frequency combs respectively to achieve time delay, and at the same time couples them into a single-mode optical fiber. The second-order nonlinear effect of a photodetector is used for beating frequency to realize and an electrical filter is used to complete addition, and finally the convolution of two matrices is realized. At the same time, this device performs all-optical calculation, consumes less time, and has less calculation consumption. This convolutional device can be directly integrated into an optical neural network to realize an all-optical network system, which is an important development direction for future optical computing.

Claims

1. An all-optical convolutional device based on dual-combs, characterized in that: It includes a dual optical frequency comb light source, an optical filter, an optical modulator, a photodetector, and an electrical filter; The dual optical frequency comb light source includes two lasers, two optical amplifiers, and two on-chip microresonators. One laser, one optical amplifier, and one on-chip microresonator form one path, with a total of two paths. The wavelength range of the lasers is the entire C band. The laser light emitted by the lasers is amplified by the optical amplifiers and then incident into the two on-chip microresonators to generate soliton optical frequency combs. Two optical frequency combs A and B are generated separately for the two paths. After the two optical frequency combs A and B are output, they are respectively connected in series with the two optical filters one by one; The central wavelengths of optical frequency combs A and B are both in the range of 1549 - 1551 nm, the repetition frequency is greater than 50 GHz, the repetition frequency difference f △rep is less than 1 GHz, and the intensity of a single wavelength is greater than -25 dBm; both optical frequency combs A and B have N 2 (M - N + 1) 2 optical frequency comb teeth, N > 3, M > 5, M > N, to implement the operation of convolving an M×M matrix with an N×N matrix; There are two optical filters, each with a filtering bandwidth of 0.5 - 1.5 nm. After filtering the optical frequency combs A and B with wavelengths covering the entire C band, they are respectively divided into K optical outputs with different wavelengths. The adjacent wavelength interval is greater than 0.4 nm, and K > N 2 (M - N + 1) 2 ; The frequency difference ω of the first - path output of the two optical filters △1 is less than 1 GHz, and the frequency difference of the n - th path output is ω △1 +(n - 1)f △rep , n ∈ N 2 (M - N + 1) 2 ; After the outputs of the two optical filters, they are connected in series with two optical modulators; The trigger threshold of the optical modulator is -25 dBm, the modulation depth is greater than 3 dB, and the modulation speed is greater than 40 GHz. The two optical frequency combs A and B after passing through the optical filter are intensity modulated, and the intensity modulation method is as follows: A matrix of size N×N is flattened row by row and then column by column and repeated (M - N + 1) 2 times to obtain a vector A` with N 2 (M - N + 1) 2 data. The data of the vector A` is modulated onto N 2 (M - N + 1) 2 frequencies of the optical frequency comb A; A matrix of size M×M is split row by row and then column by column into (M - N + 1) 2 matrices of size N 2 and arranged row by row and then column by column into a vector B` with N 2 (M - N + 1) 2 data. The data of the vector B` is modulated onto N 2 (M - N + 1) 2 frequencies of the optical frequency comb B; After the optical frequency combs A and B are output by the two optical modulators, these two groups of light are simultaneously coupled into the photodetector; The photodetector performs optical beat frequency on the two groups of light output by the two optical modulators and outputs the electrical signal after optical beat frequency to the electrical filter; The electric filter filters the electric signals of all received frequencies in groups of every N according to the calculation requirements of frequency kernel convolution, and sums them up. Finally, (M - N + 1) electric signals are obtained. The rearrangement of the (M - N + 1) electric signals is the convolution result of the two matrices. 2 2 2 ​​​ 2. The all-optical convolutional device based on dual combs according to claim 1, wherein: The specific process of the photodetector for realizing optical beat frequency is as follows: The frequency of the nth comb tooth of the optical frequency comb A is ω An , and the intensity is E An ; the frequency of the nth comb tooth of the optical frequency comb B is ω Bn , and the intensity is E Bn . When the comb teeth with frequencies ω An and ω Bn enter the photodetector simultaneously, a signal with a frequency of ω An - ω Bn and an amplitude of is observed in the frequency spectrum; therefore, when the modulated optical frequency combs A and B enter the photodetector together, N 2 (M - N + 1) 2 electrical signals with different frequencies will be generated in the frequency domain. The intensity of each electrical signal is the product of the intensities of the two corresponding wavelength lights. Finally, the electrical signal after optical beat frequency is output from the photodetector to the electrical filter.

3. The all-optical convolutional device based on dual combs according to claim 1, wherein: The filtering bandwidth of the optical filter is 0.6 nm. After filtering the optical frequency comb with a wavelength covering the entire C band, it is divided into 81 optical outputs with different wavelengths, and the adjacent wavelength interval is 0.8 nm; The electrical filter filters and sums the electrical signals of 81 frequencies in groups of 9 according to the frequency. Finally, 9 electrical signals are obtained, and the rearrangement of the 9 electrical signals is the product of two matrices.

Images