Optical Convolution Acceleration Device and Method Based on Time-Domain Talbot Effect

Abstract

The present invention discloses an optical convolution acceleration device and method based on the time-domain Talbot effect. The device includes: an optical pulse generation module configured to generate a first optical pulse cluster, wherein the amplitude of the optical pulses in the first optical pulse cluster represents the data to be convolved, the optical pulses in the first optical pulse cluster have a comb-shaped spectrum, and the comb-shaped spectrum has at least one spectral component; a convolution calculation module configured to edit the spectral components to obtain optical pulses with the target convolution kernel loaded on the spectrum, and perform convolution calculation on the data to be convolved based on the time-domain Talbot effect, and output a second optical pulse cluster; and a balanced photodetector configured to convert the second optical pulse cluster into an electrical signal and output the electrical signal as the convolution result. By representing the data to be convolved with the amplitude of the optical pulses, loading the target convolution kernel onto the spectral components of the optical pulses, and then performing convolution calculation based on the time-domain Talbot effect, the speed of convolution calculation is improved by utilizing the advantages of large bandwidth, low latency, and high speed of photon technology.

Description

Technical Field

[0001] The present invention relates to the technical field of microwave photonics, and particularly to an optical convolution acceleration device and method based on the time-domain Talbot effect. Background Art

[0002] A convolutional neural network is an abstraction and simulation of several basic characteristics of the human brain or biological neural networks, and is a network system with a signal discrimination function formed by a large number of neurons interconnected with each other. The initial purpose of proposing a convolutional neural network was to enable it to solve problems in the same way as the human brain. However, over time, the current focus of convolutional neural networks has become how to use them to complete specific functions. Currently, convolutional neural networks have achieved great success in many research fields and are widely applied in aspects such as speech recognition, machine vision, decision-making, unmanned driving, and artistic creation, solving many difficult problems that are difficult for modern computers to solve. Moreover, with the development of modern technology and the improvement of hardware performance, the role of convolutional neural networks has become increasingly important.

[0003] In the process of implementing the present invention, the inventors found that in the related art, convolutional neural networks based on electricity use CPUs, GPUs, ASICs, or FPGAs for calculation. Although they have been developed for a long time and are relatively mature, with the explosive growth of data volume, convolutional neural networks based on electricity face technical problems such as processing speed, latency, and energy consumption. Summary of the Invention

[0004] In view of this, the main object of the present invention is to provide an optical convolution acceleration device and method based on the time-domain Talbot effect, in order to at least partially solve at least one of the above-mentioned technical problems.

[0005] To achieve the above object, the technical solution of the present invention includes:

[0006] As one aspect of the present invention, there is provided an optical convolution acceleration device based on the time-domain Talbot effect, including:

[0007] An optical pulse generation module, configured to generate a first optical pulse cluster, wherein the amplitude of the optical pulses in the first optical pulse cluster represents the data to be convolved, the optical pulses in the first optical pulse cluster have a comb-shaped spectrum, and the comb-shaped spectrum has at least one spectral component;

[0008] A convolution calculation module, configured to edit the spectral components to obtain optical pulses with a target convolution kernel loaded on the spectrum; and perform convolution calculation on the data to be convolved based on the time-domain Talbot effect, and output a second optical pulse cluster; and

[0009] A balanced photodetector, configured to convert the second optical pulse cluster into an electrical signal and output the electrical signal as a convolution result.

[0010] As another aspect of the present invention, an optical convolution acceleration method based on the time-domain Talbot effect is provided, including:

[0011] Using an optical pulse generation module to generate a first optical pulse cluster, wherein the amplitude of the optical pulses in the first optical pulse cluster represents the data to be convolved, the optical pulses in the first optical pulse cluster have a comb-shaped spectrum, and the comb-shaped spectrum has at least one spectral component;

[0012] Using a convolution calculation module to edit the spectral components to obtain optical pulses with the target convolution kernel loaded on the spectrum; and performing convolution calculation on the data to be convolved based on the time-domain Talbot effect, and outputting a second optical pulse cluster; and

[0013] Using a balanced photodetector to convert the second optical pulse cluster into an electrical signal and output the electrical signal as the convolution result.

[0014] Based on the above technical solutions, the present invention has at least one or some of the following beneficial effects compared with the prior art:

[0015] By representing the data to be convolved with the amplitude of optical pulses, loading the target convolution kernel onto the spectral components of the optical pulses, and then performing convolution calculation on the optical pulses and spectral components based on the time-domain Talbot effect, an optical convolution acceleration device is realized, and the speed of convolution calculation is improved. Description of the Drawings

[0016] Figure 1 is a schematic diagram of an optical convolution acceleration device based on the time-domain Talbot effect provided by an embodiment of the present invention;

[0017] Figure 2 is a schematic diagram of the optical pulse generation module provided by an embodiment of the present invention;

[0018] Figure 3 is a schematic diagram of the convolution calculation module provided by an embodiment of the present invention;

[0019] Figure 4 is a schematic diagram of the principle of generating the time-domain Talbot effect in a dispersive medium provided by an embodiment of the present invention;

[0020] Figure 5 is a schematic diagram of the convolution calculation module provided by another embodiment of the present invention;

[0021] Figure 6 is a schematic diagram of an optical convolution acceleration device including three target convolution kernels provided by an embodiment of the present invention;

[0022] Figure 7 is a schematic diagram of an optical convolution acceleration device provided by another embodiment of the present invention;

[0023] Figure 8 It is a schematic diagram of an optical convolution acceleration device provided by another embodiment of the present invention;

[0024] Figure 9 It is a schematic diagram of an optical convolution acceleration device provided by another embodiment of the present invention;

[0025] Figure 10 It is a schematic diagram of an optical convolution acceleration device provided by another embodiment of the present invention;

[0026] Figure 11a It is the original image of some pictures in the MNIST dataset;

[0027] Figure 11b It is to use the optical convolution acceleration device of the embodiment of the present invention for Figure 11a the convolution result after processing the picture in; and

[0028] Figure 12 It is a flowchart of an optical convolution acceleration method based on the time-domain Talbot effect provided by the embodiment of the present invention. Detailed implementation manners

[0029] The present invention provides an optical convolution acceleration device based on the time-domain Talbot effect, including an optical pulse generation module, a convolution calculation module, and a balanced photodetector.

[0030] The optical pulse generation module is configured to generate a first optical pulse cluster, wherein the amplitude of the optical pulses in the first optical pulse cluster represents the data to be convolved, the optical pulses in the first optical pulse cluster have a comb-shaped spectrum, and the comb-shaped spectrum has at least one spectral component;

[0031] The convolution calculation module is configured to edit the spectral components to obtain optical pulses with the target convolution kernel loaded on the spectrum; and perform convolution calculation on the data to be convolved based on the time-domain Talbot effect, and output a second optical pulse cluster; and

[0032] The balanced photodetector is configured to convert the second optical pulse cluster into an electrical signal and output the electrical signal as the convolution result.

[0033] By representing the data to be convolved with the amplitude of the optical pulse, loading the target convolution kernel onto the spectral components of the optical pulse, and then performing convolution calculation based on the time-domain Talbot effect, an optical convolution acceleration device is realized. Since photon technology has advantages such as large bandwidth, low latency, low power consumption, and immunity to electromagnetic interference, the speed of convolution calculation can be improved.

[0034] The following will make a detailed description of the specific components and structures of the optical convolution acceleration device based on the time-domain Talbot effect of the present invention with reference to the accompanying drawings.

[0035] Specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the spirit of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0036] As Figure 1 shown, the present invention provides an optical convolution acceleration device based on the time-domain Talbot effect, including an optical pulse generation module 1, a convolution calculation module 2, and a balanced photodetector 3.

[0037] The optical pulse generation module 1 is configured to generate a first optical pulse cluster, wherein the amplitude of the optical pulses in the first optical pulse cluster represents the data to be convolved, and the optical pulses in the first optical pulse cluster have a comb-shaped spectrum with at least one spectral component.

[0038] According to an embodiment of the present invention, the first optical pulse cluster may include at least one first optical pulse, and the number of the first optical pulses can be flexibly set according to the data to be convolved. The embodiments of the present invention do not specifically limit the number of the first optical pulses in the first optical pulse cluster.

[0039] The convolution calculation module 2 is configured to edit the spectral components to obtain an optical pulse with a target convolution kernel loaded on the spectrum; and perform convolution calculation on the data to be convolved based on the time-domain Talbot effect, and output a second optical pulse cluster.

[0040] According to an embodiment of the present invention, by using the convolution calculation module 2, the target convolution kernel can be loaded onto the spectral components of the first optical pulse cluster, and since the amplitude of the first optical pulse cluster represents the data to be convolved, thus, based on the time-domain Talbot effect, the target convolution kernel can be used to perform optical convolution calculation on the data to be convolved.

[0041] The balanced photodetector 3 is configured to convert the second optical pulse cluster into an electrical signal and output the electrical signal as the convolution result.

[0042] According to an embodiment of the present invention, referring to Figure 2 , the optical pulse generation module 1 includes a mode-locked laser 11, an arbitrary waveform generator 12, and an electro-optic modulator 13.

[0043] The mode-locked laser 11 is configured to generate an initial optical pulse cluster with a comb-shaped spectrum.

[0044] According to an embodiment of the present invention, the mode-locked laser 11 can shape the spectrum of the optical pulse cluster to generate an initial optical pulse cluster with a comb-shaped spectrum.

[0045] According to an embodiment of the present invention, the mode-locked laser 11 can be replaced by a combination of a continuous laser and an electro-optic modulator 13 modulated by an electrical pulse signal, but not limited thereto. The mode-locked laser 11 can also be replaced by a directly modulated laser modulated by an electrical pulse signal.

[0046] An arbitrary waveform generator 12, configured to generate a voltage waveform corresponding to the data to be convolved.

[0047] According to an embodiment of the present invention, the arbitrary waveform generator 12 can be replaced by a programmable pulse generator (PPG), a combination of any one or more of a field programmable gate array (FPGA), a central processing unit (CPU), a graphics processing unit (GPU), and an application specific integrated circuit (ASIC) and an analog-to-digital converter, but not limited thereto. It can also be a combination of other units capable of performing logical operations and a digital-to-analog converter or other devices capable of generating arbitrary waveforms.

[0048] An electro-optic modulator 13, configured to load the voltage waveform onto the initial optical pulse cluster, so that the amplitude of the initial optical pulse cluster represents the data to be convolved, thereby generating a first optical pulse cluster.

[0049] According to an embodiment of the present invention, the mode-locked laser 11 and the electro-optic modulator 13 can be connected by an optical fiber jumper.

[0050] According to an embodiment of the present invention, the arbitrary waveform generator 12 and the electro-optic modulator 13 can be connected by a cable.

[0051] According to an embodiment of the present invention, the mode-locked laser 11 can generate an initial optical pulse cluster with a comb-shaped spectrum; then the data to be convolved is input into the arbitrary waveform generator 12, and the arbitrary waveform generator 12 generates a voltage waveform corresponding to the data to be convolved; then the initial optical pulse cluster and the voltage waveform generated by the arbitrary waveform generator 12 are input into the electro-optic modulator 13, so that the electro-optic modulator 13 can modulate the amplitude of the initial optical pulse cluster according to the voltage waveform corresponding to the data to be convolved, so as to load the data to be convolved onto the amplitude of the optical pulse cluster.

[0052] According to an embodiment of the present invention, referring to Figure 3 , the convolution calculation module 2 includes a dispersion medium 21 and a waveform shaper 22.

[0053] The dispersion medium 21, configured to generate a time-domain Talbot effect to delay-align different spectral components of different optical pulses in the first optical pulse cluster, generating a third optical pulse cluster.

[0054] According to an embodiment of the present invention, the dispersion medium 21 can realize the delay alignment of different spectral components in different optical pulses by generating a time-domain Talbot effect, and the process of delay alignment is the process of realizing convolution calculation.

[0055] The waveform shaper 22 is configured to shape the spectral components of the third optical pulse cluster so that the target convolution kernel is loaded onto the third optical pulse cluster, and perform beam splitting according to the convolution kernel represented by each spectral component in the third optical pulse cluster and the positive or negative value of the value in the convolution kernel.

[0056] According to an embodiment of the present invention, the number of target convolution kernels can be one or more.

[0057] According to an embodiment of the present invention, the second optical pulse cluster includes the cluster spectral components of the second optical pulses representing the non-negative values in the convolution kernel and the spectral components of the second optical pulse cluster representing the negative values in the convolution kernel; the output second optical pulse cluster includes: outputting the cluster spectral components of the second optical pulses representing the non-negative values in the convolution kernel and the spectral components of the second optical pulse cluster representing the negative values in the convolution kernel based on different output ports.

[0058] According to an embodiment of the present invention, the principle of generating the time-domain Talbot effect in the dispersion medium 21 is as Figure 4 shown. As an example, Figure 4 the spectrum of the initial optical pulse cluster generated by the mode-locked laser 11 in includes four spectral components. However, it should be noted that in the actual application process, the number of spectral components can be determined according to the number of parameters in the convolution kernel and the number of convolution kernels. For example, when the target convolution kernel includes two 3x3 convolution kernels, the spectrum of the initial optical pulse cluster can include 18 spectral components.

[0059] As an example, the following takes the optical pulse generation module 1 generating the first optical pulse containing only four spectral components as an example to introduce the delay alignment and calculation process of the optical pulse in the dispersion medium 21.

[0060] Figure 4 In X i1 to X i4 respectively represent the first optical pulse 1 to the first optical pulse 4 input into the dispersion medium 21. The amplitude of the first optical pulse represents the data to be convolved. X o1 to X o4 represent the third optical pulse 1 to the third optical pulse 4 output from the dispersion medium 21. W 1 to W 4 represent the four spectral components of the first optical pulse cluster.

[0061] After generating the time-domain Talbot effect in the dispersion medium 21, different spectral components from the optical pulses X i1 ~X i4 are respectively delayed by 0 to 3 pulse repetition periods due to the dispersion effect. Then, when the third optical pulse cluster is output from the dispersion medium 21, the amplitude of X o1 is the superposition of different spectral components from the initial optical pulse 1 to the initial optical pulse 4, that is: Xo1 = X i1 W 1 + X i2 W 2 + X i3 W 3 + X i4 W 4 .

[0062] The third optical pulse cluster output by the dispersion medium 21 is input to the waveform shaper 22. First, the waveform shaper 22 shapes the spectral components of the third optical pulse cluster, so that the target convolution kernel can be loaded onto the spectral components of the third optical pulse cluster to represent the target convolution kernel through optical information. For example, the target convolution kernel can be a 2x2 convolution kernel The waveform shaper 22 shapes the four spectral components W 1 to W 4 of the third optical pulse cluster according to the absolute values of the parameters in the 2x2 convolution kernel. Thus, W 1 can represent the absolute value 2 of the parameter -2 in the first row and first column of the convolution kernel, W 2 can represent the absolute value 2 of the parameter -2 in the first row and second column of the convolution kernel, and so on. W 4 can represent the parameter 1 in the second row and second column of the convolution kernel.

[0063] In the above formula X ol = X il W 1 + X i2 W 2 + X i3 W 3 + X i4 W 4 , for example, X il to X i4 are 1, 2, 3, and 4 respectively. After the waveform shaper 22 shapes the spectral components of the third optical pulse cluster, W 1 to W 4 represent 2, 2, 1, and 1 respectively. Therefore, the above formula can be expressed as X o1 = 1×2 + 2×2 + 3×1 + 4×1. Among them, the sign of the negative value in the convolution kernel is realized by taking the difference between the spectral components representing non-negative values and negative values in the balanced photodetector 3.

[0064] After the waveform shaper 22 shapes the spectral components of the third optical pulse cluster, the waveform shaper 22 can determine from which port to output this spectral component according to the positive or negative value of the median of the convolution kernel represented by the spectral component and the convolution kernel to which the spectral component belongs. The spectral components from the same convolution kernel are output from two ports. Among them, the spectral components representing the negative values in the convolution kernel are output from one port, and the spectral components representing the non-negative values in the convolution kernel are output from the other port. Taking the target convolution kernel including the above 2x2 convolution kernel and the third optical pulse cluster including four spectral components W 1 to W 4 as an example, the values in the convolution kernel represented by the spectral components W 1 and W 2 are negative, and the values in the convolution kernel represented by W 3 and W 4 are non-negative. Then W 1 and W 2 are output from the first port, and W 3 and W 4 are output from the second port. If the target convolution kernel includes another 2x2 convolution kernel in addition to the above 2x2 convolution kernel, among the four spectral components corresponding to the other 2x2 convolution kernel, the spectral components representing the negative values in the convolution kernel are output from the third port, and the spectral components representing the non-negative values in the convolution kernel are output from the fourth port.

[0065] According to an embodiment of the present invention, the spectral components of the negative second optical pulse cluster may include W 1 and W 2 ; the spectral components of the non-negative second optical pulse cluster may include W 3 and W 4 .

[0066] According to an embodiment of the present invention, the waveform shaper 22 can be used to shape the spectral components and can also be used to split the spectral components. Specifically, the waveform shaper 22 can shape the spectral components of the comb-shaped spectrum, so as to load the target convolution kernel onto the spectrum; another function of the waveform shaper 22 is to split different spectral components of the comb-shaped spectrum according to the target convolution kernel and the positive or negative value of the value in the target convolution kernel. Among them, the number of output ports of the waveform shaper 22 is twice the number of target convolution kernels. The spectral components representing a target convolution kernel are output from two ports of the waveform shaper 22 according to the positive or negative value of the value in the convolution kernel, with negative values output from one port and non-negative values output from one port.

[0067] According to another embodiment of the present invention, referring to Figure 5 , the convolution calculation module 2 includes a waveform shaper 22, a dispersion medium 21, and a beam splitter 23.

[0068] The waveform shaper 22 is configured to shape the spectral components of the first optical pulse cluster to load the target convolution kernel onto the first optical pulse cluster and output the optical pulse cluster to be processed.

[0069] The dispersion medium 21 is configured to generate a time-domain Talbot effect to delay-align different spectral components of different optical pulses from different optical pulse clusters to be processed, and generate the fourth optical pulse cluster.

[0070] The beam splitter 23 is configured to split the fourth optical pulse cluster loaded with the target convolution kernel according to the spectral components based on the parameters in the target convolution kernel, and output the second optical pulse cluster.

[0071] The second optical pulse cluster includes the spectral components of the second optical pulse cluster representing the non-negative values in the convolution kernel and the spectral components of the second optical pulse cluster representing the negative values in the convolution kernel; outputting the second optical pulse cluster includes: outputting the spectral components of the second optical pulse cluster representing the non-negative values in the convolution kernel and the spectral components of the second optical pulse cluster representing the negative values in the convolution kernel based on different output ports.

[0072] The waveform shaper 22 can shape the spectral components of the first optical pulse cluster, so that the target convolution kernel can be loaded onto the spectral components of the first optical pulse cluster to represent the target convolution kernel by optical information. For example, the target convolution kernel can be a 2x2 convolution kernel. The waveform shaper 22 shapes the four spectral components W 1 to W 4 of the first optical pulse cluster according to the absolute values of the 2x2 convolution kernel parameters, so that, W 1 can represent the absolute value 2 of the parameter -2 in the first row and first column of the convolution kernel, W 2 can represent the absolute value 2 of the parameter -2 in the first row and second column of the convolution kernel, and so on, W 4 can represent the parameter 1 in the second row and second column of the convolution kernel.

[0073] After the waveform shaper 22 shapes the spectral components of the first optical pulse cluster, the generated optical pulse cluster to be processed is input into the dispersion medium 21. After the time-domain Talbot effect is generated in the dispersion medium 21, the spectral components W 1 to W 4 of the optical pulse cluster to be processed are respectively delayed by 0 to 3 pulse repetition periods due to the dispersion effect to implement the convolution calculation and generate the fourth optical pulse cluster.

[0074] The beam splitter 23 can determine from which port to output this spectral component according to the positive or negative value of the median of the convolution kernel represented by the spectral component and the convolution kernel to which the spectral component belongs. The spectral components from one convolution kernel are output from two ports, the spectral components representing the negative values in the convolution kernel are output from one port, and the spectral components representing the non-negative values in the convolution kernel are output from the other port. Taking the target convolution kernel including the above-mentioned 2x2 convolution kernel and the third optical pulse cluster including four spectral components W 1 to W 4 as an example, the values in the convolution kernel represented by the spectral components W 1 and W 2 are negative values, and the values in the convolution kernel represented by W 3 and W 4 are non-negative values. Then W 1 and W 2 are output from the first port, and W 3 and W 4 are output from the second port. If the target convolution kernel includes another 2x2 convolution kernel in addition to the above-mentioned 2x2 convolution kernel, among the four spectral components corresponding to the other 2x2 convolution kernel, the spectral components representing the negative values in the convolution kernel are output from the third port, and the spectral components representing the non-negative values in the convolution kernel are output from the fourth port.

[0075] According to an embodiment of the present invention, the spectral components of the negative second optical pulse cluster may include W 1 and W 2 ; the spectral components of the non-negative second optical pulse cluster may include W 3 and W 4 .

[0076] According to an embodiment of the present invention, the number of balanced photodetectors 3 matches the number of target convolution kernels; each balanced photodetector 3 includes a first input port and a second input port. Among them, the first input port is configured to receive the spectral components representing the non-negative second optical pulse cluster in one convolution kernel, and the second input port is configured to receive the spectral components representing the negative values in the same convolution kernel. Alternatively, the first input port is configured to receive the spectral components representing the non-positive second optical pulse cluster in the convolution kernel, and the second input port is configured to receive the spectral components representing the positive second optical pulse cluster in the convolution kernel.

[0077] According to an embodiment of the present invention, referring to Figure 4 , for example, the spectral components of the non-negative third optical pulse cluster include W 1 and W 2 , the spectral components of the negative third optical pulse cluster include W 3 and W 4 , and the first input port of the balanced photodetector 3 receives X il W 1 and Xi2 W 2 At the second input port of the balanced photodetector 3, X is received i3 W 3 and X i4 W 4 The output of the balanced photodetector is Xo 1 =-X i1 W 1 -Xi 2 W 2 +X i3 W 3 +X i4 W 4 .

[0078] According to an embodiment of the present invention, the number of balanced photodetectors 3 can be the same as the number of target convolution kernels. Refer to Figure 6 , Figure 6 FIG. is a schematic diagram of an optical convolution acceleration device when including three target convolution kernels

[0079] According to an embodiment of the present invention, the balanced photodetector 3 is configured to subtract the spectral components of the non-negative second optical pulse cluster from the spectral components of the negative second optical pulse cluster, convert it into an electrical signal, and output the electrical signal as the convolution result

[0080] According to an embodiment of the present invention, the subtraction of the spectral components of the second optical pulse cluster representing non-negative values in the convolution kernel from the spectral components of the second optical pulse cluster representing negative values in the convolution kernel in the balanced photodetector 3 can be expressed by the following formula

[0081] X o1 =α(X i1 W 1 +X i2 W 2 -X i3 W 3 -X i4 W 4 );

[0082] wherein, α is the response of the balanced photodetector 3, and X o1 represents the optical calculation result of the data to be convolved and the target convolution kernel

[0083] According to an embodiment of the present invention, the balanced photodetector 3 can be replaced by a combination of two photodetectors with the same characteristics and a differentiator, or can also be replaced by a combination of two photodetectors with the same characteristics, an analog-to-digital converter, and a logic operation unit. After performing analog-to-digital conversion on the data collected by the two photodetectors, the logic operation unit is used for subtraction

[0084] According to an embodiment of the present invention, the initial optical pulse train generated by the mode-locked laser has a pulse repetition period. Among them, the dispersion coefficient and length of the dispersion medium 21 and the pulse repetition period of the initial optical pulse train generated by the mode-locked laser 11 satisfy the conditions of the time-domain Talbot effect:

[0085]

[0086] where s is an arbitrary natural number, T is the pulse repetition period, is the dispersion coefficient, and L is the length of the dispersion medium 21.

[0087] According to an embodiment of the present invention, the dispersion medium 21 is any one of a dispersion compensation fiber, a chirped fiber grating, an ordinary single-mode fiber, or a multimode fiber.

[0088] According to an embodiment of the present invention, the optical convolution acceleration device based on the time-domain Talbot effect further includes an electrical amplifier 4.

[0089] According to an embodiment of the present invention, referring to Figure 7 , an electrical amplifier 4 can be connected between the electro-optic modulator 13 and the arbitrary waveform generator 12 to increase the driving voltage of the electro-optic modulator 13.

[0090] According to an embodiment of the present invention, the optical convolution acceleration device based on the time-domain Talbot effect further includes any number of optical amplifiers 5.

[0091] According to an embodiment of the present invention, referring to Figure 8 , an optical amplifier 5 can be connected between the dispersion medium 21 and the waveform shaper 22, but not limited thereto. The optical amplifier 5 can be connected to the output end of any device that can output optical pulses to increase the optical power.

[0092] According to an embodiment of the present invention, the optical convolution acceleration device based on the time-domain Talbot effect further includes any number of optical filters 6.

[0093] According to an embodiment of the present invention, referring to Figure 9 , an optical filter 6 can be connected to the output end of the electro-optic modulator 13, but not limited thereto. The optical filter 6 can be connected to the output end of any device that can output optical pulses to filter out unnecessary spectral components.

[0094] According to an embodiment of the present invention, referring to Figure 10 , the optical convolution acceleration device based on the time-domain Talbot effect further includes a radio frequency biaser 7.

[0095] According to an embodiment of the present invention, a radio frequency biaser 7 can be connected between the arbitrary waveform generator 12 and the electro-optic modulator 13 to change the range of the voltage driving the electro-optic modulator 13.

[0096] As shown in FIG. 11, Figure 11a is the original image of some pictures in the MNIST dataset, Figure 11b is the convolution result after processing the pictures in using the optical convolution acceleration device of the embodiment of the present invention and using Figure 11a as the convolution kernel.

[0097] The optical convolution accelerator based on the time-domain Talbot effect provided by the present invention uses the amplitudes of the spectral components of the comb spectrum to represent the values in the convolution kernel and has the ability of full reconfigurability. Without changing the structure of the accelerator, functions such as changing the convolution kernel and changing the size of the convolution kernel can be achieved only by editing the spectrum in the waveform shaper 22. In addition, the optical convolution accelerator based on the time-domain Talbot effect provided by the embodiment of the present invention utilizes the advantages of large bandwidth and high speed of light. Through time, space, and frequency multiplexing, multiple convolution kernels can perform convolution calculations simultaneously in the same convolution accelerator, having the advantages of parallelism and high speed.

[0098] On the other hand, an embodiment of the present invention provides an optical convolution acceleration method based on the time-domain Talbot effect. Referring to Figure 12 , the optical convolution acceleration method includes operations S1201 to S1203.

[0099] In operation S1201, a first optical pulse cluster is generated by using the optical pulse generation module 1. Among them, the amplitude of the optical pulses in the first optical pulse cluster represents the data to be convolved, and the optical pulses in the first optical pulse cluster have a comb spectrum, and the comb spectrum has at least one spectral component.

[0100] In operation S1202, the spectral components are edited by using the convolution calculation module 2 to obtain optical pulses with the target convolution kernel loaded on the spectrum; and convolution calculation is performed on the data to be convolved based on the time-domain Talbot effect, and a second optical pulse cluster is output.

[0101] In operation S1203, the second optical pulse cluster is converted into an electrical signal by using the balanced photodetector 3, and the electrical signal is output as the convolution result.

[0102] By representing the data to be convolved with the amplitude of the optical pulse, loading the target convolution kernel onto the spectral components of the optical pulse, and then performing convolution calculation on the optical pulse and the spectral components based on the time-domain Talbot effect, an optical convolution acceleration device is realized. Since photon technology has the advantages of large bandwidth, low latency, low power consumption, and immunity to electromagnetic interference, the speed of convolution calculation can be improved.

[0103] It should be noted that in the embodiments of the present invention, the part of the optical convolution acceleration method based on the time-domain Talbot effect corresponds to the part of the optical convolution acceleration device based on the time-domain Talbot effect. For the description of the part of the optical convolution acceleration method, please specifically refer to the part of the optical convolution acceleration device based on the time-domain Talbot effect, and will not be elaborated here.

[0104] The above specific embodiments further elaborate on the purpose, technical solutions, and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. An optical convolution acceleration device based on the time-domain Talbot effect, comprising: An optical pulse generation module configured to generate a first optical pulse cluster, wherein the amplitude of the optical pulses in the first optical pulse cluster represents the data to be convolved, the optical pulses in the first optical pulse cluster have a comb-shaped spectrum, and the comb-shaped spectrum has at least one spectral component; A convolution calculation module configured to edit the spectral components to obtain optical pulses with the target convolution kernel loaded on the spectrum; and perform convolution calculation on the data to be convolved based on the time-domain Talbot effect, and output a second optical pulse cluster; the convolution calculation module includes: A dispersive medium configured to generate the time-domain Talbot effect to delay-align different spectral components of different optical pulses in the first optical pulse cluster to generate a third optical pulse cluster; A waveform shaper configured to shape the spectral components of the third optical pulse cluster so that the target convolution kernel is loaded on the third optical pulse cluster, and split the beam according to the convolution kernel represented by each spectral component in the third optical pulse cluster and the positive and negative values of the convolution kernel values to output the second optical pulse cluster; and A balanced photodetector configured to convert the second optical pulse cluster into an electrical signal and output the electrical signal as the convolution result; wherein the second optical pulse cluster includes the spectral components of the second optical pulse cluster representing the non-negative values in the convolution kernel and the spectral components of the second optical pulse cluster representing the negative values in the convolution kernel; Said outputting the second optical pulse cluster includes: Outputting the spectral components of the second optical pulse cluster representing the non-negative values in the convolution kernel and the spectral components of the second optical pulse cluster representing the negative values in the convolution kernel based on different output ports.

2. The optical convolution acceleration device according to claim 1, wherein, The optical pulse generation module includes: A mode-locked laser configured to generate an initial optical pulse cluster with a comb-shaped spectrum; An arbitrary waveform generator configured to generate a voltage waveform corresponding to the data to be convolved; An electro-optic modulator configured to load the voltage waveform onto the initial optical pulse cluster so that the amplitude of the optical pulses in the initial optical pulse cluster represents the data to be convolved, thereby generating the first optical pulse cluster.

3. The optical convolution acceleration device according to claim 1, wherein, The convolution calculation module includes: A waveform shaper configured to shape the spectral components of the first optical pulse cluster to load the target convolution kernel onto the first optical pulse cluster and output a to-be-processed optical pulse cluster; A dispersive medium configured to generate the time-domain Talbot effect to delay-align different spectral components of different optical pulses in the to-be-processed optical pulse cluster to generate a fourth optical pulse cluster; A beam splitter configured to split the fourth optical pulse cluster loaded with the target convolution kernel based on the parameters in the target convolution kernel according to the spectral components and output the second optical pulse cluster.

4. The optical convolution acceleration device according to claim 1, wherein, The number of the balanced photodetectors matches the number of the target convolution kernels; Each of the balanced photodetectors includes a first input port and a second input port. Among them, the first input port is configured to receive the spectral components of a second optical pulse representing non-negative values in the convolution kernel, and the second input port is configured to receive the spectral components of a second optical pulse representing negative values in the convolution kernel; or the first input port is configured to receive the spectral components of a second optical pulse representing non-positive values in the convolution kernel, and the second input port is configured to receive the spectral components of a second optical pulse representing positive values in the convolution kernel.

5. The optical convolution acceleration device according to claim 4, wherein, the balanced photodetector is configured to subtract the spectral components of the second optical pulse cluster representing non-negative values in the convolution kernel from the spectral components of the second optical pulse cluster representing negative values in the convolution kernel, convert the result into an electrical signal, and output the electrical signal as the convolution result.

6. The optical convolution acceleration device according to claim 2, the initial optical pulse generated by the mode-locked laser has a pulse repetition period; wherein, the dispersion coefficient and length of the dispersion medium satisfy the conditions of the time-domain Talbot effect with the pulse repetition period of the initial optical pulse generated by the mode-locked laser: ； where s is an arbitrary natural number, T is the pulse repetition period, is the dispersion coefficient, and is the length of the dispersion medium.

7. The optical convolution acceleration device according to claim 6, wherein: the dispersion medium is any one of dispersion compensation fiber, chirped fiber grating, ordinary single-mode fiber or multi-mode fiber.

8. An optical convolution acceleration method based on the time-domain Talbot effect, implemented by using the optical convolution acceleration device according to any one of claims 1 to 7. The optical convolution acceleration method includes: generating a first optical pulse cluster by using an optical pulse generation module. Among them, the amplitude of the optical pulses in the first optical pulse cluster represents the data to be convolved. The optical pulses in the first optical pulse cluster have a comb-shaped spectrum, and the comb-shaped spectrum has at least one spectral component; editing the spectral components by using a convolution calculation module to obtain an optical pulse with a target convolution kernel loaded on the spectrum; and performing convolution calculation on the data to be convolved based on the time-domain Talbot effect, and outputting a second optical pulse cluster; and converting the second optical pulse cluster into an electrical signal by using a balanced photodetector, and outputting the electrical signal as the convolution result.

Images