System for accelerating convolution operation layers in optical neural networks

Abstract

The present disclosure relates to the technical field of photonic computing, and particularly relates to an acceleration system of a convolution operation layer in an optical neural network. The system comprises: a light source array, a first modulator array, a first optical switch, a second modulator, a second optical switch, a delay array, a wavelength division multiplexer and a photodetector; wherein the first modulator array is configured to modulate elements in a convolution kernel matrix into light signals of at least one wavelength respectively; the first optical switch is configured to arrange the first modulated light signals of at least one wavelength according to a preset time sequence to obtain first combined light signals, and repeat the first combined light signals in time according to the number of elements in an input signal matrix; the second modulator is configured to modulate elements in the input signal matrix into the second combined light signals respectively; and the second optical switch is configured to decompose the second combined modulated light signals into the second modulated light signals of at least one wavelength. The present disclosure adopting the above scheme can improve the convenience of system use.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of photonic computing, and in particular, to an acceleration system of a convolution operation layer in an optical neural network. BACKGROUND

[0002] In a conventional electronic-based neural network accelerator, the bandwidth, power consumption and delay are limited by the development of Moore's Law, and the electronic-based neural network accelerator is challenged in terms of both delay and power consumption. Most of the power consumption in the electronic-based accelerator is generated in linear operation. In comparison, photonic devices are also used to accelerate the process of linear operation due to their higher bandwidth and parallelism.

[0003] In the related art, when a photonic device is used for convolution operation in an optical neural network, the convolution operation can only realize sliding in one dimension of a two-dimensional image (2x2). If sliding in another dimension is to be realized, another modulator channel needs to be added, the entire system needs to be duplicated, and the sliding in the other dimension of the two-dimensional image needs to be synchronized and parallel. If three-dimensional image (3x3) convolution operation is to be realized, the system needs to be duplicated again to process the image information of the third dimension, which increases the complexity of the system, reduces the flexibility of the system, and is not conducive to system expansion, thereby reducing the convenience of using the system. SUMMARY

[0004] The present disclosure provides an acceleration system of a convolution operation layer in an optical neural network, which mainly aims to improve the convenience of using the system.

[0005] According to an aspect of the present disclosure, an acceleration system of a convolution operation layer in an optical neural network is provided, comprising: a light source array, a first modulator array, a first optical switch, a second modulator, a second optical switch, a delay array, a wavelength division multiplexer and a photodetector; wherein,

[0006] The light source array is configured to output at least one wavelength of optical signal.

[0007] The first modulator array is connected with the light source array and configured to modulate elements in a convolution kernel matrix into the at least one wavelength of optical signal to obtain at least one wavelength of first modulated optical signal.

[0008] The first optical switch is connected with the first modulator array and configured to arrange the at least one wavelength of first modulated optical signal according to a preset time sequence to obtain first combined optical signal, and repeat the first combined optical signal in time according to the number of elements in an input signal matrix to obtain second combined optical signal.

[0009] The second modulator is connected with the first optical switch, and is used for modulating elements in the input signal matrix into the second combined optical signal respectively to obtain a second combined modulation optical signal;

[0010] The second optical switch is connected with the second modulator, and is used for decomposing the second combined modulation optical signal into second modulation optical signals of at least one wavelength;

[0011] The delay array is connected with the second optical switch, and is used for adjusting output time of the second modulation optical signals of at least one wavelength to obtain third modulation optical signals of at least one wavelength;

[0012] The wavelength division multiplexer is connected with the delay array, and is used for combining and outputting the third modulation optical signals of at least one wavelength into a third combined optical signal;

[0013] The photodetector is connected with the wavelength division multiplexer, and is used for extracting at least one target combined optical signal from the third combined optical signal, and performing photoelectric conversion on the at least one target combined optical signal to obtain an electrical signal set corresponding to the convolution result.

[0014] Optionally, in an embodiment of the present disclosure, the light source array includes N lasers, the first modulator array includes N first modulators, the first optical switch is an Nx1 optical switch, the second optical switch is a 1xN optical switch, the delay array includes N delay lines, and N=i 2 , i is a non-zero integer; wherein,

[0015] The laser corresponds to the first modulator one by one, the first modulator corresponds to the input end of the first optical switch one by one, and the wavelengths of the optical signals output by each laser are different;

[0016] The output end of the second optical switch corresponds to the delay array one by one, and the wavelengths of the second modulation optical signals output by each output end of the second optical switch are different;

[0017] The number of elements in the convolution kernel matrix is N, and the elements in the convolution kernel matrix correspond to the optical signals one by one.

[0018] Optionally, in an embodiment of the present disclosure, the first modulator is an amplitude modulator, and the delay time corresponding to the delay line is adjustable.

[0019] Optionally, in an embodiment of the present disclosure, the modulation rate corresponding to the second modulator is the ratio between the modulation rate corresponding to the first modulator and the number of the first modulators.

[0020] Optionally, in an embodiment of the present disclosure, the second modulator is a Mach-Zehnder modulator.

[0021] Optionally, in an embodiment of the present disclosure, the number of elements in the input signal matrix is not less than the number of elements in the convolution kernel matrix.

[0022] Optionally, in an embodiment of the present disclosure, the light source array, the first modulator array, the first optical switch, the second modulator, the second optical switch are integrated in a first chip, the delay array is a separate device, and the wavelength division multiplexer and the photoelectric detector are integrated in a second chip.

[0023] Alternatively,

[0024] The light source array, the first modulator array, the first optical switch, the second modulator, the second optical switch, the delay array, the wavelength division multiplexer and the photoelectric detector are integrated in a third chip.

[0025] Optionally, in an embodiment of the present disclosure, the number of elements in the input signal matrix is the same as the number of times of repetition of the first combined optical signal in the second combined optical signal.

[0026] Optionally, in an embodiment of the present disclosure, the first modulated optical signal of each wavelength in the first combined optical signal corresponds to a preset time length.

[0027] The time length corresponding to the first combined optical signal is one bit.

[0028] Optionally, in an embodiment of the present disclosure, the elements in the input signal matrix correspond to the first combined optical signal in the second combined optical signal one by one.

[0029] In one or more embodiments of the present disclosure, an acceleration system of a convolution operation layer in an optical neural network includes an optical source array, a first modulator array, a first optical switch, a second modulator, a second optical switch, a delay array, a wavelength division multiplexer, and a photodetector; wherein the optical source array is configured to output optical signals of at least one wavelength; the first modulator array is connected to the optical source array and configured to modulate elements in a convolution kernel matrix into the optical signals of at least one wavelength respectively to obtain first modulated optical signals of at least one wavelength; the first optical switch is connected to the first modulator array and configured to arrange the first modulated optical signals of at least one wavelength according to a preset time sequence to obtain a first combined optical signal, and repeat the first combined optical signal in time according to the number of elements in an input signal matrix to obtain a second combined optical signal; the second modulator is connected to the first optical switch and configured to modulate elements in the input signal matrix into the second combined optical signal respectively to obtain a second combined modulated optical signal; the second optical switch is connected to the second modulator and configured to decompose the second combined modulated optical signal into second modulated optical signals of at least one wavelength; the delay array is connected to the second optical switch and configured to adjust the output time of the second modulated optical signals of at least one wavelength to obtain third modulated optical signals of at least one wavelength; the wavelength division multiplexer is connected to the delay array and configured to combine and output the third modulated optical signals of at least one wavelength as a third combined optical signal; and the photodetector is connected to the wavelength division multiplexer and configured to extract at least one target combined optical signal from the third combined optical signal and perform photoelectric conversion on the at least one target combined optical signal to obtain an electrical signal set corresponding to a convolution result. Therefore, by repeating the first combined optical signal in time according to the number of elements in the input signal matrix through the first optical switch, and modulating elements in the input signal matrix into the second combined optical signal respectively through the second modulator, multi-dimensional convolution operation can be completed, the number of channels of the modulator can be increased and the entire system can be copied to synchronize and parallelize sliding in another dimension, the complexity of the system can be reduced, the flexibility of the system can be improved, the power consumption of the system can be reduced, and the scalability of the system can be improved, thereby improving the convenience of using the system.

[0030] Additional aspects and advantages of the present disclosure will be made apparent from the following description, which, taken together with the accompanying drawings, describes or illustrates various embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0031] The above and / or additional aspects and advantages of the present disclosure will become apparent and more readily appreciated from the following description, taken in conjunction with the accompanying drawings, in which:

[0032] Figure 1 A structure schematic diagram of an acceleration system of a convolution operation layer in a first optical neural network is shown;

[0033] Figure 2A structural schematic diagram of an acceleration system of a convolution operation layer in a second optical neural network provided by an embodiment of the present disclosure is shown.

[0034] Figure 3 A flowchart of a linear multiplication operation provided by an embodiment of the present disclosure is shown.

[0035] Figure 4 A signal schematic diagram of a second combined optical signal provided by an embodiment of the present disclosure is shown.

[0036] Figure 5 A signal schematic diagram of an input signal provided by an embodiment of the present disclosure is shown.

[0037] Figure 6 A signal schematic diagram of a second combined modulation optical signal provided by an embodiment of the present disclosure is shown.

[0038] Figure 7 A flowchart of a summation operation provided by an embodiment of the present disclosure is shown.

[0039] Figure 8 A convolution calculation process schematic diagram provided by an embodiment of the present disclosure is shown.

[0040] Figure 9 An extraction schematic diagram of a target combined optical signal provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0041] Embodiments of the present disclosure are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numbers represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present disclosure, and cannot be understood as a limitation of the present disclosure. On the contrary, the embodiments of the present disclosure include all changes, modifications and equivalents falling within the spirit and scope of the appended claims.

[0042] In the related art, the convolution operation method in the optical neural network converts the convolution corresponding element multiplication operation into a matrix point multiplication operation. Specifically, first, the input image is preprocessed by padding: the input image is divided into multiple small blocks, and the length of each block is the same as the size of the weight matrix. Then each block is flattened as each row of the matrix, and the column number of the matrix corresponds to the number of blocks of the input image. Finally, the weight matrix is flattened as a vector and multiplied with the input image matrix.

[0043] In some embodiments, when the weight matrix is flattened as a vector and the matrix multiplication operation is performed with the matrix of the input image, the information of the weight matrix can be first loaded onto the optical signal of different frequency components. Then the gray value of the input image is flattened into a one-dimensional vector, and each element in the vector is distributed in different time periods, and the value is the electrical signal of the corresponding time period modulator. Then the optical signal containing frequency components is used as the input signal of the modulator to realize linear operation. Finally, the dispersion optical fiber is used to separate the optical signals of different frequencies, so that the corresponding weight component is multiplied by the corresponding input component, thereby realizing the convolution operation.

[0044] It is easy to understand that the convolution operation in the optical neural network can only realize the sliding of one dimension in the two-dimensional image (2x2). If the sliding of another dimension is to be realized, another channel of the modulator needs to be added, and the entire system needs to be duplicated, synchronized and parallel to realize the sliding of another dimension in the two-dimensional image. If the convolution operation of a three-dimensional image (3x3) is to be realized, the system needs to be duplicated again to process the image information of the third dimension, thereby increasing the complexity of the system and reducing the flexibility of the system, which is not conducive to the expansion of the system.

[0045] The present application will be described in detail below with reference to specific embodiments.

[0046] In the first embodiment, as shown in Figure 1 , Figure 1 Fig. 1 shows a structure schematic diagram of an acceleration system of a convolution operation layer in a first optical neural network provided by the embodiments of the present disclosure.

[0047] Specifically, the acceleration system of the convolution operation layer in the optical neural network comprises a light source array, a first modulator array, a first optical switch, a second modulator, a second optical switch, a delay array, a wavelength division multiplexer and a photodetector.

[0048] The light source array is configured to output at least one wavelength of optical signal.

[0049] The first modulator array is connected with the light source array and configured to modulate the elements in the convolution kernel matrix into the at least one wavelength of optical signal respectively, to obtain at least one wavelength of first modulated optical signal.

[0050] The first optical switch is connected with the first modulator array and configured to arrange the at least one wavelength of first modulated optical signal according to a preset time sequence, to obtain first combined optical signal, and repeat the first combined optical signal in time according to the number of elements in the input signal matrix, to obtain second combined optical signal.

[0051] The second modulator is connected with the first optical switch and configured to modulate the elements in the input signal matrix into the second combined optical signal respectively, to obtain second combined modulated optical signal.

[0052] The second optical switch is connected with the second modulator, and is used for decomposing the second combined modulation optical signal into the second modulation optical signal of at least one wavelength;

[0053] The delay array is connected with the second optical switch, and is used for adjusting the output time of the second modulation optical signal of at least one wavelength to obtain the third modulation optical signal of at least one wavelength;

[0054] The wavelength division multiplexer is connected with the delay array, and is used for combining the third modulation optical signal of at least one wavelength to output as the third combined optical signal;

[0055] The photodetector is connected with the wavelength division multiplexer, and is used for extracting at least one target combined optical signal from the third combined optical signal, and performing photoelectric conversion on the at least one target combined optical signal to obtain an electrical signal set corresponding to the convolution result.

[0056] According to some embodiments, the light source array is not limited to a fixed light source array. For example, the light source array can be a laser array.

[0057] According to some embodiments, the convolution kernel matrix is not limited to a fixed matrix. For example, the convolution kernel matrix can be a 2x2 matrix. The convolution kernel matrix can also be a 3x3 matrix.

[0058] In some embodiments, the input signal matrix is not limited to a fixed matrix. For example, the input signal matrix can be a 2x2 matrix. The input signal matrix can also be a 3x3 matrix.

[0059] In some embodiments, when the second modulator modulates the elements in the input signal matrix into the second combined optical signal to obtain the second combined modulation optical signal, the second combined modulation optical signal includes the result of multiplying each element in the convolution kernel matrix with each element in the input signal matrix, that is, the multiplication operation of the convolution kernel matrix and the input signal matrix in the convolution linear operation is completed (w i x i ), where w i is the i th element in the convolution kernel matrix, and x i is the i th element in the input signal matrix.

[0060] According to some embodiments, the wavelength division multiplexer (Wavelength Division Multiplexing, WDM) refers to an optical passive device for separating and combining optical wavelengths. When the wavelength division multiplexer combines the third modulation optical signal of at least one wavelength to output as the third combined optical signal, the summation operation in the convolution linear operation can be completed

[0061] In the embodiments of the present disclosure, the light source array includes N lasers, the first modulator array includes N first modulators, the first optical switch is an Nx1 optical switch, the second optical switch is a 1xN optical switch, the delay line array includes N delay lines, and N = i 2 , i is a non-zero integer; wherein

[0062] The laser corresponds to the first modulator in one-to-one correspondence, the first modulator corresponds to the input end of the first optical switch in one-to-one correspondence, and the wavelengths of the optical signals output by each laser are different;

[0063] The output end of the second optical switch corresponds to the delay line array in one-to-one correspondence, and the wavelengths of the second modulated optical signals output by each output end of the second optical switch are different;

[0064] The number of elements in the convolution kernel matrix is N, and the elements in the convolution kernel matrix correspond to the optical signals in one-to-one correspondence.

[0065] In the embodiments of the present disclosure, the first modulator is an amplitude modulator, and the delay time corresponding to the delay line is adjustable.

[0066] According to some embodiments, the amplitude modulator (AM) refers to a device that controls the amplitude of the modulated wave to change with the modulation signal. The amplitude modulator does not refer to a specific fixed modulator. For example, the amplitude modulator includes but is not limited to a transistor amplitude modulator, a differential pair amplitude modulator, a ring amplitude modulator, etc.

[0067] In some embodiments, when the first modulator is an amplitude modulator, the first modulator array can be an amplitude modulator array.

[0068] In the embodiments of the present disclosure, the modulation rate corresponding to the second modulator is the ratio between the modulation rate corresponding to the first modulator and the number of first modulators.

[0069] According to some embodiments, by changing the adjustment rate corresponding to at least one first modulator in the first modulator array, the adjustment rate corresponding to the second modulator, and the delay time corresponding to at least one delay line in the delay line array, a reconfigurable optical convolution operation process can be realized.

[0070] In the embodiments of the present disclosure, the second modulator is a Mach-Zehnder modulator.

[0071] According to some embodiments, the Mach-Zehnder modulator (MZM) refers to a device for converting changes in an electrical signal into changes in an optical signal to achieve modulation of optical intensity. The Mach-Zehnder modulator does not refer to a specific fixed modulator. For example, the Mach-Zehnder modulator can be a wide-bandwidth Mach-Zehnder modulator.

[0072] In the embodiments of the present disclosure, the number of elements in the input signal matrix is not less than the number of elements in the convolution kernel matrix.

[0073] According to some embodiments, the elements in the input signal matrix can be gray values of the identified object image.

[0074] In some embodiments, the input signal matrix can be directly obtained by one-dimensional expansion of the identified object image, and the identified object image can not need to be preprocessed, thereby reducing the requirement for input signal data preprocessing in the electrical domain and improving the convenience of system use.

[0075] In the embodiments of the present disclosure, the light source array, the first modulator array, the first optical switch, the second modulator, and the second optical switch are integrated on a first chip, the delay element array is a separate device, and the wavelength division multiplexer and the photodetector are integrated on a second chip.

[0076] Alternatively,

[0077] The light source array, the first modulator array, the first optical switch, the second modulator, the second optical switch, the delay element array, the wavelength division multiplexer, and the photodetector are integrated on a third chip.

[0078] According to some embodiments, when the light source array, the first modulator array, the first optical switch, the second modulator, and the second optical switch are integrated on the first chip, the delay element array is a separate device, and the wavelength division multiplexer and the photodetector are integrated on the second chip, the low-speed convolution operation can be performed by using the acceleration system of the convolution operation layer in the optical neural network.

[0079] In some embodiments, when the light source array, the first modulator array, the first optical switch, the second modulator, the second optical switch, the delay element array, the wavelength division multiplexer, and the photodetector are integrated on the third chip, monolithic integration of the acceleration system of the convolution operation layer in the optical neural network can be implemented, and thereby the high-speed convolution operation can be performed by using the acceleration system of the convolution operation layer in the optical neural network.

[0080] In the embodiments of the present disclosure, the number of elements in the input signal matrix is the same as the number of times of repetition of the first combined optical signal in the second combined optical signal in time.

[0081] In the embodiments of the present disclosure, the time length corresponding to each wavelength of the first modulated optical signal in the first combined optical signal is a preset time length.

[0082] The time length corresponding to the first combined optical signal can be one bit (Bit).

[0083] According to some embodiments, the first combined optical signal is periodically repeated with a preset time length as a period in time, and the wavelength division time division multiplexing function can be implemented.

[0084] In the embodiments of the present disclosure, the elements in the input signal matrix correspond to the first combined optical signals in the second combined optical signals one by one.

[0085] In the embodiments of the present disclosure, Figure 2 A structural schematic diagram of an acceleration system of a convolution operation layer in a second optical neural network provided by the embodiments of the present disclosure is shown. As Figure 2 shown, the acceleration system of the convolution operation layer in the optical neural network includes a laser array composed of four lasers, an amplitude modulator array composed of four amplitude modulators, a 4x1 optical switch, an MZM, a 1x4 optical switch, a delay line array composed of four delay lines, a wavelength division multiplexer, and a photodetector; wherein the four lasers respectively emit laser with wavelength λ1, laser with wavelength λ2, laser with wavelength λ3, and laser with wavelength λ4.

[0086] In some embodiments, when the convolution kernel matrix and the input signal matrix are both 2x2 matrices, the acceleration system of the convolution operation layer in the optical neural network performs convolution operation, including the following steps:

[0087] S101, Figure 3 A flowchart of linear multiplication operation provided by the embodiments of the present disclosure is shown. As Figure 3 shown, the laser array outputs four separate lasers with different wavelengths: laser with wavelength λ1, laser with wavelength λ2, laser with wavelength λ3, and laser with wavelength λ4;

[0088] S102, as Figure 3 shown, different weight signals are applied to each amplitude modulator according to the elements in the convolution kernel matrix, so that the lasers with different wavelengths carry different weight information, obtaining first modulated optical signals with wavelengths λ1, λ2, λ3, and λ4.

[0089] wherein the convolution kernel matrix has four elements, which can be represented by .

[0090] S103, as Figure 3 shown, the four first modulated optical signals with wavelengths λ1, λ2, λ3, and λ4 are distributed in four different time periods t0 by using a 4x1 optical switch, obtaining a first combined optical signal, and the first combined optical signal is repeated four times in time to obtain a second combined optical signal O1. Wherein, Figure 4 A signal schematic diagram of the second combined optical signal provided by the embodiments of the present disclosure is shown. Wherein, 4t0 is one Bit.

[0091] S104, determining the input signal according to the input signal matrix, Figure 5A signal diagram of the input signal provided by the embodiment of the present disclosure is shown. As Figure 5 shown, the duration of each element in the input signal matrix is 4t0.

[0092] The input signal matrix has 4 elements, which can be represented as .

[0093] S105, as Figure 3 shown, the input signal is modulated to the second combined optical signal as the electrical signal of the MZM to obtain the second combined modulation optical signal O2. Wherein, Figure 6 A signal diagram of the second combined modulation optical signal provided by the embodiment of the present disclosure is shown.

[0094] S106, Figure 7 A flow diagram of the summation operation provided by the embodiment of the present disclosure is shown. As Figure 7 shown, the second combined modulation optical signal O2 is used as the input signal of the summation operation, and is decomposed into second modulation optical signals with wavelengths λ1, λ2, λ3 and λ4 through a 1x4 optical switch, and is input into four channels respectively.

[0095] S107, as Figure 7 shown, the linear multiplication operation results of different time instants in the second modulation optical signals with wavelengths λ1, λ2, λ3 and λ4 are normalized to the same time instant by using a delay array, to obtain third modulation optical signals with wavelengths λ1, λ2, λ3 and λ4.

[0096] S108, as Figure 7 shown, the third modulation optical signals with wavelengths λ1, λ2, λ3 and λ4 are added by using a wavelength division multiplexer to perform summation operation, to obtain a third combined optical signal.

[0097] S109, as Figure 7 shown, the target combined optical signal y1 is extracted from the third combined optical signal by using a photodetector, and the target combined optical signal y1 is optoelectronically converted to obtain an electrical signal corresponding to the convolution result.

[0098] In the embodiment of the present disclosure, Figure 8 A convolution calculation process diagram provided by the embodiment of the present disclosure is shown. As Figure 8As shown, the convolution kernel matrix is a 2x2 matrix, the input signal matrix is a 3x3 matrix, the target combined optical signal y1 is the result of convolution calculation of the sub-input signal matrix corresponding to X1, X2, X4 and X5 in the input signal matrix and the convolution kernel matrix; the target combined optical signal y2 is the result of convolution calculation of the sub-input signal matrix corresponding to X2, X3, X5 and X6 in the input signal matrix and the convolution kernel matrix; the target combined optical signal y3 is the result of convolution calculation of the sub-input signal matrix corresponding to X4, X5, X7 and X8 in the input signal matrix and the convolution kernel matrix; and the target combined optical signal y4 is the result of convolution calculation of the sub-input signal matrix corresponding to X5, X6, X8 and X9 in the input signal matrix and the convolution kernel matrix.

[0099] In some embodiments, since the input signal matrix contains 9 elements, the first combined optical signal is periodically repeated nine times in time to obtain the second combined optical signal. Meanwhile, the target combined optical signals y1, y2, y3 and y4 can be extracted from the third combined optical signal by using a photodetector, and the target combined optical signals y1, y2, y3 and y4 are optoelectronically converted to obtain an electrical signal set corresponding to the convolution result. Finally, performing nonlinear operation on the elements in the electrical signal set in the electrical domain can complete the image recognition of the entire optical neural network.

[0100] In some embodiments, Figure 9 An extraction schematic diagram of the target combined optical signal provided by the embodiments of the present disclosure is shown. As shown in the figure, Figure 9 As shown, the convolution result corresponding to y1 is X1w1+X2w2+X4w3+X5w4; the convolution result corresponding to y2 is X2w1+X3w2+X5w3+X6w4; the convolution result corresponding to y3 is X4w1+X5w2+X7w3+X8w4; and the convolution result corresponding to y4 is X5w1+X6w2+X7w3+X8w4.

[0101] In some embodiments, when the target combined optical signals y1, y2, y3 and y4 are extracted from the third combined optical signal by using a photodetector, the target combined optical signals y1, y2, y3 and y4 can be extracted in sequence by adjusting the delay time length corresponding to at least one delay line in the delay line array.

[0102] In summary, the system provided in the embodiments of the present disclosure includes a light source array, a first modulator array, a first optical switch, a second modulator, a second optical switch, a delay array, a wavelength division multiplexer, and a photodetector. The light source array is configured to output light signals of at least one wavelength. The first modulator array is connected to the light source array and configured to modulate elements in a convolution kernel matrix into the light signals of at least one wavelength respectively to obtain first modulated light signals of at least one wavelength. The first optical switch is connected to the first modulator array and configured to arrange the first modulated light signals of at least one wavelength according to a preset time sequence to obtain a first combined light signal, and repeat the first combined light signal in time according to the number of elements in an input signal matrix to obtain a second combined light signal. The second modulator is connected to the first optical switch and configured to modulate elements in the input signal matrix into the second combined light signal respectively to obtain a second combined modulated light signal. The second optical switch is connected to the second modulator and configured to decompose the second combined modulated light signal into second modulated light signals of at least one wavelength. The delay array is connected to the second optical switch and configured to adjust the output time of the second modulated light signals of at least one wavelength to obtain third modulated light signals of at least one wavelength. The wavelength division multiplexer is connected to the delay array and configured to combine and output the third modulated light signals of at least one wavelength as a third combined light signal. The photodetector is connected to the wavelength division multiplexer and configured to extract at least one target combined light signal from the third combined light signal and perform photoelectric conversion on the at least one target combined light signal to obtain an electrical signal set corresponding to a convolution result. Therefore, by using the first optical switch to repeat the first combined light signal in time according to the number of elements in the input signal matrix, and by using the second modulator to modulate elements in the input signal matrix into the second combined light signal respectively, a multi-dimensional convolution operation can be completed, the number of channels of the modulator can be reduced, the entire system can be copied to synchronize and parallelize sliding in another dimension, the complexity of the system can be reduced, the flexibility of the system can be improved, the power consumption of the system can be reduced, and the scalability of the system can be improved, thereby improving the convenience of using the system.

[0103] It should be noted that in the description of the present disclosure, the terms "first", "second", etc. are only for the purpose of description and cannot be understood or implied as indicating or suggesting relative importance. In addition, in the description of the present disclosure, the meaning of "a plurality of" is two or more, unless otherwise specified.

[0104] Any procedural or methodological descriptions in flow charts or otherwise described herein can be understood to represent modules, segments, or portions of code that include executable instructions for implementing the specific logical functions or steps, and the scope of preferred embodiments of the present disclosure includes additional implementations in which the functions are performed in an order different from that shown or discussed, including substantially simultaneously, or in reverse order, as will be understood by those skilled in the art to which the embodiments of the present disclosure pertain.

[0105] It should be understood that portions of the present disclosure can be implemented in hardware, software, firmware, or combinations thereof. In the above-described embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and as in another embodiment, implementation can be in any one or a combination of the following technologies that are known in the art: discrete logic circuitry having logic gates for implementing logic functions upon an application of data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0106] Those skilled in the art can understand that all or part of the steps carried out by the above-described embodiments can be completed by programs instructing relevant hardware, and the programs can be stored in a computer readable storage medium, and when executed, include one or a combination of steps of the method embodiments.

[0107] In addition, each functional unit in various embodiments of the present disclosure can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The above-mentioned integrated module can be realized in the form of hardware or in the form of a software functional module. When the integrated module is realized in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0108] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0109] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0110] Although the embodiments of the present disclosure have been shown and described above, it is understood that the above-described embodiments are exemplary, and it is not construed that the present disclosure is limited to the above-described embodiments, and a person of ordinary skill in the art can make changes, modifications, replacements, and variations to the above-described embodiments within the scope of the present disclosure.

Claims

1. An acceleration system of a convolution operation layer in an optical neural network, characterized in that, The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device.

2. The system of claim 1, wherein, The light source array comprises N lasers, the first modulator array comprises N first modulators, the first optical switch is an Nx1 optical switch, the second optical switch is a 1xN optical switch, and the delay line array comprises N delay lines, N=i 2 , i is a non-zero integer; wherein, The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device.

3. The system of claim 2, wherein, The application relates to a convolutional neural network (CNN) optical computing device.

4. The system of claim 2, wherein, The application relates to a convolutional neural network (CNN) optical computing device.

5. The system of claim 1, wherein, The application relates to a convolutional neural network (CNN) optical computing device.

6. The system of claim 1, wherein, The application relates to a convolutional neural network (CNN) optical computing device.

7. The system of claim 1, wherein, The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device. The application relates to a convolutional neural network (CNN) optical computing device.

8. The system of claim 1, wherein, The number of elements in the input signal matrix is the same as the number of times the first multiplexed optical signal in the second multiplexed optical signal repeats in time.

9. The system of claim 1, wherein, The time length corresponding to each wavelength of the first multiplexed optical signal is a preset time length. The time length corresponding to the first multiplexed optical signal is one bit.

10. The system of claim 1, wherein, The elements in the input signal matrix correspond one-to-one to the first multiplexed optical signal in the second multiplexed optical signal.

Images