Optoelectronic hybrid computing system

Abstract

The invention relates to an opto-electronic hybrid computing system capable of completing accurate matrix vector multiplication computation in a shorter clock period. Comprises: a storage unit storing a first input vector composed of K * b-bit elements and a first weight matrix; the driving unit is used for respectively splitting the first input vector and the first weight matrix into K second input vectors and K second weight matrixes consisting of b-bit elements, and respectively carrying out digital-to-analog conversion on the K second input vectors and the K second weight matrixes into K groups of first electric signals and K groups of second electric signals; and modulating the K groups of first electric signals and the K groups of second electric signals into corresponding groups of first optical signals of the K groups of first optical signals based on wavelength, and performing product accumulation, intra-group summation and photoelectric conversion on the obtained K groups of second optical signals to obtain 2K-1 groups of third electric signals, and the photoelectric hybrid calculation unit is used for carrying out analog-to-digital conversion, shift register and summation on the corresponding third electric signals in each group of third electric signals so as to obtain an output vector.

Description

Technical Field

[0001] The present invention relates to optoelectronic hybrid computing, and in particular to an optoelectronic hybrid computing system that applies optical computing to matrix-vector multiplication. Background Art

[0002] With the emergence of computing-intensive applications like artificial intelligence and virtual reality, accelerating matrix-vector multiplication (MVM) has become increasingly important. However, due to the slowdown of Moore's Law, traditional electronic chips like GPUs are approaching a computing bottleneck, forcing manufacturers to exponentially increase costs to achieve computing power gains. However, model parameters continue to grow exponentially, making it increasingly difficult to increase computing power through the development of traditional electronic chips.

[0003] To this end, in the past, a variety of architectures have been proposed to implement complex operations through optical computing. Optical computing is a new computing paradigm that uses light as a propagation carrier, modulates signals on light, detects them through photodiodes (PDs), and uses the amplified voltage to represent the results of operations. Compared with traditional electrical computing, optical computing has the characteristics of large modulation bandwidth, multiple parallel dimensions, high energy efficiency per symbol, and low transmission loss. It is suitable for application scenarios that require large-scale computing power, such as large model prediction and precision medicine. Currently, there are more than 10 architectures for optical computing, which can realize complex operations such as convolution, MVM, and even tensor multiplication. Summary of the Invention

[0004] Technical problems to be solved by the present invention

[0005] Most existing optoelectronic computing architectures focus on neural network inference tasks with relatively simple data sets, so the requirements for computational accuracy are not high. However, for high-precision MVM scenarios, it is necessary to repeatedly call the optical computing chip and perform high-precision calculations on the electrical chip through summation. As a result, the impact of inter-chip communication delays and weight updates becomes increasingly significant, which can offset the advantages of optical computing and make it impossible to ensure system computing power.

[0006] Furthermore, in existing optoelectronic computing architectures that use a crossbar switch array, a large number of electro-optical modulators are typically required to calculate accurate results within a single clock cycle, which results in a larger computing system and increased costs.

[0007] The present invention is completed in view of the above-mentioned problems, and its purpose is to provide an optoelectronic hybrid computing system that can complete accurate matrix-vector multiplication calculations in fewer clock cycles and reduce the number of required modulators, thereby effectively reducing chip area and power consumption and lowering costs.

[0008] Technical solutions to technical problems

[0009] In order to solve the above problems, the first aspect of the present invention involves an optoelectronic hybrid computing system, which uses a system with b-bit computing accuracy to complete K×b-bit matrix-vector multiplication, where K and b are both positive integers. The optoelectronic hybrid computing system is characterized in that it includes: a storage unit, which stores a first input vector and a first weight matrix composed of K×b-bit elements; a driving unit, which splits the first input vector and the first weight matrix into K second input vectors and second weight matrices composed of b-bit elements, and converts the K second input vectors and The elements of the K second weight matrices are respectively digital-to-analog converted into K groups of first electrical signals and K groups of second electrical signals; and an optoelectronic hybrid computing unit is provided, which modulates the K groups of first electrical signals and the K groups of second electrical signals into the first optical signals of the corresponding groups of the first optical signals divided into K groups based on wavelength, performs multiplication and accumulation, intra-group summation and optoelectronic conversion on the K groups of second optical signals obtained thereby, obtains 2K-1 groups of third electrical signals, performs analog-to-digital conversion, shift registration and summation on the corresponding third electrical signals in each group of the third electrical signals, thereby obtaining the output vector of the matrix-vector multiplication.

[0010] Optionally, the driving unit includes: a data splitting module, which splits the first input vector and the first weight matrix into K second input vectors and K second weight matrices respectively; a time division multiplexing module, which extracts the absolute values ​​of the elements in the second input vector and the second weight matrix with the same signs according to the signs of each element of the second input vector and the second weight matrix in the first clock cycle, and combines them with 0 to reconstruct the second input vector and the second weight matrix, and extracts the absolute values ​​of the elements in the second input vector and the second weight matrix with different signs in the second clock cycle, and combines them with 0 to reconstruct the second input vector and the second weight matrix; and a digital-to-analog conversion module, which converts the reconstructed K second input vectors and K second weight matrices into K groups of first electrical signals and K groups of second electrical signals respectively.

[0011] Optionally, the optoelectronic hybrid computing unit includes: a laser module, which outputs K lasers with different wavelengths; a photon computing module, which divides the K lasers into K groups of first optical signals, the wavelength of the first optical signals in each group being the same, the photon computing module modulates the K groups of first electrical signals and K groups of second electrical signals into the K groups of first optical signals respectively to obtain K groups of second optical signals, performs multiplication and accumulation on the K groups of second optical signals to obtain 2K-1 groups of third optical signals, and performs photoelectric conversion on the 2K-1 groups of third optical signals to obtain 2K-1 groups of fourth electrical signals; an electronic computing module, which performs intra-group summation on the 2K-1 groups of fourth electrical signals to obtain 2K-1 groups of third electrical signals, performs analog-to-digital conversion, shift registration, and summation on the corresponding third electrical signals in each group of the third electrical signals, thereby calculating the product of the matrix vector; and a difference computing module, which subtracts the products calculated in the first clock cycle and the second clock cycle to obtain the output vector.

[0012] Optionally, the photonic computing module includes: a first spectrometer, which splits the laser of each wavelength into a group of first optical signals with the same wavelength; an electro-optical modulator, which modulates the first electrical signal into the first optical signal; a weight controller, which modulates the second electrical signal into the first optical signal modulated with the first electrical signal to obtain the second optical signal; and a multiplication and accumulation device, which multiplies and accumulates the corresponding second optical signals in K groups of the second optical signals between each group to obtain the third optical signal, and converts the third optical signal into the fourth electrical signal.

[0013] Optionally, the product-accumulator comprises: a first demultiplexer, which demultiplexes K groups of the second optical signals to obtain K 2 a first multiplexer that multiplies and accumulates the fourth optical signals except the first and last two groups of fourth optical signals between groups, and combines the fourth optical signals with the first and last two groups of fourth optical signals to form 2K-1 groups of third optical signals; and a first photoelectric converter that converts the third optical signal into the fourth electrical signal.

[0014] Optionally, the optoelectronic hybrid computing unit includes: a laser module, which outputs K groups of lasers with different wavelengths, and the wavelengths of the lasers in each group are different; a photon computing module, which uses the K groups of lasers as the K groups of first optical signals, and modulates the K groups of first electrical signals and the K groups of second electrical signals into the K groups of first optical signals respectively to obtain K groups of second optical signals, performs multiplication and accumulation and intra-group summation on the K groups of second optical signals to obtain 2K-1 groups of third optical signals, and performs photoelectric conversion on the third optical signals to obtain 2K-1 groups of third electrical signals; an electronic computing module, which performs analog-to-digital conversion, shift registration and summation on the corresponding third electrical signals in each group of the third electrical signals, thereby calculating the product of the matrix vector; and a difference calculation module, which subtracts the products calculated by the first clock cycle and the second clock cycle to obtain the output vector.

[0015] Optionally, the photonic computing module includes: an electro-optical modulator, which modulates the first electrical signal into the first optical signal; a weight controller, which modulates the second electrical signal into the first optical signal modulated with the first electrical signal to obtain the second optical signal; and a multiplication and accumulation device, which multiplies and accumulates the corresponding second optical signals in K groups of the second optical signals between each group, performs intra-group summation, obtains the third optical signal, and converts the third optical signal into the third electrical signal.

[0016] Optionally, the product-accumulator comprises: a second demultiplexer, which demultiplexes K groups of the second optical signals to obtain K 2 a second multiplexer, which, for all rows except the last row of the matrix-vector multiplication, performs inter-group summing and multiplication accumulation on the fourth optical signals except the first and last two groups of the fourth optical signals, and the fourth optical signals, together with the first and last two groups of the fourth optical signals, form 2K-1 groups of fifth optical signals; a third multiplexer, which, for the last row of the matrix-vector multiplication, performs intra-group summing and multiplication accumulation on the fourth optical signals except the first and last two groups of the fourth optical signals and the fifth optical signals except the first and last two groups of the fifth optical signals, and the sixth optical signal obtained by intra-group summing of the first and last two groups of the fourth optical signals and the first and last two groups of the fifth optical signals, form 2K-1 groups of third optical signals; and a second photoelectric converter, which converts the third optical signal into the third electrical signal.

[0017] Optionally, the driving unit includes: a data splitting module, which splits the first input vector and the first weight matrix into K second input vectors and K second weight matrices respectively; and a digital-to-analog conversion module, which converts each element of the K second input vectors and the K second weight matrices into K groups of first electrical signals and K groups of second electrical signals respectively.

[0018] Optionally, the optoelectronic hybrid computing unit includes: a laser module, which outputs K lasers with different wavelengths; a photon computing module, which splits each of the K lasers into two to obtain K first splits and K second splits, divides the K first splits into K groups of first optical signals, and the wavelengths of the first optical signals in each group are the same, and divides the K second splits into K groups of seventh optical signals, and the wavelengths of the seventh optical signals in each group are the same, and the photon computing module modulates the K groups of first electrical signals and the K groups of second electrical signals into the K groups of first optical signals to obtain K groups of second optical signals, and demultiplexes and sums the second optical signals in each group to obtain K groups of 2 The eighth optical signal is grouped and the eighth optical signal is photoelectrically converted based on the seventh optical signal to obtain K 2 Group fifth electrical signal; and an electronic computing module, the electronic computing module K 2 The fifth electrical signals of the groups are multiplied and added to obtain 2K-1 groups of the third electrical signals, and the corresponding third electrical signals in each group of the third electrical signals are analog-to-digital converted, shifted and registered, and summed, so as to calculate the product of the matrix vector as the output vector.

[0019] Optionally, the photonic computing module includes: a second spectrometer, which splits K of the lasers into K first spectrometers and K second spectrometers; a third spectrometer, which splits the first spectrometer of each wavelength into a group of the first optical signals with the same wavelength; a fourth spectrometer, which splits the second spectrometer of each wavelength into a group of the seventh optical signals with the same wavelength; an electro-optical modulator, which modulates the first electrical signal into the first optical signal; a weight controller, which modulates the second electrical signal into the first optical signal modulated with the first electrical signal to obtain the second optical signal; a multiplier-accumulator, which demultiplexes and sums each group of the second optical signals within the group to obtain the eighth optical signal, and performs photoelectric conversion on the eighth optical signal based on the seventh optical signal to obtain the fifth electrical signal.

[0020] Optionally, the product accumulator includes: a third demultiplexer, which demultiplexes K groups of the second optical signals to obtain K2 a fourth optical signal of each wavelength in the group; a fourth multiplexer, the fourth multiplexer summing the fourth optical signals of all rows except the last row from the matrix-vector multiplication with the fourth optical signal of the last row of the matrix-vector multiplication within the group to obtain K 2 a fifth optical splitter, which splits the K groups of the seventh optical signal into K 2 a ninth optical signal; and a third photoelectric converter configured to perform photoelectric conversion on the eighth optical signal based on the ninth optical signal to obtain the fifth electrical signal.

[0021] Effects of the Invention

[0022] The optoelectronic hybrid computing system according to the present invention can complete accurate matrix-vector multiplication calculations in fewer clock cycles and reduce the number of required modulators, thereby effectively reducing chip area and power consumption and lowering costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following detailed description, they are used to explain the embodiments of the present application but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings:

[0024] Figure 1 This is a block diagram showing the overall configuration of the optoelectronic hybrid computing system according to the first embodiment of the present invention.

[0025] Figure 2 It is a schematic diagram showing the structure of the drive unit according to the first embodiment of the present invention.

[0026] Figure 3 It is a schematic diagram showing the structure of the photonic computing module involved in embodiment 1 of the present invention.

[0027] Figure 4 Yes Figure 3 Schematic diagram of the structure of the weight controller in .

[0028] Figure 5 Yes Figure 3 Schematic diagram of a structural example of a multiply-accumulator when K=2.

[0029] Figure 6 Yes Figure 3 Schematic diagram of a structural example of a multiply-accumulator when K=3.

[0030] Figure 7 This is a schematic diagram showing the structure of an electronic calculation module according to the first embodiment of the present invention.

[0031] Figure 8 It is a schematic diagram showing the structure of the photonic computing module involved in embodiment 2 of the present invention.

[0032] Figure 9 Yes Figure 8 Schematic diagram of a structural example of a multiply-accumulator when K=2.

[0033] Figure 10 Yes Figure 8 Schematic diagram of a structural example of a multiply-accumulator when K=3.

[0034] Figure 11 It is a schematic diagram showing the structure of an electronic calculation module involved in Embodiment 2 of the present invention.

[0035] Figure 12 It is a schematic diagram showing the structure of a drive unit according to Embodiment 3 of the present invention.

[0036] Figure 13 It is a schematic diagram showing the structure of the photonic computing module involved in embodiment 3 of the present invention.

[0037] Figure 14 Yes Figure 13 Schematic diagram of a structural example of a multiply-accumulator when K=2 and n<N-1.

[0038] Figure 15 Yes Figure 13 Schematic diagram of a structural example of a multiply-accumulator when K=2 and n=N-1.

[0039] Figure 16 This is a schematic diagram showing the structure of an electronic calculation module involved in embodiment 3 of the present invention. DETAILED DESCRIPTION

[0040] To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0041] It should be noted that the technical solutions between the various embodiments of the present application can be combined with each other, but it must be based on the fact that ordinary technicians in this field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0042] Implementation Method 1

[0043] The optoelectronic hybrid computing system of the present invention mainly solves the problem of how to use an optoelectronic hybrid computing system architecture with lower computing accuracy to implement matrix-vector multiplication (MVM) of the two operands when the number of bits of the two operands is too high. In the present invention, the so-called "computational accuracy" refers to the number of bits of the operands involved in the calculation. In other words, when the number of bits of the operands exceeds the computational accuracy, the entire computing system will not be able to ensure the accuracy of the calculation results. For example, the computational accuracy can be expressed as the maximum number of bits of the operand that can ensure that the electro-optical modulator used in photonic computing can correctly modulate the electrical signal into the optical signal.

[0044] Taking the MVM calculation of an M×N matrix W and an N×1 vector X as an example, the binary representation of each operand requires, for example, B = Kb bits, where K and b are both positive integers. b represents the "computational precision" of the optoelectronic hybrid computing system architecture, and B represents the number of binary bits of each element constituting the matrix W and vector X, with B being K times b. The final product Y can be expressed as the following equation (1).

[0045] [Mathematical formula 1]

[0046]

[0047] in,

[0048]

[0049] Represents the element ω of the W matrix mn The decimal number represented by the (i-1)×b+1th to the i×bth digits of (m=0,…,M-1,n=0,…,N-1), Then it represents the element X of vector X n ,n=0,…,N-1, the decimal number represented by the (j-1)×b+1th to j×bth digits.

[0050] Assuming that for the current dimensions M and N, the existing optoelectronic hybrid computing system can only guarantee that the calculation result is correct when the two operands are b bits, it is necessary to repeatedly call the current optoelectronic hybrid computing system K 2 times to realize MVM with B-bit operands. In addition, up to (K 2 -1)MN additional additions and multiplications or shifts (multiplication by 2 bk ), the power consumption and delay consumption are very large.

[0051] In contrast, the optoelectronic hybrid computing system of the present invention utilizes a system with b-bit computational precision to perform K×b-bit matrix-vector multiplication. Specifically, through the scheme described below, the parallel dimension of light is exploited to achieve bit-splitting of operands. This reduces the latency and power consumption of the aforementioned conventional optoelectronic hybrid computing systems, significantly increasing the system's computing power.

[0052] The overall structure of the optoelectronic hybrid computing system of the present invention is described below.

[0053] Figure 1 : is a block diagram showing the overall structure of the optoelectronic hybrid computing system involved in this embodiment 1. Figure 1 As shown, the optoelectronic hybrid computing system includes a storage unit 10 , a driving unit 20 and an optoelectronic hybrid computing unit 30 .

[0054] The storage unit 10 stores an N×1-dimensional vector X (hereinafter referred to as "first input vector X") composed of K×b=B-bit elements, and an M×N-dimensional matrix W (hereinafter referred to as "first weight matrix W") composed of K×b=B-bit elements. Specifically, the storage unit 10 can be composed of various existing memories such as ROM (Read Only Memory), RAM (Random Access Memory), or flash memory.

[0055] The driving unit 20 reads the first input vector X (x0, . . . , x N-1 ), and the first weight matrix W(ω 00 ,…,ω mn ,…,ω M-1，N-1 ), split the first input vector X and the first weight matrix W into K second input vectors consisting of b-bit elements respectively and the second weight matrix And each element of the K second input vectors and the K second weight matrices is digitally converted into K groups of first electrical signals and K groups of second electrical signals. That is, each element in the second input vector and the second weight matrix after the splitting is completed is composed of b-bit binary numbers. In addition, the driving unit also digitally converts the K second input vectors and the K second weight matrices into K groups of first electrical signals and K groups of second electrical signals, and the voltage of each electrical signal in each group of first electrical signals and second electrical signals corresponds to the size of each element in the second input vector and the second weight matrix. That is, the voltage of the K groups of first electrical signals is equal to Correspondingly, the voltage of the second electrical signal of group K is The specific embodiment of the driving unit 20 will be described in detail below.

[0056] The optoelectronic hybrid calculation unit 30 obtains the first electrical signal and the second electrical signal from the driving unit 20, modulates the K groups of first electrical signals and the K groups of second electrical signals into the first optical signals of the corresponding groups of the first optical signals divided into K groups based on wavelength, performs multiplication and accumulation, intra-group summation and optoelectronic conversion on the K groups of second optical signals thus obtained, obtains 2K-1 groups of third electrical signals, performs analog-to-digital conversion, shift register and summation on the corresponding third electrical signals in each group of third electrical signals, thereby obtaining an output vector Y (Y0, ..., Y M-1 ). Specifically, the optoelectronic hybrid calculation unit 30 completes the multiplication operation between the elements in the MVM by modulating K groups of first electrical signals and K groups of second electrical signals into the first optical signals of the corresponding groups respectively. The corresponding second optical signals in each group of second optical signals modulated with the first electrical signal and the second electrical signal are multiplied and accumulated between each group, and the corresponding electrical signals in each group of third electrical signals are converted into analog-to-digital, shifted and registered, and summed to complete the multiplication operation between the corresponding elements in each group. The summation of the parallel multiplication results of each group is completed by summing within the group. The conversion of optical signals to electrical signals is completed by optoelectronic conversion. Thus, the operation of the MVM of B=Kb bits is finally completed, and the output vector Y (Y0,…,Y M-1 The specific embodiment of the optoelectronic hybrid calculation unit 30 will be described in detail below.

[0057] In the prior art, for a weight matrix W with a dimension of M×N and an input vector X with a dimension of N×1, if the accuracy that can be achieved by the existing device differs by K times from the actual required weight or input vector accuracy, then if accurate MVM is required, then for the traditional optoelectronic hybrid computing system, K 2 Secondary optoelectronic hybrid computing system, that is, K 2 Clock cycles are required to achieve accurate MVM. In contrast, the optoelectronic hybrid computing system according to the present invention modulates K groups of electrical signals into K groups of optical signals, which are also divided into K groups based on wavelength, and utilizes the parallel dimension of light to simultaneously perform multiplication operations on multiple groups of electrical signals. Therefore, within 1 to 2 clock cycles, a system with a precision of b bits can accurately complete K×b-bit matrix-vector multiplication, thereby significantly shortening calculation time and improving system computing power while ensuring calculation accuracy.

[0058] In addition, taking the crossbar (crossbar switch matrix or vertical and horizontal switching matrix) architecture as an example, in the prior art, the number of weights and vector modulators required is N and MN respectively. If an accurate result needs to be calculated within one clock cycle, K is required. 2 N+K2 MN modulators. In contrast, the optoelectronic hybrid computing system according to the present invention only requires K(N+MN) modulators, which is K times less than the number of modulators in the crossbar architecture. Therefore, it can effectively reduce chip area and power consumption.

[0059] like Figure 1 As shown, the optoelectronic hybrid computing unit 30 includes a laser module 1, a photon computing module 2 and an electronic computing module 3. Figures 2 to 7 , a specific structural example of the optoelectronic hybrid computing system involved in this embodiment is described. Figures 2 to 6 In the figure, the solid arrows indicate the direction of optical signal flow, and the dotted arrows indicate the direction of electrical signal flow.

[0060] Figure 2 Schematic diagram showing the structure of the drive unit 20a according to the first embodiment. Figure 2 As shown, the driving unit 20a uses a data splitting module (not shown) to convert the first input vector X (x0, ..., x N-1 ) and the first weight matrix W(ω 00 ,…,ω mn ,…,ω M-1，N-1 ) are split into K second input vectors and K second weight matrices in, represent The binary representation of represent The binary representation of and Represents two operands respectively and The sign bit of .

[0061] In addition, in this embodiment, the driving unit 20a further includes a plurality of XOR gates, switch gates, and digital-to-analog converter modules (DACs). The XOR gate first extracts two operands. and The sign bit and And perform XOR operation, so that when the two operands have the same sign, the XOR gate outputs 0, otherwise it outputs 1. The switch gate is responsible for inputting the valid binary number to the input end of the digital-to-analog converter module (DAC) at different times. For example, its output can be expressed as the following formula (2).

[0062] [Mathematical formula 2]

[0063]

[0064] For the second input vector, the expression of its output can be deduced by analogy with the above formula (2).

[0065] Therefore, in this embodiment, a time division multiplexing module is formed by an exclusive OR gate (XOR) and a switch gate (Switch), and the time division multiplexing module is based on the second input vector and the second weight matrix The signs of each element of are extracted in the first clock cycle (i.e., t is an even-numbered clock cycle) and the absolute values ​​of the elements with the same signs as the multiplied elements in the second input vector and the second weight matrix are combined with 0 to reconstruct the second input vector and the second weight matrix. The absolute values ​​of the elements with different signs as the multiplied elements in the second input vector and the second weight matrix are extracted in the second clock cycle (i.e., t is an odd-numbered clock cycle) and the absolute values ​​of the elements are combined with 0 to reconstruct the second input vector and the second weight matrix.

[0066] In addition, in this embodiment, the digital-to-analog conversion module (DAC) converts the reconstructed K second input vectors and the elements of the K second weight matrix into K groups of first electrical signals. and K group second electrical signal

[0067] Figure 3 Schematic diagram showing the structure of the photonic computing module 2a according to the first embodiment. Figure 1 and Figure 3 In this embodiment, the laser module 1 transmits K laser beams with different wavelengths λ0...λ K-1 The photon calculation module 2a converts the input K lasers λ0...λ K-1 The first optical signals are divided into K groups, and the wavelengths of the first optical signals in each group are the same. Specifically, Figure 3 As shown, the photon computing module 2a includes K first optical splitters (optical splitters 1:N), each of which splits the laser light of each wavelength into a group of first optical signals with the same wavelength. For example, the laser light with a wavelength of λ0 is split into a group of N first optical signals with the same wavelength of λ0, the laser light with a wavelength of λ1 is split into a group of N first optical signals with the same wavelength of λ1, and so on. K-1 The laser is divided into 1 group of N wavelengths, each with λ K-1 The first optical signal.

[0068] The photon computing module 2a converts the K groups of first electrical signals and K group second electrical signal The K groups of second optical signals are respectively modulated into the K groups of first optical signals to obtain K groups of second optical signals.

[0069] Specifically, if Figure 3 As shown, the photonic computing module 2a further includes a plurality of (K×N in this embodiment) electro-optical modulators EOM and a plurality of (K×M×N in this embodiment) weight controllers WCU. Each electro-optical modulator EOM converts the first electrical signal Each weight controller WCU modulates the second electrical signal into the first optical signal. Modulation to modulation with a first electrical signal The second optical signal is obtained from the first optical signal ( Figure 3 The signal output from each weight controller WCU to each multiplier accumulator MAC) can complete the point product of each element in the matrix-vector multiplication.

[0070] Figure 4 Yes Figure 3 Schematic diagram of the structure of the weight controller WCU in . Figure 4 As shown, the weight controller WCU includes an adjustable direct coupler TDC and an electro-optical modulator EOM, with solid lines representing effective splitting and weight data, respectively. modulation.

[0071] There is no particular limitation on the specific implementation of the electro-optical modulator (EOM) for modulating optical signals into electrical signals, and any existing modulation circuit such as photoelectric neurons can be used to complete the modulation.

[0072] In addition, if Figure 3 As shown, in this embodiment, the photonic computing module 2a also includes a plurality of (N in this embodiment) multiplexers MUX, each multiplexer MUX respectively combines different groups of first optical signals (i.e., multiple wavelengths with different wavelengths) modulated with the first electrical signal into one waveguide for transmission. Moreover, the different groups of first optical signals combined into one waveguide are optical signals modulated with the corresponding first electrical signal. In the present invention, unless otherwise specified, the so-called "corresponding" refers to corresponding to the elements of the same row / column in the input vector, or corresponding to the elements of the same row and column in the weight matrix. For example, with the first input vector X(x0,…,x N-1 ) corresponds to the element with subscript 0 in, or corresponds to the first weight matrix W(ω 00 ,…,ω mn ,…,ω M-1，N-1 The waveguides combined by the multiplexer MUX are guided to the weight controllers WCU in the same row, whereby the weight controllers WCU further modulate the second electrical signal.

[0073] The photon calculation module 2a performs multiplication and accumulation on the K groups of second optical signals to obtain 2K-1 groups of third optical signals, and performs photoelectric conversion on the 2K-1 groups of third optical signals to obtain 2K-1 groups of fourth electrical signals.

[0074] Specifically, if Figure 3 As shown, the photon computing module 2a includes multiple (M×N in this embodiment) multiplication and accumulation units MAC, each of which performs multiplication and accumulation on the corresponding second optical signals in the K groups of second optical signals between each group to obtain 2K-1 groups of third optical signals. and converting the third optical signal into a fourth electrical signal

[0075] The following describes the structures of the multiply-accumulator MAC when K=2 and K=3 as examples.

[0076] Figure 5 Yes Figure 3 Schematic diagram of the structure of the multiply-accumulate MAC when K=2. Figure 5 As shown, when K=2, the product accumulator MAC includes two first demultiplexers DeMUX, one first multiplexer MUX, three first photoelectric converters PD, three transimpedance amplifiers TIA, and three low-pass filters. The first demultiplexer DeMUX demultiplexes K groups (2 groups) of second optical signals to obtain K 2 (4 groups) of fourth optical signals of different wavelengths (λ0, λ1, λ0, λ1). The first multiplexer MUX will remove the first and last two groups of fourth optical signals ( Figure 5 The fourth optical signal ( Figure 5 The middle two fourth optical signals λ0, λ1) are multiplied and accumulated between groups and then added to the first and last two groups of fourth optical signals ( Figure 5 The two fourth optical signals λ0 and λ1 on both sides together form 2K-1 groups (3 groups) of third optical signals The first photoelectric converter PD converts the third optical signal Converted into a fourth electrical signal

[0077] Specifically, assuming that the first input vector Then pass Figure 3 The optical signal after the multiplexer MUX can be expressed as the following formula (3).

[0078] [Mathematical formula 3]

[0079]

[0080] Where c is the speed of light, Ak The wavelength is λ k The local oscillator light does not pass through The amplitude during modulation. Figure 3 After modulation by the weight controller WCU in , the optical signal at this time becomes a second optical signal, which can be expressed as the following formula (4).

[0081] [Formula 4]

[0082]

[0083] Afterwards, after Figure 5 The demultiplexing of the first demultiplexer DeMUX and the multiplication and accumulation of the first multiplexer MUX obtain 2K-1 groups (3 groups) of third optical signals It can be expressed as the following formulas (5) to (7).

[0084] [Formula 5]

[0085]

[0086] [Formula 6]

[0087]

[0088] [Formula 7]

[0089]

[0090] Assuming that the response rates of the first photoelectric converters PD are the same and are all β, the current outputs detected by the three PDs can be expressed as the following equations (8) to (10).

[0091] [Formula 8]

[0092]

[0093] [Formula 9]

[0094]

[0095] [Formula 10]

[0096]

[0097] After passing through the low-pass filter and the transimpedance amplifier TIA, and assuming that the TIA amplification factor is γ, the amplified voltage output, i.e. the fourth electrical signal It can be expressed as the following formulas (11) to (13).

[0098] [Mathematical formula 11]

[0099]

[0100] [Mathematical formula 12]

[0101]

[0102] [Mathematical formula 13]

[0103]

[0104] By calibrating and adjusting the TIA amplification factor, β γ =1, thereby ensuring that the output result is the required summation result.

[0105] In this embodiment, the fourth electrical signal is amplified and filtered by the transimpedance amplifier TIA and the low-pass filter, but the present invention is not limited thereto, and the transimpedance amplifier TIA and the low-pass filter may not be provided.

[0106] Figure 6 Yes Figure 3 Schematic diagram of the structure of the multiply-accumulate MAC when K=3. Figure 6 As shown, when K=3, the product accumulator MAC includes 3 first demultiplexers DeMUX, 3 first multiplexers MUX, 5 first photoelectric converters PD, 5 transimpedance amplifiers TIA, and 5 low-pass filters. The first demultiplexer DeMUX demultiplexes K groups (3 groups) of second optical signals to obtain K 2 (9 groups) of fourth optical signals of various wavelengths (λ0, λ1, λ2, λ0, λ1, λ2, λ0, λ1, λ2). The first multiplexer MUX will remove the first and last two groups of fourth optical signals ( Figure 6 The fourth optical signal ( Figure 5 The middle 7 fourth optical signals λ1, λ2, λ0, λ1, λ2, λ0, λ1) are multiplied and accumulated between groups and then added to the first and last 2 groups of fourth optical signals ( Figure 5 The two fourth optical signals λ0 and λ2 on both sides together form 2K-1 groups (5 groups) of third optical signals The first photoelectric converter PD converts the third optical signal Converted into a fourth electrical signal

[0107] The detailed calculation method when K=3 is the same as that when K=2, and can be deduced by analogy.

[0108] Figure 7 It is a schematic diagram showing the structure of the electronic calculation module 3a according to the first embodiment.

[0109] The electronic calculation module 3a pairs the 2K-1 group of fourth electrical signals The intra-group summation is performed to obtain 2K-1 groups of third electrical signals, and the corresponding third electrical signals in each group of third electrical signals are subjected to analog-to-digital conversion, shift registration, and summation, thereby calculating the matrix-vector product.

[0110] Specifically, if Figure 7 As shown, the electronic calculation module 3a performs intra-group summation on the fourth electrical signals in the same group, for example, Then, the analog-to-digital converter ADC is used to convert the summed results into analog-to-digital conversion, and the shift register BitShifter is used to shift and store the digital signals after analog-to-digital conversion to obtain 2K-1 groups of digital signals. Use adder Addition to add each group of digital signals The corresponding digital signal Summing to calculate the matrix-vector product

[0111] In addition, in this embodiment, the optoelectronic hybrid calculation unit 30 further includes a difference calculation module (not shown), which subtracts the product calculated in the first clock cycle and the product calculated in the second clock cycle to obtain an output vector.

[0112] Specifically, the difference calculation module can calculate the output vector Y(T) through the following formula (14).

[0113] [Formula 14]

[0114]

[0115] The calculation process of the optoelectronic hybrid computing system according to this embodiment is described below with examples.

[0116] Assume that the goal of the optoelectronic hybrid computing system is to calculate the following matrix-vector multiplication:

[0117]

[0118] The number of bitsl ice (bit slices) is K = 2, the number of bits in each slice is b = 2, and the MVM dimension is M = N = 2. Therefore, the number of bits in each operand is Kb. The combination of the high and low bits of the weight and vector can be expressed as:

[0119]

[0120] Therefore, the entire calculation process of the optoelectronic hybrid computing system can be expressed as:

[0121]

[0122] For the optoelectronic hybrid computing system involved in this embodiment, the photon computing module 2a performs the dot multiplication operation and detects the electrical signal through the first photoelectric converter PD. Where m = 0...M-1, n = 0...N-1, k = 0...2K-2. Then, the electronic calculation module 3a completes Y m The summation of (t) and the shift register operation therein, where m=0…M-1.

[0123] Right now,

[0124]

[0125] As described above, according to the optoelectronic hybrid computing system of this embodiment, Figures 2 to 7 As shown in the figure, a non-coherent architecture is used to implement K×b bit matrix-vector multiplication, without the need to convert the local oscillator light (λ0...λ K-1 ) is input to each photoelectric converter PD to complete the photoelectric conversion. Therefore, no additional optical path design is required, which can reduce the design difficulty.

[0126] In addition, the optoelectronic hybrid computing system according to this embodiment adopts a non-coherent architecture to implement matrix-vector multiplication. Therefore, the requirements for laser phase are relatively low, which can reduce the difficulty of laser calibration.

[0127] In addition, according to the optoelectronic hybrid computing system of this embodiment, Figure 7 As shown, the fourth electrical signal in the same group The intra-group summation is performed in the electrical domain, so it is easier to implement than summation in the optical domain.

[0128] Implementation Method 2

[0129] In this embodiment, the structures of the storage unit 10 and the driving unit 20 are the same as those in the above embodiment 1. The main difference between this embodiment and the above embodiment 1 is the structure of the optoelectronic hybrid computing unit 30. Figures 8-11 The optoelectronic hybrid computing system of this embodiment will be described with the structure of the optoelectronic hybrid computing unit 30 as the center. Figures 8 to 10 In the figure, the solid arrows indicate the direction of optical signal flow, and the dotted arrows indicate the direction of electrical signal flow.

[0130] Figure 8 Schematic diagram showing the structure of the photonic computing module 2b according to the second embodiment. Figure 1 and Figure 8In this embodiment, the laser module 1 outputs K groups of lasers with different wavelengths, and the wavelengths of the lasers in each group are different. The photon computing module 2b uses the K groups of lasers from the laser module 1 as the K groups of first optical signals. Specifically, Figure 8 As shown, the laser module 1 is based on the wavelength , so that K groups of lasers with different wavelengths are input into the photonic computing module 2b as the first optical signal.

[0131] The photon computing module 2b converts the K groups of first electrical signals and K group second electrical signal are modulated into the K groups of first optical signals to obtain K groups of second optical signals. Specifically, Figure 8 As shown, the photonic computing module 2b further includes a plurality of (K×N in this embodiment) electro-optical modulators EOM and a plurality of (K×M×N in this embodiment) weight controllers WCU. Each electro-optical modulator EOM converts the first electrical signal Each weight controller WCU modulates the second electrical signal into the first optical signal. Modulation to modulation with a first electrical signal The second optical signal is obtained from the first optical signal ( Figure 8 The signal output from each weight controller WCU to each multiplier accumulator MAC) can complete the point product of each element in the matrix-vector multiplication.

[0132] about Figure 8 The specific structure of the weight controller WCU is the same as that of the above-mentioned embodiment 1, and the description is omitted here.

[0133] In addition, if Figure 8 As shown, in this embodiment, the photonic computing module 2b also includes a plurality of (N in this embodiment) multiplexers MUX, each of which combines different groups of first optical signals (i.e., multiple optical signals with different wavelengths) modulated with the first electrical signal into one waveguide for transmission. Moreover, the different groups of first optical signals combined into one waveguide are optical signals modulated with the corresponding first electrical signals. The waveguides combined by the multiplexer MUX are guided to the weight controller WCU in the same row, so that the weight controller WCU further completes the modulation of the second electrical signal.

[0134] The photon calculation module 2b performs multiplication and accumulation and intra-group summation on the K groups of second optical signals to obtain 2K-1 groups of third optical signals, and performs photoelectric conversion on the 2K-1 groups of third optical signals to obtain 2K-1 groups of third electrical signals.

[0135] Specifically, if Figure 8As shown, the photon computing module 2b includes a plurality of (M×N in this embodiment) multiplier-accumulator MACs, each of which performs multiplication and accumulation on the corresponding second optical signals in the K groups of second optical signals between the groups, and performs summation within the group to obtain 2K-1 groups of third optical signals, and converts the third optical signals into a third electrical signal

[0136] The following describes the structures of the multiply-accumulator MAC when K=2 and K=3 as examples.

[0137] Figure 9 Yes Figure 8 Schematic diagram of a structural example of the multiply-accumulate unit MAC when K=2. Figure 9 The last row of the matrix-vector multiplication, Figure 8 The structure of the multiply-accumulate MAC in the last row. For the structure of the multiply-accumulate MAC other than the last row, please refer to Figure 5 The structure of each multiplier-accumulator MAC only includes two second demultiplexers DeMUX and one second multiplexer MUX. The second demultiplexer DeMUX demultiplexes K groups (2 groups) of second optical signals to obtain K 2 (4 groups) of fourth optical signals of each wavelength The second multiplexer MUX converts the fourth optical signals except the first and last two groups of fourth optical signals into After multiplication and accumulation between groups, the first and last two groups of fourth optical signals form a 2K-1 group of fifth optical signals.

[0138] like Figure 9 As shown in FIG1 , when K=2, the product accumulator MAC in the last row includes two second demultiplexers DeMUX, three third multiplexers MUX, three second photoelectric converters PD, three transimpedance amplifiers TIA, and three low-pass filters. The second demultiplexer DeMUX demultiplexes K groups (2 groups) of second optical signals to obtain K 2 (4 groups) of fourth optical signals of each wavelength The third multiplexer MUX performs the last row of matrix-vector multiplication, and removes the first and last two groups of the fourth optical signal. The fourth optical signal other than Except for the first and last two groups of fifth light signals The fifth optical signal other than After performing the summation and product accumulation within the group, the first and last two groups of fourth optical signals are combined. and the first and last two groups of fifth light signals The sixth optical signals obtained by performing the intra-group summation respectively constitute 2K-1 groups of third optical signals ( Figure 9 The second photoelectric converter PD converts the third optical signal into a third electrical signal.

[0139] In detail, Figure 9 The principle is similar. Taking K=2 as an example, the voltage output of each port can be expressed as the following equations (15) to (17).

[0140] [Mathematical formula 15]

[0141]

[0142] [Mathematical formula 16]

[0143]

[0144] [Mathematical formula 17]

[0145]

[0146] In addition, in this embodiment, the third electrical signal is amplified and filtered by the transimpedance amplifier TIA and the low-pass filter, but the present invention is not limited thereto, and the transimpedance amplifier TIA and the low-pass filter may not be provided.

[0147] Figure 10 Yes Figure 8 Schematic diagram of the structure of the multiply-accumulate MAC when K=3. Figure 10 As shown, when K=3, the product-accumulator MAC in the last row includes 3 second demultiplexers DeMUX, 5 third multiplexers MUX, 5 second photoelectric converters PD, 5 transimpedance amplifiers TIA, and 5 low-pass filters. The second demultiplexer DeMUX demultiplexes K groups (3 groups) of second optical signals to obtain K 2 (9 groups) of the fourth optical signals of each wavelength. The third multiplexer MUX performs the last row of the matrix-vector multiplication, and removes the first and last two groups of the fourth optical signals. The fourth optical signal other than Except for the first and last two groups of fifth light signals The fifth optical signal other than After performing the summation and product accumulation within the group, the first and last two groups of fourth optical signals are combined. and the first and last two groups of fifth light signals The sixth optical signals obtained by performing the intra-group summation respectively constitute 2K-1 groups of third optical signals ( Figure 10 The second photoelectric converter PD converts the third optical signal into a third electrical signal.

[0148] The detailed calculation method when K=3 is the same as that when K=2, and can be deduced by analogy.

[0149] Figure 11 It is a schematic diagram showing the structure of the electronic calculation module 3b according to the second embodiment.

[0150] The electronic calculation module 3b processes each group of third electrical signals The corresponding third electrical signal in is subjected to analog-to-digital conversion, shift registration and summation, thereby calculating the product of the matrix vector.

[0151] Specifically, if Figure 11 As shown, the electronic calculation module 3b uses the analog-to-digital converter ADC to convert each group of third electrical signals Perform analog-to-digital conversion and use the shift register BitShifter to shift and store the digital signal after analog-to-digital conversion to obtain 2K-1 groups of digital signals Use adder Addition to add each group of digital signals The corresponding digital signal Summing to calculate the matrix-vector product

[0152] In addition, in this embodiment, the optoelectronic hybrid calculation unit 30 further includes a difference calculation module (not shown), which subtracts the product calculated in the first clock cycle and the product calculated in the second clock cycle to obtain an output vector.

[0153] Specifically, the difference calculation module can calculate the output vector Y(T) through the above formula (14).

[0154] The calculation process of the optoelectronic hybrid computing system according to this embodiment is described below with examples.

[0155] Assume that the goal of the optoelectronic hybrid computing system is to calculate the following matrix-vector multiplication:

[0156]

[0157] The number of bitslices is K=2, the number of bits in each slice is b=2, and the MVM dimension is M=N=2. Therefore, the number of bits in each operand is Kb. The high and low bit combinations of weights and vectors can be expressed as:

[0158]

[0159] Therefore, the entire calculation process of the optoelectronic hybrid computing system can be expressed as:

[0160]

[0161]

[0162] For the optoelectronic hybrid computing system involved in this embodiment, the photonic computing module 2b performs the dot multiplication operation. The product accumulator MAC of the first N-1 rows of the matrix-vector multiplication directly outputs the optical signal after modulation and demultiplexing by the demultiplexer Demux. The electrical signal is obtained by detecting and summing the last row of the matrix-vector multiplication through the second optoelectronic converter PD. Where m = 0...M-1, k = 0...2K-2. Then, the electronic calculation module 3b completes Y m The summation of (t) and the shift register operation therein, where m=0…M-1.

[0163] Right now,

[0164]

[0165] According to the optoelectronic hybrid computing system of this embodiment, similar to the above embodiment 1, a non-coherent architecture is used to implement K×b bit matrix-vector multiplication, without the need to convert the local oscillator light (λ0...λ K-1 ) is input to each photoelectric converter PD to complete the photoelectric conversion. Therefore, no additional optical path design is required, which can reduce the design difficulty.

[0166] In addition, the optoelectronic hybrid computing system according to this embodiment adopts a non-coherent architecture to implement matrix-vector multiplication. Therefore, the requirements for laser phase are relatively low, which can reduce the difficulty of laser calibration.

[0167] In addition, according to the optoelectronic hybrid computing system of this embodiment, Figure 8 As shown, there is no need to use a spectrometer to split the laser light from the laser module 1 , thus reducing the size of the photonic computing chip.

[0168] Furthermore, according to the optoelectronic hybrid computing system of this embodiment, photoelectric conversion is performed only by the photoelectric converters PD in the last row, and therefore the number of photoelectric converters PD used can be significantly reduced.

[0169] Implementation 3

[0170] In this embodiment, the structure of the storage unit 10 is the same as that of the above-mentioned embodiments 1 and 2. The main difference between this embodiment and the above-mentioned embodiments 1 and 2 lies in the structure of the driving unit 20b and the optoelectronic hybrid computing unit 30. Figures 12-16 The optoelectronic hybrid computing system of this embodiment will be described with the structure of the driving unit 20b and the optoelectronic hybrid computing unit 30 as the center. Figures 13 to 15 In the figure, the solid arrows indicate the direction of optical signal flow, and the dotted arrows indicate the direction of electrical signal flow.

[0171] Figure 12 Schematic diagram showing the structure of the drive unit 20b according to the third embodiment. Figure 12 As shown, the driving unit 20a uses a data splitting module (not shown) to convert the first input vector X (x0, ..., x N-1 ) and the first weight matrix W(ω 00 ,…,ω mn ,…,ω M-1，N-1 ) are split into K second input vectors and K second weight matrices in, represent The binary representation of represent The binary representation of and Represents two operands respectively and The sign bit of .

[0172] Different from the driving unit 20a in the above-mentioned embodiments 1 and 2, the driving unit 20b in this embodiment directly inputs the sign bit and the corresponding bit number into the digital-to-analog conversion module (DAC) for subsequent operations. Specifically, the digital-to-analog conversion module directly converts the K second input vectors and the elements of the K second weight matrix into K groups of first electrical signals. and K group second electrical signal

[0173] Figure 13 Schematic diagram showing the structure of the photonic computing module 2c according to the third embodiment. Figure 1 and Figure 13 In this embodiment, the laser module 1 transmits K laser beams with different wavelengths λ0...λ K-1 The photon calculation module 2c inputs the K lasers λ0...λ K-1Each laser is split into two, obtaining K first splits and K second splits, the K first splits are divided into K groups of first optical signals, the wavelength of the first optical signals in each group is the same, and the K second splits are divided into K groups of seventh optical signals, the wavelength of the seventh optical signals in each group is the same. Specifically, Figure 13 As shown, the photon computing module 2c includes K second optical splitters (optical splitter 1:2), K third optical splitters ( Figure 13 The two lower side optical splitters 1:N are represented by the optical splitter 1:N), and K fourth optical splitters ( Figure 13 The two upper beam splitters 1:N are representative of the beam splitter 1:N). Each second beam splitter (beam splitter 1:2) splits K laser beams into K first beam splitters ( Figure 13 The light splitting input to the third light splitter) and K second light splitting ( Figure 13 Each third optical splitter splits the first split light of each wavelength into a group of first optical signals with the same wavelength. For example, the first split light of wavelength λ0 is split into a group of N first optical signals with the same wavelength λ0, the first split light of wavelength λ1 is split into a group of N first optical signals with the same wavelength λ1, and so on. K-1 The first splitting is divided into 1 group of N wavelengths, each of which is λ K-1 Each fourth optical splitter splits the second split light of each wavelength into a group of the seventh optical signals with the same wavelength. For example, the second split light with a wavelength of λ0 is split into a group of N seventh optical signals with the same wavelength of λ0, the second split light with a wavelength of λ1 is split into a group of N seventh optical signals with the same wavelength of λ1, ..., the second split light with a wavelength of λ K-1 The second splitting is divided into 1 group of N wavelengths, each of which is λ K-1 The seventh optical signal.

[0174] The photon computing module 2c converts the K groups of first electrical signals and K group second electrical signal The K groups of second optical signals are respectively modulated into the K groups of first optical signals to obtain K groups of second optical signals.

[0175] Specifically, if Figure 13 As shown, the photonic computing module 2c further includes a plurality of (K×N in this embodiment) electro-optical modulators EOM and a plurality of (K×M×N in this embodiment) weight controllers WCU. Each electro-optical modulator EOM converts the first electrical signal Each weight controller WCU modulates the second electrical signal into the first optical signal. Modulation to modulation with a first electrical signal The second optical signal is obtained from the first optical signal ( Figure 13 The signal output from each weight controller WCU to each multiplier accumulator MAC) can complete the point product of each element in the matrix-vector multiplication.

[0176] about Figure 13 The specific structure of the weight controller WCU is the same as that of the above-mentioned embodiment 1, and the description is omitted here.

[0177] In this embodiment, the electro-optical modulator (EOM) modulates the first electrical signal (i.e., the input vector) using amplitude modulation by changing the amplitude of light, and the weight controller (WCU) modulates the second electrical signal (i.e., the weight matrix) using phase modulation by changing the phase of light. However, the present invention is not limited to this. Modulation can also be achieved by changing the intensity of light.

[0178] In addition, if Figure 13 As shown, in this embodiment, the photonic computing module 2b also includes a plurality of (N in this embodiment) multiplexers MUX, each of which combines different groups of first optical signals (i.e., multiple optical signals with different wavelengths) modulated with the first electrical signal into one waveguide for transmission. Moreover, the different groups of first optical signals combined into one waveguide are optical signals modulated with the corresponding first electrical signals. The waveguides combined by the multiplexer MUX are guided to the weight controller WCU in the same row, so that the weight controller WCU further completes the modulation of the second electrical signal.

[0179] In addition, in this embodiment, each optical splitter can ensure that the static optical power (without any modulation) of the input port before each multiplexer MUX is the same.

[0180] The photon calculation module 2c demultiplexes each group of second optical signals and performs intra-group summation to obtain K 2 The eighth optical signal is grouped and the eighth optical signal is photoelectrically converted based on the seventh optical signal to obtain K 2 Group 5 electrical signal

[0181] Specifically, if Figure 13 As shown, the photonic computing module 2c includes a plurality of (M×N in this embodiment) multiplier-accumulator MACs, each of which demultiplexes and sums the second optical signals of each group to obtain an eighth optical signal, and performs photoelectric conversion on the eighth optical signal based on the seventh optical signal to obtain a fifth electrical signal.

[0182] The following describes the structure of the multiply-accumulate-accumulator MAC when K=2 as an example.

[0183] Figure 14 Yes Figure 13 Schematic diagram of the structure of the multiply-accumulate MAC when K=2 and n<N-1, Figure 15 Yes Figure 13Schematic diagram of the structure of the multiplier-accumulator MAC when K = 2 and n = N - 1. As Figure 14 shown, when n < N - 1, that is, for Figure 13 the multiplier-accumulator MACs other than the last row of multiplier-accumulator MACs in each multiplier-accumulator MAC includes 2 third demultiplexers DeMUX, and the third demultiplexer DeMUX demultiplexes the K groups (2 groups) of second optical signals except for the last row 2 to obtain the fourth optical signals of K groups (4 groups) of each wavelength.

[0184] [Mathematical formula 18]

[0185]

[0186] It can be achieved by means of a look-up table and calibration

[0187] In addition, as Figure 15 shown, when n = N - 1, that is, for Figure 13 the last row of multiplier-accumulator MACs in each multiplier-accumulator MAC includes 2 third demultiplexers DeMUX, 4 fourth multiplexers MUX, 2 fifth optical splitters (optical splitter 1:2), 4 third photoelectric converters BPD, 4 transimpedance amplifiers TIA, and 4 low-pass filters. The third demultiplexer DeMUX demultiplexes the K groups (2 groups) of second optical signals of the last row 2 to obtain the fourth optical signals of K groups (4 groups) of each wavelength. The fourth multiplexer MUX sums the fourth optical signals of all rows except the last row from the matrix-vector multiplication with the fourth optical signals of the last row of the matrix-vector multiplication 2 to obtain the eighth optical signals of K In Figure 15 the eighth optical signal is the optical signal input to each third photoelectric converter BPD. The fifth optical splitter (optical splitter 1:2) divides the K groups (2 groups) of seventh optical signals (λ0, λ1) into K 2The third photoelectric converter BPD performs photoelectric conversion on the eighth optical signal based on the input ninth optical signal, thereby obtaining a fifth electrical signal.

[0188] Specifically, for Figure 15 , when n=N-1, the eighth optical signal after passing through the fourth multiplexer MUX can be expressed as the following formula (19).

[0189] [Mathematical formula 19]

[0190]

[0191] Assume that the expressions of the local oscillator optical signal (i.e., the ninth optical signal) are and The response rate of the third photoelectric converter BPD is R, and the amplification factor of the transimpedance amplifier TIA is γ, then the voltage of the output fifth electrical signal can be expressed by the following formula (20).

[0192] [Mathematical formula 20]

[0193]

[0194]

[0195] in, set up Then the fifth electrical signal can be expressed as the following formula (21).

[0196] [Mathematical formula 21]

[0197]

[0198] The coefficient η can be made equal to 1 by adjusting the amplification factor of the transimpedance amplifier TIA or the optical power of the local oscillator light (ie, the ninth optical signal).

[0199] In addition, in this embodiment, the fifth electrical signal is amplified and filtered by the transimpedance amplifier TIA and the low-pass filter, but the present invention is not limited thereto, and the transimpedance amplifier TIA and the low-pass filter may not be provided.

[0200] The above describes the case where K=2. For the case where K>2, the output result can be derived using the above method. In this case, 0≤u, v≤K-1.

[0201] Figure 16 It is a schematic diagram showing the structure of the electronic calculation module 3c involved in this embodiment 3.

[0202] Electronic calculation module 3c to K 2Group 5 electrical signal Perform multiplication and accumulation to obtain 2K-1 groups of third electrical signals, perform analog-to-digital conversion, shift register and sum on the corresponding third electrical signals in each group of third electrical signals, and thus calculate the matrix-vector product (Y0(t), ..., Y M-1 (t)) as the output vector.

[0203] Specifically, if Figure 16 As shown, for the fifth electrical signal If u=v=0 or u=v=K-1, the electronic calculation module 3c directly converts it into a digital signal using the analog-to-digital converter ADC For other u and v values, the digital signal after product accumulation can be expressed by the following formula (22):

[0204] [Mathematical formula 22]

[0205]

[0206] The calculation process of the optoelectronic hybrid computing system according to this embodiment is described below with examples.

[0207] Assume that the goal of the optoelectronic hybrid computing system is to calculate the following matrix-vector multiplication:

[0208]

[0209] The number of bitslices is K=2, the number of bits in each slice is b=2, and the MVM dimension is M=N=2. Therefore, the number of bits in each operand is Kb. The high and low bit combinations of weights and vectors can be expressed as:

[0210]

[0211] Therefore, the entire calculation process of the optoelectronic hybrid computing system can be expressed as:

[0212]

[0213] For the optoelectronic hybrid computing system involved in this embodiment, the photon computing module 2c performs the dot multiplication operation and detects the fifth electrical signal through the third photoelectric converter BPD therein. Wherein, m=0…M-1, n=0…N-1, k=0…2K-2. Then, the electronic calculation module 3c completes Y m The summation of (t) and the shift register operation therein, where m=0…M-1.

[0214] Right now,

[0215]

[0216] According to the optoelectronic hybrid computing system of this embodiment, a coherent architecture is used to implement K×b bit matrix-vector multiplication, such as Figure 13 As shown, it is possible to use fewer lasers (λ0…λ K-1 ), and fewer balanced detectors (ie, the third photoelectric converter BPD) to complete the matrix-vector multiplication.

[0217] The above is not intended to limit the optoelectronic hybrid computing system of the present invention. The scope of the present invention is indicated by the claims, not by the above embodiments, and includes all modifications and variations within the meaning and scope equivalent to the claims.

[0218] Industrial applicability

[0219] As described above, the optoelectronic hybrid computing system according to the present invention is useful for realizing high-precision matrix-vector multiplication calculations with bit division on a chip.

[0220] Label Description

[0221] 1 laser module

[0222] 2. 2a, 2b, 2c Photonic Computing Modules

[0223] 3. 3a, 3b, 3c electronic computing modules

[0224] 10 storage units

[0225] 20, 20a, 20b drive unit

[0226] 30 optoelectronic hybrid computing units.

Claims

1. An optoelectronic hybrid computing system, wherein the optoelectronic hybrid computing system utilizes a system with b-bit computing accuracy to perform K×b-bit matrix-vector multiplication, where K and b are both positive integers, and the optoelectronic hybrid computing system is characterized by comprising: A storage unit storing a first input vector consisting of K×b-bit elements and a first weight matrix; a driving unit, the driving unit splitting the first input vector and the first weight matrix into K second input vectors and a second weight matrix each consisting of b-bit elements, and performing digital-to-analog conversion of the K second input vectors and the K second weight matrix elements into K groups of first electrical signals and K groups of second electrical signals; as well as An optoelectronic hybrid computing unit is provided, which modulates K groups of the first electrical signals and K groups of the second electrical signals into the first optical signals of corresponding groups of the first optical signals divided into K groups based on wavelength, performs multiplication and accumulation, intra-group summation and optoelectronic conversion on the K groups of second optical signals obtained thereby, obtains 2K-1 groups of third electrical signals, performs analog-to-digital conversion, shift registration and summation on the corresponding third electrical signals in each group of the third electrical signals, thereby obtaining an output vector of the matrix-vector multiplication.

2. The optoelectronic hybrid computing system according to claim 1, wherein: The driving unit includes: a data splitting module, wherein the data splitting module splits the first input vector and the first weight matrix into K second input vectors and K second weight matrices respectively; a time division multiplexing module, which extracts, according to the signs of each element of the second input vector and the second weight matrix, the absolute values ​​of the elements of the second input vector and the second weight matrix whose signs are the same, and combines them with 0 to reconstruct the second input vector and the second weight matrix in a first clock cycle; and extracts, in a second clock cycle, the absolute values ​​of the elements of the second input vector and the second weight matrix whose signs are different, and combines them with 0 to reconstruct the second input vector and the second weight matrix; and A digital-to-analog conversion module converts the reconstructed K second input vectors and the K second weight matrices into K groups of first electrical signals and K groups of second electrical signals respectively.

3. The optoelectronic hybrid computing system according to claim 2, wherein: The optoelectronic hybrid computing unit comprises: A laser module, which outputs K lasers with different wavelengths; a photon computing module, wherein the photon computing module divides K lasers into K groups of first optical signals, wherein the wavelength of the first optical signals in each group is the same, the photon computing module modulates the K groups of first electrical signals and the K groups of second electrical signals into the K groups of first optical signals to obtain K groups of second optical signals, performs multiplication and accumulation on the K groups of second optical signals to obtain 2K-1 groups of third optical signals, and performs photoelectric conversion on the 2K-1 groups of third optical signals to obtain 2K-1 groups of fourth electrical signals; an electronic calculation module, which performs intra-group summation on 2K-1 groups of the fourth electrical signals to obtain 2K-1 groups of the third electrical signals, performs analog-to-digital conversion, shift registers, and sums the corresponding third electrical signals in each group of the third electrical signals, thereby calculating a matrix-vector product; and A difference calculation module is configured to subtract the product calculated in the first clock cycle from the product calculated in the second clock cycle, thereby obtaining the output vector.

4. The optoelectronic hybrid computing system according to claim 3, wherein: The photon computing module includes: a first optical splitter, which splits the laser light of each wavelength into a group of first optical signals with the same wavelength; an electro-optical modulator, configured to modulate the first electrical signal into the first optical signal; a weight controller, configured to modulate the second electrical signal into the first optical signal modulated with the first electrical signal to obtain the second optical signal; and A multiply-accumulator is configured to perform multiplication and accumulation on the corresponding second optical signals in the K groups of the second optical signals between the groups to obtain the third optical signal, and convert the third optical signal into the fourth electrical signal.

5. The optoelectronic hybrid computing system according to claim 4, wherein: The multiply-accumulator comprises: A first demultiplexer is used to demultiplex K groups of the second optical signals to obtain K 2 a fourth optical signal of each wavelength; a first multiplexer configured to multiply and accumulate the fourth optical signals other than the first and last two groups of fourth optical signals between groups, and then combine the fourth optical signals with the first and last two groups of fourth optical signals to form 2K-1 groups of third optical signals; and A first photoelectric converter is provided, wherein the first photoelectric converter converts the third optical signal into the fourth electrical signal.

6. The optoelectronic hybrid computing system according to claim 2, wherein: The optoelectronic hybrid computing unit comprises: a laser module, the laser module outputting K groups of lasers with different wavelengths, wherein the wavelengths of the lasers in each group are different from each other; a photon computing module, wherein the photon computing module uses K groups of the lasers as K groups of the first optical signals, the photon computing module modulates the K groups of the first electrical signals and the K groups of the second electrical signals into the K groups of the first optical signals to obtain K groups of the second optical signals, performs multiplication and accumulation and intra-group summation on the K groups of the second optical signals to obtain 2K-1 groups of third optical signals, and performs photoelectric conversion on the third optical signals to obtain 2K-1 groups of third electrical signals; an electronic calculation module, which performs analog-to-digital conversion, shift registering, and summing on the corresponding third electrical signals in each group of the third electrical signals, thereby calculating a matrix-vector product; and A difference calculation module is configured to subtract the product calculated in the first clock cycle from the product calculated in the second clock cycle, thereby obtaining the output vector.

7. The optoelectronic hybrid computing system according to claim 6, wherein: The photon computing module includes: an electro-optical modulator, configured to modulate the first electrical signal into the first optical signal; a weight controller, configured to modulate the second electrical signal into the first optical signal modulated with the first electrical signal to obtain the second optical signal; and A multiply-accumulator performs multiplication and accumulation on the corresponding second optical signals in the K groups of the second optical signals between groups, performs summation within the groups, obtains the third optical signal, and converts the third optical signal into the third electrical signal.

8. The optoelectronic hybrid computing system according to claim 7, wherein: The multiply-accumulator comprises: A second demultiplexer is used to demultiplex K groups of the second optical signals to obtain K 2 a fourth optical signal of each wavelength; a second multiplexer, wherein the second multiplexer multiplies and accumulates the fourth optical signals except the first and last two groups of the fourth optical signals between groups for all rows of the matrix-vector multiplication except the last row, and forms 2K-1 groups of fifth optical signals together with the first and last two groups of the fourth optical signals; a third multiplexer, which, for the last row of the matrix-vector multiplication, performs intra-group summing and product accumulation on the fourth optical signals other than the first and last two groups of the fourth optical signals and the fifth optical signals other than the first and last two groups of the fifth optical signals, and combines the summed signals with a sixth optical signal obtained by intra-group summing of the first and last two groups of the fourth optical signals and the first and last two groups of the fifth optical signals to form 2K-1 groups of the third optical signals; and A second photoelectric converter is provided, which converts the third optical signal into the third electrical signal.

9. The optoelectronic hybrid computing system according to claim 1, wherein: The driving unit includes: a data splitting module, which splits the first input vector and the first weight matrix into K second input vectors and K second weight matrices respectively; and A digital-to-analog conversion module converts the K second input vectors and the K elements of the second weight matrix into K groups of the first electrical signals and K groups of the second electrical signals.

10. The optoelectronic hybrid computing system according to claim 9, wherein: The optoelectronic hybrid computing unit comprises: A laser module, which outputs K lasers with different wavelengths; A photon calculation module, wherein the photon calculation module splits each of the K lasers into two to obtain K first splits and K second splits, divides the K first splits into K groups of first optical signals, and the wavelengths of the first optical signals in each group are the same, and divides the K second splits into K groups of seventh optical signals, and the wavelengths of the seventh optical signals in each group are the same, and the photon calculation module modulates the K groups of first electrical signals and the K groups of second electrical signals into the K groups of first optical signals to obtain K groups of second optical signals, demultiplexes each group of second optical signals and sums them within the group to obtain K 2 The eighth optical signal is grouped and the eighth optical signal is photoelectrically converted based on the seventh optical signal to obtain K 2 a fifth electrical signal; and Electronic calculation module, which calculates K 2 The fifth electrical signals of the groups are multiplied and added to obtain 2K-1 groups of the third electrical signals, and the corresponding third electrical signals in each group of the third electrical signals are analog-to-digital converted, shifted and registered, and summed, so as to calculate the product of the matrix vector as the output vector.

11. The optoelectronic hybrid computing system according to claim 10, wherein: The photon computing module includes: a second beam splitter, which splits the K laser beams into K first beams and K second beams; a third optical splitter, configured to split the first light of each wavelength into a group of first optical signals having the same wavelength; a fourth optical splitter, configured to split the second optical signal of each wavelength into a group of seventh optical signals having the same wavelength; an electro-optical modulator, configured to modulate the first electrical signal into the first optical signal; a weight controller, configured to modulate the second electrical signal into the first optical signal modulated with the first electrical signal to obtain the second optical signal; A multiplier-accumulator is configured to demultiplex and perform intra-group summation on each group of the second optical signals to obtain the eighth optical signal, and to perform photoelectric conversion on the eighth optical signal based on the seventh optical signal to obtain the fifth electrical signal.

12. The optoelectronic hybrid computing system according to claim 11, wherein: The multiply-accumulator comprises: A third demultiplexer demultiplexes K groups of the second optical signals to obtain K 2 a fourth optical signal of each wavelength; A fourth multiplexer is configured to perform intra-group summation on the fourth optical signals of all rows except the last row from the matrix-vector multiplication and the fourth optical signal of the last row of the matrix-vector multiplication to obtain K 2 grouping the eighth optical signal; A fifth optical splitter is used to split the K groups of the seventh optical signals into K 2 a ninth optical signal; and A third photoelectric converter performs photoelectric conversion on the eighth optical signal based on the ninth optical signal, thereby obtaining the fifth electrical signal.

Images