An integrated balanced photon universal high-dimensional matrix operation system

Abstract

The disclosed embodiments relate to the field of photon computing technology, and provide an integrated balanced photon general high-dimensional matrix operation system, including a silicon photonic chip and a drive control circuit chip; the silicon photonic chip is used to encode different radio frequency signal data and digital electronic signals into an optical analog domain, and based on the encoded optical analog signal, a matrix operation is performed through a balanced photon cross array to obtain a corresponding matrix operation result; the drive control circuit chip is used to generate different radio frequency signal data, drive the silicon photonic chip to encode different radio frequency signal data and digital electronic signals into the optical analog domain; and, the matrix operation result is electrically read out to obtain the final result of the matrix operation. The disclosed embodiments can realize general high-dimensional matrix operations in the entire complex domain, and have the advantages of high energy efficiency, high throughput, low latency, high scalability and flexibility, and have great application potential in artificial intelligence applications and cloud services.

Description

Technical Field

[0001] The present disclosure relates to the field of photon computing technology, and in particular to an integrated balanced photon universal high-dimensional matrix operation system. Background Art

[0002] Artificial Intelligence (AI) applications, the Internet of Things, cloud services, etc. have an increasingly urgent need to efficiently process massive amounts of data. A study found that the amount of computing required to run a top-level deep neural network (Deep Ne) is growing at twice the rate of hardware efficiency. At present, AI's capabilities and application scope in multiple disciplines and fields such as medical imaging and diagnosis, material discovery, autonomous navigation, and natural language processing are constantly expanding. For example, the existing technology already has natural language processing tools such as ChatGPT, AI painting tools Midjourney AI, and image generation systems such as Dall.E. Therefore, as the global demand for computing power continues to grow, improving computing throughput and efficiency has become an important challenge.

[0003] The limited efficiency of traditional computing hardware has prompted increased investment in hardware parallelism, but the generality and accuracy of Deep Neural Networks (DNNs) is proportional to the amount of training data and available computing, which is also their Achilles' heel. While Graphics Processing Units (GPUs) have historically facilitated continued progress in deep learning, this is due to their suitability for distributed training of DNNs on clusters of large numbers of independent nodes, rather than significantly improving the computational throughput of a single node. This distributed deep learning approach can take months, cost millions of dollars in computing services, and emit hundreds of tons of carbon dioxide. Following these trends, continued progress in the field of DNNs using traditional computing hardware is unsustainable both economically and environmentally.

[0004] In the analog domain, optical computing shows the potential to improve efficiency and throughput for AI applications and has shown significant experimental progress in the past few years. Various photonic architectures such as cascaded Mach-Zehnder interferometers, in-memory computing, reconfigurable metasurfaces, frequency comb shaping, and neuromorphic computing have demonstrated the feasibility of analog computing in the photonic domain. However, most of these approaches rely on fixed photon weights and high-speed photodetectors and analog to digital converters (ADCs) to convert the results of optical matrix-vector multiplication (MVM) back to the digital domain for further processing. Therefore, the optoelectronic readout circuit must run at the same speed as the input electro-optic modulator, which limits the overall throughput and energy efficiency of the photonic accelerator. In addition, unlike digital-to-analog conversion, which can be very efficient, the conversion from analog to digital domain is not simple, and the energy consumption increases with the operating frequency of the ADC. Therefore, the overall energy consumption of the readout circuit accounts for the majority of the power consumption of many analog computing systems, which is proportional to the matrix size. In addition, the significant difference in the physical size of photonic and electronic components poses a challenge to weight-fixed architectures, which often require constant reconfiguration of weight matrices during computation, which can result in computational efficiency that is even lower than that of traditional digital hardware. To address this challenge, Hamerly et al. proposed a new approach that implements large-scale multiplication and accumulation operations in the optical domain through in-phase detection. However, these free-space architectures require precise and stable control of multiple optical path lengths in the system to ensure temporal coherence, and customized diffractive optical elements to ensure spatial coherence. At the same time, the modulation speed of the spatial light modulators used is currently only about 1 kHz or less, which greatly reduces computational efficiency. Summary of the invention

[0005] The present disclosure aims to solve at least one of the problems existing in the prior art and provide an integrated balanced photon universal high-dimensional matrix operation system.

[0006] The present disclosure provides an integrated balanced photon universal high-dimensional matrix operation system, the system comprising:

[0007] Silicon photonic chips are used to encode different radio frequency signal data and digital electronic signals into the optical analog domain, and perform matrix operations based on the encoded optical analog signals through a balanced photon cross array to obtain corresponding matrix operation results;

[0008] A driving control circuit chip is used to generate the different RF signal data, drive the silicon photonic chip to encode the different RF signal data and the digital electronic signal into the optical analog domain; and electrically read out the matrix operation result to obtain the final result of the matrix operation.

[0009] Optionally, the silicon photonic chip includes a silicon-based laser, a silicon photonic beam splitter, a row input modulator array, a column input modulator array, a row silicon photonic DAC array, a column silicon photonic DAC array, and a silicon-based balanced photonic crossbar array; wherein,

[0010] The silicon-based laser is used to generate input light;

[0011] The silicon optical beam splitter is used to split the input light generated by the silicon-based laser into M row input lights and N column input lights;

[0012] The row input modulator array and the column input modulator array are used to load different radio frequency signal data to the corresponding row input lights and column input lights, respectively, to obtain each row modulated light and each column modulated light, respectively;

[0013] The row silicon photonic DAC array is used to perform complex domain digital encoding on each row of modulated light to obtain each row of input optical signals;

[0014] The column silicon photonic DAC array is used to perform complex domain digital encoding on each column of modulated light to obtain each column of input optical signal;

[0015] The silicon-based balanced photonic crossbar array is used to perform matrix operations based on each of the row input optical signals and each of the column input optical signals to obtain the matrix operation results.

[0016] Optionally, the silicon-based balanced photonic cross array includes M×N array elements, each of which includes a waveguide cross, two 1×2 directional couplers, a 2×2 waveguide coupler, and a waveguide balanced detector; wherein,

[0017] The input ends of the waveguide cross are respectively used to receive the corresponding row input optical signals and the column input optical signals, and the output ends of the waveguide cross are respectively connected to the input ends of each of the 1×2 directional couplers, so that the row input optical signals and the column input optical signals respectively enter the corresponding 1×2 directional couplers;

[0018] In each of the array elements, one path of output light from each of the 1×2 directional couplers enters the waveguide balanced detector after interference through the 2×2 waveguide coupler, so as to perform balanced detection and time integration through the waveguide balanced detector, thereby realizing matrix operations between the corresponding row input optical signal and the column input optical signal; the other path of output light from each of the 1×2 directional couplers is used to transmit the corresponding row input optical signal or the column input optical signal to the next array element.

[0019] Optionally, in the silicon-based balanced photon cross array, the coupling coefficient of two adjacent 1×2 directional couplers in the same row or column satisfies K n =K n+1 / (1 / η+ K n+1 ), where K n represents the coupling coefficient of the nth 1×2 directional coupler, K n+1 represents the coupling coefficient of the next 1×2 directional coupler adjacent to the nth 1×2 directional coupler in the same row or column, and η represents the insertion loss of the array element.

[0020] Optionally, the drive control circuit chip includes:

[0021] A filter array, used for performing input equalization processing on each matrix element in the matrix operation result;

[0022] The transimpedance amplifier array is used to perform photoelectric conversion and amplification processing on each matrix element after input equalization processing to obtain the final result of the matrix operation.

[0023] Optionally, the drive control circuit chip further includes:

[0024] The silicon photonics DAC driver array is used to generate a plurality of adjustable voltages to drive the row silicon photonics DAC array and the column silicon photonics DAC array through each of the adjustable voltages.

[0025] Optionally, the silicon photonic DAC driver array comprises a microcontroller unit, a multi-channel electrical DAC array and an adjustment output stage; wherein,

[0026] The microcontrol unit is used to dynamically configure the multi-channel electrical DAC array to output multiple adjustable voltages online;

[0027] The adjustment output stage is used for power promotion, outputting each of the adjustable voltages to the corresponding row silicon photonic DAC array and the column silicon photonic DAC array to drive the row silicon photonic DAC array and the column silicon photonic DAC array.

[0028] Optionally, the drive control circuit chip further includes:

[0029] A radio frequency generator array, used to generate a plurality of radio frequency signals of different frequencies;

[0030] A mixer array, used to load data into radio frequency signals of different frequencies respectively to obtain data of different radio frequency signals;

[0031] An electronic combiner array is used to respectively combine different RF signal data corresponding to the same object in the row input modulator array or the column input modulator array into one path as a corresponding RF input signal, and output each RF input signal to the corresponding object in the row input modulator array or the column input modulator array.

[0032] Optionally, the RF generator array includes a field programmable gate array and an operational amplifier stack; wherein,

[0033] The field programmable gate array is used to set the frequency of each radio frequency signal respectively;

[0034] The operational amplifier stack is used to generate corresponding radio frequency signals of each frequency according to the frequency of each radio frequency signal set by the field programmable gate array.

[0035] Optionally, the silicon photonic chip and the driving control circuit chip are integrated using an optoelectronic hybrid integration method, and the optoelectronic hybrid integration method includes any one of a monolithic optoelectronic hybrid integration method, a flip-chip bonding method, and a through silicon via three-dimensional stacking method.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. Utilize silicon-based photonic crossbar arrays and balanced detection to perform matrix operations, decoupling the modulation frequency of the optical input from the speed of the readout circuit, and eliminating the need to frequently reprogram the photon weights, significantly reducing energy consumption and latency in the limit of large matrices.

[0038] 2. The differential characteristics of balanced detection allow universal matrix operations in the entire complex domain to be achieved by controlling the phase and amplitude of the input light. Balanced detection can eliminate common-mode noise, allowing the use of extremely low optical power, greatly improving the signal-to-noise ratio.

[0039] 3. By increasing the time and radio frequency degrees of freedom, additional dimensions are added to the data, so that high-dimensional matrix operations can be obtained, which improves the system parallelism and throughput. At the same time, time and radio frequency coding makes the physical size of photonic hardware no longer limit the maximum dimension of matrix operations, greatly improving the flexibility, scalability and computing efficiency of the system.

[0040] 4. The optoelectronic hybrid integration technology is used to integrate and package the silicon photonic chip composed of silicon-based devices and the driving control circuit chip, and the wafer is taped out based on the CMOS (Complementary Metal Oxide Semiconductor) process, which can greatly improve the system integration, significantly reduce the production cost, and realize mass production. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0042] Figure 1 A schematic diagram of the structure of an integrated balanced photon universal high-dimensional matrix operation system provided in one embodiment of the present disclosure;

[0043] Figure 2 A schematic diagram of the working principle of an integrated balanced photon universal high-dimensional matrix operation system provided by another embodiment of the present disclosure;

[0044] Figure 3 A schematic diagram of an integrated balanced photon universal high-dimensional matrix computing system provided by another embodiment of the present disclosure, which is integrated by a flip-chip bonding method;

[0045] Figure 4 A schematic diagram of an integrated balanced photon universal high-dimensional matrix computing system provided in another embodiment of the present disclosure, which is integrated using a through silicon via three-dimensional stacking method. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings. However, it can be understood by those skilled in the art that in each embodiment of the present disclosure, many technical details are proposed in order to enable readers to better understand the present disclosure. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed for protection in the present disclosure can also be implemented. The division of the following embodiments is for the convenience of description and should not constitute any limitation on the specific implementation of the present disclosure. The various embodiments can be combined and referenced with each other without contradiction.

[0047] One embodiment of the present disclosure relates to an integrated balanced photon universal high-dimensional matrix operation system, such as Figure 1 As shown, it includes a silicon photonic chip 15 and a driving control circuit chip 14 .

[0048] The silicon photonic chip 15 is used to encode different radio frequency signal data and digital electronic signals into the optical analog domain, and based on the encoded optical analog signal, matrix operations are performed through a balanced photon cross array to obtain corresponding matrix operation results.

[0049] The driving control circuit chip 14 is used to generate different RF signal data, drive the silicon photonic chip 15 to encode different RF signal data and digital electronic signals into the optical analog domain; and perform electrical readout processing on the matrix operation results to obtain the final result of the matrix operation.

[0050] Compared with the existing technology, the integrated balanced photon universal high-dimensional matrix operation system provided in the embodiments of the present disclosure can realize universal high-dimensional matrix operations in the entire complex domain, has the advantages of high energy efficiency, high throughput, low latency, high scalability and flexibility, and has great application potential in artificial intelligence applications and cloud services.

[0051] For example, Figure 1 As shown, the silicon photonic chip 15 includes a silicon-based laser 1, a silicon photonic beam splitter 2, a row input modulator array 3, a column input modulator array 4, a row silicon photonic DAC array 5, a column silicon photonic DAC array 6, and a silicon-based balanced photonic crossbar array 7.

[0052] The silicon-based laser 1 is used to generate input light. The silicon optical beam splitter 2 is used to split the input light generated by the silicon-based laser 1 into M row input lights and N column input lights. Specifically, the silicon optical beam splitter 2 can first split the input light generated by the silicon-based laser 1 into two beams of light, one of which is used as a row input light source and the other as a column input light source, and then split the row input light source into M beams of light to obtain M row input lights, and split the column input light source into N beams of light to obtain N column input lights. The silicon optical beam splitter 2 is a silicon-based waveguide device, and its splitting ratio can be adjusted according to the row input light path loss and the column input light path loss to ensure that the power of each beam of light is consistent.

[0053] The row input modulator array 3 and the column input modulator array 4 are respectively used to load different RF signal data to the corresponding row input light and column input light, so as to obtain each row modulated light and each column modulated light respectively. Specifically, the row input modulator array 3 includes M modulators, each modulator corresponds to a row input light, and each modulator is used to load each RF signal data to the corresponding row input light to obtain the corresponding row modulated light. The column input modulator array 4 includes N modulators, each modulator corresponds to a column input light, and each modulator is used to load each RF signal data to the corresponding column input light to obtain the corresponding column modulated light. By loading the data encoded in different RF signals, that is, different RF signal data into the optical domain, additional dimensions can be added to the data to be processed, further improving the flexibility of the integrated balanced photonic general high-dimensional matrix operation system.

[0054] The row silicon photonic DAC array 5 is used to digitally encode each row of modulated light in the complex domain to obtain each row of input optical signals. Specifically, the row silicon photonic DAC array 5 includes M silicon-based segmented modulators, each of which includes a phase modulation part and an amplitude modulation part, and can modulate the phase and amplitude of the row input light under voltage drive, so that the digital electronic signal can be directly encoded into the optical analog domain, realizing the complex domain digital encoding of each row of modulated light, and obtaining the corresponding row input optical signal, wherein the digital electronic signal encoded to the i-th row of modulated light at time t can be expressed as .

[0055] The column silicon photonic DAC array 6 is used to digitally encode each column modulated light in the complex domain to obtain each column input optical signal. Specifically, the column silicon photonic DAC array 6 includes N silicon-based segmented modulators, each of which also includes a phase modulation part and an amplitude modulation part, which can modulate the phase and amplitude of the column input light under voltage drive, so that the digital electronic signal can be directly encoded into the optical analog domain, realizing the complex domain digital encoding of each column modulated light, and obtaining the corresponding column input optical signal, wherein the digital electronic signal encoded to the j-th column modulated light at time t can be expressed as .

[0056] This embodiment can realize high-speed digital-to-analog conversion in the entire complex domain through the row silicon photonic DAC array 5 and the column silicon photonic DAC array 6, thereby improving the computational efficiency of the system.

[0057] The silicon-based balanced photonic crossbar array 7 is used to perform matrix operations based on input optical signals in each row and input optical signals in each column to obtain matrix operation results.

[0058] Specifically, the silicon-based balanced photonic crossbar array 7 can perform balanced detection and time integration on each row of input optical signals output by the row silicon photonic DAC array 5 and each column of input optical signals output by the column silicon photonic DAC array 6, thereby realizing corresponding matrix operations and obtaining corresponding matrix operation results.

[0059] Exemplarily, the silicon-based balanced photon cross array 7 includes M×N array elements, each of which includes a waveguide cross, two 1×2 directional couplers, a 2×2 waveguide coupler, and a waveguide balanced detector. Each array element corresponds to a row input optical signal and a column input optical signal, wherein the row input optical signal is a row input matrix element, and the column input optical signal is a column input matrix element. The silicon-based balanced photon cross array 7 realizes the operation between the corresponding row input matrix element and the corresponding column input matrix element through each array element, thereby realizing the matrix operation based on each row input optical signal and each column input optical signal.

[0060] Specifically, the waveguide cross includes two input ends and two output ends. The input ends of the waveguide cross are respectively used to receive corresponding row input optical signals and column input optical signals, and the output ends of the waveguide cross are respectively connected to the input ends of each 1×2 directional coupler so that the row input optical signal and the column input optical signal enter the corresponding 1×2 directional coupler respectively.

[0061] In each array element, one output light of each 1×2 directional coupler enters the waveguide balanced detector after interference through the 2×2 waveguide coupler, so as to perform balanced detection and time integration through the waveguide balanced detector to realize the matrix operation between the corresponding row input optical signal and the column input optical signal; the other output light of each 1×2 directional coupler is used to transmit the corresponding row input optical signal or column input optical signal to the next array element.

[0062] Specifically, in each array element, two 1×2 directional couplers respectively output one light path to the 2×2 waveguide coupler, the 2×2 waveguide coupler interferes with the two input lights, and inputs the interfered two lights to the waveguide balanced detector, the waveguide balanced detector performs balanced detection and time integration on the two input lights, and realizes the multiplication operation between the corresponding row input light signal and the corresponding column input light signal. The silicon-based balanced photon cross array 7 can realize the operation between the matrix corresponding to each row input light signal and the matrix corresponding to each column input light signal by using each array element to realize the multiplication operation between the corresponding row input light signal and the corresponding column input light signal.

[0063] For example, to obtain a specific coupling coefficient and transmission parameter, the coupling coefficient of two adjacent 1×2 directional couplers in the same row or column in the silicon-based balanced photon cross array 7 satisfies K n =K n+1 / (1 / η+ K n+1 ). Among them, K n represents the coupling coefficient of the nth 1×2 directional coupler, K n+1 K represents the coupling coefficient of the next 1×2 directional coupler adjacent to the nth 1×2 directional coupler in the same row or column, and η represents the insertion loss of the array element. In other words, K n It represents the coupling coefficient of the first 1×2 directional coupler in the two adjacent 1×2 directional couplers in the same row or column, K n+1 It represents the coupling coefficient of the latter 1×2 directional coupler among two adjacent 1×2 directional couplers in the same row or column.

[0064] For example, Figure 1 As shown, the drive control circuit chip 14 includes a filter array 8 and a trans-impedance amplifier (TIA) array 9. The filter array 8 is used to perform input equalization processing on each matrix element in the matrix operation result. The trans-impedance amplifier array 9 is used to perform photoelectric conversion and amplification processing on each matrix element after input equalization processing to obtain the final result of the matrix operation.

[0065] Specifically, the filter array 8 and the transimpedance amplifier array 9 are used to read out the matrix operation results. The filter array 8 includes M×N filters, each of which corresponds to an array element in the silicon-based balanced photon cross array 7. Each matrix element in the matrix operation result obtained by the silicon-based balanced photon cross array 7 is represented by a differential photocurrent output by each array element in the silicon-based balanced photon cross array 7, and each differential photocurrent is input balanced through a corresponding filter.

[0066] The transimpedance amplifier array 9 includes M×N transimpedance amplifiers, each of which corresponds to a filter in the filter array 8. Each transimpedance amplifier performs photoelectric conversion and amplification processing on each matrix element output by each filter after input equalization processing to obtain each final matrix element. Each final matrix element constitutes the final result of the matrix operation.

[0067] For example, Figure 1 As shown, the driving control circuit chip 14 further includes a silicon photonics DAC driver array 10. The silicon photonics DAC driver array 10 is used to generate a plurality of adjustable voltages to drive the row silicon photonics DAC array 5 and the column silicon photonics DAC array 6 through the adjustable voltages.

[0068] Specifically, the silicon photonic DAC driver array 10 includes a microcontroller unit (MCU), a multi-channel electrical DAC array, and an adjustment output stage. The microcontroller unit is used to dynamically configure the multi-channel electrical DAC array to output multiple adjustable voltages online. The adjustment output stage is used to perform power promotion and output each adjustable voltage to the corresponding row silicon photonic DAC array 5 and column silicon photonic DAC array 6 to drive the row silicon photonic DAC array 5 and column silicon photonic DAC array 6.

[0069] The multi-channel electrical DAC array includes M+N DACs, i.e., digital-to-analog converters, wherein the M DACs correspond to a silicon-based segmented modulator in the row silicon photonic DAC array 5, and the remaining N DACs correspond to a silicon-based segmented modulator in the column silicon photonic DAC array 6. The silicon photonic DAC driver array 10 firstly dynamically configures each DAC in the multi-channel electrical DAC array online by a microcontroller unit to output an adjustable voltage required to drive the corresponding silicon-based segmented modulator, and then adjusts the output stage for power driving, outputs the required adjustable voltage to the corresponding silicon-based segmented modulator, so as to realize driving each silicon-based segmented modulator with each adjustable voltage.

[0070] This embodiment further improves the flexibility of the integrated balanced photonic general high-dimensional matrix operation system by utilizing a microcontroller unit to dynamically configure the driving voltages of the row silicon photonic DAC array and the column silicon photonic DAC array online.

[0071] For example, Figure 1 As shown, the driving control circuit chip 14 also includes a radio frequency generator array 11 , a mixer array 12 , and an electronic combiner array 13 .

[0072] The RF generator array 11 is used to generate multiple RF signals of different frequencies. The mixer array 12 is used to load data into RF signals of different frequencies to obtain different RF signal data. The electronic combiner array 13 is used to respectively combine the different RF signal data of the same object in the corresponding row input modulator array 3 or column input modulator array 4 into one path as the corresponding RF input signal, and output each RF input signal to the corresponding object in the row input modulator array 3 or column input modulator array 4.

[0073] Specifically, in order to match the row and column scale of the matrix operation, the RF generator array 11 includes 2×M×N RF generators, the mixer array 12 includes 2×M×N mixers, and the electronic combiner array 13 includes M+N electronic combiners, wherein the M electronic combiners correspond to one modulator in the row input modulator array 3, and the remaining N electronic combiners correspond to one modulator in the column input modulator array 4. For each electronic combiner corresponding to a modulator in the row input modulator array 3, it corresponds to N RF generators and N mixers, respectively, each RF generator corresponds to each mixer one by one, and each RF generator generates a RF signal of each frequency, each mixer loads data into the RF signal generated by its corresponding RF generator, and obtains a RF signal carrying data, and the electronic combiner synthesizes the N RF signals carrying data into one path as the corresponding RF signal data, and inputs the RF signal data into the modulator in the corresponding row input modulator array 3. Similarly, for each electronic combiner of the modulator in the corresponding column input modulator array 4, it corresponds to another M RF generators and another M mixers, each RF generator corresponds to each mixer one by one, and each RF generator generates a RF signal of each frequency, each mixer loads data into the RF signal generated by its corresponding RF generator to obtain a RF signal carrying data, and the electronic combiner synthesizes the M RF signals carrying data as the corresponding RF signal data, and inputs the RF signal data into the modulator in the corresponding column input modulator array 4.

[0074] Exemplarily, the RF generator array 11 includes a field programmable gate array and an operational amplifier stack. The field programmable gate array is used to set the frequency of each RF signal respectively. The operational amplifier stack is used to generate corresponding RF signals of each frequency according to the frequency of each RF signal set by the field programmable gate array.

[0075] For example, the field programmable gate array can set the frequencies of the RF signals generated by the N RF generators corresponding to the electronic combiner of each corresponding row input modulator array 3 to ƒ1-ƒ N , the frequencies of the RF signals generated by the M RF generators corresponding to the electronic combiners of each corresponding column input modulator array 4 are set to ƒ1-ƒ M After that, the operational amplifier stack can generate a corresponding RF signal according to the frequency of the RF signal set by the field programmable gate array, and input the RF signal to the corresponding modulator in the corresponding row input modulator array 3 or column input modulator array 4.

[0076] In order to enable those skilled in the art to better understand the above embodiments, Figure 2 right Figure 1 The working principle of the integrated balanced photon universal high-dimensional matrix operation system shown is explained.

[0077] First, the silicon-based laser 1 generates input light. Then, the silicon optical beam splitter 2 first splits the input light into two beams, one beam as a row input light source and the other beam as a column input light source. Then, the silicon optical beam splitter 2 splits the row input light source into M beams as M row input lights, and splits the column input light source into N beams as N column input lights.

[0078] Afterwards, the M modulators in the row input modulator array 3 respectively load the RF signal data input by the corresponding electronic combiner in the electronic combiner array 13 to the corresponding row input light, and obtain the corresponding row modulated light. The N modulators in the column input modulator array 4 respectively load the RF signal data input by the corresponding electronic combiner in the electronic combiner array 13 to the corresponding column input light, and obtain the corresponding column modulated light. The M silicon-based segmented modulators in the row silicon photonic DAC array 5, driven by the corresponding adjustable voltage output by the silicon photonic DAC driver array 10, perform complex domain digital encoding on the corresponding row modulated light, and convert the corresponding digital electronic signal The N silicon-based segmented modulators in the column silicon photonic DAC array 6 are driven by the corresponding adjustable voltage output by the silicon photonic DAC driver array 10 to perform complex domain digital encoding on the corresponding column modulated light and convert the corresponding digital electronic signal into The corresponding column modulated light is encoded to obtain the corresponding column input light signal.

[0079] After that, each row input optical signal and each column input optical signal enter the corresponding array element in the silicon-based balanced photon cross array 7 respectively. In each array element, the row input optical signal and the column input optical signal enter the 2×2 waveguide coupler through the corresponding waveguide in the waveguide cross and the 1×2 directional coupler connected to the corresponding waveguide for interference. The row input optical signal and the column input optical signal after interference enter the waveguide balanced detector, which performs balanced detection and time integration to realize the multiplication operation between the corresponding row input optical signal and the column input optical signal, and outputs the corresponding differential photocurrent as the operation result. The differential photocurrents output by each array element constitute the matrix operation result. The filter array 8 performs input balancing processing on each differential photocurrent output by the silicon-based balanced photon cross array 7 through each filter included therein. The transimpedance amplifier array 9 performs photoelectric conversion and amplification processing on the differential photocurrent after input balancing processing through each transimpedance amplifier included therein to obtain the final result of the matrix operation, thereby realizing the readout of the matrix operation result.

[0080] The process of generating the radio frequency signal data inputted by the electronic combiner in the electronic combiner array 13 to the row input modulator array 3 or the column input modulator array 4 is as follows: Figure 2 As shown, for each electronic beam combiner corresponding to the row input modulator array 3, it corresponds to N RF generators in the RF generator array 11 and N mixers in the mixer array 12, each RF generator corresponds to each mixer one by one, and each RF generator generates a frequency of ƒ1-ƒ N The RF signal, each mixer respectively converts the data x1, x2, ..., x k The RF signal is loaded into the RF signal generated by the corresponding RF generator to obtain the RF signal carrying the data. The electronic combiner combines the N RF signals carrying the data into one path as the corresponding RF signal data, and inputs the RF signal data into the modulator in the corresponding row input modulator array 3. Similarly, for each electronic combiner corresponding to the column input modulator array 4, it corresponds to another M RF generators and another M mixers, each RF generator corresponds to each mixer one by one, and each RF generator generates a frequency of ƒ1-ƒ N The RF signal, each mixer respectively converts the data x1, x2, ..., x k The RF signal is loaded into the RF signal generated by the corresponding RF generator to obtain an RF signal carrying data. The electronic combiner combines M RF signals carrying data into one path as the corresponding RF signal data, and inputs the RF signal data into the modulator in the corresponding column input modulator array 4.

[0081] The process of generating the adjustable voltage output by the silicon photonic DAC driver array 10 is as follows: the silicon photonic DAC driver array 10 dynamically configures the multi-channel electrical DAC array online through a microcontroller unit to output the M adjustable voltages required to drive the row silicon photonic DAC array 5 and the N adjustable voltages required to drive the column silicon photonic DAC array 6, and then adjusts the output stage for power promotion to output the corresponding adjustable voltages to the row silicon photonic DAC array 5 and the column silicon photonic DAC array 6 respectively.

[0082] Exemplarily, the silicon photonic chip 15 and the driving control circuit chip 14 are integrated using an optoelectronic hybrid integration method, which includes any one of a monolithic optoelectronic hybrid integration method, a flip-chip bonding method, and a through silicon via (TSV) three-dimensional stacking method.

[0083] Specifically, the monolithic optoelectronic hybrid integration method utilizes a special CMOS process or a BiCMOS (bipolar-CMOS) process to integrate the silicon photonic chip 15 and the driving control circuit chip 14 on a single chip, which can effectively reduce parasitic effects and reduce power consumption.

[0084] The flip-chip method is to use the bump hot pressing method to flip-chip interconnect the silicon photonic chip 15 and the drive control circuit chip 14. Figure 3 When the silicon photonic chip 15 and the driving control circuit chip 14 are integrated by the flip-chip method, alignment marks are first made at the positions where the silicon photonic chip 15 and the driving control circuit chip 14 need to be connected correspondingly, and then metal bumps are prepared on one side of the electrode of the silicon photonic chip 15. After alignment by the alignment marks, the silicon photonic chip 15 is flipped on the driving control circuit chip 14 by the hot pressing / sound pressure process, and the input and output electrodes on the silicon photonic chip 15 and the input and output electrodes on the driving control circuit chip 14 are connected correspondingly, and finally cured and filled to realize the interconnection between the silicon photonic chip 15 and the driving control circuit chip 14. The driving control circuit chip 14 can be electrically connected to the ceramic substrate by wire bonding to realize ceramic tube shell packaging, thereby obtaining an integrated balanced photon universal high-dimensional matrix operation system.

[0085] The TSV three-dimensional stacking method is to thin the silicon photonic chip 15 and the drive control circuit chip 14, then etch the TSV through hole to complete the electrical interconnection between the chips, and then realize the three-dimensional stacking integration between the chips through direct bonding. Figure 4After thinning the silicon photonic chip 15 and the driving control circuit chip 14, TSV through holes are etched at the positions where corresponding connections are required, and the input and output electrodes on the silicon photonic chip 15 and the input and output electrodes on the driving control circuit chip 14 are correspondingly connected through the TSV through holes to complete the electrical interconnection between the silicon photonic chip 15 and the driving control circuit chip 14. Then, wafer-level bonding is performed through direct bonding technology to achieve three-dimensional stacking integration between the silicon photonic chip 15 and the driving control circuit chip 14, and finally an integrated balanced photonic general high-dimensional matrix computing system is obtained.

[0086] The integrated balanced photon universal high-dimensional matrix operation system provided by the embodiments of the present disclosure has the following beneficial effects compared with the prior art:

[0087] 1. Utilize silicon-based photonic crossbar arrays and balanced detection to perform matrix operations, decoupling the modulation frequency of the optical input from the speed of the readout circuit, and eliminating the need to frequently reprogram the photon weights, significantly reducing energy consumption and latency in the limit of large matrices.

[0088] 2. The differential characteristics of balanced detection allow universal matrix operations in the entire complex domain to be achieved by controlling the phase and amplitude of the input light. Balanced detection can eliminate common-mode noise, allowing the use of extremely low optical power, greatly improving the signal-to-noise ratio.

[0089] 3. By increasing the time and radio frequency degrees of freedom, additional dimensions are added to the data, so that high-dimensional matrix operations can be obtained, which improves the system parallelism and throughput. At the same time, time and radio frequency coding makes the physical size of photonic hardware no longer limit the maximum dimension of matrix operations, greatly improving the flexibility, scalability and computing efficiency of the system.

[0090] 4. The optoelectronic hybrid integration technology is used to integrate and package the silicon photonic chip composed of silicon-based devices and the drive control circuit chip, and the wafer is taped out based on the CMOS process, which can greatly improve the system integration, significantly reduce the production cost, and realize mass production.

[0091] Those skilled in the art will appreciate that the above-mentioned embodiments are specific embodiments for implementing the present disclosure, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present disclosure.

Claims

1. An integrated balanced photon universal high-dimensional matrix operation system, characterized in that: The system comprises: Silicon photonic chips are used to encode different radio frequency signal data and digital electronic signals into the optical analog domain, and perform matrix operations based on the encoded optical analog signals through a balanced photon cross array to obtain corresponding matrix operation results; A driving control circuit chip is used to generate the different radio frequency signal data, drive the silicon photonic chip to encode the different radio frequency signal data and the digital electronic signal into the optical analog domain; and perform electrical readout processing on the matrix operation result to obtain the final result of the matrix operation; The silicon photonic chip includes a silicon-based laser, a silicon photonic beam splitter, a row input modulator array, a column input modulator array, a row silicon photonic DAC array, a column silicon photonic DAC array, and a silicon-based balanced photonic crossbar array; wherein, The silicon-based laser is used to generate input light; The silicon optical beam splitter is used to split the input light generated by the silicon-based laser into M row input lights and N column input lights; The row input modulator array and the column input modulator array are used to load different radio frequency signal data to the corresponding row input lights and column input lights, respectively, to obtain the modulated lights of each row and each column, respectively; The row silicon photonic DAC array is used to perform complex domain digital encoding on each row of modulated light to obtain each row of input optical signals; The column silicon photonic DAC array is used to perform complex domain digital encoding on each column of modulated light to obtain each column of input optical signal; The silicon-based balanced photon crossbar array is used to perform matrix operations based on each of the row input optical signals and each of the column input optical signals to obtain the matrix operation results; The drive control circuit chip comprises: A filter array, used for performing input equalization processing on each matrix element in the matrix operation result; A transimpedance amplifier array, used to perform photoelectric conversion and amplification processing on each of the matrix elements after input equalization processing to obtain the final result of the matrix operation; The drive control circuit chip also includes: A radio frequency generator array, used to generate a plurality of radio frequency signals of different frequencies; A mixer array, used to load data into radio frequency signals of different frequencies respectively to obtain data of different radio frequency signals; An electronic combiner array is used to respectively combine different RF signal data corresponding to the same object in the row input modulator array or the column input modulator array into one path as a corresponding RF input signal, and output each RF input signal to the corresponding object in the row input modulator array or the column input modulator array.

2. The system according to claim 1, characterized in that The silicon-based balanced photon cross array includes M×N array elements, each of which includes a waveguide cross, two 1×2 directional couplers, a 2×2 waveguide coupler, and a waveguide balanced detector; wherein, The input ends of the waveguide cross are respectively used to receive the corresponding row input optical signals and the column input optical signals, and the output ends of the waveguide cross are respectively connected to the input ends of each of the 1×2 directional couplers, so that the row input optical signals and the column input optical signals respectively enter the corresponding 1×2 directional couplers; In each of the array elements, one path of output light from each of the 1×2 directional couplers enters the waveguide balanced detector after interference through the 2×2 waveguide coupler, so as to perform balanced detection and time integration through the waveguide balanced detector, thereby realizing matrix operations between the corresponding row input optical signal and the column input optical signal; the other path of output light from each of the 1×2 directional couplers is used to transmit the corresponding row input optical signal or the column input optical signal to the next array element.

3. The system according to claim 2, characterized in that In the silicon-based balanced photon cross array, the coupling coefficient of two adjacent 1×2 directional couplers in the same row or column satisfies K n =K n+1 / (1 / η+ K n+1 ), where K n represents the coupling coefficient of the nth 1×2 directional coupler, K n+1 represents the coupling coefficient of the next 1×2 directional coupler adjacent to the nth 1×2 directional coupler in the same row or column, and η represents the insertion loss of the array element.

4. The system according to claim 1, characterized in that The drive control circuit chip also includes: The silicon photonics DAC driver array is used to generate a plurality of adjustable voltages to drive the row silicon photonics DAC array and the column silicon photonics DAC array through each of the adjustable voltages.

5. The system according to claim 4, characterized in that The silicon photonic DAC driver array comprises a microcontroller unit, a multi-channel electrical DAC array and an adjustment output stage; wherein, The microcontrol unit is used to dynamically configure the multi-channel electrical DAC array to output multiple adjustable voltages online; The adjustment output stage is used for power promotion, outputting each of the adjustable voltages to the corresponding row silicon photonic DAC array and the column silicon photonic DAC array to drive the row silicon photonic DAC array and the column silicon photonic DAC array.

6. The system according to claim 1, characterized in that The RF generator array includes a field programmable gate array and an operational amplifier stack; wherein, The field programmable gate array is used to set the frequency of each radio frequency signal respectively; The operational amplifier stack is used to generate corresponding radio frequency signals of each frequency according to the frequency of each radio frequency signal set by the field programmable gate array.

7. The system according to any one of claims 1 to 6, characterized in that: The silicon photonic chip and the driving control circuit chip are integrated using an optoelectronic hybrid integration method, and the optoelectronic hybrid integration method includes any one of a monolithic optoelectronic hybrid integration method, a flip-chip bonding method, and a through silicon via three-dimensional stacking method.

Images