A high-precision in-memory computing method and system based on a photoelectric hybrid array

Abstract

The application discloses a high-precision in-memory computing method and system based on an optoelectronic hybrid array, and is suitable for the optoelectronic hybrid array; wherein the method comprises the following steps: obtaining a plurality of moduli corresponding to to-be-computed data, and selecting in-memory computing sub-arrays from the optoelectronic hybrid array; obtaining a plurality of weight remainders corresponding to the computing weight according to the moduli, and writing the weight remainders into the corresponding in-memory computing sub-arrays to obtain the in-memory computing array after weight coding; obtaining the input data remainder corresponding to each in-memory computing sub-array according to the moduli and the to-be-computed data, so as to input the input data remainder and a preset optical signal into the in-memory computing array, and obtain the response optical signal output by the in-memory computing array; obtaining the signal power of the response optical signal, and obtaining the phase offset of the response optical signal according to the signal power, so as to obtain the in-memory computing result of the to-be-computed data according to the phase offset, and improve the efficiency of in-memory computing.

Description

Technical Field

[0001] This invention relates to the field of in-memory computing technology, and specifically to a high-precision in-memory computing method and system based on a hybrid optoelectronic array. Background Technology

[0002] Analog in-memory computing is a computational architecture that performs multiplication and summation operations in-situ using analog memory devices. It leverages the physical properties of these devices, such as conductivity and capacitance, to naturally map multiplication and summation operations, eliminating the need for frequent data exchanges and thus improving computational efficiency and reducing latency. However, the accuracy of analog in-memory computing is severely limited by non-ideal factors in the analog domain, including: nonlinearity of the memory cell's conductivity state and weight write errors due to limited tuning accuracy; and the degradation of the signal-to-noise ratio (SNR) due to thermal noise, process mismatch, and quantization noise from analog-to-digital conversion. As the target computational accuracy increases, the required SNR at the detection end grows exponentially, typically limiting the effective accuracy of analog in-memory computing to 4 to 5 bits. Simultaneously, the energy consumption of the front-end digital-to-analog converter (DAC) and the SNR requirements of the back-end DAC further exacerbate the trade-off between accuracy and efficiency. These limitations mean that while analog in-memory computing offers high energy efficiency, it struggles to meet the computational accuracy demands of large language models and high-precision scientific computing.

[0003] To address the aforementioned technical challenges, existing technologies employ photonic memories for simulated in-memory computation. While photonic memories offer advantages such as freedom from parasitic capacitance limitations, high bandwidth, and multiple encoding dimensions, current photonic memories largely rely on phase-change materials as storage media, utilizing the optical property differences between crystalline and amorphous states to encode data. The number of reliably distinguishable optical states in these materials is limited, hindering the improvement of storage density. Furthermore, the writing process requires overcoming a high energy barrier, resulting in high energy consumption and slow speed, further reducing the efficiency of in-memory computation. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention discloses a high-precision in-memory computing method and system based on a hybrid optoelectronic array, which improves the efficiency of in-memory computing.

[0005] To achieve the above objectives, this invention discloses a high-precision in-memory computation method based on a hybrid optoelectronic array, applicable to hybrid optoelectronic arrays; wherein, the high-precision in-memory computation method includes: Obtain several moduli corresponding to the data to be calculated, and select the in-memory computing subarray corresponding to each moduli from the optoelectronic hybrid array; Based on the modulus, obtain several weight remainders corresponding to the calculated weights, and write each weight remainder into the corresponding in-memory calculation subarray to obtain the weight-encoded in-memory calculation array. Based on the modulus and the data to be calculated, the input data remainder corresponding to each in-memory computing subarray is obtained, and the input data remainder and a preset optical signal are input into the in-memory computing array to obtain the response optical signal output by the in-memory computing array. The signal power of the response optical signal is obtained, and the phase offset of the response optical signal is obtained based on the signal power, so as to obtain the in-memory calculation result of the data to be calculated based on the phase offset.

[0006] This invention discloses a high-precision in-memory computing method based on a hybrid optoelectronic array. The method first acquires several moduli corresponding to the data to be computed and selects corresponding in-memory computing subarrays, decomposing the high-precision computing task into multiple parallel low-precision modulo operation channels. Then, the weighted remainders are written into the corresponding subarrays, and the input data remainders and optical signals are input into the arrays. The computation result is obtained using optical phase offset. This scheme performs modular multiplication in situ within the optical domain, avoiding frequent data transfer and analog-to-digital conversion, significantly improving computational energy efficiency and speed, and providing a feasible technical path for realizing high-precision analog in-memory computing.

[0007] As a preferred example, the optoelectronic hybrid array includes several initial computing subarrays; wherein each of the initial computing subarrays includes the same several photonic memories; the step of acquiring several moduli corresponding to the data to be computed, and acquiring the in-memory computing subarray corresponding to each moduli from the optoelectronic hybrid array, includes: For any given modulus, obtain the phase modulation characteristic data corresponding to that modulus; For any one of the initial computation subarrays, obtain the unit phase modulation step size corresponding to each of the photonic memories in the initial computation subarray; When the unit phase modulation step size is consistent with the phase modulation characteristic data, the initial computation subarray is used as the in-memory computation subarray corresponding to the modulus.

[0008] The above scheme defines the specific method for selecting the corresponding subarray from the optoelectronic hybrid array: by comparing the phase modulation characteristic data of the module with the unit phase modulation step size of the photonic memory in the initial subarray, the initial subarray is determined as the in-memory computation subarray for that module when the step size and characteristics match. This matching method ensures that the phase modulation capability of each subarray precisely corresponds to the module requirement, laying the hardware foundation for the subsequent natural realization of analog-to-digital computation in the optical domain and avoiding calculation errors caused by modulation step size mismatch.

[0009] As a preferred example, each of the photonic memories includes a plurality of non-volatile memory devices; wherein, for any one of the in-memory computation subarrays, the number of non-volatile memory devices in each of the photonic memories in the in-memory computation subarray is equal to the number of binary bits of the modulus corresponding to the in-memory computation subarray.

[0010] The above scheme limits the number of non-volatile memory devices in each photonic memory to the number of bits of the corresponding modulus in its subarray. This feature allows the hardware configuration of the photonic memory (the number of non-volatile devices) to precisely match the bit width requirements of the modulus, ensuring that all remainder values ​​in the range from 0 to modulus -1 can be fully represented, while avoiding waste of hardware resources and improving the utilization efficiency and design flexibility of the array.

[0011] As a preferred example, for any one of the photonic memories, the optical signal phase modulation capability of all the non-volatile memory devices in the photonic memory increases proportionally; the step of obtaining a plurality of weight remainders corresponding to the calculated weights based on the modulus, and writing each weight remainder into the corresponding in-memory computing subarray to obtain the weight-encoded in-memory computing array includes: For any photonic memory in any in-memory computing subarray, obtain the computing weight corresponding to that photonic memory from the preset computing weight matrix; Obtain the remainder when the calculated weight is divided by the modulus corresponding to the in-memory computation subarray, and use the remainder as the weight remainder of the photonic memory. The weight remainder is written into each of the non-volatile memory devices of the photonic memory to obtain the weighted photonic memory. Based on all the in-memory computing subarrays and all the weighted photonic memories in each of the in-memory computing subarrays, the in-memory computing array corresponding to the data to be computed is obtained.

[0012] The above scheme defines the specific process of writing the weight remainder into the in-memory computation subarray: obtain the computation weight at the corresponding location in the photonic memory, calculate the remainder when divided by the modulus as the weight remainder, and write this weight remainder into all non-volatile storage devices in the same photonic memory. This writing method ensures that each non-volatile storage device in the same photonic memory stores the same weight value, providing a unified and stable modulation reference for subsequent multiplication operations between input and weight using area ratio (binary bit weight), while avoiding repeated loading of weight data by utilizing non-volatile characteristics.

[0013] As a preferred example, each of the non-volatile memory devices includes a gate, a source, and a drain; the step of writing the weight remainder into each of the non-volatile memory devices of the photonic memory to obtain a weighted photonic memory includes: For any non-volatile memory device in the photonic memory, a write pulse signal is applied to the gate of the non-volatile memory device, and the source and drain of the non-volatile memory device are set to 0V, so as to write the weight remainder into the non-volatile memory device.

[0014] The above scheme defines the operation of writing the weight remainder to a non-volatile memory device: a write pulse signal is applied to the gate while the source and drain are set to 0V. This writing method is simple and reliable, utilizes the non-volatile storage of weight values ​​based on the ferroelectric domain polarization state, and can be retained for a long time without refresh after writing. Moreover, the writing power consumption is low and the speed is fast, which is beneficial to improving the overall energy efficiency of the in-memory computing system.

[0015] As a preferred example, each of the in-memory computing subarrays includes an optical waveguide serving as a bit line and a switching transistor electrically connected to each non-volatile memory device; the step of obtaining the input data remainder corresponding to each of the in-memory computing subarrays based on the modulus and the data to be computed, and inputting the input data remainder and a preset optical signal into the in-memory computing array to obtain the response optical signal output by the in-memory computing array includes: For any in-memory computing subarray, obtain the remainder when the data to be computed is divided by the modulus corresponding to the in-memory computing subarray, and use it as the remainder of the input data corresponding to the in-memory computing subarray. The multi-bit binary signal corresponding to the remainder of the input data is obtained according to the bit precision corresponding to the modulus. For any non-volatile memory device on the in-memory computing subarray, determine the target data binary signal corresponding to each non-volatile memory device from the multiple bits of the data binary signal; For any one of the non-volatile memory devices, the target data binary signal is applied to the source and drain terminals of the non-volatile memory device, and the source and drain terminals of the non-volatile memory device are controlled to be in a grounded state or a high-impedance state by the corresponding switching transistor; wherein, when the source and drain terminals are in a grounded state, the non-volatile memory device is in an optical signal shielding state; when the source and drain terminals are in a high-impedance state, the non-volatile memory device is in an optical signal modulation state; A preset optical signal is input into the input optical waveguide, which serves as the bit line, so that the optical signal passes sequentially through non-volatile memory devices in each photonic memory that are set to the modulation state of the optical signal, in order to obtain the response optical signal output by each photonic memory.

[0016] The above scheme defines the process of inputting the remainder of the input data into the in-memory computing array and obtaining the response optical signal: calculating the remainder of the input data, converting it into a binary signal, applying each bit signal to the source and drain terminals of the non-volatile memory device, controlling the source and drain terminals to be in a grounded or high-impedance state by switching transistors, thereby determining whether the device is in an optical signal shielding state or an optical signal modulation state, and finally using the preset optical signal input as the optical waveguide of the bit line, passing through the modulated device in sequence to obtain the response optical signal. This scheme utilizes the shielding effect to achieve direct control of optical phase modulation by the digital input signal, eliminating the need for a front-end digital-to-analog converter, significantly reducing system power consumption, and simultaneously completing the input and weight multiplication operation in the optical domain, reducing data transfer delay.

[0017] As a preferred example, each photonic memory includes a microring resonator and a plurality of non-volatile memory devices covering the microring resonator; the step of acquiring the signal power of the response optical signal and acquiring the phase offset of the response optical signal based on the signal power, so as to acquire the in-memory calculation result of the data to be calculated based on the phase offset, includes: For any one of the response optical signals, obtain the signal power of the response optical signal; The light transmittance of the microring resonator is obtained, and the phase offset corresponding to the response light signal is obtained according to the preset phase power mapping relationship between the light transmittance and the signal power. The modulus multiplication remainder corresponding to the response optical signal is obtained by querying a preset phase remainder mapping table based on the phase offset. The remainder of the modular multiplication is used as the result of the operation of the in-memory computing subarray where the response optical signal is located; Obtain the modulo sum of all the operation results output by all the in-memory computation subarrays, and use it as the in-memory computation result of the data to be computed.

[0018] The above scheme defines the process of obtaining the phase shift and the modulo multiplication remainder from the response optical signal: the signal power is detected, the phase shift is determined using the mapping relationship between the optical transmittance of the microring resonator and the phase power, and then the modulo multiplication remainder is obtained through a phase remainder mapping table. This process fully utilizes the 2π periodicity of the optical phase, directly mapping the phase shift in the optical domain to the modulo multiplication remainder, achieving a low-complexity conversion from optical signal to digital result; at the same time, the preset mapping relationship avoids complex real-time calculations, improving detection speed and accuracy.

[0019] As a preferred example, obtaining the modulo summation of all the operation results output by all the in-memory computing subarrays as the in-memory computing result of the data to be computed includes: For any in-memory computation subarray, obtain the computation result output by that in-memory computation subarray; Based on the modulus corresponding to each in-memory computing subarray, the operation results output by each in-memory computing subarray are accumulated modulowise to obtain the modulo accumulation result for each modulus. The modulus accumulation result under each modulus is reconstructed to obtain the in-memory calculation result of the data to be calculated.

[0020] The above scheme defines a method for performing modulo summation on the operation results output by all in-memory computation subarrays and reconstructing the final computation result: the operation results output by each subarray are modulo summed separately under the modulus corresponding to each subarray to obtain the modulo summation result under each modulus, and then the high-precision in-memory computation result of the data to be computed is reconstructed using the Chinese Remainder Theorem. This realizes the coordinated combination of results from multiple RNS channels, fully leverages the parallel and carry-free advantage of the Remainder Number System, and concatenates low-precision modulo operation results into high-precision multiplication and summation results, effectively reducing the system's requirements for signal-to-noise ratio, and enabling the analog in-memory computation precision to reach more than 15 bits.

[0021] On the other hand, the present invention discloses a high-precision in-memory computing system based on a hybrid optoelectronic array, which is applicable to hybrid optoelectronic arrays; wherein, the high-precision in-memory computing system includes an array selection module, a weight writing module, an optical processing module, and an in-memory computing module; The array selection module is used to obtain several moduli corresponding to the data to be calculated, and select the in-memory computing subarray corresponding to each moduli from the optoelectronic hybrid array; The weight writing module is used to obtain several weight remainders corresponding to the calculated weights according to the modulus, and write each weight remainder into the corresponding in-memory calculation subarray to obtain the in-memory calculation array after weight encoding. The optical processing module is used to obtain the input data remainder corresponding to each in-memory computing subarray according to the modulus and the data to be calculated, so as to input the input data remainder and the preset optical signal into the in-memory computing array to obtain the response optical signal output by the in-memory computing array. The in-memory calculation module is used to obtain the signal power of the response optical signal and obtain the phase offset of the response optical signal based on the signal power, so as to obtain the in-memory calculation result of the data to be calculated based on the phase offset.

[0022] This invention discloses a high-precision in-memory computing system based on a hybrid optoelectronic array. It first acquires several modulo operations corresponding to the data to be computed and selects corresponding in-memory computing subarrays, decomposing the high-precision computing task into multiple parallel low-precision modulo operation channels. Then, the weighted remainders are written into the corresponding subarrays, and the input data remainders and optical signals are input into the arrays. The computation result is obtained using optical phase offset. This scheme performs modular multiplication in situ within the optical domain, avoiding frequent data transfer and analog-to-digital conversion, significantly improving computational energy efficiency and speed, and providing a feasible technical path for realizing high-precision analog in-memory computing.

[0023] As a preferred example, the optoelectronic hybrid array includes several initial computing subarrays; wherein each of the initial computing subarrays includes the same several photonic memories; the array selection module includes a modulus unit and a matching unit; The module unit is used to obtain the phase modulation characteristic data corresponding to any module; and to obtain the unit phase modulation step size corresponding to each photonic memory in any initial computing subarray. The matching unit is used to use the initial computation subarray as the in-memory computation subarray corresponding to the modulus when the unit phase modulation step size is consistent with the phase modulation characteristic data.

[0024] The above scheme defines the specific method for selecting the corresponding subarray from the optoelectronic hybrid array: by comparing the phase modulation characteristic data of the module with the unit phase modulation step size of the photonic memory in the initial subarray, the initial subarray is determined as the in-memory computation subarray for that module when the step size and characteristics match. This matching method ensures that the phase modulation capability of each subarray precisely corresponds to the module requirement, laying the hardware foundation for the subsequent natural realization of analog-to-digital computation in the optical domain and avoiding calculation errors caused by modulation step size mismatch. Attached Figure Description

[0025] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0026] Figure 1 This is a flowchart illustrating a high-precision in-memory computing method based on a hybrid optoelectronic array provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the three-dimensional structure of a single photonic memory in an embodiment of the present invention; Figure 3This is a top view of the layout of multiple FeFET units arranged along the circumference of the lithium niobate microring resonator in an embodiment of the present invention. Figure 4 This is a schematic diagram of the cross-sectional structure of a single photonic memory (FeFET and microring resonator) in an embodiment of the present invention; Figure 5 This is a schematic diagram of the circuit connection of multiple FeFETs in the photonic memory in an embodiment of the present invention; Figure 6 This is a schematic diagram of applying a voltage pulse to the FeFET gate for weight writing in an embodiment of the present invention; Figure 7 This is a schematic diagram illustrating the FeFET channel shielding effect principle in an embodiment of the present invention; Figure 8 This is a schematic diagram of the overall high-precision in-memory computing architecture based on a hybrid optoelectronic array in an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of a high-precision in-memory computing system based on a hybrid optoelectronic array provided in an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] Reference Figure 1 To improve the efficiency of in-memory computation, this embodiment discloses a high-precision in-memory computation method based on a hybrid optoelectronic array, applicable to hybrid optoelectronic arrays; wherein, the high-precision in-memory computation method includes: Step 101: Obtain several modules corresponding to the data to be calculated, and select the in-memory computing subarray corresponding to each module from the optoelectronic hybrid array; Step 102: Obtain several weight remainders corresponding to the calculated weights according to the modulus, and write each weight remainder into the corresponding in-memory calculation subarray to obtain the weight-encoded in-memory calculation array; Step 103: Based on the modulus and the data to be calculated, obtain the input data remainder corresponding to each in-memory computing subarray, and input the input data remainder and the preset optical signal into the in-memory computing array to obtain the response optical signal output by the in-memory computing array; Step 104: Obtain the signal power of the response optical signal, and obtain the phase offset of the response optical signal based on the signal power, so as to obtain the in-memory calculation result of the data to be calculated based on the phase offset.

[0029] In this embodiment, the modulus refers to the integer used in the remainder system for modulo operations on the original data. A large integer can be represented as a combination of multiple smaller remainders by a set of pairwise coprime moduli, thereby achieving parallel encoding of information. The optoelectronic hybrid array includes several initial computation subarrays. Each initial computation subarray includes several identical photonic memories. A photonic memory is a basic computational unit capable of modulating optical signals through an electric field. Each photonic memory includes several non-volatile memory devices. A non-volatile memory device is the smallest functional unit in a photonic memory used to achieve a unit modulation amount. In the same computation subarray, the number of non-volatile memory devices in any two photonic memories is the same, and for any photonic memory, the optical signal phase modulation capability of all non-volatile memory devices in that photonic memory increases proportionally.

[0030] This embodiment utilizes the Pockels effect (a linear electro-optic effect generated by the residual polarization electric field on the lithium niobate waveguide to achieve phase modulation of the optical signal in the waveguide) and the shielding effect of the ferroelectric field-effect transistor on the optical signal modulation. Preferably, as follows: Figure 2 As shown, a ferroelectric field-effect transistor (FeFET) is chosen to construct the photonic memory, and lithium niobate (FeFET) is chosen to be used. A microring oscillator (MRR) is used as an optical waveguide to vertically stack the photonic memory on top of the optical waveguide, forming the optoelectronic hybrid array. It is worth noting that high-κ ferroelectric thin film materials such as barium titanate (BTO) can be used instead of traditional lithium niobate to fabricate the optical waveguide. These materials have a higher Pockels coefficient, enabling the same phase change at a lower modulation voltage, thereby further reducing the voltage swing and power consumption of the peripheral driving circuit. Meanwhile, BTO offers better compatibility with silicon-based optoelectronic processes, facilitating higher-density monolithic integration; it can utilize MZI (Mach-Zehnder interferometer) structures to form optical waveguides; a FeFET-modulated phase shifter can be integrated into one of the arms of the MZI; after the input light passes through the beam splitter, the phase of one arm is modulated by the weight value stored in the FeFET, generating a phase shift corresponding to the RNS modulus; the phase shift of the output optical signal after the two arms are combined naturally characterizes the modulus multiplication or addition result; compared to MRR, the MZI structure is less sensitive to process deviations (such as linewidth and temperature), requires no thermal tuning, and is more suitable for constructing large-scale, highly robust in-memory arrays.

[0031] Specifically, the fabrication process of the photonic memory is as follows: First, lithium niobate is etched to a depth of 320 nm using reactive ion etching to form a ridge waveguide. After cleaning, a 1 μm thick silicon dioxide cladding layer is deposited by plasma-enhanced chemical vapor deposition to reduce light propagation loss. Silicon dioxide is removed in the areas where FeFETs (non-volatile memory devices) and microring resonators are to be placed using photolithography and wet etching with a buffer oxide etchant, creating windows. A 5 nm IGZO and a 3 nm ITO film are then sequentially deposited using magnetron sputtering as the channel layer for the FeFET. Pd is deposited by electron beam evaporation to form the source and drain regions of the FeFET. An 8 nm thick HZO ferroelectric film is grown above the IGZO and ITO film channel using atomic layer deposition as the non-volatile memory layer. A tungsten layer is deposited above the ferroelectric layer as a ferroelectric induction layer, and then subjected to rapid thermal annealing at 380 degrees Celsius in a nitrogen atmosphere. This sacrificial layer is then removed using a tungsten etchant. Finally, a conductive ITO layer is deposited, and the top gate electrode is formed by photolithography and chloride-based reactive ion etching.

[0032] After fabricating a single photonic memory according to the fabrication process, from Figure 2 As shown in the three-dimensional structure, a single photonic memory is a vertically stacked structure located above the LNOI microring resonator of the optical waveguide, and from... Figure 2 As shown in the three-dimensional structure and the above fabrication process, the specific structure of each photonic memory, from top to bottom, consists of a top gate electrode, a ferroelectric layer, a semiconductor channel layer, a source and a drain, and a bottom LNOI waveguide.

[0033] It should be noted that, for a single photonic memory, the storage capacity of the photonic memory can be expanded by stacking multiple independent non-volatile memory devices, such as stacking multiple FeFET cells, and an electrical interface can be provided for subsequent in-situ multiplication operations.

[0034] This embodiment takes multiple FeFETs corresponding to one photonic memory as an example. First, from... Figure 3 It can be seen that multiple FeFET cells are arranged and covered along the circumference of the same lithium niobate microring resonator; and different FeFET cells in each photonic memory have different coverage areas, which are used to control their weight in the phase modulation of the optical signal according to the size of the coverage area; it should be noted that, from Figure 3 It can be seen that the areas of different non-volatile memory devices in a single photonic memory increase proportionally. For example, taking... Figure 3In one embodiment, when a single photonic memory corresponds to 4 FeFET cells, the area ratio of the FeFET cells is 1:2:4:8. In another embodiment, when a single photonic memory corresponds to 5 FeFET cells, the area ratio of the FeFET cells is 1:2:4:8:16, and so on. Similarly, when a single photonic memory corresponds to n FeFET cells, the area ratio of the FeFET cells is 1:2:4:8:16:...: .

[0035] Furthermore, for multiple non-volatile memory devices in a single photonic memory, with Figure 4 The cross-section of the photonic memory shown and Figure 5 As shown in the circuit diagram of the photonic memory, each FeFET, i.e., each of the non-volatile memory devices, has an independent top gate electrode G for receiving externally input binary signals to independently control the polarization state of the HZO in its region according to the binary signals; each FeFET has an independent drain D, and the drains D of multiple photonic memories are connected together to form a shared drain; at the same time, any two adjacent FeFETs share a source S.

[0036] For each photonic memory containing multiple non-volatile memory devices, the electrical connection structure of the multiple FeFETs is as follows: Figure 4 As shown. From Figure 4 It is known that when an optical signal of a specific wavelength passes through each non-volatile memory device of the photonic memory, a portion of the light is coupled into the LNOI of the MRR, and then returns to the original optical waveguide through the MRR. Based on the additional phase shift generated in the MRR, interference occurs in the optical signal in the original optical waveguide, causing a phase change. It is important to note that in this embodiment, the FeFET used in each non-volatile memory device can modulate the refractive index of the MRR waveguide, thereby generating different phase shifts in the MRR when the FeFET is in different states. The interference of different phase differences causes changes in the output light power.

[0037] In this first embodiment, the FeFET superimposed on the waveguide was subjected to tests on ferroelectric polarization PV curves, channel current, leakage current, read / write durability, and duration. The test results show that the photonic memory constructed from the FeFET exhibits excellent non-volatility, with a retention time exceeding ten years and read / write durability exceeding [missing information]. Second-rate.

[0038] In summary, a single photonic memory consists of multiple FeFETs covering an MRR waveguide. Each FeFET has an independent gate and drain, and the multiple FeFETs share a common source. When the ferroelectric domain polarization of the ferroelectric layer is changed, the resulting ferroelectric field also changes, altering the modulation state of the ferroelectric field on the lithium niobate waveguide. This changes the waveguide refractive index, causing a phase change in the microring resonator waveguide beneath the FeFET, thus altering its resonance condition and resulting in different optical powers in the output optical signal depending on the stored state within the FeFET.

[0039] This embodiment first acquires several moduli corresponding to the data to be calculated and selects the corresponding in-memory computing subarrays, decomposing the high-precision computing task into multiple parallel low-precision modulo operation channels. Then, the weighted remainders are written into the corresponding subarrays, and the input data remainders and optical signals are input into the arrays. The calculation results are obtained using optical phase offset. This scheme completes the modular multiplication operation in-situ within the optical domain, avoiding frequent data transfer and analog-to-digital conversion, significantly improving computational energy efficiency and speed, and providing a feasible technical path for realizing high-precision analog in-memory computing.

[0040] In this embodiment, the optoelectronic hybrid array includes several initial computing subarrays; wherein each initial computing subarray includes the same several photonic memories; each photonic memory includes several non-volatile memory devices; wherein, for any in-memory computing subarray, the number of non-volatile memory devices in each photonic memory of the in-memory computing subarray is equal to the number of binary bits of the modulus corresponding to the in-memory computing subarray; step 101 includes: Step 1011: For any of the aforementioned moduli, obtain the phase modulation characteristic data corresponding to that moduli; Step 1012: For any one of the initial computation subarrays, obtain the unit phase modulation step size corresponding to each of the photonic memories in the initial computation subarray; Step 1013: When the unit phase modulation step size is consistent with the phase modulation characteristic data, the initial calculation subarray is used as the in-memory calculation subarray corresponding to the modulus.

[0041] In this embodiment, as Figure 4As shown, the pre-fabricated optoelectronic hybrid array comprises multiple initial computational subarrays, each consisting of an equal number of photonic memories arranged in an M x N array configuration. Each photonic memory includes a microring resonator and multiple ferroelectric field-effect transistors covering the circumference of the microring resonator. These ferroelectric field-effect transistors, acting as non-volatile memory devices, are used to modulate the phase of the optical signal through the Pockels effect generated by their residual polarization electric field. The photonic memories in each initial computational subarray are configured to have a specific unit phase modulation step size, which is determined by the perimeter of the microring resonator in that subarray.

[0042] In this embodiment, taking the calculation with 15-bit precision as an example, the set of moduli is selected as {m1=15,m2=14,m3=13,m4=11}. For each moduli in this set, a corresponding in-memory computing subarray needs to be selected from the optoelectronic hybrid array.

[0043] Specifically, taking a modulus m1=15 as an example, the phase modulation characteristic data corresponding to this modulus is first obtained. The phase modulation characteristic data refers to the unit phase modulation step size of the photonic memory required to achieve modulus multiplication in the optical domain. According to the encoding principle of the residual number system, the unit phase modulation step size generated by the photonic memory for the optical signal should be precisely equal to 2π divided by the modulus. Therefore, for a modulus of 15, its phase modulation characteristic data is 2π / 15.

[0044] Next, for each initial computational subarray in the optoelectronic hybrid array, the unit phase modulation step size corresponding to each photonic memory in that initial computational subarray is obtained. This unit phase modulation step size is determined by the perimeter of the microring resonator in the photonic memory and is an inherent parameter determined during manufacturing. In this embodiment, the microring resonator in one initial computational subarray is designed to have a specific perimeter, such that the unit phase modulation step size of each photonic memory in that subarray is 2π / 15; the unit phase modulation step size of another initial computational subarray is 2π / 14; another initial computational subarray has a unit phase modulation step size of 2π / 13; and yet another initial computational subarray has a unit phase modulation step size of 2π / 11.

[0045] Then, the phase modulation characteristic data corresponding to the modulus is compared with the unit phase modulation step size of each initial computational subarray. When the unit phase modulation step size of an initial computational subarray matches the phase modulation characteristic data corresponding to the modulus, that is, when the unit phase modulation step size is equal to 2π divided by the modulus, the initial computational subarray is determined as the in-memory computational subarray corresponding to that modulus.

[0046] In this embodiment, for modulus 15, its phase modulation characteristic data is 2π / 15. Therefore, the initial computation subarray with a unit phase modulation step size of 2π / 15 is used as the in-memory computation subarray corresponding to modulus 15. Similarly, for modulus 14, the initial computation subarray with a unit phase modulation step size of 2π / 14 is used as its corresponding in-memory computation subarray; for modulus 13, the initial computation subarray with a unit phase modulation step size of 2π / 13 is used as its corresponding in-memory computation subarray; and for modulus 11, the initial computation subarray with a unit phase modulation step size of 2π / 11 is used as its corresponding in-memory computation subarray.

[0047] Through the above matching process, four in-memory computation subarrays were obtained, corresponding to moduli 15, 14, 13, and 11, respectively. Each in-memory computation subarray is responsible for multiplication and accumulation operations at its corresponding moduli. The subarrays operate independently and in parallel without carry interaction. This matching method ensures that the phase modulation capability of each subarray precisely corresponds to the moduli requirement, laying the hardware foundation for subsequent analog-to-digital computation using the natural characteristics of optical phase cycling in the optical domain.

[0048] In this first embodiment, for the in-memory computing subarray used, in order for the in-memory computing subarray to receive externally input binary signals, it is necessary to configure the electrical connections of the in-memory computing array composed of the in-memory computing subarrays. Specifically, from... Figure 5 As shown in the electrical connection diagram, in order for a single photonic memory to receive binary signals, the state of each non-volatile memory device needs to be mapped to a corresponding binary state. Specifically, this can be achieved by applying 0V or a high-impedance state to the source and drain terminals of the non-volatile memory device, thus mapping the modulation state of each non-volatile memory device to a binary number with corresponding bit precision. However, since the state of a single non-volatile memory device is not a fixed voltage state, it cannot be broadcast via conventional word lines. Therefore, this embodiment uses multiple NFETs (field-effect transistors) with the same number of non-volatile memory devices as in a single photonic memory. Specifically, using... Figure 3 Taking the four non-volatile memory devices shown as an example, four NFETs are obtained, and their sources are sequentially connected to the independent source and drain terminals outside the four FeFETs. The drain terminals of the NFETs are all set to 0V. At this time, the gate of each NFET is controlled. When the gate voltage of the transistor is lower than the threshold voltage, a high-resistivity state can be applied to the FeFET. Thus, the photonic memory and the NFET together constitute the basic unit of memory computing.

[0049] Preferably, after the field-effect transistors are electrically connected to the non-volatile memory devices, the N basic memory computing units are arranged horizontally, resulting in 4N NFET gate ports. In each basic memory computing unit, four NFETs are connected to the source and drain terminals of FeFETs with an area ratio of 1:2:4:8. The gates of the FeFETs with an area ratio of 8 in the N basic memory computing units are connected together through a metal layer wiring. When a 0V voltage is applied to this wiring, all NFETs connected to this line are turned off, achieving high-impedance state broadcasting. When a VDD voltage is applied to this line, all NFETs connected to this line are turned on, setting the source and drain terminals of the FeFETs they control to 0V. This principle is followed to connect to other non-volatile memory devices. At this point, the N memory computing units appear externally as four digital input ports, enabling the broadcasting of 4-bit binary input values ​​to the N photonic memories. This 4-bit digital input metal layer wiring is the word line of the in-memory computing subarray.

[0050] Finally, to obtain the modulated optical signal of each photonic memory, the N basic memory units are arranged horizontally and connected by digital word lines. Now, the N×M basic memory units are arranged to form an M-row, N-column array; each row is connected by digital bit lines, and each column is coupled to all photonic memories in that column via optical waveguides. A laser is added at the initial end as the optical signal input, at which point the output of all photonic memories in that column can be read from the optical waveguide. Therefore, the optical waveguides constitute the bit lines of the photonic memory's internal computing array.

[0051] The above implementation defines a specific method for selecting the corresponding subarray from the optoelectronic hybrid array: by comparing the phase modulation characteristic data of the modulus with the unit phase modulation step size of the photonic memory in the initial subarray, the initial subarray is determined as the in-memory computation subarray for that modulus when the step size and characteristics match. This matching method ensures that the phase modulation capability of each subarray precisely corresponds to the modulus requirement, laying the hardware foundation for the subsequent natural implementation of modulus arithmetic in the optical domain and avoiding calculation errors caused by modulation step size mismatch. Simultaneously, it limits the number of non-volatile memory devices in each photonic memory to the number of bits corresponding to the modulus in its subarray. This technical feature enables a precise match between the hardware configuration of the photonic memory (the number of non-volatile devices) and the bit width requirement of the modulus, ensuring the complete representation of all remainder values ​​in the range from 0 to modulus -1 while avoiding waste of hardware resources, thus improving array utilization efficiency and design flexibility.

[0052] In this embodiment, for any one of the photonic memories, the optical signal phase modulation capability of all the non-volatile memory devices in the photonic memory increases proportionally; each of the non-volatile memory devices includes a gate, a source, and a drain; step 102 includes: Step 1021: For any photonic memory in any in-memory computing subarray, obtain the computing weight corresponding to the photonic memory from the preset computing weight matrix; Step 1022: Obtain the remainder when the calculated weight is divided by the modulus corresponding to the in-memory computation subarray, and use the remainder as the weight remainder of the photonic memory; Step 1023: Write the weight remainder into each of the non-volatile memory devices of the photonic memory to obtain a weighted photonic memory; wherein, for any non-volatile memory device in the photonic memory, a write pulse signal is applied to the gate of the non-volatile memory device, and the source and drain of the non-volatile memory device are set to 0V to write the weight remainder into the non-volatile memory device; Step 1024: Based on all the in-memory computing subarrays and all the weighted photonic memories in each of the in-memory computing subarrays, obtain the in-memory computing array corresponding to the data to be computed.

[0053] In this embodiment, before weight encoding, weights need to be written to each non-volatile memory device in the photonic memory to change the polarization state of the non-volatile memory device according to the writing, thereby enabling each non-volatile memory device to perform phase modulation of the optical signal. The polarization state refers to the orientation state of the ferroelectric domains of the ferroelectric material in the non-volatile memory device. This state can be maintained for a long time after power is turned off (i.e., it is non-volatile), determining the intensity of the phase modulation of the optical signal by the non-volatile memory device in the startup state. Preferably, as follows... Figure 6 As shown, by applying a voltage pulse to the gate of the FeFET The polarization state of the ferroelectric domains in the ferroelectric layer is changed, and the weighting information is written into the FeFET in a non-volatile manner. Then, after the weighting information is written into the FeFET, the residual polarization electric field in the FeFET changes the refractive index of the lithium niobate waveguide through the Pockels effect, thereby generating phase modulation on the optical signal transmitted in the waveguide.

[0054] In this embodiment, before writing the weights, it is necessary to write the weights to each non-volatile memory device in the photonic memory. This writing changes the polarization state of the non-volatile memory device, thereby enabling each non-volatile memory device to perform phase modulation on the optical signal. The polarization state refers to the orientation state of the ferroelectric domains of the ferroelectric material in the non-volatile memory device. This state can be maintained for a long time after power is turned off (i.e., it is non-volatile), which determines the intensity of the phase modulation of the optical signal by the non-volatile memory device in the startup state.

[0055] In this embodiment, the floating-point quantization in the modular multiplication algorithm is converted to integers to perform high-precision vector-matrix multiplication; taking y = 1·W as an example, where x is the input vector and W is an N×M weight matrix, each element after quantization is an integer with the corresponding bit precision. For each weight... For example, j and k correspond to the j-th row and k-th column in the matrix, and their modulus is calculated through preprocessing. remainder = mod The remainder is the weight remainder.

[0056] Next, the weight remainder is written into each non-volatile memory device of the photonic memory. Each photonic memory includes multiple ferroelectric field-effect transistors (FETs) covering the circumference of the micro-ring resonator. These FETs, as non-volatile memory devices, have a number equal to the number of bits in the binary representation of the modulus corresponding to the subarray. For a modulus of 15, the number of bits is 4, therefore each photonic memory contains 4 FETs. During writing, the same weight remainder is written to all 4 FETs in the photonic memory, ensuring that the weight values ​​stored in each non-volatile memory device within the same photonic memory are consistent.

[0057] In this first embodiment, the process of writing the weight remainder is as follows: First, the ferroelectric state of the ferroelectric field-effect transistor is reset. Specifically, from... Figure 4As can be seen, in a photonic memory, the gate of the FeFET is located above the LN waveguide constituting the MRR, and the source and drain span across both sides of the waveguide. Therefore, they are distinguished as the source and drain inside the MRR ring and the source and drain outside the ring (because FeFETs do not distinguish between source and drain; they are equivalent). The source and drain inside the MRR ring are connected together by metal wiring to form a unified inner electrode. At this time, the source, drain, and gate of the four FeFETs outside the ring are independent of each other. First, the inner electrode is set to 0V, and the independent electrodes of the four FeFETs outside the ring are also set to 0V. Then, the gate state of each FeFET is reset to +3V with an electrical pulse of 10μs. The non-volatile ferroelectric domains in the ferroelectric layers of the four FeFETs on a photonic memory are uniformly inverted, thus completing the state reset of the four FeFETs. If, with the gate and source electrodes on both sides set to 0V, a pulse is applied to the gate of a single FeFET, the state of that FeFET can be reset independently, while the states of the other FeFETs remain unchanged. The ferroelectric state refers to the polarization orientation state of the ferroelectric domains in the ferroelectric material. Different ferroelectric states correspond to different residual polarization intensities, thereby generating electric fields of different intensities.

[0058] Next, after the state reset, the gate and source terminals of the FeFET to be encoded are first set to 0V. Then, a write pulse with an amplitude of -6V and a duration of 10μs is applied to the gate of this FeFET, while the gates of the other FeFETs remain at 0V. Depending on the number of write pulses applied, different degrees of ferroelectric domain inversion occur in the FeFET. This step is repeated to encode all FeFETs in a photonic memory differently until all non-volatile memory devices in the photonic memory are written with the same weight remainder. After writing, the residual polarization electric field of each ferroelectric field-effect transistor generates phase modulation of the optical signal in the microring resonator through the Paulcke effect, and the modulation intensity corresponds to the weight remainder. The above steps are repeated for each photonic memory in the current in-memory computing subarray until all photonic memories in the subarray have completed the weight remainder writing. At this point, all photonic memories in the subarray are weight-encoded, resulting in a weight-encoded photonic memory.

[0059] Finally, the above writing process is performed on each in-memory computing subarray corresponding to each modulus. In this embodiment, for each photonic memory in the four in-memory computing subarrays corresponding to moduli 15, 14, 13, and 11, the weight remainder is calculated according to its respective modulus and written. After the weight writing of all subarrays is completed, all in-memory computing subarrays and all weighted photonic memories in each subarray together constitute the in-memory computing array corresponding to the data to be calculated.

[0060] After encoding all FeFET states, the inner electrode of the MRR loop is set to 0V, and the outer electrode is left open in a high-impedance state. At this point, the total phase shift generated in the MRR photonic memory is the sum of the waveguide phase modulations of all FeFET ferroelectric fields. Specifically, by controlling the drain level of the FeFET (0V or high-impedance state), the electric field modulation of the waveguide by the FeFET is controlled to determine whether it is shielded, thereby controlling the analog-to-digital multiplication operation of the digital input signal. When both the source and drain of the FeFET are set to 0V, as... Figure 7 As shown, a 0V voltage applied to the source and drain of the FeFET causes a screening layer to form in the channel. This screening layer then cancels the electric field generated by the ferroelectric domain polarization states (i.e., weighting information) stored in the FeFET. Thus, when both the source and drain are at 0V, the screening layer forms, and the electric field generated by the ferroelectric state W is shielded, preventing phase modulation of the waveguide.

[0061] At this point, the outer source / drain electrodes of one FeFET are set to 0V, while the outer source / drain electrodes of the other FeFETs remain in an open-circuit high-resistivity state. Shielding charge accumulates within the FeFET channel, forming a shielding layer. This prevents the ferroelectric field intensity encoded in the FeFET's ferroelectric layer from being applied to the waveguide, thus shielding the FeFET from phase modulation of the waveguide. The shielding layer refers to the charge layer accumulated in the FeFET channel due to the attraction of the ferroelectric field to the charge. This shielding layer can counteract the effect of the ferroelectric field on the underlying waveguide, thereby preventing phase modulation. Specifically, as... Figure 7 As shown, an optical signal is input through an optical waveguide. When the optical signal passes through the MRR, it is modulated by the FeFET electric field. The output optical signal carries the stored weight information, thus realizing optical reading.

[0062] In this embodiment, to read the stored information of a specific FeFET in the photonic memory, the remaining FeFETs can be shielded according to the shielding steps described above. Only the ferroelectric field of that FeFET can modulate the waveguide phase. A fixed-wavelength optical signal is applied to the straight waveguide coupled to the MRR. Since different FeFETs have different phase modulations, these different phase modulations result in different transmittances of the MRR for that wavelength of light, thus producing different optical powers. A photodiode is used at the end of the straight waveguide to detect the transmitted optical signal power, thus completing the reading of the stored information in the FeFET. If all FeFETs are considered as a whole, without shielding any FeFETs, a fixed-wavelength optical signal is directly applied to the straight waveguide for reading. The read optical power is related to the sum of the phase modulations of the FeFETs.

[0063] Through the above method, the optical processing state settings of all non-volatile memory devices within each photonic memory in the optoelectronic hybrid array are completed, resulting in an in-memory computing array. The weight information of each photonic memory in this in-memory computing array is stored non-volatilely in the ferroelectric domain polarization states of the non-volatile memory devices, providing a physical basis for subsequent in-situ modular multiplication operations. Specifically, the weight remainders... Write as mod This corresponds to the residual polarization state of each FeFET in the subarray. The write process is completed by applying a short pulse to the gate with the FeFET source and drain grounded, resulting in non-volatile storage. The residual polarization state refers to the polarization intensity that the ferroelectric material can retain after the external electric field is removed. This polarization intensity determines the strength of the phase modulation generated by the FeFET on the waveguide, i.e., the corresponding weight value.

[0064] The above implementation defines the specific process of writing the weight remainder into the in-memory computing subarray: the computational weight at the corresponding location in the photonic memory is obtained, the remainder when divided by the modulus is calculated as the weight remainder, and this weight remainder is written into all non-volatile memory devices within the same photonic memory. This writing method ensures that each non-volatile memory device in the same photonic memory stores the same weight value, providing a unified and stable modulation reference for subsequent multiplication operations between input and weight using area ratio (binary bit weight). Simultaneously, the non-volatile nature of the data avoids repeated loading of weight data. Furthermore, the operation of writing the weight remainder into the non-volatile memory device is defined: a write pulse signal is applied to the gate, while the source and drain are set to 0V. This writing method is simple and reliable, utilizing the non-volatile storage of weight values ​​in the ferroelectric domain polarization state. The weight values ​​can be retained long-term without refresh after writing, and the writing power consumption is low and the speed is fast, which is beneficial for improving the overall energy efficiency of the in-memory computing system.

[0065] In this embodiment, each of the in-memory computing subarrays includes an optical waveguide serving as a bit line and a switching transistor electrically connected to each non-volatile memory device; step 103 includes: Step 1031: For any in-memory computing subarray, obtain the remainder of the data to be computed divided by the modulus corresponding to the in-memory computing subarray, and use it as the remainder of the input data corresponding to the in-memory computing subarray. Step 1032: Obtain the multi-bit binary signal corresponding to the remainder of the input data according to the bit precision corresponding to the modulus; Step 1033: For any non-volatile memory device on the in-memory computing subarray, determine the target data binary signal corresponding to each non-volatile memory device from the multiple bits of the data binary signal; Step 1034: For any one of the non-volatile memory devices, the target data binary signal is applied to the source and drain terminals of the non-volatile memory device, and the source and drain terminals of the non-volatile memory device are controlled to be in a grounded state or a high-impedance state by the corresponding switching transistor; wherein, when the source and drain terminals are in a grounded state, the non-volatile memory device is in an optical signal shielding state; when the source and drain terminals are in a high-impedance state, the non-volatile memory device is in an optical signal modulation state; Step 1035: Input the preset optical signal into the input optical waveguide that serves as the bit line, so that the optical signal passes sequentially through the non-volatile memory devices in each photonic memory that are set to the modulation state of the optical signal, so as to obtain the response optical signal output by each photonic memory.

[0066] In this embodiment, an in-memory computing subarray corresponding to a modulus of 15 is used as an example. Each photonic memory in this subarray has its corresponding weight remainder written according to the aforementioned embodiment. For the data to be computed, x_j (a 15-bit integer), the input data remainder of this data under the current modulus is first obtained. Specifically, for the current in-memory computing subarray, the remainder of the data to be computed divided by the modulus corresponding to the subarray is calculated and used as the input data remainder for that subarray. In this embodiment, the modulus is 15, and the data to be computed, x_j = 23, so the remainder of 23 divided by 15 is 8, therefore the input data remainder is 8.

[0067] Then, based on the bit precision corresponding to the modulus, the multi-bit binary signal corresponding to the remainder of the input data is obtained. The binary number of a modulus of 15 is 4 bits, so the remainder of the input data 8 is converted into a 4-bit binary representation, i.e., 1000. The bits from least significant to most significant are: bit 0 (least significant bit) is 0, bit 1 is 0, bit 2 is 0, and bit 3 (most significant bit) is 1.

[0068] Next, for any non-volatile memory device (i.e., ferroelectric field-effect transistor) on the in-memory computing subarray, the target data binary signal corresponding to each non-volatile memory device is determined from the multi-bit data binary signal. In this embodiment, each photonic memory contains four ferroelectric field-effect transistors, whose coverage areas are in a binary ratio (1×, 2×, 4×, 8×), corresponding to the 0th, 1st, 2nd, and 3rd bits of the data binary signal, respectively. According to the bit weight matching principle, the least significant bit (0th bit) of the binary signal is mapped to a 1× area ferroelectric field-effect transistor, the 1st bit is mapped to a 2× area ferroelectric field-effect transistor, the 2nd bit is mapped to a 4× area ferroelectric field-effect transistor, and the most significant bit (3rd bit) is mapped to an 8× area ferroelectric field-effect transistor. For an input data remainder of 8, its binary representation is 1000. Therefore, the target data binary signal corresponding to the ferroelectric field-effect transistor with an area of ​​8× is 1, and the target data binary signal corresponding to the ferroelectric field-effect transistors with areas of 4×, 2×, and 1× is 0.

[0069] Then, for each non-volatile memory device, its corresponding target data binary signal is applied to the source and drain terminals of that non-volatile memory device. Specifically, the source and drain terminals of the non-volatile memory device are controlled to be either grounded or in a high-impedance state by the corresponding switching transistor (N-type field-effect transistor). When the target data binary signal is 0, the switching transistor is turned on, pulling down the source and drain terminals of the non-volatile memory device to 0V (grounded state), at which time the non-volatile memory device is in an optical signal shielding state; when the target data binary signal is 1, the switching transistor is turned off, the source and drain terminals of the non-volatile memory device are in a high-impedance state, at which time the non-volatile memory device is in an optical signal modulation state.

[0070] In this embodiment, the target data binary signal corresponding to the 8× area ferroelectric field-effect transistor is 1. Therefore, its source and drain terminals are set to a high-impedance state, which is in the optical signal modulation state. The weighted remainder (i.e., the residual polarization electric field) written in this device can generate phase modulation of the optical signal. The target data binary signals corresponding to the 4×, 2×, and 1× area ferroelectric field-effect transistors are all 0. Therefore, their source and drain terminals are set to 0V ground, which is in the optical signal shielding state. Even if the weighted remainder has been written, its ferroelectric field is canceled by the shielding layer formed in the channel, and it cannot generate phase modulation of the optical signal.

[0071] After completing the above settings, use the preset optical signal input as the input optical waveguide of the bit line. For example... Figure 4As shown, a laser source generates continuous wavelength light, which is then split evenly by a beam splitter and input into each subarray. For the current subarray, the optical signal passes sequentially through each photonic memory in the subarray via the input optical waveguide. At each photonic memory, the optical signal is coupled into a microring resonator and sequentially passes through a non-volatile memory device (i.e., an 8× ferroelectric field-effect transistor) set to optical signal modulation mode. This device modulates the optical signal according to its stored weight remainder. Multiple modulated optical signals are superimposed in the microring resonator, producing a total phase shift. Since the optical phase has a 2π periodicity, this phase shift, modulo 2π, changes the phase of the output light from the microring resonator, thus obtaining a response optical signal carrying the modulus multiplication result.

[0072] For all photonic memories in the subarray, the above process is repeated to obtain the response optical signal output by each photonic memory. These response optical signals are transmitted to the back-end detection circuit via the output optical waveguide for subsequent phase offset extraction and modulus multiplication remainder calculation.

[0073] In this embodiment one implementation, as Figure 8 As shown, the in-memory computing array is connected to a beam splitter; the beam splitter is connected to a laser source, which generates continuous wavelength light, which is then split evenly by the beam splitter and input to each subarray. In each subarray, the light is split into M waveguides through a y-branch or MMI (multimode interference coupler), with each waveguide corresponding to an output vector element. Because the photonic memory has a large half-maximum width (FWHM) to achieve detectable mode multiplication output, it will modulate all wavelengths. Therefore, a wavelength isolator is used when connecting the photonic memory to the waveguides, ensuring that only a single wavelength of light enters the photonic memory, and each photonic memory corresponds to a different wavelength. For the k-th output waveguide of subarray i, the light flows sequentially through N photonic memories. At each photonic memory: only a single wavelength light signal is captured by the optical wavelength isolator, and the input electrical signal determines whether the FeFET applies phase modulation through the shielding effect; if applied, a phase shift of (position weight * 2π / mi) is generated. Due to the cyclic nature of optical phase, the optical phase output by the photonic memory is actually equal to the product of the input data remainder and the weight remainder, naturally realizing modular multiplication. Since the optical fields output by all photonic memories on this waveguide have different wavelengths, sequential multiplication results can be read out through a time-delay waveguide, allowing the detection end to obtain multiplication results at different time intervals.

[0074] In terms of modulo operations, since the optical phase exhibits a 2π periodicity, the transmittance of the MRR for a specific wavelength of optical signal also exhibits a periodicity with the phase shift. By applying a fixed wavelength optical signal in a straight waveguide coupled to the MRR, the result of the modulo multiplication operation can be obtained by detecting the transmitted optical power using a photodiode. Since this output result is symmetrical about π, a perturbation is applied to the MRR through EO modulation to further verify its range and accurately determine the modulo remainder value corresponding to the phase shift.

[0075] The above implementation defines the process of inputting the remainder of the input data into the in-memory computing array and obtaining the response optical signal: calculating the remainder of the input data, converting it into a binary signal, applying each bit signal to the source and drain terminals of the non-volatile memory device, controlling the source and drain terminals to be in a grounded or high-impedance state by a switching transistor, thereby determining whether the device is in an optical signal shielding state or an optical signal modulation state, and finally using a preset optical signal input as an optical waveguide as a bit line, which sequentially passes through the modulated device to obtain the response optical signal. This scheme utilizes the shielding effect to achieve direct control of optical phase modulation by the digital input signal, eliminating the need for a front-end digital-to-analog converter, significantly reducing system power consumption, and simultaneously completing the input and weight multiplication operation in the optical domain, reducing data transfer delay.

[0076] In this embodiment, each photonic memory includes a microring resonator and a plurality of non-volatile memory devices covering the microring resonator; step 104 includes: Step 1041: For any one of the response optical signals, obtain the signal power of the response optical signal; Step 1042: Obtain the light transmittance of the micro-ring resonator, and obtain the phase offset corresponding to the response light signal according to the preset phase power mapping relationship between the light transmittance and the signal power; Step 1043: Query the preset phase remainder mapping table according to the phase offset to obtain the modulus multiplication remainder corresponding to the response optical signal; Step 1044: Use the modulo multiplication remainder as the operation result of the in-memory computing subarray where the response optical signal is located; Step 1045: Obtain the modulo sum of all the operation results output by all the in-memory computing subarrays, and use it as the in-memory computation result of the data to be computed; wherein, for any in-memory computing subarray, obtain the operation result output by that in-memory computing subarray; according to the modulus corresponding to each in-memory computing subarray, perform modulo summation on the operation results output by each in-memory computing subarray to obtain the modulo summation result under each modulus; reconstruct the modulo summation result under each modulus to obtain the in-memory computation result of the data to be computed.

[0077] In this embodiment, the optical signal undergoes modular multiplication via a photonic memory. Different wavelengths are delayed using delay lines, and the optical signal is read out using photodiodes at different time windows. An ADC converts the signal into corresponding digital values, which are then decoded using a lookup table. Modular accumulation is then performed in the back-end digital domain. Due to the closed nature of RNS coding domain computation, multiple arrays can perform parallel computations at low precision without worrying about carry control. The high-precision computation result is obtained by concatenating the results of each modular operation. This scheme effectively reduces the signal-to-noise ratio requirement for high-precision computation, enabling analog in-memory computation precision up to 15 bits. Compared to in-memory computation schemes based on signal strength for the same precision, its energy efficiency is significantly improved.

[0078] like Figure 8 As shown, this embodiment uses a delay line to stagger the timing of optical signals of different wavelengths in the optical domain, ensuring that the multiplication results of each channel arrive at the back-end detection circuit (photodiode) in a specific time sequence. This avoids conflicts caused by simultaneous signal arrival and facilitates the sequential reading of each multiplication-accumulation result in the time domain. Next, the photodiode converts the optical signal modulated by the photonic memory into a photocurrent, which is then converted into a voltage by a TIA (transimpedance amplifier), and subsequently quantized by an ADC. In this embodiment, a 4-bit ADC is used. The digital value quantized by the ADC represents the power information of the response optical signal. Preferably, as... Figure 8 As shown, the Output Decoder decodes and reconstructs the digital signal after photoelectric conversion and quantization to obtain the final high-precision calculation result. Specifically, the optical signal output from the delay line is converted into current by a photodiode, then into voltage by a transimpedance amplifier (TIA), and then quantized by a low-precision ADC (e.g., 4-bit). Due to the cyclic characteristic (modulo 2π) of optical phase modulation, the quantized value output by the ADC needs to be mapped through a lookup table (LUT) to restore the corresponding modulo multiplication remainder. Finally, the remainder results of each RNS channel (corresponding to different modulo numbers) are accumulated modulowise in the digital domain, and finally, the remainders of all channels are concatenated using the Chinese Remainder Theorem (CRT) to reconstruct a high-precision (e.g., 15-bit) multiplication-accumulation result.

[0079] The transmittance refers to the transmission efficiency of optical signals at a specific wavelength. This transmittance is an inherent parameter that can be obtained through pre-calibration. The preset phase power mapping relationship refers to the correspondence between the phase offset and the power of the output optical signal. This mapping relationship is determined by the interference characteristics of the MRR. Based on the obtained signal power and transmittance of the response optical signal, the phase offset corresponding to the response optical signal can be obtained by querying this phase power mapping relationship.

[0080] Since the phase shift generated within the MRR is affected by the 2π periodicity of the optical phase, the actual detectable phase shift is the result of taking the total phase shift modulo 2π. Furthermore, the transmittance of the MRR for a specific wavelength of light also exhibits periodic variation with the periodicity of the phase shift; therefore, the phase shift can be inferred by detecting the optical power.

[0081] In this embodiment, a lookup table is used in the digital domain to decode the ADC quantization result. The phase remainder mapping table pre-stores the correspondence between different phase offsets and modular multiplication remainders. Due to the cyclic nature of optical phase, the optical phase output by the photonic memory naturally implements modular multiplication. By querying the phase remainder mapping table, the phase offset is converted into the corresponding modular multiplication remainder value. Since the output result is symmetrical about π, a perturbation is applied in the MRR through EO modulation to further verify its interval, so as to accurately determine the modular multiplication remainder value corresponding to the phase offset and avoid ambiguity caused by symmetry.

[0082] The decoded results are accumulated using a modulo accumulator to obtain the subarray output remainder. Specifically, when processing only a single piece of data to be calculated, this data generates a response optical signal in each in-memory computing subarray. Each response optical signal is decoded to obtain a corresponding modulo multiplication remainder. Each modulo multiplication remainder is used as the operation result of its respective in-memory computing subarray. The operation results of each in-memory computing subarray are added together to obtain the sum. It should be noted that the addition here refers to modulo addition under the modulus of each channel; the operation results of each channel are accumulated independently under their respective modulus.

[0083] For each output, the results of all subarrays are collected, and a high-precision value is reconstructed in the digital domain using the Chinese Remainder Theorem (CRT). In this embodiment, the calculation results of the outputs of four subarrays (corresponding to modulo 15, 14, 13, and 11 respectively) are collected, and a 15-bit precise value is reconstructed using the Chinese Remainder Theorem, which serves as the in-memory calculation result of the data to be calculated.

[0084] When processing multiple data to be computed, for any given in-memory computing subarray, each data to be computed generates its corresponding photonic memory in that channel, which generates its own response optical signal. After processing, the modulus multiplication remainder corresponding to each photonic memory is obtained. The modulus multiplication remainders corresponding to all photonic memories within the in-memory computing subarray are then output to obtain the result vector output by the in-memory computing subarray.

[0085] After obtaining the computation results from all in-memory computation subarrays, the results of all subarrays are collected, and high-precision accurate values ​​are reconstructed in the digital domain using the Chinese Remainder Theorem (CRT), which serves as the in-memory computation result for the multiple data to be computed. This method achieves parallel processing of the multiplication and accumulation operations of multiple input data and their corresponding weights across multiple analog-to-digital channels, and the merging of the final results. It is suitable for batch data processing scenarios in neural networks, improving the throughput efficiency of large-scale computations.

[0086] The above implementation defines the process of obtaining the phase offset and modulo multiplication remainder from the response optical signal: detecting the signal power, determining the phase offset using the mapping relationship between the optical transmittance of the microring resonator and the phase power, and then obtaining the modulo multiplication remainder through the phase remainder mapping table. This process fully utilizes the 2π periodicity of the optical phase, directly mapping the phase offset in the optical domain to the modulo multiplication remainder, achieving a low-complexity conversion from optical signal to digital result; at the same time, it avoids complex real-time calculations through a preset mapping relationship, improving detection speed and accuracy; it defines the method of modulo accumulation of the operation results output by all in-memory computing subarrays and reconstructing the final calculation result: the operation results output by each subarray are modulo-accumulated separately under the modulus corresponding to each subarray, obtaining the modulo accumulation result under each modulus, and then reconstructing the high-precision in-memory calculation result of the data to be calculated using the Chinese Remainder Theorem, realizing the collaborative combination of results from multiple RNS channels, fully leveraging the parallel and carry-free advantage of the remainder system, splicing low-precision modulo operation results into high-precision multiplication and accumulation results, effectively reducing the system's signal-to-noise ratio requirements, and enabling analog in-memory calculation accuracy to reach more than 15 bits.

[0087] On the other hand, refer to Figure 9 This embodiment also discloses a high-precision in-memory computing system based on a hybrid optoelectronic array, which is applicable to hybrid optoelectronic arrays; the system includes an array selection module 1, a weight writing module 2, an optical processing module 3, and an in-memory computing module 4.

[0088] The array selection module 1 is used to obtain several moduli corresponding to the data to be calculated, and select the in-memory computing subarray corresponding to each moduli from the optoelectronic hybrid array; The weight writing module 2 is used to obtain several weight remainders corresponding to the calculated weights according to the modulus, and write each weight remainder into the corresponding in-memory computing subarray to obtain the in-memory computing array. The optical processing module 3 is used to obtain the input data remainder corresponding to each in-memory computing subarray according to the modulus and the data to be calculated, so as to input the input data remainder and the preset optical signal into the in-memory computing array to obtain the response optical signal output by the in-memory computing array. The in-memory calculation module 4 is used to obtain the signal power of the response optical signal and obtain the phase offset of the response optical signal based on the signal power, so as to obtain the in-memory calculation result of the data to be calculated based on the phase offset.

[0089] In this first embodiment, the optoelectronic hybrid array includes several initial computing subarrays; each initial computing subarray includes the same several photonic memories; the array selection module 1 includes a modulus unit and a matching unit. The module unit is used to obtain the phase modulation characteristic data corresponding to any module; and to obtain the unit phase modulation step size corresponding to each photonic memory in any initial computing subarray. The matching unit is used to use the initial computation subarray as the in-memory computation subarray corresponding to the modulus when the unit phase modulation step size is consistent with the phase modulation characteristic data.

[0090] This embodiment discloses a high-precision in-memory computing method and system based on a hybrid optoelectronic array. By combining in-memory computing technology with the RNS algorithm, a high-precision hybrid optoelectronic analog in-memory computing system is realized, achieving high-precision analog signal output with low power consumption and signal-to-noise ratio requirements. In this embodiment, the architectural accuracy of the hybrid optoelectronic array depends only on the choice of the analog-to-digital set, not the array size, because the computation results are not truncated. Furthermore, this embodiment co-locates the input and weights, thereby reducing the insertion loss caused by modulation. This enables support for larger-scale arrays within the same energy budget, thus reducing power consumption at both the input and output ends. In addition, the lithium niobate-based RNS photonic in-memory computing architecture features decoupling of write, control, and read operations, while RNS encoding allows the array to support signed number inputs, improving the normalized energy efficiency of the computation.

[0091] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A high-precision in-memory computing method based on a hybrid optoelectronic array, characterized in that, Applicable to optoelectronic hybrid arrays; wherein, the high-precision in-memory calculation method includes: Obtain several moduli corresponding to the data to be calculated, and select the in-memory computing subarray corresponding to each moduli from the optoelectronic hybrid array; Based on the modulus, obtain several weight remainders corresponding to the calculated weights, and write each weight remainder into the corresponding in-memory calculation subarray to obtain the weight-encoded in-memory calculation array. Based on the modulus and the data to be calculated, the input data remainder corresponding to each in-memory computing subarray is obtained, and the input data remainder and a preset optical signal are input into the in-memory computing array to obtain the response optical signal output by the in-memory computing array. The signal power of the response optical signal is obtained, and the phase offset of the response optical signal is obtained based on the signal power, so as to obtain the in-memory calculation result of the data to be calculated based on the phase offset.

2. The high-precision in-memory computing method based on a hybrid optoelectronic array according to claim 1, characterized in that, The optoelectronic hybrid array includes several initial computing subarrays; wherein each initial computing subarray includes the same several photonic memories; the step of acquiring several moduli corresponding to the data to be computed, and selecting the in-memory computing subarray corresponding to each moduli from the optoelectronic hybrid array, includes: For any given modulus, obtain the phase modulation characteristic data corresponding to that modulus; For any one of the initial computation subarrays, obtain the unit phase modulation step size corresponding to each of the photonic memories in the initial computation subarray; When the unit phase modulation step size is consistent with the phase modulation characteristic data, the initial computation subarray is used as the in-memory computation subarray corresponding to the modulus.

3. The high-precision in-memory computing method based on a hybrid optoelectronic array according to claim 2, characterized in that, Each of the photonic memories includes a plurality of non-volatile memory devices; wherein, for any one of the in-memory computation subarrays, the number of non-volatile memory devices in each of the photonic memories in the in-memory computation subarray is equal to the number of binary bits of the modulus corresponding to the in-memory computation subarray.

4. A high-precision in-memory computing method based on a hybrid optoelectronic array according to any one of claims 2-3, characterized in that, For any one of the photonic memories, the optical signal phase modulation capability of all the non-volatile memory devices in the photonic memory increases proportionally; the step of obtaining several weight remainders corresponding to the calculated weights based on the modulus, and writing each weight remainder into the corresponding in-memory computing subarray to obtain the weight-encoded in-memory computing array includes: For any photonic memory in any in-memory computing subarray, obtain the computing weight corresponding to that photonic memory from the preset computing weight matrix; Obtain the remainder when the calculated weight is divided by the modulus corresponding to the in-memory computation subarray, and use the remainder as the weight remainder of the photonic memory. The weight remainder is written into each of the non-volatile memory devices of the photonic memory to obtain the weighted photonic memory. Based on all the in-memory computing subarrays and all the weighted photonic memories in each of the in-memory computing subarrays, the weighted in-memory computing array corresponding to the data to be computed is obtained.

5. The high-precision in-memory computing method based on a hybrid optoelectronic array according to claim 4, characterized in that, Each of the aforementioned non-volatile memory devices includes a gate, a source, and a drain; the step of writing the weight remainder into each of the aforementioned non-volatile memory devices of the photonic memory to obtain a weighted photonic memory includes: For any non-volatile memory device in the photonic memory, a write pulse signal is applied to the gate of the non-volatile memory device, and the source and drain of the non-volatile memory device are set to 0V, so as to write the weight remainder into the non-volatile memory device.

6. The high-precision in-memory computing method based on a hybrid optoelectronic array according to claim 1, characterized in that, Each of the in-memory computing subarrays includes an optical waveguide serving as a bit line and a switching transistor electrically connected to each non-volatile memory device; the step of obtaining the input data remainder corresponding to each of the in-memory computing subarrays based on the modulus and the data to be computed, and inputting the input data remainder and a preset optical signal into the in-memory computing array to obtain the response optical signal output by the in-memory computing array includes: For any in-memory computing subarray, obtain the remainder when the data to be computed is divided by the modulus corresponding to the in-memory computing subarray, and use it as the remainder of the input data corresponding to the in-memory computing subarray. The multi-bit binary signal corresponding to the remainder of the input data is obtained according to the bit precision corresponding to the modulus. For any non-volatile memory device on the in-memory computing subarray, determine the target data binary signal corresponding to each non-volatile memory device from the multiple bits of the data binary signal; For any one of the non-volatile memory devices, the target data binary signal is applied to the source and drain terminals of the non-volatile memory device, and the source and drain terminals of the non-volatile memory device are controlled to be in a grounded state or a high-impedance state by the corresponding switching transistor; wherein, when the source and drain terminals are in a grounded state, the non-volatile memory device is in an optical signal shielding state; when the source and drain terminals are in a high-impedance state, the non-volatile memory device is in an optical signal modulation state; A preset optical signal is input into the input optical waveguide, which serves as the bit line, so that the optical signal passes sequentially through non-volatile memory devices in each photonic memory that are set to the modulation state of the optical signal, in order to obtain the response optical signal output by each photonic memory.

7. The high-precision in-memory computing method based on a hybrid optoelectronic array according to claim 6, characterized in that, Each photonic memory includes a microring resonator and a plurality of non-volatile memory devices covering the microring resonator; the step of acquiring the signal power of the response optical signal and acquiring the phase offset of the response optical signal based on the signal power, and acquiring the in-memory calculation result of the data to be calculated based on the phase offset, includes: For any one of the response optical signals, obtain the signal power of the response optical signal; The light transmittance of the microring resonator is obtained, and the phase offset corresponding to the response light signal is obtained according to the preset phase power mapping relationship between the light transmittance and the signal power. The modulus multiplication remainder corresponding to the response optical signal is obtained by querying a preset phase remainder mapping table based on the phase offset. The remainder of the modular multiplication is used as the result of the operation of the in-memory computing subarray where the response optical signal is located; Obtain the modulo sum of all the operation results output by all the in-memory computation subarrays, and use it as the in-memory computation result of the data to be computed.

8. The high-precision in-memory computing method based on a hybrid optoelectronic array according to claim 7, characterized in that, The step of obtaining the modulo summation of all the operation results output by all the in-memory computing subarrays as the in-memory computing result of the data to be computed includes: For any in-memory computation subarray, obtain the computation result output by that in-memory computation subarray; Based on the modulus corresponding to each in-memory computing subarray, the operation results output by each in-memory computing subarray are accumulated modulowise to obtain the modulo accumulation result for each modulus. The modulus accumulation result under each modulus is reconstructed to obtain the in-memory calculation result of the data to be calculated.

9. A high-precision in-memory computing system based on a hybrid optoelectronic array, characterized in that, Suitable for optoelectronic hybrid arrays; wherein, the high-precision in-memory computing system includes an array selection module, a weight writing module, an optical processing module, and an in-memory computing module; The array selection module is used to obtain several moduli corresponding to the data to be calculated, and select the in-memory computing subarray corresponding to each moduli from the optoelectronic hybrid array; The weight writing module is used to obtain several weight remainders corresponding to the calculated weights according to the modulus, and write each weight remainder into the corresponding in-memory calculation subarray to obtain the in-memory calculation array after weight encoding. The optical processing module is used to obtain the input data remainder corresponding to each in-memory computing subarray according to the modulus and the data to be calculated, so as to input the input data remainder and the preset optical signal into the in-memory computing array to obtain the response optical signal output by the in-memory computing array. The in-memory calculation module is used to obtain the signal power of the response optical signal and obtain the phase offset of the response optical signal based on the signal power, so as to obtain the in-memory calculation result of the data to be calculated based on the phase offset.

10. A high-precision in-memory computing system based on a hybrid optoelectronic array according to claim 9, characterized in that, The optoelectronic hybrid array includes several initial computation subarrays; each of the initial computation subarrays includes the same several photonic memories; the array selection module includes a modulus unit and a matching unit; The module unit is used to obtain the phase modulation characteristic data corresponding to any module; and to obtain the unit phase modulation step size corresponding to each photonic memory in any initial computing subarray. The matching unit is used to use the initial computation subarray as the in-memory computation subarray corresponding to the modulus when the unit phase modulation step size is consistent with the phase modulation characteristic data.

Images