Signal processing method and device based on optical computing chip, and optoelectronic fusion computing system

Abstract

The application relates to the field of optical computing, and provides a signal processing method and device based on an optical computing chip and an optoelectronic fusion computing system. The signal processing device comprises an optical input modulation module, a first weight mapping module, a second weight mapping module and an optical output combining module. The optical input modulation module converts an input signal into a modulated optical signal. The first weight mapping module obtains a first output optical signal based on a first weight. The second weight mapping module obtains a second output optical signal based on a second weight. The optical output combining module obtains a comprehensive output signal based on the first output optical signal and the second output optical signal. The second weight mapping module comprises an optical dimension reduction unit, a low-dimensional optical modulation unit and an optical dimension increase unit. The optical dimension reduction unit performs low-dimensional projection on the modulated optical signal. The low-dimensional optical modulation unit adjusts the second weight based on a second weight update signal, and obtains a low-dimensional processed optical signal. The optical dimension increase unit performs high-dimensional projection on the low-dimensional processed optical signal.

Description

Signal processing methods and devices based on optical computing chips, and optoelectronic fusion computing systems Technical Field

[0001] This application relates to the field of optical computing, specifically to a signal processing method and apparatus based on an optical computing chip, and an optoelectronic fusion computing system. Background Technology

[0002] In recent years, foundational models (FMs), represented by Large Language Models (LLMs), have demonstrated outstanding capabilities in various tasks such as natural language processing and data analysis. However, the parameter scale of foundational models (FMs), represented by Large Language Models (LLMs), often reaches billions or even trillions, making full parameter fine-tuning extremely demanding on computing resources and storage capacity, resulting in high costs for model optimization and application. When performing large model inference computations on traditional electronic hardware such as GPUs, there are significant problems of excessive energy consumption and prominent latency.

[0003] Optical computing chips, with their core characteristics of high speed, high bandwidth, and low power consumption, are a key candidate technology for meeting the computing power demands of artificial intelligence. However, the scale of the computing matrices supported by existing optical computing chips is far smaller than the requirements of large models, making it impossible to support large-scale matrix calculations. Summary of the Invention

[0004] In view of this, the embodiments of this application are committed to providing a signal processing method and apparatus based on optical computing chips, and an optoelectronic fusion computing system, to solve the technical problem that the scale of the computing matrix supported by existing optical computing chips is far smaller than the requirements of large models, and thus cannot support large-scale matrix computing.

[0005] The first aspect of this application provides a signal processing device based on an optical computing chip, comprising:

[0006] An optical input modulation module, configured to convert an input signal into a modulated optical signal corresponding to the input signal;

[0007] A first weight mapping module is configured to obtain a first output optical signal corresponding to the modulated optical signal based on a first weight and the modulated optical signal corresponding to the input signal.

[0008] The second weight mapping module is configured to obtain a second output optical signal corresponding to the modulated optical signal based on the second weight and the modulated optical signal corresponding to the input signal, wherein the first output optical signal and the second output optical signal have the same dimension.

[0009] An optical output combining module, wherein the optical output combining module is configured to obtain a combined output signal based on the first output optical signal and the second output optical signal;

[0010] The second weight mapping module includes:

[0011] An optical dimension reduction unit is configured to perform a low-dimensional projection on the modulated optical signal to obtain a dimension-reduced optical signal corresponding to the modulated optical signal.

[0012] A low-dimensional optical modulation unit is configured to adjust a second weight based on a second weight update signal, and to obtain a low-dimensional processed optical signal based on the second weight and the reduced-dimensional optical signal.

[0013] An optical upscaling unit is configured to project the low-dimensional processed optical signal into a high-dimensional dimension to obtain an upscaled optical signal corresponding to the modulated optical signal, wherein the upscaled optical signal is the second output optical signal corresponding to the modulated optical signal.

[0014] In one embodiment of this application, the low-dimensional optical modulation unit includes at least one of a phase modulator, a Mach-Zehnder modulator array, an electro-optic modulator, and a thermo-optic control unit.

[0015] In one embodiment of this application, the optical dimension reduction unit includes at least one of a multimode interferometer, an arrayed waveguide grating, and a diffractive optical element;

[0016] And / or, the optical dimensionality enhancement unit includes at least one of a multimode interferometer, an arrayed waveguide grating, and a diffractive optical element.

[0017] In one embodiment of this application, the optical dimension reduction unit includes at least one of a phase modulator, a Mach-Zehnder modulator array, an electro-optic modulator, and a thermo-optic control unit;

[0018] And / or, the optical dimensionality enhancement unit includes at least one of a phase modulator, a Mach-Zehnder modulator array, an electro-optic modulator, and a thermo-optic control unit.

[0019] In one embodiment of this application, the first weight mapping module is configured to adjust the first weight based on the first weight update signal, and to obtain the first output optical signal corresponding to the modulated optical signal based on the first weight and the modulated optical signal corresponding to the input signal.

[0020] In one embodiment of this application, the first weight mapping module includes at least one of Clements interferometer network, Reich interferometer array, cross switch structure, multimode interferometer, or programmable diffraction network.

[0021] In one embodiment of this application, the optical output combining module includes at least one of an interferometric optical combiner, an optical power combiner, a photoelectric detection weighting structure, or a phase-sensitive interferometric structure.

[0022] In one embodiment of this application, at least two second weight mapping modules are provided, and the corresponding second weights of each second weight mapping module are different;

[0023] The signal processing device based on the optical computing chip further includes:

[0024] A path selection module is configured to, based on preset conditions, input at least a portion of the modulated optical signal corresponding to the input signal into the second weight mapping module to obtain at least one second sub-output optical signal.

[0025] An output weighting module is configured to weight and combine the second sub-output optical signals output by each of the second weight mapping modules to obtain a second output optical signal;

[0026] The optical output combining module is configured to obtain a combined output signal based on the first output optical signal and the second output optical signal.

[0027] In one embodiment of this application, the optical input modulation module is configured to encode the modulated optical signal using an encoding method including pulse amplitude modulation or pulse width modulation, and to multiplex the modulated optical signal using at least one of wavelength multiplexing, mode multiplexing, and time division multiplexing.

[0028] The optical output combining module is configured to demultiplex the first output optical signal and the second output optical signal, and then obtain a combined output signal based on the demultiplexed first output optical signal and the second output optical signal.

[0029] In one embodiment of this application, the optical input modulation module includes at least one of an arrayed waveguide grating, a mode demultiplexer, a multimode interference coupler, or a diffraction grating, and is configured to multiplex the modulated optical signal using at least one of the arrayed waveguide grating, the mode demultiplexer, the multimode interference coupler, or the diffraction grating in a wavelength multiplexing and / or mode multiplexing manner;

[0030] The optical output combining module includes at least one of an arrayed waveguide grating, a mode demultiplexer, a multimode interference coupler, or a diffraction grating, and is configured to demultiplex the first output optical signal and the second output optical signal using at least one of the arrayed waveguide grating, the mode demultiplexer, the multimode interference coupler, or the diffraction grating.

[0031] In one embodiment of this application, the optical input modulation module includes at least one of an optical delay line, a ring optical waveguide, or an optical path folding structure, and is configured to use at least one of the optical delay line, ring optical waveguide, or optical path folding structure to multiplex the modulated optical signal in a time-division multiplexing manner.

[0032] The optical output combining module is configured to demultiplex the first output optical signal and the second output optical signal using at least one of an optical delay line, a ring optical waveguide, or an optical path folding structure.

[0033] Another aspect of this application provides a signal processing method based on an optical computing chip, comprising:

[0034] The input signal is converted into a modulated optical signal corresponding to the input signal;

[0035] Based on the first weight and the modulated optical signal corresponding to the input signal, the first output optical signal corresponding to the modulated optical signal is obtained;

[0036] Based on the second weight and the modulated optical signal corresponding to the input signal, a second output optical signal corresponding to the modulated optical signal is obtained, wherein the first output optical signal and the second output optical signal have the same dimension;

[0037] A combined output signal is obtained based on the first output optical signal and the second output optical signal;

[0038] The step of obtaining the second output optical signal corresponding to the modulated optical signal based on the second weight and the modulated optical signal corresponding to the input signal includes:

[0039] The modulated optical signal is projected in a low dimension to obtain the dimension-reduced optical signal corresponding to the modulated optical signal.

[0040] The second weight is adjusted based on the second weight update signal, and the low-dimensional processed optical signal is obtained based on the second weight and the reduced-dimensional optical signal.

[0041] The low-dimensional processed optical signal is projected into a high dimension to obtain the up-dimensional optical signal corresponding to the modulated optical signal, wherein the up-dimensional optical signal is the second output optical signal corresponding to the modulated optical signal.

[0042] In one embodiment of this application, the step of obtaining the first output optical signal corresponding to the modulated optical signal based on the first weight and the modulated optical signal corresponding to the input signal includes:

[0043] The first weight is adjusted based on the first weight update signal, and the first output optical signal corresponding to the modulated optical signal is obtained based on the first weight and the modulated optical signal corresponding to the input signal.

[0044] In one embodiment of this application, the step of obtaining the second output optical signal corresponding to the modulated optical signal based on the second weight and the modulated optical signal corresponding to the input signal includes:

[0045] Based on at least two different second weights and the modulated optical signal corresponding to the input signal, at least one second sub-output optical signal corresponding to the modulated optical signal is obtained;

[0046] The individual second output optical signals are weighted and combined to obtain the second output optical signal.

[0047] In one embodiment of this application, the step of converting the input signal into a modulated optical signal corresponding to the input signal includes:

[0048] The input signal is converted into a modulated optical signal corresponding to the input signal, and the modulated optical signal is encoded by pulse amplitude modulation or pulse width modulation, and the modulated optical signal is multiplexed by at least one of wavelength multiplexing, mode multiplexing, and time division multiplexing.

[0049] The step of obtaining the combined output signal based on the first output optical signal and the second output optical signal includes:

[0050] After demultiplexing the first output optical signal and the second output optical signal, a combined output signal is obtained based on the demultiplexed first output optical signal and the second output optical signal.

[0051] In one embodiment of this application, both the input signal and the combined output signal are vector data; the signal processing method is applied to matrix-vector multiplication; or, both the input signal and the combined output signal are tensor data; the signal processing method is applied to tensor operations.

[0052] Another aspect of this application provides an optoelectronic fusion computing system, including the aforementioned signal processing device based on an optical computing chip, an optoelectronic hybrid computing device, an electronic computing device, and a main control unit;

[0053] The main control unit is configured to control the signal processing device based on the optical computing chip, the optoelectronic hybrid computing device, and the electro-computing device to perform signal processing individually or jointly.

[0054] The signal processing device based on an optical computing chip in this application embodiment constructs a dual-path parallel structure consisting of an optical input modulation module, a first weight mapping module, a second weight mapping module, and an optical output combining module. This enables a hardware device capable of signal weight mapping and fusion output through optical signal transmission and processing. The parallel characteristics of the optical domain significantly reduce power consumption and increase signal processing bandwidth, effectively adapting to large-scale artificial intelligence model inference scenarios, improving model inference efficiency and adaptability, and effectively adapting to novel large-scale model structures such as Low-Rank Adaptation fine-tuning and Expert-Selection. When applied to large-scale model fine-tuning scenarios using Low-Rank Adaptation, the signal processing device based on an optical computing chip in this application embodiment can effectively achieve the coordinated operation of low-speed stable mapping of pre-trained weights and high-speed dynamic updating of weights. This effectively solves the problem that existing signal processing devices cannot simultaneously achieve complete and accurate weight mapping and high-speed weight updating, balancing the high precision of optical computing with the high flexibility of weight updates. The fusion processing of the dual-path output signals ensures the accuracy of the final output result, making it suitable for high-bandwidth, low-power inference acceleration of large-scale artificial intelligence models. Attached Figure Description

[0055] It should be understood that the following figures only illustrate certain embodiments of this application and should not be construed as limiting the scope.

[0056] It should be understood that the same or similar reference numerals are used in the accompanying drawings to denote the same or similar elements.

[0057] It should be understood that the accompanying drawings are only schematic, and the dimensions and scales of the elements in the drawings are not necessarily precise.

[0058] Figure 1 is a schematic diagram of the structure of a signal processing device based on an optical computing chip according to an embodiment of this application.

[0059] Figure 2 is a schematic diagram of the specific structure of the signal processing device based on the optical computing chip according to an embodiment of this application.

[0060] Figure 3 is a schematic diagram of another signal processing device based on an optical computing chip according to an embodiment of this application.

[0061] Figure 4 is a schematic diagram of the steps of the signal processing method based on the optical computing chip according to an embodiment of this application.

[0062] Figure 5 is a schematic diagram of another step of the signal processing method based on an optical computing chip according to an embodiment of this application.

[0063] Figure 6 is a schematic diagram of another step of the signal processing method based on an optical computing chip according to an embodiment of this application.

[0064] Figure 7 is a schematic diagram of another step of the signal processing method based on an optical computing chip according to an embodiment of this application.

[0065] Figure 8 is a schematic diagram of another step of the signal processing method based on an optical computing chip according to an embodiment of this application.

[0066] Attached image labels:

[0067] 10. Optical input modulation module; 20. First weight mapping module; 30. Second weight mapping module; 31. Optical dimensionality reduction unit; 32. Low-dimensional optical modulation unit; 33. Optical dimensionality increase unit; 40. Optical output combining module; 50. Path selection module; 60. Output weighting module. Detailed Implementation

[0068] Numerous specific details are set forth below to provide an understanding of the structure, function, and use of the embodiments described and illustrated in the specification and figures. It is to be understood that the embodiments described and illustrated herein are non-limiting examples, and thus it will be appreciated that the particular structural and functional details disclosed herein are representative and exemplary. Variations and changes may be made to these embodiments without departing from the scope of the claims.

[0069] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0070] This application provides a signal processing device based on an optical computing chip. This signal processing device based on a photonic computing chip can be integrated on various optical computing chips such as thin-film lithium niobate (TFLN) and packaged with electrical control, driving and readout circuits through optoelectronic hybrid integration.

[0071] As shown in Figures 1 and 2, the signal processing device based on a photonic computing chip in this application embodiment may include an optical input modulation module 10, a first weight mapping module 20, a second weight mapping module 30, and an optical output combining module 40.

[0072] The optical input modulation module 10 is configured to convert the input signal into a modulated optical signal corresponding to the input signal. The optical input modulation module 10 can convert the input signal in electrical form into a modulated optical signal that can be transmitted and processed in the optical computing chip; that is, the optical input modulation module 10 serves as the interface between electrical domain input and optical domain processing. The input signal may include input feature data of an artificial intelligence model, and may include data in vector or tensor form. The modulated optical signal is an optical signal carrying the input signal information; information encoding can be achieved by modulating parameters such as the pulse width or pulse amplitude of the light, and it can be transmitted in the optical computing chip structure.

[0073] The first weight mapping module 20 is configured to obtain the first output optical signal corresponding to the modulated optical signal based on the first weight and the modulated optical signal corresponding to the input signal.

[0074] Specifically, the first weight mapping module 20 is a functional unit capable of performing weight mapping operations between the first weight and the modulated optical signal. In the application of the signal processing device based on an optical computing chip in this embodiment of the invention, when used in a large model fine-tuning scenario of Low-Rank Adaptation, the first weight can be a pre-trained weight W0, corresponding to the static weight parameters of the model, requiring no frequent updates. The first weight mapping module 20 can perform stable linear transformations and can employ a low-speed adjustable optical structure to achieve accurate optical mapping of the pre-trained weight W0. The first output optical signal is the optical signal output by the first weight mapping module 20 after completing the weight mapping operation, capable of carrying the result information after the first weight and the input signal are processed.

[0075] The second weight mapping module 30 is configured to obtain a second output optical signal corresponding to the modulated optical signal based on the second weight and the modulated optical signal corresponding to the input signal. The first output optical signal and the second output optical signal have the same dimension. Dimensionality, also known as the number of dimensions, is the number of independent parameters in the data contained in the modulated optical signal, the first output optical signal, and the second output optical signal.

[0076] Specifically, the second weight mapping module 30 is a functional unit capable of performing weight mapping operations between the second weight and the modulated optical signal. In the application of the signal processing device based on an optical computing chip in this embodiment of the invention, when used in a large model fine-tuning scenario of Low-Rank Adaptation, the second weight can be a high-speed update weight ΔW, enabling high-speed updates and fine-tuning to meet the dynamic adaptation requirements of the model. The second output optical signal is the optical signal output by the second weight mapping module 30 after completing the weight mapping operation. It carries the result information after the second weight and the input signal are processed. The second output optical signal has the same dimension as the first output optical signal, ensuring effective fusion can be achieved subsequently.

[0077] Specifically, as shown in Figures 1 and 2, the second weight mapping module 30 includes an optical dimensionality reduction unit 31, a low-dimensional optical modulation unit 32, and an optical dimensionality increase unit 33. The optical dimensionality reduction unit 31 is configured to perform low-dimensional projection on the modulated optical signal to obtain the dimensionality-reduced optical signal corresponding to the modulated optical signal. The optical dimensionality reduction unit 31 can reduce the complexity of subsequent processing and improve the efficiency of weight updates by reducing the signal dimension. The dimensionality-reduced optical signal is the optical signal output by the optical dimensionality reduction unit 31 after completing the low-dimensional projection; its dimension is lower than the original modulated optical signal, effectively preserving the core feature information of the input signal.

[0078] The low-dimensional optical modulation unit 32 is configured to adjust the second weights based on the second weight update signal, and obtain a low-dimensional processed optical signal based on the second weights and the dimensionality-reduced optical signal. The low-dimensional optical modulation unit 32 can process the dimensionality-reduced optical signal in a low-dimensional space and can be tuned at high speed to adapt to dynamic weight update requirements, thereby realizing second weight adjustment and weight mapping. The second weight update signal can trigger and control the second weight adjustment, and can be generated according to model training or task adaptation requirements, driving parameter changes in the low-dimensional optical modulation unit 32. The low-dimensional processed optical signal is the optical signal output by the low-dimensional optical modulation unit 32 after completing the second weight mapping, and can carry the low-dimensional result information after the second weights and the dimensionality-reduced optical signal are processed.

[0079] The optical upscaling unit 33 is configured to project the low-dimensional processed optical signal into a high-dimensional space to obtain the upscaled optical signal corresponding to the modulated optical signal. The upscaled optical signal is the second output optical signal corresponding to the modulated optical signal. The optical upscaling unit 33 can project the low-dimensional processed optical signal back into the high-dimensional space, restoring it to the same dimension as the first output optical signal, ensuring that the output optical signals of the two paths can be effectively fused.

[0080] The optical output combining module 40 is configured to obtain a combined output signal based on the first output optical signal and the second output optical signal. The optical output combining module 40 can fuse the first output optical signal and the second output optical signal in the optical domain or the electrical domain to generate a final combined output signal. The combined output signal carries the result of the first weight and the second weight acting on the input signal and can be converted into an electrical signal for subsequent applications.

[0081] Specifically, in the operation of the signal processing device based on the optical computing chip in this application embodiment, the optical input modulation module 10 first receives the input signal in the form of an external electrical signal, and converts the electrical signal into a modulated optical signal through an electro-optic modulation structure. Specifically, the input signal is encoded by adjusting parameters such as the pulse width or pulse amplitude of the light, so that the optical parameters of the modulated optical signal correspond to the values ​​of the input signal, thereby completing the conversion from the electrical domain signal to the optical domain signal.

[0082] Subsequently, the modulated optical signal is simultaneously transmitted to the first weight mapping module 20 and the second weight mapping module 30, which are set up in parallel. When the modulated optical signal enters the first weight mapping module 20, the weight mapping operation between the first weight and the modulated optical signal is completed, and finally the first output optical signal carrying the operation result is output.

[0083] When the modulated optical signal enters the second weight mapping module 30, it first enters the optical dimensionality reduction unit 31 to achieve dimensionality reduction and mixing of the high-dimensional signal, projecting the high-dimensional modulated optical signal onto the low-dimensional subspace and outputting the dimensionality-reduced optical signal, thus completing the linear transformation from high-dimensional to low-dimensional. Subsequently, the dimensionality-reduced optical signal enters the low-dimensional optical modulation unit 32, where the relevant weight control unit generates a second weight update signal according to model adaptation requirements, achieving dynamic adjustment of the second weight. The dimensionality-reduced optical signal undergoes weight mapping operations between the adjusted second weight and the dimensionality-reduced optical signal in the low-dimensional optical modulation unit 32, outputting the low-dimensional processed optical signal. Then, the low-dimensional processed optical signal enters the optical dimensionality enhancement unit 33 to achieve projection from the low-dimensional signal to the high-dimensional signal. The dimensionality-enhanced optical signal is then output, which is the second output optical signal, and its dimension is consistent with the first output optical signal.

[0084] Finally, the first and second output optical signals are transmitted to the optical output combining module 40, which performs fusion processing on the two optical signals. The two optical signals undergo optical interference or weighted superposition within the optical output combining module 40, ultimately outputting a composite output signal carrying the result of the comprehensive calculation. This composite output signal can be converted into an electrical signal and transmitted to subsequent signal processing units to complete the entire signal processing flow.

[0085] Therefore, the signal processing device based on the optical computing chip in this application embodiment can realize a hardware device that can achieve signal weight mapping and fusion output through optical signal transmission and processing by constructing a dual-path parallel structure of optical input modulation module 10, first weight mapping module 20, second weight mapping module 30 and optical output combining module 40. It significantly reduces power consumption and increases signal processing bandwidth through the parallel characteristics of the optical domain. It can effectively adapt to large-scale artificial intelligence model inference scenarios, improve the efficiency and adaptability of model inference, and effectively adapt to new large model structures such as Low-Rank Adaptation fine-tuning and Expert-Selection.

[0086] When the signal processing device based on the optical computing chip in this embodiment is applied to the fine-tuning of large models in Low-Rank Adaptation, it can effectively achieve the coordinated operation of slow and stable mapping of pre-trained weights and dynamic updating of weights at high speed. This effectively solves the problem that existing signal processing devices cannot simultaneously achieve complete and accurate weight mapping and high-speed weight updating, balancing the high precision of optical computing with the high flexibility of weight updates. The fusion processing of dual-path output signals ensures the accuracy of the final output result, making it suitable for high-bandwidth, low-power inference acceleration of large-scale artificial intelligence models.

[0087] In one embodiment of this application, the low-dimensional optical modulation unit 32 includes at least one of a phase modulator, a Mach-Zehnder modulator array, an electro-optic modulator, and a thermo-optic control unit.

[0088] Understandably, the low-dimensional optical modulation unit 32 can process the dimensionality-reduced optical signal in a low-dimensional space and can be tuned at high speed to adapt to the dynamic weight update requirements, thereby realizing the second weight adjustment and weight mapping. The low-dimensional optical modulation unit 32 typically has high-speed tuning capability to adapt to the dynamic update requirements of the second weight. The low-dimensional optical modulation unit 32 may include any one of a phase modulator, a Mach-Zehnder modulator array, an electro-optic modulator, and a thermo-optic control unit, or a combination of at least two of these devices.

[0089] Phase modulators can change the phase of optical signals through physical mechanisms such as electro-optic and thermo-optic effects, achieving weight encoding through phase changes. They allow for precise control of the phase parameters of the optical signal, enabling fine-tuning of weights, and offer fast response times, making them suitable for high-speed weight update scenarios. Mach-Zehnder modulator arrays consist of multiple Mach-Zehnder modulators arranged according to a fixed rule. Each Mach-Zehnder modulator can independently achieve dual amplitude and phase control. The array design supports multi-channel parallel modulation, adapting to the simultaneous processing of multi-channel signals in low-dimensional space. Electro-optic modulators utilize the electro-optic effect—the change in the refractive index of a material under the influence of an electric field—to modulate optical signals. They can rapidly change the amplitude, phase, and other parameters of the optical signal by adjusting the applied voltage, offering fast modulation speed and short response time, making them core devices for high-speed weight updates. Thermo-optic control units utilize the thermo-optic effect—the change in the refractive index of a material caused by temperature changes—to control optical signals. By changing the temperature of the waveguide region through heating elements, they adjust the transmission characteristics of the optical signal to achieve weight encoding. They feature simple structure and high stability.

[0090] Specifically, as shown in Figure 2, in one embodiment of this application, the low-dimensional optical modulation unit 32 adopts a combination structure of Mach-Zehnder modulator array and phase modulator to realize high-speed adjustment and weight mapping of the second weight.

[0091] The specific working process is as follows: First, the relevant weight control unit generates a second weight update signal according to the model's adaptation requirements. This signal is divided into two paths and transmitted to the Mach-Zehnder modulator array and the phase modulator, respectively, to achieve coordinated control. The Mach-Zehnder modulator array adopts a multi-channel structure, with each channel corresponding to one dimensionality-reduced optical signal. Each Mach-Zehnder modulator integrates electro-optic modulation electrodes. The second weight update signal adjusts the voltage applied to the electrodes, using the electro-optic effect to change the refractive index of the waveguide, thereby controlling the phase difference of the two-arm optical signals. By controlling the phase difference, the amplitude of the dimensionality-reduced optical signal of the corresponding channel is modulated, completing the first stage of weight encoding. Each channel can be independently controlled, ensuring the weight independence of the multi-channel signals. Subsequently, the optical signal modulated by the Mach-Zehnder modulator array is transmitted to the phase modulator array, which corresponds one-to-one with each optical computing chip channel. The second weight update signal drives the phase modulator array to perform precise phase fine-tuning on each optical signal, further optimizing the accuracy of weight mapping. The phase modulator rapidly changes the phase of the optical signal through the electro-optic effect, with an extremely short response time, reaching the nanosecond level, ensuring high-speed updates of the second weight. Through the synergistic effect of amplitude modulation by the Mach-Zehnder modulator array and phase fine-tuning by the phase modulator, the weight mapping operation between the second weight and the dimensionality-reduced optical signal is completed, outputting the low-dimensional processed optical signal.

[0092] In another optional embodiment, the low-dimensional optical modulation unit 32 can adopt a combined structure of an electro-optic modulator and a thermo-optic control unit. The electro-optic modulator, as the main control device, achieves high-speed amplitude and phase modulation to meet the requirement of rapid update of the second weight; the thermo-optic control unit, as an auxiliary control device, fine-tunes the waveguide temperature through heating elements to compensate for modulation deviations caused by changes in ambient temperature or device aging, thereby improving the stability of weight mapping.

[0093] The signal processing device based on an optical computing chip in this embodiment employs a combination or individual use of various tunable devices, such as phase modulators, Mach-Zehnder modulator arrays, electro-optic modulators, and thermo-optic control units, to construct a low-dimensional optical modulation unit 32. This fully leverages the advantages of each device to achieve high-speed and accurate updates of the second weight. The high-speed response characteristics of electro-optic modulators ensure the required rate for second weight updates and adapt to dynamic adjustment needs; the stable characteristics of the thermo-optic control unit enhance the reliability of the modulation process; and the flexible combination of multiple devices allows the low-dimensional optical modulation unit 32 to adapt to different weight update accuracy and rate requirements, further enhancing the flexibility and applicability of the aforementioned signal processing device based on an optical computing chip, and ensuring the accuracy and efficiency of the second weight mapping.

[0094] In one embodiment of this application, the optical dimension reduction unit 31 includes at least one of a multimode interferometer, an arrayed waveguide grating, and a diffractive optical element; similarly, in one embodiment of this application, the optical dimension enhancement unit 33 includes at least one of a multimode interferometer, an arrayed waveguide grating, and a diffractive optical element.

[0095] In one embodiment of this application, the optical dimensionality reduction unit 31 and the optical dimensionality enhancement unit 33 may include passive, fixed, and non-volatile optical structures that perform linear transformations from high-dimensional to low-dimensional and from low-dimensional to high-dimensional, without relying on external dynamic control. Specifically, the multimode interferometer is a passive optical device that achieves optical signal coupling and beam splitting based on the multimode interference effect. By performing physical processes such as interference and self-imaging of optical signals transmitted in the multimode waveguide, it realizes signal power distribution and dimensional transformation. It has a compact structure, high integration, and can effectively achieve dimensional transformation. The arrayed waveguide grating includes input and output waveguides, planar waveguides, and arrayed waveguide components. It utilizes the optical path difference of different wavelength optical signals propagating in the arrayed waveguide to achieve wavelength splitting and combining. At the same time, the dimensional projection transformation of the signal can be realized through waveguide structure design, resulting in high splitting accuracy and low crosstalk. Diffractive optical elements are passive devices that utilize the diffraction effect of light to control optical signals by etching specific diffraction structures on a substrate. They can project high-dimensional signals to low-dimensional signals or recover low-dimensional signals to high-dimensional signals by designing diffraction patterns. They are flexible in structure and adaptable to various dimensional transformation requirements.

[0096] The signal processing device based on an optical computing chip in this embodiment employs passive, fixed, and non-volatile optical structures such as multimode interferometers, arrayed waveguide gratings, and diffractive optical elements to construct optical dimensionality reduction unit 31 and optical dimensionality enhancement unit 33, achieving stable and low-loss linear transformation between high-dimensional and low-dimensional signals. The passive structure eliminates the need for external dynamic control, reducing system energy consumption and complexity while ensuring the accuracy and stability of the dimensionality transformation. The flexible selection and combination of various devices are suitable for different chip integration scenarios and dimensionality transformation requirements, further improving the compatibility and reliability of the aforementioned signal processing device based on an optical computing chip, and providing a stable dimensionality transformation basis for the high-speed weight update of the second weight mapping module 30.

[0097] In another embodiment of this application, the optical dimension reduction unit 31 includes at least one of a phase modulator, a Mach-Zehnder modulator array, an electro-optic modulator, and a thermo-optic control unit; and / or, the optical dimension increase unit 33 includes at least one of a phase modulator, a Mach-Zehnder modulator array, an electro-optic modulator, and a thermo-optic control unit.

[0098] In one embodiment of this application, the optical dimensionality reduction unit 31 and the optical dimensionality enhancement unit 33 may be tunable, but they do not undergo dynamic tuning during actual operation, maintaining fixed dimensionality transformation parameters to ensure the stability of the dimensionality transformation. When a phase modulator is integrated into the optical dimensionality reduction unit 31 or the optical dimensionality enhancement unit 33, fixed phase control parameters are preset to form a fixed dimensionality transformation path, which is not dynamically adjusted with the update of the second weight. When a Mach-Zehnder modulator array is integrated into the optical dimensionality reduction unit 31 or the optical dimensionality enhancement unit 33, the amplitude and phase control parameters of each modulator are pre-calibrated and fixed, and fixed dimensionality transformation of multi-channel signals is achieved through array design. When an electro-optic modulator is integrated into the optical dimensionality reduction unit 31 or the optical dimensionality enhancement unit 33, a fixed voltage is pre-applied to put the device in a fixed modulation state, forming fixed dimensionality transformation characteristics. The thermo-optical modulation unit is a unit that uses the thermo-optical effect to modulate optical signals. When integrated into the optical dimension reduction unit 31 and the optical dimension increase unit 33, a fixed temperature is preset to keep the waveguide at a fixed refractive index and achieve a fixed dimension transformation. Parameters can be calibrated by adjusting the temperature, but the operation remains stable.

[0099] During the operation of the signal processing device based on the optical computing chip in this embodiment, the tunable devices of the optical dimensionality reduction unit 31 and the optical dimensionality enhancement unit 33 maintain fixed tuning parameters, while only the low-dimensional optical modulation unit 32 performs high-speed dynamic tuning according to the second weight update signal. If the dimensionality transformation parameters need to be adjusted subsequently, they can be fixed again after adjusting the parameters of the tunable devices.

[0100] The signal processing device based on an optical computing chip in this embodiment employs tunable devices to construct an optical dimensionality reduction unit 31 and an optical dimensionality enhancement unit 33. Utilizing the parameter calibration capability of the tunable devices, precise calibration of the dimensionality transformation parameters can be achieved, improving the accuracy of the dimensionality transformation. Simultaneously, maintaining fixed tuning parameters during operation balances the calibration flexibility of the tunable devices with the stability and reliability of the fixed structure, avoiding increased energy consumption and signal fluctuations caused by dynamic tuning. This allows the optical dimensionality reduction unit 31 and the optical dimensionality enhancement unit 33 to adapt to different dimensionality transformation requirements through parameter adjustment while maintaining stability during operation, further enhancing the adaptability and stability of the aforementioned signal processing device based on an optical computing chip.

[0101] In one embodiment of this application, the first weight mapping module 20 is configured to adjust the first weight based on the first weight update signal, and to obtain the first output optical signal corresponding to the modulated optical signal based on the first weight and the modulated optical signal corresponding to the input signal.

[0102] In the signal processing device based on an optical computing chip according to this embodiment, the first weight mapping module 20 can not only store the first weights but also perform low-speed adjustments to them. When the signal processing device based on an optical computing chip according to this embodiment is applied to a large model fine-tuning scenario of Low-Rank Adaptation, the first weights are the pre-trained basic weights of the model. The first weight mapping module 20 can not only map the pre-trained weights but also perform low-speed fine-tuning of the pre-trained weights as needed, ensuring high fidelity of the weight mapping. The first weight update signal is a control signal that can trigger and control the adjustment of the first weights. Compared with the second weight update signal, the first weight update signal has a lower update frequency and is usually used for the calibration of pre-trained weights or fine-tuning to adapt to different task scenarios. It does not perform high-frequency dynamic updates. Low-speed adjustments can be performed under the drive of the first weight update signal, and the adjusted weights remain relatively stable.

[0103] In the signal processing device based on the optical computing chip in this application embodiment, the first weight mapping module 20 has a low-speed weight adjustment capability, which enables the first weight to be flexibly fine-tuned according to task requirements, effectively adapting to a variety of different task scenarios; the low-speed fine-tuning of the first weight and the high-speed update of the second weight work together to ensure the stability and high fidelity of the basic weight, and to realize the rapid adaptation of the dynamic compensation weight, further improving the task adaptability and inference accuracy of the signal processing device based on the optical computing chip.

[0104] Specifically, in one embodiment of this application, the first weight mapping module 20 includes at least one of Clements interferometer network, Reich interferometer array, cross switch structure, multimode interferometer or programmable diffraction network.

[0105] Specifically, the first weight mapping module 20 adopts a reconfigurable interferometric network structure, which can ensure accurate mapping and low-speed fine-tuning of the first weight, adapting to the inference requirements of multi-task scenarios. The Clements interferometric network is an N×N unitary matrix network based on cascaded MZIs. Weight parameters are configured and adjusted by adjusting the phase shifters in the MZIs, exhibiting good reconfigurability and enabling arbitrary complex weight mapping, adapting to the low-speed fine-tuning of the first weight. The Reck interferometric array is a reconfigurable interferometric array based on asymmetric cascaded MZIs. Its structure is more compact and has lower loss than the Clements interferometric network. Weight adjustment is achieved by adjusting the phase shifters of the cascaded MZIs, effectively reducing the required optical computing chip layout area. The cross-switch structure is a network formed by multiple cross-connected optical switches. Signal transmission path selection and weight encoding are achieved by controlling the on / off states of the optical switches, enabling arbitrary input and output connections, exhibiting high flexibility, and adapting to multi-task weight mapping requirements. Multimode interferometers are devices that achieve signal coupling and weight mapping based on the multimode interference effect. Weight configuration can be achieved by adjusting the structural parameters of the multimode waveguide, resulting in a stable structure and high integration. Programmable diffraction networks are diffraction structures built based on subwavelength gratings or metasurfaces. Weight mapping is achieved by controlling the diffraction phase. The optical properties of the diffraction structure can be adjusted through electrical signal programming, enabling low-speed weight adjustment. They are compact and highly compatible.

[0106] The signal processing device based on an optical computing chip in this embodiment employs various reconfigurable interferometric network structures, such as Clements interferometer networks, Reichschaft interferometer arrays, and cross-switch structures, to construct a first weight mapping module 20. This fully leverages the advantages of each structure to achieve accurate mapping and flexible, low-speed adjustment of the first weights. The reconfigurable structure allows the first weight mapping module 20 to adapt to pre-trained weights for various task scenarios, improving task compatibility; low-speed adjustment ensures stability during the first weight update process and reduces energy consumption. The combination of these multiple structures further enhances the design flexibility and adaptability of the signal processing device based on an optical computing chip in this embodiment, better adapting to multi-task, high-precision artificial intelligence model inference.

[0107] In one embodiment of this application, the optical output combining module 40 includes at least one of an interferometric optical combiner, an optical power combiner, a photoelectric detection weighting structure, or a phase-sensitive interferometric structure.

[0108] The optical output combining module 40 can fuse the first and second output optical signals to generate a comprehensive output signal. Optical or electrical domain fusion methods can be selected according to requirements to ensure effective superposition of the two signals. The interferometric optical combiner is a device that combines two or more optical signals based on the principle of coherent optical interference. By adjusting the phase difference of the signals, constructive or destructive interference is achieved, completing the weighted superposition of the signals. High-precision signal fusion can be achieved directly in the optical domain. The optical power combiner includes devices that combine signals based on the principle of incoherent optical power superposition. It does not rely on the phase characteristics of the signals and directly linearly superimposes the optical power of multiple signals. It has a simple structure, high stability, and is suitable for coarse-grained signal fusion. The photoelectric detection weighted structure is a hybrid structure that first converts the optical signal into an electrical signal and then achieves weighted superposition through electrical domain circuitry. Its core includes a photodetector and an electrical domain operation circuit, enabling the conversion and fusion of optical domain signals to electrical domain signals. Phase-sensitive interference structure is an advanced structure for signal combining based on the principle of phase-sensitive interference. It performs constructive interference superposition only on signals of a specific phase, suppressing stray signals with phase mismatch, and can achieve high-precision and highly anti-interference signal fusion.

[0109] The flexible selection and combination of various combining structures in the signal processing device based on the optical computing chip in this application embodiment are suitable for different accuracy requirements and application scenarios, further improving the output accuracy, anti-interference ability and adaptability of the aforementioned signal processing device based on the optical computing chip, and ensuring the reliability of the final integrated output signal.

[0110] As shown in Figure 2, in one specific embodiment of this application, the optical input modulation module 10 in the signal processing device based on the optical computing chip may include 128 Mach-Zehnder modulators (MZIs). The optical input modulation module 10 receives an input signal in the form of an externally input electrical signal, and converts the electrical signal into a modulated optical signal through the 128 Mach-Zehnder modulators (MZIs). The modulated optical signal has a dimension of 128. In a specific embodiment, the frequency of the modulated optical signal can be 1 GHz.

[0111] The first weight mapping module 20 internally employs a cross-switching structure based on phase change materials, with a matrix size of 128×128. This structure is used to store and load the pre-trained weights W0 (i.e., the first weights) of the model. Each cross node unit is formed by the intersection of phase change material and waveguide. The light transmittance of each node is adjusted by pre-applied phase change electrical pulses to achieve fixed storage of the first weights. When the modulated optical signal enters the first weight mapping module 20, the weight mapping operation between the first weights and the modulated optical signal is completed through optical coupling and transmittance modulation during waveguide transmission. Finally, the first output optical signal carrying the operation result is output, which is still 128-dimensional. Since the first weights are pre-trained weights, they do not need to be updated frequently. The update rate of this module is set to 1kHz to ensure low power consumption and high stability.

[0112] Understandably, when a slow update of the first weight is required, the first weight mapping module 20 employs a reconfigurable interferometric network structure based on phase change materials, with a matrix size of 128×128. Each cross node unit of this structure is composed of a phase change material and a waveguide cross, and reversible light transmittance adjustment can be achieved through phase change electrical pulses, thereby realizing the storage and adjustment of the first weight. When the signal processing device starts working, the control unit corresponding to the first weight inputs an initial calibration signal to the first weight mapping module 20, loading the pre-trained first weight into the cross node unit, completing the initial configuration of the first weight. At this time, the first weight mapping module 20 performs weight mapping calculations on the modulated optical signal based on the initial pre-trained weights and outputs the first output optical signal. When the first weight needs to be updated, the control unit corresponding to the first weight generates a first weight update signal containing information on the weight parameters to be adjusted, and this signal is transmitted to the first weight mapping module 20. The first weight mapping module 20 applies a phase change electrical pulse to the corresponding cross node unit according to the signal, changing the light transmittance of the node through the phase change characteristics of the phase change material, thereby realizing the fine adjustment of the first weight. Since the first weight is the basic weight, it does not require high-frequency updates. Its update rate is set to 1kHz, which is a low-speed adjustment. During the adjustment process, the device can still perform signal processing normally, with only the weight parameters changing slowly to ensure system stability. After the first weight adjustment is completed, the first weight mapping module 20 continues to perform weight mapping calculations on the modulated optical signal based on the adjusted first weight, and outputs the adjusted first output optical signal.

[0113] The second weight mapping module 30 is used to perform the weight mapping operation between the second weight required for LoRA low-rank adaptation, namely the high-speed update weight ΔW, and the modulated optical signal.

[0114] In the first step, the modulated optical signal first enters the optical dimensionality reduction unit 31. This unit adopts a 128×8 multimode interferometer structure, which is etched with four layers of microgroove structures, each layer containing 1024 slots. Each slot is finely tuned in phase or amplitude through waveguide perturbation. After the modulated optical signal enters the multimode interferometer, a multimode interference process occurs inside. Through modulation by the stacked slots, the high-dimensional signal is reduced and mixed, projecting the 128-dimensional modulated optical signal onto an 8-dimensional low-dimensional subspace, and outputting an 8-dimensional reduced optical signal, completing the linear transformation from high-dimensional to low-dimensional.

[0115] In the second step, the dimensionality-reduced optical signal enters the low-dimensional optical modulation unit 32, which consists of an 8×8 Clements interferometer network and a 1×8 phase shifter array. The relevant control unit generates a second weight update signal according to the model adaptation requirements. This signal drives the phase shifter array to perform high-speed tuning, adjusting the phase parameters of the phase shifters through electro-optic effects, thereby achieving dynamic adjustment of the second weight. The dimensionality-reduced optical signal is transmitted in the Clements interferometer network. Through phase shift modulation and optical interference of each MZI interferometer unit, the weight mapping operation between the adjusted second weight and the dimensionality-reduced optical signal is completed, outputting an 8-dimensional low-dimensional processed optical signal. The update rate of this low-dimensional optical modulation unit 32 can reach the nanosecond level, achieving high-speed updating of the second weight.

[0116] The third step involves the low-dimensional processed optical signal entering the optical upscaling unit 33. This unit employs an 8×128 multimode interferometer structure, whose design matches that of the multimode interferometer in the optical downscaling unit 31. It also utilizes the multimode interference effect to project the low-dimensional signal to a higher-dimensional signal. The 8-dimensional low-dimensional processed optical signal undergoes multimode interference and mixing in the optical upscaling unit 33, and is projected back into a 128-dimensional high-dimensional space, outputting a 128-dimensional upscaled optical signal. This upscaled optical signal is the second output optical signal, and its dimension is consistent with the first output optical signal.

[0117] Finally, the first and second output optical signals are transmitted to the optical output combining module 40. This module employs an interferometric optical combiner structure, which fuses the two optical signals through coherent optical interference. The two optical signals converge in the coupler of the interferometric optical combiner, and constructive interference superposition is achieved by adjusting the phase difference between the two signals, ultimately outputting a comprehensive output signal carrying the result of the integrated calculation. This comprehensive output signal can be converted into an electrical signal by a photodetector and transmitted to the subsequent signal processing unit to complete the entire signal processing flow.

[0118] The energy storage device of this application embodiment, based on the signal processing device of the optical computing chip, can realize the equivalent optical calculation of y=W0×x+ΔW×x in Low-Rank Adaptation fine-tuning. According to the computing power evaluation model, the theoretical computing power of the optical dimensionality-upgrading unit and the optical dimensionality-reducing unit in the signal processing device based on the optical computing chip of this application embodiment is: T=2m×N2×r OPS; where m is the number of hidden layers, which can be taken as m=4; N is the effective matrix size, corresponding to the number of slots of each layer of the multimode interferometer, which is 1024; r is the operation rate, which is 1GHz in this embodiment. Then the computing power of a single optical dimensionality-upgrading unit or optical dimensionality-reducing unit is approximately: T=2×4×(1024)^2×10^9≈8.4×10^15 OPS, that is, about 8.4 POPS (Peta OPS).

[0119] Since the second weight mapping module 30 includes two structures—an optical dimensionality upscaling unit and an optical dimensionality downscaling unit—the total computing power can reach 16.8 POPS. Combined with the 128-channel parallel input of the optical input modulation module 10, the overall effective inference computing power of the signal processing device based on the optical computing chip in this embodiment exceeds 20 POPS, and its energy efficiency is significantly better than that of traditional electrical computing units. At an input rate of 1 GHz, the signal processing device based on the optical computing chip in this embodiment can complete a high-speed weight update ΔW compensation operation in sub-microseconds, which can meet the real-time requirements for online weight fine-tuning in large model inference.

[0120] In summary, the energy storage device of this application embodiment realizes the collaborative calculation of low-speed static weight loading and high-speed dynamic weight compensation, and can support online inference and fine-tuning of the Low-Rank Adaptation algorithm at the hardware level, with the advantages of high computing power, high bandwidth and high energy efficiency.

[0121] As shown in Figure 3, in one embodiment of this application, at least two second weight mapping modules 30 are provided, and the corresponding second weights of each second weight mapping module 30 are different; the signal processing device based on optical computing chip in this embodiment of the application also includes a path selection module 50 and an output weighting module 60.

[0122] The path selection module 50 is configured to, based on preset conditions, input at least a portion of the modulated optical signal corresponding to the input signal into the second weight mapping module 30 to obtain at least one second sub-output optical signal. That is, the path selection module 50 can selectively transmit the modulated optical signal to different second weight mapping modules 30, thereby obtaining at least one second sub-output optical signal.

[0123] The output weighting module 60 is configured to weight and combine the second sub-output optical signals output by each of the second weight mapping modules 30 to obtain the second output optical signal. The optical output combining module 40 is configured to obtain a combined output signal based on the first output optical signal and the second output optical signal.

[0124] In the signal processing device based on the optical computing chip in this application embodiment, at least two independent second weight mapping modules 30 are provided, each corresponding to a different second weight. During the operation of the signal processing device, the optical input modulation module 10 modulates the input signal into an optical signal and transmits it to the first weight mapping module 20 and the path selection module 50. The operation process of the first weight mapping module 20 is as described above and will not be repeated here.

[0125] After the modulated optical signal enters the path selection module 50, the input features carried by the modulated optical signal are first detected to determine its feature type. Based on the matching relationship between the detection result and preset conditions, the path selection module 50 controls the selective input of the modulated optical signal to at least one adapted second weight mapping module 30. The adapted second weight mapping module 30 processes the modulated optical signal respectively and outputs a second sub-output optical signal. It can be understood that different second sub-output optical signals carry different calculation results of second weights and input signals. The output weighting module 60 performs weighted combination of the second sub-output optical signals output by each second weight mapping module 30 to obtain the second output optical signal. Finally, the second output optical signal and the first output optical signal are transmitted to the optical output combining module 40, and after fusion, a comprehensive output signal is obtained, completing the signal processing flow of hybrid expert inference.

[0126] In this embodiment of the signal processing device based on an optical computing chip, the multiple second weight mapping modules 30 can be configured with different second weights to adapt to different types of input feature processing requirements; the path selection module 50 achieves selective signal transmission based on preset conditions to ensure that the input signal receives appropriate weight mapping processing; and the output weighting module 60 achieves effective fusion of multi-sub-path results to improve the accuracy of the output results. The signal processing device based on an optical computing chip in this embodiment can be configured with at least two second weight mapping modules 30 to form a multi-sub-path parallel structure. Combined with the path selection module 50 and the output weighting module 60, it implements a Mixture of Experts (MFA) inference mechanism, suitable for large-scale artificial intelligence model MFA inference scenarios, improving the parallelism and adaptability of model inference.

[0127] Specifically, in one embodiment of this application, the optical input modulation module 10 is configured to encode the modulated optical signal using an encoding method including pulse amplitude modulation or pulse width modulation, and to multiplex the modulated optical signal using at least one of wavelength multiplexing, mode multiplexing, and time division multiplexing; the optical output combining module 40 is configured to demultiplex the first output optical signal and the second output optical signal, and then obtain a combined output signal based on the demultiplexed first output optical signal and the second output optical signal.

[0128] That is, the optical input modulation module 10 of this application embodiment can encode the modulated optical signal by pulse amplitude modulation or pulse width modulation. Pulse amplitude modulation is a modulation method that encodes the input signal by adjusting the amplitude of the optical pulse. The change in the amplitude of the optical pulse corresponds to the change in the value of the input signal, which can achieve high-precision signal encoding. Pulse width modulation is a modulation method that encodes the input signal by adjusting the width of the optical pulse. The change in the width of the optical pulse corresponds to the change in the value of the input signal. It has a simple structure and strong anti-interference ability.

[0129] The optical input modulation module 10 of this application embodiment can multiplex the modulated optical signal in at least one of wavelength multiplexing, mode multiplexing, and time division multiplexing. That is, it can multiplex the modulated optical signal in any one of wavelength multiplexing, mode multiplexing, and time division multiplexing, or it can multiplex the modulated optical signal in a combination of at least two of wavelength multiplexing, mode multiplexing, and time division multiplexing.

[0130] Wavelength multiplexing encodes different input signals or weight parameters onto optical signals of different wavelengths, achieving parallel processing through transmission via the same waveguide. It utilizes the wavelength dimension of light for multiplexing. Mode multiplexing encodes different input signals or weight parameters onto optical signals of different spatial modes, achieving parallel processing through transmission via the same waveguide. It utilizes the spatial mode dimension of light to expand parallel channels.

[0131] Time-division multiplexing is a multiplexing method that encodes different input signals or weight parameters into different time slots in chronological order and achieves parallel processing through time-division transmission on the same waveguide, thereby expanding the parallel channel by utilizing the time dimension of light.

[0132] The optical output combining module 40 demultiplexes the first and second output optical signals, separates the multiplexed optical signals into independent single signals, and then obtains a combined output signal based on the demultiplexed first and second output optical signals. It is understood that the structure for demultiplexing the first and second output optical signals in the optical output combining module 40 can be selected according to the multiplexing method chosen by the optical input modulation module 10, and will not be elaborated further here.

[0133] The signal processing device based on an optical computing chip in this application embodiment employs pulse amplitude modulation or pulse width modulation for signal encoding, combined with various multiplexing methods such as wavelength multiplexing, mode multiplexing, and time division multiplexing, to fully utilize the multi-dimensional characteristics of light and significantly improve the device's computing power density and parallel processing capabilities. By multiplexing optical signals, multiple signals can be transmitted through the same waveguide, significantly improving the chip's integration and reducing energy consumption. The flexible combination of various encoding and multiplexing methods can adapt to different signal processing requirements, further enhancing the high bandwidth and low power consumption advantages of the signal processing device based on an optical computing chip in this application embodiment, making it more suitable for large-scale, high-computing-power artificial intelligence model inference scenarios.

[0134] Specifically, in one embodiment of this application, the optical input modulation module 10 includes at least one of an arrayed waveguide grating, a mode demultiplexer, a multimode interference coupler, or a diffraction grating, and is configured to use at least one of the arrayed waveguide grating, mode demultiplexer, multimode interference coupler, or diffraction grating to multiplex the modulated optical signal in a wavelength multiplexing and / or mode multiplexing manner; the optical output combining module 40 includes at least one of an arrayed waveguide grating, a mode demultiplexer, a multimode interference coupler, or a diffraction grating, and is configured to use at least one of the arrayed waveguide grating, mode demultiplexer, multimode interference coupler, or diffraction grating to demultiplex the first output optical signal and the second output optical signal.

[0135] Understandably, arrayed waveguide gratings are passive optical devices with wavelength combining and splitting capabilities. They can precisely combine (multiplex) or separate (demultiplex) optical signals of different wavelengths through the optical path difference of the arrayed waveguide, achieving high splitting accuracy and low crosstalk, thus enabling wavelength multiplexing / demultiplexing. Mode demultiplexers are optical devices with spatial mode combining and splitting capabilities. They can precisely separate (demultiplex) or combine (multiplex) optical signals of different spatial modes, utilizing the differences in propagation characteristics of different modes to achieve mode identification and combining / splitting, thus enabling mode multiplexing / demultiplexing. Multimode interference couplers are devices that achieve signal coupling and combining based on multimode interference effects. They can simultaneously achieve wavelength and mode multiplexing / demultiplexing, featuring a compact structure, high integration, and adaptability to multi-dimensional multiplexing fusion processing requirements. Diffraction gratings are devices that separate or combine wavelengths or modes based on the diffraction effect of light. They separate or combine signals of different wavelengths or modes by means of the difference in diffraction angles of the diffraction structure. They are simple in structure, low in cost, and suitable for coarse-grained multiplexing / demultiplexing scenarios.

[0136] The signal processing device based on an optical computing chip in this application embodiment achieves wavelength multiplexing and mode multiplexing by employing specialized optical devices such as arrayed waveguide gratings, mode demultiplexers, and multimode interference couplers. This significantly improves the accuracy of multiplexing and demultiplexing, reduces signal crosstalk, and ensures the transmission quality of multi-dimensional multiplexed signals. Using wavelength and mode multiplexing for optical signals can increase computing power density without increasing the area of ​​the optical computing chip, making the signal processing device based on an optical computing chip in this application embodiment more suitable for high-bandwidth, high-precision, large-scale optical computing inference scenarios.

[0137] In one embodiment of this application, the optical input modulation module 10 includes at least one of an optical delay line, a ring optical waveguide, or an optical path folding structure, and is configured to use at least one of the optical delay line, ring optical waveguide, or optical path folding structure to multiplex the modulated optical signal in a time-division multiplexing manner; the optical output combining module 40 includes at least one of an optical delay line, a ring optical waveguide, or an optical path folding structure, and is configured to use at least one of the optical delay line, ring optical waveguide, or optical path folding structure to demultiplex the first output optical signal and the second output optical signal.

[0138] Optical delay lines consist of miniature delay devices integrated on optical computing chips. They achieve precise time delays by controlling the length of the optical signal transmission path, enabling time slot division and signal alignment in time-division multiplexing to ensure orderly transmission and separation of signals in different time slots. Ring waveguides are passive waveguides with a ring structure. Optical signals circulate within the ring to achieve time delays. They are compact, offer stable delays, and effectively achieve time slot buffering and delay control. Optical path folding structures increase the transmission path length by bending and folding the waveguide. They achieve specified delays without requiring additional chip area, adapting to time-division multiplexing requirements and providing controllable delays.

[0139] The signal processing device based on an optical computing chip in this application embodiment achieves time-division multiplexing and demultiplexing by employing optical delay lines, ring optical waveguides, and optical path folding structures. This expands the parallel processing channels without increasing the number of waveguides, significantly improving computing density. The aforementioned time-division multiplexing device enables the orderly transmission and separation of time-slotted signals, with small timing deviations and low signal crosstalk, ensuring the processing accuracy of time-series signals. It requires additional power, adapting to low-power optical computing needs, and is compact, easy to integrate, and can seamlessly connect with other optical computing units.

[0140] As shown in Figure 4, this application embodiment also provides a signal processing method based on an optical computing chip, including:

[0141] S101: Converts the input signal into a modulated optical signal corresponding to the input signal.

[0142] The process involves converting an input signal in electrical form into a modulated optical signal that can be transmitted and processed within an optical computing chip, facilitating subsequent optical computation. Specifically, the input signal can include input feature data from an artificial intelligence model, and can be in vector or tensor form. The modulated optical signal, carrying the input signal information, can be encoded by modulating parameters such as pulse width or pulse amplitude, and can be transmitted within the optical computing chip structure.

[0143] S102: Based on the first weight and the modulated optical signal corresponding to the input signal, obtain the first output optical signal corresponding to the modulated optical signal.

[0144] Specifically, when the signal processing device based on the optical computing chip in this embodiment is applied to a large model fine-tuning scenario of Low-Rank Adaptation, the first weight can be a pre-trained weight W0, corresponding to the static weight parameters of the model, which does not need to be updated frequently. Based on the first weight and the modulated optical signal corresponding to the input signal, the first output optical signal corresponding to the modulated optical signal is obtained, realizing the accurate optical mapping of the pre-trained weight W0. The first output optical signal is the optical signal output by the first weight mapping module 20 after completing the weight mapping operation, which can carry the result information after the first weight and the input signal are processed.

[0145] S103: Based on the second weight and the modulated optical signal corresponding to the input signal, obtain the second output optical signal corresponding to the modulated optical signal.

[0146] It is understood that the first and second output optical signals have the same dimension. In the application of the signal processing device based on an optical computing chip in this embodiment of the invention, when used for fine-tuning large models in Low-Rank Adaptation, the second weight can be a high-speed update weight ΔW, enabling high-speed updates and fine-tuning to meet the dynamic adaptation requirements of the model. The second output optical signal is the optical signal output by the second weight mapping module 30 after completing the weight mapping operation. It can carry the result information after the second weight and the input signal are processed. The second output optical signal has the same dimension as the first output optical signal, ensuring effective fusion can be achieved subsequently.

[0147] As shown in Figure 5, step S103: obtaining the second output optical signal corresponding to the modulated optical signal based on the second weight and the modulated optical signal corresponding to the input signal includes:

[0148] S1031: Perform low-dimensional projection on the modulated optical signal to obtain the dimension-reduced optical signal corresponding to the modulated optical signal.

[0149] By performing low-dimensional projection on the modulated optical signal, the dimension-reduced optical signal corresponding to the modulated optical signal is obtained. This reduces the complexity of subsequent processing and improves the efficiency of weight update by reducing the signal dimension. The dimension-reduced optical signal is the optical signal output by the optical dimension reduction unit 31 after completing the low-dimensional projection. Its dimension is lower than that of the original modulated optical signal, which can effectively preserve the core feature information of the input signal.

[0150] S1032: Adjust the second weight based on the second weight update signal, and obtain the low-dimensional processed optical signal based on the second weight and the dimension-reduced optical signal.

[0151] By adjusting the second weight based on the second weight update signal, and obtaining a low-dimensional processed optical signal based on the second weight and the dimensionality-reduced optical signal, the dimensionality-reduced optical signal can be processed in a low-dimensional space. Furthermore, it can be tuned at high speed to adapt to dynamic weight update requirements, thereby realizing second weight adjustment and weight mapping. The low-dimensional processed optical signal is the optical signal output by the low-dimensional optical modulation unit 32 after completing the second weight mapping, and it can carry the low-dimensional result information after the calculation of the second weight and the dimensionality-reduced optical signal.

[0152] S1033: Perform high-dimensional projection on the low-dimensional processed optical signal to obtain the up-dimensional optical signal corresponding to the modulated optical signal.

[0153] Among them, the upgraded optical signal is the second output optical signal corresponding to the modulated optical signal.

[0154] By projecting the low-dimensional processed optical signal into a high-dimensional space, the modulated optical signal corresponding to the up-dimensional optical signal is obtained. This allows the low-dimensional processed optical signal to be projected back into the high-dimensional space and restored to the same dimension as the first output optical signal, ensuring that the output optical signals of the two paths can be effectively fused.

[0155] S104: A combined output signal is obtained based on the first output optical signal and the second output optical signal.

[0156] Specifically, the first and second output optical signals can be fused in the optical or electrical domain to generate a final integrated output signal. The integrated output signal carries the result of the first and second weights acting on the input signal and can be converted into an electrical signal for subsequent applications.

[0157] Therefore, the signal processing method based on optical computing chips in this application significantly reduces power consumption and increases signal processing bandwidth through the parallel characteristics of the optical domain. It can effectively adapt to large-scale artificial intelligence model inference scenarios, improving the efficiency and adaptability of model inference, and effectively adapting to novel large model structures such as Low-Rank Adaptation fine-tuning and Expert-Selection. When the signal processing method based on optical computing chips in this application is applied to large model fine-tuning scenarios of Low-Rank Adaptation, it can effectively achieve the coordinated work of low-speed stable mapping of pre-trained weights and high-speed dynamic updating of weights. This effectively solves the problem that existing signal processing devices cannot simultaneously achieve complete and accurate weight mapping and high-speed weight updating, balancing the high precision of optical computing with the high flexibility of weight updates. The fusion processing of dual-path output signals ensures the accuracy of the final output result, making it suitable for high-bandwidth, low-power inference acceleration of large-scale artificial intelligence models.

[0158] Specifically, as shown in Figure 6, in one embodiment of this application, S102: the step of obtaining the first output optical signal corresponding to the modulated optical signal based on the first weight and the modulated optical signal corresponding to the input signal includes:

[0159] S202: Adjust the first weight based on the first weight update signal, and obtain the first output optical signal corresponding to the modulated optical signal based on the first weight and the modulated optical signal corresponding to the input signal.

[0160] In the signal processing method based on optical computing chips in this application embodiment, the first weight can be adjusted at a low speed. When the signal processing method based on optical computing chips in this application embodiment is applied to the fine-tuning scenario of a large model in Low-Rank Adaptation, the first weight is the pre-trained basic weight of the model. The signal processing method based on optical computing chips in this application embodiment can not only map the pre-trained weights, but also fine-tune the pre-trained weights at a low speed as needed, ensuring high fidelity of weight mapping. The first weight update signal can trigger and control the control signal for adjusting the first weight. Compared with the second weight update signal, the first weight update signal has a lower update frequency and is usually used for the calibration of pre-trained weights or fine-tuning to adapt to different task scenarios. It does not perform high-frequency dynamic updates. It can be adjusted at a low speed under the drive of the first weight update signal, and remains relatively stable after adjustment.

[0161] In the signal processing method based on optical computing chips in this application embodiment, the low-speed fine-tuning of the first weight and the high-speed updating of the second weight work together to ensure the stability and high fidelity of the basic weights, and to achieve rapid adaptation of the dynamic compensation weights, thereby further improving the task adaptability and inference accuracy of the signal processing method based on optical computing chips.

[0162] As shown in Figure 7, in one embodiment of this application, step S103: obtaining the second output optical signal corresponding to the modulated optical signal based on the second weight and the modulated optical signal corresponding to the input signal includes:

[0163] S1034: Based on at least two different second weights and modulated optical signals corresponding to the input signals, obtain at least one second output optical signal corresponding to the modulated optical signal.

[0164] S1035: Weight the individual second output optical signals to obtain the second output optical signal.

[0165] In the signal processing method based on an optical computing chip according to the embodiments of this application, the input features carried by the modulated optical signal are first detected to determine their feature type. Based on the matching relationship between the detection result and preset conditions, at least one second output optical signal corresponding to the modulated optical signal is obtained, based on the modulated optical signal corresponding to different second weights and input signals. It can be understood that different second output optical signals carry different calculation results of the second weights and input signals. Then, the various second output optical signals are weighted and combined to obtain the second output optical signal. Finally, the second output optical signal and the first output optical signal are transmitted to the optical output combining module 40, and after fusion, a comprehensive output signal is obtained, completing the signal processing flow of hybrid expert inference.

[0166] That is, the signal processing method based on optical computing chip in this application embodiment can be configured with different second weights to adapt to different types of input feature processing requirements, ensure that the input signal is processed with appropriate weight mapping, and effectively fuse multi-sub-path results to improve the accuracy of output results, thereby realizing a Mixture of Experts inference mechanism, which is suitable for large-scale artificial intelligence model mixed expert inference scenarios and improves the parallelism and adaptability of model inference.

[0167] As shown in Figure 8, in one embodiment of this application, S101: the step of converting the input signal into a modulated optical signal corresponding to the input signal includes:

[0168] S301: Convert the input signal into a modulated optical signal corresponding to the input signal, encode the modulated optical signal by pulse amplitude modulation or pulse width modulation, and multiplex the modulated optical signal by at least one of wavelength multiplexing, mode multiplexing, and time division multiplexing.

[0169] S104: The step of obtaining the combined output signal based on the first output optical signal and the second output optical signal includes:

[0170] S304: After demultiplexing the first output optical signal and the second output optical signal, a combined output signal is obtained based on the demultiplexed first output optical signal and the second output optical signal.

[0171] The signal processing method based on the optical computing chip in this application embodiment can encode the modulated optical signal by pulse amplitude modulation or pulse width modulation, and then multiplex the modulated optical signal in at least one of wavelength multiplexing, mode multiplexing, and time division multiplexing. That is, it can multiplex the modulated optical signal in at least one of wavelength multiplexing, mode multiplexing, and time division multiplexing, or it can multiplex the modulated optical signal in a combination of at least two of wavelength multiplexing, mode multiplexing, and time division multiplexing.

[0172] The signal processing method based on the optical computing chip in this application embodiment can also demultiplex the first output optical signal and the second output optical signal, separate the multiplexed optical signals into independent single signals, and then obtain a comprehensive output signal based on the demultiplexed first output optical signal and the second output optical signal.

[0173] The signal processing method based on optical computing chips in this application can combine multiple multiplexing methods such as wavelength multiplexing, mode multiplexing, and time-division multiplexing to make full use of the multi-dimensional characteristics of light and greatly improve the computing power density and parallel processing capability of the device. The flexible combination of multiple encoding and multiplexing methods can be adapted to different signal processing requirements, making the signal processing method based on optical computing chips in this application more suitable for large-scale, high-computing-power artificial intelligence model inference scenarios.

[0174] In one embodiment of this application, both the input signal and the synthesized output signal are vector data; the signal processing method is applied to matrix-vector multiplication; or, both the input signal and the synthesized output signal are tensor data; the signal processing method is applied to tensor operations.

[0175] It's understandable that vector data, an ordered collection of data composed of single-dimensional elements, is the most basic input and output data form for artificial intelligence models, such as the model's input feature vector and output result vector. Matrix-vector multiplication, a linear algebra operation based on this, is the core operation for inference in artificial intelligence models. Tensor data, on the other hand, is an ordered collection of data composed of multi-dimensional elements, representing a higher-order extension of vectors and matrices. Tensor operations are performed on higher-order tensor data, including tensor-vector multiplication, matrix-matrix multiplication, tensor-matrix multiplication, and convolution / superposition between tensors, requiring multi-dimensional parallel processing support.

[0176] In embodiments where both the input signal and the combined output signal are vector data, the signal processing method based on an optical computing chip in this application is applied to matrix-vector multiplication. In this embodiment, the input signal can be a 128-dimensional vector, corresponding to the input feature vector of an artificial intelligence model. The first weight and the second weight are both 128×128 dimension matrices. The specific steps of the matrix-vector multiplication operation include:

[0177] First, the input signal in vector form is converted into 128 parallel modulated optical signals, with each signal corresponding to one dimension element of the vector. Then, through structures such as optical interference networks, a 128×128 first weight matrix is ​​used to perform a parallel optical domain multiplication operation with the modulated optical signal. That is, each row of the matrix weight is synchronously interfered and coupled with each dimension of the vector signal, outputting a 128-dimensional vector form first output optical signal, completing one matrix-vector multiplication operation. For the matrix-vector multiplication operation of the second weight, the 128-dimensional optical signal is first reduced to an 8-dimensional optical signal, then the 8-dimensional second weight is multiplied by the 8-dimensional optical signal, and finally the 8-dimensional optical signal is increased to a 128-dimensional optical signal, completing the multiplication operation of the second weight matrix with the input vector, outputting a 128-dimensional vector form second output optical signal. Finally, the two vector form output signals are fused to obtain a 128-dimensional vector form comprehensive output signal, completing the coordinated output of two matrix-vector multiplication operations throughout the process.

[0178] In embodiments where both the input and output signals are tensor data, the signal processing method based on an optical computing chip in this application, applied to tensor-matrix multiplication, can first decompose the tensor input signal into multiple vectors, and then transmit them in parallel using the aforementioned multiplexing method to complete the conversion from tensor to multiplexed modulated optical signal. The execution method of vector-matrix multiplication is as described above and will not be repeated here. After obtaining multiple output optical signals, dimensional concatenation can be performed to obtain a high-order tensor synthesized output signal, completing the full tensor operation.

[0179] As can be seen, the signal processing method based on optical computing chips in this application supports both basic vector data and matrix-vector multiplication operations, as well as high-order tensor data and tensor operations, adapting to the inference needs of models ranging from simple linear models to complex large-scale artificial intelligence models, thus significantly improving the method's versatility. In vector operation scenarios, parallel processing in the optical domain enables high-speed matrix-vector multiplication, greatly improving computational efficiency. In tensor operation scenarios, by multiplexing different dimensions of the tensor through wavelength, mode, and time-division multiplexing, parallel processing of high-order tensors is achieved, solving the problems of insufficient bandwidth and low efficiency in existing tensor operations.

[0180] This application also provides an optoelectronic fusion computing system, including the aforementioned signal processing device based on an optical computing chip, optoelectronic hybrid computing device, electrical computing device, and a central control unit; the central control unit is configured to control the signal processing device based on the optical computing chip, the optoelectronic hybrid computing device, and the electrical computing device to perform signal processing individually or jointly.

[0181] The optoelectronic hybrid computing device combines optical domain transmission and electrical domain control, enabling it to handle complex logical operations that are difficult to adapt to with optical computing, and achieving direct combination of the optical and electrical domains. The electrical computing device is a traditional pure electrical domain computing device, capable of handling high-precision logical operations, control command parsing, and other computational tasks. The central control unit is capable of task allocation, device coordination, parameter control, and process scheduling.

[0182] The optoelectronic fusion computing system of this application embodiment can achieve complementary advantages of different computing devices. It leverages the high parallelism, low power consumption, and high bandwidth computing power of signal processing devices based on optical computing chips, while supplementing it with optoelectronic hybrid computing devices. Furthermore, it ensures the accuracy and flexibility of complex logical operations through electrical computing devices. This enables reasonable task allocation and orderly collaboration of devices, avoiding the functional limitations of a single device, and significantly improving the overall computing efficiency and versatility of the system. It can be applied to the inference deployment of ultra-large-scale artificial intelligence models, significantly reducing overall energy consumption while ensuring computing power requirements.

[0183] It should be noted that the elements described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this application will not describe the various possible combinations separately.

[0184] It should be understood that multiple components and / or parts can be provided by a single integrated component or part. Alternatively, a single integrated component or part can be divided into multiple separate components and / or parts. The use of the public designation "a" or "an" to describe a component or part does not exclude other components or parts.

[0185] It should be understood that while terms such as "first" or "second" may be used in this application to describe various elements, these elements are not limited by these terms; these terms are merely used to distinguish one element from another. The terminology used in one or more embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of one or more embodiments of this application. The singular forms "a," "the," and "the" as used in one or more embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used in one or more embodiments of this application refers to and includes any or all possible combinations of one or more associated listed items.

[0186] In this document, terms such as "up," "down," "front," "back," "left," and "right" are used only to indicate the relative positional relationship between related parts, and not to define the absolute position of these related parts. Terms such as "equal" and "same" are not strict mathematical and / or geometric limitations, and also include errors that are understandable to those skilled in the art and permissible in manufacturing or use. Unless otherwise stated, numerical ranges in this document include not only the entire range within its two endpoints, but also several subranges contained therein.

[0187] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0188] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A signal processing device based on an optical computing chip, characterized in that, include: An optical input modulation module (10) is configured to convert an input signal into a modulated optical signal corresponding to the input signal; a first weight mapping module (20) is configured to obtain a first output optical signal corresponding to the modulated optical signal based on a first weight and the modulated optical signal corresponding to the input signal. A second weight mapping module (30) is configured to obtain a second output optical signal corresponding to the modulated optical signal based on a second weight and the modulated optical signal corresponding to the input signal, wherein the first output optical signal and the second output optical signal have the same dimension; an optical output combining module (40) is configured to obtain a combined output signal based on the first output optical signal and the second output optical signal; wherein the second weight mapping module (30) includes: an optical dimensionality reduction unit (31), the optical dimensionality reduction unit (31) being configured to... The modulated optical signal is projected in a low dimension to obtain a dimension-reduced optical signal corresponding to the modulated optical signal; a low-dimensional optical modulation unit (32) is configured to adjust the second weight based on the second weight update signal, and obtain a low-dimensional processed optical signal based on the second weight and the dimension-reduced optical signal; an optical dimension-upgrading unit (33) is configured to project the low-dimensional processed optical signal in a high dimension to obtain a dimension-upgraded optical signal corresponding to the modulated optical signal, and the dimension-upgraded optical signal is the second output optical signal corresponding to the modulated optical signal.

2. The signal processing device based on an optical computing chip according to claim 1, characterized in that, The low-dimensional optical modulation unit (32) includes at least one of a phase modulator, a Mach-Zehnder modulator array, an electro-optic modulator, and a thermo-optic control unit.

3. The signal processing device based on an optical computing chip according to claim 1, characterized in that, The optical dimension reduction unit (31) includes at least one of a multimode interferometer, an arrayed waveguide grating, and a diffractive optical element; and / or, the optical dimension enhancement unit (33) includes at least one of a multimode interferometer, an arrayed waveguide grating, and a diffractive optical element.

4. The signal processing device based on an optical computing chip according to claim 1, characterized in that, The optical dimension reduction unit (31) includes at least one of a phase modulator, a Mach-Zehnder modulator array, an electro-optic modulator, and a thermo-optic control unit; and / or, the optical dimension enhancement unit (33) includes at least one of a phase modulator, a Mach-Zehnder modulator array, an electro-optic modulator, and a thermo-optic control unit.

5. The signal processing device based on an optical computing chip according to claim 1, characterized in that, The first weight mapping module (20) is configured to adjust the first weight based on the first weight update signal, and to obtain the first output optical signal corresponding to the modulated optical signal based on the first weight and the modulated optical signal corresponding to the input signal.

6. The signal processing device based on an optical computing chip according to claim 5, characterized in that, The first weight mapping module (20) includes at least one of Clements interferometer network, Reich interferometer array, cross switch structure, multimode interferometer or programmable diffraction network.

7. The signal processing device based on an optical computing chip according to claim 1, characterized in that, The optical output combining module (40) includes at least one of the following: an interferometric optical combiner, an optical power combiner, a photoelectric detection weighting structure, or a phase-sensitive interferometric structure.

8. The signal processing apparatus based on an optical computing chip according to any one of claims 1 to 7, characterized in that, The second weight mapping module (30) is provided in at least two forms, and the corresponding second weight of each second weight mapping module (30) is different; the signal processing device based on the optical computing chip further includes: a path selection module (50), which is configured to input at least part of the second weight mapping module (30) based on preset conditions to obtain at least one second sub-output optical signal; an output weighting module (60), which is configured to weight and combine the second sub-output optical signals output by each second weight mapping module (30) to obtain a second output optical signal; and an optical output combining module (40) is configured to obtain a comprehensive output signal based on the first output optical signal and the second output optical signal.

9. The signal processing apparatus based on an optical computing chip according to any one of claims 1 to 7, characterized in that, The optical input modulation module (10) is configured to encode the modulated optical signal using an encoding method including pulse amplitude modulation or pulse width modulation, and to multiplex the modulated optical signal using at least one of wavelength multiplexing, mode multiplexing, and time division multiplexing; the optical output combining module (40) is configured to demultiplex the first output optical signal and the second output optical signal, and then obtain a combined output signal based on the demultiplexed first output optical signal and the second output optical signal.

10. The signal processing device based on an optical computing chip according to claim 9, characterized in that, The optical input modulation module (10) includes at least one of an arrayed waveguide grating, a mode demultiplexer, a multimode interference coupler, or a diffraction grating, and is configured to use at least one of the arrayed waveguide grating, the mode demultiplexer, the multimode interference coupler, or the diffraction grating to multiplex the modulated optical signal in a wavelength multiplexing and / or mode multiplexing manner; the optical output combining module (40) includes at least one of an arrayed waveguide grating, a mode demultiplexer, a multimode interference coupler, or a diffraction grating, and is configured to use at least one of the arrayed waveguide grating, the mode demultiplexer, the multimode interference coupler, or the diffraction grating to demultiplex the first output optical signal and the second output optical signal.

11. The signal processing device based on an optical computing chip according to claim 9, characterized in that, The optical input modulation module (10) includes at least one of an optical delay line, a ring optical waveguide, or an optical path folding structure, and is configured to use at least one of the optical delay line, a ring optical waveguide, or an optical path folding structure to multiplex the modulated optical signal in a time-division multiplexing manner; the optical output combining module (40) is configured to use at least one of the optical delay line, a ring optical waveguide, or an optical path folding structure to demultiplex the first output optical signal and the second output optical signal.

12. A signal processing method based on an optical computing chip, characterized in that, include: The input signal is converted into a modulated optical signal corresponding to the input signal; a first output optical signal corresponding to the modulated optical signal is obtained based on a first weight and the modulated optical signal corresponding to the input signal; a second output optical signal corresponding to the modulated optical signal is obtained based on a second weight and the modulated optical signal corresponding to the input signal, wherein the first output optical signal and the second output optical signal have the same dimension; a comprehensive output signal is obtained based on the first output optical signal and the second output optical signal; wherein the step of obtaining the second output optical signal corresponding to the modulated optical signal based on the second weight and the modulated optical signal corresponding to the input signal includes: performing a low-dimensional projection on the modulated optical signal to obtain a dimension-reduced optical signal corresponding to the modulated optical signal; adjusting the second weight based on the second weight update signal, and obtaining a low-dimensional processed optical signal based on the second weight and the dimension-reduced optical signal; performing a high-dimensional projection on the low-dimensional processed optical signal to obtain an up-dimensional optical signal corresponding to the modulated optical signal, wherein the up-dimensional optical signal is the second output optical signal corresponding to the modulated optical signal.

13. The signal processing method based on an optical computing chip according to claim 12, characterized in that, The step of obtaining the first output optical signal corresponding to the modulated optical signal based on the first weight and the modulated optical signal corresponding to the input signal includes: adjusting the first weight based on the first weight update signal, and obtaining the first output optical signal corresponding to the modulated optical signal based on the first weight and the modulated optical signal corresponding to the input signal based on the first weight.

14. The signal processing method based on an optical computing chip according to claim 12, characterized in that, The step of obtaining the second output optical signal corresponding to the modulated optical signal based on the second weight and the modulated optical signal corresponding to the input signal includes: obtaining at least one second sub-output optical signal corresponding to the modulated optical signal based on at least two different second weights and the modulated optical signals corresponding to the input signal; and weighting and combining each of the second sub-output optical signals to obtain the second output optical signal.

15. The signal processing method based on an optical computing chip according to claim 12, characterized in that, The step of converting the input signal into a modulated optical signal corresponding to the input signal includes: converting the input signal into a modulated optical signal corresponding to the input signal, encoding the modulated optical signal with pulse amplitude modulation or pulse width modulation, and multiplexing the modulated optical signal in at least one of wavelength multiplexing, mode multiplexing, and time division multiplexing; the step of obtaining a comprehensive output signal based on the first output optical signal and the second output optical signal includes: demultiplexing the first output optical signal and the second output optical signal, and obtaining a comprehensive output signal based on the demultiplexed first output optical signal and the second output optical signal.

16. The signal processing method based on an optical computing chip according to claim 12, characterized in that, Both the input signal and the combined output signal are vector data; the signal processing method is applied to matrix-vector multiplication; or, both the input signal and the combined output signal are tensor data; the signal processing method is applied to tensor operations.

17. A photoelectric fusion computing system, characterized in that, The device includes a signal processing apparatus based on an optical computing chip, a hybrid optoelectronic computing device, an electrical computing device, and a central control unit as described in any one of claims 1 to 11; the central control unit is configured to control the signal processing apparatus based on an optical computing chip as described in any one of claims 1 to 7, the hybrid optoelectronic computing device, and the electrical computing device to perform signal processing individually or jointly.

Images