A large model computing device and method based on a MZI cascade network

Abstract

A large model computing device and method based on MZI cascade network, relating to the technical field of model computing. The method solves the problem of high energy consumption of the existing Transformer model in operation by combining photonic artificial intelligence chips and FPGA technology. The invention contains a system with three main components: an FPGA control module, an optical computing module, and an FPGA computing module. The FPGA control module is responsible for digital-to-analog conversion and timing input, while the optical computing module is used for weight calculation and feature extraction of data. The FPGA computing module is responsible for all computing tasks except the optical computing module. The main purpose of the invention is to introduce optical computing into the calculation of the Transformer model and realize matrix operation with the help of photonic artificial intelligence chips, thereby reducing the power consumption of the model during operation while maintaining high performance. The combination of the computing flexibility of FPGA and the high speed and efficiency of optical computing makes the optoelectronic hybrid computing a powerful solution for low-energy consumption Transformer model computing.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an optical Transformer implementation method based on MZI cascaded networks. Background Technology

[0002] The Transformer is a neural network architecture that has achieved great success in Natural Language Processing (NLP) and other sequence data processing tasks. Compared to traditional Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs), the Transformer has many unique features and advantages. First, it employs a self-attention mechanism, which can simultaneously consider information from all positions in the input sequence, thus better capturing long-range dependencies. Second, the parallelization of the self-attention mechanism allows the Transformer to efficiently process sequence data, unaffected by the vanishing or exploding gradient problems. These advantages make it outstanding in tasks such as machine translation and text generation. Furthermore, the Transformer is not limited to NLP tasks; it can also be applied to computer vision, audio processing, and other fields. This versatility makes it promising for multimodal tasks as well.

[0003] Before the advent of Transformers, Seq2Seq models were the mainstream method for processing sequential data. These models typically used recurrent neural networks (RNNs) or long short-term memory (LSTM) networks. The foundational work of Transformer technology was the 2017 paper "Attention is All You Need" by Vaswani et al. from the Google Brain team. This paper introduced the concept of self-attention mechanism, which can effectively handle relationships in input sequences, thus achieving better performance in tasks such as machine translation. In 2018, Google released the BERT model, a pre-trained Transformer model. BERT improved the performance of natural language understanding tasks and led the research trend of using pre-trained models for transfer learning. The success of BERT inspired exploration of Transformer technology in various fields. From 2020 onwards, Transformer technology began to be applied to image processing, speech recognition, and other non-linguistic tasks, demonstrating its wide applicability. Transformer technology continues to evolve, and more innovations are likely to emerge in the future, including more efficient model architectures, applications in more fields, and explorations into model interpretability and robustness.

[0004] However, Transformers also have some limitations. First, training Transformer models requires massive amounts of data and high-performance computing resources. These models typically rely on tens of thousands of GPU or TPU cores, running continuously for weeks or even months. Such massive computational demands lead to enormous power consumption. As the model size continues to increase, this energy consumption problem becomes increasingly severe. Second, model training increases the cost of research and development. This means that only large companies or research institutions with ample funding can develop and maintain such models, thus limiting the popularization and innovation of the technology to some extent. Furthermore, once the models are trained, they also require relatively high energy consumption when performing specific tasks. Although this runtime energy consumption is usually much lower than during the training phase, the accumulated energy consumption is still a significant issue when these models are widely used in various online services and applications. Traditional single-data processing computations cannot map all input signals from the real number domain to the positive number domain that light intensity can represent, increasing computation time and power consumption during model runtime.

[0005] The emergence of photonic AI chips can effectively solve this problem, employing optical rather than traditional electronic methods for data processing and computation. This approach offers unique advantages in deep learning, particularly when dealing with complex models such as the Transformer. Photonic AI chips can significantly reduce energy consumption when processing data. This is because optical computation involves fast information transmission and generates minimal heat. In contrast, traditional electronic chips generate substantial heat when processing large amounts of data, increasing the demand for cooling systems and significantly boosting energy consumption. Therefore, applying photonic technology to AI chips, especially when running large models like the Transformer, can effectively reduce energy consumption and reliance on cooling systems. Summary of the Invention

[0006] This application provides a method, apparatus, and device for large model computation based on MZI cascaded networks to solve the problem of excessive energy consumption during Transformer large model computation. It introduces optical computing into the model computation and performs matrix operations through a photonic artificial intelligence chip, thereby reducing the power consumption of the model during operation.

[0007] Firstly, this application provides a method for large-scale model computation based on MZI cascaded networks, which includes the following steps:

[0008] Step 1: The input data is converted from digital signal to analog signal by the DA module. The converted analog signal is then input by the FPGA control module into multiple parallel optical linear computing modules.

[0009] Step 2: After receiving the input, the multiple optical linear calculation modules perform weight calculations and output light intensity. The light intensity signal is converted from an analog signal to a digital signal by the AD module. The FPGA control module sends the digital signal to the FPGA calculation module for self-attention calculation.

[0010] Step 3: The self-attention output calculated by the FPGA computing module is converted into an analog signal by the DA module;

[0011] Step 4: The analog signal converted by the DA module in Step 3 is controlled by the FPGA control module and input into the optical property calculation module to calculate the optical intensity output.

[0012] Step 5 and Step 4's light intensity output are converted into digital signal output by the AD module.

[0013] In one possible implementation, the optical computation module is divided into transforming the input signal, outputting the transformed signal through the photonic artificial intelligence chip, and then performing an inverse transformation relative to the aforementioned transformation before outputting it, thus mapping all input signals from the real number domain to the positive number domain that can be represented by light intensity.

[0014] In one possible implementation, the formula for calculating the self-attention in step 2 is:

[0015] out = QK T

[0016] out' = softmax(out)

[0017] O = Vout'.

[0018] Secondly, this application provides a large-scale model computing device based on an MZI cascaded network, comprising:

[0019] The FPGA control module is used to perform digital-to-analog conversion on the parts that require signal conversion throughout the process, and inputs the timing settings of the signals into the optical computing module. The FPGA control module is also connected to the input signals.

[0020] The optical computing module is used to perform weight calculations on the data and obtain the data features to input into the FPGA computing module.

[0021] The FPGA computing module is used for calculations other than those performed by the optical computing module throughout the process. The FPGA computing module is connected to the FPGA control module and the optical computing module, and the FPGA computing module outputs the calculation results.

[0022] In one possible implementation, the FPGA control module includes a clock module, an AD / DA module, an input / output control system, and an FPGA chip;

[0023] The FPGA chip is connected to the AD / DA module, the input / output control system, and the clock module to provide computing power;

[0024] The AD / DA module is connected to the input / output control system and provides mutual conversion between digital signals and analog signals;

[0025] The clock module is connected to the FPGA chip and provides timing signals;

[0026] The input / output control system is connected to the FPGA chip and the AD / DA module, and is used to perform communication control of the input and output signals of the FPGA chip.

[0027] In one possible implementation, the optical computing module includes at least four photonic linear computing modules, a laser, and a detector. The computing process involves the signal converted by the DA module of the FPGA control module passing through three of the photonic linear computing modules. The output is then converted by the AD module of the FPGA control module and input to the FPGA computing module for self-attention calculation. The calculation result is converted by the DA module controlled by the FPGA and input to the fourth photonic linear computing module. Its output is then converted by the AD module of the FPGA control module and input to the FPGA computing module. The photonic linear transformation computing module includes transformation calculations and inverse transformation calculations provided by the FPGA computing module, and a photonic artificial intelligence chip.

[0028] In one possible implementation, the laser is connected to a photonic AI chip, providing a continuous light source input to the chip. An FPGA control module is also connected to the photonic AI chip, which modulates the laser input based on data sent by the FPGA control module. The photonic AI chip consists of a cascaded array of Mach-Zehnder interferometers, and its modulation process is as follows:

[0029] Step 1: Adjust the modulator according to the voltage signal sent by the FPGA control module. The modulator will modulate the input signal onto the pulse signal input to the laser;

[0030] Step 2: Based on the neural network weight information sent by the FPGA control module, the modulator in the Mach-Zehnder interferometer cascade array in each photonic AI chip is adjusted by voltage to modulate the weight information onto the light pulse.

[0031] Step 3: In the optical domain, perform a multiplication operation on the input signal and the weighting signal modulated on the optical pulse.

[0032] The large model computation method, apparatus, and device based on MZI cascaded networks provided in this application introduce optical computing into the model computation and realize matrix operations through photonic artificial intelligence chips, thereby reducing the power consumption of the model during operation. By introducing optical computing into the computation of Transformer models and realizing matrix operations with the help of photonic artificial intelligence chips, the power consumption of the model during operation is reduced while maintaining high performance. This architecture not only solves the energy consumption problem, but also combines the computing flexibility of FPGA and the high speed and high efficiency of optical computing, making optoelectronic hybrid computing a powerful solution for realizing low-power Transformer model computation. Attached Figure Description

[0033] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0034] Figure 1 The main components and connection methods of the optical Transformer neural network system architecture provided in this application.

[0035] Figure 2 The main computational component in the optical Transformer neural network system architecture provided in this application.

[0036] Figure 3 The composition of the photonic linear computing module provided in this application.

[0037] Figure 4 This application provides a diagram illustrating the process of modulating input and weight information onto light.

[0038] Figure 5 The diagram shows the FPGA control module and FPGA computing module provided in this application.

[0039] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0040] The technical solutions and hardware structures of the embodiments of the present invention will be fully described below with reference to the accompanying drawings. Unless otherwise specifically stated, the relative arrangement of the components and steps described in these embodiments does not limit the scope of the present invention. To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0041] Based on photonic artificial intelligence chips, this invention proposes an optical Transformer neural network system architecture that can reduce the energy consumption of Transformer model operation, thereby significantly reducing the cost of training and running large Transformer-based models.

[0042] See attached document Figure 1 This system comprises an FPGA control module, an optical computing module, and an FPGA computing module, which work together to process data. Input first passes through the FPGA control module, then is processed by the optical computing module. During this process, the FPGA module performs necessary nonlinear and linear computational operations.

[0043] See attached document Figure 2 The key computational components for implementing the Transformer model include the DA / AD module, the linear computation module, and the FPGA computation module. The main steps in its implementation are as follows:

[0044] Step 1: The input data is converted from digital signals to analog signals by the DA module. These converted analog signals are then input by the FPGA control module into three parallel optical linear computing modules.

[0045] Step 2: After receiving the input, the three optical linear calculation modules perform weight calculations and output light intensity. These light intensity signals are converted from analog signals to digital signals by the AD module. The FPGA control module sends these digital signals to the FPGA calculation module for self-attention calculation.

[0046] Step 3: The self-attention output calculated by the FPGA computing module is converted into an analog signal by the DA module.

[0047] Step 4: The analog signal converted by the DA module in Step 3 is controlled by the FPGA control module and input into the fourth optical property calculation module to calculate the optical intensity output.

[0048] Step 5: The light intensity output mentioned in step 4 is converted into a digital signal output by the AD module.

[0049] The above steps are the key computational steps for implementing the Transformer model. During this process, the calculations of the ray scattering module are referenced in the appendix. Figure 3 The process can be divided into two parts: transforming the input signal, outputting the transformed signal through the photonic AI chip, and then performing an inverse transformation on the output signal. Since the input to the photonic AI chip is the light intensity value, and the input signal may include negative values, this transformation maps all input signals from the real number domain to the positive number domain that light intensity can represent.

[0050] See attached document Figure 4 The laser is connected to the photonic AI chip, providing a continuous light source input. The FPGA control module is also connected to the photonic AI chip, which modulates the laser input based on data sent from the FPGA control module. The photonic AI chip consists of a cascaded array of Mach-Zehnder interferometers, and its modulation process is as follows:

[0051] Step 1: Adjust the modulator according to the voltage signal sent by the FPGA control module. The modulator will modulate the input signal onto the pulse signal input to the laser.

[0052] Step 2: Based on the neural network weight information sent by the FPGA control module, the modulators in the cascaded array of Mach-Zehnder interferometers in each photonic AI chip are adjusted by voltage to modulate the weight information onto the light pulse.

[0053] Step 3: In the optical domain, perform a multiplication operation on the input signal and the weighting signal modulated on the optical pulse.

[0054] Using FPGAs for optoelectronic hybrid computing offers several advantages, particularly in handling high-performance computing tasks such as deep learning and big data analytics. FPGAs combine the flexibility of traditional electronic computing with the speed and efficiency of optical computing, making optoelectronic hybrid computing a promising research and application area.

[0055] The structure of the FPGA's computing and control modules can be referenced in the appendix. Figure 5 The FPGA control module, used to realize real-time high-speed communication between the optical computing module and the multi-task neural network module, and to realize real-time weight updates of the optical computing module, includes: an FPGA chip, an AD / DA module, an input / output control system, and a clock module. The functions of the FPGA computing module include: providing basic multiplication, summation, nonlinear calculations, and linear transformation calculations, providing computational functions for the FPGA chip.

[0056] The FPGA chip mentioned above is connected to the AD / DA module, input / output control system, and clock module to provide computing power;

[0057] The AD / DA module is connected to the input / output control system, providing mutual conversion between digital and analog signals.

[0058] The clock module is connected to the FPGA chip and provides timing signals;

[0059] The input / output control system is connected to the FPGA chip and the AD / DA module, and is used to perform communication control of the input and output signals of the FPGA chip.

[0060] Based on the above description, the main objective of this invention is to introduce optical computing into the computation of Transformer models, using photonic AI chips to perform matrix operations, thereby reducing power consumption during model operation while maintaining high performance. The system comprises three components: an FPGA control module, an optical computing module, and an FPGA computing module. The FPGA control module is responsible for signal analog-to-digital conversion and timing input, while the optical computing module is used for data weight calculation and feature extraction. The FPGA computing module handles all computational tasks except for the optical computing module. This method combines photonic AI chips and FPGA technology, leveraging the computational flexibility of FPGAs and the high speed and efficiency of optical computing to make optoelectronic hybrid computing a powerful solution for achieving low-power Transformer model computation. This application introduces optical computing into model computation, uses photonic AI chips to perform matrix operations, and solves the problem of excessive power consumption during the computation of large Transformer models, thereby reducing power consumption during model operation, achieving the objective of this invention, and reaching the desired technical effect.

Claims

1. A large model calculation method based on a MZI cascade network, characterized by The large model calculation device based on the MZI cascade network comprises: The FPGA control module is used for digital-to-analog conversion of parts needing signal conversion in the whole process, and inputting the timing of the signal setting into the optical computing module, and the FPGA control module is connected with the input signal; The optical computing module is used for weight calculation of data, and feature input of the data is obtained into the FPGA computing module; The FPGA computing module is connected with the FPGA control module and the optical computing module, and the FPGA computing module outputs the calculation result; The method comprises the following steps: Step 1, the input data is converted from digital signal to analog signal by the DA module, and the converted analog signal is input into multiple parallel optical linear computing modules by the FPGA control module; Step 2, the multiple optical linear computing modules perform weight calculation after receiving the input, and output light intensity, the light intensity signal is converted from analog signal to digital signal by the AD module, and the FPGA control module sends the digital signal to the FPGA computing module for self-attention calculation; Step 3, the self-attention output calculated by the FPGA computing module is converted into analog signal by the DA module; Step 4, the analog signal converted by the DA module in step 3 is input into the optical linear computing module by the FPGA control module to obtain the light intensity output after calculation; Step 5, the light intensity output in step 4 is converted into digital signal output by the AD module; The optical computing module comprises at least four optical linear computing modules, a laser and a detector, and the calculation process is that the signal converted by the DA module of the FPGA control module passes through three optical linear computing modules, the output is input into the FPGA computing module for self-attention calculation after being converted by the AD module of the FPGA control module, the calculation result is input into the fourth optical linear computing module after being converted by the DA module of the FPGA control module, and the output is input into the FPGA computing module after being converted by the AD module of the FPGA control module.

2. The large model calculation method based on the MZI cascade network according to claim 1, characterized in that, The FPGA control module comprises a clock system, an AD / DA conversion system, an input / output control system and an FPGA chip.

3. The large model calculation method based on the MZI cascade network according to claim 1, characterized in that, The optical linear computing module comprises transformation calculation and inverse transformation calculation provided by the FPGA computing module, a photonic artificial intelligence chip, a laser connected with the photonic artificial intelligence chip, and the FPGA control module connected with the photonic artificial intelligence chip.

4. The large model calculation method based on the MZI cascade network according to claim 3, characterized in that, The photonic artificial intelligence chip is composed of a Mach-Zehnder interferometer cascade array.

5. The large model calculation method based on the MZI cascade network according to claim 1, characterized in that, The formula of the self-attention calculation method in step 2 is: 。 6. The large model calculation method based on the MZI cascade network according to claim 1, characterized in that, The FPGA computing module is used for providing the calculation function of the FPGA chip, including providing basic multiplication, summation, nonlinear calculation and linear transformation calculation.

7. The large model calculation method based on the MZI cascade network according to claim 4, characterized in that, The modulation process is: Step 1, according to the voltage signal sent by the FPGA control module, the modulator is adjusted, and the modulator will modulate the input signal into the pulse signal input into the laser; Step 2, according to the neural network weight information sent by the FPGA control module, the modulator in the Mach-Zehnder interferometer cascade array in each photonic artificial intelligence chip is adjusted by voltage, and the weight information is modulated onto the optical pulse; Step 3, in the optical domain, the input signal and the weight signal modulated on the optical pulse are multiplied.

Images