Interferometer-free hybrid linear algebra optical processing unit

Abstract

Hybrid Linear Algebra Optical Processing Unit without Interferometer: An optical processing unit for performing tensor multiplication on a first tensor value and a second tensor value. The optical processing unit includes: - a first converter configured to convert the value of the first tensor into a first analog signal, - a first logarithmic amplifier to convert the first analog signal into a second analog signal representing the logarithm of the first tensor value, - a first modulator and a first light source to convert the second analog signal into a first light beam, - a second converter configured to convert the value of the second tensor into a third analog signal, - a second logarithmic amplifier to convert the third analog signal into a fourth analog signal representing the logarithm of the second tensor value.- a second modulator and a second light source to convert the fourth analog signal into a second light beam, and - an optical combiner to add the first light beam with the second light beam to obtain a resulting light beam representing the logarithm of the value of the first tensor multiplied by the value of the second tensor, or the logarithm of the value of the first tensor added to the logarithm of the value of the second tensor.

Description

Title of the invention: Hybrid linear algebra optical processing unit without interferometer Background

[0001] The current rate of growth in computing and energy demands for artificial intelligence (including machine learning and inference) is unsustainable. Some predict that by the end of the decade, artificial intelligence (“AI”) data centers could consume up to 20–25 percent of the United States’ energy needs (https: / / www.wsj.com / tech / ai / artificial-intelligences-insatiable-energy-needs-not-sustainable-arm-ceo-says-all218c9). After an estimated total consumption of 460 terawatt-hours (TWh) in 2022, total data center electricity consumption could reach over 1,000 TWh by 2026. This electricity demand is equivalent to the electricity consumption of Japan. International Energy Agency “IEA”, Electricity 2024, Analysis and Forecast to 2026, pg. 8.

[0002] OpenAI has prepared a report predicting that AI training far exceeds the computing power of modern computers. Since 2012, the amount of computing power used in the largest AI training cycles has increased exponentially, with a doubling time of 3.4 months. In contrast, Moore's Law describes a doubling time of two years. The computing power used for AI training has increased more than 300,000-fold since 2012, a rate of increase that far exceeds the prediction of Moore's Law, which forecasts a sevenfold increase over the same period. Neural Internet, Future of AI: Compute is King: $2 trillion market boom and computational revolution, March 9, 2024; see also OpenAI, Research AI and Compute, May 16, 2018.

[0003] The exponential growth of floating-point operations used to train AI models since 2018, as shown in [Fig.1], gives a graphical overview of the increasing energy demand created by machine learning (https: / / ourworldindata.org / grapher / artificial-intelligence-training-computation?time=2009-06-15..latest).

[0004] Another risk lies in the expansion of data centers and the resulting increase in energy and water consumption. Large companies like Microsoft (which is reportedly considering a $100 billion data center project with OpenAI called "Stargate") are beginning to consider attaching new energy sources such as small modular nuclear reactors to their data centers, and work is underway to find less energy-intensive alternatives to existing AI infrastructure. But resistance to these data centers has Increased across the board, from Ireland to West Virginia, in recent years, and the cascading demands of new AI models are only amplifying this resistance. See Fortune.com, The cost of training AI could soon become too much to bear. April 4, 2024.

[0005] Over time, the demand for computing resources for training will only increase with the accelerating demand for investment in AI and the computing resources needed to train increasingly complex and sophisticated AI models. These problems are compounded by the difficulty of obtaining the advanced chips needed to perform these training operations. For example, there are critical shortages of advanced chips, such as Nvidia's H100 and A100, which are essential for training, https: / / www.wired.com / story / nvidia-chip-shortages-leave-ai-startups-scrambling-for-computing-power / .

[0006] In addition to computing and energy requirements, training AI models also entails significant environmental impacts. A 2019 study identified an environmental cost "due to the carbon footprint required by modern tensor processing hardware." The environmental cost of training an NLP model is five times the CO2 emissions typically associated with driving a car over its lifetime. [Emma Strubell et al., Energy and Policy Considerations for Deep Learning in NLP, 57th Annual Meeting of ACL, Florence, Italy, (July 2019)]. See also, International Energy Agency "IEA", Electricity 2024, Analysis and Forecast to 2026.

[0007] The rapid growth of the AI ​​market and the proportional demand for computing resources are forcing the industry to consider new and more efficient approaches to overcome the limitations of Moore's Law, reduce environmental impacts, and increase the concentration of available AI resources held by only a few market players. Furthermore, training AI models requires significant capital. For example, training GPT-4 cost over $100 million, and Mistral, a Paris-based competitor of OpenAI, raised $400 million to train its models. Many companies do not use machine learning because they cannot outsource training due to confidentiality concerns, and training the models themselves is too expensive.Reducing training costs through more efficient computing methods is essential to enabling more businesses, academic and research institutions, and others to develop their own machine learning models and further democratize AI. https: / / sifted.eu / articles / mistral-releases-first-ai-model, https: / / www.bloomberg.com / news / articles / 2023-12-04 / openai-rival-mistral-nears-2-billion-valuation-with-nvidia-funding?sref=gni836kR.

[0008] Tensors (e.g., vectors or matrices) play a crucial role in machine learning and inference. In machine learning and inference, tensors are used to represent input values ​​and weights that serve to link elements together in the neural network. In machine learning, steps called forward and backward passing or forward and backward propagation are used to train the weights by performing numerous gradient descent calculations using linear algebra. These operations require many tensor product operations. Depending on the amount of data used in the machine learning process, several million, or even several trillion, tensor products may be required to complete the learning process.Tensor multiplication is the most important bottleneck operation in machine learning and inference. The more efficient this step is, the faster and more energy-efficient the learning process. Performing tensor multiplications on a matrix (which is a 2D tensor) of size (N, N) involves NA3 operations, whereas other operations in neural networks involve NA2. Since N is typically between 1,000 and 1,000,000, it is clear that tensor multiplications are very resource-intensive.

[0009] The basic step in tensor products is to multiply the data and weights in the neural network. Figure 2 represents the neural network and the computation of a single data point X and a single weight W to obtain an output O. This same representation can be applied with multiple data inputs X1 and X2 and weights W1 and W2. Instead of performing a single computation, as shown in Figure 2, there are three computations, as shown in Figure 3. Since the tensors are 2x2 vectors, only two products and a sum are needed to compute the output. Figure 3 represents the computations for this single-layer neural network.

[0010] The two neural networks described so far have only one output. In practice, neural networks have many more outputs. Adding additional outputs requires more weights to link the outputs to the inputs. If there are two outputs, the number of weights will be increased by two for a total of four, as shown in [Fig. 4]. The tensor products described so far correspond to a vector-vector multiplication ([Fig. 2]) and a vector-matrix multiplication ([Fig. 4]).

[0011] Rather than trying to overcome current computing limitations with traditional processors and electronic computing, simply by injecting more computing resources into the problem—which would entail all the problems already mentioned—a different approach should be explored. In the past, light has proven very effective for digital communications. To increase the To increase bandwidth and reduce the energy costs of communications, optical fibers are used instead of electrical wires. Optical data links have replaced copper wires for long-distance communication lines and shorter spans, all the way to bay-to-bay communications in data centers. Optical data communications are much faster and require less energy. Ryan Hamerly, "Future of Deep Learning Is Photonics," IEEE Spectrum (June 29, 2021). The use of optics (in the form of photonic processors) to improve computational efficiency for inference is also promising. Photonic processors have been presented as a way to use light rather than digital processors to perform the linear algebra calculations required for tensor products.

[0012] The energy requirements of current processors are due to performance limitations when scaling the number of tensor inputs. Photonic processors do not have this limitation. Therefore, photonics is particularly well suited to meet the massive AI demand for low-cost, high-efficiency computing (https: / / techhq.com / 2023 / 05 / what-is-optical-computing-explained / #:~:text=The%20history%20of%20optical%20computing,arrays%20of%20semiconductor%20smart%20pixels). Early attempts to use photonics to improve the efficiency of basic tensor products were developed in the 1980s. For example, see Collins, 4,569,033. In this approach, the intensity or phase of the laser beams is modulated by the different cells of an SLM (spatial light modulator). The modulation that each cell applies to the beam depends on the electrical signal addressed.Vectors (or the rows or columns of a matrix) are encoded in arrays of one-dimensional SLM cells, and a dyadic product of two vectors is performed by passing the beams through the modulators encoding each vector. See [Fig. 6] (Collins figures) and [Fig. 7] (generalized spatial modulator from the 1980s). However, these early attempts are not suitable for larger tensor multiplication operations due to the slow refresh rate of these SLM cells. Consequently, they do not operate at frequencies high enough to meet current computing requirements.

[0013] Photonic processors for tensor multiplication have been developed in an attempt to meet the computational and energy requirements of today's artificial intelligence processes. While these implementations have improved the efficiency of tensor calculations, they have several major drawbacks that limit their performance and efficiency. The key operations of current photonic processors involve multiplications and accumulations, which include Multiplication of pairs of numbers in vectors and matrices and addition of the results. The most common way to do this today is by multiplying light beams using Mach-Zehnder interferometers (MZIs). While the original principle of the MZI was developed in the 1890s, current implementations are integrated into chips. Typically, the MZI splits the incoming light into two beams, each traveling along a different path. See [Fig. 8]. The two resulting beams are then recombined into a single beam. If the two paths are identical, the output is identical to the input. However, if one of the two beams travels farther than the other or slows down, it is out of phase with the other beam. The intensity or amplitude of the output beam can be affected by the phase difference between the two beams.If there is no phase shift between the two beams, the intensity of the output beam is the same as that of the input beam. But if the phase shift is 180 degrees, the two beams interfere destructively when recombined, resulting in no output. The amplitude / intensity of the output beam will be the amplitude of the input beam multiplied by the cosine of half the phase difference. By controlling the phase difference between the two beams, this multiplication can be achieved. David Schneider, Lightmatter's Mars Chip Performs Neural-Network Calculations at the Speed ​​of Light, IEEE Spectrum (August 29, 2020). The value of the multiplier determined by the phase difference is essentially the "weight" used in the neural network.

[0014] Although MZIs are used for forward or forward passes, they are not well suited for machine learning. This is because their resolution and accuracy are limited for tensor computations. When multiple forward and backward passes occur for training, error propagation also occurs. Training requires a very high dynamic range, especially compared to inference. See David Schneider, ibid. It is also difficult to scale systems using MZIs to larger matrices—they are currently limited to 64x64 matrices. Sunil Pai et al.Experimentally realized in situ backpropagation for deep leaming in nanophotonic neural networks, Science, (April, 27, 2023) ("We measured backpropagation gradients for phase change voltages by interfering forward and backpropagation light and simulated in situ backpropagation for 64-port photonic neural networks").

[0015] Since the forward and backward passes involve a chain of matrix multiplications, error propagation using an MZI in the networks can lead to inaccuracies in large matrix multiplications (significant errors for sizes greater than 64 by 64) in inference operations as well as in machine learning. Unacceptable error propagation This occurs for large matrices, both for the forward and backward passes. Thus, for matrices larger than 64 by 64, these photonic processors do not perform well for inference or machine learning.

[0016] There is a critical need to improve the energy efficiency of AI computing and to mitigate the previously mentioned negative impacts on GPUs / TPUs, including current photonic processor architectures. Many of the limitations of current photonic processor designs stem from the use of MZIs (and similar interferometers). A photonic processor design and implementation that can operate without MZIs could be used to improve accuracy for training and inference purposes and also allow matrix multiplications to extend well beyond 64 by 64 while using less energy and having less environmental impact. Summary

[0017] Certain aspects of this disclosure relate to performing tensor multiplication operations without requiring phase shifting or an interferometer. The embodiments of this disclosure relate to an optical processing unit and a method for performing tensor multiplication. Figure 5 graphically compares the power requirements of current GPUs / TPUs with those of the optical processing unit (GPU) proposed by this disclosure.

[0018] Certain aspects of this disclosure relate to an optical processing unit that performs a tensor multiplication on a value of a first tensor and a value of a second tensor. The optical processing unit includes a first converter configured to convert the value of the first tensor into a first analog signal, a first logarithmic amplifier to convert the first analog signal into a second analog signal that represents the logarithm of the value of the first tensor, and a first modulator and a first laser to convert the second analog signal into a first laser beam.The optical processing unit also contains a second converter configured to convert the value of the second tensor into a third analog signal, a second logarithmic amplifier to convert the third analog signal into a fourth analog signal representing the logarithm of the value of the second tensor, and a second modulator and a second laser to convert the fourth analog signal into a second laser beam. Certain aspects of this disclosure also relate to an optical combiner for adding the first laser beam to the second laser beam to obtain a laser beam representing the logarithm of the value of the first tensor multiplied by the value of the second tensor, or the logarithm of the value of the first tensor added to the logarithm of the value of the second tensor.

[0019] Certain aspects of this disclosure relate to a method for optically performing tensor multiplication on a value of a first tensor and a value of a second tensor. In some embodiments, the method comprises the following steps: converting the value of the first tensor into a first analog signal, processing the first analog signal through a first logarithmic amplifier to obtain a second analog signal that represents the logarithm of the value of the first tensor, and modulating a first laser with the second analog signal to produce a first laser beam.In some embodiments, the method also includes the following steps: converting the value of the second tensor into a third analog signal, processing the third analog signal through a second logarithmic amplifier to obtain a fourth analog signal representing the logarithm of the value of the second matrix, and modulating a second laser with the fourth analog signal to produce a second laser beam. In some embodiments, the method may also include the step of optically combining the first and second laser beams to obtain a laser beam representing the logarithm of the value of the first tensor multiplied by the value of the second tensor, or the logarithm of the value of the first tensor added to the logarithm of the value of the second tensor. Brief description of the figures

[0020] Figure 1 illustrates the growth in energy consumption for training modern machine learning models between 2012 and 2024.

[0021] Fig. 2 illustrates the representation of a neural network composed of an input (x) and a weight (w) to obtain an output and the representation of the equation, which is equivalent to a scalar-scalar multiplication.

[0022] Fig. 3 illustrates the representation of a single-layer neural network where there are two inputs (x) and two weights (w) to obtain one output and the representation of the equation, which is equivalent to a vector-vector multiplication.

[0023] Fig. 4 illustrates the representation of a neural network composed of two inputs (x) and four weights (w) to obtain two outputs, as well as the representation of the equation which is equivalent to a vector-matrix multiplication.

[0024] Figure 5 compares the energy consumed for matrix multiplication as a function of matrix size for modern GPUs and TPUs with respect to a photonic processor implemented according to an embodiment of the present disclosure.

[0025] The [Fig.6] is a first attempt to use photonics to improve the multiplication of vectors and matrices, which makes it possible to obtain a matrix.

[0026] Figure 7 illustrates a generalized vector-matrix multiplication using a spatial modulator.

[0027] Figure 8 shows an example of a simple Mach-Zehnder interferometer, an example of a way to control the phase difference between two light beams.

[0028] Fig. 9 is a diagrammatic representation detailing the steps of an example of scalar-scalar multiplication according to an embodiment of the present disclosure.

[0029] Fig. 10 shows an example of an embodiment of the components of a photonic processor for scalar-scalar multiplication used on a clock cycle.

[0030] Fig. 11 is a flowchart detailing the steps of an example of vector-vector multiplication according to an embodiment of the present disclosure.

[0031] Fig. 12 shows an example of an embodiment of the components of a photonic processor for an example of vector-vector multiplication used during a clock cycle.

[0032] The [Fig. 13] is a logic diagram detailing the steps of a vector matrix multiplication over two clock cycles, according to an embodiment of the present disclosure.

[0033] Fig. 14 shows an example of components of a photonic processor for multiplying a vector matrix over one clock cycle.

[0034] Fig. 15 represents another example of components of a photonic processor for the multiplication of a vector matrix.

[0035] Fig. 16 is a flowchart detailing the steps of the present disclosure for a matrix-matrix multiplication.

[0036] Fig. 17 shows an exemplary realization of the present disclosure for a matrix-matrix multiplication used on a clock cycle.

[0037] Fig. 18 illustrates different representations of a matrix-matrix multiplication, in the form of an equation, a clock-cycle neural network and a hardware representation of the same multiplication according to an embodiment of the present disclosure.

[0038] Figure 19 shows a logic diagram showing how signed values ​​are taken into account in tensor-tensor multiplications in one embodiment of the present disclosure.

[0039] Figure 20 shows a flow diagram of the multiplication of two tensors with signed values ​​according to an embodiment of the present disclosure.

[0040] Figure 21 shows an example of an embodiment of the components of the present disclosure for the multiplication of two tensors with signed values ​​over one clock cycle.

[0041] Figure 22 shows a flow diagram of the multiplication of two vectors with signed values ​​over two clock cycles using a photonic processor according to an embodiment of the present disclosure.

[0042] Figure 23 shows an exemplary variant of the components of the present disclosure for the multiplication of two vectors with signed values.

[0043] 24 shows a flow diagram of the multiplication of a tensor in representation binary to obtain a result with 16-bit precision.

[0044] Figure 25 shows a flow diagram of the vector-vector multiplication represented in bits to obtain a result with a precision of 16 bits.

[0045] Figure 26 represents a diagram of a linear amplifier circuit.

[0046] Figure 27 represents a diagram of an exponential amplifier circuit.

[0047] Figure 28 represents a diagram of a logarithmic amplifier circuit.

[0048] Figure [Fig. 29] represents a diagram of an integration circuit.

[0049] Fig. 30 shows the architecture of a vertical cavity surface emitting laser (VCSEL).

[0050] Fig. 31 illustrates a non-limiting set of examples of light sources that could be used in a possible embodiment: surface-emitting photonic crystal lasers (PCSELs), distributed feedback / edge-emitting lasers (EELs), and light-emitting diodes.

[0051] Fig. 32 is a cross-sectional view illustrating an embodiment of the present disclosure which includes a transmissive optical component providing a distribution (or fan) function.

[0052] Fig. 33 is a cross-sectional view illustrating an embodiment of the present disclosure which includes a reflective optical component fulfilling a distribution function which reflects light beams towards transducers located on the same substrate as the light sources.

[0053] Fig. 34 is a cross-sectional view illustrating an embodiment of the present disclosure which includes a transmissive optical component fulfilling a distribution function and a mirror for reflecting light beams to transducers located on the same substrate as the light sources.

[0054] Figure [Fig. 35] represents a non-limiting diagram of a vector-vector multiplication.

[0055] Figure [Fig. 36] represents a non-limiting diagram of a vector-matrix multiplication.

[0056] Figure [Fig. 37] represents a non-limiting diagram of a matrix-matrix multiplication. Detailed description

[0057] 1. Scalar-scalar multiplication

[0058] In the background section of [Fig. 2], we presented the simplest neural network involving a data point and a weight, or X*W=0. Figures 9 and 10 show the steps for implementing this simple neural network using embodiments of this disclosure. [Fig. 9] is a flowchart detailing the steps of one embodiment of this disclosure, and [Fig. 10] illustrates a model embodiment including the components for implementing one embodiment of this disclosure. In the first step 901, illustrated in [Fig. 9], the input data X (a binary number) is converted into an analog signal, analogx. In [Fig. 10], the model implementation of this step 901 is carried out using a readily available digital-to-analog converter 1002, or "DAC".In this exemplary embodiment, the input weight W (a binary number) is simultaneously or concurrently converted into an analog signal analogw by another digital-to-analog converter 1007. However, another implementation could allow serial operations over one or more clock cycles for the conversion of X and W into analog signals using the same DAC. In this document, when operations are described as being performed simultaneously, they may be performed by separate hardware components operating in parallel. These operations may be time-aligned (e.g., have the same start and end times) or time-shifted (e.g., have different start and / or end times), and the different operations performed by different hardware components may take different times (e.g., different numbers of clock cycles) to complete.

[0059] In the following steps 902 and 906 illustrated in Figures 9 and 10, the analog signals αx and αw are simultaneously converted into log X and log W using logarithmic amplifiers 1003 and 1008. The logarithmic amplifiers 1003 and 1008 illustrated in [Fig. 10] are described in more detail in [Fig. 28]. This logarithmic amplifier could be the amplifier illustrated and described in [Fig. 28]. Again, another approach could be to implement serial operations over additional clock cycles using a single logarithmic amplifier.

[0060] In the following steps 903 and 907 illustrated in [Fig. 9], log X and log W are simultaneously fed into two modulators 1004 and 1009 illustrated in [Fig. 10] to control the light sources referred to respectively as laserx and laserw. Although various embodiments of the present disclosure are described herein in examples using lasers as light sources, the embodiments of the present disclosure are not limited to these. In one embodiment A variety of light sources or emitters can be used, including, but not limited to, light-emitting diodes (LEDs), lasers, etc. See Figures 30, 31, and 32 and the corresponding discussion below for examples of such potential light sources. Thus, when the term "laser" is used in this document, it should be understood that the embodiments include alternatives using other types of light sources, such as LEDs, and when the term "laser beam" is used in this document, the embodiments include light beams that are not laser beams. In one possible embodiment, lasers are used. These lasers can be of various types, but in one possible embodiment, they are vertical cavity surface-emitting lasers, or "VCSELs," as shown in [Fig. 30].The output of the laserx 1005 is a laser beam with a power of Px 904 and the output of the laserw 1010 is a laser beam with a power of Pw908. However, another embodiment could consist of implementing serial operations over one or more clock cycles using only a single modulator and a single laser.

[0061] In the next step 909 illustrated in [Fig.9], the lasersx and laserw are combined with each other. The combination produces a third laser beam having the power Px + Pw 910 which represents the value of log X + log W or log (XW).

[0062] In the next step shown in Figures 9 and 10, a photodiode 1011 is used to convert the third laser into a voltage 911. In other embodiments, other components may be used to convert the optical signal into an analog electrical signal. This voltage is then used in step 912 as the input to an antilogarithmic or exponential amplifier 1012, as shown in Figures 9 and 10. This antilogarithmic amplifier 1012 may be the amplifier illustrated and described in more detail in [Fig. 27]. The output is the product of X and W or XW. This output XW is an analog signal and, in the next step 913 shown in Figures 9 and 10, this signal is converted into a digital signal using an analog-to-digital converter 1013 or "ADC".

[0063] 2. Vector-vector multiplication

[0064] Figure 35 gives a representative, non-limiting example of a vector-vector multiplication.

[0065] In [Fig. 3] (previously discussed in the background section), we also presented a neural network representing a vector-vector multiplication. As mentioned earlier, instead of performing only one calculation, as shown in [Fig. 2], [Fig. 3] presents three. Since the matrices are vectors of size 1x2 and 2x1, the three calculations for the output are two products and one sum, or XiW\ + X2W2.

[0066] Figures 11 and 12 illustrate the steps and an exemplary embodiment of this disclosure performing a vector-vector multiplication, or a vector-vector multiplication, or a vector-vector multiplication.

[0067] [Math.2] 1 «« ) ■ ( ■ Wj -i- æ» ■ «!•>

[0068] Figure 11 shows the operational steps for multiplying the vector XiX2 (a first vector) by a second vector WiW2. The steps performed in the shaded area, steps 1101 to 1112 and 1113 to 1124 of Figure 11 for XiX2, are the same as those described in Figure 9, steps 901 to 914. Similarly, the components for carrying out the steps in Figure 12 (1201 to 1205 and 1206 to 1210) are the same. The only component in Figure 12 not shown in Figure 10 is the integrator, Figure 12, 1213, for performing the accumulation in Figure 11, 1125. The operation of the accumulator / integrator will be discussed below.

[0069] In one possible embodiment, the steps of the XixWi multiplication, as shown in [Fig. 11], take place in parallel, steps 1101 to 1104 and 1105 to 1108, just as in [Fig. 9], steps 901 to 904 and 905 to 908, as previously discussed. These steps can be performed in a single clock cycle. However, other embodiments could include serial operations in order to conserve the number of components required to perform these operations. See the discussion of Figures 9 and 10 concerning another embodiment.

[0070] The steps of the X2xW2 operation in [Fig. 11] are the same as those in [Fig. 9] for the Xi xW operation. Similarly, the components 1201 to 1212 shown in [Fig. 12] for performing the X2xW2 operation are the same as those shown in [Fig. 10], 1001 to 1011. In one possible embodiment, a set of components is used (as shown in [Fig. 12]) to perform the vector multiplications for XixWi and for X2xW2. These operations 1101 to 1112 can be performed serially in two clock cycles (as shown in [Fig. 11]) to conserve the number of components (as shown in [Fig. 12]) so that, for example, one set of components is sufficient. It is possible to use fewer components if they are serialized as described above in relation to [Fig. 10]. It is also possible to use two sets of components illustrated in [Fig.12], 1201 to 1112, to allow parallel operations to take place in a single clock cycle instead of . two clock cycles for all or some of the steps illustrated in [Fig.1 1], 1101 to 1112 and 1113 to 1124.

[0071] Once the steps, illustrated in Figures 11, 1101 to 1112 and 1113 to 1124, and in Figures 12, 1201 to 1212, have been carried out, the analog signals representing XiW1 (output during clock cycle 1) and X2W2 (output during clock cycle 2) can be added in step 1125, [Fig. 11], by an integrator illustrated in [Fig. 12] to 1213, which accumulates the analog signals representing XiW1 and X2W2 over the two clock cycles, [Fig. 11] by an integrator represented in [Fig. 12] to 1213 which accumulates the analog signals representing XiW1 and X2W2 over the two clock cycles. Integrator 1213 can be the one shown and discussed later with regard to [Fig. 29]. The output of the integrator is an analog signal representing XiWi + X2 W2 as shown in step 1126 of [Fig. 11]. In step 1127, [Fig.

[11] , this analog signal is converted into a digital signal representing XiWi + X2W2 by an analog-to-digital converter or "ADC", component 1214, [Fig.12]. .

[0072] 3. Vector-matrix multiplication

[0073] Figure 36 gives a representative, non-limiting example of a vector-matrix multiplication.

[0074] Fig. 13 shows the steps for performing a vector-matrix multiplication, or

[0075] [Math.4] / Ai 1 '®12 I / / , , / , \ ( æi ) • = ( (Kj (“1 -^12 + ¾-^23) ) according to one embodiment of this disclosure. More specifically, Figure 13 shows the operational steps of the multiplication of XiX2 (a vector) and Wn Wi2, W2i W22 (a 2x2 matrix). The steps performed in the shaded area 1301 to 1312 and 1323 to 1334 of Figure 13 are substantially the same as those described in Figures 9 and 11, steps 901 to 908 and 1101 to 1108 and 1113 to 1120.

[0076] In addition, the components for carrying out the shaded steps of figures 1301 to 1312 and 1323 to 1334 are illustrated in [Fig. 14], 1401 to 1415. These components are also the same as those in [Fig. 12], 1201 to 1210.

[0077] In one possible embodiment, the execution steps of the multiplications Xi xWn, XixW[2] and the multiplications X2xW2[, X2xW22, as shown in the shaded area of ​​[Fig. 13], 1301 to 1312, take place in a first clock cycle, and steps 1323 to 1334 take place in a second clock cycle. As previously stated, these steps can take place in a single clock cycle (Figures 9 and 11) or, as shown in [Fig. 13], they can include serial operations in order to conserve the number of components required to perform these operations. See the discussion relating to figures 9 and 10 and figures 11 and 12 concerning other embodiments.

[0078] The steps for performing the shaded operations for XixWn, XixWn and the operations for X2xW2i, X2xW22 in [Fig. 13] (steps 1301 to 1312 and 1323 to 1334) are the same as in [Fig. 11] for performing the operations XixWn and XixW[2, (steps 1101 to 1108 and 1113 to 1120). Similarly, the components (1401 to 1415) shown in [Fig. 14] for performing the operations in the shaded area of ​​[Fig. 13] are the same as those shown in [Fig. 12], 1201 to 1210. In one possible embodiment, a single set of components is used (as shown in [Fig. 14]) to perform the multiplications. These operations can be performed serially over two clock cycles to conserve the number of components, so that a set of components 1401 to 1415 is sufficient. It is possible to use fewer components if they are serialized as described above in relation to [Fig. 10].It is also possible to use two sets of components illustrated in [Fig. 14] (1401 to 1415) to allow parallel operations in the same clock cycle for all or part of steps 1301 to 1334.

[0079] In [Fig. 14], the laser beam of the 1405 laser is split in two by a distribution procedure made possible by a diffractive optical element. For examples of diffractive optical elements and how they can be used and where they can be obtained from vendors such as Coherent, https: / / www.coherent.com / optics / general-optics / diffractive-optics / splitters and Holo / Or Ltd., https: / / www.holoor.co.il / structured-light-doe / . Although the distribution procedure or optical fan function is illustrated in [Fig. 14] as being implemented using a diffractive optical element or "DOE," the embodiments in this disclosure are not limited to this. Within the scope of this disclosure, other optical components that implement a distribution procedure or optical function may be substituted for the diffractive optical element.Examples of these other optical components include a grating, a beam splitter and / or a meta-lens designed to implement a distribution function.

[0080] It is possible to obtain additional parallelism for the multiplication operations performed in [Fig. 14]. This can be generalized to a matrix with more than two columns. In this case, the laser beam for the laser Xi in [Fig. 14] is split by the diffractive optical element into N laser beams, where N is the number of columns in the matrix. In [Fig. 15], clock cycle 1, for example, shows the parallel operations performed to obtain X₁xW₂ and X₁xW₂. In the following clock cycle, parallel operations can be performed to obtain X₂xW₂ and X₂xW₂.

[0081] Another embodiment may not include an optical fan and may add additional components to encode the vector value in a plurality of additional lasers. This approach would increase the number of amplifiers, modulators, and logarithmic lasers to four sets instead of three as shown in this example.

[0082] Once the steps 1313 to 1320 and 1335 to 1342 of [Fig. 13] have been carried out with the components of [Fig. 14] 1416 & 1417, and 1422 & 1423, the analog signals representing XiWn and X2W2i can be added in steps 1321 & 1322, and 1343 & 1344 of [Fig. 13] by the integrators shown in [Fig. 14] at 1418 & 1424. Integrator 1424 is the same as that of [Fig. 12] 1213, and that of [Fig. 29]. The output of the integrator is an analog signal representing XiWn+X2W2i as shown in [Fig. 13]. This analog signal is then converted into a digital signal representing XiWn+X2W2i by an analog-to-digital converter or "ADC" represented by components 1419 and 1425 in [Fig. 14]. Similar to XiWn, the analog signals representing XiWi2 and X2W22 can be added together, and the output converted into a digital signal to obtain XiW12 + X2W22.

[0083] This process and its components can be scaled accordingly for any vector with more than two inputs and any matrix with more than two rows. In general, the number K times the steps are executed in the shaded area of ​​[Fig. 13] corresponds either to the number of inputs in the vector or to the number of rows in the matrix. This means that, in one possible embodiment, the number of clock cycles used to implement the vector-matrix multiplication depends on the number of inputs in the vector or rows in the matrix. It is also possible to perform all the operations in parallel during the same clock cycle using K sets of components.

[0084] With reference to [Fig. 32], in certain embodiments of this disclosure, the laser [Fig. 14], 1405, the diffractive optical elements, and the photodiodes [Fig. 14], 1416 & 1422 are stacked vertically, perpendicular to the support plane, such that the laser or emitters are at the lower level and the photodiodes or transducers at the upper level, or vice versa. With reference to Figures 33 and 34, in other embodiments, the diffractive optical elements are reflective or combined with reflective optical elements, so that the optical signals are reflected. A diffraction grating may be used for this purpose. In these embodiments, the lasers and photodiodes are integrated on the same plane.

[0085] 4. Matrix-matrix multiplication

[0086] Figure 37 gives a representative, non-limiting example of matrix-matrix multiplication. The approach taken in the example for matrix-matrix multiplication is called dyadic product decomposition. As Figure 37 shows, the product of two matrices X and W can be written as the sum of the outer products of the columns of the first matrix with the rows of the second matrix.

[0087] A dyadic product decomposition is a linear algebra operation which takes into account two vectors (a first vector from the first matrix and a second vector from the second matrix) and produces a matrix, where the components of the matrix are the products of two elements, each pair of elements containing a value from the first vector and a value from the second vector.

[0088] As shown in [Fig.37], each value of the first vector must be multiplied by each value of the second vector and vice versa, so that the output matrix contains all possible combinations of an element of the first vector and an element of the second vector.

[0089] In practice, we use what is called a "double optical fan", where each value of the first vector is distributed with each value of the second vector, and where each value of the second vector is distributed with each value of the first vector.

[0090] The matrices resulting from the external products are added together by accumulating the values ​​as shown in [Fig.37] (one accumulation for each input of the output matrix).

[0091] Figure 16 shows the steps for multiplying one matrix by another matrix, or

[0092] [Math.6] | (He •U?-)'] 1-¾ (w «>» I ( (¾ ■ wji r ? » • w) /

[0093] according to an embodiment of the present disclosure. More specifically, [Fig. 16] shows the operational steps for multiplying together ((XnXi2), (X2iX22)) (a first 2x2 matrix) and ((WnWi2), (W2iW22)) (a second 2x2 matrix). The steps performed in the shaded region (steps 1601 to 1640, [Fig. 16]) are the same as those described in Figures 9, 11 and 13, steps 901 to 908, 1101 to 1108 and 1113 to 1120, 1301 to 1312 and 1323 to 1334.

[0094] Similarly, the components for carrying out the shaded steps of [Fig. 16], steps 1601 to 1640, are shown in [Fig. 17], 1701 to 1720 are also the same as those shown in figures 12 and 14, 1201 to 1210 and 1401 to 1415.

[0095] In one possible embodiment, the execution steps of the multiplications Xn xWn , XnxWi2 , X2ixWn , X2ixWi2 and the multiplications Xi2xW2b Xi2xW22 , X22xW2b As shown in the shaded area of ​​[Fig. 16], steps 1601 to 1620, X22xW22, run in parallel with steps 1621 to 1640, as in Figures 9, 11, and 13, examined previously. These steps can be completed in a single clock cycle or, as shown in [Fig. 16], they can include serial operations to conserve the number of components required to perform them. See the discussion of Figures 9 and 10, Figures 11 and 12, and Figures 13 and 14 for other embodiments.

[0096] The steps for performing the shaded operations for XnxWn, XnxW12, X21xWn, X21xWi2 and the operations for X12xW2b, X12xW22, X22xW2j, X22xW22 in [Fig. 16], steps 1601 to 1620, are identical to those in [Fig. 13] for performing the operations XixWn and XixWi2, steps 1301 to 1312. Similarly, the components shown in [Fig. 17] for performing the operations in the shaded area of ​​[Fig. 16] are the same as those shown in [Fig. 14], 1401 to 1415. In one possible embodiment, a single set of components is used (as shown in [Fig. 17]) to perform the multiplications. These operations can be performed serially over two clock cycles to conserve the number of components, so that, for example, a set of components 1401 to 1405 is sufficient. It is possible to use fewer components if they are serialized as described above in relation to [Fig. 10].It is also possible to use two sets of components illustrated in [Fig. 17], 1701 to 1720, to allow parallel operations to occur in the same clock cycle for all or some of the steps 1721 to 1740.

[0097] In [Fig. 17], the light sources or emitters are lasers that produce laser beams split into two beams by a distribution procedure made possible by a diffractive optical element, as discussed above. The optical fan of the two lasers can be obtained using separate diffractive optical elements or by using a common diffractive optical element. Similarly, the laser beams of the Wn and Wi2 lasers are split in two using a common diffractive optical element or separate elements for each laser beam.

[0098] The steps are generalized for matrices with a size greater than two. In this case, the laser beams corresponding to the column values ​​of the first matrix (in [Fig. 16], the values ​​Xn and X2i) are each split into N laser beams using diffractive optical elements, N being the number of columns in the second matrix. See [Fig. 18] for lasers and an optical fan, producing N = 2 laser beams for each of the values ​​Xn and X2i during a first clock cycle, and Xn and X2i during a second clock cycle. This can be done using one or more common optical elements shared by several laser beams. (For example, a common optical element shared by all laser beams or k different optical elements shared by N / k laser beams where k is a natural number, as well as embodiments where different optical elements interact with different numbers of laser or light beams) or separate optical elements for each laser beam. Similarly, the laser beams corresponding to the row values ​​of the second matrix (in [Fig. 16], the values ​​Wn and W[2]) are split into M laser beams using different diffractive optical elements, M being the number of rows in the first matrix. See [Fig. 18] for lasers and an optical array producing M=2 laser beams for each of the values ​​Wn and W[2] in a first clock cycle, and W2i and W22 in a second clock cycle.Once again, this optical fan can be obtained by a common diffractive optical element or by separate elements for each beam. The details concerning the diffractive optical elements are the same as those discussed previously. The effect of this optical fan is to produce a dyadic product of a column of the first matrix with a row of the second matrix, thus combining all possible pairs of beams encoding a value from the column of the first matrix and a value from the row of the second matrix.

[0099] In the lower part of [Fig. 18], clock cycle 1 shows the parallel operations performed to obtain XnxWn and XnxWi2 for the first element of the batch and X2ixWn and X2[xWi2 for the second element of the batch. During the following clock cycle, clock cycle 2, parallel operations can be performed to obtain Xi2xW2i and X[2xW22 for the first element of the batch, and X22xW2[ and X22xW22 for the second element of the batch. Another solution would be to omit the optical fan and add additional components to encode the matrix values ​​in a plurality of additional lasers. This approach would increase the number of amplifiers, modulators, and logarithmic lasers to eight sets instead of four in this example.

[0100] Once the steps 1601 to 1640 of [Fig. 16] have been executed as shown in [Fig. 17] 1701 to 1740 with the components of [Fig. 17] 1701 to 1740, the analog signals representing XnxW h and X[2xW2i] can be added in step 1673 of [Fig. 16] by an integrator 1723 shown in [Fig. 17]. An example of an integrator implementation is illustrated and discussed in more detail in [Fig. 29]. The output of the integrator is, for this example, an analog signal representing XnWn + X[2W2i] as shown in step 1674 of [Fig. 16]. In [Fig. 17], this analog signal is converted into a digital signal representing XnWn + X[2W2i] by an analog-to-digital converter or "ADC" shown in component 1724, [Fig. 17]. 17]. Similarly, the analog signals representing XnWn and Xi2W22 are also converted by a 1730 ADC. The signals analog signals representing X2iWn and X22W2i and analog signals representing X21W12 and X22W22 can be added together and their outputs converted into digital signals as shown in Figures 17, 1735 and 1744.

[0101] This process and its components can be generalized to larger matrices where the number of columns in the first matrix and the number of rows in the second matrix are greater than two. The number of times the steps indicated in the shaded area of ​​[Fig. 16] are executed (K) corresponds to either the number of columns in the first matrix or the number of rows in the second matrix. This means that the number of clock cycles required to perform matrix-matrix multiplications is equal to K. Alternatively, all operations can be performed in the same clock cycle using K sets of the same components in parallel.

[0102] As in the implementation of vector-matrix multiplication according to certain aspects of the embodiments of this disclosure, several geometries are to be considered. With reference to Figures 32 and 33, we can use transmissive or reflective optical elements. With regard to [Fig. 32], in certain embodiments of this disclosure, the lasers of Figures 17, 1705, 1710, 1715 and 1720, the diffractive optical elements and the photodiodes of Figures 17, 1721, 1727, 1733 and 1739 are stacked vertically, in a direction perpendicular to the support plane, such that the laser or emitters are at the lower level and the photodiodes or transducers at the upper level, or vice versa. With reference to [Fig.

[33] In other embodiments, the diffractive optical elements are reflective or combined with reflective optical elements, so that the optical signals are reflected. A diffraction grating can be used for this purpose. In these embodiments, the lasers and photodiodes are integrated on the same plane.

[0103] 5. Implementation of negative signs

[0104] Figure 19 shows a flowchart describing how negative values ​​are taken into account in tensor-tensor multiplications according to certain aspects of the embodiments of this disclosure. Since some machine learning algorithms for training and inference use data values ​​that can take positive and negative values, it is necessary to be able to support a varied range of model parameters and values ​​to support these machine learning algorithms.

[0105] Encoding negative values ​​with laser beam intensities is impractical because intensity measurements are inherently positive. To account for negative data values ​​ranging from -1 to 1 (as an example) using optical intensity, these values ​​are first fitted to a new range starting at 0 and going up to 2 in one possible embodiment. This fitted range is then set to the scale to correspond to the maximum possible intensity level, called "maximum intensity". This maximum intensity can be determined based on trade-offs such as power consumption, heat dissipation and the dynamic range of optical devices, such as non-linearities in the behavior of light sources (e.g., laser sources or light-emitting diodes), the behavior of amplifiers (e.g., input and output ranges for which amplifiers exhibit logarithmic and antilogarithmic behavior), and the saturation of photodiodes.

[0106] Since all optically processed values ​​can only be positive, the analog hardware discussed above, according to certain embodiments of this disclosure, performs what are called unsigned multiplications. In other words, this means that in one possible embodiment according to the above disclosure, the multiplication hardware only processes positive numbers.

[0107] In one embodiment of the disclosure, if the inputs 2501 and 2503, x and y, are between -1 and 1, they are adjusted to positive values ​​by adding 1 to each of them, 2502 and 2504. The product of x and y is then obtained by first calculating the product of the adjusted values, then subtracting the sum of the adjusted values ​​from the product, and finally adding 1 to this result. See steps 2507 to 2514, [Fig. 19].

[0108] The steps in [Fig. 19] are given assuming that the range of possible values ​​for the inputs x and y is between -1 and 1. However, this is arbitrary and admits a direct generalization. If the input x has a value between -Vmax and Vmax and the input y has a value between -Vmax and Vmax, the fitted values ​​are obtained by adding Vmax to x and y respectively. Other generalizations (for example, when the ranges of x and y are different) can also be applied in other circumstances.

[0109] To extend this principle to tensor multiplication with vectors and matrices containing several values, each between -1 and 1, each value is adjusted in the same way by adding 1. The dot product of these vectors is then obtained by performing the following operations. First, the product of the adjusted values ​​is calculated, the sum of the adjusted values ​​is subtracted from this product, and 1 is added to the final result. See steps 2507 to 2514, [Fig. 19]. These operations can be performed before or after the summation carried out by integrators 1213 [Fig. 12], 1418 & 1424 [Fig. 14], 1518 & 1524 [Fig. 15], 1723, 1729, 1735 & 1741 [Fig. 17], and 2316 [Fig. 23] in the case of vector-vector, vector-matrix, or matrix-matrix multiplications. In a numerical embodiment, these operations are performed directly on the accumulated sums, the sums first being converted to numerical values ​​using of one or more analog-to-digital converters. In other embodiments where the implementation is analog, these operations can be performed either on the analog signals representing the accumulated sums, or on the product of the fitted values ​​and the sum of the fitted values ​​before the integrator. In this latter case, where the operations are performed before the integrator, a plurality of implementations are possible, which are discussed below.

[0110] An embodiment of the present disclosure for multiplying two tensors with signed values ​​according to the process detailed above, and in which the operations specific to the signed multiplication are performed in the analog domain before the integrator, comprises two sets of amplifiers, modulators, and lasers. Figure 20 shows a logic diagram of the operations according to an embodiment of the present disclosure following this approach, and Figure 21 shows the implementation of the components according to an embodiment of the present disclosure following this approach. Amplifiers 2106, 2116 [Fig. 21] of one set of components comprise logarithmic amplifiers, while amplifiers 2103, 2113 [Fig. 21] of the other set comprise linear amplifiers.Linear amplifiers convert one analog signal into another analog signal, where the relationship between the input and output signals is linear, with a positive or negative proportionality coefficient, with or without an offset value, for an appropriate range of the input signal value. A typical linear amplifier is described and illustrated in more detail in [Fig. 26]. In another embodiment, a linear amplifier can be omitted (e.g., replaced by a wire or a conductive trace) in a well-calibrated implementation.

[0111] In other aspects of this disclosure, steps 2001 to 2019 of [Fig. 20] are similar to those described in [Fig. 9], [Fig. 11], [Fig. 13] and [Fig. 16]. The analog signals resulting from the measurement of the combined beams, including the laser beams from the first and second sets of components, are routed respectively and in parallel into an exponential amplifier (steps 2015 and 2019, [Fig. 20]) and (component 2110, [Fig. 21]) and into a linear amplifier (2120, [Fig. 21]). The two analog signals are then provided as inputs to an analog subtractor (2121, [Fig.2], step 2020, [Fig.20]), which has the effect of producing another analog signal representing the subtraction of the sum of the input tensor values ​​from the product of the input tensor values.The addition of a constant (e.g., 1) is also performed in this component, so that the resulting signal encodes the result of the signed multiplication. In other respects, steps 2001 to 2019 of [Fig. 20] are similar to those described in Figures 9, 11, 13, and 16.

[0112] In another embodiment illustrated by Figures 22 and 23, the operations specific to signed multiplication are performed using a single set of components over two clock cycles, relying on the 2316 integrator circuit to perform both subtraction and addition (as shown in Figures 20 and 21). Figure 22 shows a logic diagram of the operations of the present disclosure following this approach, and Figure 23 shows the implementation of the components of the present disclosure following this approach. In this embodiment, the steps illustrated in Figure 22 are similar to those in Figure 20.In addition to steps 2201 to 2209, steps 2210 to 2221 are added, including digital-to-analog converters 2302 & 2309, modulators 2306 & 2313, and lasers 2307 & 2314 representing the input tensor values, an optical combiner, and a converter 2315 to produce the sum of the two input tensor values, multiplied by a negative sign. In other respects, steps 2201 to 2209 of [Fig. 22] are similar to those described in Figures 9, 11, 13, and 16. Linear amplifiers 2303 & 2310 are added to the implementation of [Fig. 22]. 12], in parallel with the logarithmic amplifiers 2304 & 2311. A two-state selector 2305 & 2312 or another component of similar functionality is used to connect the modulators to the linear amplifiers 2303, 2310 or to the logarithmic amplifiers 2304, 2311.Branches 2305 and 2312, connected to modulators 2306 and 2313, are selected according to the chosen operating mode, as shown in [Fig. 23]. Similarly, a linear amplifier 2317 is added in parallel with the exponential amplifier 2316, and a selector switch 2317 allows the output of only one of the amplifiers to be connected to the integrator 2316, as shown in [Fig. 23]. In this embodiment, the implementation operates according to a chosen configuration, or mode. In one mode, which we call "multiplication mode," the selectors connect the logarithmic amplifiers 2304 and 2311 to the modulators 2306 and 2313, and the exponential amplifier 2316 to the integrator. In another mode, which we call "addition mode", the selectors connect the linear amplifiers 2303 & 2310 to the modulators 2306 & 2313 and the linear amplifier 2317 to the integrator 2316. In multiplication mode, the integrator accumulates the product of the input tensor values.In addition mode, the integrator accumulates the negative sum of the input tensor values. Thus, the execution of operations in both modes in successive clock cycles with the same inputs, along with the accumulation of a constant offset, is used to obtain the result of the signed multiplication.

[0113] Other embodiments of the methods and implementations described above may use different conventions to perform the multiplication of values signed, which results in different ranges for signed and unsigned values ​​and additional scaling factors.

[0114] 6. Tensor multiplication with high output accuracy

[0115] The following variants, illustrated by Figures 24 and 25, present two approaches and implementations for adding further precision to tensor multiplications performed in accordance with one of the variants of this disclosure.

[0116] Multiplication of a tensor on an analog device produces inaccuracies due to the limited read resolution and finite signal-to-noise ratio of analog signals. For example, an exact multiplication of two 8-bit positive integers gives a 16-bit result. However, it is possible that noise may render, for example, the first 8 bits of the 16 bits reliable (the least significant bits having a high probability of being incorrect) and the following 8 bits unreliable. Using inputs with larger intervals (for example, only 4 bits of effective precision) reduces the number of bits in the result of the multiplication and also reduces the probability of errors occurring.

[0117] The Karatsuba algorithm is a well-known algorithm used to obtain a greater number of bits of precision for a multiplication of two values, even if the multiplier hardware cannot accommodate the bits of precision. See https: / / www.researchgate.net / publication / 234346907_Multiplication_of_Multidigit_Numbers_on_Automata, See also https: / / en.wikipedia.org / wiki / Karatsuba algorithm

[0118] With reference to [Fig. 24], the 8-bit integer values ​​are split into two 4-bit representations or words for each of the input values. A series of partial multiplications is then performed, and the results are accumulated to produce a larger output.

[0119] Each of the boxes 2409, 2412, 2415, and 2418 marked with an X in [Fig. 24] indicates a tensor multiplication operation. Each tensor multiplication can be implemented by the embodiments of this disclosure examined herein. For example, for vector-vector multiplication illustrated in Figures 11 and 12, each of the four multiplications 2409, 2412, 2415, and 2418 marked with an X is performed by the steps and components described with reference to those figures. Similarly, this approach to multiplication can be applied to the operations and components of vector-matrix and matrix-matrix multiplication according to the embodiments of this disclosure.

[0120] Figure 25 shows another way of performing precision multiplication. By adding positive integers, the effective bit width is increased, which allows for more accurate results. When a sum is performed Between partial products and aggregation, this means that lower-resolution vector inputs must be used to obtain an exact analog vector-vector product. An implementation for two four-element vectors, for example XI, X2, X3, and X4, and W1, W2, W3, and W4, where each value contains 6 bits, has an exact representation of the result with 14 bits. This can be achieved using the process in [Fig. 25], where the multiplication units 2509, 2514, 2519, and 2524, denoted by X, are the system and methods described herein for vector-vector, vector-matrix, and matrix-matrix multiplication, and where the read precision is 8 bits.

[0121] Various circuit structures for photonic processors are described in this document with respect to various embodiments of this disclosure. These circuit structures can be represented using digital information that can be stored on a non-transient computing medium, such as flash memory (e.g., a solid-state drive), a hard disk drive, etc. These digital representations may include, for example, embodiments of this disclosure described in a hardware description language (HDL) such as Verilog and VHDL, analog circuit models described, for example, as SPICE (Simulation Program with Integrated Circuit Emphasis) models, and others, and in various forms suitable for integration into a module or subcircuit of an integrated circuit design.Digital representations may also include, for example, files representing integrated circuit (IC) designs, such as GDSII stream format files that may be provided to a foundry for the manufacture of an integrated circuit implementing photonic processors according to the embodiments of this disclosure.

[0122] According to one embodiment, an optical processing unit for performing tensor multiplication on a value of a first tensor and a value of a second tensor comprises: a first converter configured to convert the value of the first tensor into a first analog signal, a first logarithmic amplifier to convert the first analog signal into a second analog signal representing the logarithm of the value of the first tensor, a first modulator and a first light source to convert the second analog signal into a first light beam, a second converter configured to convert the value of the second tensor into a third analog signal, a second logarithmic amplifier to convert the third analog signal into a fourth analog signal representing the logarithm of the value of the second tensor,a second modulator and a second light source to convert the fourth analog signal into a second light beam, and an optical combiner to add the first light beam to the second light beam, in order to obtain a light beam representing the logarithm of the value of the first tensor multiplied by the value of the second tensor or the logarithm of the value of the first tensor added to the logarithm of the value of the second tensor.

[0123] The optical processing unit may also include a transducer to convert the resulting light beam into a fifth analog signal.

[0124] The transducer may include a photodiode.

[0125] The optical processing unit may also include an antilogarithmic amplifier to convert the fifth analog signal into a sixth analog signal that represents the antilogarithm or exponential of the fifth analog signal.

[0126] The optical processing unit may also include a third converter configured to take the sixth analog signal and convert it into a digital signal.

[0127] The first light source and the second light source may respectively comprise a first vertical cavity surface emitting laser (VCSEL) and a second VCSEL.

[0128] The first tensor and the second tensor can respectively comprise a first vector and a second vector, the optical processing unit performing a vector-vector multiplication.

[0129] The first tensor may include a vector and the second tensor may include a matrix to perform a vector-matrix multiplication.

[0130] The first tensor can include a matrix and the second tensor can include a matrix to perform a matrix-matrix multiplication.

[0131] The first light beam or the second light beam can be divided or distributed by one or more diffractive elements in order to obtain a plurality of light beams.

[0132] The first light beam and the second light beam can be divided or distributed by one or more diffractive elements in order to obtain a plurality of light beams.

[0133] The diffractive element(s) may be transmissive and located vertically between the light sources and one or more photodiodes to convert the plurality of light beams into analog signals.

[0134] The diffractive elements can be reflective and the first light source and the second light source can be located on the same substrate as one or more photodiodes to convert the plurality of light beams into analog signals.

[0135] The optical processing unit may also include: a) a first linear amplifier and a second linear amplifier for converting the first a) a third analog signal and a third analog signal into a seventh analog signal and an eighth analog signal which represent the values ​​of the first tensor and the second tensor, and b) a third modulator and a fourth modulator and a third light source and a fourth light source to convert the seventh analog signal and the eighth analog signal into a third beam of light and a fourth beam of light.

[0136] The optical processing unit may further include: a third modulator and a fourth modulator, as well as a third light source and a fourth light source to convert the seventh analog signal and the eighth analog signal into a third beam of light and a fourth beam of light.

[0137] The optical processing unit may also include a second optical combiner to add the third light beam with the fourth light beam in order to obtain a second light beam representing the value of the first tensor added to the value of the second tensor.

[0138] The optical processing unit may also include a second transducer to convert the resulting second light beam into a ninth analog signal that represents the value of the first tensor added to the value of the second tensor.

[0139] The second transducer can be a photodiode.

[0140] The optical processing unit may further include a) a third linear amplifier to convert the ninth analog signal into a tenth analog signal, b) a subtractor to take the difference between the sixth analog signal and the tenth analog signal in order to obtain an eleventh analog signal, and c) an analog-to-digital converter to convert the eleventh analog signal into a digital signal.

[0141] The optical processing unit may also include an analog-to-digital converter to convert the eleventh analog signal into a digital signal.

[0142] The optical processing unit may also include a subtractor which takes the difference between the sixth analog signal and the tenth analog signal and adds the integer value 1 to obtain an eleventh analog signal.

[0143] The optical processing unit for performing a tensor-tensor multiplication in which the value of the first tensor and the second tensor can be represented by a plurality of bits and the first binary representation of the tensor and the second binary representation of the tensor can be divided into at least two or more words.

[0144] The optical processing unit may also include at least one adder for aggregating two or more partial products into a resulting multiplication product.

[0145] According to one embodiment of the present disclosure, an optical method for performing tensor multiplication on a value of a first tensor and a value of a second tensor comprises: converting the value of the first tensor into a first analog signal, processing the first analog signal through a first logarithmic amplifier to obtain a second analog signal representing the logarithm of the value of the first tensor, modulating a first light source with the second analog signal to produce a first light beam, converting the value of the second tensor into a third analog signal, processing the third analog signal through a second logarithmic amplifier to obtain a fourth analog signal representing the logarithm of the value of the second matrix, and modulating a second light source with the fourth analog signal to produce a second light beam.and optically combine the first light beam and the second light beam to obtain a resulting light beam representing the logarithm of the value of the first tensor multiplied by the value of the second tensor, or the logarithm of the value of the first tensor added to the logarithm of the value of the second tensor.

[0146] The method may also include the step of using a transducer to convert the resulting light beam into a fifth analog signal.

[0147] The transducer may include a photodiode.

[0148] The method may also include the step of converting the fifth analog signal into a sixth analog signal representing the antilogarithm of the fifth analog signal.

[0149] The step of converting the fifth analog signal into a sixth signal may include the use of an antilogarithmic amplifier.

[0150] The modulation steps of the first light source and the second light source may include the modulation steps of the first and second vertical cavity surface-emitting lasers (VCSELs).

[0151] The first tensor and the second tensor for performing the tensor-tensor multiplication respectively may comprise a first vector and a second vector for performing the vector-vector multiplication.

[0152] The first tensor and the second tensor for performing the tensor-tensor multiplication can respectively comprise a vector and a matrix for performing the vector-matrix multiplication.

[0153] The first tensor and the second tensor for performing the tensor-tensor multiplication respectively may comprise a first matrix and a second matrix for performing the matrix-matrix multiplication.

[0154] The production steps of the first light beam or the second light beam may further include the step of distribution by one or more diffractive elements in order to provide a plurality of light beams.

[0155] The production steps of the first light beam and the second light beam may further include the step of distribution by one or more diffractive elements in order to provide a plurality of light beams.

[0156] The method may also include the following steps: a) converting the first analog signal and the third analog signal using a first linear amplifier and a second linear amplifier into a seventh analog signal and an eighth analog signal which respectively represent the values ​​of the first tensor and the second tensor, and b) converting the seventh analog signal and the eighth analog signal into, respectively, a third light beam and a fourth light beam using a third modulator and a fourth modulator and a third light beam and a fourth light beam.

[0157] The method may also include the step of converting the seventh analog signal and the eighth analog signal into a third light beam and a fourth light beam using a third modulator and a fourth modulator and a third light source and a fourth light source.

[0158] The method may also include the step of adding the third light beam to the fourth light beam in order to obtain a second light beam representing the value of the first tensor added to the value of the second tensor using a second optical combiner.

[0159] The method may also include the step of using a second transducer to convert the light beam into a ninth analog signal representing the value of the first tensor added to the value of the second tensor.

[0160] The transducer can be a photodiode.

[0161] The method may also include the following steps: a) using a third linear amplifier to convert the ninth analog signal into a tenth analog signal, b) using a subtractor to take the difference between the sixth analog signal and the tenth analog signal and add a positive constant value to obtain an eleventh analog signal, and c) using an analog-to-digital converter to convert the eleventh analog signal into a digital signal.

[0162] The method may further comprise the following steps: a) using a subtractor to take the difference between the sixth analog signal and the tenth analog signal and adding the integer value 1 to obtain an eleventh signal analog, and b) use of an analog-to-digital converter to convert the eleventh analog signal into a digital signal.

[0163] Tensor multiplication method in which the value of the first tensor and the second tensor can be represented by a plurality of bits and the first binary representation of the tensor and the second binary representation of the tensor can be divided into at least two or more words.

[0164] The method may also include the steps of using at least one adder to aggregate at least two or more partial products into a resulting multiplication product.

[0165] Certain aspects of this disclosure have been described in various embodiments, but are not limited to them. For example, all components of the disclosure, including those mentioned in the glossary, can be implemented as separate components on a motherboard, integrated into a silicon chip, and / or into firmware. Furthermore, steps performed in possible embodiments in more than one clock cycle can be performed in parallel operations in fewer clock cycles, including a single clock cycle, by adding additional and duplicated components to the possible embodiments. Adding clock cycles to the operation of the possible embodiments allows for the use of fewer components.Those with relevant expertise will recognize that a number of further modifications and improvements can be made to the disclosure without departing from its essential spirit and scope. Glossary

[0166] Linear Amplifier: A linear amplifier is a type of inverting amplifier that uses a resistor across the inverting terminal and another across the feedback loop to produce an output proportional to the input but at a higher power. The equation representing this can be shown below, where Vi is the input voltage of the circuit, Vo is the output voltage, Rf is the resistance across the feedback loop, Ri is the resistance across the inverting terminal, and Is is the saturation current.

[0167] Exponential Amplifier: An exponential amplifier is a type of inverting amplifier that uses a diode on the inversion terminal to produce an output proportional to the exponential of the diode's input. The equation representing this can be shown below, where Vi is the input voltage of the circuit, VT is the equivalent thermal voltage of the diode, Rf is the resistance, and Is is the saturation current.

[0168] Logarithmic Amplifier: A logarithmic amplifier is a type of inverting amplifier in which the feedback loop is not regulated by a resistor but replaced by a diode; the output is proportional to the natural logarithm of the input. The equation that can be represented below is as follows: Vi is the input voltage, Is is the saturation current, Ri is the resistance connected to the inverting terminal, VT is the equivalent thermal voltage of the diode, and n is the ideality parameter of the diode.

[0169] Integrator: An integrating circuit is a type of inverting amplifier in which the feedback loop is not regulated by a resistor but replaced by a capacitor. This capacitor accumulates the charges from the resistor connected to the inverting terminal of the amplifier and acts as a time integrator. The output is then equal to the time integral of the input voltage multiplied by the frequency of the RC circuit. The equation can be represented below, where Vt is the input voltage, t is time, R is the resistance, and C is the capacitance.

[0170] VCSEL: A vertical cavity surface-emitting laser is a semiconductor laser, more specifically a laser diode with a monolithic laser resonator, where the emitted light leaves the device in a direction perpendicular to the chip surface (https: / / www.rp-photonics.com / vertical_cavity_surface_emitting_lasers.html). For example, a non-limiting series of examples of light sources that could be used are: surface-emitting photonic crystal lasers (PCSELs), distributed feedback / edge-emitting lasers, light-emitting diodes, and others.

[0171] In addition, other light sources could be used, such as LEDs and others.

Claims

Demands

1. An optical processing unit for performing tensor multiplication on a value of a first tensor and a value of a second tensor, the optical processing unit comprising: a first converter configured to convert the value of the first tensor into a first analog signal, a first logarithmic amplifier for converting the first analog signal into a second analog signal representing the logarithm of the value of the first tensor, a first modulator and a first light source for converting the second analog signal into a first light beam, a second converter configured to convert the value of the second tensor into a third analog signal, a second logarithmic amplifier for converting the third analog signal into a fourth analog signal representing the logarithm of the value of the second tensor,a second modulator and a second light source to convert the fourth analog signal into a second light beam, and an optical combiner to add the first light beam with the second light beam to obtain a resulting light beam representing the logarithm of the value of the first tensor multiplied by the value of the second tensor, or the logarithm of the value of the first tensor added to the logarithm of the value of the second tensor.

2. Optical processing unit according to claim 1 further comprising a transducer for converting the resulting light beam into a fifth analog signal.

3. Optical processing unit according to claim 2 further comprising an antilogarithmic amplifier for converting the fifth analog signal into a sixth analog signal which represents the antilogarithm or exponential of the fifth analog signal.

4. Optical processing unit according to claim 3 further comprising a third converter configured to take the sixth analog signal and convert it into a digital signal.

5. Optical processing unit according to claim 1, wherein the first light source and the second light source comprise respectively a first vertical cavity surface emitting laser (VCSEL) and a second VCSEL.

6. Optical processing unit according to claim 1, wherein the first light beam or the second light beam is distributed by one or more diffractive elements to provide a plurality of light beams.

7. Optical processing unit according to claim 6, wherein one or more diffractive elements are transmissive and located vertically between the first and second light source and one or more photodiodes for converting the plurality of light beams into analog signals.

8. Optical processing unit according to claim 6, wherein one or more diffractive elements are reflective and the first light source and the second light source are located on the same substrate as one or more photodiodes for converting the plurality of light beams into analog signals.

9. Optical processing unit according to claim 4 further comprises: a) a first linear amplifier and a second linear amplifier for converting the first analog signal and the third analog signal into a seventh analog signal and an eighth analog signal which represent the values ​​of the first tensor and the second tensor, and b) a third modulator and a fourth modulator and a third light source and a fourth light source for converting the seventh analog signal and the eighth analog signal into a third light beam and a fourth light beam.

10. Method for optically performing a tensor multiplication on a value of the first tensor and a value of the second tensor using the optical processing unit of any one of claims 1 to 9.

Images