calculator

Abstract

The present disclosure relates to a calculator comprising: at least one spatial light modulator, SLM, layer, comprising at least one programmable SLM device; at least one sensor-and-emitter layer, SEL, each of the at least one SEL comprising at least one electromagnetic, EM, radiation sensor, at least one programmable EM radiation emitter and a processing unit, wherein a first SEL of the at least one SEL is configured to emit first combined modulated EM radiation to the at least one SLM layer, wherein the SLM device is configured to further modulate the first combined modulated EM radiation to obtain a second combined modulated EM radiation, wherein the first SEL or a second SEL of the at least one SEL is configured to sense the second combined modulated EM radiation from the SLM layer. The present disclosure further relates to methods thereof and apparatus comprising the disclosed calculator.

Description

[0001] CALCULATOR

[0002] Technical field

[0003] The present invention relates to an electro-magnetic radiation based calculator, a method thereof and apparatus comprising the calculator.

[0004] Background

[0005] A spatial light modulator (SLM) is an optical device that imposes some form of spatially varying modulation on a beam of light or electromagnetic (EM) radiation.

[0006] Spatial light modulators are developing quickly, offering faster update frequencies and higher resolutions and smaller pixel sizes. SLMs come in many variants supporting both phase and amplitude modulation.

[0007] Some of the most used SLM technologies are:

[0008] - Digital micromirror devices (DMDs): DMDs are a type of SLM that uses an array of tiny mirrors to modulate the light. Each mirror can be individually tilted, which allows the SLM to project a wide variety of patterns. DMDs are very fast and typically used for applications where high temporal resolution modulation is required.

[0009] - Liquid crystal on silicon (LCOS) SLMs: LCOS SLMs are a type of SLM that uses liquid crystals to modulate the light. LCOS SLMs can be made to modulate either phase or amplitude of the incoming light. LCoS devices are widely available on the market. LCD with glass substrate. They can be reflective or transmissive (transparent)

[0010] - Metamaterial-based SLMs are still in research, and early commercialization phase, but have the benefits of enabling sub -wavelength pixel size and rapid update frequency. The company Swave (swave.io), in particular, develops a SLM system based on phase change materials with pixel size down to 300 nm or below.

[0011] An example of LCOS SLM is the Holoeye Leto-3, with a pixel pitch of 6.4 pm and a resolution of 1920x1080 pixels.

[0012] Performing calculations by the use of light or EM radiation can have benefits over traditional electronic systems in that optical signals and optical components (lenses, mirrors, etc...) can effectively represent matrices and matrix operations, all performed in parallel at the speed of light. SLMs can be the tool used to perform complex optical calculations, which then can be used for a variety of tasks, such as image processing and pattern recognition. In artificial neural networks, large network depth and layer matrix sizes require many and computationally heavy calculations (mostly matrix-vector multiplication accumulations). To reduce the complexity, mitigations like, among others, network pruning, weight quantization,low resolution input and better algorithms are researched and used to lower the number of needed calculations.

[0013] It is known that matrix multiplication operations can be performed by controlling a light waveform that is passing through or reflecting from a SLM or static hologram. An example of this using a single SLM layer for a vector multiplication is shown in prior art document: “Fully reconfigurable coherent optical vector-matrix multiplication”, by, James Spall et al. Vol. 45, No. 20 / 15 October 2020 / Optics Letters Optical society of America. As shown in Fig. 1 of this first prior art document, a vector input is encoded into columns of a “vector SLM”, while matrix elements are encoded into a “matrix SLM”. As light is passed through these two SLMs and then through a cylindrical lens, one can read the result of the vectormatrix multiplication as the intensity of light through a slit on the output after the lens.

[0014] It is also known that a stack of diffractive layers with a fixed optical structure according to the outcome of a training process, can function as an artificial neural network by passing a laser beam through it, and be used to perform image classification. The calculation may be performed with the speed of light as a single pass-through is needed. This is explained and shown in further prior art document “All-optical machine learning using diffractive deep neural networks”, by Xing Lin et al, Science 361, 10004-1008 (2018). In this reference the use of diffractive layers is also demonstrated with static (3D printed) diffractive layers. In this reference, pixels of diffractive layers represent nodes of a neural network and provide a complex coefficient (based on amplitude and phase modulation).

[0015] In a further prior art document, that is “https: / / www.microsoft.com / en-us / research / blog / unlocking-the-future-of-computing-the-analog-iterative-machines-lightning-fast-approach-to-optimization / ”, from summer 2023, it is proposed to introduce a non-linear electronic feedback loop to an optical system. The presented system in this prior art document does not include the diffractive patterns but is more to be seen as an optical analog weighting. Present solutions of optical calculations, especially directed to neural networks, are difficult or impossible to integrate in micro electronics devices, such as microchips, that can be used in consumer electronics.

[0016] In addition, optical neural networks, although being able to function at light speed, suffer from the main disadvantage that most optical systems are linear systems. As such, fully optical neural networks normally will not include a nonlinear activation function as is required for neural networks to be able to map to nonlinear problems. Without nonlinear activation, the networks will only be able to learn linear relationships between input and output. However, many real-world problems involve non-linear relationships, so a neuralnetwork without nonlinearity would be unable to learn these relationships. In addition, an optical neural network represented by one SLM or similar per layer in the network, will in practice be restricted to network models of limited depth. This is due to that the physical network size (i.e. number of optically aligned SLMs) will grow in proportion to the model depth. Light or EM radiation based calculators disclosed in the prior art may not be fully suitable for integration in micro electronics devices that can be used in consumer electronics or may be limited in speed.

[0017] Summary

[0018] An object of the present disclosure is to overcome the problems indicated above.

[0019] According to a first aspect, the present disclosure relates to a calculator comprising: at least one spatial light modulator, SLM, layer, comprising at least one programmable SLM device; at least one sensor-and-emitter layer, SEL, each of the at least one SEL comprising at least one electromagnetic, EM, radiation sensor, at least one programmable EM radiation emitter and a processing unit,

[0020] wherein a first SEL of the at least one SEL is configured to emit first combined modulated EM radiation to the at least one SLM layer,

[0021] wherein the SLM device is configured to further modulate the first combined modulated EM radiation to obtain a second combined modulated EM radiation,

[0022] wherein the first SEL or a second SEL of the at least one SEL is configured to sense the second combined modulated EM radiation from the SLM layer.

[0023] According to a further aspect, the present disclosure relates to a method of operating the presently disclosed calculator, the method comprising: emitting first combined modulated EM radiation from the first SEL to the SLM layer; further modulating, at the SLM layer, the first combined modulated EM radiation to obtain the second combined EM radiation; sensing, at the first or at the second SEL, the second combined modulated EM radiation.

[0024] According to a further aspect, the present disclosure relates to an apparatus comprising the presently disclosed calculator, wherein said apparatus is a user equipment, UE, Augmented Reality, AR, glasses, Virtual Reality, VR, glasses, or a network node or a repeater for wireless communications or a laptop or a computing device or a consumer electronics device or an electrical device or an electronic device.Further embodiments are defined in the dependent claims. It should be emphasized that the term “comprises / comprising” when used in this specification is taken to specify the presence of stated features, integers, steps, or components, but does not preclude the presence or addition of one or more other features, integers, steps, components, or groups thereof.

[0025] Brief description of the drawings

[0026] Fig. 1 illustrates embodiments of the presently disclosed calculator.

[0027] Fig. 2 illustrates embodiments of the presently disclosed calculator.

[0028] Fig. 3 illustrates embodiments of the presently disclosed calculator.

[0029] Fig. 4 illustrates a flow diagram.

[0030] Fig. 5a illustrates a flow diagram.

[0031] Fig. 5b illustrates a flow diagram.

[0032] Fig. 6 illustrates one embodiment of the presently disclosed calculator.

[0033] Fig. 7 illustrates one embodiment of the presently disclosed calculator.

[0034] Fig. 8 illustrates one embodiment of an apparatus or electronic device comprising the presently disclosed calculator.

[0035] Fig. 9 illustrates one embodiment of an apparatus or electronic device comprising the presently disclosed calculator.

[0036] Detailed description

[0037] Several embodiments of the present disclosure are described in detail below. Each embodiment has exemplary value. In the present description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be readily apparent to one of ordinary skill in the art that the present disclosure may be practiced without limitation to these specific details. In this description, well known methods and structures have not been described in detail so as not to unnecessarily obscure the present disclosure, and the skilled person, after reading this disclosure, may realize that other embodiments are possible, or other ways to implement the claimed subject matter. Words like “non-linear”, “nonlinear” and / or “non linear”, when refereed to operations, are construed to have the same meaning.Fig.l illustrates at least one embodiment of the presently disclosed calculator. Other embodiments that are not captured by Fig.l are covered by the claims of the present application. In the embodiments of Fig.l, the calculator comprises at least one sensor and emitter layer (SEL), of which a first SEL 103 a, and a second sensor and emitter layer (SEL) 103b. The calculator may comprise a spatial light modulator (SLM) layer, 101. In one embodiment the first SEL may be transparent and comprises at least one sensor and at least one emitter. In one embodiment, the first SEL comprises a plurality of sensors 1041 - 104nand a plurality of emitters 1051 - 105n. In one embodiment, sensors 1041 - 104nand emitters 105i - 105nare placed in pairs in common locations, which may be referred to as SEL pixels 112. First SEL 103a may be configured to receive an incoming electromagnetic (EM) radiation 109. The incoming EM radiation 109 may be generated by an electromagnetic (EM) source, for example a laser source, such as a pulsed laser. The incoming EM radiation 109 may be a coherent EM radiation. The incoming EM radiation 109 may have a wavelength in the range of visible light or infrared (IR) or ultraviolet (UV) or other wavelengths.

[0038] In one embodiment the at least one SEL and the SLM may have a planar shape comprising two planar faces or planar sides and other lateral faces or lateral sides perpendicular to the planar faces or planar sides. SEL and SLM may be facing at each other, being placed on parallel surfaces, wherein said parallel surfaces are parallel to planar faces of SEL and SLM.

[0039] The inventors have realized that the presently disclosed calculator is configured for being integrated in a single device or apparatus, such as AR glasses.

[0040] In one embodiment the first SEL 103a may be transparent, such that incoming EM radiation 109 may be propagated from a top face of the first SEL to a bottom face of the first SEL, after being modulated into first combined modulated EM radiation 107. A bottom face of the first SEL may correspond to a face of the first SEL facing towards the SLM. A top face of the first SEL corresponds to an opposite face to the bottom face of the first SEL. In Fig.l this corresponds to a vertical propagation from a top face to a bottom face of the first SEL 103a. In the embodiment where the first SEL is transparent, the calculator may advantageously comprise polarizers or may be designed to accept a loss in a returning EM radiation.In another embodiment the incoming EM radiation 109 may be injected in the first SEL 103 a from a lateral side, such as from a right or left side of the SEL and emitters may be configured to modulate the incoming EM radiation 109 and emit towards the SLM layer. In this embodiment, where the incoming EM radiation 109 is injected by a lateral side of the first SEL, the SEL does not need to be transparent.

[0041] In one embodiment, the at least one SEL and the SLM layer are planar layers facing towards each other. In one embodiment the SLM is in between the first SEL and the second SEL.

[0042] In the embodiments of Fig.1, where the first SEL is configured to receive an incoming EM radiation 109, each of the emitters of the first SEL is configured to modulate in phase and / or amplitude the incoming EM radiation 109. Therefore, the SEL is configured to emit a first combined modulated EM radiation 107 to the at least one SLM layer 101. The first combined modulated EM radiation 107 results from a combination of EM radiation emitted by each emitter of the first SEL 103a. SEL may comprise processing unit 106. Each emitter of the first SEL may be programmable, for example by means of processing unit 106. In particular amplitude and / or phase modulation of each emitter is programmable based on a first data vector. In one embodiment amplitude modulation comprises attenuation or gain. For example the first data vector may comprise a plurality of data elements, each data element being, for example, a complex number. It is clear to the skilled person, that a real number is also a complex number with phase zero. Each or a portion of the emitters 105i - 105nin the first SEL 103a may modulate the incoming EM radiation 109 by amplitude and / or phase based on at least one of said data elements of said first data vector. Amplitude and / or phase modulation of each emitter may be based on amplitude and / or phase of a complex element of the first data vector. In one embodiment a plurality of EM radiations are emitted by emitters of the first SEL 103a, having the SEL, in one embodiment, a plurality of emitters 105i - 105n. The EM radiations of each emitter is combined in a first combined EM radiation 107 which effectively represents or encodes the first data vector.

[0043] In the embodiments of Fig.1, the presently disclosed calculator comprises a spatial light modulator (SLM) layer, 101. In those embodiments the SLM is a transparent SLM, that is configured to propagate EM radiation from one side or face, such as a top side or face, to another side or face, such as a bottom side or face, of the layer, after modulation. A top side orface of the SLM is construed as being facing towards the first SEL and a bottom side is opposite to the top side. In the figure the SLM is configured to propagate EM radiation from a top side to a bottom side of the SLM layer, after modulation. The SLM layer comprises an SLM device. An SLM device may comprise a programmable optical matrix configured to further modulate the first combined modulated EM radiation 107, coming from the first SEL 103a to obtain a second combined modulated EM radiation 108. The SLM device may comprise N times M (NxM) pixels, wherein each pixel may be configured to further modulate the first combined modulated EM radiation 107 in phase and / or amplitude. Arrangement of pixel configurations on the SLM is often referred to as a Computer Generated Hologram (CGH). The first combined modulated EM radiation 107 may represent a first data vector. Pixels of the SLM device may represent a second vector, for example a vector of size N, or a first matrix, for example a NxM matrix, being N and M integers. Pixels of the SLM device may be spatially disposed at the SLM device in linear or rectangular form. For example pixels of the SLM device may be disposed spatially on N columns and M rows along a plane of the SLM device. Coefficients of said second vector and / or said first matrix may be complex numbers. Amplitude and / or phase modulation at each pixel of the SLM may correspond to coefficients of said second vector or first matrix. When the first combined modulated EM radiation 107 interacts with the SLM device, the first combined modulated EM radiation 107 is further modulated into a second combined modulated EM radiation 108. Said second combined modulated EM radiation 108 represents a result of a vector to vector or matrix to vector multiplication. In particular said second combined modulated EM radiation 108 represent the result of the multiplication of the first matrix (or second vector), that is the matrix encoded in the SLM device, and the first data vector, that is the vector represented by the first combined EM radiation.

[0044] In the embodiments of Fig.1, the presently disclosed calculator comprises at least one SEL, and a second SEL 103b. SEL 103b comprises at least one sensor 104i - 104nand at least one emitter 105i — 105n. In some embodiments, SEL 103b may comprise a plurality of sensors 104i- 104nand emitters 105i — 105n.

[0045] Second SEL 103b may sense, by means of the at least one sensor or by means of a plurality of sensors 1041 - 104n, second combined modulated EM radiation 108 from the SLM layer 101. As second combined modulated EM radiation 108 represents a result of a matrix by vector multiplication, second SEL 103b may sense such result, which may be a second data vector. SEL 103b, as well as each of the at least one SEL, comprises a processing unit.Second SEL 103b is further configured to read first values from sensors 1041 - 104nof the SEL; process said first values to obtain second values or processed values; drive emitters 1051 - 105nof the SEL to emit EM radiation based on the processed second values, collectively emitting a third combined modulated EM radiation towards the SLM layer. In one embodiment, the second values are equal to the first values. In one embodiment, processing is performed at least partially in the analog domain, with advantages in speed, low power consumption and low complexity. In one embodiment, processing is performed at least partially in the digital domain, with advantages in flexibility and programmability.

[0046] The first values that are being read from the sensors 1041 - 104ncorrespond to the result of matrix (or vector) by vector operation. They are then processed. In neural network applications the processing unit of the second SEL may process said read values from the sensors using linear and / or nonlinear operations. Said operations may be applied on single (scalar) or multiple values. Examples of such operations may be sigmoid function, maxpool, avgpool, Rectified Linear Unit (ReLu). In one embodiment, the processing unit of the at least one SEL may comprise a plurality of sub-processors for processing of the sensor values. Having a processing unit in the SEL for applying non-linearities contributes to improved speed of each iteration, because the processing unit may process values directly read from sensors present on the same layer (SEL) with no need for an external processing block for introducing the linearities. By applying the non-linearities directly at SEL the calculator is compact and may be integrated in a consumer device or apparatus.

[0047] In the embodiments of Fig.1, values of pixels of the SLM device are configured to be updated to values corresponding to a second matrix at a second iteration, or a kth matrix, at a kth iteration (being k an integer), and said update is in synchronization with at least one operation of the second SEL or the first SEL. Update of SLM pixel values may be performed by means of memory 111.

[0048] In the embodiments of Fig.1, the second SEL is further configured to emit a third combined modulated EM radiation (not shown in the figure) to the SLM layer, for further matrix (or vector) to vector multiplication to happen at the SLM. The second SEL emits a third combined modulated EM radiation that represents a processed data vector, wherein said processed data vector results from emitting EM radiation from emitters of the second SELbased on processed values, wherein said processes values are processed based on operations performed on first values read from the sensors of the second SEL.

[0049] Said differently: the embodiments of the calculator shown in Fig. 1 are configured to: modulate input EM radiation 109 at the first SEL to obtain a first combined modulated EM radiation 107 from the first SEL 103a to the SLM layer 101, said first combined modulated EM radiation 107 corresponding to a first data vector; modulate at the SLM device 102 the first combined modulated EM radiation to obtain a second combined modulated EM radiation 108 from the SLM to the second SEL 103b, said second combined modulated EM radiation 108 representing a result of a matrix to vector multiplication; sense the second combined modulated EM radiation 108 at sensors of the second SEL. In one embodiment the presently disclosed calculator is further configured to: read, at the second SEL, sensed values from the second combined modulated EM radiation 108; process said read values to obtain processed values or second values; emit, from the second SEL a third combined modulated EM radiation (not shown in the figure) based on the processed values at the second SEL, wherein said third combined modulated EM radiation represents a third data vector. In one embodiment, said third data vector is a result of processing a second data vector, wherein the second data vector is the vector sensed at the second SEL. In one embodiment, said second data vector is the result of the multiplication between the first data vector encoded at the first SEL by the first matrix corresponding to first CGH of the SLM device 102. The embodiments of the calculator shown in Fig. 1 are further configured to: update pixel values of the SLM based on coefficients of a second matrix; modulate the third combined modulated EM radiation at the SLM to obtain a fourth combined modulated EM radiation from the SLM to the first SEL 103a, wherein said fourth combined modulated EM radiation represents a matrix to vector multiplication of the second matrix to the third vector; sense the fourth combined modulated EM radiation at sensors of the first SEL; read, at the first SEL, fourth sensed values from the fourth combined modulated EM radiation; process said read fourth values to obtain processed fourth values; emit, from the first SEL a fifth combined modulated EM radiation (not shown in the figure) based on the processed fourth values at the first SEL, wherein said fifth combined modulated EM radiation represents a further data vector.

[0050] The presently disclosed calculator is configured to be operated according an iterative process of an integer number of matrix to vector multiplications obtained in the electromagnetic domain and data processing comprising non-linearities, obtained in the processing unit of the at least one SEL, wherein said process is suitable for neural networks.The claimed calculator is configured for n iterations where n is an integer. Each iteration through the programmable SLM device may correspond to a layer of a neural network. In one example, when the EM radiation has traversed the SLM L times, that may correspond to operation of a neural network of L layers. The presently disclosed calculator is therefore configured for inference and / or training of neural networks.

[0051] Fig. 2 illustrates further embodiments of the presently disclosed calculator. In this embodiment the first SEL 103e is not a transparent SEL. Other embodiments that are not captured by Fig.2 are covered by the claims of the present application. In one embodiment emitters of the at least one SEL, in particular the first SEL, may be synchronized in such a way that the first combined EM radiation is coherent. In particular, emitters of the at least one SEL, in particular the first SEL, may be synchronized, that is they have phases relative to a common reference phase in order to obtain a coherent combined radiation. In this embodiment emitters of the at least one SEL, in particular the second SEL, may be synchronized in such a way that the third combined EM radiation is coherent, that is they have phases relative to a common reference phase. In the embodiment of Fig. 2 the first SEL 103e may be an opaque SEL and emits the first combined modulated EM radiation 107 by means of emitters 105i - 105n. In this embodiment, emitters may be active emitters such as light emitting diodes. In one embodiment emitting the first combined EM radiation is based on synchronized emitters of the at least one SEL and no incoming EM radiation EM is required. In one embodiment an incoming EM radiation 109 may be injected by a lateral side of the first SEL 103e. A lateral side may be perpendicular to a top face of the SEL or to a plane of a planar shape of the SEL. In another embodiment no incoming EM radiation 109 is required and emitters of the at least one SEL may be synchronized, that is their phases are correlated in such a way that emitters modulating EM radiation based on a same and equal data value, such as a same and equal complex number, are such that the emitted EM radiations are aligned in phase, thus the first combined EM radiation being coherent. The working principle of this embodiment is similar to the working principle of the embodiment of Fig. 1.

[0052] In the embodiments of Fig.2, the calculator comprises at least one sensor and emitter layer (SEL), of which a first SEL 103e, and a second sensor and emitter layer (SEL) 103b. The calculator may comprise a spatial light modulator (SLM) layer, 101. In one embodiment the first SEL comprises at least one sensor and at least one emitter. In some embodiments, the SEL 103e comprises a plurality of sensors 104i - 104nand a plurality of emitters 1051 - 105n. In one embodiment, sensors 1041 - 104nand emitters 1051 - 105nare placed in pairs incommon locations, which may be referred to as SEL pixels. First SEL 103e may be configured to receive an incoming electromagnetic (EM) radiation 109 from a lateral side. In one embodiment an incoming EM radiation 109 may be generated by a laser source, such as a pulsed laser. The incoming EM radiation 109 may be a coherent EM radiation. The incoming EM radiation 109 may have a wavelength in the range of visible light or infrared (IR) or ultraviolet (UV) or other wavelength.

[0053] In one embodiment an incoming EM radiation 109 (not shown in Fig.3) may be injected in the SEL 103e from a lateral side such as from a right or left side of the SEL and emitters may be configured to modulate it and emit down towards the SLM layer.

[0054] In one embodiment, no incoming EM radiation 109 is injected and emitters of the incoming EM radiation are synchronized in such a way to be configured to emit, as a whole, a first combined EM radiation 107 which may be coherent.

[0055] In one embodiment, the at least one SEL and the and the SLM layer are planar layers facing towards each other. In one embodiment the SLM may be in between the first SEL and the second SEL.

[0056] In any case, the SEL 103e is configured to emit a first combined modulated EM radiation 107 to the at least one SLM layer 101. The first combined modulated EM radiation 107 results from a combination of all the EM radiations emitted by each emitter of the first SEL 103e. SEL may comprise processing unit 106. Each emitter of the first SEL may be programmable, for example by mean of processing unit 106. In particular amplitude and / or phase modulation of each emitter is programmable based on a first data vector. In one embodiment amplitude modulation comprises attenuation or gain. For example the first data vector may comprise a plurality of data elements, each data element being, for example, a complex number. Each or a portion of the emitters 105i - 105nin the first SEL 103e may modulate the incoming EM radiation 109 by amplitude and / or phase based on at least one of said data elements of said first data vector. Amplitude and / or phase modulation of each emitter may be based on amplitude and / or phase of a complex element of the first data vector. This way a plurality of EM radiations are emitted by emitters of the first SEL 103e, having the SEL, in one embodiment, a plurality of emitters 1051 - 105n. The EM radiations of each emitter iscombined in a first combined EM radiation 107 which effectively represents or encodes the first data vector.

[0057] In the embodiments of Fig.2, the presently disclosed calculator comprises a spatial light modulator (SLM) layer, 101. In this embodiment the SLM is a transparent SLM, that is configured to propagate EM radiation from one side, such as a top side, to another side, such as a bottom side, of the layer, after modulation. A top side of the SLM is construed as being facing towards the first SEL and a bottom side is opposite to the top side. In the figure the SLM is configured to propagate EM radiation from a top side to a bottom side of the SLM layer, after modulation. The SLM layer comprises an SLM device. An SLM device may comprise a programmable optical matrix configured to further modulate the first combined modulated EM radiation 107, coming from the first SEL 103e to obtain a second combined modulated EM radiation 108. The SLM device may comprise NxM pixels, wherein each pixel may be configured to further modulate the first combined modulated EM radiation 107 in phase and / or amplitude. Pixels of the SLM are relative to what is often referred to as computer generated holography (CGH). The first combined modulated EM radiation 107 may represent a first data vector. Pixels of the SLM device may represent a second vector, for example a vector of size N, or a first matrix, for example a NxM matrix, being N and M integers. Pixels of the SLM device may be spatially disposed at the SLM device in linear or rectangular form. For example pixels of the SLM device may be disposed spatially on N columns and M rows along a plane of the SLM device. Coefficients of said second vector and / or said first matrix may be complex numbers. Amplitude and / or phase modulation at each pixel of the SLM may correspond to coefficients of said vector or matrix. When the first combined modulated EM radiation 107 interacts with the SLM device, the first combined modulated EM radiation 107 is further modulated into a second combined modulated EM radiation 108. Said second combined modulated EM radiation 108 represents the result of a vector to vector or matrix to vector multiplication. In particular said second combined modulated EM radiation 108 represent the result of the multiplication of the first matrix (or vector), that is the matrix encoded in the SLM device at an initial step, and the first data vector.

[0058] In the embodiments of Fig.2, the presently disclosed calculator comprises at least one SEL, and a second SEL 103b. Second SEL 103b comprises at least one sensor 104i - 104nand atleast one emitter 1051 - 105n. In some applications, SEL 103b may comprise a plurality of sensors 104i — 104nand emitters 105i — 105n.

[0059] Second SEL 103b may sense, by means of the at least one sensor or by means of a plurality of sensors 1041 - 104n, second combined modulated EM radiation 108 from the SLM layer 101. As second combined modulated EM radiation 108 represents a result of a matrix by vector multiplication, second SEL 103b may sense such result, which may be a second data vector. SEL 103b, as well as each of the at least one SEL, comprises a processing unit.

[0060] Second SEL 103b is therefore further configured to read first values from sensors 1041 - 104nof the SEL; process said first values to obtain second values or processed values; drive emitters 1051 - 105nof the SEL to emit EM radiation based on the processed second values, collectively emitting a third combined modulated EM radiation 110 towards the SLM layer. In one embodiment, the second values are equal to the first values. In one embodiment, processing is performed at least partially in the analog domain, with advantages in speed, low power consumption and low complexity. In one embodiment, processing is performed at least partially in the digital domain, with advantages in flexibility, programmability.

[0061] The first values that are being read from the sensors 1041 - 104ncorrespond to the result of matrix (or vector) by vector operation. They are then processed. In neural network applications the processing unit of the second SEL may process said read values from the sensors using linear or non linear operations. Said operations may be applied on single (scalar) or multiple values preferentially using local data from neighbouring SEL pixels. Examples of such operations may be sigmoid function, maxpool, avgpool, Rectified Linear Unit (ReLu). In one embodiment, the processing unit of the at least one SEL may comprise a plurality of sub-processor for processing of the sensor values.

[0062] In the embodiments of Fig.2, pixels of the SLM device are configured to be updated to values corresponding to a second matrix or further matrix, in synchronization with at least one operation of the second SEL or the first SEL. Such pixel update of the SLM device pixel values may be done at each iteration, by means of memory 111.

[0063] In the embodiments of Fig.2, the second SEL is further configured to emit a third combined modulated EM radiation 110 to the SLM layer, for further matrix (or vector) to vector multiplication to happen at the SLM. The second SEL emits a third combined modulated EM radiation 110 that represents a processed data vector, wherein said processeddata vector results from emitting EM radiation from emitters of the second SEL based on processed values, said processed values based on nonlinear operations of first read values from the sensors of the second SEL.

[0064] In other words, the embodiments of the calculator shown in Fig. 2 are configured to: emit a first combined modulated EM radiation 107 from the first SEL 103e to the SLM layer 101, said first combined modulated EM radiation 107 corresponding to a first data vector; modulate at the SLM device 102 the first combined modulated EM radiation to obtain a second combined modulated EM radiation 108 from the SLM to the second SEL 103b, said second combined modulated EM radiation 108 representing a result of a matrix to vector multiplication; sense the second combined modulated EM radiation 108 at sensors of the second SEL. In one embodiment the calculator is further configured to: read, at the second SEL, sensed values from the second combined modulated EM radiation 108; process said read values to obtain processed values; emit, from the second SEL a third combined modulated EM radiation 110 based on the processed values at the second SEL, wherein said third combined modulated EM radiation represents a third data vector. In one embodiment said third data vector is a result of processing a second data vector, wherein the second data vector, whose values are also referred to as first read valued, is the vector sensed at the second SEL. In one embodiment said second data vector is the result of the multiplication between the first data vector encoded at the first SEL by the first matrix corresponding to the first CGH of the SLM device 102. The embodiments of the calculator shown in Fig. 2 are further configured to: update pixel values of the SLM based on coefficients of a second matrix; modulate the third combined modulated EM radiation at the SLM to obtain a fourth combined modulated EM radiation from the SLM to the first SEL 103e, wherein said fourth combined modulated EM radiation represents a matrix to vector multiplication of the third vector to the second matrix. In one embodiment the presently disclosed calculator is further configured to sense the fourth combined modulated EM radiation at sensors of the first SEL; read, at the first SEL, fourth sensed values from the fourth combined modulated EM radiation; process said read fourth values to obtain processed fourth values; emit, from the first SEL a fifth combined modulated EM radiation (not shown in the figure) based on the processed fourth values at the first SEL, wherein said fifth combined modulated EM radiation represents a further data vector.Pixel values of the SLM may correspond to properties of the SLM such as voltages, current levels, optical state like mirrorlike versus absorbing, or a specific phase shift based on changing refractive index, etc.... Pixel values may be relative to a CGH.

[0065] The embodiments of the calculator shown in Fig. 2 are configured to be operated according to an iterative process of an integer number of matrix to vector multiplications obtained in the electromagnetic domain and data processing comprising nonlinearities, obtained in the processing unit of the at least one SEL, wherein said process is suitable for neural networks. The inventors have realized that the claimed calculator is configured for n iterations where n is an integer. Each iteration through the programmable SLM device may correspond to a layer of a neural network. In one example, when the EM radiation has traversed the SLM L times, that corresponds to operation of a neural network of L layers or L+l layers. The presently disclosed calculator is therefore configured for inference and / or training of neural networks.

[0066] Fig. 3 illustrates a further embodiment of the presently disclosed calculator. In this embodiment there is only one SEL, that may be referred to as the first SEL 103c or the SEL 103c. . In this embodiment the SLM may be reflective. In this embodiment the first combined modulated EM radiation 107 is still from the first SEL to the SLM. In this embodiment the second combined modulated EM radiation 108 is still from the SLM to the at least one SEL, shown in the figure as 103c. However, in this case, because the SLM is reflective, a physical direction of the first combined modulated EM radiation 107 (from SEL to SLM) and a physical direction of the second combined modulated EM radiation 108 (from SLM to SEL) are opposite.

[0067] The skilled person will realize that the working principle of this embodiment is similar to the working principle of the embodiment of Fig. 2 However, in the embodiment of Fig. 3, only one SEL is present and the SLM is reflective.

[0068] Fig. 3 illustrates further embodiments of the presently disclosed calculator. Other embodiments that are not captured by Fig.3 are covered by the claims of the present application. In one embodiment emitters of the SEL 103 c may be synchronized in such a way that the first combined EM radiation is coherent. In one of the embodiments of Fig. 3 the SEL 103c may be an opaque SEL and emits the first combined modulated EM radiation 107 by means of emitters 1051 - 105n. In this embodiment, emitters may be active emitters such aslight emitting diodes and may be synchronized. In one embodiment an incoming EM radiation 109 may be injected by a lateral side of the first SEL 103 c. In another embodiment no incoming EM radiation 109 is not required and emitters of the at least one SEL may be synchronized. The skilled person will realize that the working principle of this embodiment is similar to the working principle of the embodiment of Fig. 1 and 2, with a reflective SLM instead and only one SEL.

[0069] In the embodiments of Fig.3, the calculator comprises at least one sensor and emitter layer (SEL) 103c. The calculator may comprise a spatial light modulator (SLM) layer, 101. In one embodiment the first SEL comprises at least one sensor and at least one emitter. It is clear to the skilled person that, in one embodiment, the first SEL comprises a plurality of sensors 1041 - 104nand a plurality of emitters 105i - 105n. In one embodiment, sensors 104i - 104nand emitters 1051 - 105nare placed in pairs in common locations, which may be referred to as SEL pixels. First SEL 103c may be configured to receive an incoming electromagnetic (EM) radiation 109. In the embodiments that require an incoming EM radiation 109, the incoming EM radiation 109 may be generated by a laser source, such as a pulsed laser. The incoming EM radiation 109 may be a coherent EM radiation. The incoming EM radiation 109 may have a wavelength in the range of visible light or infrared (IR) or ultraviolet (UV) or other wavelength.

[0070] In one embodiment an incoming EM radiation 109 (not shown in Fig.3) may be injected in the first SEL 103 c from a lateral side such as from a right or left side of the SEL and emitters may be configured to modulate it and emit towards the SLM layer.

[0071] In one embodiment, no incoming EM radiation 109 is injected and emitters of the incoming EM radiation are synchronized in such a way to be configured to emit, as a whole, a first combined EM radiation 107 which may be coherent.

[0072] In one embodiment, the at least one SEL and the and the SLM layer are planar layers facing towards each other. In one embodiment the SLM top side or top face is facing towards the SEL whereas the SELs bottom face or side is facing towards the SLM.

[0073] The first SEL 103c is configured to emit a first combined modulated EM radiation 107 to the at least one SLM layer 101. The first combined modulated EM radiation 107 results froma combination of all the EM radiations emitted by each emitter of the first SEL 103 c. Each emitter of the first SEL may be programmable. In particular amplitude and / or phase modulation of each emitter is programmable based on a first data vector. In one embodiment amplitude modulation comprises attenuation or gain. For example the first data vector may comprise a plurality of data elements, each data element being, for example, a complex number. Each or a portion of the emitters 1051 - 105nin the first SEL 103e may modulate the incoming EM radiation 109 by amplitude and / or phase based on at least one of said data elements of said first data vector. The skilled person may understand that amplitude and / or phase modulation of each emitter may be based on amplitude and / or phase of a complex element of the first data vector. This way a plurality of EM radiations are emitted by emitters of the first SEL 103 c, having the SEL, in one embodiment, a plurality of emitters 1051 - 105n. The EM radiations of each emitter is combined in a first combined EM radiation 107 which effectively represents or encodes the first data vector.

[0074] In the embodiments of Fig.3, the presently disclosed calculator comprises a spatial light modulator (SLM) layer, 101. In this embodiment the SLM is a reflective SLM, that is configured to reflect EM radiation after modulation, that is after matrix to vector multiplication. The SLM layer comprises an SLM device. An SLM device may comprise a programmable optical matrix configured to further modulate the first combined modulated EM radiation 107, coming from the first SEL 103a to obtain a second combined modulated EM radiation 108. The SLM device may comprise NxM pixels, wherein each pixel may be configured to further modulate the first combined modulated EM radiation 107 in phase and / or amplitude. Pixels of the SLM may in this disclosure being referred to as computer generated holography (CGH) as they are relative to a CGH. The first combined modulated EM radiation 107 may represent a first data vector. Pixels of the SLM device may represent a second vector, for example a vector of size N, or a first matrix, for example a NxM matrix, being N and M integers. Pixels of the SLM device may be spatially disposed at the SLM device in linear or rectangular form. For example pixels of the SLM device may be disposed spatially on N columns and M rows along a plane of the SLM device. Coefficients of said second vector and / or said first matrix may be complex numbers. Amplitude and / or phase modulation at each pixel of the SLM may correspond to coefficients of said vector or matrix. When the first combined modulated EM radiation 107 interacts with the SLM device, the first combined modulated EM radiation 107 is further modulated into a second combined modulated EM radiation 108. Said second combined modulated EM radiation 108 representsthe result of a vector to vector or matrix to vector multiplication. In particular said second combined modulated EM radiation 108 represent the result of the multiplication of the first matrix (or vector), that is the matrix encoded in the SLM device, and the first data vector. In the embodiments of Fig.3, second combined modulated EM radiation 108 may have an opposite direction of first combined modulated EM radiation 107 as the SLM is reflective. However, also in these embodiment, direction of the second combined modulated EM radiation 108 is from the SLM to the at least one SEL.

[0075] In the embodiments of Fig.3, the presently disclosed calculator comprises one SEL 103c. The SEL comprises at least one sensor 104i - 104nand at least one emitter 105i - 105n. In one embodiment, SEL 103b may comprise a plurality of sensors 104i — 104nand emitters 105i — 105n.

[0076] First SEL 103 c may sense, by means of the at least one sensor 104i - 104nor by means of a plurality of sensors 1041 - 104n, second combined modulated EM radiation 108 from the SLM layer 101. As second combined modulated EM radiation 108 represents a result of a matrix by vector multiplication, second SEL 103c may sense and / or read such result, which may be a second data vector. SEL 103c comprises a processing unit. SEL 103c is further configured to read first values from sensors 1041 - 104nof the SEL; process said first values to obtain second values or processed values; drive emitters 1051 - 105nof the SEL to emit EM radiation based on the processed second values, collectively emitting a third combined modulated EM radiation towards the SLM layer. In one embodiment, the second values are equal to the first values. In one embodiment, processing is performed at least partially in the analog domain, with advantages in speed, low power consumption and low complexity. In one embodiment, processing is performed at least partially in the digital domain, with advantages in flexibility, programmability.

[0077] The first values that are being read from the sensors 1041 - 104ncorrespond to the result of matrix (or vector) by vector operation. They are then processed. In neural network applications the processing unit of the second SEL may process said read values from the sensors using linear or nonlinear operations. Said operations may be applied on single (scalar) or multiple values. Examples of such operations may be sigmoid function, maxpool, avgpool, Rectified Linear Unit (ReLu). In one embodiment, the processing unit of the at least one SEL may comprise a plurality of sub-processor for processing of the sensor values.In the embodiments of Fig.3, pixels of the SLM device are configured to be updated to values corresponding to a second matrix, or further matrix, in synchronization with at least one operation of the SEL. Update of SLM pixel values may be performed by means of memory 111.

[0078] In the embodiments of Fig.3, the first SEL 103c is further configured to emit a third combined modulated EM radiation 110 to the SLM layer, for further matrix (or vector) to vector multiplication to happen at the SLM. The first SEL emits a third combined modulated EM radiation 110 that represents a processed data vector, wherein said processed data vector results from emitting EM radiation from emitters of the first SEL based on processed first values read from the sensors of the first SEL.

[0079] The embodiments of the calculator shown in Fig. 3 are configured to: emit a first combined modulated EM radiation 107 from the first SEL 103c to the SLM layer 101, said first combined modulated EM radiation 107 corresponding to a first data vector; modulate at the SLM device 102 the first combined modulated EM radiation to obtain a second combined modulated EM radiation 108 from the SLM to the first SEL 103c, said second combined modulated EM radiation 108 representing a result of a matrix to vector multiplication; sense the second combined modulated EM radiation 108 at sensors of the first SEL. In one embodiment the presently disclosed calculator is further configured to: read, at the first SEL, sensed values from the second combined modulated EM radiation 108; process said read values to obtain processed values or second values; emit, from the first SEL a third combined modulated EM radiation 110 based on the processed values at the second SEL, wherein said third combined modulated EM radiation represents a third data vector, wherein said third data vector is a result of processing a second data vector, wherein the second data vector is the vector sensed at the first SEL and being the result of the multiplication between the first data vector encoded at the first SEL by the first matrix corresponding to the SLM device 102. The embodiments of the calculator shown in Fig. 3 are further configured to: update pixel values of the SLM based on coefficients of a second matrix; modulate the third combined modulated EM radiation at the SLM to obtain a fourth combined modulated EM radiation from the SLM to the first SEL 103c.

[0080] The embodiments of the calculator shown in Fig. 3 are configured to be operated according an iterative process of an integer number of matrix to vector multiplications obtained in the electromagnetic domain and data processing comprising non-linearities, obtained in theprocessing unit of the at least one SEL, wherein said process is suitable for neural networks. The inventors have realized that the claimed calculator is configured for n iterations where n is an integer. Each iteration through the programmable SLM device may correspond to a layer of a neural network. In one example, when the EM radiation has traversed the SLM L times, that corresponds to operation of a neural network of L layers or L+l layers. The presently disclosed calculator is therefore configured for inference and / or training of neural networks.

[0081] Fig. 6 illustrates further embodiments of the presently disclosed calculator. One embodiment of the presently disclosed calculator comprises a first SEL 103e and a second SEL 103d. It also comprises a transparent SLM layer 101. In this embodiment, emitters 105u of the first SEL 103e may emit EM radiation at a first wavelength that may be a different wavelength than a second wavelength, the second wavelength being a wavelength of emitters 105L of the second SEL 103d. For example, emitters 105u of the first SEL (upper SEL in Fig.

[0082] 6) may emit at a first wavelength of 500 nm, and emitters 105L of the second SEL (Lower SEL in Fig. 6) may emit at a second wavelength of 400 nm. In this embodiment EM radiation from the second SEL 103d is emitted while still reading sensor values at the second SEL 103d. In this embodiment sensors at the first SEL are sensitive to EM radiation of the second wavelength and sensors of the second SEL are sensitive to EM radiation at the first wavelength, in virtue of applied filters covering respective sensors. In this embodiment, pixel values of the SLM device are adjusted to specific wavelength. In this embodiment sensors of the SEL is / are covered by a filter (114, 113) to avoid that the sensor is affected by stray light or reflected light from the emitter from the same SEL. Filter 114 of first SEL filters EM radiation at the first wavelength. Filter 113 of the second SEL filters EM radiation at the second wavelength. In this embodiment parallel independent calculations may be performed at the different wavelengths. For example two separate neural network computations may be performed. Or operations of a same neural network that are independent may be performed. In one embodiment more than 2 different wavelengths may be used.

[0083] In one embodiment a portion of the emitters of the first SEL may emit at a first wavelength and a different portion of the emitters of the first SEL may emit at a third wavelength.

[0084] Similarly, a portion of the emitters of the second SEL may emit at a second wavelength and a different portion of the emitters of the first SEL may emit at a fourth wavelength. W different wavelengths may be used, with W being an even integer.Fig.7 illustrates a further embodiment of the presently disclosed calculator. In this embodiment sensors of the at least one SEL are based on Light Field Display (LFD) system. In this embodiment, EM radiations may be emitted with different intensity in different directions by emitters 105F of the at least one SEL. In this embodiment sensors of the at least one SEL are based on Plenoptic camera 104d. This way the sensor, by use of, for example, microlenses, are able to direct the light from specific directions to a sensitive area of the sensor. This way, by using the direction of light as an additional parameter, besides amplitude and phase, additional calculation opportunities are obtained. In one example emitters may be regular emitters and sensors may be Plenoptic Cameras 104d

[0085] In one embodiment, the first combined modulated EM radiation 107 spatially represents at least a first data vector, and / or wherein each one of the at least one EM emitter ( 1051 - 105n) in the at least one SEL (103a, 103b, 103c, 103d, 103e) is programmable and is configured to independently emit EM radiation with amplitude and / or phase based on the first data vector. When the at least one emitter is a plurality of emitters, the first combined modulated EM radiation 107 is formed by a combination of EM radiations from the emitters. In one example each emitter emits an EM radiation that corresponds to a complex number, wherein an amplitude of the EM radiation emitted by one emitter is proportional to an amplitude of a complex number in a polar coordinates representation, and a phase of the EM radiation emitted by the same emitter is proportional to a phase of the same complex number in polar coordinates representation. In one embodiment a wavelength of all emitters in one SEL is the same. The first data vector, in one example, may comprise a plurality of complex numbers. As emitters emit light or EM radiation representing said complex numbers, the first combined modulated EM radiation 107 may spatially represent a data vector. In one example the data vector is mono-dimensional and in this example emitters of the SEL may be advantageously be placed linearly on the SEL.

[0086] In one embodiment, pixels (102i-102n) of the SLM device (102) are programmable, and / or pixels (102i-102n) of the SLM device correspond to elements or coefficients of a second vector or a first matrix.In one embodiment the SLM device may be implemented as Digital Micro Mirrors Devices (DMDs).

[0087] In one embodiment the SLM device may be implemented as Liquid crystal on silicon (LCOS).

[0088] In one embodiment the SLM device may be implemented as Liquid crystal on glass, or other type of Liquid Crystal.

[0089] In one embodiment the SLM device may be implemented as Metamaterials. In one example the metamaterials are developed by the company Swave. Metamaterials developed by Swave are typically reflective.

[0090] In one embodiment amplitude and / or phase modulation of each pixel of the at least one SLM device is based on a computer-generated hologram, CGH.

[0091] In one embodiment of the present disclosure, the at least one SLM layer is transparent. In one embodiment the at least one SLM layer comprises a plurality of SLM layers. This has the advantage of stacking SLM layers which speeds up the computations by adding further additional layers of computations in one pass of the EM radiation from a first SEL to a second SEL.

[0092] In one embodiment the SLM device is linear, rectangular or square.

[0093] In one embodiment the at least one SLM layer is a single SLM layer.

[0094] In one embodiment the at least one SEL is offset by a distance from the SLM layer. Said distance may be in the order of several millimeters, preferably less than 5 mm, more preferably less than 3 mm, even more preferably less than 2 mm. A distance between the at least one SEL and the SLM may be proportional to a planar size of the SEL and / or the SLM: That is, if the SEL and / or SLM have a planar size of 5 mm by 5 mm, such distance may be 5 mm, or less than 5 mm, preferably less than 3 mm, even more preferably less than 2 mm. However, if the SEL and / or SLM have a planar size of 10 mm by 10 mm, such distance maybe 10 mm, or less than 10 mm, preferably less than 5 mm, even more preferably less than 3 mm.

[0095] In one embodiment each pixel 102i-102nof the SLM device is configured to independently modulate, by independent amplitude and / or phase, the first combined modulated EM radiation 107 to obtain the second combined modulated EM radiation 108. In one embodiment the SLM devices represents or encodes a matrix or a second vector, wherein each pixel of the SLM represents a coefficient of said matrix or said second vector. The first combined modulated EM radiation may represent a first data vector. The second combined modulated EM radiation represents a result of a matrix to vector multiplication or vector to vector multiplication.

[0096] In one embodiment the first SEL 103a, 103b, 103c, 103d, 103e is further configured to modulate an incoming EM radiation 109 from a first EM radiation source to provide the first combined modulated EM radiation 107, and each one of the at least one EM emitter ( 1051 -105n) is configured to independently modulate the incoming EM radiation (109) in amplitude and / or phase based on at least one value of the first data vector. In this embodiment, emitters of the at least one SEL are configured to modulate in amplitude and / or phase the incoming radiation.

[0097] In one embodiment the incoming EM radiation 109 is coming from one top face of the SEL opposite to the face facing the SLM, as in Fig.1.

[0098] In another embodiment the incoming EM radiation 109 comes from a lateral side of the at least one SEL.

[0099] In one embodiment the incoming EM radiation is generated by a laser diode, such as a pulsed laser. In one embodiment, iterations of a calculation within the calculator are timed by pulses of said laser.

[0100] In one embodiment each of the at least one SEL 103 a, 103b, 103 c, 103 d, 103e is further configured to: read first values from sensors 1041 - 104nof the SEL; process said first values to obtain second values; drive emitters 1051 - 105nof the SEL to emit EM radiation based onthe processed second values, collectively emitting a third combined modulated EM radiation 110 towards the SLM layer.

[0101] In one embodiment processing first values to obtain second values or processed values comprises applying nonlinear operations, such as data pooling or max pooling or average pooling or ReLu. In this embodiment linear operations may also be applied. In this embodiment, linear and / or non-linear operations may be applied to each single value of the first values, or to combination of said values, or to all said values. In this embodiment the processing is performed by the processing unit in the at least one SEL, said processing unit may be distributed within the SEL. As the non-linearities are applied at the SEL, the presently disclosed calculator does not need an off-chip computing device for the calculations and is therefore compact and can be fully integrated in a microchip. In addition a processing unit 106 integrated in the SEL contributes to a compact design that can be integrated in electronic devices or apparatuses for consumer products.

[0102] In one embodiment, the at least one SEL 103a, 103b, 103c, 103d, 103e is configured to emit a third modulated EM radiation when or after a refresh time of the SLM device 102 has elapsed. In one embodiment the at least one SEL 103a, 103b, 103c, 103d, 103e is configured to emit the third modulated EM radiation at termination of a refresh time of the SLM device. An SLM device typically requires some time for an update of the pixels. Said time may be referred to as refresh time of the SLM. The inventors have realized that it is advantageous to update the pixels before or just before the third modulated EM radiation, that is the radiation from a next iteration of the calculation, reaches the SLM. This way the matrix represented by the SLM is updated to a next step in a neural network computation.

[0103] In one embodiment, pixel configuration of the SLM device 102 is stored in a memory operatively coupled to the SLM device, wherein the SLM layer comprises said memory. In one embodiment said memory is an analog memory. This has the advantage of being low power and fast. The time it takes to program the SLM pixels based on the coefficients stored in the memory affects the speed of the iterations of the calculator as, for example, it may affect a refresh time of the SLM device. The memory may store coefficients of all matrixes needed in a neural network operation. The pixels of the SLM device are updated, at each iteration, based on coefficients stored in the memory and corresponding to said iteration. Datavectors modulated by the at least one SEL may reach the SLM after the SLM pixels have been updated and represent a matrix corresponding to a next iteration. Since memory Ill is integrated in the SLM layer, the time it takes to program or load the SLM device can be reduced as compared to using off-chip devices, because the memory is local and directly coupled to the SLM device on the same layer, the SLM layer. In addition a memory integrated in the SLM contributes to a compact design that can be integrated in electronic devices for consumer products.

[0104] In one embodiment the memory 111 is operatively coupled to the at least one SEL 103a, 103b, 103c, 103d, 103e for at least synchronization. In this embodiment the memory is synchronized with the SEL for operation. This means that the at least one SEL, in one embodiment, is synchronized with the memory and / or the SLM layer. In one example, the SEL starts emitting the third combined modulated EM radiation only after receiving a synchronization signal, said synchronization signal signaling that, for example, the SLM pixel configuration update, based on the coefficients stored in the memory, has finished. Said synchronization signal may be optical or electrical. In some embodiments the at least one SEL may emit first or further combine EM radiation responsive to a synchronization signal, wherein the synchronization signal may be one pulse of the incoming EM radiation 109, such as a pulsed laser. Using pulses of the source for the incoming EM radiation (109) for synchronization has the advantage of using a same source for both emission of the first combined EM radiation and for synchronization, saving hardware components. In one embodiment, memory 111 may initiate update of pixels of SLM device 102 upon arrival of the first combined EM radiation (107) from the at least one SEL, or upon arrival of further combined EM radiation from the at least one SEL.

[0105] In some embodiments the at least one SEL may emit a first combined EM radiation 107 or further combine EM radiation, responsive to a synchronization signal, wherein said synchronization signal is generated by a synchronizer. In some embodiments said synchronizer may be optical, which has the advantage of not requiring wiring from SEL to SLM or from the synchronizer to SEL and / or SLM; in some embodiments said synchronizer may be electrical, requiring wiring.

[0106] In one embodiment, the presently disclosed calculator further comprises a synchronizer operatively coupled to the SLM layer (101) and / or each of the at least one SEL (103a, 103b,103c, 103d, 103e), the synchronizer configured to generate at least a first signal, the first signal comprising at least an integer number of occurrences, wherein the at least one SEL is configured to initiate emitting the first combined EM radiation or further combined EM radiations responsive to at least one occurrence of said first signal, and / or the SLM layer is configured to initiate update of pixel configuration of the SLM device (102) responsive to at least one occurrence of said first signal.

[0107] In one embodiment the memory Ill is configured to initiate an update of pixel configuration of the SLM device 102 based on arrival of the first combined modulated EM radiation 107 at the SLM layer. Once the first combined modulated EM radiation 107 has reached the SLM and the SLM has obtained a second combined modulated EM radiation, pixel update of the SLM may be initiated, to save time and obtain faster calculations. In addition, this way no wiring between SEL and SLM is required as the memory 111 will start the update of the SLM pixels responsive to an optical trigger, wherein said optical trigger is arrival of the first combined EM radiation to the SLM. The skilled person realizes that the SLM modulates the first combined EM radiation into the second combined EM radiation at speed of light, whereas the memory 111 may have a setup time before initiating the transfer of the new pixel values to the SLM device and the SLM device may have a refresh time.

[0108] Therefore the memory 111 may be advantageously triggered, for update of pixel configuration of the SLM, by arrival of the first combined EM radiation to the SLM.

[0109] In one embodiment, the presently disclosed calculator comprises a synchronizer operatively coupled to the SLM layer 101 and / or each of the at least one SEL 103a, 103b, 103c, 103d, 103e, the synchronizer configured to generate a first signal, wherein the memory 111 is configured to initiate an update of the pixel configuration of the SLM device 102 based at least on said first signal.

[0110] In one embodiment the presently disclosed calculator comprises a first EM radiation source, such a pulsed laser source, wherein the first EM radiation source provides at least one pulse and wherein the at least one SEL is configured to emit the first combined modulated EM radiation 107 or further combined EM radiations in synchronization with the at least one pulse. In one embodiment the memory Ill is configured to initiate an update of pixel configuration of the SLM device 102 based on arrival of the first combined modulated EMradiation 107 at the SLM layer. Once the first combined modulated EM radiation 107 has reached the SLM and the SLM has obtained a second combined modulated EM radiation, pixel update of the SLM may be initiated, to save time and obtain faster calculations. This has advantage of using a same source for both emission of the incoming EM radiation 109 and for synchronization and does not require wiring between SEL and SLM.

[0111] In one embodiment the presently disclosed calculator further comprises a synchronizer operatively coupled to the SLM layer (101) and / or each of the at least one SEL (103a, 103b, 103c, 103d, 103e), the synchronizer configured to generate at least a first signal, the first signal comprising at least an integer number of occurrences, wherein the at least one SEL is configured to initiate emitting the first combined EM radiation or further combined EM radiations responsive to at least one occurrence of said first signal, and / or the SLM layer is configured to initiate update of pixel configuration of the SLM device (102) responsive to at least one occurrence of said first signal.

[0112] In one embodiment said synchronizer may be an optical synchronizer. In one embodiment the synchronizer comprises a second EM radiation source, configured to generate at least one EM pulse for synchronization.

[0113] In one embodiment the synchronizer comprises an electronic signal generator, configured to generate at least one synchronization signal.

[0114] In one embodiment the synchronizer is electrically connected to the SLM layer and / or to the at least one SEL.

[0115] In one embodiment, a linear pixel size of the SLM device 102 is in a same order of magnitude of a wavelength of the EM radiation. . A pixel pitch, center-to-center distance, may be in the order of a wavelength of the EM radiation or smaller. The smaller the more control of the light or EM radiation but more calculations are needed to obtain the CGH.

[0116] In one embodiment, sensors and / or emitters 105i — 105nof the SEL 103a, 103b, 103c, 103d, 103e represent nodes of a programmable layer of a neural network, and pixels 102i-102nof the SLM device 102 represent coefficients of a weight matrix corresponding to connections parameters between layers of the neural network.

[0117] In one embodiment, emitters 105u (Upper emitters in Fig.6) of the first SEL 103e emit at a first wavelength, for example 500 nm, and emitters 105L (Lower emitters L in Fig. 6) of the second SEL 103d emit at a second wavelength, for example 400 nm, wherein the first wavelength is different from the second wavelength, and wherein the first SEL (103e) is configured to emit an EM radiation while still reading data at the first SEL, and / or the second SEL (103d) is configured to emit an EM radiation while still reading data at the second SEL.

[0118] In one embodiment, the first SEL (103e) and the second SEL (103d) have emitters working at different wavelengths, allowing emitting an EM radiation from the second SEL (103d) while still reading data at the second SEL and vice-versa. In this embodiment sensors of the SEL is / are covered by a filter 113, 114 to avoid that the sensor is affected by stray light or reflected light from the emitter from the same SEL. For example, if the first wavelength is 500 nm and the second wavelength is 400 nm, sensors of the first SEL may be covered by filters 114 that filter EM radiation corresponding to the first wavelength, and / or sensors of the second SEL may be covered by filters 113 that filter EM radiation at the second wavelength. This embodiment has the advantage of allowing parallel operations in the calculators, wherein a first portion of the calculation is performed at a first wavelength and a second portion of the calculation is performed at a second wavelength. Other embodiments comprise usage of W wavelengths, wherein W is an even integer number.

[0119] In one embodiment, the first and / or second EM radiation is visible light, or ultra-violet, UV, EM radiation, or infra-red, IR, EM radiation, or radio-frequency, RF, EM radiation, or X-ray EM radiation.

[0120] In one embodiment the incoming EM radiation (109) from the first EM radiation source is a coherent EM radiation.

[0121] and the EM radiation source is a laser or a pulsed laser.

[0122] The first or second SEL may be configured to emit a third combined modulated EM radiation based on the second combined modulated EM radiation and based on processing of read values.In one embodiment the second combined modulated EM radiation represents a second numerical vector or array corresponding to a result of a vector-matrix multiplication of the first data vector and the second matrix or a vector-vector multiplication of the first data vector and the second vector.

[0123] In one embodiment non-linear operations may be one or more of the following: logarithm, inversion, square root, exponential, rectified linear unit operation, maximum search operation, sigmoid function, maxpool, avgpool, Rectified Linear Unit (ReLu).

[0124] In one embodiment the at least one SEL and the SLM layers are planar layers. In one embodiment the at least one SEL and the SLM layers are planar layers facing each other, by a distance of a few mm, in some embodiments preferably less than 10 mm. The SEL and SLM being planar and facing to each other has the advantage of a compact design suitable for integration in a microchip.

[0125] In one embodiment the at least one emitters in the at least one SEL comprise a plurality of emitters

[0126] In one embodiment the at least one sensor in the at least one SEL comprise a plurality of sensors.

[0127] In one embodiment the SLM layer comprises a polarizer and / or an anti -reflective coatings to reduce unwanted reflections from the SLM. This is particular advantageous in the embodiments where the first SEL is transparent. When EM radiation 107 passes through the SLM layer it turns into EM radiation 108 on the other side. However, if some part is reflected back towards first SEL 103a it may disturb the calculation. This can be reduced using antireflective coatings.

[0128] In one embodiment the at least one SEL is based on a Light Field Display, LFD, system.

[0129] In one embodiment the at least one emitter of the at least one SEL comprise micro machines mechanical systems, MEMS.In one embodiment the at least one emitter of the at least one SEL comprise phase shifting material.

[0130] In one embodiment the at least one emitter of the at least one SEL comprise integrated micro lasers.

[0131] Material between the at least one SEL and the at least one SLM layer may be air, inert gas, transparent glass, transparent glue, transparent plastic. Hard materials have advantages in withholding mechanical stress. Air has the advantage of being light. Inert gas has the advantage of being light durable. The material can be chosen with a refractive index to suitably control direction of Em radiations 107 and / or 108.

[0132] A planar size of the at least one SEL and / or the SLM layer may be lOxlOmm or less, more preferably 5x5mm or less.

[0133] Embodiments of this disclosure relate to methods for operating the presently disclosed calculator. The presently disclosed method has the advantage of providing calculations of neural networks, which often require a number of n iterations, being n an integer number, within the presently disclosed calculator, which can be integrated in a microchip, in a time efficient way.

[0134] Fig. 4 illustrates a flow diagram of one method 300 for operating a embodiments of the presently disclosed calculator, the method comprising: emitting first combined modulated EM radiation from the first SEL to the SLM layer 301; further modulating, at the SLM layer 101, the first combined modulated EM radiation 107 to obtain the second combined EM radiation 302; sensing, at the first or at the second SEL, the second combined modulated EM radiation 303. The method 300 may be repeated an integer number of times, wherein said integer number is predetermined or corresponds to an exit condition, wherein said exit condition may be an end of a neural network calculation, such as inference.

[0135] Fig. 5a illustrates a flow diagram of some steps of one method for operating embodiments of the presently disclosed calculator.The presently disclosed method 300, 400 further comprises: loading Neural Network (NN) parameters as CGH patterns onto SLM memory 401; loading parameters corresponding to first data vector in first SEL 402. Actions or steps 401 and 402 are preparatory actions to set an initial state of the calculator, before the calculator can iteratively operate to perform all calculations required, such as operations required in the neural network, such as inference and / or learning. Preparation or setup phase 400 comprises steps 401 and 402. CGH patterns correspond to pixel values of the SLM device and represent matrix coefficients. Steps 401 and / or 402 are preparatory steps, preferably performed before steps 301-303 of the presently disclosed method. In one embodiment, loading Neural Network (NN) parameters as CGH patterns onto memory 401 means that NN parameters are converted to CGH patterns before being loaded in memory.

[0136] Fig. 5b illustrates a flow diagram of one method for operating embodiments of the presently disclosed calculator. In one embodiment, steps 401 and / or 402 are preparatory steps, preferably performed before steps of Fig. 5b of the presently disclosed method.

[0137] In one embodiment the presently disclosed calculator 100 may comprise a source for coherent EM radiation generation, or a source for incoming EM radiation 109, such as a pulsed laser source, configured to emit n laser pulses, being n an integer number corresponding to a number of iterations in a calculation of the presently disclosed calculator. Integer n may be predetermined, predefined, or it may correspond to an exit condition of the calculations.

[0138] A first pulse 50 h from said laser source may initiate or trigger steps 602i - 606i of the method illustrated in Fig. 5b. The presently disclosed method may comprise initiating steps 6021 - 606i triggered based on a first pulse 50 L, said first pulse being generated by a source of coherent EM radiation generation, such as a synchronizer or such as a first EM radiation source for incoming EM radiation 109.

[0139] A second pulse 50h from said laser source may initiate or trigger steps 6022 - 6O62 of the method illustrated in Fig. 5b. The presently disclosed method may comprise initiating steps 6022 - 6O62 triggered based on a second pulse 50h, said first pulse being generated by asource coherent EM radiation generation, such as a synchronizer or such as a first EM radiation source for incoming EM radiation 109.

[0140] A nth pulse, 501n, with n being an integer number, from said laser source may initiate or trigger steps 602n - 606n of the method illustrated in Fig. 5b. The presently disclosed method may comprise initiating steps 602n- 606ntriggered based on a nth pulse 50 ln, said pulse being generated by a source coherent EM radiation generation, such as a synchronizer or such as a first EM radiation source for incoming EM radiation 109.

[0141] After step n, at step n+1, the presently disclosed method may comprise step 607 of the method illustrated in Fig. 5b.

[0142] In other words, the presently disclosed method may comprise steps 602i - 606i of Fig. 5b. The presently disclosed method may further comprise steps 6022 - 6O62 of Fig. 5b.

[0143] The presently disclosed method may further comprise steps 602n- 606nof Fig. 5b.

[0144] The presently disclosed method may further comprise step 607 of Fig. 5b.

[0145] The presently disclosed method may further comprise steps 401 - 402 of Fig. 5a.

[0146] At step 602i the first SEL of the presently disclosed calculator emits the first combined EM radiation 107. Step 602i of Fig. 5b corresponds to step 301 of Fig. 4. Steps 602i corresponds to emitting the first combined EM radiation 107 from the first SEL based on a pre-loaded first data vector, wherein said first data vector was loaded at step 402 of setup or preparation phase 400. If no preparation phase 400 has been done, the first data vector may be random or null at step 602i. In one embodiment, the incoming EM radiation 109 comprises the first pulse 5011, and said first pulse synchronizes and triggers emitters of the first SEL to modulate the incoming EM radiation 109 and emit the first combined EM radiation 107. Wherein the incoming EM radiation 109 may be a pulsed laser. In another embodiment, where emitters of the first SEL are synchronized by synchronization means and no incoming EM radiation 109 is required, the source for coherent EM generation may be a synchronizer only used for generating synchronization signals, wherein synchronization signals may comprise EM pulses, such as laser pulses.At step 603i the SLM further modulates the first combined EM radiation 107 to obtain the second combined EM radiation 108. Step 6031 corresponds to further modulating, at the SLM, the first combined EM radiation 107 to obtain the second combined EM radiation 108. In other words, step 6031 in Fig. 5b corresponds to step 302 of Fig. 4. At step 6031 the obtained second combined EM radiation 108 represents a result of a matrix to vector multiplication between the first data vector and the pre-loaded matrix in the SLM, said matrix having been loaded during the setup or preparation phase 400 at step 401. If no preparation phase 400 has been done, pixels at the SLM device may have random or null values at step 6031.

[0147] At step 604i the memory 111 in the SLM initiates update of pixels in the SLM device 102 responsive to the first combined EM radiation 107 or to the second combined EM radiation 108.

[0148] Step 604i corresponds to initiating update or updating pixels in the SLM device 102. In step 604i the SLM device is loaded with a second CGH or second pixel values, corresponding to a second matrix. Said second matrix will be used at step 6032. Step 604i may be run in parallel to step 6031.

[0149] At step 605i the first or second SEL senses the second combined EM radiation. Step 6051 corresponds to sensing and or reading values from the second combined EM radiation. That is step 605i in Fig. 6 corresponds to step 303 in Fig. 4.

[0150] At step 6O61 the first or second SEL reads and processes read values into processed values, wherein processing read values may comprise applying nonlinearities. Step 6O61 corresponds to reading first values at sensors of the SEL and processing read values to obtain processed values or second values at the at least one SEL.

[0151] A second pulse 50h from said laser source may initiate steps 6022 - 6O62 of the method illustrated in Fig. 5b.

[0152] Steps 6022 - 6O62 of the method are similar to steps 602i - 6O61 of the same method.In some embodiments steps 6022, 6052, 6O62 of the method are performed by the first SEL, that is, in the embodiments that comprise only one SEL 103c and steps 6032, 6042 are performed by the SLM.

[0153] In some embodiments, that, for example comprise a first and a second SEL, the following actions or steps are performed.

[0154] At step 6022 the second SEL of the presently disclosed calculator emits the third combined EM radiation 110. Steps 6022 corresponds to emitting the third combined EM radiation 110 from the second SEL based on processed data values or second values, wherein said processed data values are a result of processing values sensed at step 6051, wherein processing 6O61 may comprises applying nonlinearities. In one embodiment the incoming EM radiation 109 comprises the second pulse 5012 and said second pulse synchronizes emitters of the first SEL to emit the third combined EM radiation 110. In another embodiment, the source for coherent EM generation may be a synchronizer only used for generating synchronization signals, wherein synchronization signals may comprise EM pulses, such as laser pulses.

[0155] At step 6032 the SLM further modulates the third combined EM radiation 110 to obtain a fourth combined EM radiation. Steps 6032 corresponds to further modulating, at the SLM, the third combined EM radiation 110 to obtain a fourth combined EM radiation. At step 6032 the obtained fourth combined EM radiation represents a result of a second matrix to vector multiplication between a data vector corresponding to processed values at step 6O61 and the second matrix, or CGH, or SLM pixel values, loaded in the SLM device at step 604i.

[0156] At step 6042 the memory 111 in the SLM initiates update of pixels in the SLM device 102 responsive to the fourth combined EM radiation or to a further combined EM radiation.

[0157] Step 6042 corresponds to initiating update or updating pixels in the SLM device 102. In step 6042 the SLM device is loaded with a third CGH or third pixel values, corresponding to a third matrix. Said third matrix will be used at step 603s (not shown in the figure).

[0158] At step 6052 the first or second SEL senses the fourth combined EM radiation. Step 6052 corresponds to sensing and / or reading values from the fourth combined EM radiation.At step 606i the first or second SEL processes read values into processed values, wherein processing read values comprises applying nonlinearities. Step 6O62 corresponds to processing read values to obtain processed values at the at least one SEL.

[0159] The method further comprises generating n pulses 50 ln, where n is an integer that is based on a required number of iterations, said n based for example on a depth L of a Neural Network.

[0160] For each integer k comprised between 1 and n and including 1 and n, a kth pulse, 501k, from said laser source may initiate steps 602k - 606k (not sown in the figure) of the method illustrated in Fig. 5b.

[0161] The method may further comprise at step n+1, outputting 607 a result of a full calculation, said calculation comprising n iterations, wherein outputting the result may be done electronically by the first or the second SEL via an interface. The first and / or second SEL may comprise an interface for outputting the result of the calculations. Step 607 is not necessarily triggered by a laser pulse. Step 607 may be performed when all steps 1-n, that is all steps triggered by pulses 5011-50 ln, have finished to be performed, with n being a predetermined integer, or with n corresponding to a number of steps satisfying an exit condition. The exit condition may be, for example, a conclusion of a neural network calculation such as inferring. In one embodiment, output step 607 may happen in synchronization with pulse 50 ln+i.

[0162] The inventors have realized that, by means of the optimized synchronization based on pulsed laser, the presently disclosed method has the advantage to obtain a faster and optimized use of the presently disclosed calculator, therefore complete neural network calculations such as inference and / or learning comprising n iterations are possible in a relatively compact calculator that is configured for being integrated in a consumer electronic device or apparatus, at a relatively fast speed.

[0163] The embodiments where the pulsed laser is also generating the incoming EM radiation 109, has the advantage of being simple as a same source is used both for triggering and synchronizing the SEL and for emitting the first combine EM radiation 107 by modulation at the first SEL by means of the SEL’s emitters. Such simplicity contributes to compactness and ease of integration.In one embodiment, emitting first combined modulated EM radiation 107 comprises emitting independently, at the at least one emitter 1051 - 105nof the SEL 103 a, 103b, 103 c, 103d, 103e, EM radiation with amplitude and / or phase based on a first data vector. When the at least one emitter of the at least one SEL comprises a plurality of emitters, each emitter emits independently according to an amplitude and phase for each emitter, to obtain a combined EM radiation which is the combination of the EM radiations emitted by each emitter.

[0164] In one embodiment, further modulating, at the SLM layer 101, the first combined modulated EM radiation 107 to obtain the second combined EM radiation 108, comprises independently modulating, by independent amplitude and / or phase, at pixels 102i-102nof the SLM device 102, the first combined modulated EM radiation (107). Pixels of the SLM device, which are or relative to a CGH, modulate independently the first combined modulated EM radiation 107 to obtain the second combined EM radiation 108. That is each pixel is configured to further modulate the first combined modulated EM radiation 107 independently, based on amplitude and / or phase values corresponding to a given CGH. If, for example, the CGH comprises a plurality of complex coefficients of a matrix, pixels of the SLM device independently modulate the first combined modulated EM radiation 107 in amplitude and phase corresponding to said coefficients.

[0165] In one embodiment, emitting first combined modulated EM radiation 107 to the at least one SLM layer 101 comprises modulating an incoming EM radiation 109 from a first EM radiation source to provide the first combined modulated EM radiation 107, by, at the at least one EM emitters, independently modulating the incoming EM radiation in amplitude and / or phase, based on a value of a first data vector. In embodiments where the first SEL receives an incoming EM radiation EM, each emitter modulates said incoming EM radiation independently in amplitude and phase, said amplitude and phase being based on complex values of a first data vector. The inventors have realized that in these embodiments where the incoming EM radiation triggers the emitters of the SEL, no additional synchronization of the emitters within the SEL is needed as the emitters act in synchronization with pulses of the incoming EM radiation.

[0166] In one embodiment, emitting first combined modulated EM radiation 107 to the at least one SLM layer 101 comprises emitting, at each emitter of the SEL, independent EMradiation, wherein each emitter emits an EM radiation with same wavelength, and amplitude and phase corresponding to a complex value of a first data vector, wherein emitters are active and are synchronized to each other in order to be able to emit a coherent first combined modulated EM radiation 107 to the at least one SLM layer 101.

[0167] In one embodiment, emitting, at the at least one SEL, the first combined modulated EM radiation 107 is in synchronization with or responsive to at least one pulse of the incoming EM radiation 109 and / or emitting, at the SEL, further combined modulated EM radiations, is in synchronization with at least one pulse, such as a further pulse, of the incoming EM radiation 109.

[0168] In one embodiments the incoming EM radiation 109 may comprise EM radiation pulses, such as laser pulses, and pulses of the incoming EM radiation (109) may trigger emission of combined EM radiation at the at least one SEL. For example a first pulse may trigger emission of the first combined modulated EM radiation 107 and further pulses may trigger further combined modulated EM radiation 107.

[0169] In one embodiment, emitting the first combined EM radiation or further combined EM radiations is responsive to at least one occurrence of a first signal, and / or initiating updating of pixel configuration of the SLM device 102 is responsive to at least one occurrence, such as another occurrence, of said first signal. In one embodiment, a first signal, such as an optical or electrical system, may synchronize and time operations of the presently disclosed calculator.

[0170] In one embodiment, the method (300) may further comprise: upon sensing the second combined modulated EM radiation 108, reading, at the at least one SEL 103a, 103b, 103c, 103d, 103e, first values from sensors 104i - 104nof the at least one SEL 103a, 103b, 103c, 103d, 103e; processing said first values to obtain second values; driving, by the at least one SEL 103 a, 103b, 103 c, 103 d, 103e, emitters (105i — 105n) of the at least one SEL 103 a, 103b, 103c, 103d, 103e to emit EM radiation based on the processed second values, collectively emitting a third combined modulated EM radiation 110. In this disclosure, said second values are also referred to as processed values.

[0171] In one embodiment, the first or second SEL reads values corresponding to the second combined EM radiation 108 from the SLM to the first or second SEL respectively. Said first values correspond to a matrix by vector multiplication. Further, the SEL processes said values, preferably using nonlinear operations. Further, the SEL drives emitters to emit EMradiation based on said processed values, to obtain a third combined EM radiation towards the SEL.

[0172] In one embodiment, processing first values to obtain second values (or processed values) comprises applying nonlinear operations.

[0173] In one embodiment, the presently disclosed method, further comprises initiating update of pixel configuration of the SLM device 102 upon emitting, at the first SEL, the first combined EM radiation or upon obtaining, by the SLM layer (101), the second combined EM radiation or upon sensing, at the at least one SEL (103a, 103b, 103c, 103d, 103e), the second combined EM radiation (108). In this embodiment, as soon as the first combined EM radiation is being emitted or arrives at the SLM, the SLM initiates update of the pixels for a next iteration. That corresponds to steps 604i-604nin Fig. 5b. In one embodiment the SLM initiates update of pixel configuration of the SLM device 102 upon obtaining, by the SLM layer (101), the second combined EM radiation or upon sensing, at the at least one SEL 103a, 103b, 103 c, 103 d, 103e, the second combined EM radiation 108. In any of these embodiments, synchronization of the SLM and the SEL is automatic and is based on the emitted EM radiation.

[0174] The presently disclosed method further comprises updating pixel configuration of the SLM device 102 by usage of a memory 111 integrated in the SLM layer 101, said memory 111 storing coefficients corresponding to pixel configurations of the SLM device.

[0175] In one embodiment, memory Ill is integrated in the SLM layer. In one embodiment memory Ill is loaded with pixel values of the SLM device. Loading of the memory 111 with pixel values, or GCH, of the SLM device corresponds to step 401 in Fig. 5a. The inventors have realized that said loading at step 401 is advantageously performed before emitting the first combined EM radiation and, in general, before performing any of the steps in Fig. 5b. The inventors realize that said loading at step 401 is a preparatory step. In one embodiment memory Ill is large enough to contain all pixel values for all iterations of a calculation of the calculator, with the advantage that that memory 111 may be loaded only once initially. In one embodiment memory 111 may contain only pixel values corresponding to a portion P of the n iterations: in this case the calculation is interrupted at step P, memory is loaded withremaining values, and further iterations are executed. In one embodiment, memory 111 may be an analog memory, which provides fast writing of the pixels to the SLM device. The inventors realize that a time delay of one iteration of the calculator depends on the time it takes for the memory 111 to transfer pixel values, that is CGH, to the SLM device. Memory 111 is integrated in the SLM device, that has the advantage to reduce the time it takes for the memory 111 to write or transfer pixel values to the SLM device and it contributes advantageously to a compact design that can be integrated in electronic devices or apparatuses for consumer products such as AR glasses.

[0176] In one embodiment, the method further comprises synchronizing at least one operation of the SLM layer with at least one operation of the SEL. Synchronization is achieved by use pulses of the incoming EM radiation 109 and / or by emission or reception of EM radiation from SEL to SLM. In another embodiment, the calculator may comprise a synchronizer emitting a synchronization signal, and operations of SEL and / or SLM may be synchronized and / or triggered by occurrences of said synchronization signal, which has the advantage of avoiding wiring between SEL and SLM. In one embodiment said signal may be an optical signal.

[0177] In one embodiment, the presently disclosed method further comprises programming emitters 1051 - 105nof the at least one SEL 103 a, 103b, 103 c, 103 d, 103e, with values representing nodes of a programmable layer of a neural network, and timely programming the SLM device 102 with values representing a weight matrix corresponding to connections parameters between layers of the neural network, wherein timely programming the SLM device corresponds to updating pixel values of the SLM device by means of memory 111. The inventors have realized that emitters and / or sensors of the presently disclosed calculator may represent nodes of a neural network and SLM device may represent a weight matrix corresponding to connections parameters between layers of the neural network. This way, in one embodiment, the presently disclosed method may perform operations related to neural network, such as inference and / or learning.

[0178] In one embodiment, the presently disclosed method further comprises emitting an EM radiation from a first SEL 103e while, at said first SEL still reading data from a second SEL 103d and vice-versa, and wherein EM radiation emitted by the first SEL 103e has at least one different wavelength than at least one wavelength of an EM radiation emitted by the secondSEL 103d. In one example, emitting EM radiation from a first SEL at a first wavelength, for example 500 nm, may be done while still receiving or processing data at the first SEL from an EM radiation at a second wavelength, for example 400 nm. In this embodiment, the SLM is configured to independently modulate EM radiation at a first wavelength and a EM radiation at a second wavelength. In this embodiment, parallel calculations of independent branches of a same neural network, or separate neural networks, may be performed, with clear advantages on the calculation time and parallelism of calculations, which is improved as compared to the embodiment where only one wavelength is used.

[0179] In one embodiment, an even number greater than 2 of wavelengths is used, which further increases the level of parallelism.

[0180] In one embodiment, an apparatus 200 comprises at least one embodiment of the presently disclosed calculator, wherein said apparatus is a user equipment, UE, Augmented reality, AR, glasses, Virtual Reality, VR, glasses, or a network node or a repeater for wireless communications or a laptop or a computing device or a consumer electronics device or an electrical device or an electronic device.

Claims

CLAIMS1. A calculator (100) comprising:at least one spatial light modulator, SLM, layer (101), comprising at least one programmable SLM device (102);at least one sensor-and-emitter layer, SEL, (103a, 103b, 103c, 103d, 103e) each of the at least one SEL (103a, 103b, 103c, 103d, 103e) comprising at least one electromagnetic, EM, radiation sensor ( 104i - 104n), at least one programmable EM radiation emitter ( 1051 - 105n) and a processing unit (106),wherein a first SEL (103 a, 103 c, 103e) of the at least one SEL (103 a, 103b, 103 c, 103d, 103e) is configured to emit first combined modulated EM radiation (107) to the at least one SLM layer (101),wherein the SLM device (102) is configured to further modulate the first combined modulated EM radiation (107) to obtain a second combined modulated EM radiation (108), wherein the first SEL (103 a, 103 c, 103e) or a second SEL (103b) of the at least one SEL (103a, 103b, 103c, 103d, 103e) is configured to sense the second combined modulated EM radiation (108) from the SLM layer (101).

2. The calculator (100) according to claim 1, wherein the first combined modulated EM radiation (107) spatially represents at least a first data vector, and / or wherein each one of the at least one EM emitter ( 1051 - 105n) in the at least one SEL (103 a, 103b, 103 c, 103d, 103e) is programmable and is configured to independently emit EM radiation with amplitude and / or phase based on the first data vector.

3. The calculator (100) according to claim 1 or 2, wherein pixels (102i-102n) of the SLM device (102) are programmable, and / or, wherein pixels (102i-102n) of the SLM device correspond to elements or coefficients of a second vector or a first matrix.

4. The calculator (100) according to any one of the preceding claims, wherein each pixel (102i-102n) of the SLM device is configured to independently modulate, by independent amplitude and / or phase, the first combined modulated EM radiation (107), to obtain the second combined modulated EM radiation (108).

5. The calculator (100) according to any one of the preceding claims, wherein the first SEL (103a, 103b, 103c, 103d, 103e) is further configured to modulate an incoming EM radiation (109) from a first EM radiation source to provide the first combined modulated EM radiation (107), wherein each one of the at least one EM emitter ( 1051 - 105n) is configured to independently modulate the incoming EM radiation (109) in amplitude and / or phase based on at least one value of the first data vector.

6. The calculator (100) according to claim 5, wherein the first EM radiation source provides at least one pulse and wherein the at least one SEL is configured to emit the first combined modulated EM radiation (107) in synchronization with or responsive to a first pulse of the at least one pulse and / or the at least one SEL is configured to emit further combined EM radiations in synchronization with or responsive to further pulses of the at least one pulse.

7. The calculator (100) according to any one of claims 1-4, further comprising a synchronizer operatively coupled to the SLM layer (101) and / or each of the at least one SEL (103a, 103b, 103c, 103d, 103e), the synchronizer configured to generate at least a first signal, the first signal comprising at least an integer number of occurrences, wherein the at least one SEL is configured to initiate emitting the first combined EM radiation or further combined EM radiations responsive to at least one occurrence of said first signal, and / or the SLM layer is configured to initiate update of pixel configuration of the SLM device (102) responsive to at least one occurrence of said first signal.

8. The calculator (100) according to any one of the preceding claims, wherein each of the at least one SEL (103a, 103b, 103c, 103d, 103e) is further configured to:read first values from sensors ( 104i - 104n) of the SEL;process said first values to obtain second values;drive emitters ( 1051 - 105n) of the SEL to emit EM radiation based on the processed second values, collectively emitting a third combined modulated EM radiation (110) towards the SLM layer.

9. The calculator (100) according to claim 8, wherein processing first values to obtain second values comprises applying nonlinear operations.

10. The calculator (100) according to any one of claims 8-9, wherein the at least one SEL (103a, 103b, 103c, 103d, 103e) is configured to emit the third combined EM radiation when or after a refresh time of the SLM device (102) has elapsed.

11. The calculator (100) according to any one of the preceding claims, wherein pixel configuration of the SLM device (102) is stored in a memory (111) operatively coupled to the SLM device, and wherein the SLM layer comprises said memory.

12. The calculator (100) according to claim 11, wherein said memory (111) is operatively coupled to the at least one SEL (103a, 103b, 103c, 103d, 103e) for at least synchronization.

13. The calculator (100) according to any one of claims 11-12, wherein the memory (111) is configured to initiate an update of pixel configuration of the SLM device (102) responsive to arrival of the first combined modulated EM radiation (107) at the SLM layer.

14. The calculator (100) according to any one of the preceding claims, wherein sensors and / or emitters of the SEL (103a, 103b, 103c, 103d, 103e) represent nodes of a programmable layer of a neural network, and pixels (102i-102n) of the SLM device (102) represent coefficients of a weight matrix corresponding to connections parameters between layers of the neural network.

15. The calculator (100) according to any one of the preceding claims, wherein emitters of the first SEL (103e) emit at a first wavelength and emitters of the second SEL (103 d) emit at a second wavelength, wherein the first wavelength is different from the second wavelength, and wherein the first SEL (103e) is configured to emit an EM radiation while still reading data at the first SEL, and / or the second SEL (103d) is configured to emit an EM radiation while still reading data at the second SEL.

16. A method (300) of operating a calculator according to any one of claims 1-15, the method comprising:emitting first combined modulated EM radiation (107) from the first SEL (103a, 103b, 103c, 103d, 103e) tothe SLM layer (301);further modulating, at the SLM layer (101), the first combined modulated EM radiation (107) to obtain the second combined EM radiation (108); sensing, at the first SEL or at the second SEL (103a, 103b, 103c, 103d, 103e), the second combined modulated EM radiation (108).

17. The method (300) according to claim 16, wherein emitting first combined modulated EM radiation (107) comprises emitting independently, at the at least one emitter ( 1051 - 105n) of the SEL (103 a, 103b, 103 c, 103 d, 103e), EM radiation with amplitude and / or phase based on a first data vector.

18. The method (300) according to any one of claims 16-17, wherein further modulating, at the SLM layer (101), the first combined modulated EM radiation (107) to obtain the second combined EM radiation (108), comprises independently modulating, by independent amplitude and / or phase, at pixels (102i-102n) of the SLM device (102), the first combined modulated EM radiation (107).

19. The method (300) according to any one of claims 16-18, wherein emitting first combined modulated EM radiation (107) to the at least one SLM layer (101) comprises modulating an incoming EM radiation (109) from a first EM radiation source to provide the first combined modulated EM radiation (107), by, at emitters of the at least one SEL, independently modulating the incoming EM radiation in amplitude and / or phase, based on a value of a first data vector.

20. The method (300) according to claim 19, wherein emitting, at the at least one SEL, the first combined modulated EM radiation (107) is in synchronization with or responsive to at least one pulse of the incoming EM radiation (109) and / or emitting, at the SEL, further combined modulated EM radiations, is in synchronization with at least one pulse of the incoming EM radiation (109).

21. The method (300) according to any one of claims 16-18, wherein emitting the first combined EM radiation or further combined EM radiations is responsive to at least oneoccurrence of a first signal, and / or initiating updating of pixel configuration of the SLM device (102) is responsive to at least one occurrence of said first signal.

22. The method (300) according to any one of claims 16-21, further comprising:- upon sensing the second combined modulated EM radiation (108), reading, at the at least one SEL (103a, 103b, 103c, 103d, 103e), first values from sensors (104i- 104n) of the at least one SEL (103a, 103b, 103c, 103d, 103e); processing said first values to obtain second values;driving, by the at least one SEL (103a, 103b, 103c, 103d, 103e), emitters (1051 - 105n) of the at least one SEL (103 a, 103b, 103 c, 103 d, 103e) to emit EM radiation based on the processed second values, collectively emitting a third combined modulated EM radiation (110).

23. The method (300) according to claim 22, wherein processing first values to obtain second values comprises applying nonlinear operations.

24. The method (300) according to any one of claims 16-23, further comprising initiating update of pixel configuration of the SLM device (102) upon emitting, at the first SEL, the first combined EM radiation or upon obtaining, by the SLM layer (101), the second combined EM radiation or upon sensing, at the at least one SEL (103a, 103b, 103c, 103d, 103e), the second combined EM radiation (108).

25. The method (300) according to any one of claims 16-22, further comprising updating pixel configuration of the SLM device (102) by usage of a memory (111) integrated in the SLM layer (101), said memory (111) storing coefficients corresponding to pixel configurations of the SLM device.

26. The method (300) according to any one of claims 16-25, further comprising synchronizing at least one operation of the SLM layer with at least one operation of the SEL.

27. The method (300) according to any one of claims 16-26, further comprising programming emitters ( 1051 - 105n) of the at least one SEL (103 a, 103b, 103 c, 103 d, 103e), with values representing nodes of a programmable layer of a neural network,and timely programming the SLM device (102) with values representing a weight matrix corresponding to connections parameters between layers of the neural network.

28. The method (300) according to any one of claims 16-27, further comprising emitting an EM radiation from a first SEL (103e) while, at said first SEL still reading data at the sensors of said first SEL, and wherein EM radiation emitted by the first SEL (103a) has at least one different wavelength than at least one wavelength of an EM radiation emitted by the second SEL (103d).

29. An apparatus (200) comprising the calculator (100) according to any one of claims 1- 15, wherein said apparatus is a user equipment, UE, Augmented Reality, AR, glasses, Virtual Reality, VR, glasses, or a network node or a repeater for wireless communications or a laptop or a computing device or a consumer electronics device or an electrical device or an electronic device.

Images