Spatial diffraction optical neural network processor

Abstract

The invention provides a spatial diffraction optical neural network processor, and relates to the technical field of spatial diffraction. The spatial diffraction optical neural network processor comprises a light field signal input unit used for converting to-be-calculated data into an input light signal and transmitting the input light signal; the first spatial diffraction optical neural network processing unit and the second spatial diffraction optical neural network processing unit are symmetrically arranged and are used for receiving input optical signals to carry out optical field regulation and control operation and reflecting and outputting, and phase regulation and control units are integrated on the first spatial diffraction optical neural network processing unit and the second spatial diffraction optical neural network processing unit; each phase regulation and control unit is used for carrying out light field regulation and control operation on an input light signal; the phase regulation and control units integrated on the first and second spatial diffraction optical neural network processing units are arranged in a staggered manner so as to realize optical signal connection through a reflection law; and the photoelectric detector array is used for converting the input optical signal subjected to the optical field regulation and control operation of the first and second spatial diffraction optical neural network processing units into an electric signal.

Description

Technical Field

[0001] The present application relates to the field of spatial diffraction technology, and in particular, to a spatial diffraction optical neural network processor. Background Art

[0002] The complexity of intelligent algorithms represented by neural networks has shown explosive growth, and the huge computing power requirements have brought great pressure to existing digital processors. Optical neural networks use complex light fields as carriers for neural network calculations and data processing, and have unique advantages such as photon transmission, low crosstalk, low transmission delay, high-speed conductivity, weak coupling with ordinary environments, and powerful parallel processing capabilities. Therefore, optical neural network processors can effectively circumvent the energy and time consumption problems of data access in the von Neumann architecture, and become a highly potential new generation of computing candidate technology to alleviate the contradiction between the growth rate of Moore's Law and the growth rate of computing power requirements.

[0003] As the complexity of executable tasks continues to grow, the number of optical elements involved in optical neural networks shows a quadratic growth trend with the size of the computing matrix, which puts higher requirements on the integration and scalability of optical neural network processors, especially spatial diffraction optical neural network processors.

[0004] Currently, there is a physical implementation method of spatial diffraction optical neural network, which mainly manipulates the phase distribution of the light beam by sequentially arranging multiple phase control units in an open space in a layered structure, and realizes calculation and data processing by modulating the amplitude and phase parameters of the light field and transmitting the process. However, this scheme of sequentially arranging multiple phase control units in a layered structure has a large dissipation of light field energy. As the number of layered structures increases, the light field energy that reaches the subsequent phase control units for calculation decays, resulting in the problem that the reasoning accuracy of the optical neural network processor based on spatial diffraction optics decreases with the increase of the number of layers; at the same time, the construction of this layered spatial diffraction optical neural network often requires the use of large optical elements, which will increase the complexity of the task of increasing the network scale, and bring huge challenges to reducing the size of the processor and realizing structural integration, which will seriously limit the application of spatial diffraction optical neural network in the fields of autonomous driving, speech and image processing. Summary of the invention

[0005] In order to solve at least one of the above problems, the present application proposes an integrated and expandable spatial diffraction optical neural network processor.

[0006] According to the first aspect of the present application, at least one embodiment of the present application provides a spatial diffraction optical neural network processor, including: a light field signal input unit, used to convert the data to be calculated into an input light signal and transmit it; a first spatial diffraction optical neural network processing unit and a second spatial diffraction optical neural network processing unit, the first spatial diffraction optical neural network processing unit and the second spatial diffraction optical neural network processing unit are symmetrically arranged, used to receive the input light signal to perform light field control operations, and reflect the output, wherein: the first spatial diffraction optical neural network processing unit is integrated with N phase control units, and the second spatial diffraction optical neural network processing unit is integrated with M phase control units, each of the phase control units is used to perform light field control operations on the input light signal, and N and M are both integers greater than or equal to 1; the phase control units integrated on the first spatial diffraction optical neural network processing unit and the second spatial diffraction optical neural network processing unit are arranged in an interlaced manner to achieve optical signal connection through the law of reflection; a photodetector array, used to convert the input light signal that has passed the light field control operation of the first spatial diffraction optical neural network processing unit and the second spatial diffraction optical neural network processing unit into an electrical signal.

[0007] For example, in some embodiments of the present application, the light field signal input unit generates the input light signal by performing phase, amplitude, polarization, frequency, wavelength and / or angular momentum encoding processing on the data to be calculated.

[0008] For example, in some embodiments of the present application, the light field signal input unit includes: an array laser, a light emitting diode array, a liquid crystal display, a digital micromirror array illuminated by a single light source, and / or a liquid crystal spatial light modulator.

[0009] For example, in some embodiments of the present application, the photodetector array includes: a charge coupled device, a complementary metal oxide semiconductor detector and / or a photodiode array.

[0010] For example, in some embodiments of the present application, it also includes: a first lens, which is arranged between the light field signal input unit and the first spatial diffraction optical neural network processing unit; a second lens, which is arranged between the first spatial diffraction optical neural network processing unit or the second spatial diffraction optical neural network processing unit and the photodetector array.

[0011] For example, in some embodiments of the present application, the phase control unit has a reflective characteristic, and the phase control unit includes a reflective metasurface and / or a reflective spatial light modulator.

[0012] For example, in some embodiments of the present application, the phase control unit has a transmissive characteristic, and the phase control unit includes a transmissive metasurface and / or a transmissive spatial light modulator.

[0013] For example, in some embodiments of the present application, it also includes: a mirror reflection unit, which is arranged at the bottom of each of the phase control units.

[0014] For example, in some embodiments of the present application, the phase control unit includes a metasurface, a digital micromirror array and / or a spatial light modulator; the phase control unit includes L*Z optical neurons, each optical neuron is a pixel point on the metasurface, the digital micromirror array and / or the spatial light modulator, and L and Z are both integers greater than or equal to 1.

[0015] For example, in some embodiments of the present application, it also includes: a programmable control unit controller, which is connected to the first spatial diffraction optical neural network processing unit and the second spatial diffraction optical neural network processing unit to dynamically adjust each optical neuron, including adjusting the sound field, light field, temperature, voltage, current and / or magnetic field strength.

[0016] Through the above-mentioned example embodiments, the present application proposes an integrable and expandable spatial diffraction optical neural network processor. By adjusting the number and arrangement of phase control units on the symmetrically placed spatial diffraction optical neural network processing units, an integrable and expandable spatial diffraction optical neural network processor architecture is realized. Logically adjacent phase control units realize data connection through spatial diffraction optical transmission. The arrangement method can be given by the law of optical mirror reflection, allowing the flexible expansion of the processor computing scale by only changing the number of arrangement of phase control units and the size of the phase modulation weight matrix of the phase control units without the need for more optical elements. It can overcome the defects of the existing spatial diffraction optical neural network processor that is difficult to integrate and expand, enhance the multi-task parallel processing capability, greatly improve the integration of the processor, ensure high scalability and integration capability, and reduce the problem of light field energy dissipation caused by the increase of layered structures. The spatial diffraction optical neural network processor can be further integrated with intelligent computing scenarios such as autonomous driving and voice image processing.

[0017] The spatial diffraction optical neural network processor provided in the present application introduces a light field signal emission array in the input area for loading the light field data to be calculated, and introduces a photodetector array in the output area for receiving and outputting the results after the optical neural network operation. Among them, the input light field data and output accuracy can be numerically adjusted through common control devices such as 16 bits, 10 bits, and 8 bits, so as to realize the input-output all-optical neural network calculation with adjustable accuracy.

[0018] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] By describing in detail exemplary embodiments thereof with reference to the accompanying drawings, the above and other objects, features and advantages of the present application will become more apparent. The accompanying drawings described below are only some embodiments of the present application, and are not intended to limit the present application.

[0020] Figure 1 A schematic diagram showing an integrable and scalable spatial diffraction optical neural network processor according to an exemplary embodiment;

[0021] Figure 2 A schematic diagram showing a plurality of phase control units integrated on each spatial diffraction optical neural network processing unit of an exemplary embodiment;

[0022] Figure 3A A schematic diagram showing a logical connection mode of a spatial diffraction optical neural network processor using a reflective spatial diffraction optical neural network processing unit according to an exemplary embodiment;

[0023] Figure 3B Another embodiment of a schematic diagram showing a logical connection mode of a spatial diffraction optical neural network processor using an exemplary reflective spatial diffraction optical neural network processing unit is shown;

[0024] Figure 3C Another embodiment of a schematic diagram showing a logical connection mode of a spatial diffraction optical neural network processor using an exemplary reflective spatial diffraction optical neural network processing unit is shown;

[0025] Figure 4A A schematic diagram showing a logical connection mode of a spatial diffraction optical neural network processor using a transmission-type spatial diffraction optical neural network processing unit according to an exemplary embodiment;

[0026] Figure 4B Another embodiment of a schematic diagram showing a logical connection mode of a spatial diffraction optical neural network processor using an exemplary transmission-type spatial diffraction optical neural network processing unit is shown;

[0027] Figure 4C Another embodiment of a schematic diagram showing a logical connection mode of a spatial diffraction optical neural network processor using an exemplary transmission-type spatial diffraction optical neural network processing unit is shown;

[0028] Figure 5A A schematic diagram showing a spatial diffraction optical neural network processor including a programmable control unit controller according to an exemplary embodiment;

[0029] Figure 5BAnother embodiment of a schematic diagram showing an exemplary spatial diffractive optical neural network processor including a programmable control unit controller;

[0030] Figure 5C A schematic diagram showing a connection mode between a programmable control unit controller and a spatial diffraction optical neural network processing unit according to an example embodiment;

[0031] Fig. 6A A schematic diagram showing the classification task results;

[0032] Figure 6B A schematic diagram showing the results of the classification task;

[0033] Figure 6C A schematic diagram showing the classification task results;

[0034] Fig. 7A A schematic diagram showing a phase control unit of an exemplary embodiment;

[0035] Figure 7B A schematic diagram showing a phase modulation unit constructed by using a liquid crystal spatial light modulator after loading phase modulation weight parameters;

[0036] Figure 7C The three-dimensional structure layout of the phase modulation unit manufactured using photolithography technology is shown. DETAILED DESCRIPTION

[0037] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this application will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0038] The described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of these specific details, or other modes, components, materials, devices, etc. may be adopted. In these cases, known structures, methods, devices, implementations, materials or operations will not be shown or described in detail.

[0039] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0040] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0041] Those skilled in the art will appreciate that the drawings are merely schematic diagrams of example embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing the present application, and therefore cannot be used to limit the scope of protection of the present application.

[0042] Figure 1 A schematic diagram showing an integrable and scalable spatial diffraction optical neural network processor according to an exemplary embodiment.

[0043] like Figure 1 As shown, the integrable and expandable spatial diffraction optical neural network processor includes: a light field signal input unit 20, a first spatial diffraction optical neural network processing unit 40, a second spatial diffraction optical neural network processing unit 41 and a photodetector array 30.

[0044] The light field signal input unit 20 is used to convert the data to be calculated into an input light signal and transmit it. The spatial diffraction optical neural network processing unit can be implemented by a spatial light modulation device, a reconfigurable metasurface array, etc.

[0045] According to some embodiments, the light field signal input unit 20 includes at least one analog light field signal output device, which receives external input data to be calculated, converts the data to be calculated into an input light signal, and can encode light field parameters such as phase, amplitude, polarization, frequency, wavelength and / or angular momentum for different data to be calculated, so as to convert and load the data to be calculated into a numerical value in the complex domain represented by the analog light field signal, generate an input light signal and emit it. The light field signal input unit 20 includes: an array laser, a light emitting diode (LED) array, a liquid crystal display (LCD), a digital micromirror array (DMD) illuminated by a single light source, and / or a liquid crystal spatial light modulator (SLM). The emission area and modulation method of the input light signal can be configured according to the needs of the computing task.

[0046] N phase control units 50 are integrated on the first spatial diffraction optical neural network processing unit, and M phase control units 50 are integrated on the second spatial diffraction optical neural network processing unit, where N and M are both integers greater than or equal to 1. Each phase control unit 50 is used to perform light field control operations on the input optical signal, and can independently perform simulated light field input value modulation and spatial diffraction transmission. The phase control units 50 integrated on the first spatial diffraction optical neural network processing unit 40 and the second spatial diffraction optical neural network processing unit 41 are arranged in an interlaced manner to achieve optical signal connection through the law of reflection. Spatially adjacent phase control units are phase control units located on the left or right side of the phase control unit on the spatial diffraction optical neural network processing unit on this side, and logically adjacent phase control units are phase control units located on the spatial diffraction optical neural network processing unit on the opposite side, and the calculation relationship is the input or output of the phase control unit.

[0047] According to some embodiments, the spatial diffraction optical neural network processing unit on each side can configure the number and size of the phase control units on this side according to the computing task.

[0048] like Figure 3C and Figure 4C As shown, the single-sided spatial diffraction optical neural network processing unit integrates 1 phase control unit and 2 phase control units.

[0049] According to some embodiments, the spatial diffraction optical neural network processing unit on each side can implement multiplexing calculations of one or more light field parameters such as light field polarization direction and wavelength by integrating different types of phase control units.

[0050] According to example embodiments, the phase control unit 50 includes a metasurface, a digital micromirror array, and / or a spatial light modulator. Figure 2 As shown, the phase control unit 50 includes L*Z optical neurons 90, each optical neuron 90 corresponds to a pixel point on the metasurface, digital micromirror array and / or spatial light modulator, and L and Z are both integers greater than or equal to 1.

[0051] According to some embodiments, the phase control unit 50 includes N*N optical neurons 90, each of which has a corresponding phase modulation weight parameter, such as Fig. 7A As shown; Figure 7B A phase modulation unit constructed by loading phase modulation weight parameters using a liquid crystal spatial light modulator; Figure 7C The three-dimensional structure layout of the phase modulation unit can be used for manufacturing by photolithography technology.

[0052] After the light emitted by the light field signal input unit 20 is modulated by the optical neurons in the phase modulation unit, the multiple diffracted light rays emitted by the modulated optical neurons are transmitted to the optical neurons of the next layer of phase modulation units through reflection and spatial diffraction. Each neuron of the phase modulation unit has a set phase modulation weight.

[0053] The spatial diffraction optical neural network processing unit provided by the present application realizes data calculation and transmission between logically adjacent phase control units through the spatial diffraction transmission process of complex light field signals, and can realize the optical signal coupling function of adjacent phase control units without adding other spatial optical devices such as optical fibers and lenses. Among them, the control weights of optical neurons can be set by pre-setting weight parameters, and can be manufactured and integrated on semiconductor material substrates such as silicon or quartz glass with the help of optical machining technology, thereby constructing a spatial diffraction optical neural network processing unit.

[0054] The phase control unit array arrangement and the number of computing units on the spatial diffraction optical neural network processing unit provided in this application can be flexibly adjusted according to the scale and type of computing tasks. Each phase control unit is composed of a pre-designed phase modulation weight matrix, and each optical neuron can independently perform light field amplitude, phase and other parameter modulation, and realize calculation and data processing during the transmission process.

[0055] According to the exemplary embodiment, the spacing between the first spatial diffraction optical neural network processing unit 40 and the second spatial diffraction optical neural network processing unit 41 is D. After the operation data is output through the corresponding area of ​​the light field signal input unit 20, it is incident on the phase control unit 1 inside the first spatial diffraction optical neural network processing unit 40 at a fixed angle θ, and the phase-modulated light field is reflected to the phase control unit 1 inside the second spatial diffraction optical neural network processing unit 41 by light reflection. The center position offset distance Δ between the phase 1 inside the first spatial diffraction optical neural network processing unit 40 and the phase control unit 1 inside the second spatial diffraction optical neural network processing unit 41 is given by the reflection law: Δ=D×tanθ, and the spatial distance of the light field transmitted between the phase control units adjacent in the calculation logic can be calculated by the equation: l=D / cosθ. The reflected light field only participates in the connection between the phase control units with the same row number in the first spatial diffraction optical neural network processing unit 40 and the second spatial diffraction optical neural network processing unit 41, and the phase control units with adjacent row numbers have no connection relationship.

[0056] The spatial diffraction optical neural network processing unit 40 and the spatial diffraction optical neural network processing unit 41 can be silicon wafers, silicon nitride, quartz glass and other wafers used for semiconductor optical machining. The phase control unit 50 can be used to make optical neurons 90 on the surface of the wafer according to the phase control parameter matrix designed by the algorithm through photolithography, electron beam etching and other technologies; the already manufactured phase control unit can also be attached to the surface of the wafer through optical glue to achieve the integration and expansion of the phase control unit; it can also be a spatial light modulation device such as SLM, etc., through the programmable control unit controller 80 of the spatial light modulation device, the phase control parameters designed by the algorithm are loaded in a specific form to the specific area of ​​the spatial light modulation device to achieve the integration and expansion of the phase control unit, and each loading of a phase control parameter forms a corresponding optical neuron 90; it can also be a reconfigurable metasurface material, through the programmable control unit controller 80 to achieve the integration and expansion of the phase control unit 50, and adjust the intensity parameters such as the sound field, light field, temperature, voltage, current, magnetic field, etc. applied to the spatial diffraction optical neural network processing units 40 and 41, so that the phase control unit 50 loads the phase control parameter matrix to achieve the optical neuron 90.

[0057] The photodetector array 30 is used to convert the input optical signal after the light field control operation of the first spatial diffraction optical neural network processing unit 40 and the second spatial diffraction optical neural network processing unit 41 into an electrical signal, and realize nonlinear operation processing during the conversion process.

[0058] According to some embodiments, the photodetector array 30 includes a detector array that can realize photoelectric conversion, such as a charge coupled device (CCD), a complementary metal oxide semiconductor detector (CMOS) and / or a photodiode array. The area size of the photodetector array 30 can be configured according to the computing task requirements.

[0059] The spatial diffraction optical neural network processor provided in the present application introduces a light field signal emission array in the input area for loading the light field data to be calculated, and introduces a photodetector array in the output area for receiving and outputting the results after the optical neural network operation. Among them, the input light field data and output accuracy can be numerically adjusted through common control devices such as 16 bits, 10 bits, and 8 bits, so as to realize the input-output all-optical neural network calculation with adjustable accuracy.

[0060] The single-sided spatial diffraction optical neural network processor can integrate more than one reflective or transmissive phase control unit 50 .

[0061] In the case where the phase control unit 50 is a reflective type, the phase control unit has a reflective characteristic, such as Figure 3A-3C The phase control unit includes a reflective metasurface and / or a reflective spatial light modulator (SLM).

[0062] In the case where the phase control unit 50 is a transmissive type, the phase control unit has a transmissive characteristic. The phase control unit includes a transmissive metasurface and / or a transmissive spatial light modulator. A mirror reflection unit 70 is provided at the bottom of the phase control unit, such as Figure 4A-4C shown.

[0063] According to an exemplary embodiment, the spatial diffraction optical neural network processor further includes: a first lens 60 and a second lens 61. The first lens 60 is disposed between the light field signal input unit 20 and the first spatial diffraction optical neural network processing unit 40. The second lens 61 is disposed between the first spatial diffraction optical neural network processing unit 40 or the second spatial diffraction optical neural network processing unit 41 and the photodetector array 30. Figure 3B , Figure 4B , Figure 5B shown.

[0064] This application takes the example of the second lens 61 being arranged between the first spatial diffraction optical neural network processing unit 40 and the photodetector array 30, but the second lens 61 can also be arranged between the second spatial diffraction optical neural network processing unit 41 and the photodetector array 30, and this application is not limited to this.

[0065] According to an exemplary embodiment, the spatial diffraction optical neural network processor further includes: a programmable control unit controller 80, such as Figure 5A , Figure 5B The programmable control unit controller 80 is connected to the first spatial diffraction optical neural network processing unit 40 and the second spatial diffraction optical neural network processing unit 41 to dynamically adjust each optical neuron 90, including adjusting the sound field, light field, temperature, voltage, current and / or magnetic field strength. Figure 5C A schematic diagram showing the logical connection of an exemplary spatial diffraction optical neural network processor using a transmission-type spatial diffraction optical neural network processing unit is shown.

[0066] The phase control unit 50 can realize separate calculations on single optical field parameters such as wavelength, amplitude, polarization state, etc., and can also realize multiplexed calculations on multiple wavelengths, polarization states, etc. For example, the optical field at the optical signal input port 20 contains n wavelengths of light, and the light fields of different wavelengths are loaded with corresponding calculation data to realize the parallel input of different calculation tasks. The selected optical field control calculation is performed through the spatial diffraction optical neural network, and the processed output light is received by the photodetector array and the calculation results are output, thereby realizing the parallel calculation of multiple tasks.

[0067] For example, Fig. 6A and Figure 6B The classification tasks shown are loaded simultaneously in the same area of ​​DMD, where Fig. 6A The neural network for data calculation uses 532nm laser calculation to Figure 6B 488nm laser illumination operation is used for data calculation. Both tasks use the same phase control unit for calculation. After reflection modulation, the outgoing light field is received by different areas of the photodetector array 30. In this process, nonlinear calculation of the light field intensity is realized, and finally the classification result is directly output in the form of an electrical signal. Fig. 6A The target image required for classification is loaded by 532nm laser illumination of the digital micromirror array DMD, and the signal reflected by the digital micromirror array is used as the light field signal input unit 20 of the processor. Figure 6B The target image required for classification is loaded by 488nm laser illumination of the digital micromirror array DMD, and the signal reflected by the digital micromirror array is used as the light field signal input unit 20 of the processor.

[0068] Both groups of input optical signals are incident at an angle of 45 degrees to the first row of phase control units 1 inside the spatial diffraction optical neural network processing unit 40. The area of ​​each phase control unit is 0.8mm×0.8mm, the size of each optical neuron is 4um×4um, and each phase control unit contains 200×200 optical neurons. After phase modulation by the phase control unit 1, it is reflected to the phase control unit 1 staggered inside the spatial diffraction optical neural network processing unit 41. After four times of phase modulation and spatial transmission, the outgoing light field is received by the photodetector array 30. In this process, nonlinear calculation of the light field intensity is realized, and finally the classification result is directly output in the form of an electrical signal.

[0069] The parallel computing capability of the spatial diffraction optical neural network processor 100 can also be achieved by expanding the phase control units 50 into multiple rows in the column direction. Each row of phase control units 50 can perform single-task or multi-task parallel computing, thereby realizing parallel computing of multiple rows of phase control units.

[0070] For example, Figure 6C The data operation uses 635nm laser as the operation signal. This task is performed on the phase control unit in the second row. The input light signal is incident at a 45-degree angle to the phase control unit 1 in the second row inside the spatial diffraction optical neural network processing unit 40. The input light signal is received by another area of ​​the photodetector array 30 after 4 reflection phase adjustments in the second row. In this process, nonlinear operation of the light field intensity is realized, and the classification result is finally directly output in the form of an electrical signal.

[0071] The multi-task parallel computing capability of the spatial diffraction optical neural network processor 100 provided in the present application can realize multi-task parallel processing by multiplexing light field parameters, and can also realize multi-task parallel processing by expanding the number of phase control unit rows.

[0072] The present application provides a spatial diffraction optical neural network processor, which realizes an integrable and expandable spatial diffraction optical neural network processor architecture by adjusting the number and arrangement of phase control units on symmetrically placed spatial diffraction optical neural network processing units. Logically adjacent phase control units realize data connection through spatial diffraction optical transmission, and the arrangement method can be given by the law of optical mirror reflection, allowing the flexible expansion of the processor computing scale by only changing the number of arrangement of phase control units and the size of the phase modulation weight matrix of the phase control units without the need for more optical elements. It can overcome the defects of existing spatial diffraction optical neural network processors that are difficult to integrate and expand, enhance multi-task parallel processing capabilities, greatly improve the integration of the processor, ensure high scalability and integration capabilities, and reduce the problem of light field energy dissipation caused by the increase of layered structures. The spatial diffraction optical neural network processor can be further integrated with intelligent computing scenarios such as autonomous driving and voice and image processing.

[0073] It should be clearly understood that the present application describes how to form and use specific examples, but the present application is not limited to any details of these examples. On the contrary, based on the teaching of the content disclosed in the present application, these principles can be applied to many other embodiments.

[0074] In addition, it should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present application, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0075] The exemplary embodiments of the present application are specifically shown and described above. It should be understood that the present application is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present application is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended claims.

Claims

1. A spatial diffraction optical neural network processor, characterized in that: include: A light field signal input unit, used to convert the data to be calculated into an input light signal and transmit it; A first spatial diffraction optical neural network processing unit and a second spatial diffraction optical neural network processing unit, wherein the first spatial diffraction optical neural network processing unit and the second spatial diffraction optical neural network processing unit are symmetrically arranged, and are used to receive the input light signal to perform light field control operation, and reflect the output, wherein: The first spatial diffraction optical neural network processing unit is integrated with N phase control units, and the second spatial diffraction optical neural network processing unit is integrated with M phase control units, each of the phase control units is used to perform light field control operation on the input optical signal, and N and M are both integers greater than or equal to 1; The phase control units integrated on the first spatial diffraction optical neural network processing unit and the second spatial diffraction optical neural network processing unit are arranged in an interlaced manner to achieve optical signal connection through the law of reflection; The photodetector array is used to convert the input light signal after the light field control operation of the first spatial diffraction optical neural network processing unit and the second spatial diffraction optical neural network processing unit into an electrical signal.

2. The spatial diffraction optical neural network processor according to claim 1, characterized in that: The optical field signal input unit generates the input optical signal by performing phase, amplitude, polarization, frequency, wavelength and / or angular momentum encoding processing on the data to be calculated.

3. The spatial diffraction optical neural network processor according to claim 1 or 2, characterized in that: The light field signal input unit includes: an array laser, a light emitting diode array, a liquid crystal display, a digital micromirror array illuminated by a single light source, and / or a liquid crystal spatial light modulator.

4. The spatial diffraction optical neural network processor according to claim 1, characterized in that: The photodetector array includes: a charge coupled device, a complementary metal oxide semiconductor detector and / or a photodiode array.

5. The spatial diffraction optical neural network processor according to claim 1, characterized in that: Also includes: A first lens is disposed between the light field signal input unit and the first spatial diffraction optical neural network processing unit; The second lens is arranged between the first spatial diffraction optical neural network processing unit or the second spatial diffraction optical neural network processing unit and the photodetector array.

6. The spatial diffraction optical neural network processor according to claim 1, characterized in that: The phase control unit has a reflective characteristic, and the phase control unit includes a reflective metasurface and / or a reflective spatial light modulator.

7. The spatial diffraction optical neural network processor according to claim 1, characterized in that: The phase control unit has a transmission characteristic, and the phase control unit includes a transmission type metasurface and / or a transmission type spatial light modulator.

8. The spatial diffraction optical neural network processor according to claim 7, characterized in that: Also includes: The mirror reflection unit is arranged at the bottom of each of the phase control units.

9. The spatial diffraction optical neural network processor according to claim 1, characterized in that: The phase control unit includes a metasurface, a digital micromirror array and / or a spatial light modulator; The phase control unit includes L*Z optical neurons, each optical neuron is a pixel point on a metasurface, a digital micromirror array and / or a spatial light modulator, and L and Z are both integers greater than or equal to 1.

10. The spatial diffraction optical neural network processor according to claim 9, characterized in that: Also includes: A programmable control unit controller is connected to the first spatial diffraction optical neural network processing unit and the second spatial diffraction optical neural network processing unit to dynamically adjust each optical neuron, including adjusting the sound field, light field, temperature, voltage, current and / or magnetic field intensity.

Images