A diffractive optical neural network computing system and method implementing pure optical nonlinearity
By integrating spatial light modulators and digital micromirror devices into a diffractive light neural network, pure optical nonlinear activation is achieved, solving the problems of low efficiency, high delay, and complex structure in existing technologies, and realizing high-energy-efficiency and low-latency optical computing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENZHEN UNIV
- Filing Date
- 2026-02-24
- Publication Date
- 2026-05-29
AI Technical Summary
Existing diffractive light neural networks lack efficient and low-power pure optical nonlinear activation mechanisms, resulting in low system efficiency, high latency, and complex structure.
By sequentially integrating a spatial light modulator and a digital micromirror device in the optical path, and using a specific training algorithm, pure optical nonlinear activation is achieved, eliminating the photoelectric conversion stage. A pure optical computing path without photoelectric conversion is formed by directly coupling a linear optical computing unit with a nonlinear optical activation unit.
It achieves high-energy-efficiency, low-latency pure optical neural network computing, has light-speed-level inference latency, supports multi-layer network expansion, and is suitable for high-speed real-time processing of large amounts of data such as images and videos.
Smart Images
Figure CN122114033A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of photonic computing and artificial intelligence hardware technology, and in particular to a diffractive light neural network computing system and method for realizing pure optical nonlinearity. Background Technology
[0002] Diffractive optical neural networks are a cutting-edge photonic computing technology that utilizes the diffraction propagation of light waves in free space or a medium for parallel computation. They possess inherent advantages such as ultra-high speed, low latency, high energy efficiency, and large-scale parallel processing, demonstrating great potential in performing complex linear transformations (such as convolution and matrix multiplication) in tasks like image processing and pattern recognition.
[0003] However, the development of this technology has long been constrained by a core challenge: the lack of an efficient, low-power pure optical nonlinear activation mechanism. Nonlinear activation functions are the cornerstone of deep neural networks for achieving complex function fitting and intelligent decision-making. Existing diffractive light neural network solutions mostly employ a hybrid optoelectronic approach. This involves first performing linear calculations using an optical system (such as a spatial light modulator), then converting the optical signal into an electrical signal using a photoelectric sensor (such as a CMOS camera), implementing a nonlinear activation function (such as the ReLU function) in the electronic domain (such as a CPU or GPU), and finally potentially converting it back to an optical signal using an electro-optic modulator for further processing. This hybrid optoelectronic approach has the following inherent drawbacks: Low energy efficiency: Frequent light-to-electricity-to-light conversion processes introduce additional energy consumption; High system latency: The photoelectric conversion and electronic processing processes severely limit the overall computing speed of the system, making it impossible to fully utilize the speed advantages of optical computing; The system is complex: it requires the integration of multiple heterogeneous components such as photodetectors, electronic processors and electro-optic modulators, which is not conducive to the integration, miniaturization and stability of the system. Summary of the Invention
[0004] This invention aims to address the problems of low system efficiency, high latency, and complex structure in existing diffractive optical neural networks due to the lack of purely optical nonlinear units. To achieve the above objective, this invention provides a diffractive optical neural network computing system and method that realizes purely optical nonlinearity. By sequentially integrating a spatial light modulator (SLM) and a digital micromirror device (DMD) in the optical path, and in conjunction with a specific training algorithm, complete, photoelectric-free, purely optical neural network inference is realized.
[0005] According to one aspect of the present invention, a diffraction light neural network computing system for realizing pure optical nonlinearity is provided, comprising: A coherent light source is used to provide a computational carrier beam; A linear optical computing unit is used to perform programmable phase modulation on the carrier beam to physically execute the linear transformation operation of the neural network. A nonlinear optical activation unit is used to perform programmable binary amplitude modulation on the beam modulated by the linear optical computing unit, thereby physically executing the nonlinear activation operation of the neural network; and A photodetector unit is used to acquire the output light field signal modulated by the nonlinear photoactivator unit. The linear optical computing unit and the nonlinear optical activation unit are directly coupled in the optical path, forming a pure optical computing path that does not require photoelectric conversion.
[0006] Preferably, the system further includes a beam shaping module disposed between the laser source and the spatial light modulator, used to shape the beam emitted by the laser source into a collimated parallel beam with a shape matching the input data format. The beam shaping module may include a beam expanding and collimating lens group and a square aperture arranged sequentially. The lens group expands and collimates the laser beam, and the square aperture clips the circular light spot into a square shape to match the typically rectangular input image data, reducing ineffective illumination areas and improving the signal-to-noise ratio.
[0007] Preferably, the system further includes a polarizer disposed between the laser source and the spatial light modulator, used to make the light beam incident on the spatial light modulator linearly polarized in a specific direction. Since commonly used liquid crystal spatial light modulators typically have the best phase modulation efficiency for light beams with a specific polarization direction, this polarizer is used to optimize the polarization state of the incident light and ensure the modulation effect.
[0008] Preferably, in order to increase the depth of the network or adapt to the optical path layout, one or more reflective elements (such as plane mirrors or right-angle prisms) for changing the direction of the optical path can be provided between the spatial light modulator and the digital micromirror device to form folded or multi-reflection optical paths, thereby realizing more computing layers without increasing the size of physical devices.
[0009] According to another aspect of the present invention, a method for calculating a diffractive light neural network to achieve pure optical nonlinearity is provided, comprising the following steps: S1. Model Training and Hardware Mapping Stage: On an electronic computing device, a neural network model containing at least one linear layer and one nonlinear layer is trained; wherein, by optimizing a customized loss function, the output value of the nonlinear layer is made to converge to binary; the parameters of the trained linear layer are converted into a phase map, and the binary output of the nonlinear layer is converted into a binary mask; S2, Hardware Configuration Stage: Load the phase map into the linear light computing unit, and load the binary mask into the nonlinear light activation unit; S3, Optical Calculation Stage: The coherent light source is activated, and the light beam passes sequentially through the linear light calculation unit loaded with the phase map and the nonlinear light activation unit loaded with the binary mask to complete the all-optical forward inference calculation. S4. Result Acquisition Stage: The final light field distribution is captured by the light detection unit and used as the calculation result of the neural network.
[0010] Preferably, in step S1, the customized loss function L is based on the nonlinear activation function of the neural network. Constructed from the output, represented as , where the function The design enables gradients during training. exist The value range of has opposite signs, thus driving The output converges toward both extreme values.
[0011] Preferably, in step S2, the target nonlinear activation function is a modified Sigmoid function: ; in and It is an adjustable constant; The customized loss function is: .
[0012] By training the network by minimizing this loss function, its mathematical properties will naturally influence the output value of the Sigmoid function. It converges to either 0 or 1. After training, the output of the nonlinear layer is directly forced to binarize (e.g., setting values greater than 0.5 to 1 and values less than or equal to 0.5 to 0), thus generating a binary mask that perfectly matches the physical characteristics of the DMD's "on / off" binary modulation. In a specific example, the constant... Take 10, Take 0.5, that is .
[0013] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the neural network training steps involved in the above-described calculation method, particularly the training process based on the customized loss function, to generate a phase map loaded onto the SLM and a binary mask loaded onto the DMD.
[0014] Therefore, the diffraction light neural network computing system and method of the present invention, which adopts the above structure to realize pure optical nonlinearity, has the following beneficial effects: (1) This invention completely eliminates the photoelectric conversion link by directly coupling linear optical computing units (such as SLM) and nonlinear optical activation units (such as DMD) in the optical path, realizing complete optical domain processing from linear to nonlinear computation. Moreover, the entire computation is completed by light, which has high energy efficiency and light-speed-level inference delay in physical nature, which is significantly better than the traditional electronic and optoelectronic hybrid scheme.
[0015] (2) This invention enables multi-layer network expansion through optical path folding design, possessing good potential for practical application and miniaturization. Furthermore, through customized loss function training, it efficiently maps Sigmoid-type continuous nonlinear functions to binary modulation devices, solving the algorithmic challenge of optical nonlinearity implementation. Simultaneously, the system provided by this invention can process the entire two-dimensional light field at once, making it suitable for high-speed real-time processing scenarios involving large amounts of data such as images and videos.
[0016] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0017] Figure 1 This is a schematic diagram of the optical path structure of a pure optical neural network system according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the steps of the all-optical neural network computation method of the present invention; Figure Labels 1-Laser, 2-First lens, 3-Second lens, 4-Square aperture, 5-Polarizer, 6-First mirror, 7-Spatial light modulator, 8-Second mirror, 9-Digital micromirror device, 10-Photodetector unit. Detailed Implementation
[0018] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.
[0019] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0020] Example 1 like Figure 1 As shown, this invention provides a diffractive light neural network computing system for realizing pure optical nonlinearity. The system mainly includes, along the optical path, a coherent light source module, a beam preprocessing module, a linear light computing unit, a nonlinear light activation unit, and a light detection unit. All units are sequentially connected along the optical path, forming a closed-loop pure optical computing path without photoelectric conversion.
[0021] In this embodiment, the coherent light source module uses a 532nm continuous-wave solid-state laser 1 as the coherent light source. This wavelength is located in the visible light band, matching the efficient operating band of most subsequent commercial spatial light modulators and digital micromirror devices. Laser 1 has an output power of approximately 50mW and exhibits good spatial and temporal coherence, providing a stable optical carrier for subsequent diffraction interferometry calculations. In other embodiments, lasers of 635nm, 780nm, or other wavelengths can be selected based on the optimized operating wavelengths of the SLM and DMD, or a pulsed laser can be used to match specific timing calculation requirements.
[0022] The beam preprocessing module is responsible for processing the small Gaussian beam emitted from laser 1 into a large-area, uniform, polarization-matched parallel beam suitable for subsequent modulation. Specifically, it includes: Beam expanding and collimating unit: consisting of a negative lens (first lens 2, focal length...) ) and a positive lens (second lens 3, focal length) This constitutes the structure of the Galilean telescope. Two lenses are placed coaxially with a distance of approximately 125 mm between them, which expand and collimate the approximately 1 mm diameter beam emitted from laser 1 into a uniform parallel beam with a diameter of approximately 20 mm.
[0023] The beam formatting unit includes an adjustable square aperture 4 and a linear polarizer 5. The square aperture 4 is positioned behind the collimated beam, and its aperture size is adjustable (e.g., adjusted to the same 16:9 ratio as the input image data) to crop the circular light spot into a rectangle, thereby matching common image input formats, reducing ineffective illumination areas, and improving the system's signal-to-noise ratio and energy efficiency. The polarizer 5 (e.g., a wire-grid polarizer optimized for a 532nm wavelength) is used to filter linearly polarized light with a specific vibration direction (e.g., the horizontal direction). Since the liquid crystal spatial light modulator 7 used subsequently in this embodiment is sensitive to the polarization state of the incident light, this polarizer ensures that the polarization state of the incident light is consistent with the optimal modulation direction of the SLM, thereby achieving a high-contrast, low-crosstalk phase modulation effect.
[0024] The linear optical computing unit is used to perform linear transformation operations (such as convolution and fully connected layers) in neural networks. In this embodiment, the unit consists of a first reflector 6 and a pure phase-type liquid crystal spatial light modulator 7.
[0025] The first reflector 6 is a planar reflector with a high-reflectivity dielectric film (reflectivity >99% for 532nm wavelength) coated on its surface. It is mainly used for optical path deflection, guiding the beam from the preprocessing module to the modulation surface of the SLM7 to achieve a compact system layout.
[0026] In this embodiment, the core function of the spatial light modulator 7 (SLM) is to load a phase distribution map (usually a grayscale image with grayscale values corresponding to 0 to 2) generated by a neural network training algorithm. (Phase delay). When coherent parallel light shines on an SLM7 surface loaded with a phase map, the wavefront phase of the beam is spatially modulated. Subsequently, the modulated light wave propagates through Fresnel diffraction in free space. This physical process of "phase modulation + free-space diffraction" is mathematically rigorously equivalent to performing a complex linear transformation (matrix multiplication), thereby realizing a layer of neural network linear computation. The programmability of SLM allows this linear transformation to be flexibly configured according to different tasks.
[0027] The nonlinear optical activation unit is used to directly implement the nonlinear activation function in the optical domain of a neural network. In this embodiment, the unit consists of a second reflector 8 and a digital micromirror device 9.
[0028] The second reflector 8 is a high-reflectivity plane mirror used to precisely guide the complex light field, which has been linearly modulated and diffracted by SLM, to the surface of the micromirror array of the digital micromirror device 9 (DMD).
[0029] In this embodiment, the digital micromirror device 9 (DMD) uses a micromirror with a resolution of 1024×768 pixels and a micromirror size of [missing information]. The DMD (Digital Micromirror Device) uses a micromirror that is independently addressable and can switch rapidly between two stable states (e.g., +12° "on" and -12° "off") (switching time on the order of microseconds). The digital micromirror device 9 is loaded with a binary mask (a black-and-white image, where "1" represents on and "0" represents off) generated by a specific training algorithm of this invention. Based on the value of each pixel on the mask, the corresponding micromirror reflects the incident light into the effective optical path (to be received by a subsequent detector) or deflects it off the optical path (into the optical trap). This "all-pass" or "all-block" operation is essentially spatial binary modulation of the optical field amplitude. Through the training method of this invention, this physical process is used to approximate continuous nonlinear activation functions such as the Sigmoid with high precision, thereby realizing purely optical nonlinear activation at the hardware level.
[0030] The photodetector unit is used to capture and quantify the results of optical calculations. In this embodiment, the photodetector unit 10 is a CMOS camera. The camera's pixel size matches the micromirror size of the digital micromirror device 9 (DMD), resulting in low readout noise and a high frame rate. The CMOS camera is precisely positioned in the optical path to receive the reflected light from the DMD in its "on" state, and is used to acquire a two-dimensional light intensity distribution image of the final output light field after processing by the entire system (SLM linear transformation + DMD nonlinear activation). This image represents the inference result of the all-optical neural network on the input data; for example, in an image classification task, the light intensity peak in a specific region of the output image corresponds to the identified category.
[0031] To further increase network depth (i.e., stack more linear and nonlinear layers), the optical path can be carefully designed between the SLM and DMD, utilizing multiple mirrors to form a multi-reflection loop. For example, the beam can be arranged according to... The path loops multiple times, with each pass through the SLM and DMD equivalent to passing through a layer of the network. This approach can significantly improve the expressive power and complexity of the network without adding additional core modulation devices.
[0032] Example 2 This embodiment describes a specific calculation method applied to the above system, the process of which is as follows: Figure 2 As shown, it mainly includes two stages: offline training and online optical inference.
[0033] S1: Model Training and Hardware Mapping Phase This stage is completed on a general-purpose electronic computer.
[0034] Building the software model: Construct a feedforward neural network model using a deep learning framework (such as PyTorch or TensorFlow). This model must contain at least one linear layer (corresponding to the Spatial Light Modulator 7 (SLM)) and one nonlinear layer (corresponding to the Digital Micromirror Device 9 (DMD)). To accommodate the binary nature of the DMD, an improved Sigmoid function with a steep transition region is chosen as the nonlinear activation function. ; in, Steepness factor (e.g.) ), c is the offset (e.g. Increase The value can make the function in The more drastic the changes in the vicinity, the more easily the output becomes saturated.
[0035] Designing a custom loss function: This is crucial for achieving hardware mapping. Besides the main loss function tailored to a specific task (such as cross-entropy loss for classification),... In addition, the output of the nonlinear layer A special auxiliary loss function is introduced: ; The mathematical principle behind it is: calculating the loss pair gradient, Analysis shows that: when When the gradient is positive, backpropagation will cause... Increase.
[0036] when When the gradient is negative, it will cause backpropagation to... Decrease.
[0037] This mechanism generates a powerful "binarization driving force" during training, continuously... The output value is pushed from the middle 0.5 to the two ends 0 or 1.
[0038] Joint training: based on total losses The network is trained for the objective function, where It is a hyperparameter that balances the weights of the two loss terms. After sufficient training, the output value of the nonlinear layer of the network... Highly polarized, the vast majority of them are very close to 0 or 1.
[0039] Parameter hardware mapping includes linear layer mapping and nonlinear layer mapping, as detailed below: Linear layer mapping: The trained linear layer weight matrix is calculated using scalar diffraction theory (such as angular spectrum propagation) to transform it into a phase map (.bmp or .tiff format, grayscale values corresponding to...) that can be displayed on an SLM. Phase).
[0040] Nonlinear layer mapping: Apply a simple thresholding operation (e.g., set the threshold to 0.5, set it to 1 if it is greater than 0.5, otherwise set it to 0) to the output tensor of the nonlinear layer of the trained network to generate a binary mask file (black and white .bmp image) with the same physical resolution as the DMD.
[0041] S2: Hardware Configuration Stage Load the phase map file generated in stage S1 into the controller memory of the SLM 7 via a video interface (such as HDMI). Load the generated binary mask file onto its micromirror array via the DMD 9's dedicated control software (such as Texas Instruments' DLP® Discovery™ software). Ensure both are in a ready-to-trigger state.
[0042] S3: Optical computing (inference) stage This stage is a purely physical optical process, extremely fast. The input data to be processed (e.g., a handwritten digit image to be classified) is encoded into the amplitude or phase of the incident light field via an additional input SLM (not shown in the figure, which can be placed before the beam preprocessing module). Laser 1 is turned on, and the beam carrying the input information automatically begins its computational journey: first, it undergoes a linear transformation by the SLM, then diffracts a distance in the air, and finally, it is nonlinearly activated by the DMD. The entire process is completed in the instant of light propagation (on the order of nanoseconds to microseconds), without any electronic processor intervention.
[0043] S4: Results Acquisition Stage A CMOS camera located at the end of the optical path performs a single exposure, capturing the light intensity distribution on the output plane. Using simple image processing algorithms (e.g., finding the pixel region with the highest light intensity, or reading the average light intensity of a preset ROI region), the inference results of the neural network, such as the image's classification label, can be directly interpreted.
[0044] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.
Claims
1. A computational system for a diffractive light neural network that realizes pure optical nonlinearity, characterized in that, Including those arranged sequentially along the optical path: A coherent light source is used to provide a computational carrier beam; A linear optical computing unit is used to perform programmable phase modulation on the carrier beam to physically execute the linear transformation operation of the neural network. The nonlinear optical activation unit is used to perform programmable binary amplitude modulation on the beam modulated by the linear optical computing unit, so as to physically execute the nonlinear activation operation of the neural network. as well as A photodetector unit is used to acquire the output light field signal modulated by the nonlinear photoactivator unit. The linear optical computing unit and the nonlinear optical activation unit are directly coupled in the optical path, forming a pure optical computing path that does not require photoelectric conversion.
2. The diffraction light neural network computing system for realizing pure optical nonlinearity according to claim 1, characterized in that, The linear optical computing unit includes a spatial optical modulator for loading a phase map corresponding to the weights of the linear layers of the neural network; the nonlinear optical activation unit includes a digital micromirror device for loading a binary mask corresponding to the nonlinear activation function of the neural network.
3. The diffraction light neural network computing system for realizing pure optical nonlinearity according to claim 1, characterized in that, It also includes a beam preprocessing module, which is disposed between the coherent light source and the linear light computing unit. The beam preprocessing module includes a lens group for beam expansion and collimation and an aperture for beam shaping.
4. The diffraction light neural network computing system for realizing pure optical nonlinearity according to claim 1, characterized in that, At least one optical path folding element is provided between the linear optical computing unit and the nonlinear optical activation unit to guide the output optical field of the linear optical computing unit to the input surface of the nonlinear optical activation unit.
5. A diffraction light neural network computing system for realizing pure optical nonlinearity according to claim 4, characterized in that, The optical path folding element is a plane mirror or a right-angle prism. By setting multiple optical path folding elements, the light beam is reflected multiple times between the linear optical computing unit and the nonlinear optical activation unit to increase the equivalent network depth.
6. A method for calculating a diffractive light neural network that achieves pure optical nonlinearity, applied to a diffractive light neural network calculation system that achieves pure optical nonlinearity as described in any one of claims 1-5, characterized in that, Includes the following steps: S1. Model Training and Hardware Mapping Stage: On an electronic computing device, a neural network model containing at least one linear layer and one nonlinear layer is trained; wherein, by optimizing a customized loss function, the output value of the nonlinear layer is made to converge to binary; the parameters of the trained linear layer are converted into a phase map, and the binary output of the nonlinear layer is converted into a binary mask; S2, Hardware Configuration Stage: Load the phase map into the linear light computing unit, and load the binary mask into the nonlinear light activation unit; S3, Optical Calculation Stage: The coherent light source is activated, and the light beam passes sequentially through the linear light calculation unit loaded with the phase map and the nonlinear light activation unit loaded with the binary mask to complete the all-optical forward inference calculation. S4. Result Acquisition Stage: The final light field distribution is captured by the light detection unit and used as the calculation result of the neural network.
7. The method for calculating a diffraction light neural network to achieve pure optical nonlinearity according to claim 6, characterized in that, In step S1, the customized loss function L is based on the nonlinear activation function of the neural network. Constructed from the output, represented as , where the function The design enables gradients during training. exist The value range of has opposite signs, thus driving The output converges toward both extreme values.
8. The method for calculating a diffraction light neural network to achieve pure optical nonlinearity according to claim 7, characterized in that, The nonlinear activation function The customized loss function is a Sigmoid type function. .
9. The method for calculating a diffraction light neural network to achieve pure optical nonlinearity according to claim 8, characterized in that, The Sigmoid type function is ,in A steepness factor greater than 1 This is a threshold parameter; after training, the threshold is set to... The output is binarized to generate the binary mask.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it is used to implement the steps of the model training and hardware mapping stage in the diffraction light neural network calculation method for realizing pure optical nonlinearity as described in any one of claims 6 to 9.