Programmable diffraction photoelectric fusion computing system and method based on heterogeneous SLM collaboration

Abstract

The invention relates to the technical field of computing systems, and particularly discloses a programmable diffraction photoelectric fusion computing system and method based on heterogeneous SLM collaboration. The system constructs a four-layer collaborative architecture comprising a hardware layer, a control driving layer, an algorithm layer and an application interface layer. The basic principle is as follows: in a hardware layer, a high-speed digital micromirror device module and a high-phase-precision silicon-based liquid crystal spatial light modulator module are connected through a 4f filtering system light path, and are respectively fixed as a high-speed data coding front end and a precision wavefront calculation core to realize heterogeneous division of labor; in the algorithm layer, hardware physical characteristics are integrated into model optimization through a physical embedding adaptive training framework; and nanosecond synchronization of multiple devices is realized on the control driving layer. The method has the core effects that the technical contradiction that a single programmable SLM system cannot meet high-speed data input and high-precision phase modulation at the same time is overcome, and the refresh speed and the analog calculation precision are both improved in one system.

Description

Technical Field

[0001] This invention relates to the field of computing system technology, specifically to a programmable diffractive optoelectronic fusion computing system and method based on heterogeneous SLM collaboration. Background Technology

[0002] In recent years, optical neural network technology, which utilizes optical diffraction for parallel simulation computing, has developed rapidly, providing a new path to overcome the bottlenecks of power consumption and parallelism in traditional electronic computing. Early diffraction neural networks mostly used fixed diffraction elements manufactured using photolithography or 3D printing techniques. Once fabricated, their function could not be changed, limiting their flexibility and applicability. With the development of spatial light modulator technology, especially the maturity of programmable devices such as silicon-based liquid crystals and digital micromirror devices, research has shifted to programmable diffraction computing systems. This makes it possible to dynamically reconstruct optical computing functions through electronic control, significantly improving the system's versatility and task adaptability.

[0003] However, most existing programmable diffraction computing systems are built upon a single type of spatial light modulator. Whether based on pure phase modulation (LCOS, Liquid Crystal on Silicon) or intensity modulation (DMD, Digital Micro-mirror Device), their physical characteristics inherently limit their performance: LCOS can achieve high-precision, continuous phase modulation, but its refresh rate is relatively low, making it difficult to handle the demands of high-speed dynamic data input; while DMD boasts extremely high refresh rates of tens of thousands of hertz, its modulation method is essentially binary or pulse-width modulation of intensity, making it unable to directly perform high-precision continuous wavefront manipulation. Therefore, a single SLM (Spatial Light Modulator) architecture faces a core contradiction in system design: the inability to simultaneously achieve both speed and accuracy. This severely restricts the application of programmable diffraction computing systems in scenarios requiring high throughput and high precision.

[0004] To improve performance, existing technologies employ an "optoelectronic fusion" approach, combining optical computing with subsequent electronic computing. This involves using fixed diffraction elements for feature extraction, followed by processing with analog or digital electrical circuits. While this approach combines the advantages of both light and electricity to some extent, its optical computing component is often static or functionally limited. It cannot achieve dynamic, programmable collaboration between front-end data input and the core computing unit across the entire chain. The overall system reconfigurability, computational agility, and hardware utilization still have significant room for improvement. Summary of the Invention

[0005] To address the aforementioned problems in the prior art, this invention provides a programmable diffraction optoelectronic fusion computing system and method based on heterogeneous SLM collaboration.

[0006] According to a first aspect of the present invention, a programmable diffraction optoelectronic fusion computing system based on heterogeneous SLM collaboration is proposed, comprising a hardware layer, a control driving layer, an algorithm layer, and an application interface layer connected in sequence, wherein: The hardware layer includes: The core heterogeneous SLM includes a digital micromirror device module and a silicon-based liquid crystal SLM module; among them, the digital micromirror device module serves as the front end for data scheduling and feature extraction, and its refresh rate is greater than the first frame rate threshold; the silicon-based liquid crystal SLM module serves as the core for wavefront transformation and connection. A laser source and optical beam expander system are used to provide the input optical signal for the core heterogeneous SLM; a 4f filtering system is used to filter the input optical signal; and an image sensor is used to detect the output optical signal. The control and driving layer connects to and drives the hardware layer, and is used to synchronously drive and manage the data flow of the digital micromirror device module and the silicon-based liquid crystal SLM module. The algorithm layer interacts with the control and drive layer, including a diffraction neural network model library and a physical embedding adaptive training framework, which is used to train and optimize the diffraction neural network model according to the physical characteristics of the hardware layer. The application interface layer is used to provide users with software development kits to deploy computing task models.

[0007] According to some embodiments, in the system of the first aspect of the present invention, the digital micromirror device module and the silicon-based liquid crystal SLM module are optically connected through a 4f filtering system, which includes two lenses and an adjustable aperture disposed on the confocal plane of the two lenses.

[0008] According to some embodiments, in the system of the first aspect of the present invention, an adjustable aperture is used to filter out diffraction noise that is higher than a set spatial frequency generated after being modulated by a digital micromirror device module.

[0009] According to some embodiments, in the system of the first aspect of the present invention, the control driving layer includes: The synchronous drive module is used to send synchronous trigger signals to the digital micromirror device module and the silicon-based liquid crystal SLM module. The data flow management module is used to coordinate the data timing between pattern loading of the digital micromirror device module, phase map loading of the silicon-based liquid crystal (SLM) module, and image acquisition of the image sensor.

[0010] According to some embodiments, in the system of the first aspect of the present invention, the system reconstruction time is less than a first time threshold. The reconstruction time refers to the total time from receiving a new task instruction to completing the parameter loading of the digital micromirror device module and the silicon-based liquid crystal SLM module and making the system output stable.

[0011] According to some embodiments, in the system of the first aspect of the present invention, a physically embedded adaptive training framework is used to perform end-to-end gradient optimization by incorporating the optical modulation characteristics of the hardware layer, the physical parameters of the digital micromirror device module and the silicon-based liquid crystal SLM module, and the transfer function of the 4f filter system as part of the forward propagation model during training.

[0012] According to some embodiments, in the system of the first aspect of the present invention, the hardware layer further includes a polarization control element, which is disposed in the optical path of the 4f filter system and is used to adjust the polarization direction of the light signal incident on the silicon-based liquid crystal SLM module.

[0013] According to a second aspect of the present invention, a programmable diffraction photoelectric fusion calculation method based on heterogeneous SLM collaboration is proposed, applied to a system as described in the first aspect of the present invention, the method comprising: The application interface layer receives user-defined computational tasks and corresponding diffraction neural network models. In the algorithm layer, the physical embedding adaptive training framework is used to train the diffraction neural network model in combination with the physical parameters of the hardware layer, so as to obtain the optimized model parameters adapted to the hardware layer. The optimized model parameters include the encoding pattern to be loaded into the digital micromirror device module and the continuous phase map to be loaded into the silicon-based liquid crystal SLM module. By controlling the driving layer, the encoded pattern and continuous phase map in the optimized model parameters are loaded into the digital micromirror device module and silicon-based liquid crystal SLM module in the hardware layer in a synchronous timing sequence. The laser source in the hardware layer is activated, and the input optical signal is sequentially encoded and modulated by the digital micromirror device module, filtered by the 4f filter system, and phase-modulated by the silicon-based liquid crystal SLM module. The image sensor then detects and outputs the calculation results.

[0014] According to some embodiments, in the method of the second aspect of the present invention, training is performed using a physical embedding adaptive training framework, specifically including: Forward propagation step: The modulation characteristics of the digital micromirror device module, the phase modulation response curve of the silicon-based liquid crystal SLM module, and the filtering characteristics of the 4f filter system in the hardware layer are integrated into the calculation graph of the diffraction neural network model to simulate the physical propagation process of optical signals in real hardware. Backpropagation and optimization steps: The loss is calculated based on the difference between the output of the forward propagation step and the expected target, and the weight parameters of the diffraction neural network model and the simulated values ​​of the adjustable hardware physical parameters are updated simultaneously through the gradient backpropagation algorithm.

[0015] According to some embodiments, in the method of the second aspect of the present invention, the diffraction neural network model is a diffraction deep neural network, a diffraction recurrent neural network, or a network architecture in a diffraction network.

[0016] The present invention has the following beneficial effects: 1. To address the core contradiction that a single type of programmable spatial light modulator (SLM) cannot simultaneously meet the requirements of high-speed data input and high-precision phase modulation, this invention constructs a heterogeneous collaborative hardware architecture that uses a digital micromirror device (DMD) module as a high-speed data front end and a liquid crystal on silicon (LCOS) spatial light modulator module as a precision computing core. This architecture achieves a synergistic improvement in system processing speed and computing accuracy, effectively overcoming the performance limitations of a single modulator.

[0017] 2. To address the problem of fragmented hardware, control, algorithm, and application layers in optoelectronic computing systems, which makes global optimization and efficient collaboration difficult, this invention designs a multi-layered hardware and software collaborative system architecture that includes a hardware layer, a control driver layer, an algorithm layer, and an application interface layer. This architecture enables end-to-end collaborative management from physical computing to task deployment, improving the overall optimization capability, reconfigurability, and ease of use of the system.

[0018] 3. To address the problem that non-ideal characteristics exist between ideal algorithm models and real physical hardware, leading to a decrease in the performance of simulation training results on actual systems, this invention introduces a physical embedding adaptive training framework at the algorithm layer. This framework models the physical characteristics of the hardware and integrates them into the training process of the neural network, achieving deep adaptation of the algorithm to the hardware system and improving the accuracy and robustness of the computational model in actual deployment.

[0019] 4. To address the issue that general-purpose spatial light modulators cannot fully meet the requirements of dedicated optical computing in terms of key performance indicators, this invention adopts dedicated DMD modules and LCOS modules optimized for optical computing as the core engine, providing a high-performance hardware foundation for the system, thereby supporting the realization of low-latency, high-stability and large-scale parallel optical computing. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of an embodiment 1000 of a programmable diffraction optoelectronic fusion computing system based on heterogeneous SLM collaboration according to the present invention; Figure 2 for Figure 1 A schematic diagram of the control drive layer 102 in Example 1000; Figure 3 This is a schematic diagram of an embodiment 2000 of a programmable diffraction optoelectronic fusion computing system based on heterogeneous SLM collaboration according to the present invention; Figure 4 This is a flowchart illustrating an embodiment 3000 of the programmable diffraction photoelectric fusion calculation method based on heterogeneous SLM collaboration according to the present invention. Figure 5 for Figure 4 A flowchart illustrating step S2 in Example 3000. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Figure 1 This is a schematic diagram of an embodiment 1000 of a programmable diffraction optoelectronic fusion computing system based on heterogeneous SLM collaboration according to the present invention. Figure 1 As shown, embodiment 1000 includes a hardware layer 101, a control driver layer 102, an algorithm layer 103, and an application interface layer 104.

[0023] Optionally, the hardware layer 101 in embodiment 1000 includes a core heterogeneous SLM 1011, a laser source and optical beam expander system 1012, a 4f filter system 1013, and an image sensor 1014.

[0024] The core heterogeneous SLM1011 includes a digital micromirror device module a and a silicon-based liquid crystal SLM module b. Specifically, the digital micromirror device module a serves as the front end for data scheduling and feature extraction, with a refresh rate greater than the first frame rate threshold; the silicon-based liquid crystal SLM module b serves as the core for wavefront transformation and connection.

[0025] In some specific embodiments, the core heterogeneous SLM1011 in this invention abandons the conventional approach of using a single type of SLM, creatively combining a digital micromirror device (DMD) module a with a liquid crystal on silicon (LCOS) SLM module b, and assigning them fixed and collaborative computing roles. The DMD module a, leveraging the physical characteristics of its micromirror mechanical flipping, is responsible for high-speed data scheduling and feature extraction. The LCOS SLM module b, utilizing the electrically controlled birefringence effect of liquid crystal molecules, is responsible for high-precision wavefront transformation and linear calculation. This heterogeneous division of labor—"DMD front-end + LCOS core"—is the core difference between this embodiment and existing technologies, aiming to unify the system's speed and accuracy requirements at the hardware level.

[0026] Optionally, the operations performed by the digital micromirror device module a include: the system receiving a digital image to be processed; the control driver layer 102 converting the digital image into a series of binary or pulse-width modulation (PWM) grayscale bitmap sequences; these bitmaps being loaded into the display buffer of the DMD via a high-speed interface; and millions of micromirrors on the DMD switching between "on" and "off" states based on the bitmap pixel values, modulating the incident uniform laser into a dynamic light field carrying input information, with a refresh rate far exceeding that of traditional LCOS.

[0027] Optionally, the operation performed by the silicon-based liquid crystal SLM module b includes: illuminating the LCOS panel with a light field modulated by DMD and filtered by the 4f system. The LCOS has been pre-loaded with a continuous phase map optimized by algorithm layer 103 for a specific computational task. This phase map precisely modulates the phase delay of the wavefront of the incident light field by adjusting the orientation of the liquid crystal molecules on each pixel, essentially performing complex matrix multiplication operations in the optical domain, equivalent to a hidden layer in a diffraction neural network.

[0028] The laser source and optical beam expander system 1012 is used to provide the input optical signal for the core heterogeneous SLM 1011. In some specific embodiments, the main function of the laser source and optical beam expander system 1012 is to provide a highly stable and highly coherent light source and shape it into a uniform plane wave to illuminate the DMD panel.

[0029] Specifically, the laser source and optical beam expander system 1012 employs a helium-neon (He-Ne) laser or a corresponding semiconductor laser. The laser beam first passes through a spatial filter consisting of a microscope objective and a pinhole to filter out higher-order mode noise. It is then expanded and collimated by a beam expander composed of two aberration-correcting lenses with different focal lengths. The final output is a circular spot with a flat wavefront, covering the entire DMD chip.

[0030] In some specific embodiments, the 4f filter system 1013 is used to filter the input optical signal. The 4f filter system 1013 is one of the key optical innovations of Embodiment 1000; it is not a simple imaging lens, but rather performs the dual functions of noise filtering and optical field reshaping. Because the binary modulation of the DMD generates strong, information-independent high-order diffraction noise, direct imaging will introduce interference at the LCOS plane. By setting up the 4f filter system 1013 and adding an adjustable aperture to its Fourier surface, these high-frequency noises can be effectively filtered out.

[0031] Optionally, the digital micromirror device module a and the silicon-based liquid crystal SLM module b are optically connected via a 4f filtering system 1013. The 4f filtering system 1013 includes two lenses and an adjustable aperture disposed on the confocal plane of the two lenses. The adjustable aperture is used to filter out diffraction noise generated by the digital micromirror device module a that is higher than a set spatial frequency.

[0032] In some specific embodiments, the 4f filtering system 1013 consists of two aberration-correcting lenses L1 and L2 with the same focal length, arranged coaxially at a distance of twice the focal length. The DMD is located at the front focal plane of L1, and the LCOS is located at the rear focal plane of L2. A mechanical aperture with a precisely adjustable aperture is placed on the confocal plane (i.e., the Fourier plane) of L1 and L2. During operation, the aperture is first opened to its maximum to observe the light spot on the rear CMOS, and then the aperture is gradually narrowed. When the aperture is reduced to allow only the brightest zero-order diffraction spot in the center, i.e., the light spot carrying the main information, to pass through, higher-order diffraction light is blocked. At this point, the light spot on the image plane is the purest, and the signal-to-noise ratio is the highest.

[0033] Optionally, the image sensor 1014 is used to detect the output light signal. Specifically, the image sensor 1014, as the output terminal of the system, converts the final light field after DMD encoding, 4f filtering, and LCOS modulation into a digital signal for subsequent processing or as a direct calculation result.

[0034] The control and driving layer 102 connects to and drives the hardware layer 101, and is used for synchronous driving and data flow management of the digital micromirror device module a and the silicon-based liquid crystal SLM module b. Specifically, the control and driving layer 102 includes: a synchronous driving module for sending synchronous trigger signals to the digital micromirror device module and the silicon-based liquid crystal SLM module; and a data flow management module for coordinating the data timing between pattern loading of the digital micromirror device module, phase map loading of the silicon-based liquid crystal SLM module, and image acquisition by the image sensor.

[0035] In some specific embodiments, the algorithm layer 103 interacts with the control driving layer 102 and includes a diffraction neural network model library and a physical embedding adaptive training framework, used to train and optimize the diffraction neural network model based on the physical characteristics of the hardware layer 101. The physical embedding adaptive training framework, during training, uses the optical modulation characteristics of the hardware layer 101, the physical parameters of the digital micromirror device module a and the silicon-based liquid crystal SLM module b, and the transfer function of the 4f filter system 1013 as part of the forward propagation model for end-to-end gradient optimization.

[0036] Optionally, the Diffractive Neural Network Model Library serves as a pre-built, pre-optimized algorithm repository, providing various deployable diffractive neural network architectures, such as feedforward diffractive deep neural networks, diffractive recurrent neural networks suitable for sequence processing, and network architectures within diffractive networks with more complex connection patterns. Its purpose is to provide users and systems with a high starting point, avoiding the need to design networks from scratch and accelerating the task deployment process.

[0037] In traditional technical processes, applying diffractive neural networks to achieve specific tasks requires users to start from scratch, consulting literature to design the number of diffractive network layers and phase parameterization methods, and then training it in an ideal simulation environment. After training, the trained phase matrix must be manually converted into a hardware format, then uploaded to LCOS via separate hardware control software, and the DMD input must be configured separately. The entire process is fragmented, cumbersome, and cannot guarantee the performance of the simulation model on real hardware.

[0038] The implementation process of this plan includes: Model Selection: Users can view the pre-built diffraction neural network model library by calling functions through the Python SDK in the application interface layer 104. Users can load a specific task model from the diffraction neural network model library with a single line of code.

[0039] Task adaptation: Users call the interface using their own task dataset. During this fine-tuning process, the physical embedding adaptive training framework runs in the background, ensuring that the fine-tuning is performed within the constraints of the current system hardware characteristics.

[0040] One-click deployment: After fine-tuning, users simply call `deploy(model)`. The SDK automatically completes all subsequent steps, including calling the training framework for final optimization, compiling the model into hardware parameters, and deploying it through the control driver layer.

[0041] The diffractive neural network model library in this invention provides users with a proven high starting point, reducing network architecture exploration and basic training time to a few hours. More importantly, because the model structure and training process in the library are compatible with the physical embedding framework, it ensures the continuity of the entire process from pre-trained model to final hardware deployment, avoiding the separation between simulation and hardware in traditional methods.

[0042] In some specific embodiments, the function of the physically embedded adaptive training framework in this solution is to seamlessly integrate the precise physical mathematical model of hardware layer 101 into the standard training process of neural networks, achieving joint optimization of network parameters and hardware operating points. Existing conventional technical processes typically involve software simulation training with fixed parameters and hardware deployment, with the two stages being separate. The traditional two-stage process includes: Phase 1: Pure Software Simulation. A diffraction neural network is trained in a computer using an optical propagation model and a phase modulator. The training objective is to minimize the error between the digital output and the label.

[0043] Phase Two Hardware Deployment: The trained ideal phase map is directly loaded onto the actual LCOS. Due to the nonlinear voltage-phase response, inter-pixel crosstalk, and phase noise of the actual LCOS, as well as the aberrations and diffraction efficiency loss in the actual optical path, the system's recognition accuracy is much lower than the simulation results.

[0044] The framework of this solution implements hardware-in-the-loop joint training and optimization, and its core technical points are mainly reflected in the following process: Step 1: Digitalization and Differentiable Modeling of Hardware Physical Characteristics. During system initialization or periodic calibration, the real hardware is accurately characterized, and a differentiable mathematical model is established: Model a of digital micromirror device module: Modeling its fill factor, micromirror tilt angle diffraction effect, and pulse width modulation timing response for grayscale.

[0045] Model b of silicon-based liquid crystal SLM module: Establish a continuous function fitting of voltage-phase response lookup table, and include a statistical model of inter-pixel crosstalk and phase fluctuation noise.

[0046] 4f filter system 1013 model: Based on actual measurements, model the aberration function of the lens and the point spread function or optical transfer function of the adjustable aperture under a specific aperture.

[0047] Step Two: Embedding the Physical Model as a Custom Layer into the Training Graph. The aforementioned hardware physical model is encapsulated as a custom neural network layer within a deep learning framework. During the forward propagation of training, the data (simulated light field) will sequentially pass through: DMDModulationLayer (Simulates DMD modulation effects based on the input image and model) 4fFilteringLayer (simulates the filtering and transmission process of a 4f system) LCOSPhaseModulationLayer (simulating phase modulation of LCOS) These layers are differentiable, meaning that gradients can propagate back through these physical models.

[0048] Step 3: End-to-end joint gradient optimization. The error in the loss function calculation is used through backpropagation to update not only the weights of the diffraction neural network, but also selectively update certain adjustable parameters in the physical model. For example, optimizing the mapping relationship of the LCOS driver lookup table to find the optimal operating voltage range to maximize linearity; providing optimization suggestions for the PWM encoding mode of the DMD to better approximate the target grayscale with a limited number of refresh bits.

[0049] This process allows the algorithm to automatically learn how to compensate for or circumvent the non-ideal characteristics of the hardware, thereby finding the optimal model parameters for this specific hardware.

[0050] Optionally, a specific embodiment of the hardware-in-the-loop joint training and optimization process implemented by the physical embedding adaptive training framework in this invention includes: First, the digitization and differentiable modeling of the hardware's physical characteristics enable automated calibration measurements to be performed in the actual system. For LCOS module b: A series of voltages are applied pixel by pixel, and the actual phase shift is measured using an interferometer to generate a measured voltage-phase response lookup table containing millions of data points. This lookup table is then fitted using a smooth, differentiable spline function to obtain the function Φ(V), where V is the driving voltage and Φ is the output phase. Simultaneously, a phase fluctuation noise model N(σ) is obtained through statistical analysis.

[0051] For DMD module a: accurately measure the fill factor (effective reflective area ratio) and diffraction efficiency curve of its micromirrors, and model the relationship between light intensity and depth under pulse width modulation I (PWM).

[0052] For the 4f filter system 1013: Under the set working aperture, measure its point spread function or optical transfer function H(fx, fy).

[0053] This step establishes not an ideal model, but a non-ideal high-fidelity mathematical model, and all models are differentiable, laying the foundation for gradient optimization.

[0054] Then, the physical model is embedded as a custom layer into the training graph. Optionally, a custom layer named PhysicalPropagation is created in the PyTorch framework, which includes an embedded model of a digital micromirror device module a, a silicon-based liquid crystal SLM module b, and a 4f filter system 1013 model. During training, the weight parameters (i.e., the phase map) of the neural network are first passed through the PhysicalPropagation layer, which applies the aforementioned hardware model and outputs a predicted actual hardware output light field.

[0055] Finally, end-to-end joint gradient optimization is performed, specifically including: Loss Calculation and Backpropagation: The output of the PhysicalPropagation layer is compared with the true label to calculate the loss. During backpropagation, the gradient passes through all the physical models in the PhysicalPropagation layer.

[0056] Joint optimization: Gradients update the phase map parameters of the network's main weights, enabling it to automatically learn how to compensate for hardware limitations. For example, in regions with slower LCOS responses, the network might learn to assign larger weights to phase changes. Gradients also update adjustable parameters in the physical model. For instance, the framework can set certain key voltage points in the LCOS-driven lookup table as trainable variables, related to the phase... Figure 1 Simultaneous optimization. Through training, the system may automatically discover a more linear and efficient operating voltage range than the factory settings.

[0057] According to such Figure 1 The embodiment shown illustrates how the physical embedding adaptive training framework proposed in this invention achieves hardware-in-the-loop optimization. The optimization objective is not a mathematical solution in an ideal space, but rather the optimal performance solution on specific hardware. The training process not only alters the algorithm but may also fine-tune the hardware's operating point.

[0058] In some specific embodiments, the application interface layer 104 is used to provide users with a software development kit (SDK) to deploy computing task models. The application interface layer 104, through a highly abstract SSD, completely encapsulates the complex underlying optical, electrical, and algorithmic details, thus realizing a general-purpose computing platform.

[0059] Optionally, the core components and functional implementations of the application interface layer 104 include: (1) Task description and compilation interface It allows users to define computational tasks in a high-level, intuitive way. Users don't need to describe optical paths or phase maps; instead, they can specify the network architecture, dataset, and training objectives through Python classes and configuration files, just like defining a regular neural network. Classes and methods are provided; after user submission, the interface layer automatically calls the algorithm layer to perform physical embedding compilation, generating a hardware-executable parameter sequence.

[0060] (2) Hardware Abstraction and Resource Management Interface Hardware details such as DMD, LCOS, and synchronization triggering are completely hidden from the user. What the user perceives is only a virtual computing resource called OpticalComputeUnit.

[0061] It provides relevant functions. Internally, this interface communicates with the control driver layer 102, automatically handling task queue loading, synchronization timing allocation, and computing resource locking, enabling multi-task queuing or exclusive access.

[0062] (3) Data management and result processing interface Simplify the data input and result acquisition process. Provide standardized data loaders and post-processing tools, allowing users to focus on task logic. Offer data interfaces compatible with popular AI frameworks. Computation results (camera images) are automatically converted into tensors with accompanying metadata, which users can directly retrieve and analyze using get_results(task_id).

[0063] According to such Figure 1 The embodiment shown in this invention heterogeneously collaborates a high-speed digital micromirror device (DMD) and a high-precision liquid crystal on silicon (LCOS) spatial light modulator, with the former playing the roles of fixed front end and the latter the core. Combined with physical embedding adaptive training and nanosecond-level synchronous driving, a four-layer computing system with deep hardware and software coupling is constructed. This improves upon the shortcomings of traditional programmable diffraction systems that cannot simultaneously handle high-speed data input and high-precision analog calculations on a single platform, achieving a balance between speed, accuracy, and flexibility. This significantly enhances the system performance, task agility, and practicality of optical intelligent computing.

[0064] Figure 2 for Figure 1 A schematic diagram of the control drive layer 102 in Embodiment 1000. (See attached diagram.) Figure 2 As shown, the control drive layer 102 includes a synchronization drive module 1021 and a data flow management module 1022.

[0065] The primary task of driving a single SLM system in the prior art is to send data unidirectionally and sequentially to a single device. However, in the solution of this invention, the control driving layer 102 needs to address the following technical problem: the refresh rate of the digital micromirror device module a differs from that of the silicon-based liquid crystal SLM module b by hundreds of times; the data input handled by the digital micromirror device module a and the computational transformation handled by the silicon-based liquid crystal SLM module b must be strictly aligned in time and precisely correspond logically; a single misalignment will cause the entire optical computation to fail.

[0066] In some specific embodiments, the synchronization drive module 1021 is used to send synchronization trigger signals to the digital micromirror device module and the silicon-based liquid crystal SLM module. Conventional technologies may provide independent clocks for the DMD and LCOS, or only perform simple delayed triggering, which cannot solve the long-term stable alignment under frame rate multiple relationships. The synchronization drive module 1021 innovatively adopts a slave triggering architecture with LCOS frame synchronization as the master clock.

[0067] Specifically, the implementation process of the synchronization drive module 1021 includes: (1) Clock cycle definition: The synchronous drive module 1021 uses the physical refresh cycle of LCOS as the optical computing clock cycle based on the system. The start of each clock cycle means that the LCOS panel is ready to load a new phase map, which means that it represents a new computing task or task step.

[0068] (2) Master-slave triggering: At the beginning of each optical computing cycle, the synchronous drive module 1021 simultaneously sends two precisely timed hardware trigger pulses, usually TTL signals: Trigger pulse A to the DMD: This pulse commands the DMD controller to immediately apply the next frame's encoded pattern, already prepared in its internal buffer, to the micromirror array. Because the DMD refreshes extremely quickly, this operation can be completed within microseconds. Trigger pulse B to the scientific CMOS camera: This pulse commands the camera to begin an exposure that is precisely matched to the current light calculation beat.

[0069] In this way, the synchronous drive module 1021 controls the data so that, regardless of how fast the DMD refreshes, it only updates the data once at the beginning of each LCOS calculation cycle. This ensures that the light field information illuminating the light field is static and stable throughout the entire period during which the LCOS continuously modulates the light field, thus achieving phase-locking and fusion of high-speed data streams and low-speed precision calculations in the time dimension.

[0070] In some specific embodiments, the data flow management module 1022 is used to coordinate the data timing between pattern loading of the digital micromirror device module, phase map loading of the silicon-based liquid crystal SLM module, and image acquisition of the image sensor.

[0071] The timing management of the data flow management module 1022 relies on the following key technical processing that differs from conventional single-device drivers: (1) Atomized encapsulation of frame tasks: For image recognition tasks, the algorithm layer generates a phase map and multiple frames of DMD patterns. The data flow management module 1022 decomposes them into atomic frame tasks. Each frame task data packet strictly contains three parts: ① a DMD binary / grayscale pattern; ② a corresponding LCOS continuous phase map; ③ a pre-allocated memory address for storing the result image of this calculation.

[0072] This encapsulation ensures that every time hardware synchronization is triggered, all devices know what to do and where to put the results, thus achieving determinism in the computation process.

[0073] (2) Double buffering and pipeline preloading: Both DMD and LCOS hardware driver boards have an active foreground cache and a background cache for preparation.

[0074] During the current Nth optical computation cycle, i.e., while LCOS is modulating and the camera is exposing, the data stream management module 1022 silently writes the DMD pattern and LCOS phase map from the N+1th frame task into the background buffers of the two devices respectively via the high-speed bus. By hiding the time-consuming data transmission process within the computation cycle, complete parallelism between computation and transmission is achieved, eliminating system idle waiting caused by data transmission delays.

[0075] (3) Precise linkage switching with synchronous drive: When the global hardware trigger pulse issued by the synchronous drive module 1021 arrives, it will simultaneously send a software interrupt signal to the data flow management module 1022.

[0076] The data stream management module 1022 then sends a buffer switching command to the DMD and LCOS controllers. Upon receiving this command, the two device controllers atomically swap the background buffer with the foreground buffer. Physically, the micromirror array of the DMD and the liquid crystal molecules of the LCOS are aligned and updated to the state required for the next frame's task almost simultaneously. This ensures that the light field transformation is perfectly aligned in time.

[0077] (4) Active management of image acquisition: The data stream management module 1022 not only sends data, but also manages the retrieval of data. The camera is pre-configured and the image storage address is bound to the frame task ID.

[0078] After the camera completes exposure, its image data is directly written to a predetermined memory address via the bus. The data stream management module 1022 confirms the completion of data writing through polling or interruption and matches the result with the records in the task queue. This avoids image data corruption and loss, ensuring that each output image accurately corresponds to specific input and calculation parameters, which is crucial for subsequent result analysis and algorithm closed-loop optimization.

[0079] According to such Figure 2 As illustrated, the data flow management module 1022 of this invention, through this combination of atomic encapsulation, pipeline preloading, hard-triggered synchronous switching, and proactive result management mechanism, not only ensures perfect coordination of heterogeneous SLMs within every microsecond-level computation cycle, but also achieves the system's rapid reconfiguration capability to complete complex task switching on a millisecond-level scale. This fundamentally distinguishes it from simple multi-device drivers, making it a core software hub that transforms heterogeneous hardware into a high-efficiency, agile, and unified computing platform.

[0080] Optionally, the system's reconstruction time is less than a first time threshold. The reconstruction time refers to the total time from receiving a new task instruction to completing the parameter loading of the digital micromirror device module a and the silicon-based liquid crystal SLM module b, and bringing the system output to a stable state. Optionally, the first time threshold is 200ms.

[0081] In some specific embodiments, the system reconfiguration process refers to the process of switching from a deployed computing task A to a completely new task B. Its rapid implementation relies entirely on the aforementioned data flow management mechanism: Receiving instructions: When the application interface layer receives a new task instruction from the user, it immediately interrupts the pipeline loading of the current task queue.

[0082] Emergency preloading: The algorithm layer quickly compiles the first frame data packet for the new task B. The data stream management module immediately inserts it into the transmission pipeline, preempting the data of the old task A that is currently being transmitted, and loads it into the background caches of DMD and LCOS.

[0083] Synchronous switching: The system will not wait for the current clock cycle to complete. It will force a cache switch when the next synchronization trigger pulse arrives, allowing the DMD and LCOS to load the parameters of the new task B.

[0084] Stable output: Due to the physical response time of the optical system, the output light field usually stabilizes within 1-2 calculation cycles after the new parameters are loaded, and the camera can collect valid results.

[0085] Traditional systems based on fixed diffraction elements or requiring complex calibration may require manual replacement of elements or recalibration and training that takes several minutes or even hours when switching tasks, making them unable to cope with dynamic scenarios.

[0086] In some specific embodiments of the present invention, the reconstruction time is constrained within a relatively short first time threshold, which has the following beneficial technical effects: It achieves software definition, allowing users to dynamically switch the functions of the optical computing system like switching software programs on a computer, greatly improving the system's flexibility and practicality. It meets real-time requirements, enabling the system to be applied to scenarios requiring rapid response to environmental changes, such as real-time target tracking and interactive optical processing; and it reflects the system's hardware and software synergy efficiency, combining rapid hard switching with efficient soft scheduling.

[0087] Figure 3 This is a schematic diagram of an embodiment 2000 of a programmable diffraction optoelectronic fusion computing system based on heterogeneous SLM collaboration according to the present invention. Figure 3 As shown, in embodiment 2000, the control drive layer 202, algorithm layer 203, application interface layer 204, and... Figure 1 The control driver layer 102, algorithm layer 103, and application interface layer 104 in the illustrated embodiment 1000 are the same; the core heterogeneous SLM 2011, laser source and optical beam expander system 2012, 4f filter system 2013, image sensor 2014, and... Figure 1The core heterogeneous SLM 1011, laser source and optical beam expander system 1012, 4f filter system 1013 and image sensor 1014 in embodiment 1000 are the same, and will not be described again here.

[0088] The hardware layer 201 in embodiment 2000 also includes a polarization control element 2015, which is disposed in the optical path of the 4f filter system 2013 and is used to adjust the polarization direction of the light signal incident on the silicon-based liquid crystal SLM module b.

[0089] In some specific embodiments, the silicon-based liquid crystal SLM module b introduces a controllable phase delay into the incident light field by altering the alignment of liquid crystal molecules using voltage. However, this electrically controlled birefringence effect of liquid crystal molecules exhibits a strong polarization dependence: the maximum and purest phase modulation depth and efficiency can only be obtained when the linear polarization direction of the incident light aligns with the guiding axis direction of the liquid crystal molecules. If the polarization directions do not match, some light energy does not participate in effective phase modulation and may even be converted into orthogonally polarized stray light, leading to decreased modulation efficiency, reduced contrast, and the introduction of noise.

[0090] In Embodiment 2000 of this invention, the laser beam is reflected by the digital micromirror device module a and transmitted through the lens of the 4f filter system 2013. Its polarization state may undergo uncontrollable rotation or degradation. Therefore, a polarization control element 2015 is inserted into the optical path. Its core function is to actively correct the polarization state of the light signal incident on the silicon-based liquid crystal SLM module b to a state that best matches the alignment angle of the LCOS liquid crystal molecules.

[0091] Optionally, the polarization control element 2015 is a precisely rotatable linear polarizer or a polarization state adjuster composed of a λ / 2 waveplate and a fixed polarizer. Its implementation involves two steps: During system initialization or periodic maintenance calibration: A temporary, simple photodetector is placed on the Fourier surface of the 4f filter system 2013, or the system's own scientific CMOS camera is used directly. The control software drives the LCOS to load a series of known standard phase maps. Simultaneously, the polarization control element is rotated slowly, automatically or manually, and the contrast or diffraction efficiency of the light intensity signal received by the detector is monitored in real time. When the monitored signal reaches an extreme value, it indicates that the polarization direction of the incident light has achieved optimal matching with the liquid crystal alignment angle of the LCOS. At this point, the angle of the polarization element is locked or recorded.

[0092] During operation: In subsequent normal calculation tasks, the polarization control element 2015 will operate at the optimal angle obtained from calibration, ensuring that the incident light with the optimal polarization state is always provided to LCOS.

[0093] According to such Figure 3The embodiment shown in this invention introduces and correctly configures polarization control elements, bringing direct performance improvements to the system: through polarization matching, most of the incident light energy is used to generate effective phase delay, rather than being lost or converted into noise, maximizing the phase modulation efficiency and light energy utilization of LCOS; the optimization of polarization state significantly reduces leakage light and stray light that do not participate in modulation, improving the overall signal-to-noise ratio and calculation accuracy of the system; under optimal polarization conditions, the voltage-phase response curve of LCOS is closer to the ideal linear relationship and is less affected by changes in temperature and driving conditions, ensuring the linearity and stability of phase modulation characteristics.

[0094] Figure 4 This is a flowchart illustrating an embodiment 3000 of the programmable diffraction photoelectric fusion calculation method based on heterogeneous SLM collaboration according to the present invention. Figure 4 As shown, Example 3000 includes steps S1-S4.

[0095] In step S1, the system receives the user-defined computation task and the corresponding diffraction neural network model through the application interface layer. Specifically, the user submits the computation task by calling the high-level API through the Python SDK provided by the application interface layer. For example, the API can be defined as: `submit_task(task_model='D2NN', dataset=mnist_data)`, where `task_model` can be selected from the diffraction neural network model library, and `dataset` is training data conforming to a standardized format. The application interface layer converts the user's abstracted high-level instructions into a standardized task description file that the system can process internally. This step encapsulates all hardware details, eliminating the need for the user to understand the underlying optical path, thus realizing a soft-defined entry point.

[0096] In step S2, in the algorithm layer, the diffraction neural network model is trained using a physical embedding adaptive training framework combined with the physical parameters of the hardware layer to obtain optimized model parameters adapted to the hardware layer. The optimized model parameters include the encoding pattern to be loaded into the digital micromirror device module and the continuous phase map to be loaded into the silicon-based liquid crystal SLM module.

[0097] Optionally, in some specific embodiments, step S2 includes a forward propagation step and a backpropagation and optimization step. The forward propagation step integrates the modulation characteristics of the digital micromirror device module in the hardware layer, the phase modulation response curve of the silicon-based liquid crystal SLM module, and the filtering characteristics of the 4f filter system into the computational graph of the diffraction neural network model to simulate the physical propagation process of the optical signal in the real hardware. The backpropagation and optimization step calculates the loss based on the difference between the output of the forward propagation step and the expected target, and simultaneously updates the weight parameters of the diffraction neural network model and the adjustable simulated values ​​of the hardware physical parameters using a gradient backpropagation algorithm.

[0098] In step S3, the system, through the control driving layer, loads the encoded pattern and continuous phase map in the optimized model parameters into the digital micromirror device module and the silicon-based liquid crystal SLM module in the hardware layer in a synchronous timing sequence.

[0099] In some specific embodiments, in step S3, the data flow management module of the control driver layer receives parameter packets from the algorithm layer and preloads these data packets into the onboard buffers of DMD and LCOS in a pipelined manner via a high-speed bus. The synchronization driver module then generates and emits a global hardware trigger pulse according to a preset clock cycle.

[0100] A global hardware trigger pulse ensures atomic synchronization of two key operations: the DMD controller applies the next frame's encoded pattern to the micromirror array; and the LCOS controller loads the corresponding phase map onto the liquid crystal panel. This nanosecond-level synchronization ensures that the LCOS senses the correctly corresponding input image that the DMD has just refreshed the moment it begins to modulate the light field, thus solving the fundamental problem of speed mismatch in heterogeneous devices.

[0101] In step S4, the system activates the laser source in the hardware layer, causing the input optical signal to sequentially pass through the encoding modulation of the digital micromirror device module, the filtering of the 4f filtering system, and the phase modulation of the silicon-based liquid crystal SLM module, before being detected by the image sensor and the calculation result is output. Specifically, the specific operation process of step S4 includes: the laser is turned on, and the collimated and expanded laser beam sequentially passes through: DMD module: Modulated by a high-speed coded pattern, carrying input information.

[0102] 4f filter system: filters out high-order diffraction noise generated by DMD and purifies the light field.

[0103] LCOS module: Performs precise wavefront transformation on a continuous phase map to complete the core linear calculation.

[0104] Image sensor: detects and digitizes the final output light intensity distribution.

[0105] The entire optical computation process in step S4 is completed within microseconds to milliseconds, with extremely low end-to-end latency. The resulting image captured by the camera represents the inference result of the diffraction neural network on the input data.

[0106] According to such Figure 4 The embodiment shown in the invention solves the gap between simulation and reality through physical embedding training and resolves the contradiction between speed and accuracy through heterogeneous synchronous driving. Ultimately, it enables a complex heterogeneous optoelectronic system to reliably and efficiently execute programmable intelligent computing tasks, just like running software.

[0107] Figure 5for Figure 4 A flowchart illustrating step S2 in Example 3000. (See attached diagram.) Figure 5 As shown, step S2 includes forward propagation step S21 and back propagation and optimization step S22.

[0108] In the forward propagation step S21, the system integrates the modulation characteristics of the digital micromirror device module, the phase modulation response curve of the silicon-based liquid crystal SLM module, and the filtering characteristics of the 4f filter system in the hardware layer as differentiable constraints into the computational graph of the diffraction neural network model to simulate the physical propagation process of the optical signal in the real hardware. This process simulates the complete physical link effect from digital input to final optical field output, rather than the abstract transformation under an ideal mathematical model.

[0109] In some specific embodiments, step S21 is implemented by including: Model Integration: The training framework loads the accurate physical model of the current hardware layer from the database. This includes: the fill factor and diffraction angle model of the DMD module to accurately predict its binary diffraction and actual light energy distribution; the measured voltage-phase response lookup table of the LCOS module to accurately simulate its nonlinear phase modulation and spatial crosstalk; and the optical transfer function of the 4f system under the current aperture setting to characterize its low-frequency filtering and aberration characteristics.

[0110] Constructing a differentiable computational graph: The physical model described above is defined as a series of differentiable custom operation layers and inserted into the computational graph of the selected diffraction neural network. During training, each batch of input data is first simulated as an optical field, and then processed sequentially through these custom layers: simulated DMD encoding, simulated 4f system frequency domain filtering, and simulated LCOS phase modulation.

[0111] In the backpropagation and optimization step S22, the system calculates the loss based on the difference between the output of the forward propagation step and the expected target, and achieves collaborative optimization of algorithm parameters and hardware physical parameters through the gradient backpropagation algorithm. Simultaneously, it updates the weight parameters of the diffraction neural network model and the adjustable simulated values ​​of the hardware physical parameters. Optionally, in some specific embodiments, the diffraction neural network model is a diffraction deep neural network, a diffraction recurrent neural network, or a network architecture within a diffraction network.

[0112] In some specific embodiments, step S22 is implemented by including: Loss calculation: Compare the simulated output obtained in step S21, which carries the full-link hardware effects, with the task objective to calculate the loss value.

[0113] Gradient backpropagation and joint update: Through gradient backpropagation, the gradient generated by the loss value needs to be propagated backward through the entire custom computational graph containing the physical model. This not only updates the weights of the network itself, but also passes through the physical model layers, updating the adjustable parameters in the model. For example: The mapping relationship of the LCOS driver lookup table is automatically fine-tuned to find a more linear working range. The optimization results will be directly reflected in the final generated continuous phase diagram, so that the phase value and the actual applied voltage achieve optimal matching. Optimize the DMD binary coding strategy (such as the bit depth allocation of pulse width modulation) to better approximate the target gray level. This strategy will directly convert into a specific coding pattern to be loaded with the least loss under a limited number of refresh bits.

[0114] According to such Figure 5 In the illustrated implementation, the final encoded pattern and continuous phase map obtained by the present invention are a set of inseparable parameter pairs jointly optimized for collaborative operation of current heterogeneous SLM systems. They not only carry the computational task logic but also embed a compensation mechanism for hardware defects, thus enabling high-precision computation on non-ideal physical hardware. This is precisely the core technological advancement that distinguishes it from the traditional approach of first performing ideal simulation and then deploying hardware.

[0115] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0116] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A programmable diffraction optoelectronic fusion computing system based on heterogeneous SLM collaboration, characterized in that, It includes a hardware layer, a control driver layer, an algorithm layer, and an application interface layer connected in sequence, wherein: The hardware layer includes: The core heterogeneous SLM includes a digital micromirror device module and a silicon-based liquid crystal SLM module; wherein, the digital micromirror device module serves as the front end for data scheduling and feature extraction, and its refresh rate is greater than the first frame rate threshold; the silicon-based liquid crystal SLM module serves as the core for wavefront transformation and connection. A laser source and optical beam expander system are used to provide the input optical signal for the core heterogeneous SLM; a 4f filter system is used to filter the input optical signal; and an image sensor is used to detect the output optical signal. The control and driving layer connects to and drives the hardware layer, and is used to perform synchronous driving and data flow management of the digital micromirror device module and the silicon-based liquid crystal SLM module. The algorithm layer interacts with the control and drive layer and includes a diffraction neural network model library and a physical embedding adaptive training framework, used to train and optimize the diffraction neural network model according to the physical characteristics of the hardware layer. The application interface layer is used to provide users with software development kits to deploy computing task models.

2. The system according to claim 1, characterized in that, The digital micromirror device module and the silicon-based liquid crystal (SLM) module are optically connected through the 4f filtering system, which includes two lenses and an adjustable aperture disposed on the confocal plane of the two lenses.

3. The system according to claim 2, characterized in that, The adjustable aperture is used to filter out diffraction noise that is higher than the set spatial frequency after being modulated by the digital micromirror device module.

4. The system according to claim 1, characterized in that, The control drive layer includes: A synchronization drive module is used to send synchronization trigger signals to the digital micromirror device module and the silicon-based liquid crystal SLM module; The data flow management module is used to coordinate the data timing between pattern loading of the digital micromirror device module, phase map loading of the silicon-based liquid crystal SLM module, and image acquisition of the image sensor.

5. The system according to claim 4, characterized in that, The reconstruction time of the system is less than the first time threshold. The reconstruction time refers to the total time from receiving the new task instruction to completing the parameter loading of the digital micromirror device module and the silicon-based liquid crystal SLM module and making the system output stable.

6. The system according to claim 1, characterized in that, The physical embedding adaptive training framework is used to perform end-to-end gradient optimization by incorporating the optical modulation characteristics of the hardware layer, the physical parameters of the digital micromirror device module and the silicon-based liquid crystal SLM module, and the transfer function of the 4f filter system as part of the forward propagation model during training.

7. The system according to claim 1, characterized in that, The hardware layer also includes a polarization control element, which is disposed in the optical path of the 4f filter system and is used to adjust the polarization direction of the light signal incident on the silicon-based liquid crystal SLM module.

8. A programmable diffraction photoelectric fusion calculation method based on heterogeneous SLM collaboration, applied to the system as described in any one of claims 1-7, characterized in that, The method includes: The application interface layer receives user-defined computational tasks and corresponding diffraction neural network models. In the algorithm layer, the physical embedding adaptive training framework is used to train the diffraction neural network model in combination with the physical parameters of the hardware layer to obtain optimized model parameters adapted to the hardware layer. The optimized model parameters include the encoding pattern to be loaded into the digital micromirror device module and the continuous phase map to be loaded into the silicon-based liquid crystal SLM module. Through the control driving layer, the encoded pattern and continuous phase map in the optimized model parameters are loaded into the digital micromirror device module and silicon-based liquid crystal SLM module in the hardware layer in a synchronous timing sequence. The laser source of the hardware layer is activated, and the input optical signal is sequentially encoded and modulated by the digital micromirror device module, filtered by the 4f filter system, and phase-modulated by the silicon-based liquid crystal SLM module. The image sensor then detects and outputs the calculation result.

9. The method according to claim 8, characterized in that, The training using the physical embedding adaptive training framework specifically includes: Forward propagation step: The modulation characteristics of the digital micromirror device module, the phase modulation response curve of the silicon-based liquid crystal SLM module, and the filtering characteristics of the 4f filter system in the hardware layer are integrated into the calculation graph of the diffraction neural network model to simulate the physical propagation process of optical signals in real hardware. Backpropagation and optimization steps: Calculate the loss based on the difference between the output of the forward propagation step and the expected target, and update the weight parameters of the diffraction neural network model and the adjustable hardware physical parameter simulation values ​​simultaneously through the gradient backpropagation algorithm.

10. The method according to claim 9, characterized in that, The diffraction neural network model is a diffraction deep neural network, a diffraction recurrent neural network, or a network architecture within a diffraction network.

Images