Optoelectronic device with periodic modulation structure, Talbot-synchronous class projection and recursive feedback

Abstract

Optoelectronic device comprising a coherent light source, a modulation device with a periodic modulation or sampling structure and a plurality of individually controllable modulation areas for amplitude and / or phase modulation of a light field, a propagation section between the modulation device and an observation plane, a detection device arranged in the observation plane with a plurality of sampling points, and an evaluation device, characterized in that the observation plane is arranged at a Talbot-synchronous distance or in its vicinity with respect to the periodic modulation or sampling structure, and the evaluation device forms respective class channels from detection values ​​of such sampling points that occupy the same relative position within different periods, and generates a state or activation vector from the class channels that is reduced compared to the number of sampling points.

Description

Summary The invention relates to an optoelectronic device for processing coherent light fields, particularly for photonic information processing, optical neural networks, and optoelectronic reservoir systems. The device according to the invention features a periodic modulation structure, a Talbot-synchronous or near-Talbot observation plane, and a detection unit in which detection values ​​of the same relative position within different periods are grouped into class channels. This compresses the high-resolution intensity distribution physically and / or sensor-wise into a few reproducible state channels. In an advantageous embodiment, the reduced state vector is fed to a feedback device, which generates an updated modulation distribution for a subsequent time interval, thus enabling recursive state formation in the sense of an optoelectronic reservoir system.The device does not require optically nonlinear materials, as the nonlinearity is provided by intensity detection and, if necessary, thresholding in the optoelectronic path. Technical field The invention relates to optoelectronic devices for processing, projection, and evaluation of coherently propagating light fields. It is particularly suitable for diffractive and holographic information processing, physically realized neural networks, sensor-based state reduction, photonic reservoir systems, and modular research and teaching platforms in the field of coherent photonics. A current focus of such systems lies in the realization of large linear transformations with low energy consumption. Here, coherent light fields are manipulated by periodically or locally selectively controllable modulation structures, guided over defined propagation paths, and detected in an observation plane. Of particular interest are architectures in which optical propagation is used as a deterministic linear mode transformation and the measured intensity distribution is not fully read out in high dimensions, but rather structurally transformed into a reduced number of reproducible state channels. State of the art Diffractive and free-space optical computing systems are known from the prior art, in which coherent or partially coherent light fields are modulated by masks, diffractive layers, metamaterials, or other optically effective structures and, after propagation, evaluated for classification, inference, measurement, or acquisition tasks. These include, in particular, diffractive optical neural networks and related optical neural networks, in which the essential computational operation consists of linear wave propagation and field superposition (see Fu et al., Light: Science & Applications, 2024, “Optical neural networks: progress and challenges”). Furthermore, optical neural networks with explicit nonlinear activation mechanisms are known. These include optical approaches using nonlinear materials as well as optoelectronic approaches where the nonlinearity is provided by intensity detection and subsequent processing. As an example, Wang et al. describe a multilayer nonlinear optical neural network for image sensing, in which images are not mapped in the classical sense, but rather optically encoded into a low-dimensional feature space (see Wang et al., Nature Photonics, 2023, “Image sensing with multilayer nonlinear optical neural networks”). Furthermore, lensless optoelectronic neural networks are known. As an example, Shi et al. describe a “lensless opto-electronic neural network” (LOEN) for machine vision applications. They explain that known optoelectronic hybrid networks often have complex structures, frequently rely on coherent light sources, and are therefore only suitable to a limited extent for natural lighting environments. They further describe that image acquisition and image signal processing can still account for the majority of energy consumption in known optoelectronic hybrid architectures (see Shi et al., Light: Science & Applications, 2022, “LOEN: Lensless opto-electronic neural network empowered machine vision”). Furthermore, Talbot-based sensor concepts are known. For example, Wang, Gill, and Molnar describe pixel-sized sensor structures based on the Talbot effect, employing local diffraction gratings and an analyzer structure to detect intensity and incident direction. This work also explains that previous macroscale approaches for evaluating Talbot self-images require direct imaging of the self-images and significant computational effort for conversion into light field information. It further states that direct sensor resolution of Talbot structures at the microscale would require an imager structure with extremely high pixel density (see Wang et al., Applied Optics, 2009, "Light field image sensors based on the Talbot effect"). Finally, reservoir computing systems and photonic reservoir computing approaches are also known.A recent review describes reservoir computing as an established field with diverse physical implementations while also identifying ongoing challenges for its practical application. In particular, it highlights existing gaps in the transfer of theoretical RC models to physical systems and the problem of timescale matching between the computational task and the internal dynamics of the physical substrate (see Yan et al., Nature Communications, 2024, “Emerging opportunities and challenges for the future of reservoir computing”). The disadvantages of the existing solutions are therefore: In diffractive and free-space optical neural networks, optical processing is primarily based on linear wave propagation. Therefore, for more efficient multilayer processing, an additional nonlinearity must be provided. Both recent review articles and newer experimental studies explicitly identify the realization of suitable nonlinearity as a central challenge in the field (see Fu et al., 2024; Xia et al., Nature Photonics, 2024, “Nonlinear optical encoding enabled by recurrent linear scattering”). Despite optical preprocessing, known optoelectronic hybrid architectures and image-processing optical neural networks typically still require considerable effort for sensor readout, image signal processing, and electronic post-processing. Furthermore, existing solutions are sometimes structurally complex and, in some cases, dependent on coherent illumination, which complicates practical application under natural or broadband lighting conditions (see Shi et al., 2022; Wang et al., 2023). While Talbot-based sensor concepts enable the use of periodic self-images, they either result in significant computational effort for direct self-image analysis or place high demands on local sensor resolution if the Talbot structure is to be spatially resolved directly. Particularly in the microscale range, a correspondingly high-density imager structure is described as difficult to fabricate (see Wang et al., 2009). Furthermore, physical reservoir computing systems face the problem that theoretical RC models cannot be readily transferred to physical substrates and that the timescales between the task and the substrate's internal dynamics must be aligned. This complicates a simple, compact, and robustly parameterizable physical implementation of recursive systems (see Yan et al., 2024). The evaluated sources contain no indication of a device in which a periodic modulation or sampling structure is specifically combined with a Talbot-synchronous or near-Talbot observation plane in such a way that detection values ​​of the same relative position within different periods are directly aggregated into a small number of class channels, and these reduced class channels are used as a state vector for recursive feedback. This statement should be understood as a distinguishing finding compared to the sources evaluated here. Object of the invention The invention is based on the objective of providing an optoelectronic device with which a coherently modulated light field, after free-space propagation in a Talbot-synchronous or near-Talbot observation plane, can be evaluated in such a way that the high-resolution intensity distribution is transformed into a reduced, geometrically defined state representation without requiring full-area, pixel-by-pixel single-channel readout. This formulation derives directly from the disadvantages of known optical neural networks, Talbot sensors, and physical reservoir systems described above. In particular, a device is to be created in which detection values ​​of the same relative position within different periods of a periodic modulation or sampling structure are combined into class channels, so that the readout and processing effort is reduced compared to a complete pixel-wise sensor evaluation. Furthermore, the device should be designed in such a way that the reduced state vector can be fed into a feedback loop, which generates an updated modulation distribution for a subsequent time period, thus enabling recursive state formation in the sense of an optoelectronic reservoir system with reduced structural effort. Description of the invention The present invention solves the aforementioned problem by means of an optoelectronic device comprising a coherent light source, a modulation unit with a periodic modulation or scanning structure, a propagation section, a detection unit arranged in a Talbot-synchronous or near-Talbot observation plane, and an evaluation unit. The modulation unit has a plurality of individually controllable modulation areas and serves for the amplitude and / or phase modulation of an incident light field. The periodic structure can be defined by the pixel grid of the modulation unit itself and / or by an additional periodic optical structure. The free coherent propagation behind the periodic structure causes a geometrically determined reorganization of the intensity distribution in the observation plane. If the observation plane is positioned at a distance corresponding to an integer or fractional multiple of a Talbot distance, or within a calibrated range around such a distance, periodicity-related intensity patterns emerge in which detection values ​​of the same relative position within different periods systematically correspond to one another. The invention specifically exploits this relationship by not processing the detection values ​​acquired in the observation plane completely pixel-by-pixel as individual channels, but rather by grouping them into class channels according to their relative position within the periodicity. Classification is thus performed geometrically and periodically. In particular, detection values ​​that occupy the same raster phase or the same relative coordinate within different repetitions of the periodic structure are assigned to a common class. A state or activation vector, reduced compared to the number of individual detection points, is generated from the class channels. This physically and / or sensor-wise compresses the high-resolution intensity distribution into a small number of reproducible readout channels without altering the underlying linear field transformation. In an advantageous embodiment, class formation is performed one-dimensionally along a preferred periodic direction. Detection values ​​of the same raster phase within this direction are grouped together. In another advantageous embodiment, class formation is performed two-dimensionally by grouping detection values ​​according to pairs of relative positions within two periodic directions. The classes can thus be considered residual classes of a one-dimensional or two-dimensional periodic structure. The number of resulting class channels K is smaller than the number N of individual detection points, preferably significantly smaller, so that a compact and reproducible state representation is created. The summation of detection values ​​assigned to a class can be achieved through summation, averaging, weighted summation, integration over class areas, or other aggregation operations. Class channels can also be created through energy-related, threshold-based, or normalized evaluation of the associated detection values. Class formation can be performed purely computationally after detection. Alternatively, it can be implemented wholly or partially at the sensor level, particularly through electrically grouped pixel clusters, shared readout lines, grouped photodetectors, or pre-structured class areas on a sensor surface. Suitable detection devices include, in particular, CMOS sensors, CCD sensors, SPAD arrays, event-based sensors, or other groupable area sensors.In a preferred embodiment, a dense, lensless area sensor is arranged directly in or near the Talbot-synchronous observation plane. The use of a CMOS sensor is particularly advantageous, since its intensity detection provides a quadratic response and thus generates a nonlinearity usable for downstream information processing. The modulation device can be configured as a microdisplay, spatial light modulator, LCOS device, DMD, transmissive or reflective pixel matrix, phase mask, or a combination of amplitude- and phase-modulating devices. The periodic structure can be formed by the native pixel grid of this modulation device itself. Alternatively or additionally, another periodic optical structure can be provided, in particular a grating, a slit structure, an aperture array, an aperture array, or a diffractive mask. In this way, the periodicity can be defined independently of the specific modulator technology. In a preferred embodiment, the class projection according to the invention constitutes the actual optical state reduction of a photonic processing system. The reduced state or activation vector can be fed to a linear readout unit, a classification unit, a regression or control unit, or other downstream evaluation equipment. This makes the device particularly suitable for tasks involving classification, regression, prediction, control, and sensor information compression. In a particularly preferred embodiment, the device is designed as a recursive optoelectronic system. For this purpose, a feedback device is provided which feeds at least part of the reduced state or activation vector to a control or processing unit. The control or processing unit generates an updated modulation distribution for a subsequent time interval and returns this to the modulation device. In this way, a discrete recursive state formation is created, in which a sequence of linear field transformations is sequentially applied, the respective effect of which depends on the current system state. In an advantageous embodiment, the feedback process updates the phase distribution. Preferably, only a defined sub-area of ​​the modulation distribution is modified, in particular a programmable phase boundary or a boundary-defined modulation structure. This allows the state evolution to be influenced with minimal update effort, without requiring a complete recalculation and reassignment of the entire modulator area for each time step. Alternatively or additionally, an amplitude-modulating stage can be updated. In another embodiment, the detection device is designed as a spiking-based CMOS detection system in which photogenerated charge is collected in sensor elements, particularly in charge-integrating pixel structures, over an integration interval. Upon reaching a predetermined charge, voltage, and / or threshold value, an event or spike signal is generated. The event or spike signal can be configured as a binary, multi-level, and / or time-coded signal. It can be used directly for class formation and / or as a state variable for feedback. In a further embodiment, a first modulation stage is provided for amplitude-based coding of an external input signal, and a second modulation stage for phase-based or limit-based coding of the feedback state information. It is also possible to superimpose an external input signal and the feedback state information in a common modulation stage or to display them in separate spatial areas of the same modulation device. In this way, the device can be operated as an optoelectronic reservoir system with external excitation and internal state dynamics. The modulation device can influence the light field by changing the amplitude, phase, and / or polarization state.In particular, with liquid crystal or LCOS-based modulation devices, the modulation can be wholly or partially polarization-dependent, optionally in combination with at least one polarizer and / or analyzer. The optical propagation at the core of the device remains linear. The nonlinearity required for the recursive dynamics arises only through intensity detection, particularly due to the quadratic detection behavior of an image sensor, and can be further influenced by thresholding, quantization, normalization, amplification, event-based signaling, and / or time delay in the feedback. The feedback mechanism can therefore incorporate adjustable amplification, adjustable latency, normalization, a threshold function, quantization, and / or event-based signaling. This allows the state dynamics to be adapted to different tasks and timescales. In a further advantageous embodiment, not the complete sensor state, but only the reduced class vector is used for feedback. This means that recursion is realized not via a high-dimensional full-screen representation, but via a compact, hardware-defined state representation. This reduces the electronic readout and feedback overhead and simplifies the physical implementation of recursive optoelectronic systems. The class regions can be fixed, calibrated, or adapted to the geometry and sampling symmetry of the modulation device used. In particular, the spatial integration regions of the classes can be adapted to the period length, pixel pitch, propagation distance, and / or the position of the Talbot-synchronous observation plane. In this way, the class projection can be adapted to different wavelengths, period measures, modulator resolutions, and sensor geometries. In a preferred reservoir embodiment, a linear readout unit is provided, which processes the state vector into an output or decision signal by weighted linear combination. The actual training can be limited to determining the readout weights, while the optical propagation, the Talbot-synchronous class projection, and the recursive feedback provide the physical state generation. A further advantageous embodiment is that no additional trained diffractive multilayer structure is required between the modulation unit and the detection unit. In this case, state formation is essentially determined by linear free-space propagation, Talbot-synchronous class projection, intensity detection, and recursive feedback. This results in a comparatively simple, compact, and parameterizable optoelectronic architecture. In another embodiment, the modulation device, the propagation section, and the detection device are arranged in a flat, planar, layered, and / or stacked configuration. In particular, the device can be designed as a common carrier, layered, laminated, bonded, or composite structure, in which the optical distance between the modulation device and the detection device is defined by a transparent spacer layer, intermediate layer, encapsulation layer, or spacer elements. In a further embodiment, at least part of the feedback device is also integrated into this composite structure. The invention is not limited to a specific wavelength, periodic shape, or sensor technology. The periodic structure can be one-dimensional or two-dimensional, strip-shaped, raster-shaped, grid-shaped, or otherwise periodic. Likewise, the device can be designed to be monochromatic or for multiple wavelength ranges, provided that the Talbot-synchronous or near-Talbot class projection is maintained. The inventive combination of periodic modulation structure, Talbot-synchronous class projection, reduced state formation, and optional recursive feedback provides an optoelectronic device that reduces electronic readout complexity compared to full pixel-wise sensor evaluation while simultaneously enabling reproducible, parameterizable state dynamics. The device is thus suitable both as a photonic processing system and as an experimental platform for recursive coherent photonics and physical reservoir computing. QUOTES INCLUDED IN THE DESCRIPTION This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Zitierte Nicht-Patentliteratur Fu et al., Light: Science & Applications, 2024, „Optical neural networks: progress and challenges

[0004] Wang et al., Nature Photonics, 2023, „Image sensing with multilayer nonlinear optical neural networks

[0005] Shi et al., Light: Science & Applications, 2022, „LOEN: Lensless opto-electronic neural network empowered machine vision

[0006] Yan et al., Nature Communications, 2024, „Emerging opportunities and challenges for the future of reservoir computing

[0007] Fu et al., 2024; Xia et al., Nature Photonics, 2024, „Nonlinear optical encoding enabled by recurrent linear scattering

[0008] Shi et al., 2022; Wang et al., 2023

[0009]

Claims

Optoelectronic device comprising a coherent light source, a modulation device with a periodic modulation or sampling structure and a plurality of individually controllable modulation areas for amplitude and / or phase modulation of a light field, a propagation section between the modulation device and an observation plane, a detection device arranged in the observation plane with a plurality of sampling points, and an evaluation device, characterized in that the observation plane is arranged at a Talbot-synchronous distance or in its vicinity with respect to the periodic modulation or sampling structure, and the evaluation device forms respective class channels from detection values ​​of such sampling points that occupy the same relative position within different periods, and generates a state or activation vector from the class channels that is reduced compared to the number of sampling points. Device according to claim 1, characterized in that the periodic modulation or scanning structure is formed by a pixel grid of the modulation device itself and / or by an additional periodic optical structure, in particular a grating, an aperture, a slit structure, an aperture array or a diffractive mask. Device according to claim 1 or 2, characterized in that the Talbot-synchronous distance corresponds to an integer and / or fractional multiple of a Talbot distance or lies within a calibrated tolerance range around such a distance. Device according to one of the preceding claims, characterized in that the class channels are formed by summation, averaging, weighted summation, area integration and / or normalized summation of the associated detection values. Device according to one of the preceding claims, characterized in that the class formation is carried out one-dimensionally by assignment to the same grid phases and / or two-dimensionally by assignment to the same relative positions within two periodic directions. Device according to one of the preceding claims, characterized in that the class formation is carried out wholly or partially on the sensor side by electrically grouped pixel groups, common readout lines or grouped photodetectors. Device according to one of the preceding claims, characterized in that the detection device is designed as a CMOS sensor, CCD sensor, SPAD array, event-based sensor or dense lensless area sensor. Device according to one of the preceding claims, characterized in that the modulation device is configured as a microdisplay, spatial light modulator, LCOS device, DMD, reflective or transmissive pixel matrix, phase mask or as a combination of amplitude-, phase- and / or polarization-modulating device. Device according to one of the preceding claims, characterized in that the spatial class areas are adapted to the period length, the pixel division, the propagation distance, the wavelength and / or the position of the Talbot-synchronous observation plane. Optoelectronic reservoir system with a coherent light source, a modulation device, a propagation section, a detection device, an evaluation device, a control or computing unit and a feedback device, characterized in that the modulation device has a periodic modulation or sampling structure, the detection device is arranged at a Talbot-synchronous distance or in its vicinity, the evaluation device forms a reduced state vector from detection values ​​of the same relative position within different periods and the feedback device supplies this state vector at least partially to the control or computing unit, which generates an updated modulation distribution for a subsequent time period and outputs it to the modulation device, so that a recursive state formation takes place. Reservoir system according to claim 10, characterized in that the control or computing unit generates the updated modulation distribution from the reduced state vector and / or from an external input signal. Reservoir system according to claim 10 or 11, characterized in that the updated modulation distribution comprises a phase distribution, in particular a programmable phase boundary or a boundary-defined modulation structure, and / or an amplitude distribution. Reservoir system according to one of claims 10 to 12, characterized in that only a sub-area of ​​the modulation distribution is updated, while other areas of the modulation device remain unchanged. Reservoir system according to one of claims 10 to 13, characterized in that a first modulation stage is provided for amplitude-based coding of an external input signal and a second modulation stage is provided for phase-based or limit-based coding of the feedback state information. Reservoir system according to one of claims 10 to 14, characterized in that the feedback device has amplification, latency, normalization, a threshold function, quantization and / or event-based signaling. Reservoir system according to one of claims 10 to 15, characterized in that the feedback device is fed exclusively with the reduced state vector and not with a complete pixel-wise sensor readout. Reservoir system according to one of claims 10 to 16, characterized in that the detection device generates a threshold-based and / or event-based output and that this output is used for feedback. Reservoir system according to one of claims 10 to 17, characterized in that the detection device comprises a CMOS sensor with charge-integrating pixel structure in which photogenerated charge is collected over an integration interval and, upon reaching a predetermined threshold, is converted into a binary, multi-stage and / or time-coded event signal, wherein the event signal serves as a spiking-based state representation for feedback. Reservoir system according to one of claims 10 to 18, characterized in that a linear readout unit is provided which processes the reduced state vector by weighted linear combination to produce an output or decision signal. Device according to one of claims 1 to 9, characterized in that the modulation device, the propagation section and the detection device are arranged in a flat, planar, layered and / or stacked design in a common carrier, layered, laminated, bonding and / or composite structure. Device according to claim 20, characterized in that the propagation section is formed at least partially by a transparent spacer layer, intermediate layer, encapsulation layer and / or a defined spacer element, the thickness of which determines the distance between the modulation device and the detection device. Reservoir system according to one of claims 10 to 19, characterized in that the modulation device, the detection device and at least a part of the feedback device are arranged in a flat, planar, layered and / or stacked design as a compact carrier, layered, laminated, bonding and / or composite structure. Device or reservoir system according to claim 21 or 22, characterized in that a thickness of a spacer layer is selected or calibrated such that the detection device is arranged at a Talbot-synchronous distance, at a broken Talbot distance and / or in a tolerance range around such a distance.