Reconfigurable generation type all-optical chip

Abstract

The invention provides a reconfigurable generative all-optical chip, which comprises an optical encoder module, an optical hidden space module and an optical encoder module, and is characterized in that the optical encoder module is used for converting a high-dimensional image light field into a low-dimensional feature light field; the optical hidden space module is used for receiving the low-dimensional feature light field and performing optical processing on the low-dimensional feature light field in an optical hidden space to obtain optical hidden space features; the optical implicit space generator module is used for generating and outputting visual semantic information according to the optical implicit space features; the optical encoder module and the optical encoder module both adopt pluggable assembly structures and can be connected with the optical hidden space module in a pluggable manner and keep an optical path alignment relationship, and flexible switching of different semantic vision generation tasks is realized by replacing different optical encoder modules and optical encoder modules. According to the invention, the optical encoder module and the optical encoder module can be replaced in a pluggable manner to realize flexible task switching, so that multi-task reasoning and rapid upgrading are realized under the condition that the same optical hidden space is shared.

Description

Technical Field

[0001] This application relates to the field of optical computing and generative artificial intelligence chip technology, and more specifically, to a reconfigurable generative all-optical chip. Background Technology

[0002] Generative models (also known as synthetic models) learn the probability distribution of training data and, by sampling from the low-dimensional latent space corresponding to this distribution and mapping it back to the data domain, synthesize entirely new content such as images, text, or audio that are similar to the original data. The latent space serves as an abstract representation of the semantic features of high-dimensional observation data and is the core of generative models for semantic manipulation and diversified generation.

[0003] Thanks to the high speed of light parallel processing and high energy efficiency, photonic computing has made significant progress in machine learning discrimination tasks. However, generative vision systems in practical applications often need to support multiple tasks and rapid switching on the same hardware platform, and require upgrades and maintenance as the task evolves. Existing solutions typically embed the entire chip into a single structure, requiring the replacement of the entire device or relying on external switching when the task changes, which incurs additional costs and cannot cover the needs of input-side adaptation and local maintenance.

[0004] A literature search of existing technologies revealed that Chen et al. (“All-optical synthesis chip for large-scale intelligent semantic visiongeneration”) disclosed an all-optical generative LightGen, which consists of a photonic encoder, an optical latent space (OLS), and a photonic generator package. This LightGen allows the photonic generator to be replaced while keeping the photonic encoder fixed. However, neither the photonic encoder nor the photonic generator has a replaceable layer-level structure to achieve different generative tasks. Therefore, the internal structure of the photonic encoder and the photonic generator is fixed, and the internal style of a single task can only be changed by replacing the photonic generator. It is not possible to partially reconstruct and achieve inference for multiple different tasks.

[0005] Therefore, there is an urgent need for a generative all-optical chip that can maintain the advantages of all-optical inference while supporting pluggable encoders, pluggable generators, and pluggable internal layers of the encoder and generator, so as to achieve multi-task reconfiguration and maintainable upgrades. Summary of the Invention

[0006] In view of the shortcomings of the prior art, the purpose of this application is to provide a reconfigurable generative all-optical chip.

[0007] According to one aspect of this application, a reconfigurable generative all-optical chip is provided, comprising: An optical encoder module is used to convert high-dimensional image light fields into low-dimensional feature light fields; An optical latent space module is used to receive the low-dimensional feature light field and perform optical processing on it in the optical latent space to obtain optical latent space features. The optical generator module is used to generate and output visual semantic information based on the optical latent space features; Both the optical encoder module and the optical generator module adopt a pluggable assembly structure, which can be plugged and plugged into the optical hidden space module respectively and keep the optical path aligned. By replacing different optical encoder modules and optical generator modules, flexible switching of different semantic vision generation tasks can be achieved.

[0008] Optionally, both the optical encoder module and the optical generator module are provided with a layer-level fixing structure, and at least one of the optical encoder module and the optical generator module is provided with a layer-level pluggable structure. The layer-level fixing structure includes at least one layer-level fixing mounting position, and the layer-level pluggable structure includes at least one layer-level pluggable position. Each layer-level fixing mounting position and each layer-level pluggable position is used to install one diffraction layer, and each diffraction layer corresponds to one diffraction neural network layer.

[0009] Optionally, the multiple diffraction layers in the optical encoder module are arranged sequentially along the light propagation direction. The input high-dimensional image light field is sequentially phase-modulated and / or amplitude-modulated by each diffraction layer, and the light field propagates and diffracts between adjacent diffraction layers. The outgoing light field of the previous diffraction layer forms a complex amplitude distribution of superimposed interference at the next diffraction layer after diffraction propagation, thereby realizing the feature extraction of the high-dimensional image light field and obtaining the low-dimensional feature light field.

[0010] Optionally, the optical latent space module includes a single-mode fiber array, which is a two-dimensional matrix array. Each single-mode fiber in the two-dimensional matrix array corresponds to a single-mode fiber channel. The single-mode fiber array is used to couple the low-dimensional characteristic light field into multiple single-mode fiber channels to achieve downsampling. The optical latent space characteristics are characterized by the complex amplitude of the output light field of each single-mode fiber channel. The complex amplitude includes amplitude and phase information, and the coupled field formed by the single-mode fiber array provides random fluctuations for generative inference.

[0011] Optionally, the multiple diffraction layers in the optical generator module are arranged sequentially along the light propagation direction. The light field carrying the characteristics of the optical hidden space from the optical hidden space is sequentially phase-modulated and / or amplitude-modulated by each diffraction layer, and the light field propagates and diffracts between adjacent diffraction layers. The outgoing light field of the previous diffraction layer forms a complex amplitude distribution of superimposed interference at the next diffraction layer after diffraction propagation, thereby generating a light field containing semantic output results.

[0012] Optionally, the diffraction layer is a phase-type diffraction optical element, and its phase modulation structure is a surface relief phase structure or a metasurface phase structure.

[0013] Optionally, the diffraction layer is made of a phase modulation material, which is selected from one or more of dielectric materials, semiconductor materials, polymer materials, metallic materials, phase change materials, and electro-optic materials.

[0014] Optionally, the diffraction layer is a shared layer or a task layer. In the optical encoder module and the optical generator module, the shared layer is configured with at least one layer and is installed in the layer-level fixed structure. In the layer-level pluggable structure, the task layer is configured with at least one layer and can be plugged and replaced via the layer-level pluggable structure.

[0015] Optionally, the pluggable assembly structure is configured with a positioning configuration for alignment, so that after the optical encoder module and the optical generator module are plugged in and replaced, the positional alignment error between them and the optical hidden space module is maintained within a preset error range.

[0016] Optionally, the semantic visual generation task includes any one or more of semantic generation, denoising, style transfer, 3D generation, and semantic manipulation.

[0017] This application provides a reconfigurable generative all-optical chip, which consists of an optical encoder module, an optical latent space module, and an optical generator module. Because optical processing is performed within the optical latent space, semantic vision generation can be achieved at the speed of light, maintaining the advantages of all-optical computing in terms of speed and energy consumption. Both the optical encoder module and the optical generator module adopt a pluggable assembly structure, which allows for flexible task switching by plugging and replacing the optical encoder module and the optical generator module. This enables multi-task inference and rapid upgrades while sharing the same optical latent space.

[0018] Other technical effects resulting from the additional features will be further illustrated in the corresponding embodiments. Attached Figure Description

[0019] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a structural block diagram of a reconfigurable generative all-optical chip according to an embodiment of this application; Figure 2 This is a schematic diagram illustrating the plugging and unplugging of the optical encoder module and the optical generator module in one embodiment of this application.

[0020] In the figure: 100, optical encoder module; 200, optical hidden space module; 300, optical generator module; 110, diffraction layer; 100a, first optical encoder module; 100b, second optical encoder module; 300a, first optical generator module; 300b, second optical generator module. Detailed Implementation

[0021] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all fall within the protection scope of the present application. Parts not described in detail in the following embodiments can be implemented using existing technology.

[0022] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations.

[0023] In the description of the embodiments of this application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0024] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature.

[0025] In the description of the embodiments in this application, "multiple" means two or more, unless otherwise explicitly specified. In this application, unless otherwise explicitly specified and limited, the terms "installed," "connected," "linked," "fixed," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0026] The terms "comprising" and "having," and any variations thereof, in the embodiments of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or devices.

[0027] Generative models can learn the probability distribution of training data and synthesize novel images, text, or audio content similar to the original data by sampling from the low-dimensional latent space corresponding to this distribution. However, in practical applications, generative vision systems often need to support multi-task and rapid switching on the same hardware platform and require upgrades and maintenance as tasks evolve. Existing solutions typically solidify the entire chip into a single structure, requiring replacement of the entire device or external switching when tasks change, incurring additional costs and failing to cover the needs of input-side adaptation and local maintenance. To address the limitations of existing all-optical generative systems in multi-task expansion, input-side adaptation, and maintenance and upgrades, embodiments of this application provide a reconfigurable generative all-optical chip.

[0028] Reference Figure 1 As shown, this application provides a reconfigurable generative all-optical chip, including an optical encoder module 100, an optical hidden space module 200, and an optical generator module 300, wherein: The optical encoder module 100 is used to convert the high-dimensional image light field into a low-dimensional feature light field; The optical latent space module 200 is used to receive low-dimensional feature light fields and perform optical processing on them in the optical latent space to obtain optical latent space features. The optical generator module 300 is used to generate and output visual semantic information based on optical latent space features; Both the optical encoder module 100 and the optical generator module 300 adopt a pluggable assembly structure, which can be plugged and plugged into the optical hidden space module 200 respectively and maintain the optical path alignment relationship. By replacing different optical encoder modules 100 and optical generator modules 300, different semantic vision generation tasks can be flexibly switched.

[0029] It should be noted that the optical encoder module 100 and the optical generator module 300 have a pluggable assembly structure, allowing them to be swapped out and maintaining optical path alignment with the optical latent space module 200. This enables multi-task reconfiguration through swapping. The optical encoder module 100 is used for encoding, the optical latent space module 200 is used for latent space mapping, and the optical generator module 300 is used for semantic vision generation. The entire process of encoding, latent space mapping, and generation is completed in the optical domain, without involving analog-to-digital conversion or electrical computing chips.

[0030] The embodiments described above employ an optical encoder module, an optical latent space module, and an optical generator module to construct a generative all-optical chip. Since optical processing is performed within the optical latent space, semantic visual generation can be achieved at the speed of light, maintaining the advantages of all-optical computing in terms of speed and energy consumption. Both the optical encoder module and the optical generator module adopt a pluggable assembly structure, allowing for flexible task switching by plugging and unplugging the optical encoder module and the optical generator module. This enables multi-task inference and rapid upgrades while sharing the same optical latent space.

[0031] In some specific embodiments of this application, both the optical encoder module 100 and the optical generator module 300 are provided with a layer-level fixing structure. At least one of the optical encoder module 100 and the optical generator module 300 is provided with a layer-level pluggable structure. The layer-level fixing structure includes at least one layer-level fixing mounting position, and the layer-level pluggable structure includes at least one layer-level pluggable position. Each layer-level fixing mounting position and each layer-level pluggable position is used to install a diffraction layer, and each diffraction layer corresponds to a diffraction neural network layer.

[0032] Unlike existing all-optical generation systems that integrate the chip into a single structure, the above embodiments of this application design reconfigurability as an inherent structural feature of the chip: the optical encoder module 100 and the optical generator module 300 maintain optical path alignment with the optical latent space module 200 through a plug-in assembly method. This allows for task reconfiguration and switching by plugging and unplugging the optical encoder module 100 and the optical generator module 300 without replacing the optical latent space module 200. The plugging and unplugging of the optical encoder module 100 is used for input-side adaptation and feature extraction strategy switching, while the plugging and unplugging of the optical generator module 300 is used for output task switching or style switching. This enables multi-task inference and rapid upgrades while sharing the same optical latent space.

[0033] Based on the above, in order to further refine the granularity of plug-and-play reconstruction and further improve the flexibility and maintainability of reconstruction, this application further sets up a layer-level pluggable structure inside the optical encoder module 100 and the optical generator module 300, so that at least one diffraction layer can be plugged and replaced, thereby realizing partial network reconstruction and multi-task adaptation without replacing the entire optical encoder module 100 and the optical generator module 300, so as to realize partial reconstruction within the module.

[0034] In some specific embodiments of this application, multiple diffraction layers in the optical encoder module 100 are arranged sequentially along the light propagation direction. The input high-dimensional image light field is sequentially modulated by the phase and / or amplitude of each diffraction layer, and the light field propagates and diffracts between adjacent diffraction layers. The outgoing light field of the previous diffraction layer forms a complex amplitude distribution of superimposed interference at the next diffraction layer after diffraction propagation, thereby realizing the feature extraction of the high-dimensional image light field and obtaining the low-dimensional feature light field.

[0035] In the embodiments described above, the optical encoder module 100 is composed of multiple diffractive neural network layers and is used to extract low-dimensional features from a high-dimensional image light field. The number of diffractive layers, pixel scale, and pixel size of the optical encoder module 100 can be configured according to the target resolution and task complexity to match the accuracy requirements of different generation tasks. Input-side adaptation can be achieved by plugging and unplugging different optical encoder modules 100 to support switching between different input resolutions or different feature extraction targets.

[0036] In some specific embodiments of this application, the optical latent space module 200 includes a single-mode fiber array, which is a two-dimensional matrix array. Each single-mode fiber in the two-dimensional matrix array corresponds to a single-mode fiber channel. The single-mode fiber array is used to couple low-dimensional feature light fields into multiple single-mode fiber channels to achieve downsampling. The optical latent space features are characterized by the complex amplitude of the output light field of each single-mode fiber channel. The complex amplitude includes amplitude and phase information, and the coupling field formed by the single-mode fiber array provides random fluctuations for generative inference.

[0037] In the embodiments described above, the low-dimensional feature light field output by the optical encoder module 100 is coupled by the optical latent space module 200 to form optical latent space features. Specifically, the optical latent space module 200 is composed of a single-mode fiber array, used to downsample the low-dimensional features and simultaneously encode the amplitude and phase of the light. Unsupervised learning is used to extract semantics. The complex coupled field formed by the single-mode fiber array provides crucial random fluctuations for the generative model. The single-mode fiber array is a matrix-arranged single-mode fiber array. The unsupervised training method refers to determining and optimizing the diffraction sheet parameters of the optical encoder module 100 and / or the optical generator module 300 through an unsupervised training algorithm during the training phase. This ensures that the encoded samples form optical latent space features in the optical latent space module 200 that satisfy preset distribution constraints and have semantic aggregation characteristics. After training, during the inference phase, the optical latent space module 200 downsamples the coupled light field and utilizes the complex coupled field to provide the random fluctuations required by the generative model.

[0038] In some specific embodiments of this application, multiple diffraction layers in the optical generator module 300 are arranged sequentially along the light propagation direction. The light field carrying the characteristics of the optical hidden space (i.e., diverse information) from the optical hidden space is sequentially phase-modulated and / or amplitude-modulated by each diffraction layer, and the light field propagates and diffracts between adjacent diffraction layers. After the outgoing light field of the previous diffraction layer is propagated by diffraction, it forms a complex amplitude distribution of superimposed interference at the next diffraction layer, thereby generating a light field containing semantic output results.

[0039] In the embodiments described above, the optical generator module 300 is composed of multiple diffractive neural network layers. It is used to re-embed features in the optical latent space back into the optical field and output semantic generation results. Specifically, it receives optical latent space features, performs optical processing to generate or reconstruct these features, generates a high-dimensional visual semantic light field, and achieves a visual representation of the high-dimensional manifold, outputting corresponding visual semantic information. The optical processing includes performing optical diffraction, amplitude and phase modulation, and optical reconstruction on the features. The optical generator module 300 can be configured as a pluggable generator module corresponding to multiple tasks, allowing different optical generator modules to correspond to different generative tasks or different style domains. In multi-task scenarios, a structure combining a joint optical encoder module and a switchable optical generator module can be used to achieve multi-task reconstruction, allowing different optical generator modules to correspond to different tasks or different styles, thus enabling task expansion and switching under shared latent space conditions.

[0040] In some specific embodiments of this application, the diffraction layer is used as a phase-type diffraction optical element, and its phase modulation structure is a surface relief phase structure or a metasurface phase structure, which corresponds one-to-one with the corresponding diffraction neural network layer. The phase-type diffraction optical element has the advantages of high-precision phase modulation and low-loss optical field control, and can be used as the core element of the diffraction layer to realize precise amplitude and phase modulation of the optical field.

[0041] In the above embodiments of this application, the diffraction layer is made of a phase modulation material, which is selected from one or more of dielectric materials, semiconductor materials, polymer materials, metallic materials, phase change materials, and electro-optic materials.

[0042] In some specific embodiments of this application, the diffraction layer is a shared layer or a task layer. In the optical encoder module and the optical generator module, the shared layer is configured with at least one layer and is installed in the layer-level fixed structure; in the layer-level pluggable structure, the task layer is configured with at least one layer and can be plugged and replaced through the layer-level pluggable structure.

[0043] For example, each layer's fixed mounting position is used to install a shared layer, and each layer's plug-in position is used to install a task layer.

[0044] In the embodiments described above, the layer-level pluggable structure is used to realize a pluggable reconfiguration method that combines shared layers and task layers. The shared layer is at least one diffraction layer that remains unchanged in the optical encoder module 100 or the optical generator module 300, and the task layer is at least one diffraction layer that can be plugged in and replaced in the optical encoder module 100 or the optical generator module 300. Reconfiguration for different tasks or different styles can be achieved by simply plugging in and replacing the task layer.

[0045] Furthermore, through the layer-level pluggable structure, the optical network can be divided into shared layers and task layers. This allows for rapid task reconstruction or style transfer reconstruction by simply replacing the back-end task layers while keeping the front-end shared layers unchanged. In the event of performance drift caused by manufacturing errors, assembly deviations, or long-term use, the network can be restored by replacing the subsequent task layers, reducing the cost and time overhead of replacing the entire module.

[0046] In some specific embodiments of this application, the pluggable assembly structure is configured with a positioning configuration for alignment, so that after the optical encoder module 100 and the optical generator module 300 are plugged in and replaced, the positional alignment error between them and the optical hidden space module 200 is maintained within a preset error range.

[0047] In the above embodiments of this application, the pluggable assembly structure includes a positioning reference or limiting structure, i.e., a positioning mechanism, for repeated alignment, so that the position alignment error between the optical encoder module 100 and the optical generator module 300 and the optical hidden space module 200 remains within a preset error range after plugging and unplugging. The preset error range can be set to ±2 micrometers, but is not limited thereto.

[0048] In some specific embodiments of this application, the semantic visual generation task includes any one or more of semantic generation, denoising, style transfer, 3D generation, and semantic manipulation.

[0049] In the above embodiments of this application, the generative all-optical chip supports a variety of large-scale generative tasks, including any one or more of semantic generation, denoising, style transfer, 3D generation, and semantic manipulation. The generative tasks are implemented by plugging and unplugging the optical encoder module 100, the optical generator module 300, and the task layer.

[0050] This application proposes an all-optical chip based on optical latent space and achieving multi-task generation through pluggable reconstruction. This chip enables feature extraction, dimensionality transformation, latent space sampling, and semantic generation within the optical domain, thereby achieving high-resolution semantic visual generation at the speed of light while maintaining the advantages of all-optical computing in terms of speed and energy consumption. This application can automatically learn data distribution under unsupervised conditions while providing random noise to realize the optical latent space construction scheme.

[0051] This application proposes a hardware reconfiguration method for multi-task generative inference, enabling the same chip platform to switch tasks and upgrade capabilities by plugging in key optical networks without relying on large-scale external electrical reconfiguration or replacing the entire optical path. This adapts to different task requirements such as semantic generation, denoising, style transfer, 3D generation, and semantic manipulation. Furthermore, this pluggable reconfigurable device-level structure allows for the pluggable replacement of both the optical encoder and optical generator modules. Within these modules, layer-level pluggable replacement is achieved, further refining the reconfiguration from complete modules to partial diffraction layers. This enables lower-cost, shorter-cycle multi-task adaptation, maintenance calibration, and iterative upgrades while sharing optical latent space.

[0052] Furthermore, the reconfigurable generative all-optical chip of this application embodiment constructs an efficient, low-power, scalable, and upgradable all-optical generative chip architecture by completing feature extraction, latent space mapping, and semantic generation in the optical domain; and through a hierarchical reconfiguration mechanism with pluggable optical encoder modules, pluggable optical generator modules, and pluggable layers, the same chip platform can achieve rapid switching and continuous iteration for multi-task generative inference, and execute large-scale semantic vision generation tasks at the speed of light without analog-to-digital or digital-to-analog conversion.

[0053] The following examples will be used to further illustrate this application in order to better understand the above-mentioned technical solutions. It should be understood that the following are only some examples and are not intended to limit this application.

[0054] Application Example 1: A Reconfigurable Generative All-Optical Chip like Figure 2 As shown, the optical hidden space module 200 remains in the same assembly position as a shared optical base. Its input side and the output side of the optical encoder module 100 are aligned by a plug-in assembly, and its output side and the input side of the optical generator module 300 are aligned by a plug-in assembly.

[0055] Furthermore, Figure 2 The diagram shows a replaceable first optical encoder module 100a and a second optical encoder module 100b (both belong to the optical encoder module 100). These modules are used to achieve input-side adaptation and feature extraction strategy switching by plugging and unplugging without replacing the optical latent space module 200. When it is necessary to adapt to different input resolutions, different input modes, or different feature extraction targets, the current optical encoder module can be removed from the docking position of the optical latent space module 200 and another optical encoder module can be inserted, so that the optical latent space module 200 can still receive the low-dimensional feature light field that matches its channel array.

[0056] Furthermore, Figure 2 The diagram illustrates replaceable first optical generator module 300a and second optical generator module 300b (both belong to optical generator module 300). These modules are used to switch tasks or styles by plugging and unplugging without replacing the optical latent space module 200. When different generative tasks need to be performed, the current optical generator module can be extracted from the docking position of the optical latent space module 200 and another optical generator module can be inserted, allowing the same optical latent space features to be reconstructed into the semantic output of the corresponding task by different optical generator modules. This structure can be used for rapid switching in multi-task generative reasoning.

[0057] The above application examples of this application realize dimensional transformation and semantic manipulation in the entire optical domain through optical latent space, and realize multi-task reconstruction and maintainable upgrades through the plug-and-play of optical encoder module, optical generator module and layer plug-and-play. It can perform large-scale generative vision tasks at the speed of light without analog-to-digital / digital-to-analog conversion, and has good scalability and upgradeability.

[0058] The preferred features in the above embodiments can be used individually in any embodiment, or in any combination thereof, provided they do not conflict with each other. Furthermore, parts not described in detail in the embodiments can be implemented using existing technologies.

[0059] The foregoing has described some specific embodiments of this application. It should be understood that this application is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the substantive content of this application. The above-described preferred features can be used in any combination without conflict.

Claims

1. A reconfigurable generative all-optical chip, characterized in that, The application relates to a reconfigurable generative all-optical chip. An optical encoder module is used for converting a high-dimensional image light field into a low-dimensional feature light field; An optical latent space module is used for receiving the low-dimensional feature light field and performing optical processing on the low-dimensional feature light field in an optical latent space to obtain an optical latent space feature; An optical generator module is used for generating and outputting visual semantic information according to the optical latent space feature; The optical encoder module and the optical generator module are both provided with a pluggable assembly structure, can be respectively plugged into the optical latent space module and kept in optical path alignment, and different semantic visual generation tasks can be flexibly switched by replacing different optical encoder modules and optical generator modules.

2. The reconfigurable generative all-optical chip according to claim 1, wherein The optical encoder module and the optical generator module are both provided with a wafer-level fixing structure, at least one of the optical encoder module and the optical generator module is internally provided with a wafer-level pluggable structure, the wafer-level fixing structure comprises at least one wafer fixing mounting position, and the wafer-level pluggable structure comprises at least one wafer plugging position; each wafer fixing mounting position and each wafer plugging position are used for mounting a diffraction wafer, and each diffraction wafer corresponds to a diffraction neural network layer.

3. The reconfigurable generative all-optical chip according to claim 2, wherein The plurality of diffraction wafers in the optical encoder module are arranged in sequence along the light propagation direction; the input high-dimensional image light field is subjected to phase and / or amplitude modulation by each diffraction wafer in sequence, and the light field propagates and diffracts between adjacent diffraction wafers; the emergent light field of the previous diffraction wafer forms a superimposed interference complex amplitude distribution at the next diffraction wafer after diffraction propagation, so that the feature extraction of the high-dimensional image light field is realized, and a low-dimensional feature light field is obtained.

4. The reconfigurable generative all-optical chip according to claim 2, wherein The optical latent space module comprises a single-mode fiber array, the single-mode fiber array is a two-dimensional matrix array, each single-mode fiber in the two-dimensional matrix array corresponds to a single-mode fiber channel, the single-mode fiber array is used for coupling the low-dimensional feature light field into a plurality of single-mode fiber channels to realize downsampling, the optical latent space feature is represented by the complex amplitude of the output light field of each single-mode fiber channel, the complex amplitude contains amplitude and phase information, and the coupling field formed by the single-mode fiber array provides random fluctuations for generative inference.

5. The reconfigurable generative all-optical chip according to claim 2, wherein The plurality of diffraction wafers in the optical generator module are arranged in sequence along the light propagation direction; the light field carrying the optical latent space feature from the optical latent space is subjected to phase and / or amplitude modulation by each diffraction wafer in sequence, and the light field propagates and diffracts between adjacent diffraction wafers; the emergent light field of the previous diffraction wafer forms a superimposed interference complex amplitude distribution at the next diffraction wafer after diffraction propagation, so that a light field containing semantic output results is generated.

6. The reconfigurable generative all-optical chip according to claim 2, wherein The diffraction layer sheet is a phase-type diffractive optical element, and the phase modulation structure thereof is a surface relief phase structure or a metasurface phase structure.

7. The reconfigurable generative optical chip of claim 2, wherein, The diffraction layer sheet is made of a phase modulation material selected from one or more of a dielectric material, a semiconductor material, a polymer material, a metal material, a phase change material, and an electro-optic material.

8. The reconfigurable generative optical chip of claim 2, wherein, The diffraction layer sheet is a shared layer sheet or a task layer sheet, the shared layer sheet is configured with at least one layer and installed in the layer-level fixed structure in the optical encoder module and the optical generator module, and the task layer sheet is configured with at least one layer and capable of being plugged in and replaced through the layer-level pluggable structure.

9. The reconfigurable generative optical chip of claim 2, wherein, The pluggable assembly structure is configured with a positioning configuration for alignment, so that the position alignment error between the optical encoder module and the optical generator module and the optical hidden space module is maintained within a preset error range after the pluggable replacement.

10. The reconfigurable generative photonic chip of claim 1, wherein, The semantic visual generation task includes any one or any multiple of semantic generation, denoising, style transfer, three-dimensional generation, and semantic manipulation.

Images