Training method for an all-optical generative chip

Abstract

The application provides a training method of an all-optical generative chip, comprising: constructing a simulation module; inputting input data into the simulation module to obtain simulation output data through forward propagation; in the process of forward propagation, quantizing continuous optical weights in the simulation module into preset discrete levels, and participating in light field simulation with the quantized weights of the discrete levels; introducing alignment error disturbance into the simulation module; constructing an unsupervised loss function; according to the loss value, performing gradient approximation transmission on the quantization operation in the back propagation through a straight-through estimator, updating the continuous optical weights; repeating the forward and back propagation until the training converges, quantizing the converged continuous optical weights to obtain quantized weights for chip manufacturing. The optical weights obtained by the application can be directly adapted to the actual manufacturing process and maintain stable performance in the presence of assembly deviation, thereby improving the practicality and robustness of the all-optical generative chip under real manufacturing and packaging conditions.

Description

Technical Field

[0001] This application relates to the fields of optoelectronic computing and machine learning technology, and more specifically, to a training method for all-optical generative chips. Background Technology

[0002] Generative models are a class of machine learning models that learn the distribution and patterns of training data and generate entirely new content, such as images, text, or audio, that is similar to the original data. Unlike discriminative models that focus on classification and prediction, the core goal of generative models is "creation"—generating new samples by modeling the inherent structure of the data. Typical examples include variational autoencoders, generative adversarial networks, and diffusion models.

[0003] In recent years, all-optical diffraction deep neural networks have made progress in classification and prediction tasks. For example, handwritten digit recognition or image classification can be achieved through supervised training of multi-level optical phase modulation layers, demonstrating advantages such as low power consumption and high-speed forward inference. However, when attempting to extend such optical networks to generative tasks, existing training methods face fundamental limitations: generative models typically rely on large amounts of unlabeled data, requiring unsupervised learning paradigms and loss functions that accurately characterize the latent distribution of the data, which traditional supervised training strategies are difficult to adapt to.

[0004] More importantly, even if an optical architecture suitable for generation tasks is designed, it still faces the following two obstacles when it comes to practical chip manufacturing and packaging: (1) Process discretization constraints: Due to manufacturing processes such as photolithography, etching or material polymorphism, optical phase weights can only achieve a limited number of discrete levels. If the conventional process of "continuous training first and then quantitative deployment" is adopted, it will lead to significant performance degradation, requiring repeated iterative parameter tuning, which seriously hinders engineering applications. (2) Packaging alignment error: Geometric deviations (such as translation, rotation, interlayer distance deviation, etc.) are inevitably introduced during the manufacturing and packaging process of multilayer optical diffraction phase modulation structures and fiber arrays. Existing training methods are all based on the ideal alignment assumption, and the obtained weights are highly sensitive to the above errors, resulting in a significant decrease in actual generation quality and high post-calibration and calibration costs.

[0005] Patent CN202410826636.5 discloses a reflective multilayer cascaded diffractive optical neural network system. Although the performance is improved through structural design, its training process does not take into account actual physical limitations such as phase discretization and alignment errors, making it difficult to guarantee the consistency between simulation and measured performance. Summary of the Invention

[0006] In view of the deficiencies or one of the deficiencies in the prior art, the purpose of this application is to provide a training method for an all-optical generative chip.

[0007] A first aspect of this application provides a training method for an all-optical generative chip, comprising: Construct a simulation module to simulate the propagation of light fields in a chip; Input data is input into the simulation module, and simulation output data is obtained through forward propagation. During the forward propagation process, the continuous optical weights in the simulation module are quantized into preset discrete levels, and the quantized weights of the discrete levels participate in the light field simulation. Alignment error perturbation is introduced into the simulation module to simulate geometric deviations in the manufacturing and packaging process. Construct an unsupervised loss function and calculate the loss value based on the input data and the simulation output data; Based on the loss value, the continuous optical weights are updated by performing gradient approximation propagation of the quantization operation in backpropagation through the pass-through estimator. Repeat the forward and backward propagation until training converges. Quantize the converged continuous optical weights to obtain quantized weights for chip manufacturing.

[0008] Optionally, the simulation module includes an encoder simulation module, a fiber array simulation module, and a generator simulation module, wherein: The encoder simulation module includes several cascaded optical diffraction phase modulation layers, used to calculate the encoder output light field corresponding to the input data. The fiber array simulation module includes an array of several single-mode fibers, which is used to downsample the output light field of the encoder to obtain the latent space code. The generator simulation module includes several cascaded optical diffraction phase modulation layers, used to calculate the output data corresponding to the latent space encoding; The optical diffraction phase modulation layer loads optical weights in the form of optical diffraction phases and modulates the phase of the input light field.

[0009] Optionally, quantizing the continuous optical weights in the simulation module into preset discrete levels includes: The continuous phase weights in the optical diffraction phase modulation layer of the simulation module are discretely quantized at level K, where K is a positive integer and K represents the number of discrete phase levels that the chip can actually achieve. The continuous phase weights are mapped to the nearest preset discrete phase level to obtain the quantized phase weights.

[0010] Optionally, the introduction of alignment error perturbation into the simulation module specifically involves: A geometric perturbation is applied to the interlayer propagation process between at least one optical diffraction phase modulation layer in the simulation module and / or the coupling process between the optical diffraction phase modulation layer and the fiber array. The geometric perturbation includes translation error, rotation error, and / or interlayer propagation distance deviation, wherein: The translation error and rotation error are used to characterize the intralayer alignment deviation of the same optical layer in multiple photolithography or etching processes, as well as the interlayer relative offset of different optical layers during the packaging and assembly process. The interlayer propagation distance deviation is used to characterize the deviation between the actual spacing between adjacent optical layers and the design value during the packaging process.

[0011] Optionally, when introducing alignment error perturbation, the geometric parameters of the optical diffraction phase modulation layer and / or fiber array are randomly sampled according to a preset error distribution, wherein the error distribution includes a Gaussian distribution, a uniform distribution, or a combination thereof, and the sampling range is bounded by the ratio of pixel size or the packaging and adjustment accuracy.

[0012] Optionally, updating the continuous optical weights by performing gradient approximation propagation of the quantization operation in backpropagation based on the loss value via a pass-through estimator includes: During backpropagation, a pass-through estimator is used to approximate the gradient of the quantization operation as an identity mapping or a piecewise identity mapping to obtain an approximate gradient. The continuous optical weights are updated using the gradient descent algorithm based on the approximate gradient.

[0013] Optionally, the loss function includes a weighted mean square loss function, a Kullback-Leibler divergence loss function, and a light field penalty function, wherein: The mean square loss function is used to characterize the numerical difference between the current output data and the corresponding sampling points of the input data; The Kullback-Leibler divergence loss function is used to characterize the difference between the current latent space coding distribution and the expected latent space coding prior distribution; the prior distribution includes a skewed Gaussian distribution, or adopts an equivalent distribution constraint form; The optical field penalty function is used to characterize the intensity or energy setting of the output optical field of each optical diffraction phase modulation layer.

[0014] Optionally, during training, for multiple alignment error perturbation samples, the total loss value obtained by weighted summation of the mean square loss function, the Kullback-Leibler divergence loss function, and the light field penalty function is calculated respectively; the average or weighted summation of the multiple total loss values ​​is used as the final optimization target.

[0015] A second aspect of this application provides a training system for an all-optical generative chip, comprising: An all-optical generation chip parameter simulation module is constructed to simulate the propagation of light fields in a chip. The simulation module performs forward propagation on the input data to obtain simulation output data, including: The training quantization module quantizes the continuous optical weights in the simulation module into preset discrete levels, and uses the quantized weights of the discrete levels to participate in the optical field simulation. The alignment error modeling module introduces alignment error perturbation into the simulation module to simulate geometric deviations during the manufacturing and packaging process; The optical generative model loss function module constructs an unsupervised loss function and calculates the loss value based on the input data and the simulation output data. The optical parameter optimization module, based on the loss value, performs gradient approximation propagation of the quantization operation during backpropagation through a pass-through estimator to update the continuous optical weights; repeats forward and backward propagation until training converges, and quantizes the converged continuous optical weights to obtain quantized weights for chip manufacturing.

[0016] A third aspect of this application provides a terminal including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to execute the training method of any of the all-optical generation chips described in the present invention, or to run the training system of the all-optical generation chip.

[0017] This application provides a training method for an all-optical generative chip. By directly using quantized discrete optical weights for light field simulation during the forward propagation of training, and combining this with a pass-through estimator for gradient backpropagation, the model consistently aligns with the discrete phase constraints of the actual manufacturing process during optimization. Simultaneously, alignment error perturbations are introduced into the internal coupling of the simulation module during the forward propagation of training to simulate geometric deviations in the manufacturing and packaging processes. This ensures that the trained optical weights maintain stable performance even with assembly deviations. This avoids the precipitous performance drop caused by "continuous training followed by quantization and deployment," significantly improving the deployability and consistency of the trained weights on real chips, effectively supporting the engineering implementation of all-optical generative chips. The optical weights trained using this method, after being loaded onto an actual chip, are applicable to various visual generation tasks such as image generation, style transfer, and 3D light field reconstruction, while maintaining good performance.

[0018] Other technical effects resulting from the additional features will be further illustrated in the corresponding embodiments. Attached Figure Description

[0019] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating a training method for an all-optical generative chip according to an exemplary embodiment; Figure 2 This is a schematic diagram of the structure of a training system for an all-optical generative chip according to an exemplary embodiment; Figure 3 This is a flowchart illustrating a training method for an all-optical generative chip according to an exemplary embodiment. Detailed Implementation

[0020] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all fall within the protection scope of the present application. Parts not described in detail in the following embodiments can be implemented using existing technology.

[0021] Existing training methods do not consider real manufacturing and packaging conditions, resulting in reduced accuracy of the trained weights in practical applications. To address these issues, this application provides a training method for an all-optical generative chip.

[0022] Reference Figure 1 As shown, a training method for an all-optical generative chip includes the following steps: S100, a simulation module for simulating the propagation of light fields in a chip; S200 inputs the input data into the simulation module and obtains the simulation output data through forward propagation. During the forward propagation process, the continuous optical weights in the simulation module are quantized into preset discrete levels, and the quantized weights of the discrete levels participate in the light field simulation. Alignment error perturbation is introduced into the simulation module to simulate geometric deviations in the manufacturing and packaging process. S300: Construct an unsupervised loss function and calculate the loss value based on the input data and simulation output data; S400 updates the continuous optical weights by approximating the gradient of the quantization operation during backpropagation based on the loss value through the pass-through estimator. The S500 repeats forward and backward propagation until training converges, and then quantizes the converged continuous optical weights to obtain quantized weights for chip manufacturing.

[0023] The above embodiments discretize the continuous optical weights during the forward propagation of training and use discrete-level optical weights to participate in the light field simulation. At the same time, alignment error perturbation is introduced to simulate geometric deviations in the manufacturing and packaging process. Combined with the through-pass estimator, gradient transfer under quantization constraints is realized in the back propagation. This allows the trained optical weights to be directly adapted to the actual manufacturing process and maintain stable performance in the presence of assembly deviations, thereby improving the practicality and robustness of all-optical integrated chips under real manufacturing and packaging conditions.

[0024] In order to better establish the simulation module, in some specific embodiments of this application, the all-optical generation chip parameter simulation module of S100 is constructed based on angular spectrum theory.

[0025] Specifically, the angular spectrum method decomposes the input light field into an angular spectrum representation of a plane wave under the scalar diffraction approximation, and calculates the diffraction propagation of the light field in free space through Fourier transform and propagation transfer function. This allows for high-precision simulation of the post-propagation light field distribution at different propagation distances while meeting sampling conditions.

[0026] The simulation model constructed using the angular spectrum method in the above embodiments is particularly suitable for modeling light propagation behavior under different interlayer spacing conditions; this characteristic is crucial for the subsequent introduction of alignment error perturbations (such as interlayer propagation distance deviation), enabling various geometric deviations to be accurately simulated within a physically consistent framework.

[0027] It should be understood that in other embodiments, other optical field propagation models can also be used to construct the simulation module, such as Fresnel diffraction approximation, Rayleigh-Sommerfeld diffraction integral, or data-driven differentiable optical surrogate models, as long as they can meet the accuracy and differentiability requirements for training.

[0028] In order to make the established simulation model compatible with the generative model, in some specific embodiments of this application, the simulation model established in S100 consists of an encoder simulation module, a fiber array simulation module, and a generator simulation module. The encoder simulation module consists of several optical diffraction phase modulation layers connected in series, and is used to calculate the encoder output light field corresponding to the input image. The fiber array simulation module consists of an array of several single-mode fibers, which is used to downsample the output optical field of the encoder to obtain the latent space code. The generator simulation module consists of several optical diffraction phase modulation layers connected in series, and is used to calculate the output image corresponding to the latent space coding. An optical diffraction phase modulation layer is used to load optical weights in the form of optical diffraction phases and to phase modulate the input light field.

[0029] The simulation module structure of the above embodiment simulates the all-optical generation process, realizes end-to-end unsupervised training, makes the simulation model highly compatible with generative tasks, and ensures the physical realizability of latent space and the quality of generated images.

[0030] To ensure that the training process aligns with the discrete level constraints of the actual manufacturing process, in some specific embodiments of this application, the continuous optical weights in the simulation module are quantized into preset discrete levels in S200, and these quantized weights are used in the optical field simulation. This can be achieved through the following steps: S201.1, Based on the manufacturing process (such as photolithography or etching capability), set the phase quantization level K and define the corresponding discrete phase set.

[0031] Specifically, K represents the number of discrete phase levels that the optical phase modulation unit in the all-optical chip can achieve, and is a positive integer determined by the manufacturing process capability.

[0032] In actual manufacturing, due to limitations in process such as lithography resolution, etching depth control precision, and material refractive index stability, continuously adjustable phase delay cannot be achieved; instead, selection is limited to a finite number of preset phase values. These preset values ​​constitute a discrete phase set, the size of which is K.

[0033] S201.2, for each continuous phase value in the optical diffraction phase modulation layer, map it to the nearest discrete phase level.

[0034] Specifically, the initial continuous optical weights are obtained through random initialization during the initial training phase, for example, by sampling in a uniform distribution within the interval [0, 2π).

[0035] In S201.3, the original weights are replaced with the phase weights mapped and quantized in S201.2. In this forward propagation optical field simulation, all phase modulation operations use mapped and quantized weights instead of the original continuous values.

[0036] For example, suppose the continuous phase weight at a certain pixel position is... θ =1.8 radians (approximately 103 degrees), while the manufacturing process only supports 4-level phase: .

[0037] Calculate distance: , The rest are even further away.

[0038] Therefore, the nearest neighbor is determined to be... ; Therefore, the phase at this position is quantized as And use this value in light field simulation.

[0039] The above embodiments quantize the continuous phase weights into discrete levels supported by the process in real time during the forward propagation of training, and use the quantized values ​​to participate in the light field simulation. This allows the model to perceive manufacturing constraints during the training phase, avoiding the performance abrupt changes caused by the traditional method of continuous training followed by quantization deployment. At the same time, since the quantization operation and light field propagation are jointly modeled, the optimized continuous weights can still maintain the structural integrity of the generated image and the rationality of the light field energy distribution after discretization. This ensures that the weights obtained from training can be directly used for chip manufacturing, significantly improving the first-pass fabrication success rate and actual generation accuracy of all-optical generation chips.

[0040] In order to explicitly model assembly errors during training, in some specific embodiments of this application, the following operation is performed on the simulation module during the forward propagation of training in S200: an alignment error perturbation is introduced into the simulation module to simulate geometric deviations in the manufacturing and packaging process.

[0041] The above embodiments effectively improve the robustness of the model to manufacturing and packaging deviations by explicitly introducing alignment error perturbations during training, making the obtained optical weights more stable and generalizable in actual hardware deployment.

[0042] In one specific embodiment, the alignment error disturbance includes both intra-layer alignment error and inter-layer alignment error.

[0043] Specifically, intra-layer alignment error is used to characterize the overlay deviation of the same optical diffraction phase modulation layer in multiple photolithography or etching processes; inter-layer alignment error is used to characterize the relative positional offset between different optical layers during packaging and assembly.

[0044] Accordingly, the implementation of introducing alignment error perturbation into the simulation module includes: for the interlayer propagation process between at least one optical diffraction phase modulation layer and / or the coupling process between the optical diffraction phase modulation layer and the fiber array, the geometric perturbation includes translation error, rotation error and / or interlayer propagation distance deviation.

[0045] Translation and rotation errors can simulate both intralayer overlay deviations and interlayer relative offsets, while interlayer propagation distance deviations are specifically used to characterize situations where the spacing between adjacent optical layers deviates from the design value.

[0046] The above implementation method introduces intra-layer and inter-layer alignment error perturbations during training, enabling the model to actively learn stable optical field propagation characteristics under various manufacturing and assembly deviations during the optimization process. The obtained optical weights have inherent robustness to unavoidable overlay deviations, inter-layer offsets and spacing fluctuations in actual chips, significantly reducing the risk of performance degradation after packaging, reducing the need for later calibration, and improving the mass production yield and deployment reliability of all-optical generation chips.

[0047] In one specific embodiment, the geometric parameters of the optical diffraction phase modulation layer and / or the fiber array are randomly sampled according to a preset error distribution. These geometric parameters include translational errors, rotational errors, and / or interlayer propagation distance deviations used to characterize non-ideal alignment during manufacturing and packaging. Translational and rotational errors characterize the pose offset of the phase modulation layer relative to its designed position, and / or the relative pose offset between the phase modulation layer and the input surface of the fiber array; interlayer propagation distance deviations characterize the deviation of the actual spacing between adjacent optical layers from the designed value.

[0048] Specifically, for translational errors, rotational errors, and / or interlayer propagation distance deviations, perturbation parameters are randomly sampled from a preset error distribution, which may include Gaussian distribution, uniform distribution, or a combination thereof. In the forward propagation simulation, translational and rotational errors are applied to the phase weight distribution of the phase modulation layer, causing it to undergo corresponding translation and / or rotation relative to the design position; interlayer propagation distance deviations are applied to the propagation distance parameters between adjacent optical layers. The sampling range is bounded by a pixel size ratio (e.g., ±0.5 pixels) or actual packaging and assembly accuracy (e.g., ±2 μm translational deviation, ±0.5° rotational deviation).

[0049] The above implementation provides a specific method for applying alignment error perturbation, and combined with the physical constraints of manufacturing and assembly, ensures that the introduced error conforms to the actual process conditions, thereby improving the robustness and deployability of the model in a real hardware environment.

[0050] In order to adapt to the generative model's need to learn the potential distribution of data, in some specific embodiments of this application, S300, an unsupervised loss function is constructed, and the loss value is calculated based on the input data and simulation output data. This can be achieved by the following steps: S301, Construct the mean squared loss function This is used to characterize the difference between the values ​​at the corresponding sampling points of the current output data and the input data; S302, Construct the Kullback-Leibler divergence loss function This is used to characterize the difference between the current latent space coding distribution and the expected latent space coding prior distribution. Furthermore, considering that the light intensity is non-negative, the prior distribution adopts a biased Gaussian distribution, or adopts an equivalent distribution constraint form to construct an improved Kullback-Leibler divergence loss function, so as to prevent the interlayer propagation light field from being unfavorable to the actual physical realization.

[0051] S303, Establish the light field penalty function It is used to characterize whether the current light field is ill-conditioned or too dim, so as to meet the requirements of actual physical realization and signal-to-noise ratio, and to prevent the light field propagating between layers from being detrimental to actual physical realization.

[0052] S304, weight the above loss functions ( , , Sum the results to construct the total loss function:

[0053] The loss function in the above embodiments integrates reconstruction error, latent space distribution constraints, and the physical feasibility of the optical field, enabling the unsupervised training process to simultaneously satisfy generation fidelity, encoding rationality, and optical realizability. This weighted synergistic optimization avoids training optical fields that are theoretically feasible but practically unobservable or unmanufacturable, thereby improving the generation quality and stability of the all-optical generation chip on real hardware.

[0054] To improve alignment robustness, in some specific embodiments of this application, the total loss function corresponding to multiple alignment error perturbation samples is calculated during the training process, and the expected estimate or weighted summation of the multiple loss values ​​is performed as the final optimization target.

[0055] The above embodiments significantly improve the robustness of the model to alignment errors in manufacturing and packaging through multi-perturbation joint optimization, enabling the obtained optical weights to stably generate high-quality light fields even under actual non-ideal assembly conditions.

[0056] Backpropagation using the loss value typically employs gradient descent, but quantization operations, due to their non-differentiable discreteness, cannot directly propagate gradients. To achieve effective training under quantization constraints, this application introduces a straight-through estimator in backpropagation to approximate the gradient of the quantization process. In some specific embodiments of this application, S400, based on the loss value, the continuous optical weights are updated by approximating the gradient of the quantization operation during backpropagation using the straight-through estimator, which can be achieved through the following steps: S401, obtain the total loss value calculated by S300; S402, during the forward propagation process, the continuous phase weights are quantized to obtain the quantized weights: Let , where Q(·) is the quantization operator; Indicates quantization weights, Indicates continuous weights; During backpropagation, a straight-through estimator is applied to the quantization operator in S502: the gradient of the quantization operator is approximated as an identity mapping, i.e., let...

[0057] Therefore, according to the chain rule, ; Alternatively, during backpropagation, the quantization operator in S502 applies a pass-through estimator: approximating the gradient of the quantization operator as a piecewise identity mapping, i.e., letting: ,in, This represents the preset effective phase range. This piecewise identity mapping indicates that gradients are only propagated within a reasonable physical or quantization range of the input processing, and gradients are blocked in other cases, thereby improving training robustness and convergence.

[0058] S403, using the above approximate gradient, update the continuous optical weights through a gradient descent optimizer (such as Adam); S404, repeat the S200-S400 iterative training until the loss converges.

[0059] Then, the S500 is implemented. After training, the final continuous optical weights are quantized once to derive optical weights that meet the preset discrete level, which are then used in actual chip manufacturing.

[0060] In the above embodiments, the pass-through estimator maintains the use of discrete weights in forward propagation to match process constraints while enabling the backpropagation gradient to be effectively propagated back to continuous parameters, thereby achieving end-to-end quantization-aware training and avoiding post-quantization performance loss.

[0061] Based on the same technical concept, other embodiments of this application provide an unsupervised training system for an all-optical generative chip, such as... Figure 2 As shown, it includes: All-optical generation chip parameter simulation module 100, which is used to simulate the propagation of light fields in the chip; Specifically, the simulation module also includes an encoder simulation module 110, a fiber array simulation module 120, and a generator simulation module 130.

[0062] Simulation module 100 performs forward propagation on the input data to obtain simulation output data, including: In training, the quantization module 140 quantizes the continuous optical weights in the simulation module into preset discrete levels, and uses the quantized weights of the discrete levels to participate in the optical field simulation. Alignment error modeling module 150 introduces alignment error perturbation into the simulation module to simulate geometric deviations in the manufacturing and packaging process; The optical generative model loss function module 200 constructs an unsupervised loss function and calculates the loss value based on the input data and simulation output data. The optical parameter optimization module 300 updates the continuous optical weights by performing gradient approximation propagation of the quantization operation through the pass-through estimator during backpropagation based on the loss value. The forward and backward propagation are repeated until the training converges, and the converged continuous optical weights are quantized to obtain the quantized weights used for chip manufacturing.

[0063] The specific implementation techniques of each module / unit in the above examples of this application can be referred to the steps of the training method of the all-optical generative chip in the above embodiments, and will not be repeated here.

[0064] The preferred features in the above embodiments can be used individually in any embodiment, or in any combination thereof, provided they do not conflict with each other. Furthermore, parts not described in detail in the embodiments can be implemented using existing technologies.

[0065] The following examples and comparative examples will be used to further illustrate this application in order to better understand the above-mentioned technical solutions. It should be understood that the following are only some examples and are not intended to limit this application.

[0066] Application Example 1: like Figure 3 As shown, the training process for the all-optical image generation chip (used for unconditional image generation or input-guided visual content synthesis) is as follows: S1, Constructing an all-optical generative chip parameter simulation module: An end-to-end optical field propagation model is built based on angular spectrum theory, including an encoder composed of multi-level diffraction phase layers (compressing the input optical field into an implicit representation), a single-mode fiber array (achieving spatial downsampling and forming implicit space coding), and a generator composed of multi-level diffraction phase layers (reconstructing the implicit coding into an output image). The entire simulation module accurately simulates the diffraction behavior of light in free space and micro / nano structures.

[0067] S2, Construct and select a suitable training dataset: We employ a large-scale unlabeled natural image dataset (such as the CelebA face dataset or a subset of ImageNet), dividing it into training, validation, and test sets. All images are input as coherent light intensity distributions, requiring no class labels, thus adapting to unsupervised generation tasks.

[0068] S3, Constructing the loss function module and optical parameter optimization module: The design incorporates a composite loss function, including mean squared reconstruction loss (ensuring pixel-level fidelity), improved KL divergence (constraining latent space encoding to follow a biased Gaussian prior, consistent with the physical characteristics of light intensity), and a light field energy penalty term (preventing excessive signal attenuation during propagation); the Adam optimizer is also configured for gradient updates.

[0069] S4 introduces quantization and alignment error modeling during training and performs unsupervised training optimization: In each forward propagation, the continuous phase weights are quantized in real time into four discrete values ​​supported by the process, and the optical field simulation is performed using these discrete weights. Simultaneously, translation, rotation, and interlayer spacing disturbances are randomly injected into the coupling interface of the encoder, fiber array, or generator to simulate photolithography overlay errors and packaging assembly deviations. By employing a straight-through estimator (STE) to bypass quantization during backpropagation and propagate gradients, continuous weights are iteratively optimized, enabling the model to jointly minimize the total loss under various error scenarios.

[0070] S5 obtains the final weight data for actual hardware manufacturing: After training convergence, the quantized discrete phase weights are derived and mapped to the etching depth or geometry of each diffraction unit on the chip, which can then be directly used for photolithography mask design and wafer fabrication.

[0071] The chip takes a two-dimensional image as input, including a random noise map, the image of the content to be edited, or a style reference image, which is loaded onto the input plane in the form of intensity and / or phase distribution. The output is a generated or reconstructed two-dimensional image, obtained directly from the intensity distribution of the output plane, or reconstructed into a visible image after obtaining the complex amplitude distribution through interferometry under coherent detection conditions. The entire forward inference process relies on only a single light propagation, without the need for electronic computation or post-calibration, thus achieving high-quality image generation or visual content synthesis.

[0072] The above embodiments discretize the continuous optical weights during the forward propagation of training and use discrete-level optical weights to participate in the light field simulation. At the same time, alignment error perturbation is introduced to simulate geometric deviations in the manufacturing and packaging process. Combined with the through-pass estimator, gradient transfer under quantization constraints is realized in the back propagation, so that the trained optical weights can be directly adapted to the actual manufacturing process and maintain stable performance in the presence of assembly deviations.

[0073] Crucially, if only quantization is introduced without considering errors, the model may perform well under ideal alignment, but its performance will drop sharply under actual assembly deviations. If only error perturbations are introduced without quantization, the model will have some robustness, but its performance will still suffer a precipitous drop during deployment due to weight discretization. Quantization constraints and alignment error perturbations are simultaneously embedded in the training process, forcing the model to jointly optimize under the stringent conditions of discrete weights and various geometric deviations, thereby learning weights that satisfy both process discreteness and good robustness to assembly deviations.

[0074] Based on the same technical concept, other embodiments of this application also provide a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to execute the training method of the all-optical generation chip described in any one of the claims, or to run the training system of the all-optical generation chip.

[0075] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs and functional modules that implement the above methods), computer instructions, etc., and the aforementioned computer programs and computer instructions can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.

[0076] The aforementioned computer programs, computer instructions, etc., can be stored in partitions within one or more memory locations. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by a processor.

[0077] A processor is used to execute a computer program stored in memory to implement the various steps of the methods involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0078] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0079] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0080] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0081] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0082] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0083] The foregoing has described some specific embodiments of this application. It should be understood that this application is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the substantive content of this application. The above-described preferred features can be used in any combination without conflict.

Claims

1. A training method for an all-optical generative chip, characterized in that, include: Construct a simulation module to simulate the propagation of light fields in a chip; The input data is fed into the simulation module, and the simulation output data is obtained through forward propagation. During the forward propagation process, the continuous optical weights in the simulation module are quantized into preset discrete levels, and the quantized weights of the discrete levels participate in the optical field simulation. Alignment error perturbation is introduced into the simulation module to simulate geometric deviations during manufacturing and packaging. Construct an unsupervised loss function and calculate the loss value based on the input data and the simulation output data; Based on the loss value, the continuous optical weights are updated by performing gradient approximation propagation of the quantization operation in backpropagation through the pass-through estimator. Repeat the forward and backward propagation until training converges. Quantize the converged continuous optical weights to obtain quantized weights for chip manufacturing.

2. The training method for the all-optical generative chip according to claim 1, characterized in that, The simulation module includes an encoder simulation module, a fiber array simulation module, and a generator simulation module, wherein: The encoder simulation module includes several cascaded optical diffraction phase modulation layers, used to calculate the encoder output light field corresponding to the input data. The fiber array simulation module includes an array of several single-mode fibers, which is used to downsample the output light field of the encoder to obtain the latent space code. The generator simulation module includes several cascaded optical diffraction phase modulation layers, used to calculate the output data corresponding to the latent space encoding; The optical diffraction phase modulation layer loads optical weights in the form of optical diffraction phases and modulates the phase of the input light field.

3. The training method for the all-optical generative chip according to claim 2, characterized in that, The step of quantizing the continuous optical weights in the simulation module into preset discrete levels includes: The continuous phase weights in the optical diffraction phase modulation layer of the simulation module are discretely quantized at level K, where K is a positive integer and K represents the number of discrete phase levels that the chip can actually achieve. The continuous phase weights are mapped to the nearest preset discrete phase level to obtain the quantized phase weights.

4. The training method for the all-optical generative chip according to claim 1, characterized in that, The specific steps for introducing alignment error perturbation into the simulation module are as follows: A geometric perturbation is applied to the interlayer propagation process between at least one optical diffraction phase modulation layer in the simulation module and / or the coupling process between the optical diffraction phase modulation layer and the fiber array. The geometric perturbation includes translation error, rotation error, and / or interlayer propagation distance deviation, wherein: The translation error and rotation error are used to characterize the intralayer alignment deviation of the same optical layer in multiple photolithography or etching processes, as well as the interlayer relative offset of different optical layers during the packaging and assembly process. The interlayer propagation distance deviation is used to characterize the deviation between the actual spacing between adjacent optical layers and the design value during the packaging process.

5. The training method for the all-optical generative chip according to claim 4, characterized in that, When an alignment error disturbance is introduced, the geometric parameters of the optical diffraction phase modulation layer and / or fiber array are randomly sampled according to a preset error distribution, wherein the error distribution includes a Gaussian distribution, a uniform distribution, or a combination thereof, and the sampling range is bounded by the pixel size ratio or the packaging and assembly accuracy.

6. The training method for the all-optical generative chip according to claim 1, characterized in that, The step of updating the continuous optical weights by performing gradient approximation propagation of the quantization operation through a pass-through estimator during backpropagation based on the loss value includes: During backpropagation, a pass-through estimator is used to approximate the gradient of the quantization operation as an identity mapping or a piecewise identity mapping to obtain an approximate gradient. The continuous optical weights are updated using the gradient descent algorithm based on the approximate gradient.

7. The training method for the all-optical generative chip according to claim 2, characterized in that, The loss function includes a weighted mean square loss function, a Kullback-Leibler divergence loss function, and a light field penalty function, wherein: The mean square loss function is used to characterize the numerical difference between the current output data and the corresponding sampling points of the input data; The Kullback-Leibler divergence loss function is used to characterize the difference between the current latent space coding distribution and the expected latent space coding prior distribution; the prior distribution includes a skewed Gaussian distribution, or adopts an equivalent distribution constraint form; The optical field penalty function is used to characterize the intensity or energy setting of the output optical field of each optical diffraction phase modulation layer.

8. The training method for the all-optical generative chip according to claim 7, characterized in that, During training, for multiple alignment error perturbation samples, the total loss value obtained by weighted summation of the mean square loss function, the Kullback-Leibler divergence loss function, and the light field penalty function is calculated respectively; the average or weighted summation of the multiple total loss values ​​is used as the final optimization objective.

9. A training system for an all-optical generative chip, characterized in that, include: An all-optical generation chip parameter simulation module is constructed to simulate the propagation of light fields in a chip. The simulation module performs forward propagation on the input data to obtain simulation output data, including: The training quantization module quantizes the continuous optical weights in the simulation module into preset discrete levels, and uses the quantized weights of the discrete levels to participate in the optical field simulation. The alignment error modeling module introduces alignment error perturbation into the simulation module to simulate geometric deviations during the manufacturing and packaging process; The optical generative model loss function module constructs an unsupervised loss function and calculates the loss value based on the input data and the simulation output data. The optical parameter optimization module, based on the loss value, performs gradient approximation propagation of the quantization operation during backpropagation through a pass-through estimator to update the continuous optical weights; repeats forward and backward propagation until training converges, and quantizes the converged continuous optical weights to obtain quantized weights for chip manufacturing.

10. A terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it can be used to perform the method of any one of claims 1-8, or to run the system of claim 9.

Images