All-optical image generation method based on diffractive neural network and storage medium

Abstract

The application relates to a diffractive neural network-based all-optical image generation method and a storage medium. First, the application performs spatial filtering on the initialized first light field generated by a laser through all-optical sparse coding to form a second light field with a sparse activation mode. Then, the second light field is input into a diffractive neural network generator to transform the spatial distribution characteristics of the second light field layer by layer through physical propagation to obtain a target optical representation corresponding to the second light field. Finally, the output target optical representation is focused on an image sensor through a focusing lens device to obtain a target image. By adopting an all-optical architecture for image generation and introducing an all-optical sparse coding mechanism in the input stage, the image generation can be completed in a pure optical path, the energy consumption and delay problem is reduced, the separability and expression ability of the generator latent space are improved, and thus the speed and efficiency of image generation are improved.

Description

Technical Field

[0001] This application relates to the field of intelligent optical computing technology, specifically to a method for generating all-optical images based on a diffraction neural network and a storage medium. Background Technology

[0002] Image generation, as an important research direction in computer vision, has wide applications in various tasks such as artistic creation, autonomous driving, and image restoration. Currently, mainstream image generation methods mainly rely on digital neural networks, with typical examples including generative adversarial networks, variational autoencoders, and diffusion-based generative models. While these models have achieved significant results in terms of generation quality and expressive power, they suffer from slow inference speed, high energy consumption, and limited system scalability due to their reliance on electronic computing, making it difficult to meet the practical requirements of high speed and low energy consumption.

[0003] In recent years, optical computing has become an important direction for building new intelligent computing systems due to its parallelism, passivity, and the characteristics of light propagation. In the traditional process of optical computing to generate images, digital encoders or latent variable design modules are introduced to improve the generator's ability to represent the latent space. Although this enhances input control capabilities, it also compromises the all-optical nature of the system, leading to increased energy consumption, higher computational latency, and a weakening of the advantages of optical computing in terms of speed and parallelism. Summary of the Invention

[0004] The purpose of this application is to provide a fully optical image generation method and storage medium based on diffraction neural networks, in order to solve the problems of computational delay and high energy consumption when generating images using optical computation.

[0005] To achieve the above objectives, the first aspect of this application provides a method for generating all-optical images based on a diffraction neural network, comprising:

[0006] The first input optical field is initialized. The first optical field is generated by a laser and constructed into a random distribution of amplitude or phase by a programmable optical device.

[0007] The first light field is spatially filtered by all-optical sparse coding to form a second light field with sparse activation mode.

[0008] The second light field is input into the diffraction neural network generator, and the spatial distribution characteristics of the second light field are transformed layer by layer through physical propagation to obtain the target optical representation corresponding to the second light field;

[0009] The output optical representation of the target is focused onto the image sensor by a focusing lens device to obtain the target image.

[0010] A second aspect of this application provides a computer-readable storage medium storing a program that can be loaded by a processor and executed by the above-described all-optical image generation method based on a diffraction neural network.

[0011] The beneficial effects of this application are:

[0012] This application first spatially filters the initial light field generated by the laser through all-optical sparse coding to form a second light field with a sparse activation mode. Then, the second light field is input into a diffraction neural network generator, and its spatial distribution characteristics are transformed layer by layer through physical propagation to obtain the target optical representation corresponding to the second light field. Finally, the output target optical representation is focused onto an image sensor through a focusing lens device to obtain the target image. This application adopts an all-optical architecture, utilizing the diffraction and propagation characteristics of light for image generation. Furthermore, the introduction of an all-optical sparse coding mechanism at the input stage allows image generation to be completed in a purely optical path without the intervention of electronic computing units. This reduces the energy consumption and latency issues in traditional electronic computing processes, improves the separability and expressive power of the generator's latent space, and thus enhances the speed and efficiency of image generation.

[0013] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating a fully optical image generation method based on a diffraction neural network provided in an embodiment of this application.

[0015] Figure 2 This is a schematic diagram of an all-optical sparse coding process provided in the embodiments of this application. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified. In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to implement and use this application. In the following description, details are set forth for illustrative purposes. It should be understood that those skilled in the art will recognize that this application can be implemented without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid unnecessary detail that would obscure the description of this application. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0018] Figure 1 This is a flowchart illustrating a fully optical image generation method based on a diffraction neural network provided in an embodiment of this application. Figure 1 As shown, the method may include steps 101-104, which will be described in detail below.

[0019] Step 101: Initialize the input first optical field. The first optical field is generated by a laser and constructed into a randomly distributed optical representation by loading a set random distribution of amplitude or phase through a programmable optical device.

[0020] The first light field refers to the initial light field in the image generation process, serving as the starting point for image generation. Initializing the first light field is the initial step in the all-optical image generation method, providing the foundation for subsequent image generation.

[0021] In one example, a laser can be used to generate a light field. Lasers provide highly coherent and monochromatic light sources. Programmable optics, such as spatial light modulators (SLMs), are devices that can dynamically control the amplitude and phase of the light wavefront, allowing a randomly distributed pattern to be loaded into the light field. By combining the light field generated by the laser with the random distribution loaded by the programmable optics, a randomly distributed optical representation can be constructed as the starting point for the image generation process.

[0022] Step 102: Spatial filtering of the first light field is performed through all-optical sparse coding to form a second light field with sparse activation mode.

[0023] To address the shortcomings of traditional image generation techniques, such as insufficient latent space representation and reliance on electronic modules for input encoding, this application introduces an all-optical sparse coding mechanism in the input stage. By enhancing the discriminability and orthogonality between spatial variables of the input random light field through optical means, the generator's expressive power can be improved without the need for additional telecommunications modulators, thereby maintaining the all-optical architecture of the system.

[0024] In this embodiment, the second light field refers to the light field obtained by fully optically sparsely encoding the first light field. Fully optically sparse encoding is a process of spatial filtering the first light field. Spatial filtering removes unnecessary information from the first light field, retains key features, and forms a second light field with a sparse activation mode. This sparse activation mode helps improve the efficiency and quality of image generation. Obtaining the second light field through fully optically sparse encoding of the first light field results in a light field with clearer features and less redundancy, providing higher-quality input for subsequent image generation and making the generated image more efficient and accurate.

[0025] Step 103: Input the second light field into the diffraction neural network generator, and transform the spatial distribution characteristics of the second light field layer by layer through physical propagation to obtain the target optical representation corresponding to the second light field.

[0026] In this embodiment, the diffraction neural network generator is a neural network based on the principle of diffraction, capable of simulating the propagation and transformation of a light field in a multi-layered medium. The second light field propagates layer by layer in the diffraction neural network generator, with each layer simulating the diffraction and propagation process of the second light field in a specific medium. Through layer-by-layer transformation, the spatial distribution characteristics of the second light field can be progressively adjusted and optimized to generate a target optical representation with specific features. Here, the target optical representation refers to the light field processed by the diffraction neural network generator, possessing the required spatial distribution characteristics, which can be used to generate the desired target image. The diffraction neural network generator can learn and simulate complex transformations of the light field, thereby generating an optical representation with specific features, providing high-quality optical signals for the final image generation.

[0027] Step 104: Focus the output target optical representation onto the image sensor using a focusing lens device to obtain the target image.

[0028] In this embodiment, the focusing lens device is an optical element capable of focusing a light field onto a specific plane, such as an image sensor. Through the focusing lens device, the optical representation of the target can be focused onto the image sensor to form a clear image. The image sensor is a device capable of converting light signals into electrical signals, used to capture and record images to generate the target image. The target image refers to the image ultimately formed on the image sensor; it is the output of the all-optical image generation method, possessing the required features and quality. Thus, the all-optical image generation process is complete.

[0029] The all-optical image generation method based on diffraction neural networks in this application can generate a high-quality target image by starting from a randomly distributed optical representation and going through processes such as spatial filtering, physical propagation, and focusing. This application employs an all-optical architecture, utilizing the diffraction and propagation characteristics of light for image generation. Furthermore, it introduces an all-optical sparse coding mechanism at the input stage, enabling image generation to be completed in a purely optical path without the intervention of electronic computing units. This reduces the energy consumption and latency issues of traditional electronic computing processes, improves the separability and expressive power of the generator's latent space, and thus enhances the speed and efficiency of image generation.

[0030] Figure 2 This is a schematic diagram illustrating an all-optical sparse coding process provided in an embodiment of this application. For example... Figure 2 As shown, in step 102, the first light field is first spatially expanded using an amplifying optical device with a set magnification, resulting in multiple amplified regions. The spatial expansion of the first light field using the amplifying optical device facilitates subsequent sparsification processing.

[0031] As an example, sampling points can be obtained from the first light field. A sampling point is a point that needs to be processed, representing key information in the first light field. The size of the sampling point is magnified using a magnifying optical device with a set magnification, resulting in multiple magnified regions, each corresponding to one sampling point. The magnification determines the size of the magnified region and can be set according to requirements.

[0032] Then, the target areas of multiple amplified regions are retained through a pre-defined pinhole array to obtain the second optical field. The pinhole array is a precisely designed structure for spatial filtering of the amplified first optical field, allowing only the target area of ​​each amplified region to transmit light. For example, the target area can be the central region of each amplified region, thereby forming a sparse activation mode.

[0033] As an example, firstly, the first spatial coordinates of multiple magnified regions are calculated based on the magnification of the magnifying optical device. Then, based on these first spatial coordinates, the second spatial coordinates of the target region within the magnified regions are determined. Here, the first spatial coordinates refer to the spatial coordinates of the magnified regions, and the second spatial coordinates refer to the spatial coordinates of the target region. Next, the positions of the pinholes are marked in the mask design diagram of the pinhole array, with each pinhole position corresponding to a second spatial coordinate. The pinhole array is then obtained based on the mask design diagram. The pinhole array is used to selectively transmit light to a specific region in the light field, i.e., the target region. Finally, the light from the target region of the multiple magnified regions is transmitted through the pinhole array, resulting in a second light field with a sparse activation mode.

[0034] The all-optical sparse coding process essentially introduces position selectivity and structural sparsity into a two-dimensional light field, resulting in higher independence and orthogonality of inputs at different spatial locations. This helps improve the subsequent generator's ability to represent complex latent spaces and promotes the generation of diverse images. In one example, this sparse coding process can be achieved through an optical combination of a convex lens system and a precision pinhole mask, completed entirely in a pure optical path without the need for electronic control or digital computing units. Sparse coding enhances the statistical negative correlation between input sampling points of the diffractive neural network, thereby increasing spatial orthogonality and providing a clearer input signal for the diffractive neural network generator.

[0035] In this embodiment, the diffraction neural network generator consists of a multi-layer propagation network, with each layer including a spatial light modulation unit and a free-space propagation unit. Specifically, the diffraction neural network generator is composed of multiple alternating spatial light modulation units and free-space propagation units, forming an optical propagation system similar to a layered neural network.

[0036] In step 103, the second light field is first input to the diffraction neural network generator. Then, for each propagation network layer, a two-dimensional phase mask of the second light field is loaded through a spatial light modulation unit to modulate the second light field, thereby precisely controlling its amplitude and phase. Next, the modulated second light field is subjected to interference and diffraction through a free-space propagation unit to obtain the mapping and feature transformation of the second light field corresponding to each propagation network layer. This step simulates the natural propagation behavior of light in free space, including interference and diffraction phenomena. Finally, based on the mapping and feature transformation of the second light field corresponding to each propagation network layer, the spatial distribution characteristics of the second light field are adjusted and optimized layer by layer to obtain the target optical representation of the second light field. The target optical representation is the final output after processing by the diffraction neural network generator and is used to generate the target image. The diffraction neural network generator can adopt various topologies, including linear stacking, parallel, and skip-connection types, and the number of modulation units, propagation distance, and mask size can be flexibly adjusted according to application requirements.

[0037] To significantly reduce the reliance on and cost of pixel-by-pixel paired annotation data during the training of the diffraction neural network generator, this application employs an unsupervised learning strategy based on generative adversarial networks. On one hand, to alleviate the gradient response mismatch problem between the generative neural network generator and the electronic discriminator, the Lipschitz constant of the electronic discriminator is restricted to a finite range during training to stabilize the adversarial training process and improve convergence efficiency. On the other hand, to improve the physical realizability and energy efficiency of the all-optical generation model, this application designs a cooperative loss function that integrates diffraction efficiency constraints, based on the energy transfer characteristics of the optical system. This aims to effectively improve the light energy conversion rate and detection sensitivity in the actual system while maintaining the quality of the generated image.

[0038] Based on this, in this embodiment of the application, the all-optical image generation method further includes a step of training a diffraction neural network generator. First, a training sample set is obtained, which may include multiple first sample light fields. The first sample light fields are the initial sample light fields during the training process, used to train the diffraction neural network generator so that the generator can learn to generate images similar to real images.

[0039] Next, the first sample light field is initialized, and spatial filtering is performed on it using all-optical sparse coding to form a second sample light field with a sparse activation mode. This allows the sample light field to have clearer features and less redundancy, providing high-quality input for subsequent image generation. For example, the first sample light field can be spatially expanded using a magnifying optical device with a set magnification factor to obtain multiple magnified sample regions. Then, using a set pinhole array, the sample target regions of the multiple magnified sample regions are retained to obtain the second sample light field. The specific process can be referred to the sparse activation mode generation method described above.

[0040] Then, the second sample light field is input into the diffraction neural network generator to be trained, resulting in the generated image output by the generator. Through layer-by-layer transformation, the spatial distribution characteristics of the second sample light field can be progressively adjusted and optimized to generate a sample optical representation with specific features, ultimately producing a high-quality generated image. In one example, the second sample light field can be input into the diffraction neural network generator, and its spatial distribution characteristics can be transformed layer by layer through physical propagation to obtain the corresponding sample optical representation. Then, the output sample optical representation is focused onto an image sensor using a focusing lens device to obtain the generated image. In this way, the entire optical sample image generation process can be completed.

[0041] This application employs an unsupervised learning strategy to train a diffraction neural network generator, allowing the generated image to be fed into an electronic discriminator for evaluation and feedback. The electronic discriminator, composed of an electronic neural network, determines the consistency in distribution between the generated and real images and constructs an adversarial loss function accordingly. The electronic discriminator obtains the classification result through forward inference and, based on a backpropagation algorithm, transmits the gradient information of the loss function back to the diffraction neural network generator to update the phase parameters of each layer of the spatial light modulator in the diffraction neural network, achieving system-wide collaborative optimization. Since the diffraction neural network generator is an all-optical architecture while the electronic discriminator is digitally implemented, there are inherent differences in the scale and continuity of their gradient responses. Therefore, a continuity constraint strategy is introduced into the electronic discriminator during training to limit its Lipschitz continuity constant to a finite range, thereby improving the stability and convergence efficiency of adversarial training. Specifically, the electronic discriminator, with the added continuity preset strategy, calculates the loss signals of the generated and real images and transmits them back to the diffraction neural network generator to adjust its phase parameters.

[0042] Then, the second sample light field is input into the diffraction neural network with adjusted phase parameters, and the loss signal is iterated through an electronic discriminator until the index of the loss signal meets the set index conditions, thus obtaining the trained diffraction neural network generator. The set index conditions can be either reaching a set number of iterations or the loss signal meeting a set loss signal range.

[0043] Specifically, the Wasserstein distance can be used to construct the adversarial loss function for the diffraction neural network generator and the electronic discriminator. The Wasserstein distance is a more stable distance metric suitable for training generative adversarial networks. Then, a gradient penalty term is added to the adversarial loss function to constrain the Lipschitz constant of the electronic discriminator within a set range. This range can be set based on requirements. This ensures that the discriminator's output does not change too drastically, thereby improving training stability.

[0044] During the reverse propagation process, the phase parameters of the diffraction neural network generator are first fixed, and the first weight gradient of the electron discriminator is clipped. The first weight gradient is the weight gradient of the electron discriminator, which makes the Lipschitz constant of the electron discriminator within a set range. The discriminator parameters of the electron discriminator are then updated based on the clipped first weight gradient.

[0045] Then, with the discriminator parameters of the electronic discriminator fixed, the adversarial-diffractive synergistic loss of the diffraction neural network generator is calculated. The second weight gradient of the diffraction neural network generator is then calculated using the backpropagation algorithm. This second weight gradient is the weight gradient of the diffraction neural network generator, and the phase parameters of the diffraction neural network generator are updated based on this second weight gradient. This invention proposes a loss function design method suitable for all-optical image generators. This method combines traditional image quality optimization with diffraction efficiency constraints in the physical system to form an engineering-feasible adversarial-diffractive synergistic loss.

[0046] The adversarial-diffraction cooperative loss function can be obtained by the following formula:

[0047] ;

[0048] in, To counteract diffraction-coordinated loss, For the original diffraction neural network generator, adversarial loss, It is the energy utilization rate of the diffraction neural network. To adjust the weighting factor between image fidelity and light energy utilization.

[0049] These steps effectively train the diffraction neural network generator corresponding to the all-optical image, ensuring the stability and convergence of the generator and discriminator during the training process. This improves the performance of the diffraction neural network generator, ensures high-quality generated images, and optimizes the efficiency of the diffraction system.

[0050] After training, the electronic discriminator module can be omitted during the inference stage, retaining only the optical input, encoding, generation, and imaging components to achieve image generation entirely based on optical structures. This stage requires no electronic computing unit intervention and can complete image generation on the nanosecond scale, possessing extremely high speed advantages and energy efficiency, making it suitable for high-speed, low-power edge intelligent image synthesis scenarios.

[0051] In step 104, the output target optical representation is first focused onto the image plane of the image sensor, and the focused light field intensity distribution is recorded. Using a lens or other optical element to focus the target optical representation ensures that the light field distribution on the image sensor is as clear as possible, and the image plane position is the target position where the light field is focused. The light field intensity distribution reflects the amplitude and phase information of the light field.

[0052] Then, based on the light field intensity distribution, the target optical representation is converted into a target electrical signal. The target electrical signal is an electrical signal generated based on the target optical representation. The target electrical signal represents the intensity information of the light field. In one example, the target electrical signal can also undergo preliminary processing, such as amplification and filtering, to improve signal quality.

[0053] The target electrical signal is then converted from analog to digital to generate a discretized digital image matrix, thus obtaining the initial digital image. Converting continuous electrical signals into discrete digital signals is fundamental to digital image processing. The generated digital image matrix contains the pixel values ​​of the image, with each pixel value corresponding to a photosensitive unit on the image sensor. The image is reconstructed using the digital image matrix to obtain the initial digital image, where each pixel value can correspond to a point in the image. The generated initial digital image can be a grayscale image or a color image, depending on the type and configuration of the image sensor.

[0054] Finally, the initial digital image undergoes noise reduction and gain preprocessing to obtain the target image. For example, noise reduction techniques such as Gaussian filtering and median filtering can be applied to reduce noise in the initial digital image. The visual effect of the initial digital image is improved by adjusting its brightness and contrast. If the initial digital image is a color image, color correction can also be performed to ensure color accuracy and consistency.

[0055] This application also provides a computer-readable storage medium storing a program that can be loaded and executed by a processor, which is any of the all-optical image generation methods and storage media based on diffraction neural networks in this application.

[0056] Those skilled in the art will understand that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to achieve the above functions. For example, the program can be stored in the memory of a device, and when the program in the memory is executed by the processor, all or part of the above functions can be achieved. In addition, when all or part of the functions in the above embodiments are implemented by computer programs, the program can also be stored in a server, another computer, disk, optical disk, flash drive, or external hard drive, etc., and can be downloaded or copied to the memory of a local device, or the system of the local device can be updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be achieved.

[0057] The above examples illustrate this application only to aid understanding and are not intended to limit its scope. Those skilled in the art to which this application pertains can make various simple deductions, modifications, or substitutions based on the ideas presented.

Claims

1. A diffractive neural network based all-optical image generation method, characterized by, The method comprises the following steps: initializing a first light field input, the first light field being generated by a laser and constructed into a random distribution of optical representation by a programmable optical device loaded with a random distribution of set amplitude or phase; acquiring sampling points in the first light field; enlarging the size of the sampling points by an amplification optical device with a set amplification ratio to obtain a plurality of enlarged regions, each enlarged region corresponding to one sampling point; calculating first spatial region coordinates of the plurality of enlarged regions according to the amplification ratio of the amplification optical device; determining second spatial region coordinates of target regions of the plurality of enlarged regions based on the first spatial region coordinates of the plurality of enlarged regions; marking the positions of pinholes in a mask design diagram of a pinhole array, and obtaining the pinhole array based on the mask design diagram, wherein the position of each pinhole corresponds to one second spatial region coordinate; transmitting the light of the target regions of the plurality of enlarged regions through the pinhole array to obtain a second light field; inputting the second light field into the diffractive neural network generator, the diffractive neural network generator being composed of a plurality of propagation networks, each propagation network comprising a spatial light modulation unit and a free-space propagation unit; for each propagation network, loading a two-dimensional phase mask of the second light field by the spatial light modulation unit to modulate the second light field; forming interference and diffraction of the modulated second light field through the free-space propagation unit to obtain mapping and feature transformation of the second light field corresponding to each propagation network; obtaining a target optical representation of the second light field based on the mapping and feature transformation of the second light field corresponding to each propagation network; focusing the output target optical representation to an image sensor through a focusing lens device to obtain a target image.

2. The plenoptic image generation method of claim 1, wherein, The method further comprises a step of training the diffractive neural network generator, which comprises: obtaining a training sample set, the training sample set comprising a plurality of first sample light fields; initializing the first sample light fields; spatially filtering the first sample light fields by full-light sparse coding to form second sample light fields of sparse activation patterns; inputting the second sample light fields into the diffractive neural network generator to be trained to obtain generated images output by the diffractive neural network generator; calculating a loss signal of the generated images and real images by an electronic discriminator added with a continuity preset strategy, and reversely transmitting the loss signal to the diffractive neural network generator to adjust phase parameters of the diffractive neural network generator; inputting the second sample light fields into the diffractive neural network with adjusted phase parameters, and iterating the loss signal by the electronic discriminator until an index of the loss signal meets a set index condition to obtain the trained diffractive neural network generator.

3. The plenoptic image generation method of claim 2, wherein, The step of calculating a loss signal of the generated images and real images by an electronic discriminator added with a continuity preset strategy, and reversibly transmitting the loss signal to the diffractive neural network generator to adjust phase parameters of the diffractive neural network generator comprises: The Wasserstein distance is used to construct an adversarial loss function of the diffractive neural network generator and the electronic discriminator; A gradient penalty term is added to the adversarial loss function to limit the Lipschitz constant of the electronic discriminator within a set range; During the back propagation process, the phase parameters of the diffractive neural network generator are fixed, the first weight gradient of the electronic discriminator is clipped so that the Lipschitz constant of the electronic discriminator is within a set range, and the discriminator parameters of the electronic discriminator are updated based on the clipped first weight gradient; The discriminator parameters of the electronic discriminator are fixed, and an adversarial-diffractive collaborative loss of the diffractive neural network generator is calculated, a second weight gradient of the diffractive neural network generator is calculated by a back propagation algorithm, and the phase parameters of the diffractive neural network generator are updated based on the second weight gradient; The adversarial-diffractive collaborative loss is obtained by the following formula: ； wherein, is the adversarial-diffraction synergistic loss for the adversarial-diffraction synergistic loss, is the original diffraction neural network generator adversarial loss, is the energy utilization of the diffraction neural network, is a weight factor to adjust the fidelity of the image and the utilization of light energy.

4. The plenoptic image generation method of claim 2, wherein, The second sample light field is input into the diffractive neural network generator to be trained to obtain a generated image output by the diffractive neural network generator, including: The second sample light field is input into the diffractive neural network generator to transform the spatial distribution characteristics of the second sample light field layer by layer through physical propagation to obtain a sample optical representation corresponding to the second sample light field; The output sample optical representation is focused on an image sensor through a focusing lens device to obtain the generated image.

5. The plenoptic image generation method according to any one of claims 1 to 4, characterized in that, The output target optical representation is focused on the image plane position of the image sensor, and the light field intensity distribution after focusing is recorded; Based on the light field intensity distribution, the target optical representation is converted into a target electrical signal; The target electrical signal is analog-digital converted to generate a discretized digital image matrix to obtain an initial digital image; The initial digital image is preprocessed by noise reduction and gain to obtain the target image. The computer readable storage medium stores a program, and the program can be loaded and executed by the processor to perform the diffractive neural network based full optical image generation method of any one of claims 1 to 5.

6. A computer-readable storage medium, characterized in that, ​

Images