A method and system for complex image restoration based on a dual-cycle optimized diffusion model

Abstract

The application discloses a complex image recovery method and system based on a double-loop optimization diffusion model. The application adopts the double-loop optimization diffusion model to realize complex image recovery, which comprises outer loop optimization and inner loop optimization. In the outer loop optimization, the time step is decreased from the maximum iteration step T to 0, and the diffusion model is used at each time step to gradually reduce the noise in the image, so that the reconstruction of the amplitude and phase of the complex image is realized from complete random noise to a clear image. The inner loop optimization optimizes the image after denoising at each time step in the outer loop optimization and serves as the initial value of the next time step in the outer loop optimization. The application can effectively recover the amplitude and phase images of the complex image, significantly improve the quality of the reconstructed amplitude and phase images by relying on the powerful learning and reconstruction capability of the diffusion model, and exhibits excellent performance advantages in the field of complex image recovery.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method and system for complex image restoration based on a dual-cycle optimized diffusion model. Background Technology

[0002] Fourier Ptychography (FP), as a cutting-edge computational imaging technique, has demonstrated enormous application potential in numerous fields. Its emergence and development are driven by profound technological backgrounds and practical needs. In the 1950s, Fourier optics emerged as a branch of modern optics, combining mathematics, electronics, communication theory, and optics. It utilizes Fourier analysis methods and the concept of spatial spectrum to analyze the propagation, diffraction, and imaging of light, providing a solid theoretical foundation for Fourier Ptychography. In the field of optical imaging, large field-of-view and high-resolution imaging have always been relentlessly pursued goals. Traditional microscopy techniques, limited by the space-bandwidth product, struggle to achieve both large field-of-view and high-resolution imaging simultaneously. For example, biomedical applications require observing large samples and distinguishing cell structures, while industrial inspection demands the detection of large material surfaces and the discovery of minute defects. However, traditional imaging techniques cannot meet these demands, failing to provide sufficiently clear and comprehensive image information. This predicament has become a bottleneck restricting the development of related fields.

[0003] With the rapid development of computer technology, computational imaging has naturally become a research focus in the imaging field, injecting strong momentum into the innovation of imaging technology. Computational imaging, by tightly integrating optical systems with digital signal processing algorithms, breaks through many limitations of traditional imaging systems, opening up new paths for achieving more advanced imaging functions. Fourier layered imaging, based on the idea of ​​computational imaging, acquires multiple low-resolution images and uses algorithms to reconstruct high-resolution, large-field-of-view images, thus meeting the needs of various fields for high-quality imaging. Due to the rapid rise of deep learning technology, neural networks have begun to be introduced into Fourier layered imaging, hoping to leverage their powerful feature learning and pattern recognition capabilities to overcome the limitations of traditional methods. However, the reconstruction effect is still constrained by the difficulty of phase retrieval and insufficient adaptability to complex scenes.

[0004] Diffusion models, as an emerging force in generative modeling, have demonstrated strong potential in image generation and editing tasks in recent years. Their core principle is to simulate the process of data gradually diffusing and then recovering from noise. By learning from large amounts of data, they grasp the data distribution characteristics and can then generate high-quality samples. In image generation tasks, diffusion models start from pure noise and use neural networks to gradually remove noise, generating realistic images. In image restoration tasks, missing parts can be filled in through a reverse diffusion process based on information surrounding the damaged image. However, directly applying diffusion models to complex image restoration faces many challenges. The unique complex representation of complex images makes it difficult for models to effectively learn and handle the complex relationship between amplitude and phase, and complex image restoration requires extremely high accuracy and detail preservation. Therefore, there is an urgent need to explore innovative diffusion model-based methods suitable for complex image restoration. Summary of the Invention

[0005] The purpose of this invention is at least to address the problem that existing methods for recovering the amplitude and phase of complex images are not very effective. This invention proposes a method and system for complex image recovery based on a dual-cycle optimized diffusion model. By introducing a dual-cycle optimized diffusion model into the recovery of the amplitude and phase information of complex images, the quality of the reconstructed amplitude and phase images is improved. This invention can effectively recover the amplitude and phase of complex images and, relying on the powerful learning and reconstruction capabilities of the diffusion model, significantly improves the quality of the reconstructed amplitude and phase images, demonstrating superior performance advantages in the field of complex image recovery.

[0006] In a first aspect, the present invention provides a method for complex image restoration based on a dual-cycle optimized diffusion model, the method comprising:

[0007] Obtain the target complex image and the corresponding low-resolution intensity image of the real scene, and construct a dataset;

[0008] A dual-loop optimization diffusion model is constructed and pre-trained using a dataset. The dual-loop optimization diffusion model includes an outer loop optimization and an inner loop optimization. In the outer loop optimization, the time step decreases from the maximum iteration step T to 0. At each time step, the diffusion model is used to gradually reduce the noise in the image, ultimately reconstructing the amplitude and phase of the complex image from completely random noise to a clear image. The inner loop optimization optimizes the denoised image at each time step of the outer loop optimization and uses it as the initial value for the next time step of the outer loop optimization.

[0009] The amplitude and phase images of the low-resolution intensity image to be recovered are optimized in the outer loop of a pre-trained double-loop optimized diffusion model to recover the amplitude and phase images of the corresponding complex image.

[0010] Preferably, the outer loop optimization includes two identical diffusion models, both of which use a Gaussian noise image as the input image to predict the amplitude image and phase image of the corresponding target complex image, respectively.

[0011] More preferably, each diffusion model includes a forward Markov chain process and a reverse Markov chain process;

[0012] The forward Markov chain process includes multiple time-step stages; each time-step stage gradually adds Gaussian noise to the input image, eventually forming a pure noise image.

[0013] The reverse Markov chain process includes multiple time-step stages; a random Gaussian noise tensor is used as the initial state of the amplitude image or phase image during the recovery process, and the amplitude image and phase image are gradually denoised in each time-step stage to reconstruct a high-quality amplitude image or phase image.

[0014] Preferably, the process of acquiring the target complex image and the corresponding low-resolution intensity image in the real scene is as follows:

[0015] By illuminating the target complex image from multiple angles using an LED array, and based on the limitations of the objective lens NA and the illumination wave vectors at different angles, the pupil function under illumination at different angles is derived and obtained.

[0016] The point spread function is obtained by performing an inverse Fourier transform on the pupil function;

[0017] The target complex image is convolved with a point spread function to generate a low-resolution intensity image.

[0018] More preferably, the process of acquiring the target complex image and the corresponding low-resolution intensity image in the real scene is as follows:

[0019] The two input images are subjected to grayscale conversion and data processing, and the values ​​of OR and OI are assigned as amplitude and phase information, respectively. Then, the real part of the target complex image is calculated using the amplitude and phase information. and the virtual part Combine the real and imaginary parts to form the target complex image O;

[0020] A plane wave is emitted from an LED array to illuminate the target complex image sample from different angles.

[0021] The pupil function is constructed by calculating the incident wave vector of a plane wave and retaining only the frequency components within the numerical aperture limit of the objective lens.

[0022] The point spread function is obtained by inverse Fourier transform of the pupil function;

[0023] The point spread function is multiplied by the phase modulation factor to simulate the point spread function with the influence of the incident direction.

[0024] A low-resolution intensity image is generated by convolving the target complex image with a point spread function that is affected by the incident direction.

[0025] Preferably, the inner loop optimization uses a convolutional neural network to optimize the denoised image at each time step of the outer loop optimization.

[0026] Preferably, the process of using a convolutional neural network to iteratively optimize the denoised image at each time step specifically involves:

[0027] The denoised amplitude and phase images at each time step of the inverse Markov chain process of the two diffusion models are used as inputs to the convolutional neural network. A point spread function is then used to convolve the denoised amplitude and phase images. After processing through activation and addition layers, the predicted intensity image is finally captured. The process is represented as follows:

[0028] ;

[0029] in and Let be the real and imaginary parts of the point spread function, respectively. and The images are the amplitude and phase images after denoising at each time step of the inverse Markov chain process for the two diffusion models, respectively.

[0030] Based on gradient descent backpropagation of the loss function, the denoised amplitude image and phase image are used as input images for the outer loop optimization of the two diffusion models in the next time step, and the denoised image input to the convolutional neural network is updated accordingly. The process is represented as follows:

[0031] ;

[0032] in, The image is a real low-resolution intensity image; batchSize represents the number of images transmitted in the same batch; i is a self-defined value, i≥1.

[0033] Secondly, the present invention provides a complex image restoration system based on the above method, comprising:

[0034] The data acquisition module is responsible for acquiring the amplitude and phase images of the low-resolution intensity image to be recovered;

[0035] The data processing module is responsible for optimizing the amplitude and phase images of the low-resolution intensity image to be recovered through the outer loop of a pre-trained double-loop optimized diffusion model to recover the amplitude and phase images of the corresponding complex image.

[0036] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the image restoration method.

[0037] Fourthly, the present invention provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the image restoration method.

[0038] Compared with the prior art, the present invention has the following advantages:

[0039] This invention proposes a complex image restoration method based on a dual-loop optimized diffusion model, which combines the denoising capability of the diffusion model with the detail restoration capability of the convolutional neural network (CNN) in the Fourier forward imaging process to achieve complementarity, thus overcoming the limitations of a single model in processing complex images.

[0040] This invention constructs a hierarchical dual-loop optimization mechanism. Based on an outer loop using a diffusion model for progressive denoising, it combines this with an inner loop employing gradient descent based on a loss function for backward optimization, forming a coarse-to-fine dual processing chain. The two loops work together to optimize the image. Specifically, the outer loop performs denoising at each time step, generating an intermediate amplitude / phase image. The inner loop minimizes the intensity image loss, driving the CNN to update the input image and using it as the initial value for the next time step of the outer loop. This forms a closed loop of "diffusion denoising → physical constraint optimization → re-diffusion," iteratively refining the process to address the error accumulation problem of traditional end-to-end models in complex degradation scenarios, thereby improving reconstruction accuracy. Based on model combination and the dual-loop optimization strategy, this invention effectively improves the quality and accuracy of amplitude and phase recovery for complex images.

[0041] Therefore, the method of the present invention is more competitive in the field of image restoration, especially in applications involving the amplitude and phase of complex images. Attached Figure Description

[0042] Figure 1 This is a flowchart of a complex image restoration method based on a dual-loop optimized diffusion model provided by the present invention;

[0043] Figure 2 This is a flowchart of the simulation and generation of low-resolution intensity images in a real scene in this invention;

[0044] Figure 3 This is a diagram of the network structure of the convolutional neural network (CNN) used in this invention to model the forward modeling process of Fourier images;

[0045] Figure 4This is a schematic diagram illustrating the principle of iterative restoration of complex images using a combination of diffusion model and convolutional neural network in this invention.

[0046] Figure 5 The images shown are the results of recovering complex images using the dual-loop optimized diffusion model of this invention. (a) and (b) are the input images that serve as amplitude and phase information, respectively. (c) and (d) are the amplitude and phase results of the complex images recovered by the dual-loop optimized diffusion model. Detailed Implementation

[0047] To better illustrate the invention and advantages of this project, the invention will be further explained below with reference to the accompanying drawings and examples.

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0049] At least one embodiment provides a complex image restoration method based on a dual-loop optimized diffusion model. By combining the diffusion model with a convolutional neural network model for restoration, the method recovers the amplitude and phase images of complex images, improving the restoration quality and enabling the recovery of higher-quality complex image amplitude and phase images. This expands the application scope of Fourier layered imaging, enabling its application in multiple fields such as biomedicine and digital pathology. See Appendix. Figure 1 The above method includes the following steps:

[0050] Step S1: Obtain the target complex image and the corresponding low-resolution intensity image of the real scene, and construct a dataset;

[0051] Furthermore, the model simulates and acquires low-resolution intensity images of real-world scenes, mimicking complex illumination conditions that are difficult to acquire in actual scenarios, thus improving the model's adaptability to real-world imaging environments. The process of generating low-resolution intensity images incorporates multi-angle illumination technology. By superimposing wave vectors at different angles during the Fourier transform process, complex illumination conditions that are difficult to acquire in real-world scenes can be simulated, further enhancing the model's adaptability to real-world imaging environments. Specifically:

[0052] By illuminating the target complex image from multiple angles using an LED array, and based on the limitations of the objective lens's numerical aperture (NA) and the illumination wave vectors at different angles, the pupil function under different illumination angles is derived and obtained. This embodiment uses an LED array to illuminate the target complex image from different incident angles to acquire low-resolution intensity images containing information from different angles, fully exploring the potential features of the image at different angles.

[0053] The pupil function is inversely transformed by Fourier transform to obtain the point spread function (PSF); the target complex image is convolved with the PSF to generate a low-resolution intensity image.

[0054] As an example, see Appendix Figure 2 More specifically, it could be:

[0055] Step S1.1: Obtain the target complex image O

[0056] like Figure 2 As shown, this embodiment first constructs a simulation model of a microscope platform and sets its relevant physical parameters. Regarding the simulation of the target complex image, two images are input and integrated into a single complex image as the simulation target. First, the two input images are subjected to grayscale conversion and data processing, and their values ​​are assigned amplitude information (OR) and phase information (OI), respectively. Then, the real part of the target complex image is calculated using the amplitude and phase information. and the virtual part The calculation formula is:

[0057] Equation (1)

[0058] Equation (2)

[0059] Then, the real and imaginary parts are combined to form the target complex image O, that is:

[0060] Equation (3)

[0061] Where j represents the imaginary unit;

[0062] Specifically, the data processing involves converting the data type of the grayscale converted image into a double-precision floating-point number, and normalizing the image as phase information and mapping it to [-Π / 2, Π / 2].

[0063] This processing method uses the two input images as carriers of amplitude and phase information, respectively, and finally synthesizes the target complex image.

[0064] Step S1.2: After simulating and generating the target complex image, a 15×15 plane wave is emitted from an LED array to illuminate the target complex image sample from different angles.

[0065] One implementation uses an "S"-shaped scanning trajectory for the LED array, which can efficiently traverse the image area with a limited number of frames, maximizing the capture of spatial information. This scanning method is based on a spiral structure, with the core idea being to start from the image center and gradually expand spirally in a clockwise or counterclockwise manner, sequentially covering all points in the two-dimensional scanning area. Compared to traditional linear scanning, this method has advantages such as uniform coverage and strong scanning continuity, effectively improving the correlation between sampled images and spatial reconstruction capabilities.

[0066] Step S1.3: Construct the pupil function by calculating the incident wave vector of the plane wave and retaining only the frequency components within the numerical aperture (NA) constraint range. .

[0067] Equation (4)

[0068] in The wave vectors representing the incident directions along the x and y axes. Represents the wavenumber of light in a vacuum. ;

[0069] Step S1.4: Obtain the point spread function (PSF) by performing an inverse Fourier transform on the pupil function:

[0070] Equation (5)

[0071] Step S1.5: Multiply the point spread function (PSF) by the phase modulation factor to simulate the PSF with the influence of the incident direction. The process is expressed as follows:

[0072] Equation (6)

[0073] in and This is the nth wave vector along the incident directions of the x-axis and y-axis; This represents two-dimensional spatial coordinates, and i represents the imaginary unit.

[0074] Step S1.6: Convolve the target complex image with the point spread function PSF to generate a low-resolution intensity image; the process is represented as follows:

[0075] Equation (7)

[0076] in and They are respectively The real and imaginary parts, and The amplitude and phase images of the target complex image.

[0077] The training data of this invention generates low-resolution intensity images covering complex scenes by simulating actual optical limitations and multi-angle illumination effects.

[0078] Step S2: Construct a double-loop optimized diffusion model and pre-train the double-loop optimized cyclic diffusion model using the dataset;

[0079] The dual-loop optimized diffusion model includes outer loop optimization and inner loop optimization.

[0080] (1) In the outer loop optimization, the time step decreases from the maximum iteration step T to 0. At each time step, the diffusion model is used to gradually reduce the noise in the image, and finally the amplitude and phase of the complex image are reconstructed from completely random noise to a clear image.

[0081] In one implementation, the outer loop optimization includes two identical diffusion models, each including a forward Markov chain process and a reverse Markov chain process; the two identical diffusion models respectively use a Gaussian noise image as the initial state of the input image (amplitude image / phase image) in the recovery process, and respectively predict the amplitude image and phase image of the corresponding target complex image.

[0082] As an example, the diffusion model in this embodiment is a DDPM-CD diffusion model trained using 3-band remote sensing images taken by Sentinel-2.

[0083] The forward Markov chain process includes multiple time step stages;

[0084] Gaussian noise is gradually added to the input image at each time step, eventually forming a pure noise image, represented as:

[0085] Equation (8)

[0086] in The image with noise represents time t. This represents the cumulative noise attenuation coefficient. , ; This represents the Gaussian noise added at time t-1; This represents the noise term that conforms to a standard Gaussian distribution. Represents a clear, original image; This indicates the proportion of the original signal retained in step i.

[0087] The reverse Markov chain process includes multiple time-step stages; a random Gaussian noise tensor is used as the initial state of the amplitude image or phase image during the recovery process, and the amplitude image and phase image are gradually denoised in each time-step stage to reconstruct a high-quality amplitude image or phase image.

[0088] The process of progressively denoising the image at each time step can be represented as:

[0089] Equation (9)

[0090] in This represents a pre-trained noise prediction network. Image representing time step t The variance of , z represents the auxiliary term conforming to Gaussian noise and z=0 at t=1. For the amplitude or phase image at time step t;

[0091] (2) In the inner loop optimization, the convolutional neural network (CNN) is used to optimize the denoised image at each time step of the inverse Markov chain process of the two diffusion models in the outer loop optimization, and it is used as the initial value for the next time step of the outer loop optimization.

[0092] The inner-loop optimized convolutional neural network (CNN) focuses on modeling local details of the Fourier forward imaging process, performing refined recovery of the amplitude and phase information of complex images. This network complements the diffusion model in the outer-loop optimized system, constructing a complete complex image restoration system and further improving the quality and accuracy of the restored images.

[0093] See appendix Figure 3 The input to the convolutional neural network (CNN) is the denoised amplitude image and phase image at each time step. This input is treated as a two-channel learnable filter for a convolutional layer, with a stride of 4 used for the convolutional layer.

[0094] The point spread function is first convolved with the amplitude Or and phase Oi of the model to be trained. After processing through activation and additive layers, the output is the predicted intensity image finally captured by the network model. The process can be represented as follows:

[0095] Equation (10)

[0096] in and Let be the real and imaginary parts of the point spread function, respectively. and These are the denoised amplitude and phase images for each time step, respectively. This invention explicitly embeds a physical imaging model into the inner-loop CNN, aligning the optimization process with the physical rules of real imaging.

[0097] The process of restoring the amplitude and phase of a complex image in a network model is represented as follows:

[0098] Equation (11)

[0099] in, For true low-resolution intensity images, The inner loop optimizes the captured predicted intensity images, where batchSize represents the number of images passed forward / backward in one pass; i is a user-defined value, i≥1.

[0100] During backpropagation, the difference between the predicted intensity image and the acquired true intensity image... The loss function is backpropagated to the convolutional layers, updating the dual-channel objects accordingly. Therefore, the training process of a CNN model can be viewed as minimizing the loss function, achieving accurate recovery of the amplitude and phase information of the complex image through iterative optimization. The recovered amplitude and phase images in the network model are then used as input to the next time step in the diffusion model for the next round of image denoising. After the next round of denoising, the images are fed back into the network model for training and optimization. This iterative optimization process ultimately reconstructs high-quality amplitude and phase images of the complex image.

[0101] This invention uses random Gaussian noise as the initial state of amplitude and phase images, and utilizes the progressive denoising of the diffusion model and the local detail modeling of CNN to achieve approximate reconstruction from the initial state of pure noise to the target amplitude and phase images, thus constructing a complete complex image dual-component recovery link.

[0102] The diffusion model of this invention utilizes its progressive denoising capability to gradually reconstruct the global structure of the amplitude / phase image from the initial state of random Gaussian noise. The CNN model refines the deviation between the diffusion model output and the real physical process by locally modeling the Fourier forward imaging process. This solves the problem that traditional single models cannot simultaneously handle complex representation relationships and high-precision reconstruction, significantly improving the fidelity of the restored image. Complementary optimization is achieved through a dual-model collaborative mechanism.

[0103] Step S3: Using the amplitude and phase images of the low-resolution intensity image to be recovered, respectively, pass them through the inverse Markov chain processes of the two pre-trained diffusion models to recover the amplitude and phase images of the corresponding high-resolution complex image. See Appendix Figure 5 ,by Figure 5 (a) Figure 5 (b) The two images in the middle are used as the two input images for amplitude and phase information in step 1.1, respectively. The amplitude and phase of the complex image are recovered by the dual-loop optimized diffusion model. See the image below. Figure 5 (c) Figure 5 (d) shows that the dual-cycle optimized diffusion model of the present invention can better remove noise and reconstruct high-quality amplitude and phase images.

[0104] This embodiment also provides a complex image restoration system based on the above method, including:

[0105] The data acquisition module is responsible for acquiring the amplitude and phase images of the low-resolution intensity image to be recovered;

[0106] The data processing module is responsible for optimizing the amplitude and phase images of the low-resolution intensity image to be recovered through the outer loop of a pre-trained double-loop optimized diffusion model to recover the amplitude and phase images of the corresponding complex image.

[0107] This invention provides an electronic device, specifically, the electronic device includes a memory and a processor, the memory stores executable code, and when the processor executes the executable code, it implements the method described in any of the embodiments.

[0108] The memory may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk drive. Communication between this system network element and at least one other network element is achieved through at least one communication interface (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0109] The bus can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc.

[0110] The memory is used to store programs. After receiving an execution instruction, the processor executes the program. The method executed by the device for defining the flow process disclosed in any of the foregoing embodiments of the present invention can be applied to the processor or implemented by the processor.

[0111] The processor may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above methods can be completed by integrated logic circuits in the processor's hardware or by software instructions. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0112] The computer program product of the readable storage medium provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the foregoing method embodiments. For specific implementation, please refer to the foregoing method embodiments, which will not be repeated here.

[0113] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0114] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0115] The above detailed description further illustrates the purpose, technical solution, and beneficial effects of the invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A complex image restoration method based on a double-circulation optimized diffusion model, characterized in that, The method comprises: acquiring a target complex image and a low-resolution intensity image corresponding to a real scene, and constructing a dataset; constructing a double-cycle optimization diffusion model, and pre-training the double-cycle optimization diffusion model using the dataset; the double-cycle optimization diffusion model comprises outer cycle optimization and inner cycle optimization; in the outer cycle optimization, a time step is decremented from a maximum iteration step T to 0, and a diffusion model is used at each time step to gradually reduce noise in an image, so that the amplitude and phase of the complex image are finally reconstructed from complete random noise to a clear image; the inner cycle optimization optimizes the image after denoising at each time step in the outer cycle optimization, and uses the image as an initial value for the next time step in the outer cycle optimization; using the amplitude image and the phase image of the low-resolution intensity image to be restored as input images of the outer cycle optimization of the pre-trained double-cycle optimization diffusion model, to obtain the amplitude image and the phase image of the corresponding complex image; the outer cycle optimization comprises two diffusion models; the inner cycle optimization uses a convolutional neural network to optimize the image after denoising at each time step in the outer cycle optimization, specifically: the amplitude image and the phase image after denoising at each time step of the reverse Markov chain process of the two diffusion models are used as input images of the convolutional neural network, the amplitude image and the phase image after denoising are convolved with a point spread function, and then the convolutional neural network is processed through an activation layer and an addition layer, so that a predicted intensity image is finally captured; based on gradient descent reverse optimization of a loss function loss, the amplitude image and the phase image after denoising are used as input images of the two diffusion models of the outer cycle optimization at the next time step, and the input image after denoising of the convolutional neural network is updated accordingly.

2. The complex image restoration method based on the double-circulation optimized diffusion model according to claim 1, wherein, The two diffusion models have the same architecture, and both use a Gaussian noise image as an input image to predict the amplitude image and the phase image of the corresponding target complex image.

3. The complex image restoration method based on the double-circulation optimized diffusion model according to claim 2, characterized in that, Each diffusion model comprises a forward Markov chain process and a reverse Markov chain process. The forward Markov chain process comprises a plurality of time step stages; each time step stage gradually adds Gaussian noise to the input image, and finally forms a pure noise image; The reverse Markov chain process comprises a plurality of time step stages; a random Gaussian noise tensor is used as an initial state of the amplitude image or the phase image in the recovery process, and each time step stage gradually denoises the amplitude image and the phase image to reconstruct a high-quality amplitude image or phase image.

4. The complex image restoration method based on the double-circulation optimized diffusion model according to claim 1, wherein, The implementation process of acquiring the target complex image and the low-resolution intensity image corresponding to the real scene is specifically as follows: a target complex image is irradiated at multiple angles by an LED array, and a pupil function under different angle irradiation is derived and acquired according to the limitation of the objective lens NA and the illumination wave vector under different angles; inverse Fourier transform is performed on the pupil function to obtain a point spread function; the target complex image is convolved with the point spread function to generate a low-resolution intensity image.

5. The method of claim 4, wherein the method of recovering a complex image based on a double-circulation optimized diffusion model is characterized by, The implementation process of acquiring the target complex image and the low-resolution intensity image corresponding to the real scene is specifically as follows: The input two pictures are respectively subjected to gray scale conversion and data processing, and are respectively assigned as or and oi as amplitude and phase information, and then the real part and imaginary part of the target complex image are calculated by using the amplitude and phase information and the imaginary part ; the real part and the imaginary part are combined into the target complex image O; a target complex image sample is irradiated at different angles by an LED array emitting a plane wave; A pupil function is constructed by calculating the incident wave vector of the plane wave and only retaining the frequency components within the numerical aperture limit of the objective lens; An inverse Fourier transform of the pupil function is obtained to obtain a point spread function; The point spread function is multiplied by a phase modulation factor to simulate a point spread function with an incident direction effect; The target complex image is convolved with the point spread function with the incident direction effect to generate a low-resolution intensity image.

6. A complex image restoration system implementing the method of any one of claims 1 to 5, characterized in that, It comprises: A data acquisition module is responsible for acquiring the amplitude image and phase image of the low-resolution intensity image to be recovered; A data processing module is responsible for optimizing the amplitude image and phase image of the low-resolution intensity image to be recovered through the outer loop optimization of the pre-trained double-loop optimization diffusion model to recover the corresponding amplitude image and phase image of the complex image.

7. A computer readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method of any one of claims 1-5.

8. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the method of any one of claims 1-5.

Images