Pattern illumination imaging method, system, electronic device and storage medium

Abstract

The present application relates to the technical field of computational imaging, and specifically discloses a pattern illumination imaging method, a pattern illumination imaging system, an electronic device and a storage medium, wherein the pattern illumination imaging method comprises the following steps: first, a unified model containing position error, illumination, noise and optical transfer function is established to truly fit the actual system; then, frequency domain sub-pixel correction is performed to realize high-precision position alignment; next, the loss of the logarithmic focal point frequency is strengthened to enhance high frequency and suppress interference; then, the position is embedded into iteration as an optimizable variable to realize multiple error collaborative correction, so that a reconstructed image with high resolution, low artifact and high precision can be stably output in a complex environment, and the present application has good imaging stability and practicability.

Description

Technical Field

[0001] This invention belongs to the field of computational imaging technology, and specifically relates to a pattern illumination imaging method. Background Technology

[0002] PiFP (Pattern-illuminated Fourier ptychography) is a super-resolution computational imaging technique developed based on traditional Fourier ptychography. Its basic principle is to use unknown random illumination patterns to scan the surface of an illuminated object along a fixed path. By using random illumination patterns, the spatial high-frequency components of the target object are shifted into the system's imaging passband. The detector acquires multiple frames of intensity images under different illumination patterns, and combined with iterative reconstruction algorithms, high-resolution imaging of the target is achieved.

[0003] The theoretical model of pattern illumination Fourier stacked imaging technology is based on the assumption of a relatively ideal linear time-invariant system, which assumes that the position of the illumination pattern is precisely known and there is no positional error, the light intensity is stable without fluctuation, and the system is drift-free. However, in actual imaging, it is difficult to eliminate factors such as scanning position deviation, uneven distribution of illumination patterns, light intensity fluctuations, and system noise, which can affect the image quality.

[0004] In related technologies, the deviation between the imaging model and the actual system is usually addressed by correcting various errors separately. However, this method is difficult to model multiple error factors in a unified manner. In the case of multiple interference factors coupled together, the reconstructed image is prone to phenomena such as reduced resolution and increased artifacts, which limits the imaging stability and practicality. Summary of the Invention

[0005] This invention aims to at least address the technical problems existing in related technologies, such as the difficulty in uniformly modeling multiple error factors, which easily leads to reduced resolution and increased artifacts in reconstructed images, resulting in poor imaging stability and practicality. Therefore, the purpose of this invention is to propose a pattern illumination imaging method, system, electronic device, and storage medium.

[0006] In a first aspect, the present invention proposes a pattern illumination imaging method, comprising the following steps: constructing a forward imaging model for pattern illumination Fourier stacked imaging, including scan position error, illumination mode, optical transfer function, and composite noise; initializing the complex amplitude wavefront to be optimized according to the forward imaging model, and constructing a target batch dataset from multiple acquired images; wherein the complex amplitude wavefront to be optimized includes the target complex amplitude to be reconstructed and the mother illumination mode complex amplitude to be optimized; based on the target batch dataset, correcting the scan position error using frequency domain correlation subpixel displacement calculation to obtain a position-corrected image sequence; constructing a feature domain loss function based on logarithmic focal frequency loss based on the position-corrected image sequence; using the scan position error as an optimizable variable, and combining the forward imaging model, the feature domain loss function, and the complex amplitude wavefront to be optimized, iteratively updating the target complex amplitude to be reconstructed and the mother illumination mode complex amplitude to be optimized; after iterative convergence, outputting the iteratively updated target complex amplitude to be reconstructed as a high-resolution target image, and outputting the iteratively updated mother illumination mode complex amplitude to be optimized as a mother illumination mode image.

[0007] According to the pattern illumination imaging method of the present invention, firstly, a forward imaging model for pattern illumination Fourier stacked imaging, including scan position error, illumination mode, optical transfer function, and composite noise, is constructed. Secondly, based on the forward imaging model, the complex amplitude wavefront to be optimized is initialized, and multiple frames of acquired images are constructed into a target batch dataset. Then, based on the target batch dataset, the scan position error is corrected using frequency domain correlation subpixel displacement calculation to obtain a position-corrected image sequence. Then, based on the position-corrected image sequence, a feature domain loss function based on logarithmic focal frequency loss is constructed. Finally, the scan position error is used as an optimizable variable, and the forward imaging model, feature domain loss function, and complex amplitude wavefront to be optimized are iteratively updated. The complex amplitude of the target to be reconstructed and the complex amplitude of the mother illumination mode to be optimized are iteratively converged. The iteratively updated complex amplitude of the target to be reconstructed is output as a high-resolution target image, and the iteratively updated complex amplitude of the mother illumination mode to be optimized is output as a mother illumination mode image. In this way, a unified model including position error, illumination, noise and optical transfer function is established to truly fit the actual system. High-precision position alignment is achieved through frequency domain subpixel correction, high frequency is enhanced and interference is suppressed through logarithmic focal frequency loss, and multiple error collaborative correction is achieved by embedding position as an optimizable variable in the iteration. This ensures stable output of high-resolution, low-artifact, and high-precision reconstructed images in complex environments, with good imaging stability and practicality.

[0008] In addition, the pattern illumination imaging method according to embodiments of the present invention may also have the following additional technical features: Furthermore, the construction of the forward imaging model for pattern illumination Fourier stacked imaging, which includes scanning position error, illumination mode, optical transfer function, and composite noise, includes: scanning the illumination target with an unknown illumination mode along a fixed spiral path, establishing the optical field coupling relationship between the target, illumination mode, scanning position error, and optical transfer function; and superimposing image domain composite noise on the basis of the optical field coupling relationship to form a forward imaging expression that includes scanning position error, illumination mode, optical transfer function, and composite noise.

[0009] Furthermore, the step of initializing the complex amplitude wavefront to be optimized according to the forward imaging model and constructing a target batch dataset from multiple acquired images includes: based on the forward imaging model, initializing the target complex amplitude to be reconstructed that matches the target size and the mother illumination pattern complex amplitude to be optimized that matches the illumination pattern size, which together constitute the complex amplitude wavefront to be optimized as the initial variable for iterative updates; and randomly shuffling and grouping all low-resolution intensity images acquired by illumination scanning to form the target batch dataset.

[0010] Furthermore, the step of correcting the scanning position error by calculating the frequency domain correlation subpixel displacement based on the target batch dataset to obtain a position-corrected image sequence includes: using the central scanning point image of the target batch dataset as a reference image; calculating the frequency domain cross-correlation between each image frame and the reference image to obtain the subpixel displacement corresponding to the scanning position error; and performing frequency domain shift compensation based on the subpixel displacement to obtain the position-corrected image sequence.

[0011] Furthermore, the step of constructing a feature domain loss function based on logarithmic focal frequency loss based on the position-corrected image sequence includes: generating a position-corrected predicted image according to the forward imaging model and the complex amplitude wavefront to be optimized; performing Fourier transforms on the predicted image and the position-corrected measured image respectively; calculating the logarithmic domain difference between the real and imaginary parts of the Fourier transform result, merging them into the L2 norm and setting the central DC component to zero; calculating the logarithmic attenuation error of the complex difference modulus, multiplying it by the frequency weights and averaging it in the frequency domain to obtain the logarithmic focal frequency loss function.

[0012] Furthermore, the step of using the scanning position error as an optimizable variable, combined with the forward imaging model, the feature domain loss function, and the complex amplitude wavefront to be optimized, to iteratively update the complex amplitude of the target to be reconstructed and the complex amplitude of the parent illumination mode to be optimized, includes: using the complex amplitude wavefront to be optimized as the initial variable for iteration, calculating the gradients of the complex amplitude of the target to be reconstructed and the complex amplitude of the parent illumination mode to be optimized based on the forward imaging model and the logarithmic focal frequency loss function; converting the gradients into complex amplitude gradients, and propagating them back to the object plane via the optical transfer function; relocating each illumination mode gradient to the parent illumination mode according to the scanning position; and using a root mean square propagation optimizer to synchronously iteratively update the complex amplitude of the target to be reconstructed and the complex amplitude of the parent illumination mode to be optimized.

[0013] Furthermore, the iterative update of the complex amplitude of the target to be reconstructed and the complex amplitude of the mother illumination mode to be optimized includes: performing scan position error correction cyclically in each iteration, calculating the loss value of the feature domain loss function based on the correction result, and updating the complex amplitude based on the correction result and the loss value, so as to achieve synchronous joint correction of scan position error, illumination change, light intensity fluctuation and mixed noise.

[0014] Secondly, based on the same inventive concept, this invention also proposes a pattern illumination imaging system, comprising: a first construction module for constructing a forward imaging model of pattern illumination Fourier stacked imaging including scan position error, illumination mode, optical transfer function, and composite noise; a second construction module for initializing the complex amplitude wavefront to be optimized according to the forward imaging model, and constructing a target batch dataset from multiple acquired images; wherein the complex amplitude wavefront to be optimized includes the target complex amplitude to be reconstructed and the complex amplitude of the parent illumination mode to be optimized; and a correction module for correcting the scan based on the target batch dataset using frequency domain correlation subpixel displacement calculation. The system corrects for positional errors to obtain a position-corrected image sequence. A third construction module is used to construct a feature domain loss function based on the position-corrected image sequence, using the logarithmic focal frequency loss. An update module is used to iteratively update the complex amplitude of the target to be reconstructed and the complex amplitude of the mother illumination mode to be optimized, taking the scanning position error as an optimizable variable and combining it with the forward imaging model, the feature domain loss function, and the wavefront of the complex amplitude to be optimized. An output module is used to output the iteratively updated complex amplitude of the target to be reconstructed as a high-resolution target image and the iteratively updated complex amplitude of the mother illumination mode to be optimized as a mother illumination mode image after iterative convergence.

[0015] The pattern illumination imaging system provided in the second aspect has the same technical features as the pattern illumination imaging method provided in the first aspect, and therefore also has similar technical effects as the first aspect, which will not be described in detail here.

[0016] Thirdly, based on the same inventive concept, the present invention also proposes an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store a computer program; and the processor is used to execute the computer program stored in the memory to implement the steps of the pattern illumination imaging method as described in the above embodiments of the present invention.

[0017] The electronic device provided in the third aspect has the same technical features as the pattern illumination imaging method provided in the first aspect, and therefore also has similar technical effects as the first aspect, which will not be described in detail here.

[0018] Fourthly, based on the same inventive concept, the present invention also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the pattern illumination imaging method as described in the above embodiments of the present invention.

[0019] The computer-readable storage medium provided in the fourth aspect has the same technical features as the pattern illumination imaging method provided in the first aspect, and therefore also has similar technical effects as the first aspect, which will not be described in detail here.

[0020] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0021] The above and additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart of a pattern illumination imaging method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a pattern illumination imaging apparatus according to an embodiment of the present invention; Figure 3 This is a low-resolution original image of pattern illumination with noise perturbation according to a specific embodiment of the present invention; Figure 4 This is a first-iteration reconstruction result diagram of PiFP according to a specific embodiment of the present invention; Figure 5 This is a first-iteration reconstruction result image of a pattern illumination imaging method according to a specific embodiment of the present invention; Figure 6 This is a first low-resolution original image of pattern illumination with positional perturbation according to a specific embodiment of the present invention; Figure 7This is a diagram showing the second iteration reconstruction result of PiFP according to a specific embodiment of the present invention; Figure 8 This is a second iterative reconstruction result image of a pattern illumination imaging method according to a specific embodiment of the present invention; Figure 9 This is a second low-resolution original image of pattern illumination with positional perturbation according to a specific embodiment of the present invention; Figure 10 This is a diagram showing the third iteration reconstruction result of PiFP according to a specific embodiment of the present invention; Figure 11 This is a reconstruction result of the third iteration of the pattern illumination imaging method according to a specific embodiment of the present invention; Figure 12 This is a structural block diagram of a pattern illumination imaging system according to an embodiment of the present invention; Figure 13 This is a structural block diagram of an electronic device according to an embodiment of the present invention.

[0022] Figure label: 100 - Mode illumination imaging system; 110 - First building block; 120 - Second building block; 130 - Correction module; 140 - Third building block; 150 - Update module; 160 - Output module. Detailed Implementation

[0023] The embodiments of the present invention are described in detail below, and the embodiments described with reference to the accompanying drawings are exemplary.

[0024] Currently, the theoretical model of pattern illumination Fourier stacked imaging technology is based on the assumption of a relatively ideal linear time-invariant system, which assumes that the illumination pattern position is precisely known and free from positional errors, the light intensity is stable and without fluctuations, and the system is drift-free. However, in actual experimental environments, its forward imaging model often suffers from severe mismatch due to the following reasons: 1. Illumination Pattern Error: In real-world systems, the illumination pattern is unknown and is prone to tilting, rotation, and dynamic changes. Furthermore, the strong coupling between the illumination pattern and the target makes non-convex optimization of the illumination pattern more challenging, and blind inversion becomes extremely difficult.

[0025] 2. Uneven illumination and intensity fluctuations: Since the intensity of the lighting source is not constant, there are spatial non-uniformities and temporal fluctuations. This time-varying effect further undermines the linear assumption of the model.

[0026] 3. Position error: Spatial misalignment or high-dimensional non-rigid errors may occur during the illumination pattern scanning process, such as wavefront distortion and illumination pattern deformation. These errors are difficult to correct with simple translation or rotation.

[0027] 4. System noise interference: During the imaging process, there are multiple noise sources such as shot noise and detector noise, which further reduce the quality of the acquired raw images.

[0028] Under the aforementioned multiple model mismatch conditions, pattern illumination Fourier layered imaging faces the following prominent problems: First, due to the unknown nature of the illumination pattern and positional errors, the resolution of the reconstructed illumination pattern and target drops sharply, significantly lower than that of the theoretical imaging model, making it difficult to distinguish high-frequency details of the target. Second, traditional pixel-domain-based iterative reconstruction methods cannot effectively cope with various systematic disturbances and noise interference, resulting in a decrease in the signal-to-noise ratio and contrast of the reconstructed image, and a significant decline in image quality. Finally, its imaging applicability is limited; pattern illumination Fourier layered imaging technology is difficult to apply to complex real-world scenarios such as industrial inspection and long-range target detection.

[0029] Therefore, how to address the scanning position error correction problem while improving imaging quality under multiple mismatch conditions such as unknown illumination mode, position error interference, uneven illumination, and system noise is a core issue that urgently needs to be solved.

[0030] To address the aforementioned problems, this invention provides a pattern illumination imaging method, system, electronic device, and storage medium. By embedding the scan position error as an optimizable variable into the feature domain iterative reconstruction process, a unified framework is constructed. This simultaneously corrects for scan position errors and various system errors (including illumination pattern variations, light intensity fluctuations, and mixed noise), achieving joint modeling and collaborative optimization of position errors and imaging mismatch factors. This significantly improves the imaging resolution and quality of pattern illumination Fourier stacks, expanding their applicability and robustness in complex system environments. (See below for reference.) Figures 1-13 A modal illumination imaging method, system, electronic device, and storage medium according to embodiments of the present invention are described.

[0031] Figure 1 This is a flowchart of a pattern illumination imaging method according to an embodiment of the present invention. Figure 1 As shown, a pattern illumination imaging method according to an embodiment of the present invention includes the following steps: Step S1: Construct a forward imaging model for pattern illumination Fourier stacked imaging that includes scan position error, illumination mode, optical transfer function, and composite noise.

[0032] In a specific embodiment, the forward imaging model is an expression that uses mathematical formulas to describe the complete physical imaging process of "target object → illumination mode → scanning position → optical system → noise → final image acquisition".

[0033] Specifically, the embodiments of the present invention construct a forward imaging model for pattern illumination Fourier stacked imaging that includes scanning position error, illumination mode, optical transfer function, and composite noise. This model models all position offsets, illumination distortions, system bandwidth limitations, and noise in the real system, making the forward model no longer idealistic and consistent with the actual physical process. It provides accurate mathematical constraints for subsequent iterations, avoiding reconstruction blur and artifacts due to model mismatch. At the same time, it achieves a unified description of error coupling, laying the foundation for subsequent synchronous correction of multiple errors.

[0034] Step S2: Based on the forward imaging model, initialize the complex amplitude wavefront to be optimized, and construct the target batch dataset from multiple acquired images; wherein, the complex amplitude wavefront to be optimized includes the target complex amplitude to be reconstructed and the complex amplitude of the mother illumination mode to be optimized.

[0035] In a specific embodiment, the complex amplitude wavefront to be optimized is a set of iterative initial variables that include the complex amplitude of the target to be reconstructed and the complex amplitude of the mother illumination mode to be optimized. It also carries amplitude and phase information to characterize the light field distribution. The complex amplitude wavefront to be optimized serves as the initial optimization object for iterative reconstruction. By continuously updating and approximating the real target and illumination mode, it provides a basis for super-resolution complex amplitude reconstruction.

[0036] In a specific embodiment, the target batch dataset, also known as the minibatch, is a small subset of data formed by randomly shuffling and grouping low-resolution intensity images acquired from multiple frames. The batch size of the minibatch dataset is an adjustable parameter. Minibatch datasets help reduce the computational cost per iteration and improve optimization stability, prevent overfitting, and accelerate convergence.

[0037] Specifically, the embodiments of the present invention optimize the target and illumination mode simultaneously to avoid the deviation caused by only reconstructing the target and ignoring the error of the illumination mode. In addition, the complex amplitude form includes amplitude and phase, which conforms to the physical laws of light field propagation and ensures the feasibility of super-resolution reconstruction. By shuffling the groups in small batches, the amount of computation per cycle can be reduced, which helps to improve the stability of iteration, prevent overfitting and accelerate convergence.

[0038] Step S3: Based on the target batch dataset, frequency domain correlation subpixel displacement calculation is used to correct the scanning position error and obtain a position-corrected image sequence.

[0039] In a specific embodiment, frequency-domain correlated subpixel displacement is obtained by performing phase correlation / cross-power spectrum calculation on two images in the frequency domain to obtain a translation offset with an accuracy of less than 1 pixel. It is based on the Fourier translation property to perform spatial image translation, which generates a linear phase difference in the corresponding frequency domain. By calculating the normalized cross-power spectrum, the phase difference is extracted and the peak value is fitted to obtain the subpixel-level displacement.

[0040] Specifically, the embodiments of the present invention utilize the frequency domain cross-correlation to enhance noise immunity compared to the spatial domain, enabling precise extraction of displacement at low signal-to-noise ratios and achieving sub-pixel-level position correction, which is far superior to the traditional pixel-level matching accuracy. This eliminates spectral aliasing and edge ghosting caused by scanning displacement errors, providing a strictly aligned image sequence for subsequent feature domain reconstruction, thereby ensuring that high-frequency information can be effectively superimposed.

[0041] Step S4: Based on the position-corrected image sequence, construct a feature domain loss function based on logarithmic focal frequency loss.

[0042] Specifically, the embodiments of the present invention construct a feature domain loss function based on logarithmic focal frequency loss, which shifts from the pixel domain to the logarithmic frequency domain, focuses on high-frequency details (such as edges and textures), suppresses the influence of strong and weak light fluctuations through logarithmic compression, is robust to uneven illumination, avoids the defects of pixel domain loss averaging error and dilution of high-frequency gradients, strengthens high-frequency weights, and makes the reconstruction resolution higher, the details clearer, and the artifacts less.

[0043] Step S5: Using the scanning position error as an optimizable variable, and combining the forward imaging model, the feature domain loss function, and the complex amplitude wavefront to be optimized, iteratively update the complex amplitude of the target to be reconstructed and the complex amplitude of the mother illumination mode to be optimized.

[0044] Specifically, in this embodiment of the invention, the position error is no longer a fixed prior, but is continuously optimized with iteration and dynamically adapted to system drift, thereby achieving simultaneous joint correction of position error, illumination change, light intensity fluctuation and noise without separate processing. Furthermore, the target and illumination mode mutually constrain each other and converge together, which is more accurate than single target reconstruction.

[0045] Step S6: After iterative convergence, the iteratively updated complex amplitude of the target to be reconstructed is output as a high-resolution target image, and the iteratively updated complex amplitude of the mother illumination mode to be optimized is output as a mother illumination mode image.

[0046] Specifically, the image output by the embodiments of the present invention after iterative convergence has the advantages of high signal-to-noise ratio, no obvious artifacts, and resolution close to the theoretical limit. Thus, even without precise position priors and multiple system errors, it still outputs super-resolution reconstruction results and synchronously outputs illumination modes, which can be used for system calibration, error analysis, and subsequent imaging acceleration.

[0047] Therefore, according to the pattern illumination imaging method of the present invention, firstly, a forward imaging model of pattern illumination Fourier stacked imaging, including scan position error, illumination mode, optical transfer function, and composite noise, is constructed. Secondly, based on the forward imaging model, the complex amplitude wavefront to be optimized is initialized, and multiple frames of acquired images are constructed into a target batch dataset. Then, based on the target batch dataset, the scan position error is corrected using frequency domain correlation subpixel displacement calculation to obtain a position-corrected image sequence. Then, based on the position-corrected image sequence, a feature domain loss function based on logarithmic focal frequency loss is constructed. Finally, the scan position error is used as an optimizable variable, and combined with the forward imaging model, the feature domain loss function, and the complex amplitude wavefront to be optimized, the method iteratively improves the image structure. The complex amplitude of the target to be reconstructed and the complex amplitude of the mother illumination mode to be optimized are combined. After the final iteration converges, the iteratively updated complex amplitude of the target to be reconstructed is output as a high-resolution target image, and the iteratively updated complex amplitude of the mother illumination mode to be optimized is output as a mother illumination mode image. In this way, a unified model including position error, illumination, noise and optical transfer function is established to truly fit the actual system. High-precision position alignment is achieved through frequency domain subpixel correction, high frequency is enhanced and interference is suppressed through logarithmic focal frequency loss, and multiple error collaborative correction is achieved by embedding position as an optimizable variable in the iteration. This ensures stable output of high-resolution, low-artifact, and high-precision reconstructed images in complex environments, and has good imaging stability and practicality.

[0048] In other words, the pattern illumination imaging method of this invention constructs a forward imaging model that includes scanning position error, illumination mode, optical transfer function, and composite noise. This model incorporates position offset, illumination distortion, system bandwidth limitations, and noise interference in the actual imaging process into a unified mathematical description, providing precise physical constraints for subsequent iterative optimization and effectively avoiding reconstruction blurring and artifacts caused by model mismatch with the actual scene. Based on this model, a wavefront containing the complex amplitude of the target to be reconstructed and the complex amplitude of the mother illumination mode to be optimized is initialized, and a target batch dataset is constructed. This achieves bivariate synchronous optimization of the target and illumination mode, avoiding the result deviation caused by only reconstructing the target while ignoring the illumination mode error. At the same time, the target batch grouping process reduces the computational load of a single iteration, improves the stability of the optimization process, prevents overfitting, and accelerates the convergence speed. Based on the target batch dataset, the position error is corrected by calculating subpixel displacement using frequency domain correlation. With the strong anti-noise capability of frequency domain cross-correlation, subpixel-level displacement is accurately extracted under low signal-to-noise ratio conditions, eliminating spectral aliasing and edge ghosting caused by scanning displacement error. This provides strictly aligned high-quality image data for subsequent feature domain reconstruction, ensuring high-frequency signal-to-noise ratio. Effective information superposition; a logarithmic focal frequency loss function is constructed based on the position-corrected image sequence, transforming the constraints from the pixel domain to the logarithmic frequency domain, focusing on high-frequency details such as image edges and textures. The influence of illumination intensity fluctuations is suppressed through logarithmic compression, avoiding the defects of pixel domain loss averaging error and dilution of high-frequency gradients, strengthening the weight of high-frequency components, resulting in higher resolution, clearer details, and fewer artifacts in the reconstructed image; the scanning position error is embedded as an optimizable variable into the iterative process, allowing the position error to be continuously optimized during the iteration process, dynamically adapting to system drift, and realizing synchronous joint correction of position error, illumination change, light intensity fluctuation, and mixed noise. The target and illumination mode are mutually constrained and jointly converged, without relying on manual annotation and precise position priors throughout the process, significantly improving the degree of automation and ease of engineering use; finally, under the condition of no precise position priors and multiple system error interference, a high-resolution target image and a parent illumination mode image are stably output. The synchronously obtained parent illumination mode image can be used for system calibration, error analysis, and subsequent imaging acceleration. The output image has a high signal-to-noise ratio, few artifacts, and a resolution close to the theoretical limit of the system, greatly improving the resolution, signal-to-noise ratio, and scene applicability of pattern illumination imaging in complex environments.

[0049] In one embodiment of the present invention, step S1 constructs a forward imaging model of pattern illumination Fourier stacked imaging, which includes scanning position error, illumination mode, optical transfer function and composite noise. This includes: scanning the illumination target with an unknown illumination mode along a fixed spiral path, establishing the optical field coupling relationship between the target, illumination mode, scanning position error and optical transfer function; and superimposing image domain composite noise on the optical field coupling relationship to form a forward imaging expression that includes scanning position error, illumination mode, optical transfer function and composite noise.

[0050] Figure 2 This is a schematic diagram of a pattern illumination imaging apparatus according to an embodiment of the present invention. Figure 2 As shown, in a specific embodiment, this pattern illumination imaging device moves and scans the unknown illumination pattern on the target surface according to a known fixed spiral sequence path, acquiring low-resolution target images under a series of illumination patterns. This process can be described as follows: In the formula, This is the low-resolution intensity image captured in the nth frame. Let r be the complex amplitude of the target to be reconstructed at spatial location r. Let n be the illumination pattern image at a known fixed position, where , The actual scan position of the illumination mode in the nth frame. Given the location, For unknown position errors in the system, OTF stands for Optical Transfer Function. The system's image domain is subject to composite noise interference. and These represent the Fourier transform and the inverse Fourier transform, respectively.

[0051] To simplify the representation, let The above process can be written as: In the formula, For positive imaging operators, characterizing the complex amplitude from the target and actual scan location The process of generating noise-free, low-resolution images.

[0052] In the high-resolution reconstruction of complex amplitude images, it typically relies on gradient descent updates of the L2 (also called LSE (Least Squared Error), usually equivalent to MSE (Mean Square Error)) loss in the image domain between the forward imaging model's predicted image and the actual measured intensity image, gradually converging to the optimal complex amplitude solution, thereby achieving high-resolution reconstruction of the target. This loss function can be expressed as: In the formula, Let L2 be the pixel-domain loss function for the target complex amplitude O, used to measure the difference between the predicted and measured images. The square root of the measured image corresponds to the amplitude of the measured complex amplitude. To predict the amplitude of the complex amplitude, The mean square error between the predicted amplitude and the measured amplitude is calculated using the L2 norm squared, which is the L2 loss in the image domain.

[0053] Specifically, this embodiment of the invention establishes an optical field coupling relationship between the target, illumination mode, scanning position error, and optical transfer function by scanning the illuminated target along a fixed spiral path with an unknown illumination mode. The spiral scanning path achieves uniform, blind-spot-free coverage of the target area, resulting in higher sampling efficiency and facilitating the stitching of frequency domain information and the complete recovery of high-frequency components. The optical field coupling relationship accurately describes the physical interactions between the target, illumination, position, and system, ensuring the physical reliability of the reconstruction process. On this basis, image domain composite noise is superimposed, making the forward imaging model more consistent with the interference of shot noise, readout noise, vignetting, etc. in the actual imaging process, providing more realistic constraints for subsequent iterative optimization, effectively improving the algorithm's anti-interference ability in noisy environments, and making the model more practical for engineering applications.

[0054] In one embodiment of the present invention, step S2 initializes the complex amplitude wavefront to be optimized according to the forward imaging model and constructs a target batch dataset from multiple frames of acquired images, including: based on the forward imaging model, initializing the target complex amplitude to be reconstructed that matches the target size and the mother illumination mode complex amplitude to be optimized that matches the illumination mode size, which together constitute the complex amplitude wavefront to be optimized as the initial variable for iterative update; and randomly shuffling and grouping all low-resolution intensity images acquired by illumination scanning to form the target batch dataset.

[0055] In a specific embodiment, wavefront functions to be optimized are created with the same size as the target size and the size of the illumination pattern. To ensure stability during the iteration process and enhance the robustness of the model, all frame data are randomly shuffled and minibatch is created, where batchsize is an adjustable parameter.

[0056] Specifically, this embodiment of the invention is based on a forward imaging model, which initializes the complex amplitude of the target to be reconstructed that matches the target size and the complex amplitude of the parent illumination pattern to be optimized that matches the size of the illumination pattern. This avoids problems such as gradient explosion and optimization collapse caused by dimensionality mismatch, and ensures the stability of the iteration process. All low-resolution intensity images acquired by illumination scanning are randomly shuffled and grouped to form a target batch dataset, which breaks the correlation between image frames, improves the generalization ability of the model, prevents overfitting, and reduces the memory usage and computational load of a single iteration, making the optimization process smoother and the convergence speed faster, and adapting to the processing needs of large-scale image sequences.

[0057] In one embodiment of the present invention, step S3 corrects the scanning position error based on the target batch dataset by using frequency domain correlation subpixel displacement calculation to obtain a position-corrected image sequence, including: using the central scanning point image of the target batch dataset as a reference image; calculating the frequency domain cross-correlation between each image frame and the reference image to obtain the subpixel displacement corresponding to the scanning position error; and performing frequency domain shift compensation based on the subpixel displacement to obtain the position-corrected image sequence.

[0058] In a specific embodiment, it is considered that the directly acquired images contain unknown positional error perturbations. The position error is corrected using a frequency domain cross-correlation subpixel displacement method: First, the image of the center scan point is used as the reference image. Calculate the frequency domain cross-correlation between all image frame data in the minibatch and the reference image: In the formula, Let be the cross-correlation function between the nth frame image and the reference image, reflecting the positional error perturbation term between the two. Similarity under This is the low-resolution intensity image captured in the nth frame. For reference image, The cross-correlation sign is represented by r, where r is the spatial coordinate. The relative displacement is obtained by expanding the above equation into a discrete summation form and maximizing the cross-correlation function. In the formula, To estimate the positional error perturbation term of the obtained nth frame image relative to the reference image, Represents taking the cross-correlation function Displacement reaching the maximum value The estimated location is used to perform frequency domain subpixel shift compensation alignment on the image and to construct a new reference image. : In the formula, The new reference image is constructed after position correction, where n is the image index and N is the batch size; the frequency domain correlation displacement calculation operation described above is performed again on the new reference image to obtain the final updated position. .

[0059] Specifically, in this embodiment of the invention, the center scan point image of the target batch data is used as the reference image. This position has the highest signal-to-noise ratio and the best imaging quality, providing a stable and reliable benchmark for position correction. By calculating the frequency domain cross-correlation between each image frame and the reference image, the sub-pixel displacement corresponding to the scanning position error is obtained. The frequency domain cross-correlation has strong anti-noise and anti-brightness variation capabilities and is more accurate than the spatial domain correlation. Frequency domain shift compensation is performed based on the sub-pixel displacement, achieving image alignment without interpolation loss and without edge blurring. This effectively eliminates spectral aliasing and edge ghosting caused by position errors, providing a strictly aligned high-quality image sequence for subsequent feature domain reconstruction, ensuring the effective superposition of high-frequency information and the accuracy of super-resolution reconstruction.

[0060] In one embodiment of the present invention, step S4 constructs a feature domain loss function based on logarithmic focal frequency loss based on the position-corrected image sequence, including: generating a position-corrected predicted image according to the forward imaging model and the complex amplitude wavefront to be optimized; performing Fourier transform on the predicted image and the position-corrected measured image respectively; calculating the logarithmic domain difference between the real and imaginary parts of the Fourier transform result, merging them into the L2 norm and setting the central DC component to zero; calculating the logarithmic attenuation error of the complex difference modulus, multiplying it by the frequency weight and averaging it in the frequency domain to obtain the logarithmic focal frequency loss function.

[0061] In a specific embodiment, in order to better extract image feature information and reduce interference such as mixed noise in the image domain and light intensity fluctuations, the above-mentioned forward model is transformed from the image domain to the logarithmic frequency domain, and the predicted image after position correction is first processed. and actual measurement images Perform a Fourier transform, and then calculate the log-space difference between the real and imaginary parts: In the formula, To predict the logarithmic difference between the real part of the frequency domain of the image and the measured image, To predict the logarithmic difference between the imaginary part of the image and the measured image in the frequency domain, The Fourier transform result of the predicted image after position correction. The Fourier transform result of the measured image. To take the real part of the complex number, To take the imaginary part of a complex number, For logarithmic operations, is a constant and The logarithmic difference between the real and imaginary parts is combined into the L2 norm, and the relative weight center is set to zero to eliminate the influence of the DC component on the loss. This allows the loss function to focus more on high-frequency details, resulting in the relative weights: In the formula, The relative weights are calculated from the logarithmic difference between the real and imaginary parts in the frequency domain; the magnitude of the complex difference is calculated, and its logarithmic attenuation error is calculated: In the formula, To calculate the logarithmic attenuation error of the frequency domain difference between the predicted image and the measured image, the weights are multiplied element-wise by the logarithmic attenuation error, and then averaged in the frequency plane. In the formula, The final constructed logarithmic frequency domain L2 norm loss function, is a weight scaling factor used to control the emphasis on relatively high frequencies, and P and Q are the height and width dimensions of the image; then the L2 norm loss function between the predicted image and the measured image in the spatial domain is transformed into the logarithmic frequency domain L2 norm loss function for both the measured image and the predicted image.

[0062] Specifically, this embodiment of the invention generates a predicted image after position correction based on a forward imaging model and the complex amplitude wavefront to be optimized. Fourier transforms are performed on both the predicted image and the position-corrected measured image to convert the constraints to the frequency domain, directly constraining spectral fidelity. The logarithmic difference between the real and imaginary parts of the Fourier transform result is calculated and merged into the L2 norm. Simultaneously, the central DC component is set to zero. Logarithmic compression suppresses the influence of dynamic range of light intensity and uneven illumination, eliminating the interference of average brightness on the loss function, making the loss more focused on high-frequency information such as structure, texture, and edges. The logarithmic attenuation error of the complex difference modulus is calculated and multiplied with the frequency weights, then averaged in the frequency domain to construct the logarithmic focal frequency loss function. This strengthens the weight of high-frequency components and weakens low-frequency components, resulting in sharper edges, clearer details, and fewer artifacts in the reconstructed image, significantly improving imaging resolution and robustness to illumination fluctuations and noise.

[0063] In one embodiment of the present invention, step S5 uses the scanning position error as an optimizable variable, and combines the forward imaging model, the feature domain loss function, and the complex amplitude wavefront to be optimized to iteratively update the complex amplitude of the target to be reconstructed and the complex amplitude of the parent illumination mode to be optimized. This includes: using the complex amplitude wavefront to be optimized as the initial variable for iteration, calculating the gradients of the complex amplitude of the target to be reconstructed and the complex amplitude of the parent illumination mode to be optimized based on the forward imaging model and the logarithmic focal frequency loss function; converting the gradients into complex amplitude gradients, and propagating them backward to the object plane via the optical transfer function; relocating the gradients of each illumination mode to the parent illumination mode according to the scanning position; and using a root mean square propagation optimizer to synchronously iteratively update the complex amplitude of the target to be reconstructed and the complex amplitude of the parent illumination mode to be optimized.

[0064] In a specific embodiment, the loss function differentiated with respect to the frequency domain is: In the formula, Let the logarithmic frequency domain loss function of the current image be denoted as: Then its spatial domain gradient is expressed as: In the formula, Let the gradient of the loss function with respect to the complex field of the image surface be the gradient. For the complex amplitude of the image plane, The inverse Fourier transform converts the frequency domain gradient back to the spatial domain; the gradient is then converted to a complex amplitude gradient, backpropagated in the frequency domain, multiplied by the OTF, and backpropagated back to the object plane. The resulting gradient of the target and the corresponding illumination mode gradient are expressed as follows: , In the formula, The loss function is the gradient of the complex amplitude O of the target to be reconstructed. For the lighting mode of the nth frame The complex conjugate, For the complex conjugate of the optical transfer function (OTF), To express [the opinion / towards] Fourier transform, The loss function is the complex amplitude of the lighting mode in the nth frame. gradient, The complex conjugate of the complex amplitude O of the target to be reconstructed.

[0065] After shifting the gradient of each frame's illumination mode back to its corresponding position relative to the parent illumination mode, the average gradient is calculated by calculating the gradients of all images in a mini-batch. Then, gradient descent learning is performed on the target and parent illumination mode using root mean square propagation. The above operations are iteratively executed until the algorithm reaches convergence, and finally, the updated high-resolution target image and parent illumination mode image are obtained.

[0066] Specifically, this embodiment of the invention uses the complex amplitude wavefront to be optimized as the initial variable for iteration. Based on the forward imaging model and the logarithmic focal frequency loss function, the gradients of the complex amplitude of the target to be reconstructed and the complex amplitude of the parent illumination mode to be optimized are calculated, while the amplitude and phase are updated simultaneously, conforming to the physical laws of light wave propagation. The gradient is converted into a complex amplitude gradient and propagated back to the object plane via the optical transfer function, strictly following the physical constraints of the imaging system, making the gradient calculation more accurate and the reconstruction process more stable. The gradients of each illumination mode are returned to the parent illumination mode according to the scanning position, maintaining the spatial structural integrity of the parent illumination mode and avoiding distortion and confusion of the illumination mode. A root mean square propagation optimizer is used to synchronously iterate and update the complex amplitude of the target to be reconstructed and the complex amplitude of the parent illumination mode to be optimized. This optimizer has an adaptive learning rate, fast convergence speed, insensitivity to initial values, and stable iteration that is not prone to oscillation and divergence, ensuring the joint convergence of the target and the illumination mode and improving the reconstruction accuracy and stability.

[0067] In one embodiment of the present invention, iteratively updating the complex amplitude of the target to be reconstructed and the complex amplitude of the mother illumination mode to be optimized includes: performing scan position error correction cyclically in each iteration, calculating the loss value of the feature domain loss function based on the correction result, and updating the complex amplitude based on the correction result and the loss value, so as to achieve synchronous joint correction of scan position error, illumination change, light intensity fluctuation and mixed noise.

[0068] Specifically, in each iteration, the embodiments of the present invention iteratively perform scanning position error correction and feature domain loss function construction, so that the position error is continuously refined with the iteration process, which can dynamically adapt to errors caused by system drift, environmental changes and long-term operation, and continuously improve the position correction accuracy. At the same time, the feature domain loss function constrains high-frequency details in each iteration, strengthens high-frequency information, suppresses noise interference, and further improves reconstruction quality. In this way, the scanning position error, illumination change, light intensity fluctuation and mixed noise are synchronously and jointly corrected. Multiple errors are not isolated, do not drag each other, and do not accumulate step by step, realizing the global optimal solution. Even under the conditions of long-term operation, scene change and illumination change, it can still maintain high resolution, high stability and low artifact reconstruction effect, which greatly improves the robustness and engineering practicality of the algorithm.

[0069] The advantages and benefits of the pattern illumination imaging method of the present invention described above will be explained below with reference to a specific embodiment. In this specific embodiment, Figures 3-11 This is a schematic diagram comparing the simulation reconstruction results of the pattern illumination imaging method according to a specific embodiment of the present invention with those of the conventional method in terms of noise resistance and position error interference. Figures 3-11 This paper provides a simulation and experimental comparison of the pattern illumination imaging method of the above embodiments of the present invention and the conventional method (PiFP (Pattern-illuminated Fourier ptychography)).

[0070] Specifically, Figure 3 This is a low-resolution original image of pattern illumination with noise perturbation according to a specific embodiment of the present invention. Figure 4 This is a first-iteration reconstruction result image of PiFP according to a specific embodiment of the present invention. Figure 5 This is a first-iteration reconstruction result image of a pattern illumination imaging method according to a specific embodiment of the present invention. Figures 3-5 As shown, Figure 3 The image has a low signal-to-noise ratio, blurred details, and high-frequency information is overwhelmed by noise. Figure 4 The images exhibit significant noise amplification, artifact retention, blurred edges, and a significant decrease in resolution under noise interference, failing to effectively suppress the impact of noise on the reconstruction process; Figure 5 The image can effectively suppress noise interference under the same noise conditions, with clear image edges, complete texture, significantly reduced artifacts, significantly improved signal-to-noise ratio and contrast, and complete restoration of high-frequency details.

[0071] Figure 6 This is a first low-resolution original image of pattern illumination with positional perturbation according to a specific embodiment of the present invention. Figure 7This is a second iteration reconstruction result diagram of PiFP according to a specific embodiment of the present invention. Figure 8 This is a second-iteration reconstruction result image of a pattern illumination imaging method according to a specific embodiment of the present invention. Figures 6-8 As shown, Figure 6 The image shows a clear scan position offset; Figure 7 The images are affected by positional errors, resulting in severe spectral aliasing, structural misalignment, artifacts, and ghosting, leading to a sharp decrease in resolution. Figure 8 The image is optimized by frequency domain correlation subpixel displacement correction and position joint optimization, which can accurately compensate for position errors, eliminate spectral aliasing and structural misalignment, and reconstruct the image with complete structure, sharp edges and no obvious artifacts. The super-resolution reconstruction effect is stable and reliable.

[0072] from Figures 6-8 As can be seen, the pattern illumination imaging method provided by this invention significantly improves the robustness to noise, illumination fluctuations and other interferences by constraining the feature domain of logarithmic focal frequency loss, avoiding the defects of traditional pixel domain loss that is sensitive to noise and has diluted high-frequency gradients. At the same time, the scanning position is embedded as an optimizable variable into the iterative process to achieve dynamic correction of position error, solving the problem of reconstruction failure under position mismatch in traditional methods. It can still maintain high-quality super-resolution reconstruction under the coupled interference of noise and position error, and its robustness is far superior to that of traditional methods.

[0073] Figure 9 This is a second low-resolution original image of pattern illumination with positional perturbation according to a specific embodiment of the present invention. Figure 10 This is a diagram showing the third iteration reconstruction result of PiFP according to a specific embodiment of the present invention. Figure 11 This is a reconstruction result image of the third iteration of the pattern illumination imaging method according to a specific embodiment of the present invention. Figures 9-11 As shown, Figure 9 The image has a low signal-to-noise ratio, blurred details, and high-frequency information is overwhelmed by noise. Figure 10 The image is affected by multiple coupling errors such as actual system position error, uneven illumination, and noise, resulting in blurred image, severe artifacts, loss of detail, and low resolution. Figure 11 Even under real and complex system error conditions, the image can still effectively correct position errors and suppress interference such as illumination fluctuations and noise. The reconstructed image has clear details, sharp edges, very few artifacts, and its resolution and imaging quality are significantly better than traditional methods.

[0074] from Figures 9-11It can be seen that the pattern illumination imaging method provided by this invention has strong practicality and engineering robustness in its framework of joint modeling and collaborative correction of position error and multiple system errors. It can work stably under non-ideal and complex real-world imaging conditions, breaking through the strong dependence of traditional methods on precise position calibration, ideal illumination, and low-noise environment, and greatly improving the applicability and reliability of pattern illumination Fourier stacked imaging in real-world scenarios.

[0075] In other words, the pattern illumination imaging method provided by this invention has the following advantages compared with the prior art: (1) Advantages of the imaging model: Some existing pattern illumination imaging schemes only correct for position errors, and the actual correction effect is very limited. At the same time, they do not model and correct other system errors that are common in the imaging system, such as uneven illumination, light source intensity fluctuation, vignetting effect and mixed noise. This method proposes a feature domain pattern illumination super-resolution imaging model that combines position error correction and logarithmic frequency loss. It embeds position error correction into the feature domain update process, which can correct various composite interference factors, including position error and noise interference.

[0076] (2) Advantages of feature extraction: Existing feature domain models usually use wavelet transform, gradient features, etc. as feature extraction operators. However, these operators usually focus on global image features and do not pay more attention to image details, thus exhibiting spectral bias, that is, inherently learning low-frequency components and making it difficult to capture high-frequency details. Spatial loss cannot offset this limitation because they uniformly average errors between pixels, thus diluting the gradient of high-frequency features. This method embeds logarithmic focal frequency loss into the feature domain model. This loss function can focus more on the high-frequency components of target edges and fine textures, thus making the resolution of the reconstructed image more advantageous than other methods.

[0077] In summary, the pattern illumination imaging method provided by this invention combines frequency domain correlation subpixel position correction with feature domain iteration of logarithmic focal frequency loss, incorporating position error as an optimizable variable into the reconstruction process. This achieves synchronous and coordinated correction of position error, illumination variation, light intensity fluctuation, and mixed noise. Under the conditions of noise interference and position error, compared with the traditional PiFP method, it has higher reconstruction resolution, fewer artifacts, better signal-to-noise ratio and contrast, and stronger robustness. It can effectively solve the problem of the sharp decline in reconstruction quality under the coupling of multiple system errors in traditional methods, and significantly expand the application range of pattern illumination Fourier stacked imaging technology in complex environments.

[0078] Further embodiments of the present invention disclose a pattern illumination imaging system. Figure 12 This is a structural block diagram of a pattern illumination imaging system according to an embodiment of the present invention. Figure 12As shown, in one embodiment of the present invention, the mode illumination imaging system 100 includes: a first building module 110, a second building module 120, a correction module 130, a third building module 140, an update module 150, and an output module 160.

[0079] Specifically, the first building module 110 is used to build a forward imaging model for pattern illumination Fourier stacked imaging, which includes scanning position error, illumination mode, optical transfer function and compound noise.

[0080] The second construction module 120 is used to initialize the complex amplitude wavefront to be optimized according to the forward imaging model, and to construct a target batch dataset from multiple frames of acquired images; wherein, the complex amplitude wavefront to be optimized includes the target complex amplitude to be reconstructed and the complex amplitude of the mother illumination mode to be optimized.

[0081] The correction module 130 is used to correct the scanning position error based on the target batch dataset by using frequency domain correlation subpixel displacement calculation to obtain a position-corrected image sequence.

[0082] The third building module 140 is used to construct a feature domain loss function based on logarithmic focal frequency loss based on the position-corrected image sequence.

[0083] The update module 150 is used to take the scanning position error as an optimizable variable, and combine it with the forward imaging model, the feature domain loss function and the complex amplitude wavefront to be optimized to iteratively update the complex amplitude of the target to be reconstructed and the complex amplitude of the mother illumination mode to be optimized.

[0084] The output module 160 is used to output the iteratively updated complex amplitude of the target to be reconstructed as a high-resolution target image after iterative convergence, and to output the iteratively updated complex amplitude of the mother illumination mode to be optimized as a mother illumination mode image.

[0085] It should be noted that the specific implementation of the pattern illumination imaging system 100 in this embodiment of the invention is similar to the specific implementation of the pattern illumination imaging method described in the above embodiment of the invention, and therefore has similar technical effects. For details, please refer to the description of the pattern illumination imaging method section. To reduce redundancy, it will not be repeated here.

[0086] Further embodiments of the present invention also disclose an electronic device, Figure 13 This is a structural block diagram of an electronic device according to an embodiment of the present invention. Figure 13 As shown, in one embodiment of the present invention, the electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; when the processor executes the computer programs stored in the memory, it implements the pattern illumination imaging method as described in any of the above embodiments of the present invention.

[0087] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0088] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0089] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0090] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0091] The method provided in this invention can be applied to electronic devices. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server, etc. No limitation is made herein; any electronic device that can implement this invention falls within the protection scope of this invention.

[0092] It should be noted that the specific implementation of the electronic device in the embodiments of the present invention is similar to the specific implementation of the mode illumination imaging method described in the above embodiments of the present invention, and therefore has similar technical effects. For details, please refer to the description of the mode illumination imaging method section. In order to reduce redundancy, it will not be repeated here.

[0093] Further embodiments of the present invention disclose a computer-readable storage medium storing a computer program that, when executed by a processor, implements the pattern illumination imaging method as described in any of the above embodiments of the present invention.

[0094] It should be noted that the specific implementation of the computer-readable storage medium in the embodiments of the present invention is similar to the specific implementation described in the pattern illumination imaging method of the above embodiments of the present invention, and therefore has similar technical effects. For details, please refer to the description in the pattern illumination imaging method section. To reduce redundancy, it will not be repeated here.

[0095] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example.

[0096] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A pattern illumination imaging method, characterized in that, Includes the following steps: Construct a forward imaging model for pattern illumination Fourier stacked imaging that includes scan position error, illumination mode, optical transfer function, and compound noise; Based on the forward imaging model, the complex amplitude wavefront to be optimized is initialized, and multiple frames of acquired images are constructed into a target batch dataset; wherein, the complex amplitude wavefront to be optimized includes the target complex amplitude to be reconstructed and the complex amplitude of the mother illumination mode to be optimized; Based on the target batch dataset, frequency domain correlation subpixel displacement calculation is used to correct the scanning position error, and a position-corrected image sequence is obtained. Based on the position-corrected image sequence, a feature domain loss function based on logarithmic focus frequency loss is constructed. Using the scanning position error as an optimizable variable, and combining the forward imaging model, the feature domain loss function, and the complex amplitude wavefront to be optimized, the complex amplitude of the target to be reconstructed and the complex amplitude of the mother illumination mode to be optimized are iteratively updated. After iterative convergence, the iteratively updated complex amplitude of the target to be reconstructed is output as a high-resolution target image, and the iteratively updated complex amplitude of the mother illumination mode to be optimized is output as a mother illumination mode image.

2. The pattern illumination imaging method according to claim 1, characterized in that, The construction of the forward imaging model for pattern illumination Fourier stacked imaging, which includes scan position error, illumination mode, optical transfer function, and composite noise, includes: The unknown illumination mode is scanned along a fixed spiral path to establish the optical field coupling relationship between the target, illumination mode, scanning position error and optical transfer function. Based on the aforementioned optical field coupling relationship, image domain composite noise is superimposed to form a forward imaging expression that includes scanning position error, illumination mode, optical transfer function, and composite noise.

3. The pattern illumination imaging method according to claim 1, characterized in that, The step of initializing the complex amplitude wavefront to be optimized according to the forward imaging model and constructing a target batch dataset from multiple acquired images includes: Based on the forward imaging model, the complex amplitude of the target to be reconstructed, which matches the target size, and the complex amplitude of the parent illumination pattern to be optimized, which matches the size of the illumination pattern, are initialized respectively. Together, they constitute the wavefront of the complex amplitude to be optimized, which serves as the initial variable for iterative updates. All low-resolution intensity images acquired by illumination scanning are randomly shuffled and grouped to form the target batch dataset.

4. The pattern illumination imaging method according to claim 1, characterized in that, The step of correcting the scanning position error based on the target batch dataset using frequency domain correlation subpixel displacement calculation to obtain a position-corrected image sequence includes: The image of the central scan point of the target batch data is used as a reference image; Calculate the frequency domain cross-correlation between each image frame and the reference image to obtain the sub-pixel displacement corresponding to the scanning position error; Frequency domain shift compensation is performed based on the subpixel displacement to obtain a position-corrected image sequence.

5. The pattern illumination imaging method according to claim 1, characterized in that, The construction of a feature domain loss function based on logarithmic focal frequency loss based on the position-corrected image sequence includes: Based on the aforementioned forward imaging model and the complex amplitude wavefront to be optimized, a predicted image after position correction is generated; Perform Fourier transforms on the predicted image and the position-corrected measured image respectively; Calculate the logarithmic difference between the real and imaginary parts of the Fourier transform result, combine them into the L2 norm, and set the central DC component to zero; The logarithmic attenuation error of the complex difference modulus is calculated, multiplied by the frequency weight, and averaged in the frequency domain to obtain the logarithmic focal frequency loss function.

6. The pattern illumination imaging method according to claim 1, characterized in that, The step of using the scanning position error as an optimizable variable, combined with the forward imaging model, the feature domain loss function, and the wavefront of the complex amplitude to be optimized, to iteratively update the complex amplitude of the target to be reconstructed and the complex amplitude of the mother illumination mode to be optimized, includes: Using the complex amplitude wavefront to be optimized as the initial variable for iteration, the gradients of the complex amplitude of the target to be reconstructed and the complex amplitude of the mother illumination mode to be optimized are calculated based on the forward imaging model and the logarithmic focal frequency loss function. The gradient is converted into a complex amplitude gradient and propagated backward to the object plane via the optical transfer function; Return the gradients of each illumination mode to the parent illumination mode according to the scan position; The root mean square propagation optimizer is used to synchronously iterate and update the complex amplitude of the target to be reconstructed and the complex amplitude of the mother illumination mode to be optimized.

7. The pattern illumination imaging method according to claim 1, characterized in that, The iterative update of the complex amplitude of the target to be reconstructed and the complex amplitude of the mother illumination mode to be optimized includes: Each iteration performs scan position error correction cyclically, calculates the loss value of the feature domain loss function based on the correction result, and updates the complex amplitude based on the correction result and the loss value, so as to achieve synchronous joint correction of scan position error, illumination change, light intensity fluctuation and mixed noise.

8. A pattern illumination imaging system, characterized in that, include: The first building module is used to build a forward imaging model of pattern illumination Fourier stacked imaging, which includes scan position error, illumination mode, optical transfer function and compound noise. The second construction module is used to initialize the complex amplitude wavefront to be optimized according to the forward imaging model, and construct a target batch dataset from multiple acquired images; wherein, the complex amplitude wavefront to be optimized includes the target complex amplitude to be reconstructed and the complex amplitude of the mother illumination mode to be optimized. The correction module is used to correct the scanning position error based on the target batch dataset by using frequency domain correlation subpixel displacement calculation to obtain a position-corrected image sequence. The third construction module is used to construct a feature domain loss function based on logarithmic focus frequency loss based on the position-corrected image sequence. The update module is used to take the scanning position error as an optimizable variable, and combine it with the forward imaging model, the feature domain loss function and the complex amplitude wavefront to be optimized to iteratively update the complex amplitude of the target to be reconstructed and the complex amplitude of the mother illumination mode to be optimized. The output module is used to output the iteratively updated complex amplitude of the target to be reconstructed as a high-resolution target image after iterative convergence, and to output the iteratively updated complex amplitude of the mother illumination mode to be optimized as a mother illumination mode image.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the steps of the pattern illumination imaging method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the pattern illumination imaging method according to any one of claims 1-7.

Images