Fourier ptychographic position mismatch calibration method under point-by-point convergence strategy

Abstract

The application provides a Fourier superposition imaging position mismatch calibration method under a point-by-point convergence strategy, and specifically comprises the following steps: S1, performing high-resolution image reconstruction according to a collected low-resolution image; S2, searching for a mismatch error and performing calibration by using a point-by-point convergence strategy; and S3, repeating S2 until all low-resolution images are calibrated, and outputting a high-resolution reconstruction spectrum. The application directly calibrates the translation error of each scanning position of a camera in a frequency domain, calibrates the picture mismatch by using a point-by-point convergence strategy, optimizes the real position of a spectrum sub-aperture by using a particle swarm algorithm, and introduces a dynamic inertia weight factor, so that the particle can be prevented from falling into a local optimum condition in an early particle evolution stage, and accurate search can be realized in a late particle evolution stage. The application can greatly improve the camera position mismatch calibration precision and speed, calibrates a long-distance imaging system after mismatch occurs, and improves the final high-resolution image reconstruction quality.

Description

Technical Field

[0001] This invention relates to the field of computational optical imaging technology, and specifically provides a method for calibrating the position mismatch of Fourier stacked imaging under a point-by-point convergence strategy. Background Technology

[0002] In optics, the spatial bandwidth product (SBP) describes the information transmission capability of an imaging system; a larger SBP means more detailed information can be resolved. However, due to diffraction and aberrations, the spatial bandwidth product of ordinary optical imaging systems is limited to the order of millions, which is insufficient to meet the needs of modern biomedical applications and other fields. Currently, methods to improve system resolution typically include synthetic aperture technology, confocal scanning technology, and light field imaging technology. However, all of these technologies suffer from high costs and limitations in field of view and resolution.

[0003] Fourier layered imaging (FP) technology, when used in long-distance applications, replaces the illumination source with lasers. By moving the camera multiple times to acquire different spatial frequency information of the object, phase retrieval technology is used to reconstruct high-resolution images, breaking through the diffraction limit of optical systems and achieving large field of view and high-resolution imaging effects.

[0004] However, macroscopic Fourier layered imaging systems have the following problems:

[0005] In macroscopic Fourier layered imaging systems, the imaging camera needs to capture a series of low-resolution images through scanning. Due to limitations of existing technology and equipment, the actual positions of different cameras are offset from the ideal positions of the imaging system. When camera mismatch occurs, artifacts will appear in the final reconstructed image, the reconstruction quality will be reduced, and the best reconstruction result cannot be achieved. Summary of the Invention

[0006] This invention is based on a particle swarm optimization algorithm for position mismatch calibration. It employs a point-by-point convergence strategy to optimize the translation mismatch parameters in the frequency domain. It also introduces dynamic inertia weights, which enable the method to have strong global search capabilities in the early stages of particle evolution and strong local optimization capabilities in the later stages of particle evolution. After optimization, high-resolution image reconstruction is achieved through phase retrieval.

[0007] This invention addresses the camera position mismatch problem in Fourier stacked imaging systems by providing a Fourier stacked imaging position mismatch calibration method based on a point-by-point convergence strategy, comprising the following steps:

[0008] S1. Reconstruct high-resolution images based on the S acquired low-resolution images, where S is a positive integer. The specific process is as follows:

[0009] S11. Based on the spectral information of the low-resolution image acquired by the camera, the complex amplitude distribution of the light field in the pupil plane of the camera's imaging lens is obtained as follows:

[0010]

[0011] Where ψ(μ,ν) represents the total spectrum, and exp() represents an exponential function with base e. The complex amplitude distribution of the optical field on the back surface of the sample is represented by λ, where λ represents the wavelength of the laser emitted light, z represents the distance between the sample and the imaging lens, j is the imaginary unit, and k is the wavenumber.

[0012] S12. Using the pupil function P(μ,ν) of the imaging lens, extract the spectral sub-region acquired by the system during a single imaging session from the total spectrum. as follows:

[0013]

[0014] Among them, C n This represents the position of the best particle in the population corresponding to the nth low-resolution image. Its subscript indicates the μ coordinate or ν coordinate, and the subscripts n and i indicate the i-th iteration of the nth low-resolution image.

[0015] S13. Keeping the phase of the spectral sub-region to be updated unchanged, perform a Fourier transform of the spectral sub-region to be updated to the spatial domain to obtain a low-resolution image estimate. Right now:

[0016]

[0017] Where F represents the Fourier transform;

[0018] S14. Replace the estimated amplitude value of the low-resolution image with the amplitude of the acquired low-resolution image, that is:

[0019]

[0020] Among them, I n This represents the intensity of the nth low-resolution image;

[0021] S15. Transform the low-resolution image obtained in S14 into a frequency domain image ψ. n ',Right now:

[0022]

[0023] S16. Using the frequency domain image ψ from S15 n Update the total spectrum target area, i.e.:

[0024] ψ n i+1 =ψn i +ψ n i -ψ n ';

[0025] S17. Repeat S12 to S16 until all low-resolution images have been updated.

[0026] S18. Repeat S12 to S18 until the set number of iterations is completed;

[0027] S2. The mismatch error is searched and calibrated using a point-by-point convergence strategy. The specific process is as follows:

[0028] S21. Randomly generate multiple particles near the center of the spectral sub-aperture of the camera. Each particle represents the initial spectral estimate of the center of the spectral sub-aperture corresponding to the low-resolution image.

[0029] S22. Calculate the fitness value p of each particle based on the evaluation function, and select the position C of the best particle in the population. best and its corresponding optimal fitness value p best ;

[0030] S23. Calculate the dynamic inertia weight of each particle as follows:

[0031]

[0032] Where w represents the dynamic inertia weight, which determines the proportion of the particle's learning of the current velocity each time; c1 = c2 = 2.05, where c1 and c2 represent the learning rate of the particle towards the current particle's historical best position and the global best particle, respectively.

[0033] The particle velocity v is updated according to the following formula:

[0034] v m,l (t+1)=w·(v m,l (t)+c1·r1(p m,l -x m,l )+c2·r2(C best -x m,l ));

[0035] Where t represents the number of iterations for mismatch calibration, l = {1,2} represents the coordinates (μ,ν) of the frequency domain sub-aperture, m represents the particle number, and r1 and r2 are uniform random numbers between [0,1].

[0036] The particle's position is updated according to the following formula:

[0037] x m,l (t+1)=x m,l (t)+v m,l(t+1);

[0038] S24. Repeat S22 to S23 until the set number of iterations is completed, and update the coordinates of the current frequency domain sub-aperture according to the final position of the particle.

[0039] S3. Repeat S2 until all low-resolution images are calibrated and output the high-resolution reconstructed spectrum.

[0040] Preferably, a phase retrieval algorithm is used to reconstruct low-resolution images affected by position mismatch errors and output them as high-quality, high-resolution images.

[0041] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0042] This invention directly calibrates the translation error of each scanning position of the camera in the frequency domain. It employs a point-by-point convergence strategy to calibrate image mismatch, optimizes the true position of the spectral sub-aperture using a particle swarm optimization algorithm, and introduces a dynamic inertia weighting factor. This avoids getting trapped in local optima in the early stages of particle evolution and enables precise searching in the later stages. This invention significantly improves the accuracy and speed of camera position mismatch calibration. By calibrating a long-distance imaging system after mismatch occurs, it enhances the quality of the final high-resolution image reconstruction. Attached Figure Description

[0043] Figure 1 This is a simplified structural diagram of a macroscopic Fourier layered imaging system provided according to an embodiment of the present invention;

[0044] Figure 2 This is a flowchart of a Fourier stacked imaging position mismatch calibration method under a point-by-point convergence strategy according to an embodiment of the present invention;

[0045] Figure 3 This is a calibration result diagram of the mismatch error provided according to an embodiment of the present invention;

[0046] Figure 4 This is a comparison image of a reconstructed high-resolution image provided according to an embodiment of the present invention.

[0047] The reference numerals in the figures include:

[0048] 1. Laser; 2. Spatial filter; 3. Focusing lens; 4. Sample; 5. Imaging lens; 6. Camera; 7. Camera scanning plane.

[0049] Ideal position of the spectrum sub-aperture 101, actual position of the spectrum sub-aperture 102, and calibrated position of the spectrum sub-aperture 103. Detailed Implementation

[0050] In the following description, embodiments of the invention will be described with reference to the accompanying drawings. In the description below, the same modules are denoted by the same reference numerals. Where the same reference numerals are used, their names and functions are also the same. Therefore, their detailed description will not be repeated.

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.

[0052] Figure 1 A macroscopic Fourier layered imaging system according to an embodiment of the present invention is shown.

[0053] like Figure 1 As shown, the macroscopic Fourier layered imaging system is a transmissive system, comprising an illumination subsystem and an imaging subsystem. The illumination subsystem includes a laser 1, a spatial filter 2, and a focusing lens 3, wherein:

[0054] The output light of laser 1 has a wavelength of 632.8nm and requires a power supply voltage of 220V. Laser 1 is used to provide illumination light.

[0055] The lens in the spatial filter 2 has a focal length of 12.5mm. The spatial filter 2 is used to expand and filter the output light of the laser 1, effectively ensuring the beam quality and spot size.

[0056] The focusing lens 3 has a focal length of 150mm, which converts the incident divergent spherical wave into a converging spherical wave that illuminates the sample, thus providing conditions for Fraunhofer diffraction.

[0057] The imaging subsystem includes a sample 4, an imaging lens 5, and a camera 6, which is connected to a computer that stores control software for Fourier layered imaging technology.

[0058] Imaging lens 5 is a 75mm DG Double Gauss series fixed focal length lens, a product of Etmont Optics.

[0059] Camera 6 is a product of Soleber, model CS165MU / M, with a CCD camera of 1440*1080 pixels and a pixel size of 3.45μm. It is used to receive low-resolution images of sample 4. Camera 6 is equipped with an imaging lens 5 at the front, which is used to perform Fourier transform of the spectrum at the lens pupil plane to the spatial domain.

[0060] After the macroscopic Fourier overlay imaging system is set up as a transmission system, sample 4 is positioned near the focal length of focusing lens 3. Once a clear image is formed on camera 6, camera 6 is moved sequentially on the camera scanning plane 7, with imaging lens 5 moving simultaneously with camera 6. A set of S low-resolution images is acquired and stored in a computer containing the control software for Fourier overlay imaging technology.

[0061] Figure 2 The flowchart of the Fourier stacked imaging position mismatch calibration method under the point-by-point convergence strategy provided by the present invention is shown.

[0062] like Figure 2 As shown, it is necessary to reconstruct low-resolution images to obtain high-resolution images containing more spatial frequency information. The Fourier stacked imaging position mismatch calibration method under the point-by-point convergence strategy proposed in this embodiment of the invention includes the following steps:

[0063] S1. Reconstruct high-resolution images based on the S acquired low-resolution images, where S is a positive integer. The specific process is as follows:

[0064] S11. Determine the intensity of the nth low-resolution image acquired by camera 6 during a single imaging session as I. n The total spectrum is ψ(x,y), where the subscript n represents the nth image acquired by camera 6, i.e., the nth low-resolution image, and the subscript i represents the number of iterations in image reconstruction. The output light of laser 1 is used as the illumination light, with a wavelength λ = 632.8 nm. The distance between sample 4 and camera 6 is denoted by z. The complex amplitude distribution of the optical field on the back surface of sample 4 is as follows: The process of the illumination beam containing information about sample 4 being transmitted to the plane (x, y) where the imaging lens 5 is located can be regarded as a Fraunhofer diffraction. Therefore, the complex amplitude distribution of the light field in the pupil plane of the imaging lens 5 of the camera 6 is as follows:

[0065]

[0066] Here, exp() represents an exponential function with base e, j is the imaginary unit, and k is the wave number.

[0067] S12. For the i-th iteration of the n-th low-resolution image, extract the spectral sub-region acquired by the macroscopic Fourier stacked imaging system during a single imaging session from the total spectrum using the pupil function P(μ,ν) of camera 6. as follows:

[0068]

[0069] Among them, C nThis indicates the position of the best particle in the population corresponding to the nth low-resolution image, with its subscript indicating the μ or ν coordinate.

[0070] S13. Keeping the phase of the spectral sub-region to be updated unchanged, perform a Fourier transform of the spectral sub-region to be updated to the spatial domain to obtain a low-resolution image estimate. Right now:

[0071]

[0072] Where F represents the Fourier transform.

[0073] S14. Replace the estimated amplitude value of the low-resolution image with the amplitude of the nth acquired low-resolution image, that is:

[0074]

[0075] S15. Transform the low-resolution image containing partial frequency information obtained in S14 to the frequency domain. The frequency domain image ψ n ' is represented as:

[0076]

[0077] S16. Using the frequency domain image ψ from S15 n The updated target region for the total spectrum is:

[0078] ψ n i+1 =ψ n i +ψ n i -ψ n '.

[0079] S17. Repeat S12 to S14 until all low-resolution images have been updated.

[0080] S18. Repeat S12 to S15 until the set number of iterations is completed. The number of iterations can be determined according to specific needs.

[0081] Figure 3 The calibration results of the mismatch error provided according to an embodiment of the present invention are shown.

[0082] like Figure 3 As shown, S2, the particle swarm optimization-based position mismatch calibration algorithm, employs a point-by-point convergence strategy to search for and calibrate the mismatch error. The specific process is as follows:

[0083] S21. The center position of the spectral sub-aperture corresponding to each position of camera 6 is (μ,v). 80 particles with corresponding initial velocities are randomly generated within the range [μ±Δμ,v±Δv]. Each particle represents the Sth...i The initial spectral estimate of the center position of the spectral sub-aperture corresponding to the S low-resolution images, where i represents the number of the S low-resolution images.

[0084] S22. Calculate the fitness value p of each particle based on the evaluation function, and select the position C of the best particle in the population. best and its corresponding optimal fitness value p best .

[0085] S23. Calculate the dynamic inertia weight w for each particle. The dynamic inertia weight for each particle is calculated as follows:

[0086]

[0087] Where w represents the dynamic inertia weight, which determines the proportion of the particle's learning of the current velocity each time; c1 = c2 = 2.05, where c1 and c2 represent the learning rates of the particle towards the current particle's historical best position and the global best particle, respectively.

[0088] And update the velocity v of 80 particles according to the following formula:

[0089] v m,l (t+1)=w·(v m,l (t)+c1·r1(p m,l -x m,l )+c2·r2(C best -x m,l ));

[0090] Where t represents the iteration number, l={1,2} represents the coordinates (μ,ν) of the frequency domain sub-aperture, m represents the particle number, and r1 and r2 are uniform random numbers between [0,1].

[0091] Update the positions of 80 particles according to the following formula:

[0092] x m,l (t+1)=x m,l (t)+v m,l (t+1);

[0093] S24. Repeat S22 to S23 until the set number of iterations is completed, search for the optimal position of the spectral sub-aperture, and update the initial spectral estimate based on the final position of the particle. In this invention, dynamic inertial weight is introduced. The initial inertial weight is large, and the global search optimization capability is strong at this time. As the iteration proceeds, the inertial weight becomes smaller, and the local search optimization capability becomes stronger at this time.

[0094] S3. Repeat S2 to calibrate the next low-resolution image until all low-resolution images are calibrated. Output the high-resolution reconstructed spectrum and use the phase recovery algorithm to output the high-resolution reconstructed spectrum as a high-resolution image containing more spatial frequency information.

[0095] Figure 4 A comparison of reconstructed high-resolution images provided according to embodiments of the present invention is shown.

[0096] like Figure 4 As shown, traditional Fourier stacked imaging reconstruction suffers from a sharp decline in image reconstruction quality due to the lack of calibration for the mismatch position of camera 6. This invention addresses this problem by using a particle swarm optimization algorithm in the frequency domain to calibrate the mismatch of camera 6 in the macroscopic Fourier stacked imaging system. The quality of the reconstructed high-resolution image is significantly improved. Verification shows that the method of this invention can accurately calibrate 225 mismatched images in about 50 seconds. This indicates that this invention effectively solves the problem of high-resolution spectral convergence error caused by camera 6 mismatch. Furthermore, this invention has the advantages of fast convergence speed and high calibration accuracy.

[0097] Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

[0098] The specific embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made in accordance with the technical concept of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for calibrating the position mismatch in Fourier stacked imaging under a point-by-point convergence strategy, applied to a macroscopic Fourier stacked imaging system, comprising a laser, a spatial filter, a focusing lens, a sample, and a camera, characterized in that... The method includes the following steps: S1. Reconstruct high-resolution images based on the S acquired low-resolution images, where S is a positive integer. The specific process is as follows: S11. Based on the spectral information of the low-resolution image acquired by the camera, the complex amplitude distribution of the optical field in the pupil plane of the camera's imaging lens is obtained as follows: ； in, Let represent the total spectrum, and exp() represent an exponential function with base e. This represents the complex amplitude distribution of the optical field on the back surface of the sample. The wavelength of the light emitted by the laser is represented by z, and the distance between the sample and the imaging lens is represented by z. j The imaginary unit, k Wave number; S12, Utilizing the pupil function of the imaging lens Extract the spectral sub-region acquired by the system during a single imaging session from the total spectrum. as follows: ； Among them, C n This represents the position of the optimal particle in the population corresponding to the nth acquired low-resolution image, where the subscript indicates the μ or ν coordinate. n , i This represents the i-th iteration of the n-th low-resolution image; S13. Keeping the phase of the spectrum sub-region to be updated unchanged, perform a Fourier transform of the spectrum sub-region to be updated to the spatial domain to obtain a low-resolution image estimate. ,Right now: ； Where F represents the Fourier transform; S14. Replace the estimated amplitude value of the low-resolution image with the amplitude of the low-resolution image acquired by the camera, that is: ； in, This represents the intensity of the nth low-resolution image; S15. Transform the low-resolution image obtained in S14 into a frequency domain image. ,Right now: ； S16. Utilize the frequency domain image from S15 Update the total spectrum target area, i.e.: ； S17. Repeat S12~S16 until all low-resolution images have been updated; S18. Repeat S12~S17 until the set number of iterations is completed; S2. The mismatch error is searched and calibrated using a point-by-point convergence strategy. The specific process is as follows: S21. Randomly generate multiple particles near the center position of the spectral sub-aperture of the camera, wherein the particles represent the initial spectral estimate of the center position of the spectral sub-aperture corresponding to the low-resolution image. S22. Calculate the fitness value of each particle based on the evaluation function. p And select the position of the best particle in the population. and its corresponding optimal fitness value ; S23. Calculate the dynamic inertia weight of each particle as follows: ； in, w The dynamic inertia weight determines the proportion of the particle's learning of the current velocity each time; c1=c2=2.05, where c1 and c2 represent the learning rate of the particle towards the current particle's historical best position and the global best particle, respectively. The velocity of the particle is updated according to the following formula. v : ； Where t represents the number of iterations for mismatch calibration. l ={1,2} represents the coordinates of the frequency domain sub-aperture. m represents the particle number, and r1 and r2 are both uniformly random numbers between [0, 1]. The position of the particle is updated according to the following formula: ； S24. Repeat S22~S23 until the set number of iterations is completed, and update the coordinates of the current frequency domain sub-aperture according to the final position of the particle. S3. Repeat S2 until all the low-resolution images have been calibrated and output the high-resolution reconstructed spectrum.

2. The Fourier layered imaging position mismatch calibration method under the point-by-point convergence strategy as described in claim 1, characterized in that, A phase retrieval algorithm is used to reconstruct the low-resolution image affected by position mismatch error and output it as a high-quality, high-resolution image.

Images