Fourier ptychographic imaging position mismatch calibration method under multi-level convergence model

Abstract

The present application provides a kind of Fourier laminated imaging position mismatch calibration method under multistage convergence model, comprising the following steps: according to the low-resolution image collected, high-resolution image reconstruction is carried out;Mismatch error of search camera position is established by hierarchical convergence model, and hierarchical calibration is carried out, hierarchical convergence model is the position mismatch calibration algorithm based on particle swarm, translation mismatch parameter is optimized using hierarchical calibration strategy, while introducing compression factor in the algorithm to improve convergence precision, natural selection process is introduced to improve convergence speed, and the reconstruction of high-resolution image is realized.The present application improves convergence precision and convergence speed, shortens the calibration time under the premise of ensuring calibration accuracy, and solves the artifact problem of reconstructed image, improves the reconstruction quality of high-resolution image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computational optical imaging, and specifically provides a Fourier ptychographic imaging position mismatch calibration method under a multi-stage convergence model. BACKGROUND

[0002] In the field of optics, Fourier ptychographic imaging technology (FP) is usually applied to long-distance scenes, a laser light source is used as an illumination light source, different spatial frequency information of an object is obtained by moving a camera multiple times, high-resolution image reconstruction is performed using a phase recovery technology, the diffraction limit of an optical system is broken through, and a large field of view and high-resolution imaging effect are achieved.

[0003] However, the macro Fourier ptychographic imaging system has the following problems:

[0004] In the macro Fourier ptychographic imaging system, an imaging camera needs to capture a series of low-resolution images by scanning, and due to limitations of existing technologies and equipment, the actual positions of different cameras relative to ideal positions of the imaging system are offset, camera mismatch problems often occur, which can cause artifacts in the final reconstructed image, the reconstructed quality is greatly reduced, and the expected effect cannot be achieved. SUMMARY

[0005] To solve the above problems, the present application provides a particle swarm-based position mismatch calibration algorithm, which specifically adopts a hierarchical calibration strategy to optimize translation mismatch parameters, introduces a compression factor to improve convergence accuracy, introduces a natural selection process to improve convergence speed, and realizes high-resolution image reconstruction.

[0006] The Fourier ptychographic imaging position mismatch calibration method under the multi-stage convergence model provided by the present application includes the following steps:

[0007] S1, high-resolution image reconstruction is performed according to S all low-resolution images collected, wherein S all is a positive integer, and the specific process is as follows:

[0008] S11, the complex amplitude distribution of the pupil plane light field of the imaging lens of the camera is obtained according to the spectral information of the low-resolution images collected by the camera as follows:

[0009]

[0010] wherein, ψ(μ,ν) represents the total spectrum, exp() represents an exponential function with e as the base, represents the complex amplitude distribution of the light field behind the sample, λ represents the wavelength of the light emitted by the laser, z represents the distance between the sample and the imaging lens, j is an imaginary unit, and k is a wave number;

[0011] S12, taking out the sub-region of spectrum from the total spectrum by using the pupil function P(μ, v) of the imaging lens of the camera As follows:

[0012]

[0013] Wherein, C n represents the position of the optimal particle of the population corresponding to the n-th acquired low-resolution image, the subscript thereof represents μ coordinate or v coordinate, wherein the subscripts n, i represent the i-th iteration of the n-th low-resolution image;

[0014] S13, keeping the phase of the sub-region of spectrum to be updated unchanged, and Fourier transforming the sub-region of spectrum to be updated to the spatial domain to obtain a low-resolution image estimation That is:

[0015]

[0016] Wherein, F represents Fourier transform;

[0017] S14, replacing the amplitude value of the low-resolution image estimation with the amplitude of the acquired low-resolution image, that is:

[0018]

[0019] Wherein, I n represents the intensity of the n-th low-resolution image;

[0020] S15, transforming the low-resolution image obtained in S14 to a frequency domain image ψ n , that is:

[0021]

[0022] S16, updating the target region of the total spectrum by using the frequency domain image ψ n in S15, that is:

[0023] ψ n i+1 = ψ n i + ψ n i - ψ n ';

[0024] S17, repeating S12-S16 until all low-resolution images are updated;

[0025] S18, repeating S12-S18 until the set number of iterations is completed;

[0026] S2. Establish a hierarchical convergence model to search for the mismatch error in camera position and perform hierarchical calibration. The specific process is as follows:

[0027] S21. Initialize a = a + 1, a = {0, 1, 2…}, starting from the central spectral sub-aperture, adjust S… a Calibrate one low-resolution image, where S a =(a+2) 2 T particles are randomly generated near the center of the camera's spectral sub-aperture, where T is a positive integer, and each particle contains S. a Initial spectral estimates of the center position of the spectral sub-aperture corresponding to a low-resolution image;

[0028] S22. Reconstruct the image based on the mismatch parameters carried by each particle;

[0029] S23. Calculate the fitness value p of each particle based on the evaluation function and select the position C of the optimal particle. best and its corresponding optimal fitness value p best ;

[0030] S24. Update the velocities of T particles according to the following formula:

[0031] vt,l,y(m+1)=σ(vt,l,y(m)+c1·r1(pt,l,y-xt,l,y(m)+c2·r2(pbest-xt,l,y(m))));

[0032] Where σ represents the compression factor, c1 = c2 = 2.05, where c1 and c2 represent the learning rates of the particle towards its current best historical position and the global best position, respectively; m represents the iteration number; t represents the particle index, t = {1, 2, ..., T}; and l represents the l-th frequency domain sub-aperture in this convergent model, l = {1, 2, ..., S}. a}, y = {1, 2} represents the index of the current frequency domain sub-aperture coordinates (μ, ν);

[0033] Update the positions of T particles according to the following formula:

[0034] x t,l,y (m+1)=x t,l,y (m)+v t,l,y (m+1);

[0035] S25. Repeat S23 to S24. If m≤15, then execute S23.

[0036] If m > 15, then natural selection is performed, and S23 is executed until the spectrum converges or m equals the preset number of iterations. At this point, the first S is obtained. aThe reconstruction spectrum of the low-resolution image and the spectrum sub-aperture optimal position;

[0037] S26, keep the previous S a The spectrum sub-aperture optimal position of the low-resolution image is unchanged, and the steps S21-S25 are repeated on the low-resolution image a+1 The spectrum sub-aperture optimal position of the low-resolution image is unchanged, and the steps S21-S25 are repeated on the low-resolution image all The spectrum sub-aperture optimal position of the low-resolution image is unchanged, and the steps S21-S25 are repeated on the low-resolution image

[0038] Preferably, the hierarchical convergence model uses a low-resolution image captured by a camera as a spatial constraint, a pupil function as a frequency domain constraint, and a particle swarm algorithm to optimize the mismatched parameters of each level position of the camera scanning process in the frequency domain, so as to realize hierarchical calibration of the camera position mismatch.

[0039] Compared with the prior art, the present application can achieve the following beneficial effects:

[0040] The present application uses an improved particle swarm algorithm in the frequency domain to calibrate the camera mismatch in the Fourier stack imaging system, introduces a compression factor to improve the convergence accuracy, introduces a natural selection process to improve the convergence speed, shortens the calibration time under the premise of ensuring the calibration accuracy, and solves the artifact problem of the reconstructed image, thereby improving the reconstruction quality of the high-resolution image. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is a structural diagram of a macro Fourier stack imaging system according to an embodiment of the present application;

[0042] Figure 2 is a flowchart of a Fourier stack imaging position mismatch calibration method under a multi-level convergence model according to an embodiment of the present application;

[0043] Figure 3 is a calibration result diagram of the mismatch error according to an embodiment of the present application;

[0044] Figure 4 is a comparison diagram of the reconstructed high-resolution image according to an embodiment of the present application.

[0045] The reference signs in the drawings include:

[0046] Laser 1, spatial filter 2, focusing lens 3, sample 4, imaging lens 5, camera 6, camera scanning plane 7;

[0047] Spectrum sub-aperture ideal position 101, spectrum sub-aperture real position 102, spectrum sub-aperture calibrated position 103. DETAILED DESCRIPTION

[0048] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the following description, the same modules are denoted by the same reference numerals. In the case of the same reference numerals, their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated.

[0049] In order to make the objects, technical solutions and advantages of the present application clearer, further detailed description will be made to the present application in combination with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and do not constitute limitation to the present application.

[0050] Figure 1 A macro Fourier stack imaging system according to an embodiment of the present application is shown.

[0051] As shown in Figure 1 , the macro Fourier stack imaging system is a transmission system, which comprises an illumination subsystem and an imaging subsystem, the illumination subsystem comprises a laser 1, a spatial filter 2 and a focusing lens 3, wherein:

[0052] The wavelength of the output light of the laser 1 is 632.8 nm, and the required power supply voltage is 220 V, and the laser 1 is used to provide illumination light.

[0053] The focal length of the lens in the spatial filter 2 is 12.5 mm, and the spatial filter 2 is used to expand and filter the output light of the laser 1, which effectively ensures the beam quality and spot size.

[0054] The focal length of the focusing lens 3 is 150 mm, which converts the incident divergent spherical wave into a convergent spherical wave to irradiate on the sample, thereby providing conditions for Fraunhofer diffraction.

[0055] The imaging subsystem comprises a sample 4, an imaging lens 5 and a camera 6, and the camera 6 is connected to a computer in which Fourier stack imaging technology control software is stored.

[0056] The resolution version number of the sample 4 is USAF1951X.

[0057] The imaging lens 5 is a DG double Gauss series fixed focus lens with a focal length of 75 mm, which is a product of the Edmund Optics Company.

[0058] The camera 6 is a product of the SONY Company, with a model of CS165MU / M, which is a CCD camera with 1440*1080 pixels and a pixel size of 3.45 μm, and is used to receive a low-resolution image of the sample 4, and the imaging lens 5 is arranged in front of the camera 6, which is used to transform the frequency spectrum at the lens pupil plane to the spatial domain.

[0059] After the macroscopic Fourier piling imaging system is built as a transmission system, the sample 4 is adjusted to a position near the focal length of the focusing lens 3, and when a clear image is formed on the camera 6, the camera 6 is moved on the camera scanning plane 7, wherein the imaging lens 5 is matched with the camera 6 and moved together. 49 low-resolution images are acquired, and the low-resolution images are stored in a computer storing Fourier piling imaging technology control software.

[0060] Figure 2 The flow of the Fourier piling imaging position mismatch calibration method under the multi-stage convergence model provided by the embodiment of the application is shown.

[0061] As shown in Figure 2 , it is necessary to reconstruct the low-resolution images to obtain high-resolution images containing more spatial frequency information. The Fourier piling imaging position mismatch calibration method under the multi-stage convergence model provided by the embodiment of the application includes the following steps:

[0062] S1, reconstructing a high-resolution image according to the 49 acquired low-resolution images, and the specific process is as follows:

[0063] S11, determining the intensity of the nth low-resolution image collected by the camera 6 during the nth imaging as I n , the total spectrum is ψ(x, y), the subscript n represents the nth camera 6 imaging, i.e. the nth low-resolution image, 1≤n≤49 is an integer, in this embodiment, only 49 images are described, and the specific value of n is not limited, the subscript i represents the iteration number of image reconstruction, the output laser of the laser 1 is used as the illumination light, the wavelength λ=632.8nm, the distance between the sample 4 and the camera 6 is represented by z, and the back surface light field complex amplitude distribution of the sample 4 is The process of transmitting the illumination light beam containing the sample 4 information to the face (x, y) where the imaging lens 5 is located can be regarded as a Fraunhofer diffraction, and therefore the complex amplitude distribution of the pupil plane light field of the imaging lens 5 of the camera 6 obtained is as follows:

[0064]

[0065] Wherein, exp() represents the exponential function with e as the base, j is the imaginary unit, and k is the wave number.

[0066] S12, for the ith iteration of the nth low-resolution image, the pupil function P(μ,ν) of the camera 6 is used to take out the spectrum sub-region from the total spectrum during single imaging collected by the macroscopic Fourier piling imaging system as follows:

[0067]

[0068] Wherein, C nrepresents the position of the optimal particle of the population corresponding to the n-th acquired low-resolution image, the subscript thereof represents the μ coordinate or the v coordinate.

[0069] S13, keeping the phase of the spectrum sub-region to be updated unchanged, Fourier transforming the spectrum sub-region to be updated to the spatial domain to obtain a low-resolution image estimate That is,

[0070]

[0071] wherein F represents Fourier transform.

[0072] S14, using the amplitude of the n-th acquired low-resolution image to replace the amplitude value of the low-resolution image estimate, to obtain a low-resolution image containing partial frequency information, that is,

[0073]

[0074] S15, transforming the low-resolution image containing partial frequency information obtained in S14 to the frequency domain, the low-resolution image in the frequency domain ψ n represents:

[0075]

[0076] S16, using the low-resolution image in the frequency domain ψ n in S15 to update to obtain the total spectrum target region, that is,

[0077] ψ n i+1 = ψ n i + ψ n i - ψ n .

[0078] S17, repeating S12-S14 until the 49 low-resolution images are all updated.

[0079] S18, repeating S12-S15 until the set number of iterations is completed, and the number of iterations can be determined according to specific requirements.

[0080] Figure 3 The calibration result of the mismatch error provided by the embodiment of the application is shown.

[0081] As Figure 3 shown, S2, the particle swarm-based position mismatch calibration algorithm, establishes a hierarchical convergence model to search for the mismatch error of the camera position and performs hierarchical calibration, and the specific process is as follows:

[0082] S21, initialize a=0, a={0, 1, 2,...}, update variable a=a+1, start calibration on S a low-resolution images from the center spectral sub-aperture, wherein S a =(a+2) 2 , i.e. S1=9, S2=16, S3=25, S4=36, S5=49, the center position of the spectral sub-aperture corresponding to each position of the camera 6 is (μ, v), 80 particles with corresponding initial velocities are randomly generated in the range of [μ±△μ, v±△v], each particle contains the initial spectral estimation value of the center position of the spectral sub-aperture corresponding to S a low-resolution images.

[0083] S22, image reconstruction according to the mismatch parameters carried by each particle;

[0084] S23, calculate the fitness value p of each particle according to the evaluation function and select the position C best of the best particle and the corresponding optimal fitness value p best ;

[0085] S24, in order to ensure the convergence accuracy in the particle update process, a compression factor σ is introduced, c1=c2=2.05, c1 and c2 represent the learning rate of the particle to the current particle historical best position and the global best particle respectively, r1 and r2 are both uniform random numbers between 0 and 1, and m represents the iteration number. Here, m and i in S1 both represent the iteration number, which can be equal or not.

[0086] The velocity update formula of the particle is as follows:

[0087] vt,l,y(m+1)=σ(vt,l,y(m)+c1·r1(pt,l,y-xt,l,y(m)+c2·r2(pbest-xt,l,y(m)))).

[0088] Wherein, t represents the particle number, t={1, 2,..., T}, l represents the lth frequency domain sub-aperture under this convergence model, l={1, 2,..., S a}, and y={1, 2} represents the serial number of the current frequency domain sub-aperture coordinate (μ, v).

[0089] It should be noted that in the multi-stage convergence model, a particle has multiple sub-aperture information. Taking the first stage convergence of 9 sub-aperture calibrations as an example, there are totally 80 particles, each of which contains 9 sub-aperture positions, and each sub-aperture position contains two coordinates. Therefore, the three subscripts of the formula are represented as follows: the first t represents the particle number, the second l represents the sub-aperture number, and the third y represents the coordinate (μ, v) number. When y = 1, μ is represented, and when y = 2, v is represented.

[0090] The position updating formula of the particle is as follows:

[0091] x t,l,y (m+1)=x t,l,y (m)+v t,l,y (m+1);

[0092] S25, a natural selection process is introduced to improve the convergence speed of the hierarchical calibration, and the updating threshold is set to 15;

[0093] S23 is repeated, and if m≤15, S23 is executed;

[0094] If m>15, the natural selection process is performed, and S23 is executed until the spectrum converges or m equals the upper limit of the preset iteration number, at which time the reconstructed spectrum of the first 9 low-resolution images and the optimal position of the spectrum sub-aperture are obtained. a

[0095] S26, after the initial hierarchical calibration, the current sub-aperture position of the first 9 reconstructed images is the optimal position of the spectrum sub-aperture, and the reconstructed spectrum of the first 9 reconstructed images is also obtained. S21-S25 are repeated to calibrate the S a+1 16 low-resolution images, that is, the first 16 reconstructed images are calibrated. In the calibration process, the first 16 reconstructed images include the first 9 calibrated reconstructed images, and the spectrum sub-aperture positions of the first 9 reconstructed images remain unchanged and do not need to be updated.

[0096] The above process is continued to be repeated until the calibration of the spectrum sub-aperture center positions of the 49 low-resolution images is completed, and the high-resolution image is reconstructed according to the calibrated spectrum sub-aperture center positions.

[0097] Figure 4 The comparison of the reconstructed high-resolution images provided by the embodiments of the present application is shown.

[0098] As Figure 4 ​As shown, the traditional PF reconstruction result causes the image reconstruction quality to sharply decrease and artifacts to appear due to the fact that the mismatched positions of the cameras are not calibrated. By using the application to grade and calibrate the camera mismatch in the Fourier ptychographic imaging system, the reconstructed high-resolution image has good pixel quality, and compared with other image reconstruction methods of the same type, the application also solves the problem of high-resolution spectrum convergence error.

[0099] Although the embodiments of the present application have been shown and described above, it should be understood by those skilled in the art that the above embodiments are exemplary and should not be construed as limiting the present application. Those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

[0100] The specific embodiments of the application described above do not constitute a limitation on the protection scope of the application. Any various other corresponding changes and modifications made according to the technical concept of the application shall be included in the protection scope of the claims of the application.

Claims

1. A Fourier ptychographic imaging position mismatch calibration method under a multi-stage convergence model, applied to a macro Fourier ptychographic imaging system, comprising a laser, a spatial filter, a focusing lens, a sample and a camera, characterized in that, The method comprises the following steps: S1, reconstructing a high-resolution image according to the collected S all low-resolution images, wherein S all is a positive integer, and the specific process is as follows: S11, obtaining the complex amplitude distribution of the pupil plane light field of the imaging lens of the camera according to the spectral information of the low-resolution image collected by the camera as follows: wherein ψ(μ,ν) represents the total spectrum, exp() represents the exponential function with base e, represents the sample back surface light field complex amplitude distribution, λ represents the wavelength of the laser emitted light, z represents the distance between the sample and the imaging lens, j is the imaginary unit, and k is the wave number; S12, taking out a spectral sub-region captured by the system in a single imaging from the total spectrum by using a pupil function P(μ,ν) of the imaging lens As follows: where C n represents the position of the best particle of the population corresponding to the n-th acquired low-resolution image, whose lower index indicates the μ or v coordinate, where the indices n, i indicate the i-th iteration of the n-th low-resolution image; S13, keeping the phase of the to-be-updated spectral sub-region unchanged, Fourier transforming the to-be-updated spectral sub-region to the spatial domain to obtain a low-resolution image estimate That is: Wherein, F represents Fourier transform; S14, replacing the amplitude value estimated by the low-resolution image with the amplitude of the low-resolution image collected by the camera, that is: where I n denotes the intensity of the nth low resolution image; S15, transforming the low-resolution image obtained in S14 to a frequency domain image ψ n i.e. S16, using the frequency domain image ψ from S15 n updating the total spectral target region, i.e.: ψ n i+1 = ψ n i + ψ n i - ψ n ' S17, repeating S12-S16 until all the low-resolution images are updated; S18, repeating S12-S18 until the set number of iterations is completed; S2, establishing a hierarchical convergence model to search for the mismatch error of the camera position and performing hierarchical calibration, and the specific process is as follows: S21, initialize a=a+1, a={0,1,2,…}, start from the center spectral sub-aperture to calibrate S a low-resolution images, wherein S a =(a+2) 2 randomly generate T particles near the center position of the spectral sub-aperture of the camera, wherein T is a positive integer, and each of the particles contains an initial spectral estimation value of the center position of the spectral sub-aperture corresponding to the S a low-resolution images; S22, performing image reconstruction according to the mismatch parameters carried by each particle; S23, calculate the fitness value p of each particle according to the evaluation function and select the position C of the best particle best and the corresponding optimal fitness value p best ; S24, update the velocities of the T particles according to the following formula: ; wherein σ denotes a compression factor, c1 = c2 = 2.05, c1, c2 represent the learning rate of the particle to the current particle historical best position and the global best particle respectively, m denotes the iteration number, t represents the particle serial number, t = {1, 2,.., T}, l represents the lth frequency domain sub-aperture under the current convergence model, l = {1, 2,.., S} a}, y = {1, 2} represents the serial number of the coordinate (μ, v) of the current frequency domain sub-aperture; The position of the T particles is updated according to the following formula: x t,l,y (m+1) = x t,l,y (m) + v t,l,y (m+1); S25, repeating S23-S24, if m≤15, executing S23; If m > 15, natural selection is performed and S23 is executed until the spectrum converges or m equals a preset iteration number, at which time the reconstruction spectrum of the previous S a low-resolution image and the spectrum sub-aperture optimal position are obtained. S26, keep the previous S a S21~S25 to the low-resolution image until the calibration of the spectral sub-aperture center position of the low-resolution image is completed S a+1 S21~S25 to the low-resolution image until the calibration of the spectral sub-aperture center position of the low-resolution image is completed S all S21~S25 to the low-resolution image until the calibration of the spectral sub-aperture center position of the low-resolution image is completed S 2. The Fourier on-lay imaging position mismatch calibration method under a multi-stage convergence model according to claim 1, wherein, The hierarchical convergence model takes the low-resolution image captured by the camera as the spatial constraint, takes the pupil function as the frequency domain constraint, and optimizes the mismatch parameters of each level position of the camera scanning process in the frequency domain through the particle swarm algorithm, so as to realize the hierarchical calibration of the camera position mismatch.

Images