Method for positioning tomography response peak value of continuous vertical scanning structure illumination microscope

Abstract

The invention belongs to the technical field of three-dimensional shape measurement, and discloses a method for positioning a tomography response peak value of a continuous vertical scanning structure illumination microscope. In the continuous vertical scanning process of the microscope, a plurality of cosine fringe patterns with fixed phase shift are sequentially and circularly projected, and a camera synchronously acquires all corresponding series of projection image signals; the probability density distribution of the maximum tomography response position is calculated by using gradient changes of the collected series of image signals in multiple dimensions, and a high-weight probability density distribution area is selected as a peak value positioning effective interval; performing Gaussian fitting on the effective interval chromatography response signal data through least square method Gaussian fitting, determining a peak position of an axial chromatography response curve, and further realizing surface three-dimensional reconstruction; according to the method, the effective area can be extracted from the axial chromatography response signal with complex noise interference, interference of noise on chromatography response curve peak value positioning is avoided, and the measurement precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional topography measurement, and relates to a method for locating the peak value of the tomography response of a continuous vertical scanning structured illumination microscope, and specifically relates to a method for locating the peak value of the tomography response of a continuous vertical scanning structured illumination microscope based on probability density distribution. Background Art

[0002] The surface topography of microstructures is an important geometric feature of manufactured products and scientific research samples. Its measurement, analysis, and evaluation are of great significance for ensuring the surface functional quality characteristics. The structured illumination microscopy measurement technology (SIM) is based on the principle of wide-field microscopy imaging. It uses structured light to modulate the surface or structural information to be measured, removes the interference of defocus signals, and combines vertical scanning to obtain optical tomography images. Then, a peak value positioning algorithm is used to extract the peak value of the tomography response curve to achieve three-dimensional measurement. This method has the characteristics of non-contact, high efficiency, and high precision, and can be applied not only to reflective surfaces but also to the topography measurement of scattering surfaces.

[0003] Since the existing structured illumination microscopy measurement method needs to stop at a fixed step position during the scanning process, project multiple sets of cosine fringe patterns onto the surface of the sample to be measured through a spatial light modulator (DMD) in the measurement system, and then continue scanning until the scanning is completed. This process takes a long time and cannot meet the requirements of rapid measurement. GERD et al. proposed the continuous vertical scanning structured illumination microscopy measurement technology (CVS-SIM) in the paper "Better three-dimensional inspection with structured illumination: speed". In the case of a fixed scanning step position, the camera only needs to be exposed once, and the axial tomography response can be calculated through the "flying phase method" to achieve three-dimensional reconstruction of continuous vertical scanning, which greatly shortens the measurement time. However, due to the limited number of collected images, the stability of the calculated tomography response signal decreases. In addition, factors such as the local slope of the sample surface, material properties, and ambient light fluctuations will significantly interfere with the collected signals, resulting in serious noise in the tomography response. The existing peak value positioning methods cannot accurately determine the maximum tomography response position in the tomography response under complex noise interference, thus reducing the measurement accuracy of CVS-SIM. Summary of the Invention

[0004] Aiming at the problem of inaccurate peak value positioning of the tomography response curve of the continuous vertical scanning structured illumination microscopy measurement technology, the present invention proposes a robust peak value positioning method for the tomography response of a continuous vertical scanning structured illumination microscope based on probability density distribution.

[0005] To achieve the above object, according to one aspect of the present invention, a method for locating the peak value of the tomographic response of a continuous vertical scanning structured illumination microscope is provided, including the following steps:

[0006] S1: Preset multiple cosine fringe patterns with fixed phase shift amounts and input them into a spatial light modulator, which is used to project one of the cosine fringe patterns with the corresponding fixed phase shift amount onto the surface of the sample to be measured at each equally spaced vertical scanning position during the continuous scanning of the microscope perpendicular to the surface to be measured; during the scanning of the entire vertical range of the sample to be measured, the multiple cosine fringe patterns with fixed phase shift amounts are projected in sequence and cyclically, and a series of image signals corresponding to all equally spaced vertical positions are synchronously collected by a camera;

[0007] S2: For each pixel position, extract the series of image signals collected in step S1 along the axial direction (vertical direction), perform a Fourier transform on the axial image signals to obtain a frequency spectrum signal, and obtain the axial tomographic response at each pixel position by extracting the sideband part on the frequency spectrum signal and performing an inverse Fourier transform.

[0008] Then, utilize the gradient changes in multiple dimensions of the collected series of image signals to calculate the probability density distribution of the maximum tomographic response position, and select the high-weight probability density distribution region as the effective interval for peak value location;

[0009] S3: Through least squares Gaussian fitting, perform Gaussian fitting on the axial tomographic response signal data within the effective interval obtained in step S2 to determine the peak position of the axial tomographic response curve, and further realize the three-dimensional reconstruction of the surface.

[0010] Preferably, the fixed phase shift amount of the cosine fringe pattern in step S1 is 2π / N, where N is the number of generated cosine fringe patterns.

[0011] Preferably, on the preset phase shift fringe image in step S1, the light intensity I DMD (x, y) is expressed as:

[0012]

[0013] where (x, y) represents the image pixel coordinates, x represents the pixel position in the fringe sequence direction, I 0 is the background light intensity, ω is the frequency of the fringe pattern, N is the number of generated phase shift cosine fringe images, and n represents the nth cosine fringe pattern.

[0014] Preferably, the image signal collected by the camera in step S1 is the image signal collected by the camera of the continuous vertical scanning structured illumination microscope.

[0015] Preferably, among the series of image signals collected by the camera during the scanning process, the axial signal I(z) of any point (x, y) is expressed as:

[0016]

[0017] where z represents the scanning position, m(z) represents the tomographic response corresponding to the point (x, y), and Δz represents the scanning step.

[0018] Preferably, step S2 specifically includes the following steps:

[0019] S21: Perform Fourier transform on the axial acquisition image signal I(z), as shown in the following formula:

[0020]

[0021] where I F (z) represents the Fourier transform form of I(z), δ represents the Dirac function, f z represents the spatial frequency domain coordinate of z, and M f represents the Fourier transform form of m;

[0022] Extract the sideband signal of I F (z) and perform inverse Fourier transform to obtain the axial tomographic response m(z), as shown in the following formula:

[0023] m(z) = |F -1 {S[I F (z)]}|

[0024] where F -1 represents the inverse Fourier transform, S[.] represents the sideband signal filter, and |.| represents calculating the modulus;

[0025] S22: Slice the series of images collected by the camera along the XOZ plane to obtain the intensity distribution Ixoz(x, z) on the sliced image, as shown in the following formula:

[0026]

[0027] where m(x, z) represents the tomographic response distribution on the XOZ plane, a represents the maximum value of the axial tomographic response curve, h(x) represents the actual height of the surface of the sample to be measured, and σ FWHM represents the parameter related to the full width at half maximum of the tomographic response curve;

[0028] S23: Calculate the gradients of the sliced images obtained in step S22 in two dimensions along the X (the direction of the fringe sequence in the image plane) and Z (the vertical scanning direction). The gradient change can reflect the interval where the maximum axial tomography response position is located. Using the gradient change, establish a probability density equation for the peak position of the axial tomography response to determine the effective interval for peak positioning.

[0029] Preferably, establish a probability density equation P xoz (x, z) as follows:

[0030]

[0031] where w represents the window size used to reduce gradient fluctuations, represents the calculation of the gradient, and P xoz (x, z) represents the modulation degree of the sample surface structure at different scanning positions on the structured illumination cosine fringe pattern, characterizing the probability density distribution of the peak position of the tomography response, that is, the high-weight region in P xoz represents the region where the tomography response is close to the peak position.

[0032] Preferably, step S3 specifically includes the following steps:

[0033] S31: Approximate the structured illumination axial tomography response m(z) with a Gaussian function g(z);

[0034] S32: Determine the peak position of the axial tomography response curve in the effective interval for peak positioning calculated in step S23 through least squares Gaussian fitting, thereby reconstructing the sample height.

[0035] Preferably, the expression in step S31 is:

[0036]

[0037] where a represents the maximum value of the axial tomography response curve, h represents the position of the maximum value of the tomography response, and σ FWHM represents a parameter related to the full width at half maximum of the tomography response curve.

[0038] Preferably, the expression in step S32 is:

[0039]

[0040] where h′ represents the calculated sample surface height, γ represents the preset weight size, and 0 < γ < 1 is selected, z i represents the tomography response data points within the γ * P xoz region, and m p (z i ) represents z iThe tomographic response intensity calculated by the position.

[0041] Generally speaking, compared with the prior art, the above technical solution conceived by the present invention has the following beneficial effects:

[0042] 1. By analyzing the gradient changes in multiple dimensions of a series of collected image points, the method of the present invention utilizes the noise suppression effect of gradient analysis to determine the probability density distribution of the maximum tomographic response position, thereby being able to avoid the interference of complex noise on the tomographic response signal. Compared with the existing methods, this method can accurately locate the maximum tomographic response position (peak positioning) under the measurement conditions of complex noise interference, and improve the peak positioning accuracy of the tomographic response.

[0043] 2. The local height change and slope change on the sample surface will seriously affect the signal-to-noise ratio of the calculated tomographic response signal. The probability density function constructed by this method reflects the modulation of the surface height and slope changes on the tomographic response, and thus reflects the noise level of the tomographic response signal. Therefore, by determining the high-weight interval of the probability density distribution, it is possible to adaptively determine the peak position fitting interval according to the noise level to avoid redundant calculations and improve the calculation speed. Description of the Drawings

[0044] Figure 1 is the flowchart of the present invention;

[0045] Figure 2 is the schematic diagram of the illumination microscope measurement system;

[0046] Figure 3 is the schematic diagram of the illumination microscope measuring the actual object;

[0047] Among them, 1 - light source, 2 - Kohler illumination module, 3 - spatial light modulator, 4 - camera, 5 - lens, 6 - beam splitter, 7 - microscope objective, 8 - sample to be measured. Detailed Embodiments

[0048] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0049] Please refer to Figure 1 and Figure 3 A method for peak positioning of tomographic response of a continuous vertical scanning structured illumination microscope provided by the present invention includes the following steps:

[0050] S1: Preset multiple cosine fringe patterns with fixed phase shift amounts and input them into a spatial light modulator, which is used to project one of the cosine fringe patterns with the corresponding fixed phase shift amount onto the surface of the sample to be measured at each equally spaced vertical scanning position during the continuous scanning of the microscope perpendicular to the surface to be measured; during the scanning of the entire vertical range of the sample to be measured, the multiple cosine fringe patterns with fixed phase shift amounts are projected in sequence and cyclically, and a series of image signals corresponding to all equally spaced vertical positions are synchronously collected by a camera;

[0051] Specifically, generate N cosine fringe patterns with a fixed phase shift amount of 2π / N and input them into the DMD, where N is the number of generated cosine fringe patterns, which is used for projection during the scanning process. For the phase-shifted fringe image preset in step S1, the light intensity I DMD (x, y) is expressed as:

[0052]

[0053] where (x, y) represents the image pixel coordinates, x represents the pixel position in the fringe sequence direction, I 0 is the background light intensity, ω is the frequency of the fringe pattern, N is the number of generated phase-shifted cosine fringe images, and n represents the nth cosine fringe pattern.

[0054] Then, during the scanning process, when moving Δz positions each time, the DMD projects a phase-shifted fringe image, which is collected by the camera, and the image signals collected by the camera of the continuous vertical scanning structured illumination microscope are obtained. For the series of image signals collected by the camera during the scanning process, the axial signal I(z) of any point (x, y) is expressed as:

[0055]

[0056] where z represents the scanning position, m(z) represents the tomographic response corresponding to the point (x, y), and Δz represents the scanning step.

[0057] S2: For each pixel position, extract the series of image signals collected in step S1 along the axial direction (vertical direction), perform a Fourier transform on the axial image signals to obtain a spectrum signal, and obtain the axial tomographic response of each pixel position by extracting the sideband part on the spectrum signal and performing an inverse Fourier transform.

[0058] Then, utilize the gradient changes in multiple dimensions of the collected series of image signals to calculate the probability density distribution of the maximum tomographic response position, and select the high-weight probability density distribution region as the effective interval for peak positioning;

[0059] Step S2 specifically includes the following steps:

[0060] S21: Fourier transform the axially acquired image signal I(z) as follows:

[0061]

[0062] where, I F (z) represents the Fourier transform form of I(z), δ represents the Dirac function, f z represents the spatial frequency domain coordinate of z, and M f represents the Fourier transform form of m.

[0063] In the above formula, the sideband signal contains the tomographic response signal. By extracting the sideband signal of I F (z) and performing the inverse Fourier transform, the axial tomographic response m(z) is obtained as follows:

[0064] m(z) = |F -1 {S[I F (z)]}|(1.4)

[0065] where, F -1 represents the inverse Fourier transform, S[.] represents the sideband signal filter, and |.| represents calculating the modulus;

[0066] Due to the interference of noise, there are many noise points in the axial tomographic response curve obtained in S21, and it is difficult to directly determine its maximum tomographic response position (i.e., the peak position of the axial response curve). Therefore, consider using the gradient changes in multiple dimensions of the image points to determine the probability density distribution of the maximum tomographic response, and then determine the interval where the peak position is located in the axial response curve, so as to improve the peak positioning accuracy of the axial tomographic response curve.

[0067] S22: Slice the series of images collected by the camera along the XOZ plane (the XOY plane is the image pixel plane, and Z is the axial scanning direction, as Figure 1 shown) to obtain the intensity distribution Ixoz(x,z) on the sliced image as follows:

[0068]

[0069] where, m(x,z) represents the tomographic response distribution in the XOZ plane, a represents the maximum value of the axial tomographic response curve, h(x) represents the actual height of the surface of the measured sample, and σ FWHM represents the parameter related to the full width at half maximum of the tomographic response curve;

[0070] S23: Calculate the gradients in two dimensions along the X (the direction of the fringe sequence in the image plane) and Z (the vertical scanning direction) of the sliced images obtained in step S22. The gradient change reflects the modulation degree of the cosine fringes in the images acquired by the camera at different scanning positions (i.e., the tomography response). Therefore, the interval where the maximum tomography response position is located can be reflected by the gradient change. Using the gradient change, establish the probability density equation P of the peak position of the tomography response xoz , P xoz represents the modulation degree of the sample surface structure at different scanning positions on the structured illumination cosine fringe pattern, reflecting the tomography response. The high-weight region in P xoz represents the region where the tomography response is close to the peak position. Selecting the points in the high-weight region corresponding to the tomography response region for fitting can avoid the interference of irrelevant noise in the axial tomography response curve to determine the effective interval for peak positioning.

[0071] Establish the probability density equation P of the peak position of the tomography response xoz (x, z), as follows:

[0072]

[0073] where w represents the window size used to reduce the gradient fluctuation, represents the calculation of the gradient, and P xoz (x, z) represents the modulation degree of the sample surface structure at different scanning positions on the structured illumination cosine fringe pattern, characterizing the probability density distribution of the peak position of the tomography response, that is, the high-weight region in P xoz represents the region where the tomography response is close to the peak position.

[0074] S3: Through the least squares Gaussian fitting, within the effective interval obtained in step S2, perform Gaussian fitting on the axial tomography response signal data to determine the peak position of the axial tomography response curve, and then realize the three-dimensional surface reconstruction.

[0075] Step S3 specifically includes the following steps:

[0076] S31: Approximate the structured illumination axial tomography response m(z) with a Gaussian function g(z), and its expression is:

[0077]

[0078] where a represents the maximum value of the axial tomography response curve, h represents the position of the maximum value of the tomography response, and σ FWHM represents the parameter related to the full width at half maximum (FWHM) of the tomography response curve.

[0079] S32: Determine the peak position of the axial tomography response curve in the peak localization effective interval calculated in step S23 through least squares Gaussian fitting, thereby reconstructing the sample height, and its expression is:

[0080]

[0081] where h′ represents the calculated sample surface height, γ represents the preset weight size, 0 < γ < 1 is selected, and preferably set to 0.5, z i represents the tomography response data points within the γ*P xoz region, and m p (z i ) represents the tomography response intensity calculated at the z i position.

[0082] Combined with the system ([ Figure 2 ) and the physical object ([ Figure 3 ) adopted in the present invention for description, it specifically includes a spatial light modulator DMD (LightCrafter 6500), a camera (acA1920 - 155um, Basler), a continuous vertical scanning mechanism, an LED light source (center wavelength is 518nm), a microscope objective lens (20×, numerical aperture 0.45, Nikon Inc), and related lenses and fixing brackets.

[0083] During use, first, generate N cosine fringe patterns with a fixed phase shift of 2π / N (N is at least 3, generally 5 is selected) and input them into the DMD. Set the total scanning stroke of the axial scanning mechanism and the fixed scanning step distance. After each advancement to a scanning step distance, the DMD projects a pre - input cosine fringe pattern onto the sample surface, and it is collected by the camera. After moving one scanning step distance next time, switch to project the next phase - shifted fringe pattern, and cycle the above steps in sequence until the scanning is completed.

[0084] After the scanning is completed, a series of collected images are obtained. Use the formulas in steps S1 and S21 to calculate the tomography response, then use the formula in step S23 to calculate the peak position probability density distribution. Set γ = 0.5, thereby determine the high - weight region, and use the formula in step S3 to reconstruct the sample surface topography.

[0085] Those skilled in the art can easily understand that the above - mentioned is only a preferred embodiment of the present invention, and it is not used to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for locating the peak value of tomographic response of a continuous vertical scanning structured illumination microscope, characterized in that: The following steps are involved: S1: Preset multiple cosine fringe patterns with fixed phase shifts and input them into the spatial light modulator, so as to project a cosine fringe pattern with a corresponding fixed phase shift onto the surface of the sample to be tested at each equally spaced vertical scanning position during continuous scanning of the microscope perpendicular to the surface to be tested; during full vertical range scanning of the sample to be tested, multiple cosine fringe patterns with fixed phase shifts are projected cyclically in sequence, and a camera synchronously collects a series of image signals corresponding to all equally spaced vertical positions; S2: For each pixel position, extract the series of image signals collected in step S1 along the axial direction, perform Fourier transform on the axial image signal to obtain a spectrum signal, extract the sideband part on the spectrum signal and perform inverse Fourier transform to obtain the axial tomographic response of each pixel position, Then, the probability density distribution of the maximum tomographic response position is calculated using the gradient changes in multiple dimensions of the acquired series of image signals, and the high-weight probability density distribution area is selected as the effective interval for peak location; S3: Perform Gaussian fitting on the axial tomography response signal data within the effective interval obtained in step S2 by least squares Gaussian fitting, thereby determining the peak position of the axial tomography response curve, and then achieving surface three-dimensional reconstruction.

2. A method for locating the peak value of a tomographic response of a continuous vertical scanning structured illumination microscope as claimed in claim 1, characterized in that: The fixed phase shift of the cosine fringe pattern in step S1 is 2π / N, where N is the number of cosine fringe patterns generated.

3. A method for locating the peak value of a tomographic response of a continuous vertical scanning structured illumination microscope as claimed in claim 1 or 2, characterized in that: The light intensity I at any point (x, y) on the phase-shift fringe image preset in step S1 is DMD (x,y) is represented as: Among them, (x, y) represents the image pixel coordinates, where x represents the pixel position in the fringe sequence direction, I0 is the background light intensity, ω is the frequency of the fringe pattern, N is the number of generated phase-shifted cosine fringe images, and n represents the nth cosine fringe pattern.

4. A method for locating the peak value of a tomographic response of a continuous vertical scanning structured illumination microscope as claimed in claim 3, characterized in that: The series of image signals collected by the camera in step S1 are series of image signals collected by the camera of the continuous vertical scanning structured illumination microscope.

5. A method for locating the peak value of a tomographic response of a continuous vertical scanning structured illumination microscope as claimed in claim 4, characterized in that: In the series of image signals collected by the camera during the scanning process, the axial signal I(z) of any point (x, y) is expressed as: Where z represents the scanning position, m(z) represents the tomographic response corresponding to the point (x, y), and Δz represents the scanning step.

6. A method for locating the peak value of a tomographic response of a continuous vertical scanning structured illumination microscope as claimed in claim 1, characterized in that: Step S2 specifically includes the following steps: S21: Perform Fourier transform on the axial image signal I(z), as shown in the following formula: Among them, I F (z) represents the Fourier transform form of I(z), δ represents the Dirac function, and f z represents the spatial frequency domain coordinate of z, M f represents the Fourier transform form of m; By extracting I F The sideband signal of (z) is transformed by inverse Fourier transform to obtain the axial tomographic response m(z), as follows: m(z)=|F -1 {S[I F (z)]}| Among them, F -1 represents the inverse Fourier transform S[.] represents the sideband signal filter, and |.| represents the calculated modulus; S22: Slice the series of images captured by the camera along the XOZ plane, and obtain the intensity distribution Ixoz(x,z) on the sliced ​​image, as shown in the following formula: Where m(x,z) represents the tomographic response distribution in the XOZ plane, a represents the maximum value of the axial tomographic response curve, h(x) represents the actual height of the sample surface, and σ FWHM represents the parameter related to the full width at half maximum of the tomographic response curve; S23: Calculate the gradient of the slice image obtained in step S22 along the two dimensions of the X and Z directions. The gradient change can reflect the interval where the maximum axial tomographic response position is located. Using the gradient change, establish the probability density equation of the axial tomographic response peak position to determine the effective interval of peak positioning.

7. A method for locating the peak value of a tomographic response of a continuous vertical scanning structured illumination microscope as claimed in claim 6, characterized in that: The probability density equation P of the peak position of the axial tomographic response is established. xoz (x,z), as follows: Among them, w represents the window size used to reduce gradient fluctuations, represents the calculation gradient, P xoz (x, z) represents the modulation degree of the sample surface structure on the structured illumination cosine fringe pattern at different scanning positions, and characterizes the probability density distribution of the peak position of the axial tomographic response, that is, P xoz The medium and high weighted regions represent tomographic responses close to the peak position.

8. The method for locating the peak value of tomographic response of a continuous vertical scanning structured illumination microscope according to claim 1, characterized in that: Step S3 specifically includes the following steps: S31: Approximate the structured illumination axial tomographic response m(z) with a Gaussian function g(z); S32: Determine the peak position of the axial tomography response curve in the peak location effective interval calculated in step S23 by least squares Gaussian fitting, thereby reconstructing the sample height.

9. A method for locating the peak value of a tomographic response of a continuous vertical scanning structured illumination microscope as claimed in claim 8, characterized in that: The expression in step S31 is: Where a represents the maximum value of the axial tomographic response curve, h represents the position of the maximum value of the tomographic response, and σ FWHM Represents a parameter related to the full width at half maximum of the tomographic response curve.

10. The method for locating the peak value of tomographic response of a continuous vertical scanning structured illumination microscope according to claim 8, characterized in that: The expression in step S32 is: Where h′ represents the calculated sample surface height, γ represents the preset weight, and 0<γ<1 is selected. i Indicates that it is located at γ*P xoz Tomographic response data points in the region, m p (z i ) represents z i The tomographic response intensity calculated at the position.

Images