An automatic leveling method for light field microscopic system based on modeling of spatial out-of-focus distribution

Abstract

The application discloses a kind of light field microscopic system automatic leveling methods based on spatial defocus distribution modeling, comprising: collecting two-dimensional original light field data, rearrange as first four-dimensional phase space data, based on the matching analysis of view angle relative to central view angle symmetric distribution, calculate parallax, based on the corresponding relationship of parallax and defocus, obtain defocus coefficient;Collect two-dimensional original light field data under current state, rearrange as second four-dimensional phase space data, and carry out block processing and parallax estimation, and obtain the local defocus of corresponding sub-block by inversion;According to the local defocus of sub-block space position, organization is carried out, constructs spatial defocus distribution function, and obtains tilt parameter by fitting spatial defocus distribution function;Control signal is generated based on tilt parameter, and the posture adjustment of corresponding direction is completed by driving objective table through control signal to execute to complete leveling.The application does not depend on specific sample or hardware parameter, with good universality and scalability.

Description

Technical Field

[0001] This invention relates to the field of microscopic imaging technology, and more specifically to an automatic leveling method for a light field microscopy system based on spatial defocus distribution modeling. Background Technology

[0002] Currently, light field microscopy is a novel microscopic imaging technology that records the light field information of an object, enabling the acquisition of three-dimensional images without mechanical scanning. It has broad application prospects in fields such as biomedicine.

[0003] In existing light field microscopy systems, autofocus methods based on multi-view parallax have emerged. For example, patent publication number CN118567084B discloses an autofocus and focus-locking method for light field microscopy systems based on parallax. This method acquires four-dimensional phase space data, calculates the parallax of each viewpoint relative to the central viewpoint, obtains a two-dimensional parallax matrix and a two-dimensional defocus coefficient matrix, and then fits the global defocus amount of the sample to drive the overall Z-axis adjustment of the shooting distance, achieving rapid autofocus and focus-locking of the sample and successfully solving the problem of overall sample defocus.

[0004] However, the aforementioned parallax-based methods still face technical bottlenecks. While these methods can acquire two-dimensional parallax / defocus distributions, their final output is a global defocus quantity. Essentially, they compress two-dimensional spatial information into a one-dimensional control quantity to drive the entire imaging system to translate along the optical axis, with the sample plane assuming it is parallel to the focal plane. In actual biological imaging, the stage or sample itself is often tilted, resulting in different optimal focal depths in different regions of the field of view. In this case, simply adjusting the Z-axis position cannot simultaneously meet the requirements for clear imaging in all regions, inevitably leading to a phenomenon of 'local sharpness and blurred edges'. Therefore, solving the problem of spatially non-uniform defocus caused by sample tilt becomes crucial for achieving clear imaging across the entire field of view.

[0005] Therefore, how to solve the problem that existing light field microscopy autofocus technology can only compensate for overall axial defocusing, but cannot sense and correct the spatial attitude deviation between the sample plane and the focal plane, resulting in defocusing at the edge of the field of view, is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] In view of the above problems, the present invention proposes an automatic leveling method for a light field microscopy system based on spatial defocus distribution modeling to overcome or at least partially solve the above problems.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] This invention provides an automatic leveling method for a light field microscopy system based on spatial defocus distribution modeling, comprising the following steps: S1. Acquire two-dimensional raw light field data under a known defocus state using a light field microscopy system, and calculate the defocus coefficient; S2. Acquire two-dimensional raw light field data in the current state through the light field microscopy system, divide the field of view into multiple sub-regions in the spatial dimension, calculate the local defocus amount of each sub-region, and organize all local defocus amounts according to the spatial position of the sub-regions to construct a spatial defocus distribution function covering the entire field of view. S3. Based on the relationship model between the spatial defocus distribution function and the spatial attitude of the stage, the tilt parameters of the stage in at least two non-optical axis directions are obtained by performing plane fitting on the spatial defocus distribution function. S4. Generate a control signal based on the tilt parameter, and drive the stage to perform a rotational motion around an axis perpendicular to the optical axis through the control signal, so as to adjust the parallelism between the sample plane and the imaging focal plane. S5. Repeat steps S2, S3, and S4 until the tilt parameter meets the preset leveling accuracy threshold.

[0009] Furthermore, the calculation of the defocus coefficient in step S1 specifically includes: Two-dimensional raw light field data were acquired using a light field microscopy system under a known defocus state and rearranged into four-dimensional phase space data. Select at least one pair of viewpoints that are symmetrically distributed with respect to the central viewpoint, and perform matching analysis on the images between the viewpoint pairs to calculate the disparity; The global defocus coefficient is obtained based on the ratio of the parallax to the known defocus amount, and the defocus coefficient is a scalar constant.

[0010] Furthermore, the calculation of the local defocus amount for each sub-region in step S2 specifically includes: For the four-dimensional phase space data within each sub-region, at least one set of symmetrical viewpoints is selected to estimate the disparity, thereby obtaining the local disparity of that sub-region. Divide the local parallax by the defocus coefficient obtained in step S1 to invert and obtain the local defocus amount of the sub-region.

[0011] Further, in step S2, dividing the field of view into multiple sub-regions in the spatial dimension includes: dividing the second four-dimensional phase space data into m rows and n columns of sub-blocks in the spatial dimension, each sub-block corresponding to a local region of the sample, and the number and size of the blocks are determined comprehensively based on the spatial resolution of the four-dimensional phase space data, the richness of sample texture, the stability of disparity estimation, and the computational complexity.

[0012] Further, in step S3, the relationship model between the spatial defocus distribution function and the spatial attitude of the stage is obtained by fitting the spatial defocus distribution function using the least squares method through planar fitting, and the expression is:

[0013] Where m and n are the number of rows and columns after the sub-block is divided, X and Y are the actual position coordinates of the sub-block in the physical space of the sample, Z is the defocus amount of the sub-block, and a, b, and c are the tilt coefficients of the sample plane along the X direction, the tilt coefficients along the Y direction, and the axial defocus offset constant, respectively.

[0014] Furthermore, in step S3, the tilt parameters of the stage in at least two non-optical axis directions include the horizontal tilt angle. and vertical tilt angle The horizontal tilt angle and vertical tilt angle The expression is:

[0015]

[0016] in, Let be the arctangent function, and a and b be the tilt coefficients of the sample plane along the X direction and the Y direction, respectively.

[0017] Furthermore, it also includes field of view compensation: after the stage performs attitude adjustment, the rotation compensation distance in the horizontal direction is calculated based on the tilt parameters and the eccentric rotation distance of the stage, and the stage is driven to perform a translation of the corresponding distance to keep the field of view position unchanged.

[0018] Furthermore, in the field-of-view compensation step, for small-angle tilts, the field-of-view compensation satisfies:

[0019] in, and These represent the tilt angles of the stage about the X-axis and Y-axis, respectively. This indicates the height offset of the center of the field of view relative to the center of rotation.

[0020] Furthermore, the preset leveling accuracy threshold in step S5 is an angle threshold. When the absolute values ​​of the two tilt parameters obtained in step S3 are both less than the angle threshold, the leveling is determined to be complete.

[0021] As can be seen from the above technical solution, compared with the prior art, the present invention discloses an automatic leveling method for a light field microscopy system based on spatial defocus distribution modeling, which has the following beneficial effects: 1. This invention obtains the local defocus amount of each sub-block by dividing the four-dimensional phase space data into blocks and constructs a spatial defocus distribution function within the field of view. Then, a geometric model between the defocus distribution and the stage attitude is established, and the tilt parameters of the stage in the horizontal and vertical directions are calculated by plane fitting. Based on this, the stage is driven to perform attitude adjustment in the corresponding direction, so that the sample plane is consistent with the focal plane of the imaging system. In the light field microscopy system, full-field automatic leveling based on spatial defocus distribution modeling is realized, overcoming the problem of defocus at the edge of tilted samples.

[0022] 2. This invention also improves the leveling accuracy and robustness by gradually approaching the ideal horizontal state through iterative closed-loop control; at the same time, it performs field-of-view compensation translation based on the geometric relationship under small-angle tilt to avoid target loss caused by the height difference between the rotation center and the field-of-view center; in addition, this method does not depend on specific samples or hardware parameters and has good versatility and scalability. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0024] Figure 1 This is a flowchart of the automatic leveling method provided in an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] This invention discloses an automatic leveling method for a light field microscopy system based on spatial defocus distribution modeling, such as... Figure 1 As shown, it includes the following steps: S1. Acquire two-dimensional raw light field data under a known defocus state using a light field microscopy system, and calculate the defocus coefficient; S2. Acquire two-dimensional raw light field data in the current state through the light field microscopy system, divide the field of view into multiple sub-regions in the spatial dimension, calculate the local defocus amount of each sub-region, and organize all local defocus amounts according to the spatial position of the sub-regions to construct a spatial defocus distribution function covering the entire field of view. S3. Based on the relationship model between the spatial defocus distribution function and the spatial attitude of the stage, the tilt parameters of the stage in at least two non-optical axis directions are obtained by performing plane fitting on the spatial defocus distribution function. S4. Generate a control signal based on the tilt parameter, and drive the stage to perform a rotational motion around an axis perpendicular to the optical axis through the control signal, so as to adjust the parallelism between the sample plane and the imaging focal plane. S5. Repeat steps S2, S3, and S4 until the tilt parameter meets the preset leveling accuracy threshold.

[0027] This invention obtains the local defocus amount of each sub-block by dividing the four-dimensional phase space data into blocks and constructs a spatial defocus distribution function within the field of view. Then, it establishes a geometric model between the defocus distribution and the stage attitude and calculates the tilt parameters of the stage in the horizontal and vertical directions by plane fitting. Based on this, it drives the stage to perform attitude adjustment in the corresponding directions to keep the sample plane consistent with the focal plane of the imaging system. This invention realizes full-field automatic leveling based on spatial defocus distribution modeling in the light field microscopy system and overcomes the problem of defocus at the edge of tilted samples.

[0028] The following is a detailed description of each of the above steps: 1. Four-dimensional phase space data rearrangement; First, the light field microscopy system is used to capture two-dimensional raw light field data. I The original two-dimensional light field image is demultiplexed according to the spatial distribution of the microlens array to establish a mapping relationship from pixel coordinates to viewpoint-spatial four-dimensional coordinates, thus obtaining four-dimensional phase space data. :

[0029] in, Represents the angular coordinates within the microlens. Indicates the index space coordinates of the microlens. This indicates the spectral resolution, which is the number of pixels within each microlens. I This is the original two-dimensional light field data.

[0030] In this embodiment, the known defocus state in step S1 refers to a preset defocus state in which the axial defocus amount is known.

[0031] 2. Defocus coefficient calibration; Two-dimensional raw light field data of the sample were captured at the z-position out of focus using the light field microscopy system. I 1, and rearranged into W 1; Select viewpoint pairs symmetrically distributed with respect to the central viewpoint and calculate the disparity S. Select viewpoint pairs in both the horizontal and vertical directions and calculate the directional disparity S. x and S y Directional disparity reflects the defocus characteristic response of a sample in different viewing directions. Taking the average of the directional disparity yields the disparity S, which can improve the robustness and accuracy of disparity calculation. Among these, the horizontal viewing direction... and Vertical perspective and , Centered perspective:

[0032] The disparity calculation step includes: performing matching analysis on the images between the viewpoint pairs to determine the relative displacement between the two viewpoint images, thereby obtaining the disparity. The matching analysis is implemented using a cross-correlation method, obtaining the disparity by calculating the correlation response and determining the location of the maximum response. In another embodiment, the matching analysis is implemented using a frequency domain method, obtaining the disparity by performing a Fourier transform on the images and calculating phase correlation. In yet another embodiment, the matching analysis is implemented using a feature matching method, obtaining the disparity by extracting image features and performing matching estimation.

[0033] The expression for the system's defocus coefficient K is:

[0034] in, S Let z be the parallax and z be the defocus amount.

[0035] 3. Spatial segmentation and local defocus calculation: Sample data was acquired in the current state using a light field microscopy system and rearranged into four-dimensional phase space data. .

[0036] Four-dimensional phase space data The sample is divided into m rows and n columns of sub-blocks, each sub-block corresponding to a local region of the sample, for a total of m... Four-dimensional phase space data of n sub-blocks ,in The number of blocks, m and n, is determined comprehensively based on the spatial resolution, disparity estimation stability, and computational complexity of the four-dimensional phase space data. When the spatial resolution of the four-dimensional phase space data is high and the sample texture is rich, the number of blocks m and n can be appropriately increased to improve the spatial fineness of the defocus distribution; when the sample texture is simple and disparity estimation is easily affected by noise, the number of blocks m and n should be reduced so that each sub-block contains more effective pixels, thereby improving the stability of disparity estimation; at the same time, the computational load should be controlled according to the real-time requirements of the system to avoid excessively dense blocks leading to increased computation time. Specifically, the four-dimensional phase space data... The height and width are h and w, respectively, and the 2D size of each block is set to 256. 256, the corresponding values ​​of m and n are as follows:

[0037] For each sub-block, the four-dimensional phase space data Disparity estimation is performed separately to obtain local disparity. And combined with the defocus coefficient K Inversion is performed to obtain the local defocus amount in this area. :

[0038] This invention divides the field of view into multiple sub-blocks and calculates the local defocus amount for each sub-block. This "divide and conquer" strategy is the key prerequisite for this invention to sense the spatial defocus distribution and thus solve the sample tilt attitude.

[0039] Unlike existing technologies that only solve for a single global defocus value, this invention constructs a spatial defocus distribution function to analyze the spatial variation of the defocus value within the field of view. This variation directly reflects the tilt orientation of the sample plane. For example, a larger fitted coefficient 'a' indicates a significant tilt of the sample plane along the X-direction. This is a crucial leap forward for this invention, moving from 'focusing' to 'leveling'.

[0040] 4. Spatial defocus distribution construction: By organizing and spatially associating the defocus amounts of all sub-blocks, a spatial defocus distribution function is constructed within the field of view based on the spatial location of each field of view. Specifically, X and Y represent the actual position coordinates of the sub-block in the physical space of the sample, which satisfy the following conditions with respect to the spatial sampling intervals Δx and Δy:

[0041]

[0042] 5. Stage attitude estimation: Based on the aforementioned spatial defocus distribution, a geometric relationship model between it and the spatial attitude of the stage is established. Specifically, when the stage tilts, the spatial defocus distribution exhibits a continuous changing trend within the field of view. By globally modeling and optimizing the defocus distribution, the tilt parameters of the stage in two directions are estimated. The following model is fitted using the least squares method, and the horizontal tilt angle of the stage is calculated. and vertical tilt angle :

[0043] Where m and n are the number of rows and columns after the sub-block is divided, X and Y are the actual position coordinates of the sub-block in the physical space of the sample, Z is the defocus amount of the sub-block, and a, b, and c are the tilt coefficients of the sample plane along the X direction, the tilt coefficients along the Y direction, and the axial defocus offset constant, respectively.

[0044] 6. Automatic leveling control: Based on the estimated tilt angle, a control signal is generated to drive the stage to perform tilt angle adjustment in the corresponding direction and horizontal adjustment. Vertical adjustment .

[0045] 7. Iterative convergence: The process iteratively executes steps such as constructing the spatial defocus distribution, estimating the stage attitude, and controlling automatic leveling. Iterative convergence improves leveling accuracy and robustness. Specifically, a convergence criterion of 0.1 degrees is set; when the calculated stage tilt angle is less than 0.1 degrees, the stage is considered essentially level, and iteration stops. 8. Field of view compensation: When the stage tilts or rolls about its horizontal axis, the tilt causes a horizontal displacement of the field of view due to the height difference between the center of rotation and the center of the field of view. For small-angle tilts, the displacement of the field of view satisfies:

[0046]

[0047] in, and These represent the tilt angles of the stage about the X-axis and Y-axis, respectively. This represents the height offset of the field of view center relative to the rotation center. Based on this displacement, a compensating translation is performed on the stage in the horizontal direction to keep the field of view position substantially unchanged.

[0048] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An automatic leveling method for light field microscopy system based on spatial out-of-focus distribution modeling, characterized in that, Includes the following steps: S1. Acquire two-dimensional raw light field data under a known defocus state using a light field microscopy system, and calculate the defocus coefficient; S2. Acquire two-dimensional raw light field data in the current state through the light field microscopy system, divide the field of view into multiple sub-regions in the spatial dimension, calculate the local defocus amount of each sub-region, and organize all local defocus amounts according to the spatial position of the sub-regions to construct a spatial defocus distribution function covering the entire field of view. S3. Based on the relationship model between the spatial defocus distribution function and the spatial attitude of the stage, the tilt parameters of the stage in at least two non-optical axis directions are obtained by performing plane fitting on the spatial defocus distribution function. S4. Generate a control signal based on the tilt parameter, and drive the stage to perform a rotational motion around an axis perpendicular to the optical axis through the control signal, so as to adjust the parallelism between the sample plane and the imaging focal plane. S5. Repeat steps S2, S3, and S4 until the tilt parameter meets the preset leveling accuracy threshold.

2. The method of claim 1, wherein, The calculation of the defocus coefficient in step S1 specifically includes: Two-dimensional raw light field data were acquired using a light field microscopy system under a known defocus state and rearranged into four-dimensional phase space data. Select at least one pair of viewpoints that are symmetrically distributed with respect to the central viewpoint, and perform matching analysis on the images between the viewpoint pairs to calculate the disparity; The global defocus coefficient is obtained based on the ratio of the parallax to the known defocus amount, and the defocus coefficient is a scalar constant.

3. The method of claim 1, wherein, The calculation of the local defocus amount for each sub-region in step S2 specifically includes: For the four-dimensional phase space data within each sub-region, at least one set of symmetrical viewpoints is selected to estimate the disparity, thereby obtaining the local disparity of that sub-region. Divide the local parallax by the defocus coefficient obtained in step S1 to invert and obtain the local defocus amount of the sub-region.

4. The method of claim 1, wherein, In step S2, dividing the field of view into multiple sub-regions in the spatial dimension includes: dividing the second four-dimensional phase space data into m rows and n columns of sub-blocks in the spatial dimension, with each sub-block corresponding to a local region of the sample. The number and size of the sub-blocks are determined comprehensively based on the spatial resolution of the four-dimensional phase space data, the richness of the sample texture, the stability of disparity estimation, and the computational complexity.

5. The method of claim 1, wherein, In step S3, based on the relationship model between the spatial defocus distribution function and the spatial attitude of the stage, the spatial defocus distribution function is fitted using the least squares method through planar fitting, and the resulting expression is: Where m and n are the number of rows and columns after the sub-block is divided, X and Y are the actual position coordinates of the sub-block in the physical space of the sample, Z is the defocus amount of the sub-block, and a, b, and c are the tilt coefficients of the sample plane along the X direction, the tilt coefficients along the Y direction, and the axial defocus offset constant, respectively.

6. The method of claim 1, wherein, In step S3, the tilt parameters of the object table in at least two non-optical axis directions include a horizontal direction tilt angle and a vertical direction tilt angle The expression of the horizontal direction tilt angle and the vertical direction tilt angle is wherein, is an arctangent function, a and b are respectively a tilt coefficient of the sample plane along the X direction and a tilt coefficient along the Y direction.

7. The method of claim 1, wherein, It also includes field of view compensation: after the stage performs attitude adjustment, the rotation compensation distance in the horizontal direction is calculated based on the tilt parameters and the eccentric rotation distance of the stage, and the stage is driven to perform a translation of the corresponding distance to keep the field of view position unchanged.

8. The method of claim 7, wherein, In the field of view compensation step, for small-angle tilts, the field of view compensation satisfies: in, and These represent the tilt angles of the stage about the X-axis and Y-axis, respectively. This indicates the height offset of the center of the field of view relative to the center of rotation.

9. The method of claim 1, wherein, The preset leveling accuracy threshold in step S5 is an angle threshold. When the absolute values ​​of the two tilt parameters obtained in step S3 are both less than the angle threshold, the leveling is determined to be complete.

Images