Structured illumination obvious micro-imaging method based on adaptive Hilbert norm

Abstract

The invention relates to a structured illumination obvious micro-imaging method based on an adaptive Hilbert norm, and aims to solve the problems that an illumination parameter estimated value obtained by an existing SIM reconstruction algorithm is inaccurate and reconstruction artifacts occur under the condition that illumination stripes are not uniform. The method comprises the following steps: decomposing a collected single-frame or multi-frame SIM original image into a structure part, a texture part and a noise part; an energy minimization model containing a total variation norm and an adaptive Hilbert norm is constructed, and a part representing the illumination pattern texture is accurately extracted; and carrying out iterative reconstruction on the extracted illumination pattern image and the SIM original image to finally obtain a super-resolution image. Compared with a traditional method, the method has the advantages that the illumination pattern can be accurately extracted from the single-frame original image, non-uniform illumination stripes with local bending are effectively processed, and a more accurate super-resolution reconstruction result is obtained. The method is suitable for various reconstruction methods and application scenes, and has a good application prospect in improvement of SIM imaging quality and imaging flux.

Description

Technical Field

[0001] This invention belongs to the field of optical super-resolution microscopy technology, and specifically relates to a structured illumination microscopy (SIM) method based on adaptive Hilbert norm. Background Technology

[0002] The brain's structure is extremely complex, ranging from dendrites and axons less than 1 micrometer in diameter, capillaries a few micrometers in diameter, to neuronal cell bodies, arterioles, and venules tens of micrometers in diameter. Whole-brain optical imaging must span several orders of magnitude to achieve imaging of these fine structures. Therefore, whole-brain optical imaging demands high-resolution and high-throughput microscopy. Structured illumination microscopy is a high-throughput super-resolution imaging technique that projects high-frequency sinusoidal stripes of structured light onto a sample. Using the principle of spectral shifting, high-frequency information in the sample that cannot be directly detected by the objective lens is moved into the detectable range. Subsequent image reconstruction algorithms then reconstruct the sample structure beyond the diffraction limit. Applying this technique to whole-brain neural circuit imaging allows for more efficient observation of neurons and other fine structures at large scales.

[0003] SIM reconstruction algorithms require accurate estimation of various parameters of the illumination fringes, including spatial frequency, phase, and modulation depth. Existing parameter estimation methods, such as frequency domain cross-correlation and autocorrelation, typically assume that the illumination fringes are uniform across the entire field of view and mostly require three or more phase-shifted images to work accurately. When faced with local bending or deformation of the fringes caused by aberrations, defocus, or other factors, these methods yield inaccurate estimates, leading to artifacts in the reconstruction results. Furthermore, they are not suitable for high-throughput imaging scenarios with only a small number of original images. Therefore, following traditional approaches to illumination parameter estimation is unlikely to lead to substantial breakthroughs in current SIM reconstruction algorithms.

[0004] By treating the original SIM image as a combination of fluorescence samples, illumination fringes, and noise, and from the perspective of image decomposition, the assumption that the frequency and phase of the illumination fringes remain consistent within the imaging field of view can be discarded, allowing for accurate extraction of the illumination fringes. However, establishing clear and effective constraints on the illumination fringes, and addressing the incompatibility between the extracted illumination fringes and traditional reconstruction methods, have become pressing issues that need to be resolved. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a robust, accurate, and high-throughput imaging method based on adaptive Hilbert norm for structured illumination microscopy. This invention, based on an adaptive Hilbert norm-based illumination fringe extraction method, treats illumination fringes as a type of strongly directional local parallel texture. By introducing the structure-texture decomposition concept from image processing, the original SIM image is decomposed into a low-frequency structural component representing the sample fluorescence distribution and a high-frequency texture component representing the illumination fringes, thus extracting the SIM illumination fringes. Iterative super-resolution reconstruction is then performed. Even when the original SIM image contains non-uniform illumination fringes caused by optical system aberrations, it can accurately extract the illumination fringes, reduce reconstruction artifacts, and obtain more accurate and stable super-resolution results.

[0006] The technical solution of this invention is: A structured illumination micro-imaging method based on adaptive Hilbert norm, characterized by the following steps: Step 1: Based on the image structure-texture decomposition model, the acquired single-frame structured illumination original image y is decomposed into a smooth structure part f representing the fluorescence sample, a high-frequency texture part I representing the illumination stripes, and a noise part n. By introducing total variation regularization to constrain the structure part and using adaptive Hilbert norm to constrain the illumination stripe part, an optimization objective is constructed.

[0007] Step 2: Solve the energy minimization problem constructed in Step 1 using the block coordinate descent algorithm to extract the illumination fringes from the input original image. The output I of this process is a two-dimensional image of the same size as the original image, which accurately represents the true illumination fringe pattern acting on the sample, including possible non-uniform features such as local curvature and phase shift. This step can be completed for a single frame of the original image without requiring multiple phase-shifted images.

[0008] Step 3: Take the illumination fringe image extracted for each original image in Step 2 as a known physical prior information and substitute it into the forward model of the iterative super-resolution reconstruction algorithm to construct the iterative optimization objective.

[0009] Step 4: Solve the optimization objective constructed in Step 3 using the alternating direction multiplier method to obtain the SIM super-resolution image.

[0010] In step 1, based on the image structure-texture decomposition model, the acquired single-frame structured illumination original image y is decomposed into a smooth structure part f representing the fluorescence sample, a high-frequency texture part I representing the illumination stripes, and a noise part n. By introducing total variation regularization to constrain the structure part and using adaptive Hilbert norm to constrain the texture part, the corresponding minimization objective function is obtained as follows:

[0011] To optimize the problem, i.e., minimize the objective function, the optimization variable is: ; The original image of a single-frame SIM detected; The fluorescence distribution of the biological sample to be decomposed, i.e., the structural part; ξ represents the illumination stripe pattern to be extracted, i.e., the texture portion; ξ represents the local frequency field characterizing the texture direction and frequency. For structural parts The total variation norm; For the lighting stripe section Regarding frequency fields An adaptive Hilbert norm, used to precisely constrain strongly directional lighting fringes, is described as follows: ; For a local Fourier framework, each For local windows Interior coordinates Fourier transform at the location; For local Fourier framework Below The decomposition operator is represented as follows: ; For frequency field The weighting coefficients are defined as follows:

[0012] in, A scale parameter reflects and The deviation. This represents the frequency field coordinates with the center of the window as the origin. Point The dominant frequency of the surrounding stripes in the frequency field The coordinates in the middle; when the point When there are obvious directional stripes around, and The two terms correspond to two frequency components that are symmetric about the center of the spectrum, such that... Minimize so that To minimize this value, the dominant frequency of the fringe is extracted during the optimization process. When the point... When there are no obvious directional stripes around, for all make This is to avoid the creation of false stripes; The parameter is positive to prevent weight vanishing and to ensure convergence; For the reason The diagonal array formed ; This is a data fidelity item to ensure that the decomposition results are consistent with the original image; To balance the nonnegative regularization coefficients of the two constraints mentioned above.

[0013] Step 2 solves the optimization problem constructed in Step 1 using the block coordinate descent algorithm, that is, for each variable ξ, , Minimize the objective function, which is: ; The optimization problem is broken down into three subproblems, and the three subproblems are solved as follows: (1) Fixed , , The main frequency in the small window It can be approximated as: ; The main frequency is extracted using a pre-set threshold parameter. (2) Fixed , , It can be calculated using the proximal operator: ; This is a proximal operator, which can be computed iteratively. (3) Fixed , , The gradient equation is: ; It is the identity matrix; It is a symmetric positive definite operator. It can be obtained through the conjugate gradient algorithm or Fourier transform.

[0014] The forward model description of the iterative super-resolution reconstruction algorithm in step 3 is as follows: ; The original image of a single-frame SIM detected; The three-dimensional fluorescence distribution of the biological sample to be reconstructed at high resolution; The image of the illumination stripes extracted in step (2); This is a discrete convolution operator for microscope optical systems, with the point spread function as its core. For downsampling operators; Noise section.

[0015] The iterative optimization objective is described as follows: ; The original image number for SIM; The least squares fidelity term has the following mathematical form: ; For regularization terms, For Hessian operators, calculate The second partial derivative at each point. for Mixed norm, its mathematical form is ; for The first-order Schatten norm at point n is defined as the matrix... The sum of all singular values; As a non-negativity constraint, it ensures that the reconstructed fluorophore density distribution has physical meaning, that is, the intensity value at each point is greater than or equal to zero; For linear operators, the mathematical form is: ; To balance the nonnegative regularization coefficients of the two constraints mentioned above.

[0016] Step 4: Solve the Lagrange equation corresponding to the optimization problem in Step 3 using the alternating direction multiplier method. ; , Auxiliary variables and their dual variables, ; , Lagrange multiplier; Using the alternating direction multiplier method, variables The iterative calculation formula is as follows: Initialize the super-resolution image A set of auxiliary variables and its dual variables : ; Update auxiliary variables: ; ; Update host variables:

[0017] ; Update dual variables:

[0018] ; Finally, SIM super-resolution images were obtained. .

[0019] Beneficial effects of this invention: 1. Illumination stripes can be extracted from a single raw image; 2. It can accurately extract illumination fringes and obtain accurate super-resolution reconstruction results even when the illumination fringes are non-uniform. Attached Figure Description

[0020] Figure 1 This is a flowchart of the present invention.

[0021] Figure 2 This is a schematic diagram showing the results of extracting illumination stripes in three different directions from a set of SIM images without rotation regions, according to the present invention.

[0022] Figure 3 This is a schematic diagram of the stripes extracted from the simulated input image in a locally rotated region according to the present invention.

[0023] Figure 4a This is a schematic diagram of the SIM input image with local rotation (red box) according to the present invention.

[0024] Figure 4b This is a schematic diagram of the true values ​​of the stripes in the rotating region of the present invention.

[0025] Figure 4c This is a partial schematic diagram of the stripes extracted by the present invention within the rotation region.

[0026] Figure 4d This is a spectrum of the true values ​​of the stripes in the rotating region of this invention.

[0027] Figure 4e This is a spectrum of the stripes extracted in this invention within the rotational portion.

[0028] Figure 4f This is a schematic diagram of the truth values ​​within the rotation region of this invention.

[0029] Figure 4g This is a reconstruction result diagram within the rotation area of ​​the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings.

[0031] This invention proposes an illumination fringe extraction method based on adaptive Hilbert norm. Illumination fringes are treated as a type of strongly directional local parallel texture. By introducing the structure-texture decomposition concept from image processing, the original SIM image is decomposed into a low-frequency structural component representing the sample fluorescence distribution and a high-frequency texture component representing the illumination fringes, thus extracting the SIM illumination fringes. Iterative super-resolution reconstruction is then performed. The specific process is as follows: Figure 1 As shown. The main steps are as follows: Step 1: Based on the image structure-texture decomposition model, the acquired single-frame structured illumination image y is decomposed into a smooth structure part f representing the fluorescence sample, a high-frequency texture part I representing the illumination fringes, and a noise part n. By introducing total variation regularization to constrain the structure part and using adaptive Hilbert norm to constrain the illumination fringes part, the following minimization objective function is constructed: ; Step 2: Solve the minimization objective function constructed in Step 1 using the block coordinate descent algorithm, and extract the illumination stripes I from the input original image y. (1) Fixed , , The main frequency in the small window It can be approximated as: ; (2) Fixed , , It can be calculated using the proximal operator: ; (3) Fixed , , The gradient equation is obtained using the conjugate gradient descent method: ; Step 3: Take the illumination fringe image extracted for each original image in Step 2 as a known physical prior information and substitute it into the forward model of the iterative super-resolution reconstruction algorithm to construct the iterative optimization objective.

[0032] The forward model is described as follows: ; The optimization objective is described as follows: ; Step 4: Solve the Lagrange equation corresponding to the optimization target in Step 3 using the alternating direction multiplier method to obtain the SIM super-resolution image. : ; , — Auxiliary variables and their dual variables, ; , — Lagrange multiplier; Using the alternating direction multiplier method, variables The iterative calculation formula is as follows: Initialize the super-resolution image A set of auxiliary variables and its dual variables : ; Update auxiliary variables:

[0033] ; Update host variables:

[0034] ; Update dual variables:

[0035] ; Finally, SIM super-resolution images were obtained. .

[0036] The invention will be further illustrated by examples in the next step.

[0037] Example: This embodiment uses two sets of SIM simulation images as simulation cases. To simulate diffraction-limited SIM images, the ground truth image is first multiplied by illumination fringes, then convolved with a point spread function, and finally background fluorescence and noise are added. Then, a 128×128 pixel region of one set of images is rotated 90° to simulate uneven illumination fringes and demonstrate the good performance of the proposed method. Illumination fringes were obtained from both sets of simulation images using an adaptive Hilbert norm-based illumination fringe extraction method. The illumination fringes extraction results for the set of SIM images excluding the rotated region are shown below. Figure 2 As shown, Figure 2 Three illumination directions are shown, with one phase selected for each direction. The frequency and phase are compared with the actual fringes, and the frequency ratio (wave vector magnitude / actual value) and phase error are shown in Tables 1 and 2.

[0038] Table 1. Phase errors of the extracted illumination fringes in various directions and phases.

[0039] Table 2 shows the frequency ratios of the extracted illumination fringes in various directions and phases (accurate to one pixel).

[0040] This method extracts stripes from a simulated input image containing a locally rotated region, such as... Figure 3 As shown in the figure. This method produces accurate reconstruction results in both the unrotated and rotated regions, without mutual interference. The super-resolution reconstruction results within the rotated region are shown in the figure. Figure 4a , Figure 4b , Figure 4c , Figure 4d , Figure 4e , Figure 4f , Figure 4g As shown, the reconstruction results of this method are consistent with the true values, with very small errors, good image detail information, and high super-resolution quality.

Claims

1. A structured illumination micro-imaging method based on adaptive Hilbert norm, characterized by the following steps: Step 1: Based on the structure-texture decomposition model of the image, the acquired single-frame structured light illumination original image y is decomposed into a smooth structure part f representing the fluorescence sample, a high-frequency texture part I representing the illumination stripes, and a noise part n. By introducing total variation regularization to constrain the structure part and using adaptive Hilbert norm to constrain the illumination stripe part, a minimum objective function is constructed. Step 2: Solve the minimized objective function constructed in Step 1 using the block coordinate descent algorithm to separate the illumination stripe image from the input original image y; Step 3: The illumination fringe images extracted for each original image in Step 2 are used as known physical prior information and substituted into the forward model of the iterative super-resolution reconstruction algorithm to construct the iterative optimization objective; Step 4: Solve the iterative optimization objective constructed in Step 3 to obtain the SIM super-resolution image.

2. The structured illumination micro-imaging method based on adaptive Hilbert norm according to claim 1, characterized in that: The minimization objective function constructed in step 1 is: ； To optimize the problem, i.e., minimize the objective function, the optimization variable is: ; The original image of a single-frame SIM detected; The fluorescence distribution of the biological sample to be decomposed, i.e., the structural part; ξ represents the illumination stripe pattern to be extracted, i.e., the texture portion; ξ represents the local frequency field characterizing the texture direction and frequency. For structural parts The total variation norm; For the lighting stripe section Regarding frequency fields An adaptive Hilbert norm, used to precisely constrain strongly directional lighting fringes, is described as follows: ; For a local Fourier framework, each For local windows Interior coordinates Fourier transform at the location; For local Fourier framework Below The decomposition operator is represented as follows: For frequency field The weighting coefficients are defined as follows: ;in, A scale parameter reflects and Deviation; This represents the frequency field coordinates with the center of the window as the origin. Point The dominant frequency of the surrounding stripes in the frequency field The coordinates in the middle; when the point When there are obvious directional stripes around, and The two terms correspond to two frequency components that are symmetric about the center of the spectrum, such that... Minimize so that To minimize this value, the dominant frequency of the stripes is extracted during the optimization process; when the point When there are no obvious directional stripes around, for all make This is to avoid the creation of false stripes; The parameter is positive to prevent weight vanishing and to ensure convergence; For the reason The diagonal array formed This is a data fidelity item to ensure that the decomposition results are consistent with the original image; To balance the nonnegative regularization coefficients of the two constraints mentioned above.

3. The structured illumination micro-imaging method based on adaptive Hilbert norm according to claim 2, characterized in that... Step 2 specifically involves minimizing the objective function from step 1: ； The optimization problem is broken down into three subproblems, and the three subproblems are solved as follows: (1) Fixed , , The main frequency in the small window Approximately: ; The main frequency is extracted using a pre-set threshold parameter. (2) Fixed , , The following is calculated using the proximal operator: ; This is a proximal operator, which can be computed iteratively. (3) Fixed , , The gradient equation is: ; It is the identity matrix; It is a symmetric positive definite operator. It can be obtained through the conjugate gradient algorithm or Fourier transform.

4. The structured illumination micro-imaging method based on adaptive Hilbert norm according to claim 3, characterized in that: Step 3 is specifically defined as follows: The forward model of the iterative super-resolution reconstruction algorithm is as follows: ; The original image of a single-frame SIM detected; The three-dimensional fluorescence distribution of the biological sample to be reconstructed at high resolution; The image of the illumination stripes extracted in step 2; This is a discrete convolution operator for microscope optical systems, with the point spread function as its core. For downsampling operators; Noise component; The iterative optimization objective is described as follows: ; The original image number for SIM; The least squares fidelity term has the following mathematical form: ; For regularization terms, For Hessian operators, calculate The second partial derivative at each point; for Mixed norm, its mathematical form is ; for The first-order Schatten norm at point n is defined as the matrix... The sum of all singular values; As a non-negativity constraint, it ensures that the reconstructed fluorophore density distribution has physical meaning, that is, the intensity value at each point is greater than or equal to zero; For linear operators, the mathematical form is: ; To balance the non-negative regularization coefficients of the aforementioned constraints.

5. The structured illumination micro-imaging method based on adaptive Hilbert norm according to claim 4, characterized in that: Step 4 specifically involves: solving the iterative optimization objective described in claim 4 using the alternating direction multiplier method, employing the corresponding Lagrange equation: ； , Auxiliary variables and their dual variables, , Lagrange multiplier; Using the alternating direction multiplier method, variables The iterative calculation formula is as follows: Initialize the super-resolution image A set of auxiliary variables and its dual variables : ； Update auxiliary variables: ； ； Update host variables: ； ； Update dual variables: ； ； Finally, SIM super-resolution images were obtained. .

Images