Interference quantitative phase imaging system based on compressed sensing and pixel super-resolution reconstruction method

Abstract

The invention relates to a camera pixel super-resolution method, which comprises a camera and a plurality of different masks, a plurality of low-resolution pictures are shot through the cooperation of the camera and the masks, and a high-resolution picture is reconstructed through a super-resolution recovery algorithm. The different masks are formed by rearranging different row vectors in a Hadamard matrix, and light field information of a sub-pixel level is coded; the mask coding uses compressed sensing to greatly reduce coding times; the camera photographs different low-resolution pictures encoded by different masks, and the super-resolution recovery algorithm constructs a pseudo-inverse matrix of a measurement matrix through singular value decomposition (SVD) to further improve an iterative hard threshold (IHT) algorithm, achieving more efficient reconstruction. According to the method, the super-resolution of the intensity image shot by the camera can be realized, and the super-resolution of the phase image shot by the camera can also be realized, that is, the method can be combined with a phase imaging technology, so that the application of the industrial camera in the biomedical field is expanded.

Description

Technical Field

[0001] The present invention belongs to the field of computational imaging, and particularly relates to an interferometric quantitative phase imaging system based on compressive sensing and a pixel super-resolution reconstruction method. Background Art

[0002] Tissues and organs are usually composed of thousands of cells, which are interconnected through a complex network of signaling pathways in their native environment. Observing the interactions between these cells is very important for studying the physiology and pathology of biological systems. A potential technique to achieve this goal is large-scale label-free optical imaging techniques, such as interferometric quantitative phase microscopy. However, the trade-off between the field of view and resolution is limited by the system's spatial bandwidth product, which hinders the application of this technique.

[0003] As an important application of compressive sensing, single-pixel reconstruction can extract sub-pixel signals in an image. It is widely used to solve the problem of insufficient sampling. Single-pixel imaging only requires a single-pixel detector to collect all signal intensities within a large field of view. It can collect weak signals and has been widely used in fluorescence detection, remote sensing, millimeter-wave imaging, and light field microscopy. It can also improve the resolution and quality of images. Single-pixel reconstruction and interferometric quantitative phase microscopy are used to observe intensity images and phase images of large fields of view. Summary of the Invention

[0004] The purpose of the present invention is to provide a compressive sensing-based interferometric quantitative phase imaging system and a pixel super-resolution reconstruction method that can extend the resolution of phase images under a large field of view and can be used to monitor large-field transparent samples such as living cell clusters and tissue sections in real time and effectively.

[0005] The technical solution adopted by the present invention to achieve the above purpose is as follows:

[0006] An interferometric quantitative phase imaging system based on compressive sensing, comprising: a laser and, in sequence on its optical path, a first lens, a second lens, a first beam splitter, a first mirror, a stage, an objective lens, a second mirror, a mask, a second beam splitter, a third lens, a diaphragm, and a camera.

[0007] The mask is specifically:

[0008] By deriving the row vectors of the Hadamard matrix, a sampling pattern is obtained, and the number of blocks composed of "±1" in the original row vectors is calculated;

[0009] The sampling pattern is transposed and rearranged into another one-dimensional row vector different from the original row vector, and then the number of blocks in the rearranged row vector is calculated;

[0010] Take the sum of the number of blocks in the original row vector and the number of blocks in the rearranged row vector as the fraction of the sampling matrix, and sort them from low to high;

[0011] Select the top 5 sampling patterns in the sorting, splice them between each pattern, and make a mask through the coating technology.

[0012] The pixel super-resolution method based on compressive sensing includes the following steps:

[0013] 1) Use the Hadamard matrix to make a mask and place it in the imaging system;

[0014] 2) Place the sample on the stage, take the refracted beam and the transmitted beam of the first beam splitter as the reference beam R(x, y) and the sample beam O(x, y) respectively, and calculate them to obtain 5 low-resolution images I(x, y), perform Fourier transform on the images, and obtain the image field Y = O(x, y)R * (x, y);

[0015] 3) Rearrange the 5 low-resolution images, and perform sub-image splitting pixel by pixel according to the low-resolution images, so that each sub-image corresponds to a pixel one by one;

[0016] 4) For each pixel, perform sub-image reconstruction through an improved hard threshold algorithm;

[0017] 5) Perform permutation and combination on the sub-image set X∈R MN×(mn) to obtain the high-resolution image I H ∈R Mm×Nn ;

[0018] 6) Remove the sample from the stage, and repeat steps 2) to 5) to obtain the high-resolution background image G H ∈R Mm×Nn ;

[0019] 7) Obtain the high-resolution phase image according to the high-resolution background image and the high-resolution image.

[0020] The method of making a mask using the Hadamard matrix is specifically as follows:

[0021] Derive the row vector of the Hadamard matrix to obtain the sampling pattern, and calculate the number of blocks composed of "±1" in the original row vector;

[0022] Transpose the sampling pattern, and rearrange the sampling pattern into another one-dimensional row vector different from the original row vector, and then calculate the number of blocks in the rearranged row vector;

[0023] Take the sum of the number of blocks in the original row vector and the number of blocks in the rearranged row vector as the fraction of the sampling matrix, and sort them from low to high;

[0024] Select the first 5 sampling patterns in the selection sorting. Each pattern is spliced and made into 5 masks through a coating technique.

[0025] The low-resolution interferogram I(x, y) is specifically:

[0026] I(x,y) = I 0 + O * (x,y)R(x,y) + O(x,y)R * (x,y)

[0027] where, I 0 is the 0th-order component, O(x,y)R * (x,y) is the image field, and O * (x,y)R(x,y) is the conjugate image field.

[0028] Step 3) is specifically:

[0029] Rearrange the 5 low-resolution images Y ∈ R 5×M×N to Y ∈ R MN×5 , where M represents the length of the low-resolution image, N represents the width of the low-resolution image, and the sampling pattern is Assume the sub-image is x ∈ R m×n , then the pixel y and its corresponding sub-image x satisfy:

[0030]

[0031] Rearrange the sampling pattern and the sub-image, and let x ∈ R 1×mn , then we get:

[0032]

[0033] where m represents the length of the sub-image, n represents the width of the sub-image, and e represents the measurement noise.

[0034] The improved hard threshold algorithm is specifically:

[0035]

[0036] where, is the pseudo-inverse matrix of the sampling pattern constructed by singular value decomposition, f s (·) is a non-linear sparse operator, specifically to find the largest s elements and set the other elements to zero, μ is the step size controlling the update rate of each iteration, y is the value of each pixel, and x t is the sub-image reconstructed at the tth iteration. After multiple iterations, a set of sub-images X ∈ R MN×(mn) is obtained.

[0037] The high-resolution phase map is specifically as follows:

[0038]

[0039] Among them, phase represents the final high-resolution phase map, Image[] represents taking the imaginary part, Real[] represents taking the real part, arctan() represents calculating the arctangent, and unwrap() represents phase unwrapping.

[0040] The present invention has the following beneficial effects and advantages:

[0041] The present invention utilizes an easily implementable optical system and a lightweight reconstruction algorithm to achieve high-resolution quantitative phase imaging in a large field of view within 20 seconds. In the field of view of a 10x lens, accurate high-resolution quantitative phase images are obtained by this method. Compared with low-resolution images, the reconstructed images are closer to those under a 40x lens. It provides the possibility for real-time monitoring of transparent samples in a large field of view by the interference method. Description of the Drawings

[0042] Figure 1 is the optical path diagram of the improved microscopic imaging system;

[0043] Figure 2 is the sorting diagram for a 16×16 Hadamard matrix;

[0044] Figure 3 is the high-resolution reconstruction flow chart;

[0045] Figure 4 is the reconstruction result diagram of the transparent grid sample;

[0046] Figure 5 is the reconstruction result diagram of the biological sample. Detailed Embodiment

[0047] The following further elaborates on the present invention in conjunction with the drawings and embodiments.

[0048] The present invention is an interference quantitative phase imaging system and pixel super-resolution reconstruction method based on compressive sensing. By using a mask to efficiently encode the image field in a large field of view, and then reconstructing the high-resolution phase for a series of encoded low-resolution images.

[0049] This imaging system is as Figure 1As shown in the figure, it includes a laser (S1), a first lens (S2), a second lens (S3), a first beam splitter (S4), a first mirror (S5), a stage (S6), an objective lens (S7), a second mirror (S8), a mask (S9), a second beam splitter (S10), a third lens (S11), a diaphragm (S12), and a camera (S13). Among them, the laser (S1), the first lens (S2), the second lens (S3), the first beam splitter (S4), the first mirror (S5), the stage (S6), the objective lens (S7), the second mirror (S8), and the second beam splitter (S10) constitute an off-axis interference phase imaging system, and the mask (S9), the third lens (S11), the diaphragm (S12), and the camera (S13) constitute a camera resolution improvement system.

[0050] The implementation principle of this system is as follows: The beam emitted by the laser (S1) passes through a beam expander composed of the first lens (S2) and the second lens (S3) and becomes a parallel beam. After passing through the first beam splitter (S4), it becomes a reference beam (left beam) and a sample beam (right beam); the sample beam is vertically irradiated on the sample (S6) through the first mirror (S5), forms an image field after passing through the objective lens (S7), and the image field is encoded by the mask (S9) after passing through the second mirror (S8). The encoded image field is combined with the reference beam by the second beam splitter (S10) to form an interference pattern; the interference pattern is collected by the camera (S13) after passing through the third lens (S11) and the diaphragm (S12).

[0051] Next, the high-resolution image reconstruction method is introduced, and the specific steps are as follows:

[0052] Step 1, mask sampling pattern sorting and selection: Each row vector of the Hadamard matrix is rearranged into a two-dimensional pattern and regarded as a sampling pattern. Since the sampling pattern is derived from the row vectors of the Hadamard basis, we directly calculate the number of blocks in the original row vector. Then, we transpose the sampling pattern and rearrange the sampling pattern into another one-dimensional row vector different from the original row vector. We also calculate the number of blocks in this rearranged row vector. The sum of these two numbers is regarded as the "score". In order to obtain more original image information at a smaller sampling rate, we sort the sampling matrix according to the "score" from low to high. The smaller the "score", the better the reconstruction effect. Figure 2 Shows the result of sampling pattern sorting for the 16×16 Hadamard matrix by us.

[0053] Step 2, obtaining low-resolution images: For the method described in Step 1, select the first 5 sampling patterns, make a mask (S9) and encode the light field to obtain 5 low-resolution images containing different information of the high-resolution image.

[0054] Step 3: Obtaining the image field from the interference pattern: Assume the reference beam is \(R(x,y)\) and the sample beam is \(O(x,y)\). Then, for the off-axis interference pattern formed by the interaction of the two beams, the resulting low-resolution interference pattern \(I(x,y)\) satisfies the relationship described by the following formula:

[0055] \(I(x,y)=I\) 0 +O * (x,y)R(x,y)+O(x,y)R * (x,y)

[0056] Performing a Fourier transform on the interference pattern, the 0th-order component \(I\) 0 , the image field \(O(x,y)R\) * (x,y) and the conjugate image field \(O\) * (x,y)R(x,y) have different positions in the Fourier domain. By using a filtering algorithm, the image field \(Y = O(x,y)R\) * (x,y) is obtained.

[0057] Step 4: Sub-image splitting: The 5 obtained low-resolution images \(Y\in\mathbb{R}\) 5×M×N are rearranged as \(Y\in\mathbb{R}\) MN ×5 , where \(M\) represents the length of the low-resolution image and \(N\) represents the width of the low-resolution image. According to the single-pixel imaging principle, the signal collected by each pixel \(y\) of the camera (S13) is the accumulation of all signals of the sub-image \(x\) it covers. The sampling pattern we designed is Assuming the sub-image is \(x\in\mathbb{R}\) m×n , then the relationship between the pixel and its corresponding sub-image is as follows:

[0058]

[0059] We rearrange the sampling pattern and the sub-image, let \(x\in\mathbb{R}\) 1×mn , then the following relationship is obtained:

[0060]

[0061] Step 5: Sub-image reconstruction: For each pixel, we reconstruct the sub-image through an improved hard threshold algorithm. The improved hard threshold algorithm is as follows:

[0062]

[0063] Since the sampling pattern is known, we can construct the pseudo-inverse matrix of the sampling pattern through singular value decomposition (SVD) \(f\) s(·) is a non-linear sparse operator that finds the largest 20 elements and sets the other elements to zero; μ is the step size that controls the update rate of each iteration, set to 0.5; y is the value of each pixel; x t is the sub-graph reconstructed at the t-th iteration. After 10 iterations, we obtain a set of sub-graphs X ∈ R MN×(mn) .

[0064] Step Six: Sub-graph stitching: Based on the set of sub-graphs X ∈ R MN×(mn) obtained in Step Five, we arrange and combine them to form the final high-resolution image I H ∈ R Mm×Nn , and compared with the low-resolution image, we increased the resolution by mn times.

[0065] Step Seven: Generation of high-resolution phase image: We repeat Steps Two to Six for the background image. At this time, we obtain the high-resolution background image G H ∈ R Mm×Nn . According to the high-resolution background image and the high-resolution image, the high-resolution phase map can be obtained according to the following relationship:

[0066]

[0067] where phase represents the final high-resolution phase map, Image[] represents taking the imaginary part, Real[] represents taking the real part, arctan() represents calculating the arctangent, and unwrap() represents phase unwrapping.

[0068] According to Step Seven, we obtained the final high-resolution phase image. The complete step flow chart is as Figure 3 shown.

[0069] Figure 4 demonstrates the imaging ability of the pixel super-resolution interferometric quantitative phase imaging system for large-field transparent samples. We used lithography technology to fabricate transparent grid patterns of different widths and compared the phase images captured from a 40× objective lens and a 10× objective lens in Figure 4 . As shown in Figure 4 a, the transparent grid pattern can hardly be observed with a 40× objective lens under bright-field conditions. However, due to the phase delay generated by the transparent sample, we used the 40× objective lens interferometric quantitative phase imaging system to image the phase signal, as shown in Figure 4 b. We further observed the same sample with a 10× objective lens in Figure 4 c. Figure 4 d shows the phase image, with obvious speckle noise at the edge of the phase image. We magnified the local area in the red rectangle and placed its phase image in Figure 4To the lower right of d. We could not distinguish the finer grid in the local area phase image. Then, we used an interferometric quantitative phase imaging system with pixel super-resolution to image the phase signal and obtained the reconstructed bright-field image, as shown in Figure 4 e. Figure 4 f shows the reconstructed phase image and the magnified local area image. Our method effectively enhanced the target grid pattern and reduced noise. Although Figure 4 there is still some white noise in the upper part of the phase image in f, it is clearer than the phase image in Figure 4 d. We show the profiles of the red lines in Figure 4 g for Figure 4 b, d, f. Due to the different resolutions of Figure 4 b, d, f, we normalized the pixel distances in Figure 4 g for easy comparison. As shown in Figure 4 g, it is difficult to determine the characteristics of the fine grid from the orange line (phase value under a 10× objective lens). Compared with the blue line (reconstructed phase value) and the green line (phase value acquired under a 40× objective lens), the peak of the orange line gradually disappears. We also observed that the blue line is closer to the green line than the orange line. We further statistically analyzed the Figure 4 phase values in Figure 4 b, d, f, and the results are shown in

[0070] Figure 5 Figure 5 shows the imaging ability of the interferometric quantitative phase imaging system with pixel super-resolution for cell cluster samples. We observed the live cell cluster samples with a large field of view using a 10× objective lens. As shown in a, we observed the cell contours in the bright-field image. In Figure 5 b - c, the phase signal near the cell nucleus is greater than other regions. However, in the monolayer cells, since the phase signal looks uniform, we could not clearly distinguish each cell from the collective cells. As shown in Figure 5 d, we further improved the resolution using the interferometric quantitative phase imaging system with single-pixel super-resolution. As shown in Figure 5 e, the phase value near the cell nucleus is significantly greater than the phase values in other regions, even in the cell monolayer. We magnified the red rectangular area in Figure 5 e, and the result is shown in Figure 5 f. We saw a relatively large cell nucleus area and intracellular reticular details.

Claims

1. Interferometric quantitative phase imaging system based on compressed sensing, characterized in that: include: A laser and a first lens, a second lens, a first beam splitter, a first reflector, a stage, an objective lens, a second reflector, a mask, a second beam splitter, a third lens, an aperture, and a camera are sequentially arranged on its optical path.

2. The interferometric quantitative phase imaging system based on compressed sensing according to claim 1, characterized in that: The mask is specifically: The sampling pattern is obtained by deriving the row vector of the Hadamard matrix and calculating the number of blocks consisting of "±1" in the original row vector; Transpose the sampling pattern and rearrange the sampling pattern into another one-dimensional row vector different from the original row vector, and then calculate the number of blocks in the rearranged row vector; The sum of the number of blocks in the original row vector and the number of blocks in the rearranged row vector is used as the score of the sampling matrix and sorted from low to high; The first five sampling patterns in the sorting are selected, each pattern is spliced ​​together, and a mask is made by coating technology.

3. A pixel super-resolution method based on compressed sensing, applied to the interferometric quantitative phase imaging system based on compressed sensing according to claim 1, characterized in that: The following steps are involved: 1) Using the Hadamard matrix to make a mask and place it in the imaging system; 2) Place the sample on the stage, use the refracted beam and the transmitted beam of the first beam splitter as the reference beam R(x, y) and the sample beam O(x, y), respectively, and calculate them to obtain 5 low-resolution images I(x, y). Perform Fourier transform on the images, and obtain the image field Y=O(x, y)R through the filtering algorithm. * (x,y); 3) Rearrange the five low-resolution images and split them into sub-images pixel by pixel according to the low-resolution images, so that each sub-image corresponds to a pixel one by one; 4) For each pixel, sub-image reconstruction is performed using an improved hard threshold algorithm; 5) For the subgraph set X∈R MN×(mn) Perform permutations and combinations to obtain a high-resolution image I H ∈R Mm×Nn ; 6) Remove the sample from the stage and repeat steps 2) to 5) to obtain a high-resolution background image G. H ∈R Mm×Nn ; 7) A high-resolution phase map is obtained based on the high-resolution background map and the high-resolution image.

4. The pixel super-resolution method based on compressed sensing according to claim 3 is characterized in that: The method of making a mask using the Hadamard matrix is ​​specifically as follows: The sampling pattern is obtained by deriving the row vector of the Hadamard matrix and calculating the number of blocks consisting of "±1" in the original row vector; Transpose the sampling pattern and rearrange the sampling pattern into another one-dimensional row vector different from the original row vector, and then calculate the number of blocks in the rearranged row vector; The sum of the number of blocks in the original row vector and the number of blocks in the rearranged row vector is used as the score of the sampling matrix and sorted from low to high; The first five sampling patterns in the sorting are selected, and each pattern is spliced ​​and made into five masks through coating technology.

5. The pixel super-resolution method based on compressed sensing according to claim 3, characterized in that: The low-resolution interference pattern I(x, y) is specifically: I(x,y)=I0+O * (x,y)R(x,y)+O(x,y)R * (x,y) Among them, I0 is the 0th level component, O(x,y)R * (x, y) is the image field, O * (x,y)R(x,y) is the conjugate image field.

6. The pixel super-resolution method based on compressed sensing according to claim 3, characterized in that: The step 3) is specifically as follows: The five low-resolution images Y∈R 5×M×N Rearrange to Y∈P MN×5 , where M represents the length of the low-resolution image, N represents the width of the low-resolution image, and the sampling pattern is Assume that the subgraph is x∈R m×n , then the pixel y and its corresponding sub-image x satisfy: Rearrange the sampling patterns and sub-images, so that x∈R 1×mn , we get: Where m represents the length of the sub-image, n represents the width of the sub-image, and e represents the measurement noise.

7. The pixel super-resolution method based on compressed sensing according to claim 3, characterized in that: The improved hard threshold algorithm is specifically: in, is the pseudo-inverse matrix of the sampling pattern constructed by singular value decomposition, f s (·) is a nonlinear sparse operator, which is to find the largest s elements and set the other elements to zero. μ is the step size that controls the update rate of each iteration. y is the value of each pixel. t is the subgraph reconstructed at the tth iteration. After multiple iterations, the subgraph set X∈R is obtained. MN×(mn) .

8. The pixel super-resolution method based on compressed sensing according to claim 3, characterized in that: The high-resolution phase image is specifically: Among them, phase represents the final high-resolution phase image, Image[] represents taking the imaginary part, Real[] represents taking the real part, arctan() represents calculating the inverse tangent, and unwrap() represents phase unwrapping.

Images