Single-frame speckle image reconstruction method under coherent light wide-field illumination

Abstract

The application relates to a single-frame speckle image reconstruction method under coherent light wide-field illumination, and the method comprises the following steps: illuminating an imaging target; collecting backscattering light of the imaging target to obtain a speckle image and record an exposure time; calculating a first moment, a second moment, a third moment and a fourth moment; calculating a first moment and a second moment of a single scattering component and a first moment and a second moment of a multiple scattering component to obtain corresponding images; and if real-time imaging is not required, a maximum likelihood estimation method can be used to obtain the corresponding images. Compared with the prior art, the application has the advantages of reducing the influence of artifacts caused by multi-frame reconstruction, improving the reconstruction accuracy and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of optical imaging, and particularly to a single-frame speckle image reconstruction method under coherent light wide-field illumination. BACKGROUND

[0002] In optical imaging technology, image reconstruction is required after microscopic imaging and endoscopic imaging. In the existing technology, the paper Miao, P., Zhang, Y., Wang, C., & Tong, S. (2022). Random matrix description of dynamically backscattered coherent waves propagating in a wide-field-illuminated random medium. Applied Physics Letters, 120(4). A method for using multiple images under coherent light wide-field illumination to generate a single scattering first-order moment image, a single scattering second-order moment image, a multiple scattering first-order moment image, and a multiple scattering second-order moment image was established and applied to brain cortex imaging (Zhang, Y., Wang, C., Tong, S., & Miao, P. (2022). Separating single- and multiple-scattering components in laser speckle contrast imaging of tissue blood flow. Biomedical Optics Express, 13(5),2881-2895.), endoscopic imaging (Guo, Y., Weng, Y., Zhang, Y., Tong, S., Liu, Y., Lu, Z., & Miao, P. (2023). Random matrix-based laser speckle contrast imaging enables quasi-3D blood flow imaging in laparoscopic surgery. Biomedical Optics Express, 14(4), 1480-1493.), and also extended to 3D endoscopes (ZL202111142881.7). The above method requires the camera to continuously take multiple images of the same position to generate a single scattering first-order moment image, a single scattering second-order moment image, a multiple scattering first-order moment image, and a multiple scattering second-order moment image for image reconstruction, but this process is time-consuming and labor-intensive. In clinical applications, due to the autonomous movement of organs, the influence of the patient's breathing and heartbeat, and the shaking effect of the doctor holding the endoscope, the method of continuously taking multiple frames for reconstruction often has a large artifact effect and cannot accurately reflect the information such as tissue absorption and blood flow in the shallow and deep layers of the tissue. Summary of the Invention

[0003] The present application aims at reducing the influence of artifacts caused by multi-frame reconstruction and improving reconstruction accuracy, and provides a single-frame speckle image reconstruction method under coherent light wide-field illumination.

[0004] The present application aims at reducing the influence of artifacts caused by multi-frame reconstruction and improving reconstruction accuracy, and provides a single-frame speckle image reconstruction method under coherent light wide-field illumination.

[0005] A single-frame speckle image reconstruction method under coherent light wide-field illumination, the method comprising the following steps:

[0006] S1, illuminating an imaging target;

[0007] S2, collecting backscattered light of the imaging target to obtain a speckle image and recording an exposure time τ cG ;

[0008] S3, calculating a first moment μ1, a second moment μ2, a third moment μ3 and a fourth moment μ4 of all gray value data in each region of the speckle image, if real-time imaging is required, executing S4, otherwise executing S17;

[0009] S4, judging whether the exposure time τ cG satisfies τ cM ≤τ cG ≤0.1τ cS , wherein τ cS and τ cM are speckle de-coherence times of single scattering components and multiple scattering components respectively, if yes, executing S5, otherwise executing S9;

[0010] S5, calculating the first moment μ S of the single scattering components;

[0011] S6, calculating the second moment

[0012] S7, calculating the first moment μ M of the multiple scattering components;

[0013] S8, calculating the second moment of the multiple scattering components, and then executing S16;

[0014] S9, calculating a covariance matrix W;

[0015] S10, performing eigenvalue decomposition on the covariance matrix W to obtain a minimum eigenvalue λ min ;

[0016] S11, solving an unbiased parameter based on the second moment μ2, the third moment μ3 and the fourth moment μ4 obtained in S3

[0017] S12, calculating the second moment of the multiple scattering components

[0018] S13. Calculate the second-order moment of the single scattering component

[0019] S14. Calculate the first-order moment μ of the single scattering component S ;

[0020] S15. Calculate the first-order moment μ of the multiple scattering component M , then execute S16;

[0021] S16. Slide the spatial window used on the entire speckle image, and execute S3 to S15 for each spatial window data as a region to obtain a first-order moment image of the single scattering component, a second-order moment image of the single scattering component, a first-order moment image of the multiple scattering component, and a second-order moment image of the multiple scattering component;

[0022] S17. Using a maximum likelihood estimation method, obtain a first-order moment image of the single scattering component, a second-order moment image of the single scattering component, a first-order moment image of the multiple scattering component, and a second-order moment image of the multiple scattering component.

[0023] Furthermore, the first-order moment μ of the single scattering component in S5 S The calculation is:

[0024]

[0025] Among them, μ2 represents the second-order moment and μ3 represents the third-order moment.

[0026] Furthermore, the second-order moment of the single scattering component in S6 The calculation is:

[0027]

[0028] Among them, μ2 represents the second-order moment and μ3 represents the third-order moment.

[0029] Furthermore, the first-order moment μ of the multiple scattering component in S7 M The calculation is:

[0030]

[0031] Where μ1 represents the first-order moment.

[0032] Furthermore, the second-order moment of the multiple scattered components in S8 The calculation is:

[0033]

[0034] Furthermore, the unbiased parameter for:

[0035]

[0036] where λ min is the minimum eigenvalue, a1, a2, a3 are real numbers in the range of -1 to 1, p1 and p2 are real numbers in the range of 0.5 to 3.5, and μ4 represents the fourth-order moment.

[0037] Further, the second-order moment of the multiple scattering component in S12 is calculated as:

[0038]

[0039] where c = M / N, <y>is a constant between -5 and 5, and M and N represent the number of rows and columns of the image region.

[0040] Further, the specific steps of S17 are:

[0041] A cost function is constructed for the first moment of the single scattering component, the first moment of the multiple scattering component, and the second moment of the multiple scattering component, the cost function is minimized, an initial value is set, and then the first moment of the single scattering component, the first moment of the multiple scattering component, the second moment of the multiple scattering component, and the first moment of the single scattering component at the minimum of the cost function are obtained.

[0042] Further, the cost function is:

[0043]

[0044] wherein l represents the cost function, erfc is a Gaussian error function, z i represents all the gray values in the image region.

[0045] Further, the illumination adopts pulsed illumination synchronized with the camera exposure time or adopts wide-open illumination of continuous wave mode coherent light.

[0046] Compared with the prior art, the present application has the following beneficial effects:

[0047] The present application starts from the statistical characteristics of the single scattering and multiple scattering light intensity components in a single frame image, and realizes the extraction of the single scattering first moment image, the single scattering second moment image, the multiple scattering first moment image, and the multiple scattering second moment image based on the first moment, the second moment, the third moment, and the fourth moment of the light intensity of a single frame image. The image extracted based on a single frame image is reconstructed, the influence of artifacts caused by multi-frame reconstruction is reduced, and the reconstruction accuracy is improved. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 is a flowchart of the present application. DETAILED DESCRIPTION

[0049] The present application will be described in detail below in combination with the drawings and specific embodiments. The present embodiment is implemented on the premise of the technical solution of the present application, and detailed implementation modes and specific operation processes are given, but the protection scope of the present application is not limited to the following embodiments.

[0050] The present application proposes a single frame speckle image reconstruction method under coherent light wide field illumination, and the method comprises the following steps:

[0051] S1, illuminating an imaging target;

[0052] S2, collecting backscattered light of the imaging target to obtain a speckle image, and recording an exposure time τ cG ;

[0053] S3, calculate the first moment μ1, the second moment μ2, the third moment μ3 and the fourth moment μ4 of all the gray value data of each region in the speckle image, if real-time imaging is required, execute S4, if real-time imaging is not required, execute S17;

[0054] S4, judge whether the exposure time τ cG satisfies τ cM ≤τ cG ≤0.1τ cS , wherein τ cS and τ cM are the speckle de-coherence time of single scattering component and multiple scattering component respectively, if yes, execute S5, otherwise execute S9;

[0055] S5, calculate the first moment μ S of single scattering component;

[0056] S6, calculate the second moment μ of single scattering component;

[0057] S7, calculate the first moment μ M of multiple scattering component;

[0058] S8, calculate the second moment μ of multiple scattering component, and then execute S16;

[0059] S9, calculate the covariance matrix W;

[0060] S10, and perform eigenvalue decomposition on the covariance matrix W, to obtain the minimum eigenvalue λ min ;

[0061] S11, solve the unbiased parameter based on the second moment μ2, the third moment μ3 and the fourth moment μ4 obtained in S3;

[0062] S12, calculate the second moment μ of multiple scattering component;

[0063] S13, calculate the second moment μ of single scattering component;

[0064] S14, calculate the first moment μ S of single scattering component;

[0065] S15, calculate the first moment μ M of multiple scattering component, and then execute S16;

[0066] S16, on the whole speckle image, using the spatial window to slide, taking each spatial window data as a region to execute S3-S15 to obtain the first moment image of single scattering component, the second moment image of single scattering component, the first moment image of multiple scattering component and the second moment image of multiple scattering component;

[0067] S17, using the maximum likelihood estimation method to obtain the first moment image of single scattering component, the second moment image of single scattering component, the first moment image of multiple scattering component and the second moment image of multiple scattering component.

[0068] The flowchart of S1-S16 is shown in Figure 1 .

[0069] The specific steps of the application are:

[0070] 1. Using continuous wave (CW) mode coherent light (laser) to wide open illumination of the imaging target

[0071] 2. Using lens or objectives to collect the backscattering light of the imaging target. The camera records the speckle image I0(x,y) formed by the backscattering light through the exposure time τ cG , and τ cS and τ cM are the speckle de-coherence times of single scattering component and multiple scattering component respectively.

[0072] 3. For each MxN region (i.e. M rows and N columns, and M≤N) in the speckle image, the first moment (mean value) μ1, the second moment (variance) μ2, the third moment μ3 (skewness) and the fourth moment μ4 (kurtosis) are calculated using all the gray value data of the MxN region.

[0073] 4. When τ cG satisfies τ mM ≤τ cG ≤0.1τ cS , the calculation of steps 4-8 is executed. When τ cG does not satisfy the above condition, the calculation of steps 9-15 is executed.

[0074] 5. The first moment of single scattering component is calculated using the formula .

[0075] 6. The second moment of single scattering component is calculated using the formula .

[0076] 7. The first moment of multiple scattering component is calculated using the formula .

[0077] 8. The second moment of multiple scattering component is calculated using the formula .

[0078] 9. Compute the covariance matrix W = (I - μJ)(I - μJ) T where I is the intensity matrix of size M x N in the image I0 and J is the all-ones matrix.

[0079] 10. Perform eigenvalue decomposition of the W matrix, whose smallest eigenvalue is λ min .

[0080] 11. Solve the unbiased parameters using the second moment (variance) μ2, the third moment μ3 (skewness) and the fourth moment μ4 (kurtosis) where a1, a2, a3 are real numbers in the range -1 to 1. p1 and p2 are real numbers in the range 0.5 to 3.5.

[0081] 12. Compute the second moment of the multiple scattering component using the Marchenko-Pastur distribution and the Tracy-Widom distribution

[0082]

[0083] where: c = M / N, <y>is a constant between -5 and 5.

[0084] 13. Calculate the second moment of single scattering component using the formula

[0085] 14. Calculate the first moment of single scattering component using the formula S S

[0086] 15. Calculate the first moment of multiple scattering component using the formula M s

[0087] 16. Slide the MxN spatial window over the whole image I0with a step of 1 pixel. For each MxN spatial window data (with the center pixel position x0,y0), repeat the calculation of steps 3-15, and put the calculation result S into the position (x0,y0) of the first moment of single scattering component image; put the calculation result into the position (x0,y0) of the second moment of single scattering component image; put the calculation result M into the position (x0,y0) of the first moment of multiple scattering component image; put the calculation result into the position (x0,y0) of the first moment of multiple scattering component image.

[0088] The laser illumination mode of step 1 can also use pulsed illumination, in which case the illumination and camera exposure need to be synchronized in time.

[0089] For the case of M>N, the image matrix I can be transposed and the same procedure can be used.

[0090] If real-time imaging is not required, the following maximum likelihood estimation method can be used to replace steps 4-15 to reconstruct the first and second moment images of single and multiple scattering components. All the gray values {z i} of the MxN region, where i=1...MxN.

[0091] The detailed steps of S17 are:

[0092] (a) Construct the cost function l(μ S ,μ M ,σ M ) for the first moment of single scattering component, the first moment of multiple scattering component, and the second moment of multiple scattering component as:

[0093]

[0094] where erfc is the error function.

[0095] ​​​​​(b) Optimization method: Gradient descent algorithm or genetic algorithm can be used to minimize the cost function.

[0096] (c) Initial value: Random value is used. μ M and The initial value of z can also use the first and second moments of the {z i} data.

[0097] (d) The first moment of the single scattering component is calculated using the formula

[0098] In the present application, the calculation result of a single frame image is often poor in signal-to-noise ratio, resulting in a decrease in resolution. To improve the imaging quality, the MxN region can be rotated at different angles to extract data and calculate The calculation results of different angles are superimposed and averaged.

[0099] The application scenarios of the present application include:

[0100] (1) Neurosurgery microscope imaging:

[0101] The neurosurgery microscope introduces continuous wave mode 785nm laser illumination, and uses a monochrome camera to record 785nm backscattering light images with a frame rate higher than 100 frames per second and an exposure time less than 1 millisecond. For each frame recorded by the camera, a single scattering first moment image, a single scattering second moment image, a multiple scattering first moment image, and a multiple scattering second moment image are obtained by using steps 3-8. When real-time imaging is needed, GPU acceleration can be used.

[0102] (2) Endoscope imaging

[0103] The medical endoscope introduces pulsed mode 842nm laser illumination, and uses the near-infrared channel of the RGBIR camera to record 842nm backscattering light images with a frame rate of 60 frames per second and an exposure time of 16.67 milliseconds. For each frame recorded by the camera, a single scattering first moment image, a single scattering second moment image, a multiple scattering first moment image, and a multiple scattering second moment image are obtained by using steps 3 and 9-15. When real-time imaging is needed, FPGA acceleration can be used.

[0104] The above detailed the preferred embodiments of the present application. It should be understood that those skilled in the art can make many modifications and changes without creative labor based on the concept of the present application. Therefore, any technical solution obtained by logical analysis, reasoning or limited experiment based on the prior art according to the concept of the present application shall be within the protection scope determined by the claims.​< / y> < / y>

Claims

1. A method for reconstructing a single-frame speckle image under coherent light wide-field illumination, characterized in that: The method comprises the following steps: S1, illuminate the imaging target; S2. Collect the backscattered light of the imaging target to obtain the speckle image and record the exposure time τ cG ; S3, calculating the first-order moment μ1, the second-order moment μ2, the third-order moment μ3 and the fourth-order moment μ4 for all gray value data of each region in the speckle image. If real-time imaging is required, execute S4; if not, execute S17; S4. Determine exposure time τ cG Whether τ is satisfied cM ≤τ cG ≤0.1τ cS , where τ cS and τ cM are the speckle decoherence times of the single scattering component and the multiple scattering component respectively. If yes, execute S5, otherwise execute S9; S5. Calculate the first-order moment μ of the single scattering component S ; S6. Calculate the second-order moment of the single scattering component S7. Calculate the first-order moment μ of the multiple scattering component M ; S8. Calculate the second-order moment of the multiple scattered components Then execute S16; S9, calculating the covariance matrix W; S10, and perform eigenvalue decomposition on the covariance matrix W to obtain the minimum eigenvalue λ min ; S11, solve the unbiased parameters based on the second-order moment μ2, third-order moment μ3 and fourth-order moment μ4 obtained in S3 S12. Calculate the second-order moment of the multiple scattered components S13. Calculate the second-order moment of the single scattering component S14. Calculate the first-order moment μ of the single scattering component S ; S15. Calculate the first-order moment μ of the multiple scattering component M , then execute S16; S16. Slide the spatial window used on the entire speckle image, and execute S3 to S15 for each spatial window data as a region to obtain a first-order moment image of the single scattering component, a second-order moment image of the single scattering component, a first-order moment image of the multiple scattering component, and a second-order moment image of the multiple scattering component; S17. Using a maximum likelihood estimation method, obtain a first-order moment image of the single scattering component, a second-order moment image of the single scattering component, a first-order moment image of the multiple scattering component, and a second-order moment image of the multiple scattering component.

2. The method for reconstructing a single-frame speckle image under coherent light wide-field illumination according to claim 1, characterized in that: The first-order moment μ of the single scattering component in S5 S The calculation is: Among them, μ2 represents the second-order moment and μ3 represents the third-order moment.

3. The method for reconstructing a single-frame speckle image under coherent light wide-field illumination according to claim 2, characterized in that: The second-order moment of the single scattering component in S6 The calculation is: Among them, μ2 represents the second-order moment and μ3 represents the third-order moment.

4. The method for reconstructing a single-frame speckle image under coherent light wide-field illumination according to claim 3, characterized in that: The first-order moment μ of the multiple scattering component in S7 M The calculation is: Where μ1 represents the first-order moment.

5. The method for reconstructing a single-frame speckle image under coherent light wide-field illumination according to claim 4, characterized in that: Second-order moments of multiple scattering components in S8 The calculation is:

6. The method for reconstructing a single-frame speckle image under coherent light wide-field illumination according to claim 1, characterized in that: Unbiased parameters for: Among them, λ min is the minimum eigenvalue, a1, a2, and a3 are real numbers between -1 and 1, p1 and p2 are real numbers between 0.5 and 3.5, and μ4 represents the fourth-order moment.

7. The method for reconstructing a single-frame speckle image under coherent light wide-field illumination according to claim 6, characterized in that: Second-order moment of the multiple scattering component in S12 The calculation is: Where c = M / N, <y> is a constant between -5 and 5, and M and N represent the number of rows and columns of the image area.< / y> 8. The method for reconstructing a single-frame speckle image under coherent light wide-field illumination according to claim 1, characterized in that: The specific steps of S17 are: Construct a cost function about the first-order moment of the single scattering component, the first-order moment of the multiple scattering component, and the second-order moment of the multiple scattering component, minimize the cost function, set an initial value, and then obtain the first-order moment of the single scattering component, the first-order moment of the multiple scattering component, the second-order moment of the multiple scattering component, and the first-order moment of the single scattering component when the cost function is minimized.

9. The method for reconstructing a single-frame speckle image under coherent light wide-field illumination according to claim 8, characterized in that: The cost function is: Among them, l represents the cost function, erfc is the Gaussian error function, z i Represents all grayscale values ​​within the image area.

10. The method for reconstructing a single-frame speckle image under coherent light wide-field illumination according to claim 1, characterized in that: The illumination may be pulsed illumination synchronized with camera exposure time or wide illumination using continuous wave mode coherent light.

Images