Method for realizing non-invasive large field of view imaging based on speckle correlation

Abstract

The present application belongs to the field of optical scattering imaging technology. When using the method based on optical memory effect to realize the imaging through scattering medium, once the size of the imaging target exceeds the range of the memory effect, it will usually lead to the failure of imaging, thereby limiting the field of view of the imaging system. The present application proposes a method of directly using correlation to expand the field of view, directly using the speckle autocorrelation of sub-target O i , the speckle cross-correlation calculation of sub-target O i and sub-target O i+1 , the relative position between sub-target image O i and O i+1 , the shift vector of the relative position between sub-target image O i and O i+1 , the correct flipping direction of sub-target image O i is determined by using the correlation between speckles, avoiding the estimation of point spread function PSF of the imaging system, thereby improving the efficiency. When the size of the target exceeds the range of the optical memory effect, the proposed method is used to more efficiently realize the non-invasive large field of view imaging through the scattering medium.

Description

Technical fields:

[0001] This invention belongs to the field of optical scattering imaging technology, specifically relating to a method for non-invasive large field-of-view imaging based on speckle correlation. Background technology:

[0002] Scattering is a widespread phenomenon in nature. Light is intensely scattered when it passes through biological tissue or is reflected by rough walls. In such cases, the image captured by the human eye or camera is a speckle image resembling white noise. Therefore, in traditional imaging systems, scattering effects are generally considered an obstacle to imaging.

[0003] However, speckle images contain information about targets hidden behind the scattering medium. Scattering imaging techniques include wavefront shaping, transfer matrix methods, optical phase conjugation, and speckle correlation imaging. Among these, non-invasive imaging techniques based on the memory effect have attracted widespread attention because they do not require invasive imaging systems.

[0004] However, due to the limitation of the field of view caused by the memory effect of the scattering medium, non-invasive imaging techniques based on the memory effect can usually only reconstruct single hidden targets smaller than the memory effect range. Once the target size exceeds the memory effect range, this method cannot recover the image information of the hidden target, resulting in reconstruction failure. Therefore, for non-invasive imaging techniques based on the memory effect to achieve broader development and wider application, it is urgent to develop a technical solution that overcomes the limitations of the memory effect and has a larger imaging field of view. Summary of the Invention:

[0005] This invention discloses a method for non-invasive large field-of-view imaging based on speckle correlation. Its purpose is to reveal a method that directly utilizes the correlation between speckles to find azimuth information, thereby expanding the non-invasive imaging field of view and avoiding the need to predict the point spread function (PSF) of the imaging system. Compared with existing non-invasive imaging methods, this method can achieve a more efficient expansion of the imaging field of view.

[0006] To achieve the above objectives, the present invention is implemented through the following solution:

[0007] A non-invasive, large-field-of-view imaging method that directly utilizes speckle correlation calculations to find relative orientations and achieve transmission through scattering media involves multiple scans of a fluorescent target O to segment it into multiple sub-targets O with similar characteristics. i From sub-target O i Sub-target image O' is obtained from speckle by phase retrieval. i Directly utilize sub-target O i speckle autocorrelation and sub-target O i With sub-target O i+1 speckle cross-correlation calculation of sub-target image O'i With O' i+1 The relative positions between them determine the sub-target image O' i With O' i+1 Shift vector of relative positions Determining sub-target image O' using the correlation between speckle patterns i The correct flipping direction is determined, and all sub-target images are superimposed according to the displacement vector to obtain the correctly stitched target O.

[0008] Furthermore, the displacement vector is calculated according to the following steps:

[0009] Step 1. After multiple scans, the fluorescent target O is segmented into multiple sub-targets with similarity. All sub-targets O are recorded using an area array photodetector. i Speckle image formed through the medium i ;

[0010] Step 2. Calculate sub-target O i Speckle image S i The autocorrelation, and the speckle image S i With sub-target O i+1 Speckle image S i+1 The cross-correlation;

[0011] Step 3. Using the speckle image S obtained in Step 2 i Autocorrelation and speckle images S i With speckle image S i+1 The cross-correlation of the speckle image S is used to determine the speckle image S. i Autocorrelation and speckle images S i With speckle image S i+1 The two-dimensional coordinates of the cross-correlation peaks can be used to calculate the sub-target image O' using the following formula. i With sub-target image O' i+1 Shift vector of relative positions

[0012]

[0013] Where ★ represents the correlation operation, and position{} represents the two-dimensional coordinates of the correlation peak position.

[0014] Furthermore, the method for determining the orientation of the sub-target image is as follows:

[0015] For sub-target image O i The four different orientation states of ' are used to calculate four different FFTs (O') using the following formula. i+1 The result is obtained by inverse fast Fourier transform (FFT). -1 (FFT(O' i+1 ))

[0016] FFT(O' i+1 ) = FFT(O' i )·FFT(S i+1 ) / FFT(S i )

[0017] Where FFT() represents Fast Fourier Transform;

[0018] The obtained results are compared with the phase-recovered sub-target image O' i+1 The shape is compared, and the image with the highest structural similarity to the sub-target image obtained from phase retrieval is identified as sub-target image O'. i The correct direction.

[0019] In summary, the invention has the following beneficial effects:

[0020] This invention first utilizes active illumination to sequentially scan a fluorescent target hidden behind a scattering medium, segmenting it into multiple fluorescent sub-targets with similar characteristics. An image of each fluorescent sub-target is reconstructed using a phase retrieval algorithm. Then, autocorrelation and cross-correlation calculations are directly performed on the speckle pattern formed by the fluorescent sub-targets through the scattering medium, avoiding the need to estimate the point spread function (PSF) of the imaging system. This allows for faster acquisition of the orientation of each sub-target image. Based on the obtained orientation, the images of each sub-target are stitched together, thereby achieving non-invasive, large-field-of-view imaging of the entire fluorescent target hidden in the scattering medium and exceeding the memory effect range. It is suitable for applications requiring imaging through scattering media such as frosted glass, diffuse reflection walls, and biological tissues, offering the advantage of more efficient non-invasive, large-field-of-view imaging.

[0021] This invention directly utilizes speckle autocorrelation and cross-correlation operations and phase recovery algorithms to achieve non-invasive large field-of-view imaging. The imaging method of this invention is non-invasive and does not require invasive "guide stars" or prior information. Compared with existing non-invasive imaging methods, this invention expands the non-invasive imaging field of view more efficiently. Attached Figure Description

[0022] Figure 1 Fluorescent targets hidden behind scattering media;

[0023] Figure 2 (a)-(h) Eight speckle images of fluorescent sub-targets, S1-S8;

[0024] Figure 3 (a)-(h) Eight sub-target images O'1-O'8 reconstructed from phase recovery;

[0025] Figure 4 (a) The relative positions of the eight reconstructed sub-target images O'1-O'8; (b) The restored overall target image O';

[0026] Figure 5 (a) A fluorescent target hidden behind a scattering medium;

[0027] Figure 6 (a) The relative positions of the 10 reconstructed sub-target images; (b) The result after magnifying the relative position information in (a);

[0028] Figure 7 (a) The recovered overall target image O'. Detailed Implementation Plan

[0029] The present invention will now be described in further detail with reference to the accompanying drawings.

[0030] It should be noted that, for ease of description, the descriptions of direction in the following text are consistent with the directions in the accompanying drawings, but they do not limit the structure of the present invention.

[0031] like Figures 1-7 As shown, this invention discloses a non-invasive large field-of-view imaging method that directly utilizes speckle correlation calculations to find relative orientations and achieve transmission through scattering media. The method involves multiple scans to illuminate a fluorescent target O, segmenting it into multiple sub-targets O with similar characteristics. i From sub-target O i Sub-target image O' is obtained from speckle by phase retrieval. i Directly utilize sub-target O i speckle autocorrelation and sub-target O i With sub-target O i+1 speckle cross-correlation calculation of sub-target image O' i With O' i+1 The relative positions between them determine the sub-target image O' i With O' i+1 Shift vector of relative positions Determining sub-target image O' using the correlation between speckle patterns i The correct flipping direction is determined, and all sub-target images are superimposed according to the displacement vector to obtain the correctly stitched target O.

[0032] The displacement vector is calculated using the following steps:

[0033] Step 1. After multiple scans, the fluorescent target O is segmented into multiple sub-targets with similarity. All sub-targets O are recorded using an area array photodetector. i Speckle image formed through the medium i When the fluorescent target O hidden behind the scattering medium exceeds the memory effect range, traditional non-invasive imaging methods will fail. In this case, the fluorescent target O can be illuminated by successive scanning, dividing it into multiple sub-targets with similarity. Based on the memory effect of the scattering medium, fluorescent sub-target O within the memory effect range i The autocorrelation of is equal to the autocorrelation of the corresponding speckle image, i.e., O i ★O i =S i ★S i S i To correspond to the fluorescent subtarget O i The speckle image. Therefore, using a phase retrieval algorithm, the speckle image S can be analyzed. i The fluorescent subtarget O was calculated from the autocorrelation. i Reconstructed sub-target image O' i .

[0034] Each sub-target O i All are within their respective memory effect ranges and satisfy O i ≈O i+1 Sub-targets whose coordinate points are similarly distributed satisfy this formula, thus allowing the sub-target O to be identified. i Phase retrieval of speckle data yields sub-target image O' i And can directly utilize O i speckle autocorrelation and O i With O i+1 The speckle cross-correlation is used to calculate the sub-target image O' i With O' i+1 The relative positions between them are used to avoid predicting the point spread function (PSF) of the imaging system, i.e.:

[0035] S i ★S i =(O i *PSF i )★(O i *PSF i )≈PSF i ★PSF i ,

[0036] S i ★S i+1 =(O i *PSF i )★(O i+1 *PSF i+1 )≈PSF i ★PSF i+1 ,

[0037] Where S i For O i speckle, S i+1 For O i+1 Scattered spots.

[0038] Step 2. Calculate sub-target O i Speckle image S i The autocorrelation, and the speckle image S i With sub-target O i+1 Speckle image S i+1 The cross-correlation;

[0039] Step 3. Using the speckle image S obtained in Step 2 i Autocorrelation and speckle images S i With speckle image S i+1 The cross-correlation of the speckle image S is used to determine the speckle image S. i Autocorrelation and speckle images S i With speckle image S i+1 The two-dimensional coordinates of the cross-correlation peaks can be used to calculate the sub-target image O' using the following formula. i With sub-target image O' i+1 Shift vector of relative positions

[0040]

[0041] Where ★ represents the correlation operation, and position{} represents the two-dimensional coordinates of the correlation peak position.

[0042] Sub-target image O' reconstructed using phase retrieval algorithm i It will lose its correct orientation information, and may have incorrect orientations such as left-right flipping, up-down flipping, or simultaneous left-right and up-down flipping. If its speckle image S i With adjacent speckle S i+1 The relative position information can be found by calculating the cross-correlation between them, indicating that the fluorescent subtarget O i With O i+1 The memory effect ranges overlap to some extent; by further utilizing the correlation between speckle patterns, sub-target images O can be found. i The correct flip direction.

[0043] The method for determining the orientation of a sub-target image is as follows:

[0044] Imaging systems that transmit through scattering media are linear systems within the memory effect range, while fluorescent subtargets O i and O i+1 Their respective memory effect ranges overlap to some extent, therefore, the location of the fluorescent subtarget O can be utilized. i and O i+1 The point spread function between the two is used to characterize the relationship between them, i.e., S. i =PSF*O' i and S i+1 =PSF*O' i+1, where * denotes convolution operation, and PSF represents the point spread function of the imaging system. Performing a Fast Fourier Transform on these two equations, i.e.

[0045] FFT(S i ) = FFT(PSF)·FFT(O' i )

[0046] FFT(S i+1 ) = FFT(PSF)·FFT(O' i+1 )

[0047] FFT() stands for Fast Fourier Transform.

[0048] For sub-target image O' i Four different orientation states are used to calculate four different FFTs (O') using the following formula. i+1 The result is obtained by inverse fast Fourier transform (FFT). -1 (FFT(O' i+1 ))

[0049] FFT(O' i+1 ) = FFT(O' i )·FFT(S i+1 ) / FFT(S i )

[0050] Where FFT() represents Fast Fourier Transform;

[0051] The obtained results are compared with the phase-recovered sub-target image O' i+1 The shape is compared, and the image with the highest structural similarity to the sub-target image obtained from phase retrieval is identified as sub-target image O'. i The correct direction. For similarity comparison, you can choose the SSIM calculation as the standard; the highest SSIM value indicates the highest similarity. Alternatively, you can choose other standards to compare similarities.

[0052] Example 1:

[0053] In this embodiment, the illumination source is controlled to sequentially scan and illuminate the fluorescent target O hidden behind the scattering medium, dividing it into eight similar fluorescent sub-targets. The fluorescent target O hidden behind the scattering medium is larger than the memory effect range of the scattering medium, while the fluorescent sub-targets O1-O8 illuminated in each scan are all within the memory effect range. Figure 1 As shown. The size of the memory effect range is indicated by the white solid circle, and the outline on the fluorescent target represents the scanning illumination. An area array photodetector acquires eight frames of speckle images S1-S8 formed by the reflected light from the fluorescent sub-target passing through the scattering medium. In this embodiment, an sCMOS camera is used as the area array photodetector. The results are as follows... Figure 2As shown in (a)-(h). Then, phase retrieval operations are performed on the acquired speckle images S1-S8 to reconstruct eight sub-target images O'1-O'8, as follows: Figure 3 As shown in (a)-(h). From the autocorrelation and cross-correlation calculations of the eight acquired speckle images S1-S8, the correct orientation information between the sub-target images O'1-O'8 is obtained, as follows: Figure 4 As shown in (a); where Next, following the method for determining the orientation of sub-target images in this invention, sub-target images O'1-O'8 with the correct flipping orientation are obtained. Finally, the sub-target images O'1-O'8 with the correct flipping orientation are... Figure 4 By shifting and superimposing the relative positions in (a), the overall target image O' can be obtained, such as... Figure 4 As shown in (b).

[0054] Example 2:

[0055] In this embodiment, the fluorescent target O hidden behind the scattering medium is as follows: Figure 5 As shown in (a), the illumination source is controlled to sequentially illuminate the fluorescent target O hidden behind the scattering medium, dividing it into 10 similar fluorescent sub-targets O1-O2. 10 ,like Figure 5 As shown in (a), a planar photodetector acquired 10 frames of fluorescent sub-targets O1-O. 10 The speckle image S1-S formed by reflected light passing through the scattering medium 10 In this embodiment, an sCMOS camera is used as the area array photodetector. Phase retrieval operations are performed on the acquired speckle images to reconstruct 10 sub-target images O'1-O'. 10 At this point, the acquired speckle images S1-S2 are processed according to the scanning illumination sequence. 10 Autocorrelation and cross-correlation operations are performed sequentially to obtain 10 sub-target images O'1-O'. 10 The correct relative position information between them, such as Figure 6 As shown in (a), its relative position information is magnified as follows: Figure 6 As shown in (b); where

[0056] According to the sub-target image orientation determination method of the present invention, a sub-target image O'1-O' with the correct flipping orientation is further obtained. 10 Finally, the sub-target image O'1-O' with the correct flip direction will be generated. 10 according to Figure 6 By shifting and superimposing the relative positions in (b), the overall target image O' can be reconstructed, as shown below. Figure 7 As shown in (a).

[0057] Example 3: Taking a CCD as an example, this example demonstrates a non-invasive large field-of-view super-resolution imaging method based on speckle correlation. In this example, the CCD is used as the area array photodetector, and everything else is the same as in Example 1.

[0058] Example 4: Taking an sCOMS area array photodetector as an example, this example demonstrates a non-invasive large field-of-view super-resolution imaging method based on speckle correlation. In this example, the area array photodetector is an sCOMS, and everything else is the same as in Example 1.

[0059] Example 5: Taking frosted glass as the scattering medium as an example, this example demonstrates a non-invasive large field-of-view imaging method based on speckle correlation. In this example, the scattering medium is frosted glass, and everything else is the same as in Example 1.

[0060] The above description is only a preferred embodiment of the present invention. The present invention is not limited to the contents of the above embodiments, and combinations of several specific embodiments can also achieve the purpose of the present invention.

Claims

1. A non-invasive large field-of-view imaging method that directly utilizes speckle correlation calculations to find relative orientation and achieve imaging through a scattering medium, characterized in that: Fluorescent target O was illuminated by multiple scans and segmented into multiple sub-targets O with similar characteristics. i From sub-target Sub-target images are obtained from speckle patterns through phase retrieval. Directly utilize sub-targets speckle autocorrelation and sub-targets With sub-targets speckle cross-correlation calculation of sub-target images and The relative positions between them determine the sub-target images. and Shift vector of relative positions Determining sub-target images by utilizing the correlation between speckle patterns. By correctly flipping the image in the correct direction, all sub-target images are superimposed according to the displacement vector to obtain the correctly stitched target O. This avoids the need to predict the point spread function (PSF) of the imaging system, allowing for faster acquisition of the orientation of each sub-target image. Based on the obtained orientation, the sub-target images are stitched together to achieve non-invasive large field-of-view imaging of the entire fluorescent target hidden in the scattering medium and exceeding the memory effect range.

2. The non-invasive large field-of-view imaging method according to claim 1, characterized in that, The displacement vector is calculated according to the following steps: Step 1. After multiple scans, the fluorescent target O is segmented into multiple sub-targets with similarity, and all sub-targets are recorded using an area array photodetector. speckle images formed through a medium ; Step 2. Calculate sub-objectives speckle image The autocorrelation and speckle images With sub-targets speckle image The cross-correlation; Step 3. Use the speckle image obtained in Step 2. Autocorrelation and speckle images With speckle image Cross-correlation to determine speckle images Autocorrelation and speckle images With speckle image The two-dimensional coordinates of the cross-correlation peaks can be used to calculate the subtarget image using the following formula. With sub-target image Shift vector of relative positions : in Indicates the relevant operations, Two-dimensional coordinates representing the location of the relevant peak.

3. The non-invasive large field-of-view imaging method according to claim 1 or 2, characterized in that, The method for determining the orientation of the sub-target image is as follows: For sub-target images The four different directional states are calculated using the following formula. The result is obtained by inverse fast Fourier transform. in, Indicates Fast Fourier Transform; The obtained results are compared with the sub-target image obtained by phase recovery. The shape is compared, and the image with the highest structural similarity to the sub-target image obtained from phase retrieval is identified as the sub-target image. The correct direction.

Images