Super memory effect range imaging method and device based on speckle pattern superposition and averaging

Abstract

The present application relates to a super memory effect range imaging method, device, computer equipment and storage medium based on speckle pattern superposition and averaging. The method comprises: obtaining speckle patterns of samples to be imaged collected at n different times, the samples to be imaged include stationary objects and moving objects in different memory effect ranges; performing superposition and averaging processing on each speckle pattern to obtain a first speckle component pattern of the stationary object; performing a differential operation on the obtained candidate speckle pattern and the first speckle component pattern to obtain a second speckle component pattern of the moving object; determining the target Fourier amplitude information of the stationary object and the moving object respectively according to the autocorrelation information of the first speckle component pattern and the second speckle component pattern; iteratively processing the Fourier amplitude information of each target through a phase recovery algorithm to complete the reconstruction of the stationary object and the moving object. The use of this method can improve the processing efficiency of super memory effect range imaging data.

Description

Technical Field

[0001] The present application relates to the field of computer imaging technology, and in particular to a super memory effect range imaging method, device, computer equipment and storage medium based on speckle pattern superposition and averaging. Background Art

[0002] Visible light imaging is an important way for people to obtain information about things in daily life. Using traditional optical imaging systems, we can clearly obtain information about objects. However, due to the uneven distribution of the refractive index of the environment, scattering is very common in nature. Light waves carrying object information will be severely disturbed after passing through scattering media, which brings great difficulties to imaging. With the application of scattering imaging technology in biomedicine, underwater detection, and atmospheric remote sensing, how to achieve high-resolution imaging through scattering media has become an important problem that needs to be solved in the field of optical imaging.

[0003] The basis of speckle correlation technology is the "optical memory effect". The "optical memory effect" means that within a certain angle, when the angle of the light source incident on the scattering medium is changed, the speckle pattern formed on the image plane remains unchanged, but the overall speckle pattern shifts. Speckle correlation technology is effective for a single object within the range of the "memory effect"; when there are two objects beyond the range of the memory effect behind the scattering medium, the autocorrelated information of the two objects overlaps and cannot be reconstructed to restore the shape of the object.

[0004] However, at present, in realizing beyond memory effect range imaging by using speckle pattern and multiple position point spread function deconvolution technology to realize beyond memory effect range reconstruction, there is a problem of low efficiency in data processing for beyond memory effect range imaging. Summary of the invention

[0005] Based on this, it is necessary to provide a super memory effect range imaging method, device, computer equipment and storage medium based on speckle pattern superposition and averaging, which can improve the processing efficiency of super memory effect range imaging data in response to the above technical problems.

[0006] A super memory effect range imaging method based on speckle pattern superposition, the method comprising:

[0007] Acquire speckle patterns of n samples to be imaged collected at different times, wherein the samples to be imaged include stationary objects and moving objects in different memory effect ranges;

[0008] Performing superposition and averaging processing on the speckle patterns to obtain a first speckle component map of the stationary object;

[0009] Acquiring a candidate speckle pattern of the sample to be imaged;

[0010] performing a difference operation on the candidate speckle pattern and the first speckle component pattern to obtain a second speckle component pattern of the moving object at a different position;

[0011] Calculating autocorrelation information of the first speckle component map and the second speckle component map;

[0012] Determine target Fourier amplitude information of the stationary object and the moving object respectively according to the autocorrelation information of the first speckle component image and the second speckle component image;

[0013] The Fourier amplitude information of each target is iteratively processed through a phase recovery algorithm to complete the reconstruction of the stationary object and the moving object.

[0014] In one embodiment, the step of acquiring the speckle patterns of the samples to be imaged collected at n different times includes:

[0015] Acquire the speckle components of the stationary object and the speckle components of the moving object at different positions collected at n different moments;

[0016] The speckle components of the stationary object and the speckle components of the moving object at different positions are superimposed to obtain n speckle patterns including the stationary object and the moving object.

[0017] In one embodiment, the step of performing superposition and averaging processing on the speckle patterns to obtain a first speckle component image of the stationary object includes:

[0018] Linearly superimposing n speckle patterns including the stationary object and the moving object to obtain a superimposed speckle pattern;

[0019] Based on the speckle intensity of the superimposed speckle pattern, the superimposed speckle pattern is averaged to obtain a first speckle component map of the stationary object.

[0020] In one embodiment, obtaining the candidate speckle pattern of the sample to be imaged includes:

[0021] Selecting a speckle pattern at any time as a candidate speckle pattern from the speckle patterns of the sample to be imaged acquired at any time among the n times;

[0022] The performing a difference operation on the candidate speckle pattern and the first speckle component pattern to obtain a second speckle component pattern of the moving object at a different position includes:

[0023] Acquire speckle information of the candidate speckle pattern and speckle information of the first speckle component pattern;

[0024] A second speckle component map of the moving object at a corresponding position is obtained by performing a difference operation on the speckle information of the candidate speckle map and the speckle information of the first speckle component map.

[0025] In one embodiment, the iterative processing of the Fourier amplitude information of each target based on the phase recovery algorithm to complete the reconstruction of the stationary object and the moving object includes:

[0026] Iteratively processing each target Fourier amplitude information based on the phase recovery algorithm until the amplitude constraint of the phase recovery algorithm is satisfied, thereby obtaining the Fourier phase information of the stationary object and the Fourier phase information of the moving object;

[0027] The reconstruction of the stationary object and the moving object is completed according to the Fourier phase information of the stationary object and the Fourier phase information of the moving object.

[0028] In one of the embodiments, determining the target Fourier amplitude information of the stationary object and the moving object respectively according to the autocorrelation information of the first speckle component image and the second speckle component image includes:

[0029] Determining the autocorrelation information of the stationary object according to the autocorrelation information of the first speckle component image;

[0030] determining the autocorrelation information of the moving object according to the autocorrelation information of the second speckle component image;

[0031] By performing Fourier transform on the autocorrelation information of the stationary object and the autocorrelation information of the moving object, target Fourier amplitude information of the stationary object and the moving object are obtained respectively.

[0032] In one embodiment, the different memory effect ranges include different axial memory effect ranges or different lateral memory effect ranges.

[0033] A super memory effect range imaging device based on speckle pattern superposition, the device comprising:

[0034] A first acquisition module is used to acquire speckle patterns of n samples to be imaged collected at different times, wherein the samples to be imaged include stationary objects and moving objects in different memory effect ranges;

[0035] A superposition and averaging processing module, used for performing superposition and averaging processing on each of the speckle patterns to obtain a first speckle component map of the stationary object;

[0036] A second acquisition module, used to acquire a candidate speckle pattern of the sample to be imaged;

[0037] a differential operation module, configured to perform a differential operation on the candidate speckle pattern and the first speckle component pattern to obtain a second speckle component pattern of the moving object at a different position;

[0038] a calculation module, configured to calculate the autocorrelation information of the first speckle component image and the second speckle component image;

[0039] a determination module, configured to respectively determine target Fourier amplitude information of the stationary object and the moving object according to the autocorrelation information of the first speckle component image and the second speckle component image;

[0040] The reconstruction module is used to iteratively process the Fourier amplitude information of each target through a phase recovery algorithm to complete the reconstruction of the stationary object and the moving object.

[0041] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0042] Collecting speckle patterns of samples to be imaged at n different moments, wherein the samples to be imaged include stationary objects and moving objects in different memory effect ranges;

[0043] Performing superposition and averaging processing on the speckle patterns to obtain a first speckle component map of the stationary object;

[0044] Acquiring a candidate speckle pattern of the sample to be imaged;

[0045] performing a difference operation on the candidate speckle pattern and the first speckle component pattern to obtain a second speckle component pattern of the moving object at a different position;

[0046] Calculating autocorrelation information of the first speckle component map and the second speckle component map;

[0047] Determine target Fourier amplitude information of the stationary object and the moving object respectively according to the autocorrelation information of the first speckle component image and the second speckle component image;

[0048] The Fourier amplitude information of each target is iteratively processed through a phase recovery algorithm to complete the reconstruction of the stationary object and the moving object.

[0049] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0050] Collecting speckle patterns of samples to be imaged at n different moments, wherein the samples to be imaged include stationary objects and moving objects in different memory effect ranges;

[0051] Performing superposition and averaging processing on the speckle patterns to obtain a first speckle component map of the stationary object;

[0052] Acquiring a candidate speckle pattern of the sample to be imaged;

[0053] performing a difference operation on the candidate speckle pattern and the first speckle component pattern to obtain a second speckle component pattern of the moving object at a different position;

[0054] Calculating autocorrelation information of the first speckle component map and the second speckle component map;

[0055] Determine target Fourier amplitude information of the stationary object and the moving object respectively according to the autocorrelation information of the first speckle component image and the second speckle component image;

[0056] The Fourier amplitude information of each target is iteratively processed through a phase recovery algorithm to complete the reconstruction of the stationary object and the moving object.

[0057] The above-mentioned super memory effect range imaging method, device, computer equipment and storage medium based on speckle pattern superposition collect speckle patterns of samples to be imaged including stationary objects and moving objects at different times; perform superposition and averaging processing on the speckle patterns at different times to obtain a first speckle component map of the stationary object; and perform a differential operation on the candidate speckle pattern of the sample to be imaged acquired at any time and the first speckle component map to obtain a second speckle component map of the moving object at any time; that is, by superposition and averaging and differential processing on the speckle patterns at different times, the speckle component maps of the stationary object and the moving object are respectively obtained, the autocorrelation information of the target object is obtained according to the autocorrelation information of the speckle component maps of the stationary object and the moving object, and the phase recovery algorithm is used to complete the reconstruction of the target object, without the need to obtain prior information in advance, reducing the amount of data processing, shortening the time consumption of reconstructing the object and improving the data processing efficiency of super memory effect range imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a diagram of the application environment of a super memory effect range imaging method based on speckle pattern superposition and averaging in one embodiment;

[0059] Figure 2 It is a schematic flow chart of a super memory effect range imaging method based on speckle pattern superposition and averaging in one embodiment;

[0060] Figure 3 It is a schematic diagram of superposition and averaging of multiple speckle patterns in one embodiment;

[0061] Figure 4 is a flow chart of a super memory effect range imaging method based on speckle pattern superposition and averaging in another embodiment;

[0062] Figure 5 This is a diagram of an application scenario of super memory effect range imaging based on speckle pattern superposition and averaging in one embodiment;

[0063] Figure 6 It is a schematic diagram of the effect of super memory effect range imaging based on speckle pattern superposition and averaging in one embodiment;

[0064] Figure 7 is a structural block diagram of a super memory effect range imaging device based on speckle pattern superposition and averaging in one embodiment;

[0065] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0067] The super memory effect range imaging method based on speckle pattern superposition and averaging provided in this application can be applied to Figure 1 In the application environment shown. The first terminal 102 communicates with the second terminal 104 through a network. The first terminal 102 obtains the speckle images of the samples to be imaged collected by the second terminal 104 at n different times, and the samples to be imaged include stationary objects and moving objects with different memory effect ranges; the first terminal 102 performs superposition and average processing on each speckle image to obtain a first speckle component image of the stationary object; obtains a candidate speckle image of the sample to be imaged; performs a differential operation on the candidate speckle image and the first speckle component image to obtain a second speckle component image of the moving object at different positions; calculates the autocorrelation information of the first speckle component image and the second speckle component image; determines the target Fourier amplitude information of the stationary object and the moving object respectively according to the autocorrelation information of the first speckle component image and the second speckle component image; iteratively processes the target Fourier amplitude information through a phase recovery algorithm to complete the reconstruction of the stationary object and the moving object. The first terminal 102 can be, but not limited to, various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the second terminal 104 can be, but not limited to, respective image acquisition devices. Optionally, the second terminal 104 may be integrated on the first terminal 102 .

[0068] In one embodiment, Figure 2 As shown in the figure, a super memory effect range imaging method based on speckle pattern superposition and averaging is provided, and the method is applied to Figure 1 Taking the first terminal in the example as an example, the method includes the following steps:

[0069] Step 202 : collecting speckle patterns of samples to be imaged at n different moments, where the samples to be imaged include stationary objects and moving objects in different memory effect ranges.

[0070] Among them, different memory effect ranges are different optical memory effect ranges, including different depths of field (different axial memory effect ranges) or different fields of view (different lateral memory effect ranges), different axial memory effect ranges include exceeding the axial memory effect range and not exceeding the axial memory effect range; different lateral memory effect ranges include exceeding the lateral memory effect range and not exceeding the lateral memory effect range; the sample to be imaged refers to the existence of stationary objects and moving objects after the light beam carrying object information passes through the scattering medium, and the size of the object in the sample to be imaged does not exceed the memory effect range of the scattering medium, but the distance exceeds the axial memory effect range or the lateral memory effect range; the number of stationary objects and moving objects behind the scattering medium of the sample to be imaged can be, but is not limited to, 1. In this embodiment, an example is given in which there is one stationary object and one moving object behind the scattering medium of the sample to be imaged.

[0071] Specifically, a light beam carrying information of at least two objects is scattered by a scattering medium, and a speckle pattern is received by a camera on the image plane. The coordinates of the two objects are different in space, and different point spread functions are transmitted from the object plane to the image plane. That is, when there are static objects and moving objects hidden behind the scattering medium of the sample to be imaged, the speckle patterns of n samples to be imaged at different times are collected by an image acquisition device (for example, a camera), and the speckle pattern collected at each time is a linear superposition of the speckle patterns of the static object and the moving object. For example, after the sample to be imaged scatters the medium, there are two objects hidden behind the medium on the object plane, namely, a static object O1 and a moving object O2. The speckle pattern I of the sample to be imaged collected by the camera at n different times is a linear superposition of the speckle patterns I1 and I2 formed by the static object O1 and the moving object O2. The speckle pattern I can be expressed as:

[0072] I=I O1 +I O2 =S1(x1,y1)*O1(x1,y1)+S2(x2,y2)*O2(x2,y2)

[0073] Among them, the coordinates Q1(x1, y1) and Q2(x2, y2) are the coordinates of objects O1 and O2, * represents the convolution operation, S is the point spread function (PSF) of the scattering system, S1(x1, y1) is the point spread function of the stationary object O1 at the coordinate (x1, y1), S2(x2, y2) is the point spread function of the moving object O2 at the coordinate (x2, y2), I O1 The speckle pattern of the stationary object O1 at a certain moment, I O2 It is the speckle pattern of the moving object O2 at a certain moment.

[0074] Step 204: Perform superposition and averaging processing on the speckle patterns to obtain a first speckle component map of the stationary object.

[0075] Specifically, the camera collects n frames of speckle patterns of the sample to be imaged at different times. When the speckle patterns are collected, the moving object O2 whose coordinates change can be considered to be different from the object O2 at the previous position. The speckle patterns formed by the stationary objects in the n frames of speckle patterns collected by the camera have the same distribution, while the speckle distributions of the moving objects are different, that is, they can be considered to be formed by different objects. The speckle patterns are superimposed on n frames to obtain the speckle pattern I. sum , the superimposed speckle pattern I sum , including the speckle patterns of n stationary objects O1 and the speckle patterns formed by n moving objects O2 at different positions, and superimposing the multi-frame speckle patterns I sum An averaging operation is applied to obtain the first speckle component of the stationary object.

[0076] Among them, the speckle pattern collected by the camera is superimposed to obtain the speckle pattern I sum It can be expressed as:

[0077]

[0078] Among them, S1 and S2 are corresponding point spread functions, O1 is a stationary object, and O2 is a moving object.

[0079] Overlay of multiple speckle patterns sum Apply the averaging operation to obtain the sub-first speckle component I of the stationary object ave It can be expressed as:

[0080]

[0081] C is the noise, that is, the first speckle component image I ave It is approximately equal to the speckle pattern I1 formed by the object O1 after the scattering medium. In the averaging operation, the speckle pattern of the moving object O2 can be approximated as noise.

[0082] like Figure 3 FIG. 1 is a schematic diagram of the superposition and averaging of n-frame speckle patterns in an embodiment. FIG. (a) is a schematic diagram of the position of a moving object at different times. FIG. n ) indicates that at t n Object distribution at each moment, letter S represents a stationary object, letter Z represents a moving object, Figure (b) collects n frames of speckle patterns at different moments, during the speckle acquisition process, the speckle patterns formed by the stationary objects in the n frames of speckle patterns collected by the camera have the same distribution, while the speckle patterns formed by the moving objects have different distributions, Figure (c) superposition of n frames of speckle patterns, Figure (d) the speckle pattern generated by the average superposition, and the speckle pattern generated by the stationary object S is obtained.

[0083] Step 206: Obtain a candidate speckle pattern of the sample to be imaged.

[0084] The candidate speckle pattern is the speckle pattern of the sample to be imaged at any time captured by the camera. For example, the c-th frame speckle pattern captured by the camera is obtained, that is, the candidate speckle pattern, and c is any time.

[0085] Step 208: performing a difference operation on the candidate speckle pattern and the first speckle component pattern to obtain a second speckle component pattern of the moving object at a different position.

[0086] Specifically, the c-th frame speckle image I captured by the camera is obtained. c The speckle pattern of the cth frame contains the speckle patterns of the stationary object O1 and the moving object O2. The speckle pattern of the cth frame and the first speckle component pattern are differentially operated to obtain the speckle component pattern of the moving object O2 at time c. It can be expressed as is the point spread function at time c; wherein the position of the moving object O2 is different at different times.

[0087] Step 210: Calculate autocorrelation information of the first speckle component image and the second speckle component image.

[0088] Specifically, the autocorrelation information of the first speckle component image and the autocorrelation information of the second speckle component image are calculated respectively; the autocorrelation information of the first speckle component image can be expressed as: That is, the autocorrelation information of the stationary object O1 can be determined by calculating the autocorrelation information of the first speckle component image; the autocorrelation information of the second speckle component image can be expressed as: It is shown that the autocorrelation information of the moving object O2 can be determined by calculating the autocorrelation information of the second speckle component image.

[0089] Step 212: Determine the target Fourier amplitude information of the stationary object and the moving object respectively according to the autocorrelation information of the first speckle component image and the second speckle component image.

[0090] Specifically, based on the speckle correlation technology, the target Fourier amplitude information of the stationary object and the moving object is determined respectively according to the autocorrelation information of the first speckle component image and the second speckle component image, that is, the target Fourier amplitude information of the stationary object O1 can be expressed as: The target Fourier amplitude information of the moving object O2 can be expressed as:

[0091] Step 214, iteratively process the Fourier amplitude information of each target through a phase recovery algorithm to complete the reconstruction of the stationary object and the moving object.

[0092] Specifically, the Fourier amplitude information of each target is iteratively processed based on the phase recovery algorithm until the amplitude constraint of the phase recovery algorithm, that is, the dynamic constraint of the real space, is met, and the Fourier phase information of the stationary object and the Fourier phase information of the moving object are obtained; based on the Fourier phase information of the stationary object and the Fourier phase information of the moving object, the reconstruction of the stationary object and the moving object is completed.

[0093] In the above-mentioned super memory effect range imaging method based on speckle pattern superposition and averaging, speckle patterns of samples to be imaged including a stationary object and a moving object are collected at different times; the speckle patterns at different times are superimposed and averaged to obtain a first speckle component map of the stationary object; and a candidate speckle pattern of the sample to be imaged acquired at any time is differentially operated with the first speckle component map to obtain a second speckle component map of the moving object at any time; that is, by superimposing, averaging and differentially processing the speckle patterns at different times, the speckle patterns of the two objects hidden in the scattering medium are separated from the speckle patterns acquired by the camera at different times, the autocorrelation information of the target object is obtained according to the autocorrelation information of the speckle component maps of the two objects, and the hidden object is reconstructed by a phase recovery algorithm, thereby improving the imaging effect, and no prior information needs to be acquired in advance, reducing the amount of data processing, shortening the time consumption of reconstructing the object, and thus improving the data processing efficiency of super memory effect range imaging.

[0094] In another embodiment, Figure 4 As shown, a super memory effect range imaging method based on speckle pattern superposition and averaging is provided, the sample to be imaged includes a stationary object and a moving object, the stationary object and the moving object in the sample to be imaged are beyond the axial memory effect range or beyond the lateral memory effect range, and the method is applied to Figure 1 Taking the first terminal in the example as an example, the method includes the following steps:

[0095] Step 402: Obtain speckle components of a stationary object and speckle components of a moving object at different positions collected at n different times.

[0096] Among them, the speckle components of the moving object at different times are different, that is, the point spread functions corresponding to the position coordinates at different times are also different.

[0097] Step 404: superimpose the speckle components of the stationary object and the speckle components of the moving object at different positions to obtain n speckle patterns including the stationary object and the moving object.

[0098] Specifically, the speckle components of the stationary object and the speckle components of the moving object at different positions collected at n different times are superimposed, that is, the moving object forms different speckle components through the scattering medium in different memory effect ranges in the axial or lateral direction, and multiple exposures are made at different times to obtain n speckle patterns containing the stationary object and the moving object. Among them, the position coordinates of the moving object O2 change with time, and the speckle pattern I2 formed by the moving object through the scattering medium is also different, which can be expressed as: I O2 =O2*S(t,r), S() is the point spread function, t represents time, and r represents the coordinates of the object; the position coordinates of the stationary object O1 do not change with time, and the speckle pattern I1 formed by the stationary object passing through the scattering medium is also the same, which is I O1 The speckle pattern collected by the camera also changes according to the change of the object O2 speckle pattern. The speckle patterns containing static objects and moving objects at different times can be expressed as:

[0099] I n =I O1 +I O2 =O1*S1+O2(t n ,r n )*S2(t n ,r n )

[0100] Step 406: Perform superposition and averaging processing on each speckle pattern to obtain a first speckle component map of the stationary object.

[0101] Specifically, the speckle patterns containing the stationary object and the moving object at n different moments are linearly superimposed to obtain a superimposed speckle pattern; based on the speckle intensity of the superimposed speckle pattern, the superimposed speckle pattern is averaged to obtain a first speckle component map of the stationary object; that is, the more positions the moving object O2 traverses in space, the less speckle pattern information of the moving object O2 in the superimposed and averaged speckle pattern, and the more speckle pattern information formed by the stationary object O1. After the speckle patterns are superimposed, the speckle intensity of the stationary object will far exceed the speckle intensity of the moving object. Based on the speckle intensity of the superimposed speckle pattern, the superimposed speckle pattern is averaged to obtain the first speckle component map of the stationary object, which can be expressed as:

[0102] Step 408 : Selecting a speckle pattern at any time as a candidate speckle pattern from the speckle patterns of the sample to be imaged acquired at any time among the n time periods.

[0103] Step 410: performing a difference operation on the candidate speckle pattern and the first speckle component pattern to obtain a second speckle component pattern of the moving object at a different position.

[0104] Specifically, speckle information of the candidate speckle pattern and speckle information of the first speckle component pattern are obtained; and a second speckle component pattern of the moving object at a corresponding moment is obtained by performing a differential operation on the speckle information of the candidate speckle pattern and the speckle information of the first speckle component pattern.

[0105] Step 412: Calculate autocorrelation information of the first speckle component image and the second speckle component image.

[0106] Step 414: Determine the target Fourier amplitude information of the stationary object and the moving object respectively according to the autocorrelation information of the first speckle component image and the second speckle component image.

[0107] Specifically, the autocorrelation information of the stationary object is determined according to the autocorrelation information of the first speckle component image; the autocorrelation information of the moving object is determined according to the autocorrelation information of the second speckle component image; and the target Fourier amplitude information of the stationary object and the moving object are obtained respectively by performing Fourier transform on the autocorrelation information of the stationary object and the autocorrelation information of the moving object.

[0108] In other words, based on the speckle correlation technology, the target Fourier amplitude information of the stationary object and the moving object is determined according to the autocorrelation information of the first speckle component image and the second speckle component image, that is, the target Fourier amplitude information of the stationary object O1 can be expressed as: The target Fourier amplitude information of the moving object O2 can be expressed as:

[0109] Step 416, iteratively process the Fourier amplitude information of each target through a phase recovery algorithm to complete the reconstruction of the stationary object and the moving object.

[0110] Specifically, the Fourier amplitude information of each target is iteratively processed based on the phase recovery algorithm until the amplitude constraint of the phase recovery algorithm, that is, the dynamic constraint of the real space, is met, and the Fourier phase information of the stationary object and the Fourier phase information of the moving object are obtained; according to the Fourier phase information of the stationary object and the Fourier phase information of the moving object, the reconstruction of the stationary object and the moving object is completed. That is, according to the phase recovery algorithm, a random phase is first added to the given object surface amplitude as the initial complex amplitude g0(x,y). The initial complex amplitude is obtained by performing a two-dimensional Fourier transform to obtain the spectrum surface function G n (u,v). Preserve the spectrum surface function G n The phase part of (u,v), the amplitude of the spectrum surface function is replaced by the target Fourier amplitude information obtained by speckle, and the new spectrum surface function G' is obtained n (u,v). Spectral function G' n (u,v) performs a two-dimensional inverse Fourier transform to obtain g' in real space n (x,y), by g' n(x, y) implements dynamic constraints for iterative processing until the dynamic constraints are met, and the Fourier phase information of the stationary object and the moving object is obtained, and the reconstruction of the stationary object and the moving object is completed. The dynamic constraints can be expressed as:

[0111]

[0112] Where Γ represents the set of dynamic constraints and β represents the parameter that controls the convergence of the algorithm.

[0113] The following is an application scenario of super memory effect range imaging based on speckle pattern superposition in one embodiment. Figure 5 As shown, (a) represents an optical scattering module, which includes a laser, frosted glass, a transmission mirror, a spatial light modulator, a scattering medium and a camera, (b) represents the information of two objects loaded on the spatial light modulator, and (c) represents the positions of the letters S and Z when collecting speckle patterns at different times.

[0114] The light source is a laser with a wavelength of 532nm. At the same time, a rapidly rotating frosted glass and a collimating transmission mirror are used to modulate the initial light of the laser to ensure that the light is incident vertically on the input surface as parallel spatial incoherent light. Two polarizers are located before and after the spatial light modulator (Spatial Light Modulator, SLM) to achieve amplitude modulation. Multiple frames of images are loaded on the SLM to simulate objects moving at different times. The light beam is scattered by the scattering medium through the SLM carrying object information. A camera is placed on the image plane behind the frosted glass to record the corresponding speckle image. The camera is used to collect speckle patterns at ten times, and the 10 frames of speckle patterns are superimposed to obtain I sum The stationary object S has the same distribution in the speckle collected in each frame. After the speckle patterns are superimposed, the speckle intensity of the stationary object S will far exceed the speckle intensity formed by the moving object Z at each position.

[0115] The speckle pattern obtained by averaging the speckle patterns after multi-frame superposition is equal to the speckle pattern generated by the stationary object S. The speckle pattern collected by the camera at any time is the superposition of the speckle patterns of object S and object Z. The speckle pattern generated by the moving object Z can be obtained by performing difference processing between any frame of speckle pattern and the average speckle pattern after multi-frame superposition.

[0116] The speckle patterns of stationary objects and moving objects can be obtained by superposition, averaging and difference, respectively. The autocorrelation of the object can be calculated by speckle correlation technology. By calculating the autocorrelation of the speckle pattern, the Fourier amplitude information of the unknown moving object Z and the stationary object S can be finally obtained. The moving object and the stationary object can be successfully reconstructed by the phase recovery algorithm, such as Figure 6As shown in the figure, (a) is the true value image of a stationary object, (b) is the true value image of a moving object, (c) is the reconstructed image of a stationary object, and (d) is the reconstructed image of a moving object. That is, reconstruction of different memory effect ranges is completed without prior information, and the algorithm reduces the reconstruction time; the moving object and the stationary object are reconstructed using the phase recovery algorithm, and the reconstructed object has a high similarity with the real object, which improves the imaging effect.

[0117] The above-mentioned super memory effect range imaging based on speckle pattern superposition and averaging obtains the speckle component of the stationary object by superposition and averaging of multiple frames of speckle patterns, then obtains the speckle component of the moving object at different positions by performing differential processing on the superposition and averaging result and the speckle pattern collected at any time, obtains the autocorrelation information of the target object according to the autocorrelation information of the speckle component images of the two objects, and reconstructs the hidden object by using a phase recovery algorithm, thereby improving the imaging effect; there is no need to invade the inside of the scattering medium to measure the point spread function in advance, and there is no need to obtain the information of some objects, thereby reducing the amount of data processing, shortening the time consumption of reconstructing the object, and thus improving the data processing efficiency of super memory effect range imaging.

[0118] It should be understood that although Figure 2 , Figure 4 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Figure 2 , Figure 4 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0119] In one embodiment, Figure 7 As shown, a super memory effect range imaging device based on speckle pattern superposition and averaging is provided, comprising: a first acquisition module 702, a superposition and averaging processing module 704, a second acquisition module 706, a differential operation module 708, a calculation module 710, a determination module 712 and a reconstruction module 714, wherein:

[0120] The first acquisition module 702 is used to acquire speckle patterns of n samples to be imaged collected at different times, where the samples to be imaged include stationary objects and moving objects in different memory effect ranges. The different memory effect ranges include different axial memory effect ranges or different lateral memory effect ranges.

[0121] The superposition and averaging processing module 704 is used to perform superposition and averaging processing on each speckle pattern to obtain a first speckle component map of a stationary object.

[0122] The second acquisition module 706 is used to acquire a candidate speckle pattern of the sample to be imaged.

[0123] The differential operation module 708 is used to perform a differential operation on the candidate speckle pattern and the first speckle component pattern to obtain a second speckle component pattern of the moving object at a different position.

[0124] The calculation module 710 is configured to calculate the autocorrelation information of the first speckle component image and the second speckle component image.

[0125] The determination module 712 is configured to determine target Fourier amplitude information of the stationary object and the moving object respectively according to the autocorrelation information of the first speckle component image and the second speckle component image.

[0126] The reconstruction module 714 is used to iteratively process the Fourier amplitude information of each target through a phase recovery algorithm to complete the reconstruction of the stationary object and the moving object.

[0127] In the above-mentioned super memory effect range imaging device based on speckle pattern superposition and averaging, speckle patterns of samples to be imaged including a stationary object and a moving object are collected at different times; the speckle patterns at different times are superimposed and averaged to obtain a first speckle component map of the stationary object; and a candidate speckle pattern of the sample to be imaged acquired at any time is differentially operated with the first speckle component map to obtain a second speckle component map of the moving object at any time; that is, by superimposing, averaging and differentially processing the speckle patterns at different times, the speckle patterns of the two objects hidden in the scattering medium are separated from the speckle patterns acquired by the camera at different times, the autocorrelation information of the target object is obtained according to the autocorrelation information of the speckle component maps of the two objects, and the hidden object is reconstructed by a phase recovery algorithm, thereby improving the imaging effect, and no prior information needs to be acquired in advance, reducing the amount of data processing, shortening the time consumption of reconstructing the object, and thus improving the data processing efficiency of super memory effect range imaging.

[0128] In another embodiment, a super memory effect range imaging device based on speckle pattern superposition and averaging is provided, which includes, in addition to a first acquisition module 702, a superposition and averaging processing module 704, a second acquisition module 706, a differential operation module 708, a calculation module 710, a determination module 712 and a reconstruction module 714: a superposition module, an averaging module and

[0129] The first acquisition module 702 is also used to acquire the speckle components of the stationary object and the speckle components of the moving object at different positions collected at n different moments.

[0130] The convolution processing module is used to perform convolution processing according to each position coordinate and the point spread function to obtain the speckle patterns containing stationary objects and moving objects at n different moments.

[0131] The superposition and averaging processing module 704 includes a superposition module and an averaging module. The superposition module linearly superimposes n speckle patterns including stationary objects and moving objects to obtain a superimposed speckle pattern.

[0132] It is also used to superimpose the speckle components of a stationary object and the speckle components of a moving object at different positions to obtain n speckle patterns containing the stationary object and the moving object.

[0133] The averaging module is used to perform averaging processing on the superimposed speckle pattern based on the speckle intensity of the superimposed speckle pattern to obtain a first speckle component map of the stationary object.

[0134] The second acquisition module 706 is further configured to select a speckle pattern at any time as a candidate speckle pattern from the speckle patterns of the sample to be imaged acquired at any time among the n acquired time periods.

[0135] The differential operation module 708 is further used to obtain speckle information of the candidate speckle pattern and speckle information of the first speckle component map; by performing a differential operation on the speckle information of the candidate speckle pattern and the speckle information of the first speckle component map, a second speckle component map of the moving object at the corresponding position is obtained.

[0136] The determination module 712 is further configured to determine the autocorrelation information of the stationary object according to the autocorrelation information of the first speckle component image; and determine the autocorrelation information of the moving object according to the autocorrelation information of the second speckle component image.

[0137] The Fourier transform processing module is used to obtain target Fourier amplitude information of the stationary object and the moving object respectively by performing Fourier transform on the autocorrelation information of the stationary object and the autocorrelation information of the moving object.

[0138] The reconstruction module 714 is also used to iteratively process the Fourier amplitude information of each target based on the phase recovery algorithm until the amplitude constraint of the phase recovery algorithm is met, thereby obtaining the Fourier phase information of the stationary object and the Fourier phase information of the moving object; and complete the reconstruction of the stationary object and the moving object based on the Fourier phase information of the stationary object and the Fourier phase information of the moving object.

[0139] In one embodiment, the speckle component of a stationary object is obtained by superimposing and averaging multiple frames of speckle patterns, and then the speckle components of a moving object at different times are obtained by performing differential processing on the superimposed and averaged result and the speckle pattern collected at any time. The autocorrelation information of the target object is obtained according to the autocorrelation information of the speckle component images of the two objects, and the hidden object is reconstructed by a phase recovery algorithm, thereby improving the imaging effect; there is no need to invade the inside of the scattering medium to measure the point spread function in advance, and there is no need to obtain the information of part of the object, thereby reducing the amount of data processing, shortening the time consumption of reconstructing the object, and thus improving the processing efficiency of super memory effect range imaging data.

[0140] For the specific definition of the super memory effect range imaging device based on the super memory effect super memory effect range imaging method based on the super memory effect ...

[0141] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a super memory effect range imaging method based on speckle pattern superposition and averaging is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0142] Those skilled in the art will understand that Figure 8The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0143] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0144] Obtaining speckle patterns of n samples to be imaged collected at different times, where the samples to be imaged include stationary objects and moving objects with different memory effect ranges;

[0145] Performing superposition and averaging processing on each speckle pattern to obtain a first speckle component image of a stationary object;

[0146] Obtaining a candidate speckle pattern of a sample to be imaged;

[0147] Performing a differential operation on the candidate speckle pattern and the first speckle component pattern to obtain a second speckle component pattern of the moving object at a different position;

[0148] Calculating autocorrelation information of the first speckle component image and the second speckle component image;

[0149] Determine target Fourier amplitude information of a stationary object and a moving object respectively according to the autocorrelation information of the first speckle component image and the second speckle component image;

[0150] The Fourier amplitude information of each target is iteratively processed through the phase recovery algorithm to complete the reconstruction of stationary objects and moving objects.

[0151] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0152] Obtaining the speckle components of the stationary object and the speckle components of the moving object at different positions collected at n different times;

[0153] The speckle components of the stationary object and the speckle components of the moving object at different positions are superimposed to obtain n speckle patterns containing the stationary object and the moving object.

[0154] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0155] Linearly superimpose n speckle patterns including stationary objects and moving objects to obtain a superimposed speckle pattern;

[0156] Based on the speckle intensity of the superimposed speckle pattern, the superimposed speckle pattern is averaged to obtain a first speckle component map of the stationary object.

[0157] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0158] Selecting a speckle pattern at any time as a candidate speckle pattern from the speckle patterns of the sample to be imaged acquired at any time among the n time periods;

[0159] Performing a differential operation on the candidate speckle pattern and the first speckle component pattern to obtain a second speckle component pattern of the moving object at different times includes:

[0160] Acquiring speckle information of the candidate speckle pattern and speckle information of the first speckle component pattern;

[0161] By performing a differential operation on the speckle information of the candidate speckle pattern and the speckle information of the first speckle component pattern, a second speckle component pattern of the moving object at a corresponding moment is obtained.

[0162] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0163] Based on the phase recovery algorithm, the Fourier amplitude information of each target is iteratively processed until the amplitude constraint of the phase recovery algorithm is met, and the Fourier phase information of the stationary object and the Fourier phase information of the moving object are obtained;

[0164] The reconstruction of the stationary object and the moving object is completed according to the Fourier phase information of the stationary object and the Fourier phase information of the moving object.

[0165] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0166] determining the autocorrelation information of the stationary object according to the autocorrelation information of the first speckle component image;

[0167] determining the autocorrelation information of the moving object according to the autocorrelation information of the second speckle component image;

[0168] By performing Fourier transform on the autocorrelation information of the stationary object and the autocorrelation information of the moving object, the target Fourier amplitude information of the stationary object and the moving object are obtained respectively.

[0169] In one embodiment, when the processor executes the computer program, it also implements the following:

[0170] The different memory effect ranges include different axial memory effect ranges or different lateral memory effect ranges.

[0171] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0172] Obtaining speckle patterns of n samples to be imaged collected at different times, where the samples to be imaged include stationary objects and moving objects with different memory effect ranges;

[0173] Performing superposition and averaging processing on each speckle pattern to obtain a first speckle component image of a stationary object;

[0174] Obtaining a candidate speckle pattern of a sample to be imaged;

[0175] Performing a differential operation on the candidate speckle pattern and the first speckle component pattern to obtain a second speckle component pattern of the moving object at a different position;

[0176] Calculating autocorrelation information of the first speckle component image and the second speckle component image;

[0177] Determine target Fourier amplitude information of a stationary object and a moving object respectively according to the autocorrelation information of the first speckle component image and the second speckle component image;

[0178] The Fourier amplitude information of each target is iteratively processed through the phase recovery algorithm to complete the reconstruction of stationary objects and moving objects.

[0179] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0180] Obtaining the speckle components of the stationary object and the speckle components of the moving object at different positions collected at n different times;

[0181] The speckle components of the stationary object and the speckle components of the moving object at different positions are superimposed to obtain n speckle patterns containing the stationary object and the moving object.

[0182] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0183] Linearly superimpose n speckle patterns including stationary objects and moving objects to obtain a superimposed speckle pattern;

[0184] Based on the speckle intensity of the superimposed speckle pattern, the superimposed speckle pattern is averaged to obtain a first speckle component map of the stationary object.

[0185] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0186] Selecting a speckle pattern at any time as a candidate speckle pattern from the speckle patterns of the sample to be imaged acquired at any time among the n time periods;

[0187] Performing a differential operation on the candidate speckle pattern and the first speckle component pattern to obtain a second speckle component pattern of the moving object at a different position includes:

[0188] Acquiring speckle information of the candidate speckle pattern and speckle information of the first speckle component pattern;

[0189] By performing a differential operation on the speckle information of the candidate speckle pattern and the speckle information of the first speckle component pattern, a second speckle component pattern of the moving object at a corresponding moment is obtained.

[0190] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0191] Based on the phase recovery algorithm, the Fourier amplitude information of each target is iteratively processed until the amplitude constraint of the phase recovery algorithm is met, and the Fourier phase information of the stationary object and the Fourier phase information of the moving object are obtained;

[0192] The reconstruction of the stationary object and the moving object is completed according to the Fourier phase information of the stationary object and the Fourier phase information of the moving object.

[0193] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0194] determining the autocorrelation information of the stationary object according to the autocorrelation information of the first speckle component image;

[0195] determining the autocorrelation information of the moving object according to the autocorrelation information of the second speckle component image;

[0196] By performing Fourier transform on the autocorrelation information of the stationary object and the autocorrelation information of the moving object, the target Fourier amplitude information of the stationary object and the moving object are obtained respectively.

[0197] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0198] The different memory effect ranges include different axial memory effect ranges or different lateral memory effect ranges.

[0199] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0200] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0201] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A super memory effect range imaging method based on speckle pattern superposition and averaging, characterized in that: The method comprises: Acquire speckle patterns of n samples to be imaged collected at different times, wherein the samples to be imaged include stationary objects and moving objects in different memory effect ranges; Performing superposition and averaging processing on the speckle patterns to obtain a first speckle component map of the stationary object; Acquiring a candidate speckle pattern of the sample to be imaged; performing a difference operation on the candidate speckle pattern and the first speckle component pattern to obtain a second speckle component pattern of the moving object at a different position; Calculating autocorrelation information of the first speckle component map and the second speckle component map; Determine the target Fourier amplitude information of the stationary object and the moving object respectively according to the autocorrelation information of the first speckle component image and the second speckle component image; The Fourier amplitude information of each target is iteratively processed through a phase recovery algorithm to complete the reconstruction of the stationary object and the moving object.

2. The method according to claim 1, characterized in that The step of obtaining the speckle patterns of the samples to be imaged collected at n different times comprises: Acquire the speckle components of the stationary object and the speckle components of the moving object at different positions collected at n different moments; The speckle components of the stationary object and the speckle components of the moving object at different positions are superimposed to obtain n speckle patterns including the stationary object and the moving object.

3. The method according to claim 2, characterized in that The step of performing superposition and averaging processing on the speckle patterns to obtain a first speckle component image of the stationary object comprises: Linearly superimposing n speckle patterns including the stationary object and the moving object to obtain a superimposed speckle pattern; Based on the speckle intensity of the superimposed speckle pattern, the superimposed speckle pattern is averaged to obtain a first speckle component map of the stationary object.

4. The method according to claim 1, characterized in that: The obtaining of a candidate speckle pattern of the sample to be imaged comprises: Selecting a speckle pattern at any time as a candidate speckle pattern from the speckle patterns of the sample to be imaged acquired at any time among the n times; The performing a difference operation on the candidate speckle pattern and the first speckle component pattern to obtain a second speckle component pattern of the moving object at a different position includes: Acquire speckle information of the candidate speckle pattern and speckle information of the first speckle component pattern; A second speckle component map of the moving object at a corresponding position is obtained by performing a difference operation on the speckle information of the candidate speckle map and the speckle information of the first speckle component map.

5. The method according to claim 1, characterized in that: The iterative processing of the Fourier amplitude information of each target based on the phase recovery algorithm to complete the reconstruction of the stationary object and the moving object includes: Iteratively processing each target Fourier amplitude information based on the phase recovery algorithm until the amplitude constraint of the phase recovery algorithm is satisfied, thereby obtaining the Fourier phase information of the stationary object and the Fourier phase information of the moving object; The reconstruction of the stationary object and the moving object is completed according to the Fourier phase information of the stationary object and the Fourier phase information of the moving object.

6. The method according to claim 1, characterized in that The determining the target Fourier amplitude information of the stationary object and the moving object respectively according to the autocorrelation information of the first speckle component image and the second speckle component image includes: Determining the autocorrelation information of the stationary object according to the autocorrelation information of the first speckle component image; determining the autocorrelation information of the moving object according to the autocorrelation information of the second speckle component image; By performing Fourier transform on the autocorrelation information of the stationary object and the autocorrelation information of the moving object, target Fourier amplitude information of the stationary object and the moving object are obtained respectively.

7. The method according to claim 1, characterized in that The different memory effect ranges include different axial memory effect ranges or different lateral memory effect ranges.

8. A super memory effect range imaging device based on speckle pattern superposition and averaging, characterized in that: The device comprises: A first acquisition module is used to acquire speckle patterns of samples to be imaged collected at n different times, wherein the samples to be imaged include stationary objects and moving objects in different memory effect ranges; A superposition and averaging processing module, used for performing superposition and averaging processing on each of the speckle patterns to obtain a first speckle component map of the stationary object; A second acquisition module, used to acquire a candidate speckle pattern of the sample to be imaged; a differential operation module, configured to perform a differential operation on the candidate speckle pattern and the first speckle component pattern to obtain a second speckle component pattern of the moving object at a different position; a calculation module, configured to calculate the autocorrelation information of the first speckle component image and the second speckle component image; a determination module, configured to determine target Fourier amplitude information of the stationary object and the moving object respectively according to the autocorrelation information of the first speckle component image and the second speckle component image; The reconstruction module is used to iteratively process the Fourier amplitude information of each target through a phase recovery algorithm to complete the reconstruction of the stationary object and the moving object.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images