Non-vision-field high-resolution scattering imaging method based on speckle fingerprints
Through a non-sight high-resolution scattering imaging method based on speckle fingerprint, the target image is reconstructed using frequency domain filtering and phase recovery algorithms, the problem of difficulty in imaging motion targets in the prior art is solved, and high-resolution imaging and tracking are realized, which is suitable for multi-field applications.
Patent Information
- Application Number
- CN202510476903.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-16
- Publication Date
- 2025-08-19
AI Technical Summary
Existing non-field-of-sight imaging technology is difficult to achieve high-resolution imaging and tracking of motion goals, especially in large-scale scenarios, multiple scattering and long-distance conditions, and requires prior knowledge and high-cost equipment.
The non-sight high-resolution scattering imaging method based on speckle fingerprint is adopted. By building a non-sight imaging optical system to capture speckle images, frequency domain filtering and speckle overlap calculation are carried out, and the target image is reconstructed in combination with the phase recovery algorithm to achieve high-resolution imaging of the moving target.
Without prior knowledge and invasive operations, the calculation process is simplified, the accuracy of image recovery is improved, and high-resolution imaging and tracking can be achieved in complex scattering environments. It is suitable for fields such as biomedical, artificial intelligence and astronomy.
Smart Images
Figure CN120510044A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of scattering imaging in the field of optical imaging, and in particular to a non-line-of-sight high-resolution scattering imaging method based on speckle fingerprints. Background Art
[0002] Non-line-of-sight imaging technologies, such as imaging through scattering media and imaging around corners, have important applications in many fields, including biomedicine, artificial intelligence, autonomous driving, and military rescue. Scattering media, such as frosted glass, clouds, biological tissue, and walls, scatter light into a random speckle distribution, making imaging difficult. Non-line-of-sight imaging uses the effective signal carried by indirect scattered light to reconstruct the image of an object obscured by an obstacle. The detector collects photons carrying target information scattered back by multiple surfaces and restores the image of the obscured scene through calculation. Based on the differences in imaging principles, existing methods can be roughly divided into two categories: imaging methods based on interference information and imaging methods based on photon time-of-flight.
[0003] Non-line-of-sight imaging methods based on interferometric information are categorized into two types: speckle pattern and spatial coherence. The speckle pattern method utilizes the speckle memory effect to reconstruct images, which helps to obtain information about obscured objects. However, its limited field of view makes it difficult to apply to large-scale scenarios. Spatial coherence methods are still in their infancy and are expensive, and reconstruction based solely on spatial coherence information has not yet been achieved. Both methods require strong signals, but multiple scattering leads to signal attenuation, limiting their application in scenarios with strong scattering and long distances.
[0004] Non-line-of-sight imaging methods based on photon time-of-flight are primarily suitable for angular, low-resolution imaging of large targets. However, practical applications of this method still face challenges such as weak effective photon signals due to multiple scattered light, long acquisition times, and noise. In multi-target scenes, differences in reflectivity lead to weak signals, affecting reconstruction results. The assumption of scene information within the field of view is impractical in practical applications. Furthermore, the high cost of required equipment, such as ultrafast pulsed lasers and high-temporal-resolution detectors, is a significant obstacle.
[0005] The above methods are difficult to directly image and track non-line-of-sight moving targets, and have problems such as insufficient resolution and limited number of scattering medium layers, which limit their scope of application and effectiveness. Summary of the Invention
[0006] Purpose of the invention: To address the problems in the prior art, the present invention proposes a non-line-of-sight high-resolution scattering imaging method based on speckle fingerprint, which can achieve high-resolution imaging and tracking of moving targets hidden in corners without the need for prior knowledge.
[0007] Technical solution: The non-line-of-sight high-resolution scatter imaging method based on speckle fingerprint described in the present invention specifically includes the following steps:
[0008] (1) Based on the constructed non-line-of-sight imaging optical system, a series of speckle images are captured;
[0009] (2) randomly selecting two speckle images from a series of speckle images taken in step (1), performing frequency domain filtering on the two speckle images, obtaining an envelope from the low-frequency information, and obtaining a spatially normalized speckle from the high-frequency information;
[0010] (3) obtaining speckle overlap information from the normalized speckle pattern, which reflects the target overlap, i.e., the target autocorrelation value; obtaining the corresponding target displacement information by calculating the cross-correlation between the corresponding envelopes of the two speckle patterns according to the envelope obtained in step (2); and calculating the corresponding position of the autocorrelation value on the target autocorrelation; combining the obtained autocorrelation value and its corresponding position into a pixel on the target autocorrelation;
[0011] (4) Through loop iteration, a complete target autocorrelation image containing all pixels is constructed, and the target image O is reconstructed through the phase recovery algorithm.
[0012] Furthermore, the non-line-of-sight imaging optical system in step (1) magnifies the speckle reflected from the visible plane and includes a light source, a target to be measured, a multi-layer scattering medium, a visible surface, a lens, and a camera; light emitted by the light source irradiates the target surface, and the object light reflected or transmitted by the object propagates along the optical axis direction, passes through the scattering medium layer, and the multi-layer scattering structure causes strong multiple scattering of the object light; the scattered object light reaches a visible surface arranged on the side of the system and at a certain angle to the light propagation direction, and is diffusely reflected on the surface to form a speckle pattern carrying target information; the speckle pattern is focused by a lens located in its reflection direction and then captured by a camera arranged on the imaging surface to obtain a two-dimensional intensity image reflecting the target spatial information.
[0013] Furthermore, the non-field-of-view imaging optical system in step (1) also includes a light shielding plate, which is placed perpendicular to the visible surface and located on one side of the light propagation path, and is used to block direct light from the light source and stray light outside the system, preventing non-object light components from directly entering the imaging path or irradiating the visible surface, causing interference signals.
[0014] Furthermore, the shooting of a series of speckle images in step (1) is specifically that the target moves relative to the medium, and the medium always remains stationary. During the movement of the target, a series of speckle images I1, ..., I n .
[0015] Furthermore, the scattering medium is frosted glass.
[0016] Furthermore, the visible surface is a white polystyrene board.
[0017] Furthermore, the implementation process of step (2) is as follows:
[0018] From a series of speckle images I1,…,I n Take any two speckle images I i and I j , the relative displacement of the target in the two speckle images is (Δx ij ,Δy ij );The speckle image I i and I j Input into the frequency domain low-pass filter, and obtain the envelope E contained in the low-frequency information through frequency domain low-frequency filtering i and E j , using the original speckle image I i and I j Divide by the envelope E i and E j , we get the spatial normalized speckle S i and S j , specifically expressed as follows:
[0019] S i (ξ,η)=∫∫O(x,y)I S (x,y)s(ξ,η;x,y)dxdy
[0020] S j (ξ,η;Δx ij ,Δy ij )=∫∫O(x-Δx ij ,y-Δy ij )I S (x,y)s(ξ,η;x,y)dxdy
[0021] In the formula, (x-Δx ij ,y-Δy ij ) represents the position of the target at the time of the i-th detection relative to the time of the j-th detection; when i = j, Δx ij =Δy ij =0.
[0022] Furthermore, the process of obtaining the overlap of the target in step (3) is as follows:
[0023] Assume that the impulse responses of different point sources with a relative distance greater than the spatial coherence length do not overlap with each other, that is, each point on the target surface has its corresponding speckle fingerprint, and the speckle fingerprints are independent of each other; extract two spatial normalized speckle patterns S by point multiplication i and S i The speckle overlap between
[0024] M ij (ξ,η;Δx ij ,Δy ij )=Si (ξ,η)S i (ξ,η;Δx ij ,Δy ij )=∫∫O(x,y)O(x-Δx ij ,y-Δy ij )I S (x,y) 2 s(ξ,η;x,y) 2 dxdy
[0025] In the above result, only the target displacement (Δx ij ,Δy ij ) The speckle fingerprint formed by the points in the overlapping area before and after is retained; the target overlap information is obtained by summing, and the specific formula is as follows:
[0026] M ij (Δx ij ,Δy ij )=∫∫M ij (ξ,η;Δx ij ,Δy ij )dξdη=∫∫O(x,y)O(x-Δx ij ,y-Δy ij )∫∫[I S (x,y)s(ξ,η;x,y)] 2 dξdηdxdy
[0027] Among them, the second integral ∫∫[I S (x,y)s(ξ,η;x,y)] 2 dξdη is the total energy of each speckle fingerprint, which is proportional to the energy of the point light source that forms the corresponding speckle fingerprint; assuming that in a small area containing an object, the intensity distribution I S (x,y) is uniform, so the second integral is considered to be an approximately equal constant, and the target overlap is:
[0028]
[0029] in, is a related operation, and const is a constant.
[0030] Furthermore, the process of combining the obtained autocorrelation values and their corresponding positions into a pixel on the target autocorrelation in step (3) is as follows:
[0031] For two envelopes E i and E j Perform cross-correlation operation and determine the relative displacement of the target (Δx ij ,Δyij ), get the autocorrelation value M ij At the position on the target autocorrelation, assign the target autocorrelation value to the position, that is, get a pixel of the target autocorrelation:
[0032] C ij (Δx ij ,Δy ij )=M ij (Δx ij ,Δy ij ).
[0033] Furthermore, the implementation process of step (4) is as follows:
[0034] By changing i and j from 1 to n in sequence, all pixels of the target O autocorrelation are constructed to obtain the complete target autocorrelation:
[0035]
[0036] According to the obtained target autocorrelation, the target image is reconstructed by the phase recovery algorithm.
[0037] Beneficial effects: Compared with the existing technology, the beneficial effects of the present invention are: the present invention can achieve high-resolution imaging and tracking of moving targets in non-line-of-sight modes such as corner type and multi-layer medium type, without the need for prior knowledge and invasive operations, simplifying the calculation process, improving the accuracy of image restoration, and providing a new and effective means to solve the non-line-of-sight imaging problem; the present invention is both innovative and practical, and has broad application prospects in biomedicine, artificial intelligence, astronomy and other fields, and is expected to provide strong technical support for research and application in related fields. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 Schematic diagram of the non-line-of-sight scattering imaging system proposed in the present invention;
[0039] Figure 2 Schematic diagram of target autocorrelation constructed for the present invention;
[0040] Figure 3 This is the target image reconstructed using the present invention. DETAILED DESCRIPTION
[0041] The present invention will be further described in detail below with reference to the accompanying drawings.
[0042] The present invention proposes a non-line-of-sight high-resolution scattering imaging method based on speckle fingerprint, which specifically includes the following steps:
[0043] Step 1: Take a series of speckle images while the target O is moving.
[0044] Build as Figure 1The non-line-of-sight imaging optical system shown in the figure primarily consists of a light source, a target to be measured, a multi-layer scattering medium, a visible surface or diffusely reflecting plane, a light shield, a lens, and a camera. It is capable of magnifying speckle reflected from a visible surface. The light source can be light emitted by the target itself, such as fluorescence from a fluorescent target or incoherent light from a target on a display screen. The corresponding emitted light is the object light. Light from the light source illuminates the target surface. The object light, reflected or transmitted by the object, propagates along the optical axis and sequentially passes through two layers of scattering medium composed of frosted glass. This multi-layer scattering structure produces strong multiple scattering of the object light. The scattered object light then reaches a visible surface (a white polystyrene plate) positioned to the side of the system at a certain angle to the light propagation direction. Diffuse reflection occurs on this surface, forming a speckle pattern that carries target information. This speckle pattern is focused by a lens positioned in the direction of reflection and captured by a camera positioned on the imaging surface, producing a two-dimensional intensity image reflecting the target's spatial information. To suppress stray light interference and improve the imaging system's signal-to-noise ratio, a light shield is placed near the visible surface. Positioned perpendicular to the surface and to one side of the light propagation path, it blocks direct light from the light source as well as stray light from outside the system, preventing non-object light components from directly entering the imaging path or illuminating the visible surface, potentially causing interference. The shield's layout is coordinated with the optical path, ensuring it doesn't affect the propagation of effective imaging light while effectively shielding extraneous light. This improves image purity and speckle contrast, enhancing the stability and accuracy of subsequent image reconstruction.
[0045] During the detection process, a transmissive target or a reflective target (three lines, 25 μm in width, 25 μm in spacing, and 125 μm in height) is hidden behind a scattering medium or blocked by a baffle. The target movement is controlled by a two-axis motorized translation stage or moves spontaneously. The target moves relative to the medium, which remains stationary. During the target movement, a series of speckle images I1,…,I n .
[0046] Step 2: Randomly select two speckle images from the series of speckle images taken in step 1, perform frequency domain filtering on the two speckle images, obtain the envelope from the low-frequency information, and obtain the spatially normalized speckle from the high-frequency information.
[0047] A series of magnified speckle images I1,…,I are continuously recorded by the camera. n Take any two speckle images I i and I j The first speckle pattern described is the speckle pattern before the target moves, and the second speckle pattern is the speckle pattern after the target moves. The displacement of the target is expressed as (Δx ij ,Δy ij ). The speckle image I i and I jInput into the frequency domain low-pass filter, filter to get the envelope E i and E j , using the original speckle image I i and I j Divide by the envelope E i and E j , we get the spatial normalized speckle S i and S j , which is expressed as follows:
[0048] S i (ξ,η)=∫∫O(x,y)I S (x,y)s(ξ,η;x,y)dxdy
[0049] S j (ξ,η;Δx ij ,Δy ij )=∫∫O(x-Δx ij ,y-Δy ij )I S (x,y)s(ξ,η;x,y)dxdy
[0050] In the formula (x-Δx ij ,y-Δy ij ) represents the position of the target at the time of the i-th detection relative to the j-th detection. When i=j, Δx ij =Δy ij =0;
[0051] Step 3: Obtain speckle overlap information from the normalized speckle pattern obtained in step 2. According to the speckle fingerprint principle, speckle overlap reflects the target overlap, that is, the target autocorrelation value. Based on the envelope obtained in step 2, the corresponding target displacement information is obtained by calculating the cross-correlation of the corresponding envelopes of the two speckle patterns. This information reflects the corresponding position of the calculated autocorrelation value on the target autocorrelation. The obtained autocorrelation value and its corresponding position are combined to form a pixel on the target autocorrelation.
[0052] Calculate the target displacement (Δx ij ,Δy ij ) The degree of speckle overlap between the two speckles before and after:
[0053] M ij (ξ,η;Δx ij ,Δy ij )=S i (ξ,η)S i (ξ,η;Δx ij ,Δy ij )=∫∫O(x,y)O(x-Δx ij ,y-Δy ij )I S(x,y) 2 s(ξ,η;x,y) 2 dxdy
[0054] In a small area containing the target, the intensity distribution I S (x,y) is almost uniform. Therefore, although the speckle fingerprints usually have different distributions, their total energy is almost equal. So the second integral of the above formula can be approximated to a constant, and the sum of overlapping speckles is:
[0055]
[0056] in is a related operation, and const is a constant.
[0057] According to the above formula, any two speckles contribute a value C to the autocorrelation of the target. ij , whose position depends on the relative displacement of the target (Δx ij ,Δy ij ). To change the displacement, you can use envelope E i and E j Because their distributions are similar, only the displacement changes statistically. The two envelopes are cross-correlated. According to the definition of the correlation operation, the position of the maximum value relative to the cross-correlation center is (Δx ij ,Δy ij ), so we get M ij The position of the value on the target autocorrelation (Δx ij ,Δy ij ). Combining the above results, the target autocorrelation pixel C can be constructed ij (Δx ij ,Δy ij ).
[0058] Step 4: Through loop iteration, a complete target autocorrelation image containing all pixels is constructed, and the target image O is reconstructed using the phase recovery algorithm.
[0059] By changing i and j from 1 to n in sequence, a complete autocorrelation image of the target is constructed:
[0060]
[0061] Figure 2 The target autocorrelation diagram is given; finally, the target image O is recovered from the constructed autocorrelation by the iterative phase recovery algorithm. Figure 3 The reconstructed target image is given. Figure 3As shown in the figure, after the target is reconstructed using the method proposed in the present invention, the target details in the obtained image are clear, the edge contours are clear, the texture features are rich, and there are no obvious artifacts or speckle noise in the image, indicating that the phase recovery process has good stability and no problems such as error accumulation or information loss occur. The reconstructed image has a high contrast, indicating that the autocorrelation calculation and normalization processing process are reasonable and can effectively enhance the image quality. The target shape contour is accurately restored, and there is no blurring or distortion in the edge area, further verifying the convergence and accuracy of the phase recovery algorithm. By comparing with the original target, the reconstructed image performs well in terms of shape fidelity and structural restoration, and can accurately restore the target's geometric and texture information. Compared with traditional methods, the method of the present invention has significant advantages in imaging quality, noise resistance and target detail retention, verifying its applicability and effectiveness in complex scattering environments. In summary, the reconstructed image results can fully demonstrate that the method of the present invention has high-precision target recovery capabilities, the reconstruction process is stable and reliable, and provides a data basis and performance guarantee for further optimization of the algorithm.
[0062] The present invention has been described in detail above with reference to specific embodiments. However, these descriptions should not be construed as limiting the present invention. Those skilled in the art will appreciate that various equivalent substitutions, modifications, or improvements may be made to the technical solutions and implementations of the present invention without departing from the spirit and scope of the present invention, all of which fall within the scope of the present invention. The scope of protection of the present invention shall be determined by the appended claims.
Claims
1. A non-line-of-sight high-resolution scattering imaging method based on speckle fingerprint, characterized in that: The following steps are involved: (1) Based on the constructed non-line-of-sight imaging optical system, a series of speckle images are captured; (2) randomly selecting two speckle images from a series of speckle images taken in step (1), performing frequency domain filtering on the two speckle images, obtaining an envelope from the low-frequency information, and obtaining a spatially normalized speckle from the high-frequency information; (3) obtaining speckle overlap information from the normalized speckle pattern, which reflects the target overlap, i.e., the target autocorrelation value; obtaining the corresponding target displacement information by calculating the cross-correlation between the corresponding envelopes of the two speckle patterns according to the envelope obtained in step (2); and calculating the corresponding position of the autocorrelation value on the target autocorrelation; combining the obtained autocorrelation value and its corresponding position into a pixel on the target autocorrelation; (4) Through loop iteration, a complete target autocorrelation image containing all pixels is constructed, and the target image O is reconstructed through the phase recovery algorithm.
2. The non-line-of-sight high-resolution scattering imaging method based on speckle fingerprint according to claim 1, characterized in that: The non-line-of-sight imaging optical system of step (1) forms a magnified image of speckles reflected from a visible plane, and includes a light source, a target to be measured, a multi-layer scattering medium, a visible surface, a lens, and a camera; light emitted by the light source is irradiated onto the surface of the target object, and object light reflected or transmitted by the object propagates along the optical axis and passes through the scattering medium layer. The multi-layer scattering structure will produce strong multiple scattering of the object light; The scattered object light reaches a visible surface in the system that is set to the side and has a certain inclination angle with the light propagation direction, and is diffusely reflected on the surface to form a speckle pattern that carries the target information. The speckle pattern is focused by a lens located in its reflection direction and then captured by a camera set on the imaging surface to obtain a two-dimensional intensity image reflecting the target spatial information.
3. The non-line-of-sight high-resolution scatter imaging method based on speckle fingerprint according to claim 1, characterized in that: The non-field-of-view imaging optical system in step (1) further includes a light shielding plate, which is placed perpendicular to the visible surface and located on one side of the light propagation path, and is used to block direct light from the light source and stray light outside the system, thereby preventing non-object light components from directly entering the imaging path or irradiating the visible surface, thereby causing interference signals.
4. The non-line-of-sight high-resolution scatter imaging method based on speckle fingerprint according to claim 1, characterized in that: The step (1) of shooting a series of speckle images is specifically that the target moves relative to the medium, and the medium always remains stationary. During the target movement, a series of speckle images I1, ..., I n .
5. The non-line-of-sight high-resolution scattering imaging method based on speckle fingerprint according to claim 2, characterized in that: The scattering medium is frosted glass.
6. The non-line-of-sight high-resolution scatter imaging method based on speckle fingerprint according to claim 2, characterized in that: The visible surface is a white polystyrene board.
7. The non-line-of-sight high-resolution scatter imaging method based on speckle fingerprint according to claim 1, characterized in that: The implementation process of step (2) is as follows: From a series of speckle images I1,…,I n Take any two speckle images I i and I j , the relative displacement of the target in the two speckle images is (Δx ij ,Δy ij );The speckle image I i and I j Input into the frequency domain low-pass filter, and obtain the envelope E contained in the low-frequency information through frequency domain low-frequency filtering i and E j , using the original speckle image I i and I j Divide by the envelope E i and E j , we get the spatial normalized speckle S i and S j , specifically expressed as follows: S i (ξ,η)=∫∫O(x,y)I S (x,y)s(ξ,η;x,y)dxdy S j (ξ,η;Δx ij ,Dy ij )=∫∫O(x-Δx ij ,y-Δy ij )I S (x,y)s(ξ,η;x,y)dxdy In the formula, (x-Δx ij ,y-Δy ij ) represents the position of the target at the time of the i-th detection relative to the time of the j-th detection; when i = j, Δx ij =Δy ij =0.
8. The non-line-of-sight high-resolution scatter imaging method based on speckle fingerprint according to claim 1, characterized in that: The process of obtaining the overlap of the target in step (3) is as follows: Assume that the impulse responses of different point sources with a relative distance greater than the spatial coherence length do not overlap with each other, that is, each point on the target surface has its corresponding speckle fingerprint, and the speckle fingerprints are independent of each other; extract two spatial normalized speckle patterns S by point multiplication i and S i The speckle overlap between M ij (ξ,η;Δx ij ,Dy ij )=S i (ξ,η)S i (ξ,η;Δx ij ,Dy ij )=∫∫O(x,y)O(x-Δx ij ,y-Δy ij )I S (x,y) 2 s(ξ,η;x,y) 2 dxdy In the above result, only the target displacement (Δx ij ,Δy ij ) The speckle fingerprint formed by the points in the overlapping area before and after is retained; the target overlap information is obtained by summing, and the specific formula is as follows: M ij (Δx ij ,Dy ij )=∫∫M ij (ξ,η;Δx ij ,Dy ij )dξdη=∫∫O(x,y)O(x-Δx ij ,y-Δy ij )∫∫[I S (x,y)s(ξ,η;x,y)] 2 dξdηdxdy Among them, the second integral ∫∫[I S (x,y)s(ξ,η;x,y)] 2 dξdη is the total energy of each speckle fingerprint, which is proportional to the energy of the point light source that forms the corresponding speckle fingerprint; assuming that in a small area containing an object, the intensity distribution I S (x,y) is uniform, so the second integral is considered to be an approximately equal constant, and the target overlap is: in, is a related operation, and const is a constant.
9. The non-line-of-sight high-resolution scatter imaging method based on speckle fingerprint according to claim 1, characterized in that: The process of combining the obtained autocorrelation values and their corresponding positions into a pixel on the target autocorrelation in step (3) is as follows: For two envelopes E i and E j Perform cross-correlation operation and determine the relative displacement of the target (Δx ij ,Δy ij ), get the autocorrelation value M ij At the position on the target autocorrelation, assign the target autocorrelation value to the position, that is, get a pixel of the target autocorrelation: C ij (Δx ij ,Δy ij )=M ij (Δx ij ,Δy ij )。 10. The non-line-of-sight high-resolution scatter imaging method based on speckle fingerprint according to claim 1, characterized in that: The implementation process of step (4) is as follows: By changing i and j from 1 to n in sequence, all pixels of the target O autocorrelation are constructed to obtain the complete target autocorrelation: According to the obtained target autocorrelation, the target image is reconstructed by the phase recovery algorithm.