Multi-scene single-frame large field-of-view scattering imaging method based on polarization encoding and spatial multiplexing

Abstract

The application discloses a multi-scene single-frame large-field-of-view scattering imaging method based on polarization encoding and spatial multiplexing, and comprises the following steps: a reflection type and a transmission type scattering imaging system are built, and the OME ranges of the two systems are measured; a linear multiplexing model is constructed by collecting multiple spatial multiplexed speckles through a single exposure of a polarization camera; a signal recognition method based on minimum mean square error and an N-FINDR algorithm are introduced to realize target number recognition and polarization-specific speckle initial value extraction; a Cauchy non-negative matrix decomposition algorithm with a graph Laplace regularization term is designed, and the extracted target number and initial value are iteratively optimized to analyze the polarization-specific speckles in different OME regions; and a phase recovery algorithm is used to reconstruct the speckle results, and hidden targets in a scattering medium are recovered. The method can expand the field of view range by 4.5 times in the transmission type / reflection type system, the fidelity of the demultiplexing result reaches 32 dB, and the hidden scene reconstruction is realized without continuous collection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of scattering imaging, and particularly to a multi-scene single-frame large field of view scattering imaging method based on polarization encoding and spatial multiplexing. BACKGROUND

[0002] Light propagates along a straight line in a uniform medium such as clean water or air, and a traditional optical imaging system can directly image it. However, when light passes through a complex scattering medium such as a cloud layer, smoke, frosted glass, turbid water, or biological tissue, it will collide with particles in the scattering medium and undergo multiple scattering, and the photon transmission path becomes randomized. At this time, a traditional optical system cannot directly image the target. Currently, there are many methods for scattering imaging, such as adaptive optics technology, wavefront shaping technology, transmission matrix measurement, and speckle correlation imaging method. The speckle correlation imaging method has the advantage of enabling single-frame non-invasive imaging without prior information. However, this method has a significant drawback: due to the range limitation of the optical memory effect (OME), the field of view (FOV) is narrow, so the traditional speckle correlation imaging method can only image a single target within the OME range, which seriously hinders the application of this method in multi-target detection beyond the field of view range.

[0003] To expand the field of view range of scattering imaging, researchers have proposed many solutions, which can be divided into four categories: the first category uses prior information (such as point spread function (PSF), target position, target number, etc.) combined with spatial demultiplexing technology to expand the imaging field of view. However, this method generally needs to invade the inside of the imaging system, which is difficult to apply in practice. The second category uses mathematical methods to measure the transmission matrix and construct the relationship between the input field and the output field. However, this method generally has a large amount of calculation and high requirements for the stability of the scattering medium. The third category uses deep learning methods to train the network to decode the speckle and analyze the statistical correlation between the target and the speckle to solve unknown targets. However, this method requires a large amount of data set annotation and a long training time, and the model adaptation speed is slow, making it difficult to apply in dynamic environments where the scattering conditions change rapidly.

[0004] The fourth category focuses on extracting effective information of different OME regions from large field-of-view speckles through component analysis and feature separation, weakening the limitation of OME range. When multiple targets with a distance greater than the OME range are illuminated simultaneously, the multiplexed speckles generated by the scattering medium can be regarded as the linear superposition of speckles generated by each sub-target. Based on this principle, in 2020, a blind target position detection method based on low crosstalk region allocation strategy was proposed (Wang X, Jin X, Li J. Blind position detection for large field-of-view scattering imaging[J / OL]. Photonics Research, 2020, 8(6): 920-928. DOI:10.1364 / PRJ.388522.), which recovered the scene of multiple isolated targets beyond the OME range without prior information. However, blind detection methods usually require complex inverse problem solving, especially in large field-of-view scenes, the increase in the number of targets and the increase in the field of view angle will cause the number of algorithm iterations and the amount of calculation to increase exponentially. The following year, the existing technology proposed to separate the speckles generated by two moving and static targets with a distance greater than the OME range using the superposition and averaging algorithm, in order to realize large field-of-view imaging (He W, Wei Y, Lu D, et al. Noninvasive imaging of two isolated objects through a thin scattering medium beyond the 3D optical memory effect by speckle-based difference strategy[J]. Optics Letters, 2021, 46(23): 5954-5957.). However, this method is only suitable for specific scenarios and has strict requirements on the number of targets. In 2023, the existing technology improved this speckle separation-based method by introducing a mixed speckle simplex separation strategy (Wei Y, Guo E, Zhao Y, et al. Prior-free mixed speckle simplex separation strategy for multi-object imaging through thin scattering media beyond the optical memory effect[J]. APL Photonics, 2023, 8(12.).), which greatly improved the separation fidelity and solving ability.However, all the above methods need to collect speckle images continuously for multiple times, do not have single-frame imaging capability, and are difficult to meet the application scenarios with high-speed imaging requirements. SUMMARY

[0005] In order to overcome the deficiencies of the prior art, the present application provides a multi-scene single-frame large-field-of-view scattering imaging method based on polarization encoding and spatial multiplexing.

[0006] The technical solution for achieving the object of the present application is as follows: a multi-scene single-frame large-field-of-view scattering imaging method based on polarization encoding and spatial multiplexing, comprising:

[0007] Step 1: build a reflection type and a transmission type scattering imaging system, respectively measure the OME range, and verify the polarization speckle linear multiplexing relationship;

[0008] Step 2: collect multiple spatial multiplexing speckles through single exposure of a polarization camera, and construct a linear multiplexing model;

[0009] Step 3: introduce a signal recognition method based on least mean square error and an N-FINDR algorithm to respectively realize polarization target quantity recognition and polarization-specific speckle initial value extraction;

[0010] Step 4: design a Cauchy non-negative matrix decomposition algorithm with a graph Laplace regularization term constraint, use the target quantity and the polarization-specific speckle initial value extracted in step 3 for iterative optimization, and analyze the polarization-specific speckle in different OME regions; use a phase recovery algorithm to reconstruct the speckle result, and recover the hidden target after the scattering medium.

[0011] An electronic device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the steps of the above method when executing the program.

[0012] A computer readable storage medium has a computer program stored thereon, and the program is executed by a processor to implement the steps of the above method.

[0013] A computer program product comprises a computer program, and the computer program is executed by a processor to implement the steps of the above method.

[0014] Compared with the prior art, the present application has the following advantages:

[0015] (1) The present application does not need continuous multiple data collection, and only needs single exposure of a polarization camera to realize full data collection;

[0016] (2) The method proposed in the present application can be applied to both through scattering medium imaging and corner scattering imaging scenarios;

[0017] (3) The specificity speckle extraction process is guided by introducing the adjacent matrix and graph Laplacian regularization term constraints into the iteration direction of the NMF algorithm by using the continuity and similarity of speckles with similar polarization angles. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 Imaging schematic diagram for polarization encoding spatial multiplexing method, where (a) is the configuration of a transmission imaging system, (b) is the configuration of a reflection imaging system, and (c) is a polarization-guided speckle demultiplexing framework.

[0019] Figure 2 Configuration of a large field of view imaging optical system beyond the OME range, where (a) is a transmission imaging system; (b) is a reflection imaging system; (c) is an OME range measuring device; and (d) is the OME range measurement results of the reflection and transmission systems respectively.

[0020] Figure 3 Experimental verification of the spatial linear multiplexing relationship between the measured speckle and the speckles of different targets with different polarization states, where (a) is a transmission system, and (b) is a reflection system.

[0021] Figure 4 Experimental results of single-frame multi-target imaging through scattering media in the 2× OME range, where (a) is the target and polarization mask position, (b) is the polarization multiplexed speckle captured by the polarization camera, (c) is the initial polarization-specific speckle demultiplexed by the N-FINDR algorithm and their respective autocorrelation results, (d) is the accurate polarization-specific speckle demultiplexed by the NMF algorithm after optimization and their respective autocorrelation results, (e) is the target structure reconstructed from the demultiplexed speckle by the HIO phase recovery algorithm, and (f) is the real target speckle and autocorrelation results as a control group.

[0022] Figure 5 Experimental results of single-frame multi-target imaging through scattering media under different target spacings, (a) is 2.5× OME; (b) is 3× OME; and (c) is 3.75× OME.

[0023] Figure 6 Experimental results of single-frame multi-target imaging through scattering media in the 2.5× OME range in a corner scene, where (a) is the target and polarization mask position; (b) is the polarization multiplexed speckle captured by the polarization camera; (c) is the initial polarization-specific speckle demultiplexed by the N-FINDR algorithm and their respective autocorrelation results; (d) is the accurate polarization-specific speckle demultiplexed by the NMF algorithm after optimization and their respective autocorrelation results; (e) is the target structure reconstructed from the demultiplexed speckle by the HIO phase recovery algorithm; and (f) is the real target speckle and autocorrelation results as a control group.

[0024] Figure 7 Imaging experimental results of single-frame multi-target corner scene under different target distances, wherein (a) is 3x OME; (b) is 3.5x OME; (c) is 4x OME; (d) is 4.5x OME. DETAILED DESCRIPTION

[0025] The application will be described in detail below with reference to the accompanying drawings and examples.

[0026] The core of the method is to extract the differential speckle information of different OME regions in a large field of view to realize the reconstruction of each region; by introducing a special polarization camera, at most 181 speckle images of different polarization states can be calculated and obtained through one exposure; by using the polarization linear multiplexing relationship between the collected speckle and the speckle corresponding to different OME regions, a polarization-guided speckle demultiplexing framework is constructed to analyze the polarization-specific speckle of different OME regions; first, initialization is performed, the target number is identified through the minimum mean square error criterion (MMSE), and the initial value estimation of the polarization-specific speckle is completed with the help of the N-FINDR algorithm. On this basis, the polarization-specific speckle corresponding to different OME regions is accurately demultiplexed through the truncated Cauchy non-negative matrix factorization algorithm (TCLF-NMF) based on the neighborhood weighted sum and graph Laplacian regularization term constraint. The field of view of the method is expanded by 4.5 times in the transmission / reflection system, the fidelity of the demultiplexing result reaches 32dB, and the hidden scene reconstruction is realized without continuous acquisition.

[0027] The application will be described in detail below with reference to the accompanying drawings and examples.

[0028] A multi-scene single-frame large field of view scattering imaging method based on polarization encoding and spatial multiplexing, comprising:

[0029] Step 1: Build a reflection and transmission scattering imaging system, measure the OME range respectively, and verify the polarization speckle linear multiplexing relationship; specifically:

[0030] Step 1-1, experimental system building: build a reflection and transmission scattering imaging system respectively, the difference between the two is that the scattering medium of the transmission scattering imaging system is a frosted glass, and the scattering medium of the reflection scattering imaging system is a ZnO coated wall instead of a frosted glass, the light source and the camera are distributed on the two sides of the scattering medium, and the direct illumination is prevented by blocking the light baffle;

[0031] Step 1-2, OME range measurement: A target plate with a 0.5mm x 0.5mm pinhole in the center was placed on the motorized displacement stage to simulate a point light source. The point light source was controlled to move in the vertical direction of the light path by the displacement stage with a step size of 0.1mm to generate speckle patterns for correlation calculation. The cross-correlation coefficient between each frame of displacement speckle and the central reference speckle was calculated, and the correlation curve was plotted. The OME range was defined as the displacement range where the speckle cross-correlation coefficient remained above 0.5, so the actual OME range of the transmission system was 2.79mm. The same method was used to measure the OME range of the reflection system, which was 1.60mm.

[0032] Step 1-3, Polarization encoding target plate design: A hollow target plate containing multiple targets was designed, and each target was attached with N linear polarized masks arranged at equal intervals in different directions. In the four-target experiment, the linear polarized masks were 0°, 45°, 90°, and 135°, respectively.

[0033] Step 1-4, Linear multiplexing relationship verification: The relationship to be verified is where I mix represents the multiplexed speckle, I i represents the sub-target speckle in different OME regions, and N represents the number of polarization targets. A target plate containing four hollow targets with letters was placed in the object plane, and each target was equipped with a different polarizer (0°, 45°, 90°, 135°). First, all targets were used to generate polarization multiplexed speckle I mea . Then, the other three targets were covered with black opaque paper one by one, and the polarization speckles I i generated by each target were collected separately. The linear superposition of I i was performed to obtain the calculated multiplexed speckle I mix . The normalized difference analysis of I mix and I mea was performed, and the results showed that the difference of all pixels in the transmission system was within [-0.14, 0.14], and the pixel-by-pixel difference result of the reflection system was within [-0.1, 0.1]. This result fully proved that the spatial linear multiplexing model in polarization encoded speckle was correct.

[0034] Step 2: Multiple spatial multiplexed speckles were collected by a single exposure of a polarization camera to construct a linear multiplexing model. Specifically:

[0035] Data acquisition: Both the reflection and transmission scattering imaging systems used the snapshot function of the polarization camera to realize complete Stokes data acquisition. The polarization camera integrated a 0°, 45°, 90°, and 135° micro-polarizer array, and through internal calculation and integration functions, all polarization state speckles were collected under single exposure conditions. The acquisition step size of the polarization camera was set to 3°, and 61 polarization multiplexed speckles Imk ;

[0036] Establishing linear multiplexing model: According to Muller matrix analysis, the polarization camera analyzer introduces a unique intensity modulation coefficient for each sub-target speckle , combined with the linear multiplexing relationship verified in step 1, we can get , where I mk represents the kth multiplexed speckle collected, , M is the number of multiplexed speckles collected by the polarization camera at different polarization angles, 0≤a i <1 and ; I mk and I i are vectorized and normalized to obtain b k and u i , so we can get ; where α k = [a1,a2,…,a N ] T is the vector set of intensity modulation coefficient a i ; let A = [α1, α2,…,α M ] represent the vector set of α k , B = [ b1,b2,…,b N ] and U = [ u1,u2,…,u N ] are the vector sets composed of vectorized multiplexed speckles b k and different polarization target speckles u i , and a linear multiplexing model is established: .

[0037] The de-multiplexing of different targets in different OME regions can be converted into a mathematical linear solution model, and the process is shown in Figure 1 . The M multiplexed speckles collected by the polarization camera are analyzed in the polarization guided speckle de-multiplexing framework, and the polarization specific speckles of N different targets are analyzed. After separation, the de-multiplexed speckles are reconstructed by phase recovery algorithm, and the real large field of view scene can be recovered.

[0038] Preferably, the collected spatial multiplexed speckles are greater than 20, and in this embodiment, 61 are taken.

[0039] Step 3: Introducing signal recognition method based on minimum mean square error and N-FINDR algorithm to realize polarization target number recognition and polarization specific speckle initial value extraction respectively; step 3 is specifically:

[0040] Step 3-1, identify the number of polarization targets by MMSE: according to , that is, the covariance matrix R B=R UA ; wherein R B = BB T / M, then wherein denotes the estimate of R UA ; performing eigenvalue decomposition on yields

[0041] (1);

[0042] wherein W = [e1, e2, …, e G ] is the eigenvector matrix, G denotes the number of elements in the polarized speckle, and∑ is a diagonal matrix containing the eigenvalues in descending order; the space is divided into two orthogonal subspaces, a j-dimensional subspace j spanned by [e1, e2, …, e ], and a (G-j)-dimensional subspace spanned by the other elements; based on the non-negativity constraint of the polarized multiplexed speckle, the mean value of B in any eigenvector direction should be non-zero; therefore, the signal subspace is determined by finding the eigenvector subset that best represents the least square mean of the data set; the mean value of the collected multiplexed polarized speckle intensity is defined as: wherein 1 M is an M-dimensional all-one vector; let s j be the projection of s on the signal subspace ; the estimate of s is obtained by projecting onto , i.e. wherein P j = W j W j T is the projection matrix; the mean square error (MSE) of s and is:

[0043] (2);

[0044] wherein E(·) denotes the mathematical expectation, and I denotes the unit matrix; since the projection matrix P j is an idempotent matrix, there is P j T = P j , P j 2 = P j , and thus (I-P j ) T (I-P j ) = (I-P j ), so the above formula is simplified as:

[0045] (3);

[0046] in , representing the total signal energy, s T P j s represents the signal in the subspace The projected energy in the subspace; therefore, the projected energy reaches its maximum value when mse(j) reaches its minimum value, and the subspace The order is:

[0047] (4);

[0048] The obtained j is the number N of the actual target polarization speckle;

[0049] Step 3-2: Extract initial values ​​of polarization-specific speckle using the N-FINDR algorithm: Each acquired polarization multiplexed speckle can be represented as a linear combination of polarization-specific speckles of different sub-targets in different OME regions; from a geometric perspective, all acquired speckle sets form a convex polyhedron in high-dimensional space, and the independent polarization speckle of each target is located at the vertex of this convex polyhedron; based on this idea, the speckle demultiplexing problem is transformed into a vertex recognition problem;

[0050] The N-FINDR algorithm achieves vertex search by maximizing the volume of a polyhedron, and the entire process consists of four steps:

[0051] The first step, to reduce computational complexity and suppress noise interference, is to perform dimensionality reduction on the speckle data using Principal Component Analysis (PCA), retaining the information of the principal components with the largest variance. The data is then projected into an (N-1)-dimensional space to generate M principal component reuse speckle vectors pb. k ;

[0052] The second step is to randomly select N multiplexed speckle patterns as candidate vertices to form an N×N augmented matrix H:

[0053] (5);

[0054] The third step is to calculate the polyhedron volume V(H) under the current vertex configuration based on the matrix determinant det(·):

[0055] (6);

[0056] Fourth step: Fix (N-1) vertices, and use the remaining pixels to replace the Nth vertex in turn. If the volume of the polyhedron increases, then the replacement is accepted.

[0057] Repeat steps two through four to iteratively optimize the volume of the polyhedron until the volume is maximized and the convergence condition is met. ,in , respectively represent the polyhedral volume at the wth and w-1th update, represents the convergence threshold. At this time, the N polarized multiplexed speckles corresponding to the vertices of the polyhedron are the initial values of the target polarized speckles.

[0058] Step 4: Design a graph Laplacian regularized NMF algorithm, and use the target number and polarized specific speckle initial value extracted in step 3 to iteratively optimize and analyze the polarized specific speckle in different OME regions; use the phase recovery algorithm to reconstruct the speckle result to recover the hidden target in the scattering medium. Specifically:

[0059] Step 4-1, truncate the Cauchy loss function: In order to enhance the robustness of the model, the truncated Cauchy loss function is used instead of the least square loss as the NMF objective function; the expression f(x) of the truncated Cauchy loss function (TCLF) is:

[0060] (7);

[0061] Wherein, r is the proportion coefficient of Cauchy distribution, c is the self-defined adjustable threshold, x represents the loss function, that is, the difference (B-AU) between the true value B and the decomposition result; when the absolute value of x is greater than the threshold, the loss function is truncated to a fixed value, which weakens the influence of high value on the objective function, so as to suppress the smooth noise and abnormal value, and enhance the robustness of the model;

[0062] The objective function J of the NMF model based on TCLF can be represented as:

[0063] (8);

[0064] Wherein G represents the number of speckle image pixels, (B-UA) lk represents the element value of the lth row and kth column of matrix B-UA;

[0065] Step 4-2, graph Laplacian regularized: According to the image theory (Graph Theory), all polarized multiplexed speckles are mapped into a complete undirected graph, and each speckle represents a vertex, so as to obtain an MxM adjacency matrix; the correlation of polarized speckles shows strong angle dependence, the closer the camera polarization angles of two multiplexed speckles b m and b n , the closer the modulation vector a between them, the stronger the similarity of multiplexed speckles, and the weight value should be larger; on the contrary, the greater the difference between the polarization angles, the weight value should tend to 0; according to the above relationship, the graph composed of M multiplexed speckles is weighted by using Gaussian kernel, and the weight value between b m and b n ​ is denoted as:

[0066] (9);

[0067] where δ is the bandwidth of the Gaussian kernel;

[0068] The adjacent multiplex speckles should have similar modulation vectors. By adding a penalty term of modulation matrix A through this relationship, the adjacent modulation vectors are mutually constrained and corrected to avoid isolated and unrealistic estimated values, guiding the iterative process to a more smooth direction within the modulation matrix. The similarity relationship can be effectively transformed through graph Laplacian regularization:

[0069] (10);

[0070] where tr(·) represents the trace of the matrix, , D represents a diagonal matrix, and the diagonal elements , is a weight matrix composed of weight coefficients , L is a Laplacian matrix, and A m , A n represent the mth column and nth column of the modulation coefficient matrix A, i.e., α m , α n ;

[0071] To reduce the calculation amount and improve the operation efficiency, the local neighborhood weighting method is used to simplify the adjacency matrix. For any point b m , only the weight coefficients within the local neighborhood are considered, and t is the self-defined neighborhood radius. The weight coefficients between all non-adjacent points (|m-n|>t) are set to 0. The sum of the weight values of all points within the local neighborhood range m of point b is defined as the neighborhood similarity coefficient , which can be expressed as:

[0072] (11);

[0073] In formula (11), . For all , the similarity measurement vector generated by the weight can be expressed as In actual calculation, to meet the matrix matching requirements, the vector C is expanded to matrix R, which is expressed as:

[0074] (12);

[0075] The objective function of the NMF model after adding the adjacency matrix and the graph Laplacian regularization term can be expressed as:

[0076] (13);

[0077] ;

[0078] where is an N-dimensional all-one row vector, is an M-dimensional all-one row vector, T denotes the transpose operation, and β is the regularization term coefficient;

[0079] Step 4-3 Model solving: Since formula (13) is nonlinear and non-convex, it is difficult to solve directly; based on the principle of Half-Quadratic Optimization, auxiliary variables are introduced, and the original problem is decomposed into a series of more easily solved sub-problems to solve; for the truncated Cauchy loss function, first, , g(y) = -f(y), is converted to ; according to the Legendre-Fenchel transform, two conjugate transformations of g(y) can be obtained:

[0080] (14);

[0081] where Z is the conjugate variable of y, denotes the conjugate function of , and when g(y) reaches the maximum value, , where denotes the partial derivative of the function g(y) with respect to g, and when y is substituted, it can be known that when U, A are fixed, the element value of the matrix Z in the lth row and the kth column is lk denoted as:

[0082] (15);

[0083] where, denotes the modulus of the element value of the matrix B-UA in the lth row and the kth column; in the case of fixed Z, the target is an optimization problem containing two variables U and A, and the most common solution scheme is to use the idea of Alternating Direction Method of Multipliers (ADMM) to convert the original problem into several simple sub-problems containing only one variable, and then optimize each sub-problem alternately to approximate the optimal solution of the original problem; the two sub-problems after conversion are:

[0084] (16);

[0085] (17);

[0086] where , is a Lagrange multiplier, denotes point multiplication;

[0087] The optimal solution of equation (16) under Karush-Kuhn-Tucker condition can be obtained, and the update rule of A is:

[0088] (18);

[0089] where the m-th column of denotes , , denotes point division operation; for the objective function (17), the update rule of U under KKT condition can be obtained:

[0090] (19);

[0091] The matrices A, Z and U are updated alternately until convergence, and the polarization-specific speckles of different targets can be separated, and then the multiple OME-range targets hidden behind the scattering medium can be reconstructed by a phase retrieval algorithm.

[0092] The matrices A, Z and U are updated alternately until the convergence condition is met where J q and J q-1 represent the values of the objective function in the q-th and (q-1)-th update respectively, and tol represents the convergence threshold. The polarization-specific speckles of different targets can be separated, and then the multiple OME-range targets hidden behind the scattering medium can be reconstructed by a phase retrieval algorithm. The specific process is shown in Table 1.

[0093] Table 1: Framework of polarization-guided speckle demultiplexing

[0094]

[0095] The present application designs two experimental configurations for imaging in a field of view beyond the OME range, which are a transmission system for transmitting through a scattering medium and a reflection system for corner imaging, respectively. Figure 2(a) is a schematic diagram of the experimental setup of the transmissive system. The natural light from a xenon lamp (Beijing Perfect Light, PLS-SXE300E) passed through a pinhole (Thorlabs, ID25SS / M, diameter 10 mm) and was homogenized by a rotating ground glass. Then the light passed through two collimating lenses to form a collimated beam. The beam passed through a narrow-band filter with a center wavelength of 520 nm to form a non-polarized monochromatic light, which was incident on the center of the target plate. N linearly polarized masks (Changcheng Advanced Materials, T4-1825TP) were attached to the target plate, which were arranged at equal intervals in different directions. In the four-target experiment, the linearly polarized masks were 0°, 45°, 90°, and 135°, respectively. The multi-target linearly polarized light was scattered by a ground glass (Thorlabs, DG100X100-220, 220 grit), and was received by a polarization camera (The Imaging Source DYK 33UX250, 2448x2048 pixels) integrated with 0°, 45°, 90°, and 135° micro-polarizers. Through the calculation function inside the polarization camera, the acquisition of all polarization states of speckles can be completed under single-exposure conditions. Figure 2 (b) is a schematic diagram of the experimental setup of the reflective system. The reflective system is identical to the transmissive system in terms of the light path before scattering, except that the scattering medium of the transmissive system is a ground glass, while the scattering medium of the reflective system is a ZnO-coated wall, which replaces the ground glass. The light source and camera are distributed on the two sides of the scattering medium, separated by a light-blocking baffle to prevent direct illumination. Both systems use the snapshot function of the polarization camera to achieve complete Stokes data acquisition.

[0096] To verify the feasibility of imaging beyond the OME range, it is necessary to first determine the specific OME range. Figure 2 (c) is the experimental setup for measuring the OME range of the transmissive imaging system. A target plate with a 0.5 mm x 0.5 mm pinhole in the center was placed on a motorized displacement stage to simulate a point light source. The displacement stage was used to control the movement of the point light source in the vertical direction of the light path, with a step size of 0.1 mm, generating speckle patterns for correlation calculation. The cross-correlation coefficient between each frame of displacement speckle and the central reference speckle was calculated, and the correlation curve shown in (d) was plotted. Figure 2 Figure 2 The OME range is defined as the displacement range within which the speckle cross-correlation coefficient remains above 0.5, i.e., the area within the two blue dotted lines in (d), so the actual OME range of the transmissive system is 2.79 mm. Similarly, the OME range of the reflective system can be measured to be 1.60 mm.

[0097] In Figure 2 ​Based on the optical system shown, the present application carries out three experimental verifications: linear multiplexing relationship verification, single-frame multi-target imaging through scattering medium and single-frame multi-target imaging at the corner. These experiments collectively verify the feasibility of the proposed large field of view imaging method beyond the range of OME.

[0098] (1) Linear multiplexing relationship verification

[0099] To verify the spatial linear multiplexing model (equation (5)), a strict verification experiment was set up for both sets of experimental devices. Figure 3 Figure (a) in the above shows the verification process of the transmission system. A target plate containing four hollow letters 'L', 'Z', '7', 'V' was placed in the object plane, each target was equipped with a different polarizer (0°, 45°, 90°, 135°), and the size of each hollow letter was 1.5 mm. The distance between each two targets was 7 mm (2.5x OME range). Natural light passed through a four-way linear polarization mask and the target plate to form four linearly polarized light beams, which produced polarization multiplexing speckles after scattering by the ground glass. Subsequently, the other three targets were covered with black opaque paper one by one, and the polarization speckles produced by each of the four targets were collected separately. Linear superposition was performed on to obtain the calculated multiplexing speckles . Comparing with , it was found that the intensity distributions of the two images were highly similar. Further normalized difference analysis of the two speckles showed that the difference values of all pixels were within [-0.14, 0.14], and most of the pixels were within [-0.1, 0.1], confirming that the linear multiplexing relationship between the measured multi-target speckles and the speckles of different targets containing different polarization states was established.

[0100] Figure 3 Figure (b) in the above is the verification process of the reflection imaging system, the distance between the targets is 4 mm (2.5x OME range), and the verification method is the same as that of the transmission system. The single-target calculated multiplexing speckles are basically consistent with the intensity distribution of the multi-target speckles obtained by experiment, and the pixel-by-pixel difference results are within [-0.1, 0.1]. The above results fully prove that the spatial linear multiplexing model in the polarization encoding speckle is correct.

[0101] (2) Single-frame multi-target imaging through scattering medium

[0102] This embodiment sets up a transmission single-frame multi-target imaging experimental system and designs four-target imaging experiments with target spacing exceeding different degrees of OME range. As shown in Figure 4 ​As shown in (a), each target is 1.5 mm in size, and the distance between any two targets is 5.5 mm (2 × OME range). Natural light passing through the targets with four-way polarization masks generates four linearly polarized beams containing target information for different OME regions. These beams spatially mix after passing through the scattering medium (frosted glass), and the resulting multiplexed speckle pattern is captured by a polarization camera. Because the polarization camera has a built-in micro-polarizer array and computational synthesis module, it can obtain multiplexed speckles with all polarization states in a single exposure, applying different intensity modulation coefficients to the polarization speckles of different targets.

[0103] The acquisition step size of the polarization camera was set to 3°, and 61 polarization multiplexed speckle images were acquired in a single exposure. ,like Figure 4 As shown in (b) above. First, the number of targets N is obtained using the MMSE algorithm, and then the N-FINDR method is used to obtain the target number N. The N speckle images with the lowest reuse rate are extracted as the initial values ​​for polarization-specific speckle. ,like Figure 4 As shown in (c). However, since each speckle image actually acquired contains at least (N-1) polarization components of the target, Other speckle interferences still exist, which can occur in their respective autocorrelation structures. This limitation is clearly evident. To overcome this limitation, this invention utilizes the designed Cauchy NMF algorithm, combined with graph Laplace regularization constraints, to find accurate solutions for different target speckle patterns. Through multiple iterative optimizations, demultiplexed polarization speckle patterns are obtained. ,like Figure 4 As shown in (d) in the figure. The polarization-specific speckle structure obtained by demultiplexing is significantly purified, and its autocorrelation results are shown. and The improvement is significant. Finally, the HIO phase retrieval algorithm is used to... The hidden target results were obtained by reconstructing them separately. ,like Figure 4 As shown in (e) in the diagram.

[0104] The speckle image and autocorrelation results obtained from the decomposition are compared with... Figure 4 (f) shows real reference speckle images of targets acquired individually at different polarization angles. and autocorrelation results By comparison, it can be seen that the initial values ​​extracted by N-FINDR... and The target structure is identifiable, but significant target crosstalk is also observed, particularly in the autocorrelation of polarization-specific speckle at 45°. In the middle, although the main component is '2', it still contains residual speckle of 0° polarization target 'L' and 90° polarization target 'X'. Due to the relatively small crosstalk speckle component and the disorder of speckle itself, the effect of crosstalk is difficult to judge directly. In this embodiment, the polarization-specific speckle , PSNR between the real target reference speckle I, which quantitatively represents the crosstalk effect, is shown in Table 2, where 0°-L represents the 0° polarization target 'L', 45°-2 represents the 45° polarization target '2', 90°-X represents the 90° polarization target 'X', and 135°-D represents the 135° polarization target 'D'. It can be seen that for each separation result, the PSNR between FS and I is significantly improved compared to the PSNR between IS and I. The PSNR between IS and I is all below 30 dB, with an average of 29.0238 dB, while the average PSNR between FS and I is 32.1052 dB. The autocorrelation structure analysis further confirms the optimization effect of the NMF algorithm with constraints: FA has stronger correlation signal peaks and clearer structural contrast than IA. Although there is still a slight loss of details compared to AC, as shown in (e) of Figure 4 , the de-multiplexed speckle after NMF algorithm optimization is sufficient to reconstruct the structure of the clearly distinguishable target.

[0105] Table 2 Figure 4 PSNR between de-multiplexed speckle and real reference speckle in

[0106]

[0107] To evaluate the fidelity of the speckle de-multiplexing results under different target distances, experiments were conducted at 2.5x OME, 3x OME, and 3.5x OME, respectively. The de-multiplexed speckle and the reconstruction results are shown in (a)-(c) of Figure 5 , respectively. The PSNR between the de-multiplexed FS and the respective real target reference speckle is generally high, with an average of 32.2887 (2.5x OME), 33.7435 (3x OME), and 30.9994 (3.5x OME), respectively. The obtained de-multiplexed speckle was subjected to phase recovery, and the results all produced clear and recognizable reconstruction results RO, indicating that the method proposed in the present application is robust in a large field of view.

[0108] The successful construction of the transmission imaging system in this embodiment proves the feasibility of realizing single-frame multi-target scattering imaging beyond the OME range through polarization encoding and spatial demultiplexing technology, which expands the effective field of view of the system to 3.75 times the original.

[0109] (3) Single-frame multi-target imaging in corner scene

[0110] Based on the above verification of the transmission model, this embodiment also extends the feasibility evaluation to the reflection system shown in (b) of Figure 2 . Since the backscattered light of the ZnO wall still retains the same imaging mechanism as the transmission mode, there is an optical memory effect, and the hidden target can be reconstructed by the speckle correlation imaging method. According to this underlying similarity, method migration can be directly performed. The feasibility verification process of the reflection imaging system is shown in Figure 6 , the size of a single target is still 1.5 mm, and the distance between targets is 2.5x OME. The polarization light containing different OME region target information is reflected by the ZnO wall, and the multiplexed speckle produced is captured by the polarization camera through one exposure. The entire recovery process, from speckle demultiplexing to phase recovery, is the same as the transmission verification process, which indicates the performance stability of the method in different scattering scenarios.

[0111] The results of quantitative verification by PSNR are shown in Table 3, where 0°-C represents the 0° polarized target 'C', 45°-K represents the 45° polarized target 'K', 90°-H represents the 90° polarized target 'H', and 135°-7 represents the 135° polarized target '7'. The results prove the high structural fidelity between the demultiplexed speckle and the real target reference speckle. All the calculation results are above 25 dB, and for each polarization direction corresponding target, the final value is always more than 3 dB higher than the initial result extracted by the N-FINDR algorithm , which indicates that the designed iterative NMF algorithm can effectively suppress the mutual interference between different targets. The autocorrelation structure also shows obvious differences, although and both have target structures in AC, but has problems such as edge curvature degradation, which may cause imbalance in the intensity of the reconstruction results and loss of structure, for example, the left and right ends of the autocorrelation result of the 0° direction polarized target 'C' have almost no curvature. In contrast, retains the complete edge curvature while improving the detail resolution, achieving high fidelity. These demultiplexed speckles can be reconstructed with high quality by the HIO phase recovery algorithm, successfully recovering all the targets in the reflection configuration.

[0112] Table 3. Results of 2.5 ME target PSNR calculation for reflection system

[0113]

[0114] To evaluate the feasibility of the method in different view ranges, four groups of experiments with target distances of 4.8 mm, 5.6 mm, 6.4 mm, and 7.2 mm were set up, and the results are shown in FIG. 6. The PSNR results between the polarization-specific speckle and the real target reference speckle I obtained by the four groups of experiments are all above 30 dB, and the average values are 30.4873 dB, 30.7070 dB, 30.2723 dB, and 30.1941 dB, respectively, proving that the method has high demultiplexing accuracy under different target distances. Phase recovery was performed on all the demultiplexed speckles, and although some distortion and discontinuity were caused by the inevitable estimation error of the reconstruction algorithm, complete target reconstruction was still achieved. Figure 7

[0115] In summary, the above experiments show that the proposed polarization encoding and spatial multiplexing method can efficiently demultiplex the polarization-specific speckle of the target in different OME regions from the multiplexed speckle, thereby successfully achieving target structure reconstruction in a large field of view beyond the OME range. The method breaks through the limitation of the OME range in both reflection and transmission configurations, and achieves a field of view expansion of at least 4.5 times in a single shot.

[0116] The present application proposes a polarization encoding and spatial multiplexing method, which overcomes the limitation of the OME range in transmission / reflection scattering imaging and achieves single-frame multi-target imaging beyond the OME range. Through physical layer modeling and polarization multiplexing mapping at the algorithm level, a polarization-guided speckle demultiplexing framework is established to reconstruct the polarization-specific speckle in different OME regions. The framework mainly includes two stages: first, the MMSE algorithm is used to identify the number of hidden targets, and the N-FINDR algorithm is used for polarization-specific speckle estimation; then, the TCLF is guided and an NMF algorithm with local neighborhood weighting and graph Laplacian regularization term constraint is proposed. Experiments in transmission and reflection prove that the method proposed in the present application can expand the field of view by 4.5 times while maintaining a demultiplexing fidelity of 32 dB. The compatibility of the proposed method for single-frame OME range imaging makes it possible for the application of speckle-related imaging methods in actual dynamic scenes.​​​

Claims

1. A multi-scene single-frame large field-of-view scattering imaging method based on polarization coding and spatial multiplexing, characterized in that, include: Step 1: Construct reflective and transmissive scattering imaging systems, measure their optical memory effect (OME) range respectively, and verify the linear multiplexing relationship of polarization speckle. The relationship to be verified is: , where I mix Indicates multiplexed speckle pattern, I i The speckle pattern represents sub-targets within different OME regions, where N represents the number of polarized targets. A target plate containing four hollowed-out letters is placed on the object plane, with each target equipped with a different polarizer. First, all targets are sampled to generate a polarization multiplexed speckle pattern I. mea Subsequently, the three targets were covered sequentially with black opaque paper, and polarized speckle patterns generated by each of the four targets were collected individually. i , will I i Linear superposition yields the calculated multiplex speckle I. mix ; will I mix with I mea Normalized difference analysis was performed on two speckle images; Step 2: Acquire multiple spatially multiplexed speckle images using a single exposure with a polarization camera to construct a linear multiplexing model. The method for constructing the linear multiplexing model is as follows: based on Muller matrix analysis, the polarization camera analyzer introduces an intensity modulation coefficient for each sub-target speckle. Combining the linear reuse relationship verified in step 1, we can obtain , among which, I mk This represents the k-th multiplexed speckle image acquired. M is the number of multiplexed speckles acquired by the polarization camera at different polarization angles, 0≤a i <1 and ; will I mk and I i We obtain b by vectorization and normalization respectively. k and u i Therefore, we can obtain ; where α k = [a1,a2,…,a N ] T The intensity modulation coefficient a i Let A = [α1, α2, ..., α] be a set of vectors. M ] represents α k The vector set, B = [b1,b2,…,b] N ],U = [ u1,u2,…,u N [b] represents the vectorized multiplexed speckle pattern. k and speckle patterns of targets with different polarizations u i The constructed vector set is used to establish a linear reuse model: ; Step 3: Introduce a signal recognition method based on minimum mean square error and the N-FINDR algorithm to realize the identification of the number of polarized targets and the extraction of the initial value of polarization-specific speckle, respectively; Step 4: Design a Cauchy nonnegative matrix factorization algorithm with Laplace regularization constraints. Iterate and optimize the number of targets extracted in Step 3 and the initial value of polarization-specific speckle to analyze the polarization-specific speckle in different OME regions. Use the phase retrieval algorithm to reconstruct the speckle results and recover the hidden target behind the scattering medium.

2. The multi-scene single-frame large field-of-view scattering imaging method based on polarization coding and spatial multiplexing according to claim 1, characterized in that, In step 1, the experimental system is built as follows: a reflective scattering imaging system and a transmissive scattering imaging system are built respectively. The difference between the two is that the scattering medium of the transmissive scattering imaging system is frosted glass, while the reflective scattering imaging system uses a ZnO coated wall instead of frosted glass as the scattering medium. The light source and camera are distributed on both sides of the scattering medium and separated by a light-blocking baffle to prevent direct illumination. The OME range measurement method is as follows: A target plate with a 0.5mm × 0.5mm pinhole cutout in the center is placed on an electric displacement stage to simulate a point light source. The displacement stage controls the point light source to move in the direction perpendicular to the optical path with a movement step of 0.1mm, generating a speckle pattern for correlation calculation. The cross-correlation coefficient between each frame of displacement speckle and the central reference speckle is calculated, and the correlation curve is plotted. The OME range is defined as the displacement range in which the speckle cross-correlation coefficient remains above 0.

5. The design method of polarization-encoded target board is as follows: design a hollow target board containing multiple targets, and attach N linear polarization masks arranged at equal intervals in different directions on each target; in the four-target experiment, the linear polarization masks are 0°, 45°, 90° and 135° respectively.

3. The multi-scene single-frame large field-of-view scattering imaging method based on polarization coding and spatial multiplexing according to claim 2, characterized in that, In step 2, more than 20 spatial multiplexed speckle images need to be collected.

4. The multi-scene single-frame large field-of-view scattering imaging method based on polarization coding and spatial multiplexing according to claim 2, characterized in that, In step 2, both the reflective and transmissive scattering imaging systems utilize the snapshot function of the polarization camera to achieve complete Stokes data acquisition. The polarization camera integrates a 0°, 45°, 90°, and 135° micro-polarizer array, and through its internal calculation and synthesis function, it completes the acquisition of all polarization speckle patterns under a single exposure. The acquisition step size of the polarization camera is set to 3°, and 61 polarization multiplexed speckle images are acquired in a single exposure. mk .

5. The multi-scene single-frame large field-of-view scattering imaging method based on polarization coding and spatial multiplexing according to claim 4, characterized in that, Step 3 specifically involves: Step 3-1, Identify the number of polarized targets using MMSE: Based on The covariance matrix R of matrix B B =R UA Among them, R B =BB T / M, then ,in R represents UA The estimated value; for Eigenvalue decomposition yields: (1)； In the formula, W = [e1,e2,…,e G [e1, e2, ..., e2] is the characteristic matrix, G represents the number of elements in the polarization speckle, and Σ is a diagonal matrix containing eigenvalues ​​arranged in descending order; the space is divided into two orthogonal subspaces, namely [e1, e2, ..., e2]. j The determined j-dimensional subspace and the (Gj)-dimensional subspace determined by other elements. Based on the non-negativity constraint of polarization multiplexing speckle, the projection of the mean of B onto any eigenvector direction should be non-zero; therefore, the signal subspace is determined by finding the feature subset that best represents the least square mean of the dataset; the mean of the acquired multiplexed polarization speckle intensity is defined as: , of which 1 M Let s be an M-dimensional, all-one vector; j For s in the signal subspace Projection on The estimated value is obtained by... Projected to Obtained from above, that is , where P j = W j W j T Let be the projection matrix; then s and The mean square error (MSE) is: (2)； Where E(·) represents the mathematical expectation, and I represents the identity matrix; since the projection matrix P j For an idempotent matrix, P j T =P j P j 2 =P j Therefore (IP) j ) T (IP j ) = (IP j Therefore, the above formula simplifies to: (3)； in Represents the total energy of the signal, s T P j s represents the signal in the subspace The projected energy in the subspace; therefore, the projected energy reaches its maximum value when mse(j) reaches its minimum value, and the subspace The order is: (4)； The obtained j is the number N of the actual target polarization speckle; Step 3-2: Extract initial values ​​of polarization-specific speckle using the N-FINDR algorithm: Each collected polarization multiplexed speckle can be represented as a linear combination of polarization-specific speckles of different sub-targets in different OME regions; the collection of all collected speckles forms a convex polyhedron in high-dimensional space, and the independent polarization speckle of each target is located at the vertex of this convex polyhedron; the speckle demultiplexing problem is transformed into a vertex recognition problem. The N-FINDR algorithm achieves vertex search by maximizing the volume of a polyhedron, and the entire process consists of four steps: The first step is to reduce the dimensionality of the speckle data using principal component analysis, retaining the information of the principal components with the largest variance, and projecting the data into an (N-1) dimensional space to generate M principal component speckle vectors pb. k ; The second step is to randomly select N multiplexed speckle patterns as candidate vertices to form an N×N augmented matrix H: (5)； The third step is to calculate the polyhedron volume V(H) under the current vertex configuration based on the matrix determinant det(·): (6)； Fourth step: Fix (N-1) vertices, and use the remaining pixels to replace the Nth vertex in turn. If the volume of the polyhedron increases, then the replacement is accepted. Repeat steps two through four to iteratively optimize the volume of the polyhedron until the volume is maximized and converged. Then the replacement loop terminates. At this point, the N polarization multiplexed speckles corresponding to the vertices of the polyhedron are the initial values ​​of the target polarization speckle.

6. The multi-scene single-frame large field-of-view scattering imaging method based on polarization coding and spatial multiplexing according to claim 5, characterized in that, Step 4 specifically involves: Step 4-1: Replace the least squares loss with the truncated Cauchy loss function as the objective function of NMF; the expression for the truncated Cauchy loss function f(x) is: (7)； Where r is the proportionality coefficient of the Cauchy distribution, c is a user-defined adjustable threshold, and x represents the loss function, which is the difference (B-AU) between the true value B and the decomposition result. When the absolute value of x is greater than the threshold, the loss function is truncated to a fixed value. The objective function J of the NMF model based on the truncated Cauchy loss function is expressed as: (8)； Where G represents the number of pixels in the speckle image, (B-UA) lk This represents the element value in the l-th row and k-th column of matrix B-UA; Step 4-2, Graph Laplacian Regularization Constraint: According to image theory, all polarization multiplexed speckles are mapped into a complete undirected graph, with each speckle representing a vertex, resulting in an M×M adjacency matrix; the closer the camera polarization angles of two multiplexed speckles, the better. m and b n The closer the modulation vectors α are, the stronger the similarity of the multiplexed speckle patterns, and the larger the weight value; conversely, the greater the difference in polarization angles, the closer the weight value is to 0; a Gaussian kernel is used to weight the graph composed of M multiplexed speckles, b m and b n Between weights Represented as: (9)； Where δ is the bandwidth of the Gaussian kernel; Transformation of similarity relations using graph Laplacian regularization: (10)； Where tr(·) represents the trace of the matrix, D represents a diagonal matrix, with diagonal elements , For weight coefficients The weight matrix is ​​formed, where L is the Laplace matrix and A is the weight matrix. m A n These represent the m-th and n-th columns of the modulation coefficient matrix A, respectively, i.e., α m α n ; The adjacency matrix is ​​simplified using a local neighborhood weighting method; for any point b m Only consider calculating the local neighborhood. The weight coefficients within the neighborhood are set to 0, where t is the custom neighborhood radius; the weight coefficients between all non-adjacent point pairs are set to 0; point b is then... m Local neighborhood range The neighborhood similarity coefficient is defined as the sum of the weights of all points within a neighborhood. , is represented as: (11)； In equation (11), For all The adjacency matrix of the polarized speckle pattern, represented by the similarity metric vector generated by the weights, is as follows: In actual calculations, to meet the matrix matching requirements, vector C is expanded into matrix R: (12)； The objective function of the NMF model after adding the adjacency matrix and graph Laplacian regularization term is expressed as: (13)； ； in It is an N-dimensional row vector. Let be an M-dimensional row vector, where T represents the transpose operation and β is the regularization coefficient; Step 4-3, Model Solution: Based on the principle of semi-quadratic optimization, auxiliary variables are introduced, and the original problem is decomposed into multiple subproblems for solution using convex conjugate theory; for the truncated Cauchy loss function, firstly, let g(y) = -f(y), Transform into According to the Legendre-Fenchel transform, performing two conjugate transformations on g(y) yields: (14)； Where Z is the conjugate variable of y. express The conjugate function of y, when the function g(y) reaches its maximum value, ,in This represents the partial derivative of function g(y) with respect to g. Substituting y into the equation, we can see that when U and A are both fixed, the element Z in the l-th row and k-th column of matrix Z is... lk Represented as: (15)； in, Let represent the modulus of the element in the l-th row and k-th column of matrix B-UA; with Z fixed, the objective is an optimization problem with two variables, U and A. The original problem is transformed into several simpler subproblems with only one variable each using the alternating direction multiplier method. Each subproblem is then optimized alternately to approximate the optimal solution of the original problem. The two transformed subproblems are: (16)； (17)； in , For Lagrange multipliers, Dot product; Finding the optimal solution to equation (16) under the Karush-Kuhn-Tucker conditions yields the update rule for A as follows: (18)； in The m-th column is represented as , , This represents the dot division operation; for the objective function (17), the update rule for U under the KKT conditions is as follows: (19)； The matrices A, Z, and U are iteratively updated until convergence, separating the polarization-specific speckle patterns of different targets. Then, the phase retrieval algorithm is used to reconstruct multiple targets in the super-OME range hidden behind the scattering medium.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1-6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1-6.

9. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-6.

Images