An image reconstruction method, apparatus, electronic device, and readable medium

Abstract

The present invention provides an image reconstruction method, apparatus, electronic device and readable medium. The method includes: sampling an observation signal by using a CUP-VISAR system to obtain a two-dimensional compressed observation image; according to the two-dimensional compressed observation image, taking the total variation prior term, tensor low-rank constraint and sparse constraint as prior constraints, constructing a tensor principal component analysis reconstruction mathematical model based on plug-and-play alternating direction multiplier; decomposing the image reconstruction mathematical model into several optimization sub-problems, and performing iterative solution to obtain a reconstructed image. Based on the plug-and-play alternating direction multiplier framework, the present invention establishes an image reconstruction mathematical model, and converts the solution of a complex optimization problem into the solution of several simple sub-problems, completes the representation of the structural characteristics of the reconstructed three-dimensional fringe image data, and avoids the loss of local information.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to an image reconstruction method, an image reconstruction device, an electronic device, and a computer-readable medium. Background Art

[0002] Laser-driven inertial confinement fusion (ICF) is a highly challenging research field in controllable nuclear fusion, and its diagnostic technology is an important research direction for promoting the development of controllable nuclear fusion. Among them, the imaging velocity interferometer system for any reflector (VISAR) is the most widely used diagnostic device in ICF, which is used to record one-dimensional wavefront information with picosecond-level time resolution generated by ICF. However, this information is only limited to the velocity change of a line on the target surface. If the slit of the streak camera is fully opened, due to the characteristics of the streak camera, the different periodic fringe images detected on the internal recording CCD will overlap, resulting in information errors. Therefore, it is usually used to detect one-dimensional spatial information of the laser-driven shock wave velocity as a function of time, and cannot provide the velocity distribution of all points on the target surface, that is, two-dimensional velocity field information with high time resolution.

[0003] In recent years, compressive sensing can sample signals under the condition of a sampling rate lower than the Nyquist sampling rate, and at the same time achieve signal compression, and reconstruct the original signal from the compressed data using a suitable reconstruction algorithm, which has been widely applied in a wide range of fields. In 2014, GAO et al. applied compressive sensing to ultrafast imaging, and achieved compressed ultrafast photography (CUP) of two-dimensional images with a time resolution of up to 2 ps and an imaging frame rate of up to 1011 frames / s. In 2019, the Wang Feng team coupled line-imaging VISAR and compressed ultrafast photography (CUP), and initially applied CUP to the field of ICF diagnosis, and achieved the reconstruction of the dynamic evolution process of the two-dimensional shock wave velocity field through a reconstruction algorithm. Subsequently, methods such as the generalized alternating projection (GAP) method based on matrix low rank and the alternating direction method of multipliers (ADMM) method have been developed.

[0004] However, the reconstruction quality of CUP-VISAR two-dimensional compressed images is not high and the accuracy is relatively low, and there are mainly the following several disadvantages:

[0005] a. During the CUP-VISAR measurement process, there are factors such as information loss and noise pollution, resulting in low-quality observed compressed images.

[0006] b. The TV constraint can promote piecewise smoothness of the image and retain edge details, but it is prone to problems such as detail loss and over-smoothing.

[0007] c. The TV-TwIST (TwIST based on total variation constraint) algorithm is vulnerable to regularization parameters, resulting in unstable reconstruction effects.

[0008] d. Matrix low rank will destroy the internal structural relationship of the three-dimensional image block. Summary of the Invention

[0009] In view of the above problems, the present invention is proposed to provide an image reconstruction method, a corresponding image reconstruction device, an electronic device, and a computer-readable medium that overcome the above problems or at least partially solve the above problems.

[0010] The present invention discloses an image reconstruction method, including:

[0011] Sampling the observation signal using a CUP-VISAR system to obtain a two-dimensional compressed observation image;

[0012] According to the two-dimensional compressed observation image, using the total variation prior term, tensor low rank constraint, and sparse constraint as prior constraints, constructing a tensor principal component analysis reconstruction mathematical model based on plug-and-play alternating direction multiplier;

[0013] Decomposing the image reconstruction mathematical model into several optimization sub-problems and performing iterative solution to obtain the reconstructed image.

[0014] As a further improvement of the present invention, the step of sampling the observation signal using a CUP-VISAR system to obtain a two-dimensional compressed observation image includes:

[0015] The VISAR system carries the target surface information to generate interference and forms an interference fringe image at the imaging plane, obtaining a time-varying two-dimensional fringe image;

[0016] Inputting the time-varying two-dimensional fringe image into the CUP system, encoding, offsetting, and superimposing the time-varying two-dimensional fringe image to form a two-dimensional compressed observation image.

[0017] As a further improvement of the present invention, the expression of the two-dimensional compressed observation image is:

[0018] Y = AI

[0019] Wherein, I is a three-dimensional fringe image with a pixel size of m×n×k that varies with time; Y is a two-dimensional observation image recorded by a streak camera, and its size is (m + k - 1)×n; A is an observation matrix, and A = MSC, where M is a spatio-temporal integration operator, S is a time shearing operator, and C is a spatial masking operator.

[0020] As a further improvement of the present invention, the step of constructing a tensor principal component analysis reconstruction mathematical model based on the plug-and-play alternating direction multiplier with the total variation prior term, tensor low-rank constraint, and sparse constraint as prior constraints according to the two-dimensional compressed observation image includes:

[0021] Adding the prior constraint to be evaluated to the expression of the two-dimensional compressed observation image, and converting the expression of the two-dimensional compressed observation image to:

[0022]

[0023] Where X is the quantity to be solved, Φ(I) is the regularization term, ||·|| 2 is the l 2 norm, ∈ is a positive parameter depending on the noise variance;

[0024] After decomposing the quantity X to be solved into a low-rank part, sparse noise, and Gaussian noise, the expression of the quantity X to be solved is:

[0025] X = L + S + R + N

[0026] Where L is the clean signal, S and R are sparse noises, and N is Gaussian noise;

[0027] Substituting the expression of the quantity X to be solved into the expression of the two-dimensional compressed observation image obtained by conversion, and applying the total variation prior term and tensor low-rank constraint at the same time, we get:

[0028]

[0029] ||·|| tv is the total variation regularization function, ||·|| 3DTNN is the three-dimensional tensor nuclear norm, ||·|| 1 is the l 1 norm, ||R|| 2，1 is the mixed norm and it is the sum of all Frobenius norms of the side slices of R, that is is the index;

[0030] Based on the plug-and-play alternating direction multiplier framework, convert the above formula into an image reconstruction mathematical model:

[0031]

[0032] In the formula, τ, λ, α, β, γ, and μ are regularization parameters, U is an auxiliary variable, Λ is a dual variable, and ρ is an augmented Lagrangian parameter.

[0033] As a further improvement of the present invention, several optimization sub-problems include: the sub-problem of the quantity X to be solved, the sub-problem of the auxiliary variable U, the sub-problem of the clean signal L, the sub-problem of the sparse noise S, the sub-problem of the sparse noise R, and the sub-problem of the Gaussian noise N.

[0034] As a further improvement of the present invention, the step of decomposing the image reconstruction mathematical model into several optimization sub-problems and solving to obtain the reconstructed image includes:

[0035] Update and solve the sub-problem of the quantity X to be solved;

[0036] Update and solve the sub-problem of the auxiliary variable U;

[0037] Update and solve the sub-problem of the clean signal L;

[0038] Update and solve the sub-problem of the sparse noise S;

[0039] Update and solve the sub-problem of the sparse noise R;

[0040] Update and solve the sub-problem of the Gaussian noise N;

[0041] Update the dual variable Λ;

[0042] Update the augmented Lagrangian parameter ρ;

[0043] If the convergence condition is satisfied, stop the update and output the updated value to be solved and the clean signal to obtain the interference fringe image.

[0044] As a further improvement of the present invention, the convergence condition includes: the error value e ζ less than or equal to the preset error value where

[0045]

[0046] The present invention also discloses an image reconstruction device for implementing the image reconstruction method of the present invention, including:

[0047] The CUP-VISAR system is used to sample the observed signal to obtain a two-dimensional compressed observed image;

[0048] The image reconstruction mathematical model construction module is used to construct a tensor principal component analysis reconstruction mathematical model based on plug-and-play alternating direction multiplier with the total variation prior term, tensor low-rank constraint, and sparse constraint according to the two-dimensional compressed observed image;

[0049] The model solving module is used to decompose the image reconstruction mathematical model into several optimization sub-problems and solve them to obtain the reconstructed image.

[0050] The present invention also discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0051] The memory is used to store computer programs;

[0052] When the processor is used to execute the program stored on the memory, it realizes the image reconstruction method as described in the present invention.

[0053] The present invention also discloses one or more computer-readable media, on which instructions are stored. When executed by one or more processors, the instructions cause the processors to execute the image reconstruction method as described in the present invention.

[0054] The present invention includes the following advantages:

[0055] In the image reconstruction method of the present invention, the observation signal is sampled by using a CUP-VISAR system to obtain a two-dimensional compressed observation image; according to the two-dimensional compressed observation image, with the total variation prior term, tensor low-rank constraint, and sparse constraint as prior constraints, a tensor principal component analysis reconstruction mathematical model based on plug-and-play alternating direction multiplier is constructed; the image reconstruction mathematical model is decomposed into several optimization sub-problems, and iterative solutions are performed to obtain the reconstructed image. Based on the plug-and-play generalized alternating projection framework, the present invention establishes an image reconstruction mathematical model, and transforms the solution of a complex optimization problem into the solution of several simple sub-problems, completes the representation of the structural characteristics of the reconstructed three-dimensional fringe image data, and avoids the loss of local information. Description of the Drawings

[0056] Figure 1 It is a flowchart of the steps of an image reconstruction method provided by the present invention;

[0057] Figure 2 It is a schematic diagram of the CUP-VISAR system of the present invention;

[0058] Figure 3 It is a schematic diagram of compressive imaging in step S1 of the present invention;

[0059] Figure 4 It is a process diagram of compressed image reconstruction of the present invention;

[0060] Figure 5 It is a flowchart of solution in step S3 of the present invention;

[0061] Figure 6 It is a one-dimensional VISAR diagram used in the verification process of the present invention;

[0062] Figure 7a is the original three-dimensional tensor fringe pattern adopted in the verification process of the present invention;

[0063] Figure 7b is the three-dimensional tensor fringe pattern with noise adopted in the verification process of the present invention;

[0064] Figure 8a is the coding pattern adopted in the verification process of the present invention;

[0065] Figure 8b is the simulated compressed image adopted in the verification process of the present invention;

[0066] Figure 9 is the fringe image reconstructed by the ADMM-TV algorithm in the verification process of the present invention;

[0067] Figure 10 is the fringe image reconstructed by the ADMM-TVLR algorithm in the verification process of the present invention;

[0068] Figure 11 is the fringe image reconstructed by using the image reconstruction method described in the present invention in the verification process of the present invention;

[0069] Figure 12a is the comparison chart of the peak signal-to-noise ratio PSNR of ADMM-TVLR, ADMM-TV, and ADMM-TLRTN;

[0070] Figure 12b is the comparison chart of the structural similarity SSIM of ADMM-TVLR, ADMM-TV, and ADMM-TLRTN;

[0071] Figure 13 is the structural block diagram of an image reconstruction device provided by an embodiment of the present invention. Detailed implementation manners

[0072] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0073] Referring to Figure 1 , there is shown an image reconstruction method provided by the present invention, including:

[0074] S1. Sampling the observation signal by using a CUP-VISAR system to obtain a two-dimensional compressed observation image.

[0075] As Figure 2As shown, in the embodiment of the present invention, a CUP-VISAR system is used to obtain a two-dimensional compressed observation image, providing basic data for subsequent image reconstruction. The CUP-VISAR system mainly includes a VISAR system and a CUP system. Sampling the observation signal using the CUP-VISAR system to obtain a two-dimensional compressed observation image includes the following steps:

[0076] S11. The VISAR system carries the target surface information to generate interference and forms an interference fringe image at the imaging plane, obtaining a time-varying two-dimensional fringe image. Specifically, first, the probe light passes through the lens L1, the compensation lens COM1, the beam splitter BS2, the mirror M1, the mirror M2, the lens L2, the lens L3 in sequence to reach the target surface TSF and then returns along the original path. After being collected and collimated by the mirror M3, it passes through the lens L4, the interferometer, and the lens L5 in sequence to form an image at the imaging point IP2. At this time, a time-varying two-dimensional fringe image is obtained.

[0077] S12. Input the time-varying two-dimensional fringe image into the CUP system, encode, offset, and superimpose the time-varying two-dimensional fringe image to form a two-dimensional compressed observation image. On the one hand, this image is collected and recorded by a streak camera with a partially opened slit through the lens L10, the mirror M7, and the lens L11. On the other hand, this image is introduced onto a digital micromirror device (DMD) through a 4f imaging system composed of the tube lens L6 and the microscope objective L7 for encoding. The encoded image passes through the lens L8 and the lens L9 and enters the streak camera, and under the offset action of the electron tube of the streak camera, the encoded images at different times are offset to complete the shearing in time. Finally, the offset encoded images are recorded and superimposed on the charge-coupled device (CCD) built in the streak camera to obtain a two-dimensional compressed observation image.

[0078] S2. Based on the two-dimensional compressed observation image, using the total variation prior term, tensor low-rank constraint, and sparse constraint as prior constraints, construct a tensor principal component analysis reconstruction mathematical model based on plug-and-play alternating direction multiplier.

[0079] Specifically, step S2 includes the following steps:

[0080] S21. First, construct an expression for the two-dimensional compressed observation image:

[0081] Y = AI (1)

[0082] Wherein, I is a three-dimensional fringe image with a pixel size of m×n×k that varies with time; Y is a two-dimensional observation image recorded by a streak camera, with a size of (m + k - 1)×n; A is an observation matrix, and A = MSC, where M is a spatio-temporal integration operator, S is a time shearing operator, and C is a spatial masking operator.

[0083] As can be seen from Equation (1), the number of unknowns m×n×k is much larger than the number of equations (m + k - 1)×n, which is a problem of solving an ill-conditioned equation. Therefore, it is necessary to add a priori constraints on the expression of the two-dimensional compressed observation image to be evaluated, and convert it into an optimization problem. The expression of the two-dimensional compressed observation image is converted to:

[0084]

[0085] Where X is the quantity to be solved, Φ(I) is the regularization term, and ||·|| 2 is the l 2 norm, and ∈ is a positive parameter depending on the noise variance.

[0086] Considering that the quantity X to be solved contains both structural information and noise, after decomposing the quantity X to be solved into a low-rank part, sparse noise, and Gaussian noise, the expression of the quantity X to be solved is:

[0087] X = L + S + R + N (3)

[0088] Where L is a clean signal, S and R are sparse noises, and N is Gaussian noise;

[0089] Substitute the expression of the quantity X to be solved into the expression of the two-dimensional compressed observation image obtained after conversion, and at the same time apply the total variation prior term and the tensor low-rank constraint to obtain:

[0090]

[0091] ||·|| tv is the total variation regularization function, ||·|| 3DTNN is the three-dimensional tensor nuclear norm, ||·|| 1 is the l 1 norm, ||R|| 2，1 is the mixed norm and it is the sum of all Frobenius norms of the side slices of R, that is is the index;

[0092] For the solution of formula (4), based on the plug-and-play alternating direction multiplier framework, formula (4) is converted into an image reconstruction mathematical model:

[0093]

[0094] In the formula, τ, λ, α, β, γ, and μ are regularization parameters, U is an auxiliary variable, Λ is a dual variable, and ρ is an augmented Lagrangian parameter.

[0095] S3. Decompose the image reconstruction mathematical model into several optimization sub-problems, and perform iterative solution to obtain the reconstructed image. Among them, the several optimization sub-problems include: the unknown quantity X sub-problem, the auxiliary variable U sub-problem, the clean signal L sub-problem, the sparse noise S sub-problem, the sparse noise R sub-problem, and the Gaussian noise N sub-problem.

[0096] As Figure 3 and Figure 4 shown, step S3 includes the following steps:

[0097] S31. Update and solve the unknown quantity X sub-problem, and obtain the unknown quantity X sub-problem from formula (5):

[0098]

[0099] Formula (6) is a least squares problem, and its solution is as follows:

[0100] X (t) =(AA T +ρ (t-1) +μ) -1 (A T Y+ρ (t-1) U (t-1) -Λ+μ(L (t-1) +S (t-1) +R (t-1) +N (t-1) )) (7)

[0101] where t (t≥1) is the iteration number.

[0102] S32. Update and solve the auxiliary variable U sub-problem. The auxiliary variable U sub-problem is given by the following formula:

[0103]

[0104] The problem of formula (8) is solved by the following total variation approximation operator:

[0105]

[0106] In the formula, D TV is the TV denoising operator.

[0107] S33. Update and solve the clean signal L sub-problem. Similarly, the clean signal L sub-problem is given by the following formula:

[0108]

[0109] Among them, and i 3 (i 3 = 1, 2,..., k) are indices, is the weight, is the result of the third-mode Fourier transform (FFT) along L, ||·|| TNN is the tensor nuclear norm. Introduce three auxiliary tensors Then Equation 10 can be transformed into the following form:

[0110]

[0111] In the formula, Λ i is the dual variable, ρ i is the augmented Lagrangian parameter. Similarly, Equation 11 can be decomposed into the following sub-problems for update:

[0112]

[0113] Its solution is as follows:

[0114]

[0115] Among them, D tnnθ (·) represents the tensor singular value threshold operator, * i (i = 1, 2, 3) is the mode-n product, are respectively the orthogonal tensor, orthogonal tensor, and diagonal tensor of the singular value decomposition of the tensor,

[0116]

[0117] Update and solve the sub-problem of sparse noise S. Similarly, the sub-problem of sparse noise S is given by the following formula:

[0118]

[0119] Its solution is as follows:

[0120]

[0121] Among them, STH ζ (·) is the soft threshold operator, and its definition is STH ζ (·) = sgn(·) max(|·| - ζ), where sgn(·) is the real number sign function.

[0122] Update and solve the sub-problem of sparse noise R. Similarly, the sub-problem of sparse noise R is given by the following formula:

[0123]

[0124] Since The sub - problem of Equation 17 is divided into (n - t) sub - problems along the second dimension, and the i 2 -th sub - problem is as follows:

[0125]

[0126] where are X(:, i 2 , :), L(:, i 2 , :), S(:, i 2 , :), R(:, i 2 , :), N(:, i 2 , :), respectively.

[0127] Equation 18 is a least - squares problem, and its closed - form solution is as follows

[0128]

[0129] where ||·|| F is the Frobenius norm.

[0130] S36. Update and solve the Gaussian noise N sub - problem. The Gaussian noise N sub - problem is given by the following formula:

[0131]

[0132] Its solution is as follows:

[0133]

[0134] S37. Update the dual variables Λ, Λ i (i = 1, 2, 3):

[0135] Λ (t) = Λ (t-1) + ρ (t-1) (X (t) - U (t) ) (22)

[0136]

[0137] (1)S38. Update the augmented Lagrangian parameter ρρ i (i = 1,2,3)

[0138]

[0139] In the formula, p and P i(i=1,2,3) is the original residual and p=||X (t) -U (t) || 2 as well as q and q i (i=1,2,3) is the dual residual and q=ρ (t-1) ||U (t-1) -U (t) || 2 as well as τ incr (τ incr >1) and τ decr (τ decr >1) is the balance factor, ε(ε>1) is the residual tolerance, usually τ inc r = 1.1, τ decr =1.1,ε=1.5.

[0140] S39, as the iteration proceeds, that is, when t=t+1 (t>1), determine whether the convergence condition is met. If the convergence condition is met, stop updating and output the updated X (t) , L (t) , the image is reconstructed and the interference fringe image is obtained. If the convergence condition is not met, the above steps are repeated to continue the iteration.

[0141] The convergence conditions include: the error value e ζ Less than or equal to the preset error value in,

[0142]

[0143] is the preset error.

[0144] Next, the effectiveness of the image reconstruction method provided by the present invention is verified:

[0145] A numerical method is used to verify the feasibility and effectiveness of the proposed method, and the alternating multiplier method based on total variation (ADMM-TV) and the alternating multiplier method based on total variation and low-rank constraint (ADMM-LRTV) are used as control groups. Peak signal-to-noise ratio and structural similarity are used to quantitatively evaluate the quality of the reconstructed image. Then, a set of simulated images are produced based on a set of shock wave velocity data measured by the line-VISAR system, such as Figure 6 As shown. Figure 6Extract 35 phases representing time points from the shown image data to generate the original third-order fringe tensor data with a size of 540×800×35 pixels. At the same time, in order to be closer to the real complex environment, Salt&Pepper noise and Gaussian noise with a variance of 0.1 are introduced into the third-order fringe tensor data. The original third-order fringe tensor data and the third-order fringe tensor with noise are shown in Figure 7. In addition, due to limitations such as actual experimental conditions and the saturation characteristics of a streak camera with a fully open slit, a coded mask with a coded aperture of 8×8 is adopted, as shown in Figure 8(a). Based on the third-order fringe tensor data with noise and the coded mask image data, by simulating the data acquisition process of CUP-VISAR, the corresponding observed image data with a size of 574×800 pixels is shown in Figure 8(b). Finally, use ADMM-TV, ADMM-TVLR, and ADMM-TLRTN (the proposed method) to reconstruct the third-order fringe tensor from the observed image data.

[0146] The reconstructed third-order fringe tensors obtained by these methods are as Figure 9 , Figure 10 and Figure 11 shown. The quality of each reconstructed fringe image obtained by ADMM-TV is blurred. In contrast, the reconstructed images obtained by ADMM-TLRTN are better and clearer. To quantitatively evaluate the quality of the reconstructed images, the PSNR and SSIM of the reconstructed third-order fringe tensors are as Figure 13 shown. As shown in Figure 12, the PSNR and SSIM of the fringe images reconstructed using ADMM-TV and ADMM-TVLR are respectively lower than 20 dB and 0.8. For the ADMM-TLRTR method, the curves of PSNR and SSIM are relatively flat, and the PSNR values and SSIM values are respectively between 18.097 dB - 21.376 dB and 0.803 - 0.887. At the same time, the PSNR and SSIM of each frame do not differ much, indicating that the inter-frame information is well preserved. In addition, compared with ADMM-TV and ADMM-TVLR, the average PSNR and average SSIM are increased by 8.426 dB, 4.586 dB, 32.6%, and 17.5% respectively. From the above results, the quality of each frame of the reconstructed images obtained by this method is more stable and better.

[0147] It should be noted that for the method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of the present invention are not limited by the described action sequences, because according to the embodiments of the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential for the embodiments of the present invention.

[0148] Reference Figure 13 , which shows an image reconstruction device provided in an embodiment of the present invention for implementing the above-mentioned image reconstruction method, including: a CUP-VISAR system, an image reconstruction mathematical model construction module, and a model solution module; the CUP-VISAR system is used to sample the observation signal to obtain a two-dimensional compressed observation image; the image reconstruction mathematical model construction module is used to construct a tensor principal component analysis reconstruction mathematical model based on plug-and-play alternating direction multiplier with total variation prior term, tensor low-rank constraint, and sparse constraint according to the two-dimensional compressed observation image; the model solution module is used to decompose the image reconstruction mathematical model into several optimization sub-problems and solve them to obtain the reconstructed image.

[0149] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the relevant parts, please refer to the partial description of the method embodiment.

[0150] In addition, an embodiment of the present invention also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus.

[0151] The memory is used to store a computer program.

[0152] The processor is used to implement the image reconstruction method as described in the above embodiment when executing the program stored in the memory.

[0153] The communication bus mentioned in the above terminal may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0154] The communication interface is used for communication between the above terminal and other devices.

[0155] The memory may include a Random Access Memory (RAM), or may also include a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0156] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU for short), a Network Processor (NP for short), etc.; it may also be a Digital Signal Processor (DSP for short), an Application Specific Integrated Circuit (ASIC for short), a Field-Programmable Gate Array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0157] In another embodiment provided by the present invention, a computer-readable storage medium is further provided. Instructions are stored in the computer-readable storage medium, and when it runs on a computer, it causes the computer to execute the image reconstruction method described in the above embodiment.

[0158] In another embodiment provided by the present invention, a computer program product containing instructions is further provided. When it runs on a computer, it causes the computer to execute the image reconstruction method described in the above embodiment.

[0159] In the above embodiment, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a Solid State Disk (SSD)).

[0160] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0161] Each embodiment in this specification is described in a related manner. For the same and similar parts between the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and reference can be made to the corresponding part of the method embodiment for the relevant content.

[0162] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are all included in the protection scope of the present invention.

Claims

1. An image reconstruction method, characterized in that: include: The CUP-VISAR system is used to sample the observation signal to obtain a two-dimensional compressed observation image. The expression of the two-dimensional compressed observation image is: Y=AI Where I is a three-dimensional streak image with a pixel size of m×m×k that varies with time; Y is a two-dimensional observation image recorded by a streak camera, with a size of (m+k-1)×n; A is the observation matrix, and A=MSC, where M is the spatiotemporal integration operator, S is the time shear operator, and C is the spatial mask operator; According to the two-dimensional compressed observation image, a tensor principal component analysis reconstruction mathematical model based on plug-and-play alternating direction multipliers is constructed with total variation prior, tensor low rank constraint and sparse constraint as prior constraints, including the following steps: Add a priori constraints to be evaluated for the expression of the two-dimensional compressed observation image and transform the expression of the two-dimensional compressed observation image into: Where X is the quantity to be determined, Φ(I) is the regularization term, ‖·‖2 is the l2 norm, and ∈ is a positive parameter that depends on the noise variance; After decomposing the quantity X into the low-rank part of the tensor, sparse noise and Gaussian noise, the expression of the quantity X is: X=L+S+R+N Where L is the clean signal, S and R are sparse noise, and N is Gaussian noise; Substitute the expression of the desired quantity X into the expression of the two-dimensional compressed observation image obtained by transformation, and apply the total variation prior term and tensor low rank constraint at the same time, and we get: ‖·‖ tv is the total variation regularization function, ‖·‖ 3DTNN is the nuclear norm of the three-dimensional tensor, ‖·‖1 is the l1 norm, ||R|| 2,1 is the mixed norm and is the sum of all Frobenius norms of the side slices of R, that is, i2 (i2=1,2,…,n) is the index; Based on the plug-and-play alternating direction multiplier framework, the above formula is converted into a mathematical model for image reconstruction: Where τ, λ, α, β, γ, μ are regularization parameters, U is the auxiliary variable, Λ is the dual variable, and ρ is the augmented Lagrangian parameter; The mathematical model of image reconstruction is decomposed into several optimization sub-problems, and the reconstructed image is obtained by iterative solution.

2. The image reconstruction method according to claim 1, characterized in that: The step of sampling the observation signal using the CUP-VISAR system to obtain a two-dimensional compressed observation image includes: The VISAR system carries the target surface information and generates interference to form an interference fringe image at the imaging plane, thus obtaining a time-varying two-dimensional fringe image. The time-varying two-dimensional fringe image is input into the CUP system, and the time-varying two-dimensional fringe image is encoded, shifted and superimposed to form a two-dimensional compressed observation image.

3. The image reconstruction method according to claim 1, characterized in that: The optimization sub-problems include: the desired quantity X sub-problem, the auxiliary variable U sub-problem, the clean signal L sub-problem, the sparse noise S sub-problem, the sparse noise R sub-problem, and the Gaussian noise N sub-problem.

4. The image reconstruction method according to claim 3, characterized in that: The step of decomposing the image reconstruction mathematical model into a plurality of optimization sub-problems and solving them to obtain the reconstructed image comprises: Update and solve the subproblem of the quantity X to be solved; Update and solve the auxiliary variable U subproblem; Update and solve the clean signal L subproblem; Update and solve the sparse noise S subproblem; Update and solve the sparse noise R subproblem; Update and solve the Gaussian noise N subproblems; Update the dual variable Λ; Update the augmented Lagrangian parameter ρ; If the convergence condition is met, the updating is stopped and the updated value to be evaluated and the clean signal are output to obtain the interference fringe image.

5. The image reconstruction method according to claim 4, characterized in that: The convergence conditions include: the error value e ζ Less than or equal to the preset error value in, 6. An image reconstruction device, characterized in that: Used to implement the image reconstruction method according to any one of claims 1 to 5, comprising: CUP-VISAR system, used to sample the observation signal and obtain a two-dimensional compressed observation image; Image inversion mathematical model construction module, which is used to construct a tensor principal component analysis reconstruction mathematical model based on plug-and-play alternating direction multipliers according to the two-dimensional compressed observation image, with total variation prior terms, tensor low-rank constraints, and sparse constraints as prior constraints; The model solving module is used to decompose the image reconstruction mathematical model into several optimization sub-problems and solve them to obtain the reconstructed image.

7. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the image reconstruction method according to any one of claims 1 to 5 when executing the program stored in the memory.

8. One or more computer-readable media having instructions stored thereon, which, when executed by one or more processors, enable the processors to perform the image reconstruction method according to any one of claims 1 to 5.

Images