Single-pixel imaging method based on non-trained residual decoder and denoising prior

Abstract

This application discloses a single-pixel imaging method based on a non-trained residual decoder and a denoising prior, relating to the field of image processing technology. The method includes: acquiring a signal sequence sampled by a single-pixel imaging device; inputting the random tensor corresponding to the imaging into a non-trained residual decoder to obtain the image to be reconstructed output by the non-trained residual decoder; constructing an optimization problem for single-pixel imaging based on the signal sequence, the image to be reconstructed, and noise terms; the optimization problem includes a denoising prior; and solving the inverse problem of the single-pixel imaging sampling model using the optimization problem to obtain the reconstructed image of the single-pixel imaging. This application proposes a low-parameter non-trained residual decoder, controlling the number of network parameters of the non-trained residual decoder to the same order of magnitude as the number of image parameters, enabling stable convergence during image reconstruction, avoiding overfitting, and reducing computational resources; the introduction of a deep denoising prior allows for good imaging results even in high-noise environments.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a single-pixel imaging method based on an untrained residual decoder and denoising prior. Background Technology

[0002] Existing untrained neural network methods typically use networks with tens of millions of parameters. The number of parameters in these networks far exceeds the number of pixel parameters in the reconstructed image, especially for 256×256 pixel grayscale images commonly used in single-pixel imaging, where the number of parameters can be hundreds of times greater than the image's parameters. Using such large network structures can lead to overfitting during iteration, fitting unnatural components introduced during imaging (such as ringing caused by undersampling and detector noise) into the reconstructed image, severely impacting image quality. Although overfitting can be avoided through methods such as early stopping and adjusting the learning rate, these methods lack versatility and stability in engineering implementation. Furthermore, networks with tens of millions of parameters consume significant computational resources during iterative solving, resulting in excessively long image reconstruction times and high hardware performance requirements. Summary of the Invention

[0003] The main objective of this application is to propose a single-pixel imaging method and related equipment based on a non-trained residual decoder and denoising prior, so as to improve the efficiency of single-pixel imaging.

[0004] To achieve the above objectives, one aspect of this application proposes a single-pixel imaging method based on a non-trained residual decoder and denoising prior, the method comprising the following steps: Acquire the signal sequence sampled by a single-pixel imaging device; The random tensor corresponding to the imaging is input into the untrained residual decoder to obtain the image to be reconstructed output by the untrained residual decoder. An optimization problem for single-pixel imaging is constructed based on the signal sequence, the image to be reconstructed, and the noise term; wherein, the optimization problem includes a denoising prior. The inverse problem of the single-pixel imaging sampling model is solved using the optimization problem to obtain the reconstructed image of the single-pixel imaging; wherein, the process of solving the inverse problem of the sampling model includes denoising prior.

[0005] In some embodiments, the untrained residual decoder includes several decoding layers and an output layer; each decoding layer is composed of the following structure in sequence: nearest neighbor upsampling, residual structure, sigmoid activation function, and layer normalization, wherein the residual structure consists of a convolutional layer with a kernel size of 3×3, a SiLU activation function, and a convolutional layer with a kernel size of 3×3, and the residual skip connection adds the input features of the previous convolutional layer with a kernel size of 3×3 and the output features of the next convolutional layer with a kernel size of 3×3 element-wise; the output layer is composed of the following structure in sequence: a convolutional layer with a kernel size of 3×3, a SiLU activation function, layer normalization, a 1×1 convolution, and a Sigmoid activation function; The step of inputting the random tensor corresponding to the imaging into the untrained residual decoder to obtain the image to be reconstructed output by the untrained residual decoder includes the following steps: The random tensor corresponding to the image The input is fed into the untrained residual decoder, and during each nearest-neighbor upsampling of the decoding layer, The height and width are both magnified by a factor of 2, and finally, the image to be reconstructed is obtained through the output layer. : ; in, Represent real numbers, and These are the height and width of the image to be reconstructed, respectively. The number of decoding layers. The number of feature map channels. For network parameter set, This indicates the forward propagation computation of the untrained residual decoder.

[0006] In some embodiments, the single-pixel imaging sampling model is constructed through the following steps: The single-pixel imaging sampling model is constructed as follows: ; in, The signal sequence is... , For the sampling matrix, , The image to be reconstructed, , For the noise term, ; This indicates the dimension of the vector.

[0007] In some embodiments, constructing the optimization problem for single-pixel imaging based on the signal sequence, the image to be reconstructed, and noise terms includes the following steps: The optimization problem is constructed as follows: ; in, The regularization penalty coefficient is... This represents the denoising prior.

[0008] In some embodiments, solving the inverse problem of the single-pixel imaging sampling model using the optimization problem to obtain the reconstructed image of the single-pixel imaging includes the following steps: The optimization problem is solved iteratively using the alternating direction multiplier method to solve the inverse problem of the single-pixel imaging sampling model, thereby obtaining the reconstructed image; wherein, the augmented Lagrangian function corresponding to the optimization problem is expressed as: ; in, The penalty coefficient for the regularization term of the constraint condition. These are Lagrange multiplier vectors; The unknowns that need to be solved in the augmented Lagrangian function are respectively , and During the solution process, each iteration updates the three unknowns sequentially, specifically including: for Updates, fixed and Find the value of and solve the following model: ; Among them, due to For the untrained residual decoder The network parameters are such that the gradient descent optimizer in deep learning is used during the solution process. Update; for Updates, fixed and Find the value of and solve the following model: ; for The update is calculated using the following iterative formula: ; in, , , , , They are respectively , and The values ​​of the j-th and (j-1)-th iterations; After k iterations and convergence, take As the reconstructed image.

[0009] In some embodiments, the denoising prior step includes the following steps: A noise level map of the noisy image in the sampling model is determined using a noise estimator; The noise level map and the noisy image are input into the U-Net network, and the U-Net network is used to perform denoising to obtain the denoised image corresponding to the noisy image.

[0010] In some embodiments, acquiring the signal sequence sampled by a single-pixel imaging device includes the following steps: The signal sequence is obtained by sampling through a spatial light modulator or a digital micromirror device.

[0011] To achieve the above objectives, another aspect of this application proposes a single-pixel imaging system based on a non-trained residual decoder and denoising prior, the system comprising: The sampling signal acquisition unit is used to acquire the signal sequence sampled by the single-pixel imaging device; An image processing unit is used to input the random tensor corresponding to the imaging into a non-trained residual decoder to obtain the image to be reconstructed output by the non-trained residual decoder. A sampling modeling unit is used to construct an optimization problem for single-pixel imaging based on the signal sequence, the image to be reconstructed, and noise terms; wherein the optimization problem includes a denoising prior. The image reconstruction unit is used to solve the inverse problem of the single-pixel imaging sampling model using the optimization problem to obtain the reconstructed image of the single-pixel imaging; wherein, the process of solving the inverse problem of the sampling model includes denoising prior.

[0012] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0013] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0014] To achieve the above objectives, another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0015] The embodiments of this application include at least the following beneficial effects: This application provides a single-pixel imaging method and related equipment based on a non-trained residual decoder and a denoising prior. The method involves acquiring a signal sequence sampled by a single-pixel imaging device; inputting the corresponding random tensor into a non-trained residual decoder to obtain the image to be reconstructed output by the non-trained residual decoder; constructing an optimization problem for single-pixel imaging based on the signal sequence, the image to be reconstructed, and noise terms; wherein the optimization problem includes a denoising prior; and solving the inverse problem of the single-pixel imaging sampling model using the optimization problem to obtain the reconstructed image of the single-pixel imaging. This application proposes a low-parameter non-trained residual decoder, controlling the number of network parameters of the non-trained residual decoder to the same order of magnitude as the number of image parameters, enabling stable convergence during the iterative process of image reconstruction. This avoids overfitting without relying on additional methods and effectively reduces the computational resources consumed. Furthermore, this application introduces a deep denoising prior, enabling good imaging results even in high-noise environments. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of a single-pixel imaging system provided in an embodiment of this application; Figure 2 An example flowchart of a single-pixel imaging method based on a non-trained residual decoder and denoising prior provided for embodiments of this application; Figure 3 A flowchart illustrating the single-pixel imaging method based on an untrained residual decoder and denoising prior provided in this application embodiment; Figure 4 This is a network structure diagram of the non-trained residual decoder provided in the embodiments of this application; Figure 5 This is a structural diagram of the denoising network provided in the embodiments of this application; Figure 6 Example diagram of the reconstruction result of a normal environment provided in the embodiments of this application; Figure 7 Example diagram of the reconstruction result of a high-noise environment provided in the embodiments of this application; Figure 8 A schematic diagram of the structure of a single-pixel imaging system based on a non-trained residual decoder and denoising prior provided in an embodiment of this application; Figure 9 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0020] Before providing a detailed description of the embodiments of this application, some related technologies involved in the embodiments of this application will be described first, as follows: Compressed sensing: Compressed sensing is a data sampling and reconstruction theory. This theory holds that if a signal is sparse in the spatial domain or in a certain transform domain, then the signal can be undersampled by using a measurement matrix that is not related to the sparse representation transform basis, and the inverse problem of the sampling model can be solved by iterative optimization algorithm. This allows the signal to be reconstructed with far fewer samplings than required by the Nyquist sampling theorem.

[0021] SiLU Activation Function: The sigmoid linear unit (SiLU) is an activation function that exhibits a smoother curve near zero, overcoming the limitations of the traditional ReLU activation function, such as non-differentiability at zero and vanishing gradients below zero. In many scenarios, SiLU performs better than ReLU. The SiLU function can be represented as SiLU(x) = x Sigmoid(x).

[0022] Alternating Direction Method of Multipliers (ADMM) is a computational framework for solving separable convex optimization problems. It solves the problem by first decomposing and then combining the solutions of the subproblems, which are relatively simpler than the original problem. ADMM is suitable for solving distributed convex optimization problems, and is mainly applied when the solution space is very large.

[0023] Single-pixel imaging: In traditional imaging techniques, spatially resolving area array sensors such as CCDs or CMOS are typically used to directly image objects. A complete image can be acquired by detecting the spatial distribution information of the reflected light from the object in a single scan. Single-pixel imaging, however, is a computational imaging method. It uses spatial light modulation techniques (such as spatial light modulators or digital micromirror devices (DMDs)) to load a time-varying sequence of coded structural patterns to modulate and encode the spatial information of the reflected light from the object. Then, a single-pixel detector (such as a photodiode) without spatial resolution performs multiple samplings to obtain a signal sequence that corresponds one-to-one with the structural pattern sequence. A sparse model is established based on the sampling process, and an optimization algorithm is used to reconstruct the image. For example, a single-pixel imaging system... Figure 1 As shown.

[0024] Image reconstruction methods based on untrained neural networks, also known as deep image prior methods, are used for low-order image restoration tasks in computer vision, such as image denoising, image super-resolution, and image deblurring. In this framework, the network does not require prior training; instead, the network structure itself serves as an image prior. With a fixed input, the image reconstruction process is transformed into a maximum likelihood estimation problem for the network parameters. During iterative optimization, an objective function is first constructed based on the image degradation model of the specific task. Then, using undersampled data, a gradient descent optimizer iteratively optimizes the network parameters, gradually fitting them to the optimal solution image. In this process, the network training is also the network inference process. Untrained neural network image reconstruction methods have a significant advantage in scenarios where data samples for neural network training are scarce.

[0025] The related technology, "Far-field super-resolution ghost imaging with a deep neural network constraint," proposes a single-pixel ghost imaging method based on an untrained neural network. This method uses a variant of the U-Net network structure, combined with a total variation regularization term, and employs the Adam optimizer to optimize the network parameters to complete image reconstruction. However, the network structure used in this method has more than 30 million parameters, far exceeding the number of pixels in the image to be reconstructed. This makes the method extremely prone to overfitting during iterative optimization, including unnatural components in the image caused by undersampling. Furthermore, hyperparameters such as the learning rate, regularization penalty coefficient, and number of iterations have a significant impact on image quality and become difficult to adjust, making it difficult to reconstruct high-quality images. In addition, the total variation constraint used in this method is a relatively traditional prior constraint in compressed sensing methods, which has limited constraint power during image reconstruction.

[0026] The patent application CN202211439367.4, entitled "A Differential Single-Pixel Imaging System and Imaging Method Based on Untrained Neural Network Constraints", proposes a method that is basically the same as the method proposed in the previous document, but the claims focus on describing a general imaging optical device for single-pixel imaging.

[0027] Another related technique proposes an algorithm for low-order computer vision tasks (image denoising, image super-resolution, and image deblurring) that embeds the denoiser prior into the depth image prior. This technique uses an encoder-decoder network structure, which suffers from problems similar to other methods, being prone to overfitting due to its more than 10 million parameters. Secondly, the denoiser prior used in this method is a traditional iterative optimization algorithm (NLM, BM3D, etc.), which consumes more computational resources and results in lower image restoration quality. Furthermore, this method is only applied to image processing and not to single-pixel imaging or other computational optical imaging fields.

[0028] To address the aforementioned issues, this application proposes a low-parameter residual decoder network, controlling the number of network parameters to be on the same order of magnitude as the number of image parameters. This ensures stable convergence during the iterative process of image reconstruction, avoiding overfitting without relying on additional methods and effectively reducing computational resources. Furthermore, this application introduces a depth denoiser prior, employing a denoising network with a noise level estimator, enabling good imaging results even in high-noise environments. Based on the physical model of single-pixel imaging, an optimization problem is established. Combining the proposed residual decoder network and depth denoiser prior, the Alternating Direction Multiplier Method (ADMM) is used for iterative solution to complete image reconstruction. The overall process is as follows: Figure 2 As shown.

[0029] Therefore, embodiments of this application provide a single-pixel imaging method and related equipment based on a non-trained residual decoder and denoising prior, relating to the field of image processing technology. The single-pixel imaging method and related equipment based on a non-trained residual decoder and denoising prior provided in this application can be applied to terminals, servers, or software running on terminals or servers. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited to these; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the server can also be a node server in a blockchain network; the software can be an application implementing the single-pixel imaging method based on a non-trained residual decoder and denoising prior, but is not limited to the above forms.

[0030] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0031] Reference Figure 3 This application provides a single-pixel imaging method based on an untrained residual decoder and denoising prior. This method may include, but is not limited to, steps S300 to S330, as follows: S300: Acquire the signal sequence sampled by the single-pixel imaging device; S310: Input the random tensor corresponding to the imaging into the untrained residual decoder to obtain the image to be reconstructed output by the untrained residual decoder; S320: Construct an optimization problem for single-pixel imaging based on the signal sequence, the image to be reconstructed, and the noise term; wherein the optimization problem includes a denoising prior; S330: Solve the inverse problem of the single-pixel imaging sampling model using the optimization problem to obtain the reconstructed image of the single-pixel imaging; wherein, the process of solving the inverse problem of the sampling model includes denoising prior.

[0032] In some embodiments, the untrained residual decoder includes several decoding layers and an output layer; each decoding layer is composed of the following structure in sequence: nearest neighbor upsampling, residual structure, sigmoid activation function, and layer normalization, wherein the residual structure consists of a convolutional layer with a kernel size of 3×3, a SiLU activation function, and a convolutional layer with a kernel size of 3×3, and the residual skip connection adds the input features of the previous convolutional layer with a kernel size of 3×3 and the output features of the next convolutional layer with a kernel size of 3×3 element-wise; the output layer is composed of the following structure in sequence: a convolutional layer with a kernel size of 3×3, a SiLU activation function, layer normalization, a 1×1 convolution, and a Sigmoid activation function; The step of inputting the random tensor corresponding to the imaging into the untrained residual decoder to obtain the image to be reconstructed output by the untrained residual decoder includes the following steps: The random tensor corresponding to the image The input is fed into the untrained residual decoder, and during each nearest-neighbor upsampling of the decoding layer, The height and width are both magnified by a factor of 2, and finally, the image to be reconstructed is obtained through the output layer. : ; in, Represent real numbers, and These are the height and width of the image to be reconstructed, respectively. The number of decoding layers. The number of feature map channels. For network parameter set, This indicates the forward propagation computation of the untrained residual decoder.

[0033] In some embodiments, the single-pixel imaging sampling model is constructed through the following steps: Construct the single-pixel imaging sampling model: ; in, The signal sequence is... , For the sampling matrix, , The image to be reconstructed, , For the noise term, ; This indicates the dimension of the vector.

[0034] In some embodiments, an optimization problem for single-pixel imaging is constructed based on the signal sequence, the image to be reconstructed, and noise terms, including the following steps: The optimization problem is constructed as follows: ; in, The regularization penalty coefficient is... This represents the denoising prior.

[0035] In some embodiments, solving the inverse problem of the single-pixel imaging sampling model using the optimization problem to obtain the reconstructed image of the single-pixel imaging includes the following steps: The optimization problem is solved iteratively using the alternating direction multiplier method to solve the inverse problem of the single-pixel imaging sampling model, thereby obtaining the reconstructed image; wherein, the augmented Lagrangian function corresponding to the optimization problem is expressed as: ; in, The penalty coefficient for the regularization term of the constraint condition. These are Lagrange multiplier vectors; The unknowns that need to be solved in the augmented Lagrangian function are respectively , and During the solution process, each iteration updates the three unknowns sequentially, specifically including: for Updates, fixed and Find the value of and solve the following model: ; Among them, due to For the untrained residual decoder The network parameters are such that the gradient descent optimizer in deep learning is used during the solution process. Update; for Updates, fixed and Find the value of and solve the following model: ; for The update is calculated using the following iterative formula: ; in, , , , , They are respectively , and The values ​​of the j-th and (j-1)-th iterations; After k iterations and convergence, take As the reconstructed image.

[0036] In some embodiments, the denoising prior step includes the following steps: A noise level map of the noisy image in the sampling model is determined using a noise estimator; The noise level map and the noisy image are input into the U-Net network, and the U-Net network is used to perform denoising to obtain the denoised image corresponding to the noisy image.

[0037] In some embodiments, acquiring the signal sequence sampled by a single-pixel imaging device includes the following steps: The signal sequence is obtained by sampling through a spatial light modulator or a digital micromirror device.

[0038] The following sections will provide a detailed description and explanation of some optional embodiments of this application, using specific application examples.

[0039] Specifically, this embodiment includes the following technical solutions: 1. Network structure of non-trained residual decoder.

[0040] This embodiment proposes a non-trained residual decoder, the network structure of which is as follows: Figure 4 As shown, the network structure consists of several decoder layers and one output layer. Each decoder layer comprises: Nearest Upsample, a residual structure, a Sigmoid LinearUnit (SiLU) activation function, and layer normalization. The residual structure consists of a 3×3 convolutional layer (3×3 Conv), a SiLU activation function, and another 3×3 convolutional layer. Residual skip connections element-wise sum the input features of the preceding 3×3 convolutional layer and the output features of the following 3×3 convolutional layer. The output layer comprises: a 3×3 convolutional layer, a SiLU activation function, layer normalization, a 1×1 convolution, and a Sigmoid activation function. The network input is a random tensor. During each upsampling operation at a decoding layer, its height and width are magnified by a factor of 2, and finally, an image is output after passing through the output layer. The output image can be represented as: ; in, and These represent the height and width of the output image, respectively. The number of decoding layers. The number of feature map channels. This is a set of network parameters.

[0041] 2. Iterative solution algorithm (ADMM).

[0042] The sampling process of a single-pixel imaging sampling model can be represented by a linear model: ; in, The sampled signal sequence, This is a sampling matrix, where each row represents the encoded pattern loaded onto the spatial light modulator during sampling. The image obtained by rearranging the image pixels. Noise term; Represent real numbers, This indicates the dimension of the vector.

[0043] Combining the above two equations, the optimization problem of the single-pixel imaging reconstruction method proposed in this embodiment can be expressed as: ; in, The regularization penalty coefficient is... This indicates a denoising operation using a deep denoising network.

[0044] This embodiment uses the alternating direction multiplier method (ADMM) for iterative solution, and its augmented Lagrangian function can be expressed as: ; in, The penalty coefficient for the regularization term of the constraint condition. Let be the Lagrange multiplier vector. In the formula, there are three sets of unknowns that need to be solved, namely... , and During the solution process, each iteration requires updating these three sets of unknowns sequentially.

[0045] for Updates need to be fixed. and Find the value of and solve the following model: ; because For the network The network parameters are such that the gradient descent optimizer commonly used in deep learning can be used during the solution process. Update.

[0046] for Updates need to be fixed. and Find the value of and solve the following model: ; for The update is calculated using the following iterative formula: ; In the above formulas, , , , , They are respectively , and The values ​​of the j-th and (j-1)-th iterations. After k iterations and convergence, take... The image obtained from the reconstruction.

[0047] 3. Deep denoising prior.

[0048] This embodiment uses a deep denoising network as a denoising prior, i.e., the one described in Section 2. Denoising Operation. Conventional denoising methods typically require explicitly specifying the noise variance of the image as a parameter. However, in the iterative solution algorithm of this embodiment, image denoising is a gradual process. Determining the noise level of the image by manually specifying or fixing it is difficult to adapt to changes in the noise level during the iteration process, and local noise levels may also vary, resulting in an unsatisfactory overall reconstruction effect. To solve these problems, this embodiment uses an adaptive denoising network structure, which consists of two stages: the first stage is a noise estimator composed of 5 convolutional layers. When a noisy image is input to the noise estimator, it can output a noise level map, representing the local noise level at different locations in the image; the second stage uses a U-Net network as a denoiser, taking both the noisy image and the estimated noise level map as input to achieve adaptive denoising. The denoising network structure used in this embodiment is as follows: Figure 5 As shown.

[0049] As an example, one possible implementation is as follows: In an embodiment, a single-pixel imaging system such as Figure 1 As shown, it includes a digital micromirror device, a single-pixel detector, a data acquisition card, an LED light source, an imaging lens, and a collecting lens; a sampling matrix. The generation of the Hadamard matrix can be represented as a set of iterative equations: ; in, For Kronecker product.

[0050] The imaging configuration was set to: image size 256×256 pixels, sampling rates of 1.56%, 3.12%, and 6.25%, respectively. The residual decoder network configuration was set to: input tensor dimension... tensor The network parameters are generated randomly and uniformly, with 6 decoding layers and 32 feature map channels, resulting in a total of approximately 121,000 parameters, which is about 1.85 times the number of image pixels. The number of iterations is fixed at 20,000.

[0051] Imaging tests were conducted in both normal and high-noise environments. The reconstruction results in the normal environment are as follows: Figure 6 As shown, the reconstruction results of the high-noise environment are as follows: Figure 7 As shown.

[0052] In summary, the beneficial effects of this embodiment include at least the following: (1) To address the problem that overfitting occurs during iterations in single-pixel imaging reconstruction methods based on untrained neural networks, where the network structure typically uses networks with tens of millions of parameters, resulting in poor image reconstruction quality, this embodiment proposes a low-parameter residual decoder network. This reduces the number of network parameters to the same order of magnitude as the number of pixels in the reconstructed image, effectively curbing overfitting during image reconstruction and significantly improving image quality even in low-sampling-rate environments in single-pixel imaging. The network structure employs a layer-by-layer progressive decoder structure, incorporating residual skip connections in each decoding layer and introducing a novel SiLU activation function. This allows the network to maximize its information representation capabilities while reducing the number of parameters.

[0053] (2) This embodiment uses an adaptive denoising network as a denoising prior, which can greatly improve the noise resistance of single-pixel imaging, so that high-quality images can be reconstructed even in high-noise environments.

[0054] Reference Figure 8 This application also provides a single-pixel imaging system based on a non-trained residual decoder and denoising prior, which can implement the above-mentioned single-pixel imaging method based on a non-trained residual decoder and denoising prior. The system includes: The sampling signal acquisition unit is used to acquire the signal sequence sampled by the single-pixel imaging device; An image processing unit is used to input the random tensor corresponding to the imaging into a non-trained residual decoder to obtain the image to be reconstructed output by the non-trained residual decoder. A sampling modeling unit is used to construct an optimization problem for single-pixel imaging based on the signal sequence, the image to be reconstructed, and noise terms; wherein the optimization problem includes a denoising prior. The image reconstruction unit is used to solve the inverse problem of the single-pixel imaging sampling model using the optimization problem to obtain the reconstructed image of the single-pixel imaging; wherein, the process of solving the inverse problem of the sampling model includes denoising prior.

[0055] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0056] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method of this application. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0057] It is understood that the content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the methods of this application, and the beneficial effects achieved are the same as those achieved by the methods of this application.

[0058] Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 101 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 102 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 102 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 102 and is called and executed by the processor 101. Input / output interface 103 is used to implement information input and output; The communication interface 104 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 105 transmits information between various components of the device (e.g., processor 101, memory 102, input / output interface 103, and communication interface 104); The processor 101, memory 102, input / output interface 103 and communication interface 104 are connected to each other within the device via bus 105.

[0059] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of this application.

[0060] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0061] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0062] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0063] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0064] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0065] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0066] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0067] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0068] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0069] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0070] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0071] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0072] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0073] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A single-pixel imaging method based on a non-trained residual decoder and denoising prior, characterized in that, The method includes the following steps: Acquire the signal sequence sampled by a single-pixel imaging device; The random tensor corresponding to the imaging is input into the untrained residual decoder to obtain the image to be reconstructed output by the untrained residual decoder. An optimization problem for single-pixel imaging is constructed based on the signal sequence, the image to be reconstructed, and the noise term; wherein, the optimization problem includes a denoising prior. The inverse problem of the single-pixel imaging sampling model is solved using the optimization problem to obtain the reconstructed image of the single-pixel imaging; wherein, the process of solving the inverse problem of the sampling model includes denoising prior.

2. The single-pixel imaging method based on a non-trained residual decoder and denoising prior according to claim 1, characterized in that, The untrained residual decoder comprises several decoding layers and an output layer. Each decoding layer consists of the following structures: nearest neighbor upsampling, a residual structure, a sigmoid activation function, and layer normalization. The residual structure comprises a 3×3 convolutional layer, a SiLU activation function, and another 3×3 convolutional layer. Residual skip connections element-wise add the input features of the preceding 3×3 convolutional layer and the output features of the following 3×3 convolutional layer. The output layer consists of the following structures: a 3×3 convolutional layer, a SiLU activation function, layer normalization, a 1×1 convolution, and a Sigmoid activation function. The step of inputting the random tensor corresponding to the imaging into the untrained residual decoder to obtain the image to be reconstructed output by the untrained residual decoder includes the following steps: The random tensor corresponding to the image is input into the untrained residual decoder. When the image passes through the nearest neighbor upsampling layer of the decoding layer, the height and width of the random tensor are magnified by a factor of 2. Finally, the image to be reconstructed is obtained by passing through the output layer.

3. The single-pixel imaging method based on a non-trained residual decoder and denoising prior according to claim 2, characterized in that, The single-pixel imaging sampling model is constructed through the following steps: A linear model is constructed as the single-pixel imaging sampling model; The output of the linear model is the signal sequence, the input of the linear model is the image to be reconstructed, the input coefficients are the sampling matrix, and the bias term of the linear model is the noise term.

4. The single-pixel imaging method based on a non-trained residual decoder and denoising prior according to claim 3, characterized in that, The optimization problem of constructing single-pixel imaging based on the signal sequence, the image to be reconstructed, and the noise term includes the following steps: The optimization problem is constructed based on the signal sequence, the image to be reconstructed, and the noise term, including a regularization penalty and a denoising prior.

5. The single-pixel imaging method based on a non-trained residual decoder and denoising prior according to claim 4, characterized in that, The process of solving the inverse problem of the single-pixel imaging sampling model using the optimization problem to obtain the reconstructed image of the single-pixel imaging includes the following steps: The optimization problem is solved iteratively using the alternating direction multiplier method to solve the inverse problem of the single-pixel imaging sampling model, thereby obtaining the reconstructed image; The steps for solving the optimization problem include: Determine the augmented Lagrangian function corresponding to the optimization problem; The unknowns that need to be solved in the augmented Lagrange function are determined to be the network parameters of the untrained residual decoder, the image to be reconstructed, and the Lagrange multiplier vector; During the solution process, the three unknown quantities are updated sequentially in each iteration; After multiple iterations and convergence, the image to be reconstructed at the convergence point is subtracted from the Lagrange multiplier vector to obtain the reconstructed image.

6. The single-pixel imaging method based on a non-trained residual decoder and denoising prior according to claim 5, characterized in that, The denoising prior step includes the following steps: A noise level map of the noisy image in the sampling model is determined using a noise estimator; The noise level map and the noisy image are input into the U-Net network, and the U-Net network is used to perform denoising to obtain the denoised image corresponding to the noisy image.

7. The single-pixel imaging method based on a non-trained residual decoder and denoising prior according to any one of claims 1 to 6, characterized in that, The acquisition of the signal sequence sampled by the single-pixel imaging device includes the following steps: The signal sequence is obtained by sampling through a spatial light modulator or a digital micromirror device.

8. A single-pixel imaging system based on a non-trained residual decoder and denoising prior, characterized in that, The system includes: The sampling signal acquisition unit is used to acquire the signal sequence sampled by the single-pixel imaging device; An image processing unit is used to input the random tensor corresponding to the imaging into a non-trained residual decoder to obtain the image to be reconstructed output by the non-trained residual decoder. A sampling modeling unit is used to construct an optimization problem for single-pixel imaging based on the signal sequence, the image to be reconstructed, and noise terms; wherein the optimization problem includes a denoising prior. The image reconstruction unit is used to solve the inverse problem of the single-pixel imaging sampling model using the optimization problem to obtain the reconstructed image of the single-pixel imaging; wherein, the process of solving the inverse problem of the sampling model includes denoising prior.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images