Frequency domain enhanced single-pixel imaging method and system

Abstract

The invention discloses a frequency domain enhanced single-pixel imaging method and a frequency domain enhanced single-pixel imaging system, which are used for acquiring a single-pixel measurement value corresponding to a target object and performing image reconstruction based on the single-pixel measurement value to obtain a target imaging result. The method comprises the following steps that a single-pixel imaging network is constructed, the single-pixel imaging network comprises an input layer, a plurality of coding layers, a plurality of decoding layers and an output layer which are connected in sequence, the number of the coding layers is the same as that of the decoding layers, and each coding layer comprises a basic convolution layer, a Swindow-Transform module and a frequency domain MLP module which are connected in sequence; based on the single-pixel measurement value, performing iterative training on the single-pixel imaging network according to a double-domain self-supervision method, and obtaining a first reconstructed image and a second reconstructed image in each iteration; and taking the optimal first reconstructed image as a target imaging result. The method can give consideration to both imaging quality and reconstruction speed.

Description

Frequency Domain Enhanced Single-Pixel Imaging Method and System Technical Field

[0001] This invention relates to the field of single-pixel imaging, and more particularly to a frequency-domain enhanced single-pixel imaging method and system. Background Technology

[0002] Single-pixel imaging (SPI) is a novel technology based on computational imaging principles. This technology encodes the light source using spatial light modulators such as digital micromirror devices (DMDs), projects different structured light patterns onto the target object, and uses a single photodetector with no spatial resolution to collect the total light intensity signal after reflection or transmission from the object. Finally, by calculating the correlation between the projection pattern and the detection signal, an image of the target object is reconstructed. Unlike traditional imaging methods that use focal plane detectors to receive two-dimensional images, SPI requires only a single detector to achieve image reconstruction. Furthermore, SPI systems are capable of operating in multiple wavelengths, including visible, ultraviolet, infrared, and even terahertz, making them suitable for various imaging scenarios that traditional cameras struggle with, such as terahertz imaging, hyperspectral imaging, microscopic imaging, and underwater imaging. However, because the single-pixel detector acquires compressed measurements of the entire target at a time, this imaging mechanism makes it difficult to reconstruct high-quality target images under low sampling rate conditions.

[0003] To overcome the aforementioned problems, existing research mainly focuses on two directions: imaging mode optimization and reconstruction algorithm improvement. The core of imaging mode optimization lies in improving the encoding efficiency of target information in the measurement signal. Commonly used modes include Hadamard, Fourier basis, and random scattering basis. However, these modes are often limited by projection equipment or specific imaging tasks, making them difficult to widely apply to various practical scenarios. At the algorithm level, traditional methods such as pseudo-inverse matrix method (Pse-Inv), differential ghost imaging (DGI), normalized ghost imaging (NGI), and compressed sensing ghost imaging based on total variational constraints (CS-TVGI) have been proposed to improve reconstruction quality, but their high computational complexity leads to long image restoration times.

[0004] In recent years, deep learning has been gradually introduced into single-pixel imaging. Lyu et al. were the first to explore the application potential of data-driven deep learning in SPI, verifying the feasibility of this direction. Wang et al. designed an end-to-end neural network and used simulation data to pre-train the model, achieving image reconstruction from one-dimensional measurements. Wu et al. further improved reconstruction accuracy by introducing structures such as residual connections, dense blocks, and attention mechanisms. Although these methods have shown some effectiveness, they usually rely on a large amount of paired data for supervised training, and the training process is time-consuming. In addition, real images of the target scene are often difficult to obtain in practice, and if the distribution of the target to be imaged differs significantly from that of the training set, the trained model needs to readjust its parameters. This strongly supervised learning mechanism greatly limits the practical application scope of deep learning in SPI.

[0005] Based on the Deep Image Prior (DIP) theory, neural networks themselves can act as effective regularizers, capturing implicit prior information in image reconstruction and thus establishing an effective connection between training-based and non-training-based methods. Utilizing this property, Liu et al. and Li et al. proposed two self-supervised learning frameworks, UNNCGI and URNet, respectively, embedding a forward physical model of single-pixel imaging into the network structure. This allows for direct reconstruction of target images from one-dimensional measurements without the need for real image labels. These methods significantly reduce the dependence of traditional deep learning methods on large-scale labeled data.

[0006] To further improve imaging efficiency, Wang et al. proposed the FPGI method, which uses only a single-layer convolutional network as the reconstruction model to achieve fast imaging with fewer iterations. However, such self-supervised methods typically only constrain by minimizing measurement domain errors, and in complex scenes, insufficient regularization can easily lead to unstable reconstruction results or degraded quality.

[0007] To enhance reconstruction accuracy, Bian et al. proposed a self-supervised dual-domain constraint imaging model: first, an initial reconstructed image is obtained using a traditional pseudo-inverse matrix method, and then input into a neural network for optimization; the network output, after an invertible transformation, serves as a virtual label, which is used to generate measurement values ​​again through an imaging feedforward model, forming a closed-loop constraint. This method introduces transform domain constraints outside the measurement domain, improving reconstruction accuracy. However, its two-stage reconstruction process relies on the preliminary results of traditional algorithms, which may lead to information loss, and the reconstruction performance is significantly affected by the chosen algorithm. Summary of the Invention

[0008] This invention addresses the shortcomings of existing single-pixel imaging schemes that lack the ability to balance reconstruction quality and reconstruction speed by providing a frequency-domain enhanced single-pixel imaging method and system.

[0009] To solve the above-mentioned technical problems, the present invention provides a frequency-domain enhanced single-pixel imaging method, which obtains the single-pixel measurement value corresponding to the target object, and performs image reconstruction based on the single-pixel measurement value to obtain the target imaging result. The method includes the following steps: constructing a single-pixel imaging network, which includes an input layer, several encoding layers, several decoding layers, and an output layer connected in sequence, wherein the number of encoding layers and decoding layers is the same, and the encoding layer consists of a basic convolutional layer, a Swin-Transformer module, and a frequency-domain MLP module connected in sequence; iteratively training the single-pixel imaging network based on the single-pixel measurement value using a dual-domain self-supervised method, obtaining a first reconstructed image and a second reconstructed image in each iteration; the first reconstructed image is the image reconstructed by the single-pixel imaging network based on the single-pixel measurement value, and the second reconstructed image is the image reconstructed by the single-pixel imaging network based on the reconstructed single-pixel measurement value corresponding to the first reconstructed image; and using the optimal first reconstructed image as the target imaging result.

[0010] As one possible implementation, in the current iteration step, the specific steps for training the single-pixel imaging network based on the dual-domain self-supervised method are as follows: inputting the corresponding single-pixel measurement values ​​into the single-pixel imaging network to obtain a first reconstructed image; performing a single-pixel forward imaging process on the first reconstructed image to obtain the corresponding reconstructed single-pixel measurement values; inputting the reconstructed single-pixel measurement values ​​into the single-pixel imaging network to obtain a second reconstructed image; calculating the test value reconstruction loss based on the single-pixel measurement values ​​and the reconstructed single-pixel measurement values; calculating the image consistency loss based on the first reconstructed image and the second reconstructed image; calculating the total variation loss of the first reconstructed image; determining the total loss based on the test value reconstruction loss, the image consistency loss, and the total variation loss; and guiding the training of the single-pixel imaging network based on the total loss.

[0011] As one possible implementation method, the formula for calculating the total variation loss is: In the formula: TV(T) is the total variation loss of the first reconstructed image T; T x,y It is the pixel value at coordinates (x, y) in the first reconstructed image T; T x+1,y It is the pixel value at coordinates (x+1, y) in the first reconstructed image T; T x,y+1 It is the pixel value at coordinates (x, y+1) in the first reconstructed image T.

[0012] As one possible implementation, the iteration step with the best reconstruction effect is determined based on the total loss corresponding to each iteration, and the first reconstructed image corresponding to this iteration step is output as the target imaging result.

[0013] One possible implementation is as follows: the basic convolutional layer includes two convolutional blocks, each of which includes a convolutional layer, a batch normalization layer, and a LeakyReLU layer connected in sequence.

[0014] One possible implementation is a single-pixel imaging network comprising an input layer, a five-layer encoder, a five-layer decoder, and an output layer connected in sequence; the input layer is a fully connected layer; the output layer is a Sigmoid layer; and the decoder comprises an upsampling layer, a convolutional layer, a batch normalization layer, and a LeakyReLU layer connected in sequence.

[0015] This invention also provides a frequency-domain enhanced single-pixel imaging system for acquiring single-pixel measurement values ​​corresponding to a target object, and performing image reconstruction based on the single-pixel measurement values ​​to obtain a target imaging result. The system includes: a network construction module for constructing a single-pixel imaging network, the single-pixel imaging network comprising an input layer, several encoding layers, several decoding layers, and an output layer connected in sequence, wherein the number of encoding layers and decoding layers is the same, and the encoding layers are sequentially connected basic convolutional layers, a Swin-Transformer module, and a frequency-domain MLP module; an imaging module for iteratively training the single-pixel imaging network based on the single-pixel measurement values ​​using a dual-domain self-supervised method, obtaining a first reconstructed image and a second reconstructed image in each iteration, the first reconstructed image being the image reconstructed by the single-pixel imaging network based on the single-pixel measurement values, and the second reconstructed image being the image reconstructed by the single-pixel imaging network based on the reconstructed single-pixel measurement values ​​corresponding to the first reconstructed image; and an output module for using the optimal first reconstructed image as the target imaging result.

[0016] The present invention has significant technical effects due to the adoption of the above technical solutions: By designing a single-pixel imaging network, the present invention retains the inherent advantages of self-supervised learning, combines a lightweight network structure, introduces a Swing-Transformer module and a frequency domain MLP module to enhance the feature representation ability of the model and perform adaptive brightness correction on the output image. This not only ensures high image reconstruction quality, but also significantly improves the reconstruction speed. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 is a schematic diagram of the network architecture of the single-pixel imaging network FDE-Net of the present invention; Figure 2 is a schematic diagram of the network architecture of each module of the encoder and the decoder in Figure 1; Figure 3 is a schematic diagram of the training process of the single-pixel imaging network based on the dual-domain self-supervised method; Figure 4 is a schematic diagram of the simulation results of the five algorithms on the Urban100 dataset in the comparative experiment, with an image size of 64×64; Figure 5 is a schematic diagram of the quantitative evaluation results of SSIM of the five algorithms at sampling rates of 20% to 40% in the comparative experiment; Figure 6 is a schematic diagram of the quantitative evaluation results of SSIM of the five algorithms at sampling rates of 2% to 10% in the comparative experiment; Figure 7 is a schematic diagram of the ablation experiment results; Figure 8 is a schematic diagram of the device used in the actual experiment (a) and a schematic diagram of the experimental reconstruction results of the five algorithms at a sampling rate of 10% (b). Detailed Implementation

[0019] The present invention will be further described in detail below with reference to the embodiments. The following embodiments are explanations of the present invention, but the present invention is not limited to the following embodiments.

[0020] Existing deep learning methods for single-pixel imaging can be divided into two categories based on their learning methods: supervised and self-supervised. Self-supervised single-pixel imaging does not rely on a large amount of paired training data and usually has better generalization performance than supervised methods. However, since self-supervised training lacks real label constraints, its regularization ability is relatively limited. In addition, single-pixel imaging itself is an ill-conditioned inverse problem, and if the network's expressive power is insufficient, the reconstruction result is prone to converge to an incorrect solution. Furthermore, existing self-supervised methods are relatively slow in reconstruction speed and need further optimization.

[0021] To address the shortcomings of existing self-supervised single-pixel imaging schemes, this application proposes a frequency-domain enhanced single-pixel imaging method, comprising the following steps: S100, acquiring the single-pixel measurement value corresponding to the target object; The single-pixel imaging (SPI) system mainly consists of a light source, a spatial light modulator (e.g., a digital micromirror device, DMD), the target to be imaged (i.e., the target object), and a single-pixel detector. Its workflow is as follows: The light beam generated by the light source is first encoded and modulated by the spatial light modulator, and then illuminates the target object. The reflected or transmitted light field is finally received by the single-pixel detector, which lacks spatial resolution, and the total light intensity information is recorded; Mathematically, the first... The light intensity signal collected by the single-pixel detector during this measurement can be expressed as:

[0022] Where P(x,y) represents the th The modulation pattern is loaded once, T(x,y) is the true image distribution of the target object, and (x,y) is the spatial coordinate; a total of m measurements are performed in the entire imaging process.

[0023] In this embodiment, the measurement value of a single pixel is determined based on the light intensity signal detected by the single pixel detector. This is prior art and will not be described in detail in this specification.

[0024] S200. Based on the single-pixel measurement value, image reconstruction is performed to obtain the target imaging result; the specific steps are as follows: S210. Construct a single-pixel imaging network FDE-Net; Referring to Figure 1, the single-pixel imaging network includes an input layer, several encoding layers, several decoding layers and an output layer connected in sequence, wherein the number of encoding layers and decoding layers is the same; in this embodiment, a five-layer encoder and a five-layer decoder are used, wherein the encoder extracts multi-scale features step by step through convolutional blocks and max pooling, and the decoder restores spatial resolution through upsampling and skip connections.

[0025] Referring to Figure 1, in this embodiment: the input layer is a fully connected layer, namely FC+ReShape in Figure 1.

[0026] The output layer is a Sigmoid layer, where Sigmoid is an existing activation function.

[0027] Referring to Figure 2, the decoder includes an upsampling layer, a convolutional layer (Conv2d), a batch normalization layer (BN), and a LeakyReLU layer connected in sequence, where LeakyReLU is an existing activation function.

[0028] Referring to Figures 1 and 2, the coding layer consists of a basic convolutional layer, a Swin-Transformer module, and a frequency domain MLP (Multilayer Perceptron) module connected in sequence. Referring to Figure 2, the basic convolutional layer includes two convolutional blocks, each of which includes a convolutional layer (Conv2d), a batch normalization layer (BN), and a LeakyReLU layer connected in sequence.

[0029] The Swin-Transformer module, STB, is a hierarchical visual transformer. Referring to Figure 2, it includes an image patch embedding layer (winembed), layer normalization (layernorm), and an attention module. The Swin-Transformer module divides the feature map into non-overlapping local windows through the window embedding layer, and uses a multi-head self-attention mechanism to calculate spatial dependencies within the windows. Combined with layer normalization to stabilize the training process, it effectively captures long-distance dependencies while maintaining computational efficiency. In this embodiment, at the bottleneck layer, the network concatenates and fuses the STB-processed features at different scales to achieve effective integration of multi-scale features.

[0030] The frequency domain MLP module, abbreviated as FreMLP, as shown in Figure 2, breaks through the limitations of traditional spatial domain processing. It transforms the features to the frequency domain through fast Fourier transform, decomposes them into amplitude spectrum and phase spectrum, and then uses a lightweight multilayer perceptron to nonlinearly enhance the amplitude spectrum. The amplitude spectrum is generally related to the brightness of the image. Then, the spatial features are recovered through inverse transform. This frequency domain processing mechanism is particularly good at performing brightness correction on images without changing the global structural information and periodic texture patterns of the image.

[0031] This embodiment effectively enhances feature representation and image brightness correction capabilities through the design of the Swing-Transformer module and the frequency domain MLP module.

[0032] S220. Based on the single-pixel measurement value, the single-pixel imaging network is iteratively trained according to the dual-domain self-supervised method. Each iteration can obtain a first reconstructed image and a second reconstructed image. The first reconstructed image is the image reconstructed and output by the single-pixel imaging network based on the single-pixel measurement value, and the second reconstructed image is the image reconstructed and output by the single-pixel imaging network based on the reconstructed single-pixel measurement value corresponding to the first reconstructed image.

[0033] Referring to Figure 3, the specific steps for training the single-pixel imaging network based on the dual-domain self-supervised method in the current iteration step are as follows: Input the corresponding single-pixel measurement value I into the single-pixel imaging network FDE-Net to obtain a first reconstructed image T; perform a single-pixel forward imaging process on the first reconstructed image T to obtain the corresponding reconstructed single-pixel measurement value I', i.e., PT in the figure. In this embodiment, the actual imaging process is simulated by multiplying and summing a reconstructed image T with the illumination mode P; input the reconstructed single-pixel measurement value I' into the single-pixel imaging network FDE-Net to obtain a second reconstructed image T'; calculate the test value reconstruction loss based on the single-pixel measurement value I and the reconstructed single-pixel measurement value I', which is a measurement domain loss term; calculate the image consistency loss based on the first reconstructed image T and the second reconstructed image T', which is an image domain loss term; use the dual-domain loss constraint to converge the FDE-Net to the correct reconstructed image.

[0034] As one possible implementation, the total loss is determined based on the test value reconstruction loss, image consistency loss, and total variation loss, and the training of the single-pixel imaging network is guided by the total loss; wherein, the formula for calculating the total variation loss is: In the formula: TV(T) is the total variation loss of the first reconstructed image T; T x,y It is the pixel value at coordinates (x, y) in the first reconstructed image T; T x+1,y It is the pixel value at coordinates (x+1, y) in the first reconstructed image T; T x,y+1It is the pixel value at coordinates (x, y+1) in the first reconstructed image T.

[0035] In this embodiment, the loss function is: LOSS = L1(I,I') + α·L1(T,T') Where: Loss represents the total loss; L1(I,I') represents the reconstruction loss of the test value calculated based on the single pixel measurement value I and the reconstructed single pixel measurement value I'; L1(T,T') is the image consistency loss calculated based on the first reconstructed image T and the second reconstructed image T'; TV(T) is the total variation loss of the first reconstructed image T; α and β are weighting parameters, which can be set by those skilled in the art according to actual needs.

[0036] The aforementioned loss function first constrains the reconstructed single-pixel measurement value I' obtained by the forward imaging model from the network output image through the measurement value reconstruction loss L1(I,I') to ensure consistency with the actual single-pixel measurement value I, thus ensuring that the reconstruction process conforms to the physical model. Secondly, by introducing the image consistency loss αL1(T,T'), the reconstructed single-pixel measurement value I' is input into the same network again to obtain the secondary reconstruction result, i.e., the second reconstructed image T', and its consistency with the initial output first reconstructed image T is constrained by the L1 norm, thereby enhancing the model's robustness to measurement noise and output stability. Finally, the total variation regularization term TV(T) applies a smoothness constraint to the initial output first reconstructed image T, increasing the regularization constraint of self-supervised training and improving the visual quality of the reconstructed image. This embodiment, through the design of the loss function, optimizes it from three dimensions: measurement consistency, self-supervised cyclic consistency, and image prior, effectively driving the network to achieve high-quality, high-reliability single-pixel image reconstruction.

[0037] S230. Use the best first reconstructed image as the target imaging result.

[0038] In this embodiment, the iteration step with the best reconstruction effect is determined based on the total loss corresponding to each iteration, and the first reconstructed image corresponding to the iteration step is output as the target imaging result.

[0039] If the iteration ends based on the convergence result, the optimal first reconstructed image is the first reconstructed image output after convergence; if the iteration ends based on the maximum number of iterations, the optimal first reconstructed image is the first reconstructed image corresponding to the iteration step with the minimum total loss.

[0040] Experimental Results and Discussion: Comparative Experiments: This paper verifies the effectiveness of the proposed method and module through comparative experiments on the Urban100 and Set11 datasets. For the Urban100 dataset, we used images of two different scales: 64×64 and 128×128, respectively. For the Urban100 dataset, except for the method presented in this paper, all other experimental results are cited from Bian's work; the weighting parameters α in the loss function are 0.2, and β are 0.2 and 0.0000002. This experiment uses the PyTorch framework to build the network model and is deployed on an NVIDIA RTX 3090 GPU platform with 24GB of VRAM. To objectively evaluate model performance, Structural Similarity (SSIM) is used as a quantitative evaluation metric. SSIM evaluates image quality by measuring the similarity of images in features such as brightness and contrast. A value close to 1 indicates that the reconstructed image is closer to the original image, and the reconstruction effect is more ideal.

[0041] For 64*64 building images from the Urban100 dataset and house images from the Set11 dataset, the reconstruction results of Differential Ghosting Imaging (DGI), Pseudo-Inverse Matrix Method (Pse-Inv), Compressed Sensing Total Variational Method (CS-TV), Bian's SD-Net method, and the proposed method FDE-Net were verified at different sampling rates, as shown in Figures 4 to 6. Figure 4 shows the reconstructed images at various sampling rates, while Figures 5 and 6 show the quantitative results of the SSIM index for each algorithm. It can be seen that the reconstructed images of DGI and Pse-Inv at various sampling rates contain a lot of noise. Although CS-TV recovers some image information, the generated images appear unnatural due to excessive smoothness. SD-Net, due to its special dual-domain structure, further improves the image reconstruction quality. The imaging method provided in this application, due to the design of the STB and FreMLP modules, can discover more image features and reconstruct more natural images. This improvement is more obvious at lower sampling rates, as shown in the index results.

[0042] The 64×64 image was reconstructed using the five methods described above. The reconstruction time and SSIM values ​​are shown in Table 1. The sampling rate was 0.5; the CS-TV iteration count was 300; SD-Net and FDE-Net achieved optimal reconstruction results at 200 and 500 iterations, respectively. (Table 1)

[0043] As shown in the table above, DGI, Pse-Inv, and CS-TV have shorter reconstruction times but poorer reconstruction results; FDE-Net offers a significant improvement in imaging speed, with a reconstruction time approximately one-tenth that of SD-Net. FDE-Net also achieves the best reconstruction results among the five algorithms.

[0044] Ablation Experiment: To verify the effect of FreMLP on image reconstruction, we conducted a corresponding ablation experiment on the FreMLP module. The results are shown in Figure 7. When the number of iterations was 1000, the output image after network convergence and the feature map of the first module of the encoder part, namely the FreMLP_1 feature map, were extracted. As shown in Figure 7, when the FreMLP module is present, the brightness of the feature map is greater than that without FreMLP, and the output image is also brighter. This indicates that the FreMLP module has an adaptive brightness adjustment function.

[0045] Actual Experiment: To further verify the effectiveness of our method in real-world scenarios, the SPI experimental system shown in Figure 8(a) was built. White light was used to directly illuminate the DMD, which was modulated by the pattern loaded in the DMD and then magnified by a lens to illuminate the object. Finally, the light was reflected to the detector for collection. Figure 8(b) shows the results of five algorithms reconstructing a 64×64 image at a 10% sampling rate. As can be seen from the figure, the reconstruction result of FDE-Net has higher image fidelity, less background noise, and better reconstruction quality than other methods.

[0046] This application also proposes a frequency-domain enhanced single-pixel imaging system for acquiring single-pixel measurement values ​​corresponding to a target object, and performing image reconstruction based on the single-pixel measurement values ​​to obtain a target imaging result. The system includes: a network construction module for constructing a single-pixel imaging network, which comprises an input layer, several encoding layers, several decoding layers, and an output layer connected in sequence, wherein the number of encoding and decoding layers is the same, and the encoding layers are sequentially connected basic convolutional layers, a Swin-Transformer module, and a frequency-domain MLP module; an imaging module for iteratively training the single-pixel imaging network based on the single-pixel measurement values ​​using a dual-domain self-supervised method, obtaining a first reconstructed image and a second reconstructed image in each iteration; the first reconstructed image being the image reconstructed by the single-pixel imaging network based on the single-pixel measurement values, and the second reconstructed image being the image reconstructed by the single-pixel imaging network based on the reconstructed single-pixel measurement values ​​corresponding to the first reconstructed image; and an output module for using the optimal first reconstructed image as the target imaging result.

[0047] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0048] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0049] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0050] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0051] These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable terminal equipment, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0052] It should be noted that the phrase "an embodiment" or "embodiment" mentioned in the specification means that a specific feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Therefore, the phrase "an embodiment" or "embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment.

[0053] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0054] Furthermore, it should be noted that the shapes and names of the parts and components described in the specific embodiments described in this specification may differ. All equivalent or simple variations made to the structure, features, and principles described in this patent concept are included within the protection scope of this patent. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to replace them, as long as they do not depart from the structure of this invention or exceed the scope defined in these claims, they should all fall within the protection scope of this invention.

Claims

1. A frequency-domain enhanced single-pixel imaging method, comprising acquiring single-pixel measurement values ​​corresponding to a target object, performing image reconstruction based on the single-pixel measurement values, and obtaining a target imaging result; characterized in that, The process includes the following steps: constructing a single-pixel imaging network, which includes an input layer, several coding layers, several decoding layers, and an output layer connected in sequence, wherein the number of coding layers and decoding layers is the same, and the coding layer consists of a basic convolutional layer, a Swin-Transformer module, and a frequency domain MLP module connected in sequence. Based on the single-pixel measurement value, the single-pixel imaging network is iteratively trained according to the dual-domain self-supervised method. Each iteration can obtain a first reconstructed image and a second reconstructed image. The first reconstructed image is the image reconstructed by the single-pixel imaging network based on the single-pixel measurement value, and the second reconstructed image is the image reconstructed by the single-pixel imaging network based on the reconstructed single-pixel measurement value corresponding to the first reconstructed image. The optimal first reconstructed image is taken as the target imaging result.

2. The frequency-domain enhanced single-pixel imaging method according to claim 1, characterized in that, In the current iteration step, the specific steps for training the single-pixel imaging network based on the dual-domain self-supervised method are as follows: input the corresponding single-pixel measurement value into the single-pixel imaging network to obtain a first reconstructed image; perform a single-pixel forward imaging process on the first reconstructed image to obtain the corresponding reconstructed single-pixel measurement value; input the reconstructed single-pixel measurement value into the single-pixel imaging network to obtain a second reconstructed image; calculate the test value reconstruction loss based on the single-pixel measurement value and the reconstructed single-pixel measurement value; calculate the image consistency loss based on the first reconstructed image and the second reconstructed image. Calculate the total variation loss of the first reconstructed image; determine the total loss based on the reconstruction loss of the test value, the image consistency loss, and the total variation loss; and guide the training of the single-pixel imaging network based on the total loss.

3. The frequency-domain enhanced single-pixel imaging method according to claim 2, characterized in that, The formula for calculating the total variation loss is as follows: In the formula: TV(T) is the total variation loss of the first reconstructed image T; T x,y It is the pixel value at coordinates (x, y) in the first reconstructed image T; T x+1,y It is the pixel value at coordinates (x+1, y) in the first reconstructed image T; T x,y+1 It is the pixel value at coordinates (x, y+1) in the first reconstructed image T.

4. The frequency-domain enhanced single-pixel imaging method according to claim 2, characterized in that, The iteration step with the best reconstruction effect is determined based on the total loss corresponding to each iteration, and the first reconstructed image corresponding to this iteration step is output as the target imaging result.

5. The frequency domain enhanced single-pixel imaging method according to any one of claims 1 to 4, characterized in that: The basic convolutional layer consists of two convolutional blocks, each of which includes a convolutional layer, a batch normalization layer, and a LeakyReLU layer connected in sequence.

6. The frequency domain enhanced single-pixel imaging method according to any one of claims 1 to 4, characterized in that: The single-pixel imaging network comprises an input layer, a five-layer encoder, a five-layer decoder, and an output layer connected in sequence; the input layer is a fully connected layer; the output layer is a Sigmoid layer; and the decoder comprises an upsampling layer, a convolutional layer, a batch normalization layer, and a LeakyReLU layer connected in sequence.

7. A frequency-domain enhanced single-pixel imaging system, used to acquire single-pixel measurement values ​​corresponding to a target object, and to perform image reconstruction based on the single-pixel measurement values ​​to obtain a target imaging result; characterized in that, include: A network construction module is used to construct a single-pixel imaging network, which includes an input layer, several coding layers, several decoding layers and an output layer connected in sequence. The number of coding layers and decoding layers is the same. The coding layers are basic convolutional layers connected in sequence, a Swin-Transformer module and a frequency domain MLP module. An imaging module is used to iteratively train the single-pixel imaging network based on the single-pixel measurement value using a dual-domain self-supervised method. Each iteration can obtain a first reconstructed image and a second reconstructed image. The first reconstructed image is the image reconstructed by the single-pixel imaging network based on the single-pixel measurement value, and the second reconstructed image is the image reconstructed by the single-pixel imaging network based on the reconstructed single-pixel measurement value corresponding to the first reconstructed image. An output module is used to take the optimal first reconstructed image as the target imaging result.

Images