Deep learning single-pixel imaging method and system under high dynamic scattering medium
By using an end-to-end deep learning reconstruction model, combined with adaptive spectral correction and Hadamard inverse transform, the problem of image reconstruction for single-pixel imaging in dynamic scattering media was solved, achieving high-quality underwater imaging results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HEFEI UNIV OF TECH
- Filing Date
- 2025-12-12
- Publication Date
- 2026-04-24
AI Technical Summary
Existing deep learning single-pixel imaging methods struggle to effectively train and reconstruct high-quality images in dynamic scattering media, especially in turbid underwater environments. Traditional methods rely on simulation data and have poor generalization capabilities.
An end-to-end deep learning reconstruction model is adopted, which combines a learnable signal preprocessing layer and an image reconstruction network. Through adaptive spectral correction and Hadamard inverse transform, the target image is directly reconstructed from a one-dimensional light intensity measurement sequence. The image reconstruction network adopts an improved Unet++ structure.
It achieves stable and high-quality imaging in highly dynamic scattering environments, simplifies the imaging process, improves adaptability, solves the problems of difficult model training and reliance on simulation data, and significantly improves the peak signal-to-noise ratio and structural similarity of image reconstruction.
Smart Images
Figure CN121921401A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of single-pixel imaging technology, and in particular to a deep learning single-pixel imaging method and system under highly dynamic scattering media. Background Technology
[0002] In the field of computational imaging, single-pixel imaging, as an advanced imaging technology, has undergone significant development and refinement since its inception. Due to its wide imaging spectral range and excellent performance in low light, it has been applied in many industrial scenarios. Single-pixel imaging is a new technology distinct from traditional imaging. By projecting encoded patterns and utilizing a single-pixel detector to obtain corresponding light intensity values, combined with corresponding mathematical transformations, non-local reconstruction of the target can be achieved.
[0003] The underwater environment contains numerous scattering media characterized by complex internal refractive and reflective index distributions. These properties cause light to deviate from its original propagation direction and reduce the signal-to-noise ratio as light passes through the scattering medium. Therefore, traditional imaging techniques struggle to achieve clear underwater images. Compared to traditional imaging methods, single-pixel imaging (SPI) offers higher sensitivity and the ability to image under extremely low light conditions, providing significant advantages for underwater imaging. Deep learning-based SPI can be categorized into data-driven and physics-driven approaches. Physics-driven deep learning models are built upon the premise of a known illumination pattern passing through the scattering medium. Turbid water, a dynamic scattering medium, differs from static scattering media like frosted glass; the illumination pattern becomes unknowable after passing through a dynamic scattering medium. Data-driven deep learning SPI methods are further divided into image-to-image and intensity-to-image approaches. Image-to-image SPI deep learning models rely on pre-reconstructing low-quality images using SPI algorithms and then inputting these low-quality images into the deep learning model; this approach is more akin to image denoising. Intensity-to-image SPI deep learning methods often face the challenge of difficult model training. Moreover, previous studies have used simulation methods to obtain underwater turbid water datasets. Specifically, the method involves using illumination patterns to form target images and adding different levels of Gaussian noise to simulate underwater conditions with varying degrees of turbidity. These simulated data are then used to train deep learning models, which are then used to reconstruct real underwater data. This method often fails to simulate the real underwater environment. Summary of the Invention
[0004] The purpose of this invention is to overcome the shortcomings of the existing technology. To achieve the above objective, a deep learning single-pixel imaging method and system under a highly dynamic scattering medium is adopted to solve the problems mentioned in the background technology.
[0005] A deep learning-based single-pixel imaging method for highly dynamic scattering media includes the following steps: Acquire a one-dimensional light intensity measurement sequence collected by a single-pixel imaging system; The one-dimensional light intensity measurement sequence is input into an end-to-end deep learning reconstruction model, which includes a learnable signal preprocessing layer and an image reconstruction network. The learnable signal preprocessing layer is used to adaptively correct the spectrum of the input one-dimensional light intensity measurement sequence according to a trainable parameter, and perform Hadamard inverse transform on the intermediate data obtained after correction to generate two-dimensional data input to the image reconstruction network. The image reconstruction network is used to receive the intermediate data and directly reconstruct the target image.
[0006] As a further aspect of the present invention: the learnable signal preprocessing layer is a Hadamard layer based on Hadamard transform, which adaptively corrects the measured values corresponding to the positive and negative Hadamard base modes in the one-dimensional light intensity measurement sequence.
[0007] As a further aspect of the present invention, the adaptive spectrum correction is achieved through the following steps: Let the measurements corresponding to the positive and negative base modes be respectively and The weighted difference is calculated using the following formula:
[0008]
[0009]
[0010] in, These are trainable parameters; The original measured value is corrected using the weighted difference to obtain the corrected value, as shown in the formula:
[0011]
[0012] right and After performing difference calculations, an inverse Hadamard transform is performed to obtain intermediate data.
[0013] As a further aspect of the present invention: the trainable parameters The optimization objective is to adaptively correct spectral distortion caused by scattering from the imaging medium, and its optimization process is independent of the parameter optimization process of the image reconstruction network.
[0014] As a further aspect of the present invention: the image reconstruction network is a neural network based on an encoder-decoder structure, and the downsampling path and upsampling path therein have been structurally improved to enhance the peak signal-to-noise ratio and structural similarity of the reconstructed image.
[0015] As a further aspect of the present invention, the image reconstruction network is an improved Unet++ network.
[0016] As a further aspect of the present invention: during model training, the trainable parameters The image reconstruction network is optimized using a first optimizer, and its parameters are optimized using a second optimizer.
[0017] As a further aspect of the present invention, the dataset of the deep learning reconstruction model is constructed by acquiring multiple sets of one-dimensional light intensity measurement sequences and their corresponding real target images in an imaging environment containing scattering media of different concentrations.
[0018] As a further aspect of the present invention, the imaging environment of different concentrations of scattering media is simulated by adding the scattering media to the water and stirring it.
[0019] The second aspect of the technical solution: a single-pixel imaging system, comprising: A single-pixel imaging device used to acquire a one-dimensional light intensity measurement sequence of a target; A processor configured to perform a single-pixel imaging method as described in any of the preceding claims to reconstruct an image of the target from the one-dimensional light intensity measurement sequence.
[0020] Compared with the prior art, the present invention has the following technical advantages: The above-described technical solution involves constructing and applying an end-to-end deep learning reconstruction model. This method first acquires a one-dimensional light intensity measurement sequence from a single-pixel imaging system, then directly inputs it into the reconstruction model. The model integrates a learnable signal preprocessing layer and an image reconstruction network: the preprocessing layer adaptively corrects the spectrum of the original light intensity sequence using a trainable parameter to compensate for distortions introduced by medium scattering, etc.; the corrected data is then directly processed by the subsequent image reconstruction network without manual intervention or traditional reconstruction algorithms, outputting the final target image. The beneficial effect of this method is that it fundamentally changes the traditional two-step process of "reconstruction first, optimization later," achieving end-to-end mapping from the original measurement signal to a high-quality image. This not only significantly simplifies the imaging process and improves adaptability to complex dynamic scattering environments, but also effectively solves key problems such as reliance on simulation data, poor generalization ability, and difficulties in end-to-end model training through the self-learning mechanism of the model's internal parameters, ultimately achieving stable and high-quality imaging under harsh conditions such as high-turbidity underwater environments. Attached Figure Description
[0021] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings: Figure 1 This is a schematic diagram illustrating the steps of a single-pixel imaging method according to an embodiment of this application; Figure 2 This is a schematic diagram illustrating the principle of an embodiment disclosed in this application; Figure 3 This is a schematic diagram of the model flow of an embodiment disclosed in this application; Figure 4 This is a diagram of the experimental apparatus according to an embodiment of this application; Figure 5 The results of handwritten Chinese character experiments are shown in the embodiments disclosed in this application. Detailed Implementation
[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0023] Please refer to Figure 1 In this embodiment of the invention, a deep learning single-pixel imaging method under a highly dynamic scattering medium includes the following steps: Step S1: Obtain the one-dimensional light intensity measurement sequence acquired by the single-pixel imaging system; The data is collected using experimental equipment to create the datasets required for the experiment. Step S2: Input the one-dimensional light intensity measurement sequence into an end-to-end deep learning reconstruction model, the reconstruction model including a learnable signal preprocessing layer and an image reconstruction network; The learnable signal preprocessing layer is used to adaptively correct the spectrum of the input one-dimensional light intensity measurement sequence according to a trainable parameter, and perform Hadamard inverse transform on the intermediate data obtained after correction to generate two-dimensional data input to the image reconstruction network. The image reconstruction network is used to receive the intermediate data and directly reconstruct the target image.
[0024] In this embodiment, the learnable signal preprocessing layer is a Hadamard layer based on Hadamard transform, which adaptively corrects the measured values corresponding to the positive and negative Hadamard base modes in the one-dimensional light intensity measurement sequence.
[0025] In this embodiment, trainable parameters The optimization objective is to adaptively correct spectral distortion caused by scattering from the imaging medium, and its optimization process is independent of the parameter optimization process of the image reconstruction network.
[0026] In this embodiment, the image reconstruction network is a neural network based on an encoder-decoder structure, and the downsampling path and upsampling path have been structurally improved to enhance the peak signal-to-noise ratio and structural similarity of the reconstructed image.
[0027] In this embodiment, the image reconstruction network is an improved Unet++ network.
[0028] In this embodiment, during model training, the trainable parameters The image reconstruction network is optimized using a first optimizer, and its parameters are optimized using a second optimizer.
[0029] Specifically, such as Figure 2 The diagram illustrates the principle of the proposed Hadamard-Unet++ single-pixel imaging scheme. Hadamard single-pixel imaging is based on the Hadamard transform; it obtains the Hadamard spectrum of the target image and then reconstructs the target image using the inverse Hadamard transform. (Hadamard matrix) It is a size of × An orthogonal matrix whose elements contain only +1 and -1. Define a 1×1 Hadamard matrix. =[1], then it can be defined recursively. The Hadamard matrix when >0.
[0030]
[0031] The principle formula for Hadamard single-pixel imaging is as follows:
[0032]
[0033] in These are coordinates in the image space domain, while These are the coordinates in the Hadamard field. (The expression is missing.) This represents the inverse Hadamard transform. Because there are no negative values in the projected image of the DMD, HSI requires a difference operation. Specifically, and The corresponding equation is the Hadamard spectrum. The equation is:
[0034] in and These represent the imaging target in the projection image. and Measurements obtained under irradiation.
[0035] Because white patterns have significantly higher light intensity compared to other patterns, this higher intensity light is retained after being absorbed by water, resulting in more noise from backscattering. Black patterns, on the other hand, have the opposite effect. For these reasons, single-pixel imaging technology is less effective in turbid water. The value is significantly higher than its value under normal conditions. This phenomenon leads to a decrease in the contrast of the spectrum and the image recovered from the spectrum. Because the higher the turbidity, the greater the contrast in the spectrum... The greater the impact, the more the Hadamard layer designed in this invention solves the parameter tuning problem under different concentrations using a trainable parameter.
[0036] In this embodiment, adaptive spectrum correction is achieved through the following steps: Let the measurements corresponding to the positive and negative base modes be respectively and The weighted difference is calculated using the following formula:
[0037]
[0038]
[0039] in, These are trainable parameters; The original measured value is corrected using the weighted difference to obtain the corrected value, as shown in the formula:
[0040]
[0041] right and After performing difference calculations, an inverse Hadamard transform is performed to obtain intermediate data.
[0042] in, This represents the first data point of the collected one-dimensional light intensity value. This represents the second data point of the collected one-dimensional light intensity value. This represents the weighted difference. minus , Plus After obtaining the spectrally corrected data, performing differential calculations on the processed data yields the data for the Hadamard inverse transform. Parameters Iterative optimization can be performed directly using the Adam optimizer, and the optimization of the Hdamard layer and the parameter optimization of Unet++ are performed separately.
[0043] like Figure 2 As shown in the diagram, this is a schematic diagram of the entire model principle. The input model is one-dimensional data, and the output model is an image, realizing an end-to-end deep learning single-pixel model and solving the problem of difficulty in training previous deep learning single-pixel imaging schemes.
[0044] like Figure 3 As shown in the diagram, the model's flowchart illustrates the process. The first processing layer of the model receives the raw one-dimensional light intensity measurement value without differential processing. It first performs spectral correction based on learnable parameters. Then, differential calculation is performed on the corrected data, and it is reconstructed into two-dimensional data through inverse Hadamard transform. Finally, the two-dimensional data is input into the core layer of the Unet++ network for further image reconstruction and optimization.
[0045] In this embodiment, the dataset for the deep learning reconstruction model is constructed by collecting multiple sets of one-dimensional light intensity measurement sequences and their corresponding real target images in an imaging environment containing scattering media of different concentrations.
[0046] In this embodiment, the imaging environment of different concentrations of scattering medium is simulated by adding scattering medium to the water and stirring it.
[0047] In a specific implementation, to construct the training dataset, handwritten Chinese characters at a resolution of 32×32 and image data of varying complexity were collected. The experimental setup used for dataset collection was as follows: Figure 4 As shown, the main components include: a projector (model DLP7001, maximum projection resolution 1024×768 pixels), a photodetector (model PDA100A2, wavelength response range 320-1100nm), a data acquisition card (model PCIe-6374), and a custom-made water tank (dimensions: length 300mm, width 300mm, height 150mm, glass thickness 6mm). In the experiment, the target image is displayed on an e-ink screen, and the playback and acquisition are controlled synchronously by a computer.
[0048] The specific operating procedure is as follows: The projected pattern passes sequentially through the glass and the water medium in the tank, illuminating the ink screen, which serves as the imaging target. The light intensity signal reflected from the target is received by a photodetector and converted into a digital signal by a data acquisition card. To simulate a high-dynamic scattering environment, milk is added to the clean water in the tank as a scattering medium, and a stirrer is used to continuously stir the water at a speed of 1000 r / min. To fully train the network, the experiment was set with five turbidity levels ranging from 0 NTU (clean water) to 80 NTU. Generally, turbidity exceeding 50 NTU is considered a high-turbidity water condition.
[0049] The generally recognized metrics in the field of digital images, such as peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), are used to quantitatively analyze the reconstruction performance of this method in terms of principle. The definition of PSNR is:
[0050] where is the quantization depth of the image, is the original image and the reconstructed image The root mean square error between them is defined as
[0051]
[0052] M and N are the horizontal and vertical resolutions of the image respectively. The larger the PSNR value, the better the quality of the simulated reconstructed image. The value of SSIM ranges from [-1, 1]. When this value is closer to 1, it indicates that the image reconstruction effect is closer to the original image. Its expression is:
[0053] where and are the average values of the original image and the reconstructed image respectively, and are the variances of i and j respectively, is and The covariance of.
[0054] As Figure 5 shown, the experiment conducted imaging tests on three Chinese characters with different stroke complexities. The column on the left marked with turbidity in the figure is the result directly reconstructed by the Hadamard single-pixel imaging technology, and the column on the right is the result restored by using the model of the present invention. The experiment shows that at a turbidity of 60 NTU, the structural similarity (SSIM) index of the images reconstructed by the method of the present invention all reaches above 0.8. Compared with the direct imaging result, the index is improved by about twice. At an ultra-high turbidity of 80 NTU, for the relatively simple Chinese characters "Pi" and "Gen", the SSIM index can still remain above 0.7; even for the relatively complex Chinese character "Dan", the SSIM index can also reach above 0.5.
[0055] The above experimental results prove that the present invention can effectively cope with the imaging challenges under high dynamic scattering media. By deeply integrating the single-pixel imaging algorithm and the adaptive spectrum correction module into the deep learning framework, a Hadamard single-pixel deep learning imaging model of end-to-end is successfully realized.
[0056] The second aspect of the technical solution: a single-pixel imaging system, comprising: A single-pixel imaging device used to acquire a one-dimensional light intensity measurement sequence of a target; A processor configured to perform a single-pixel imaging method as described in any of the preceding claims to reconstruct an image of the target from the one-dimensional light intensity measurement sequence.
[0057] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention. The scope of the invention is defined by the appended claims and their equivalents, all of which should be included within the scope of protection of the invention.
Claims
1. A deep learning single-pixel imaging method under highly dynamic scattering media, characterized in that, Includes the following steps: Acquire a one-dimensional light intensity measurement sequence collected by a single-pixel imaging system; The one-dimensional light intensity measurement sequence is input into an end-to-end deep learning reconstruction model, which includes a learnable signal preprocessing layer and an image reconstruction network. The learnable signal preprocessing layer is used to adaptively correct the spectrum of the input one-dimensional light intensity measurement sequence according to a trainable parameter, and to perform Hadamard inverse transform on the intermediate data obtained after correction to generate two-dimensional data input to the image reconstruction network. The image reconstruction network is used to receive the intermediate data and directly reconstruct the target image.
2. The deep learning single-pixel imaging method under a high dynamic scattering medium according to claim 1, characterized in that, The learnable signal preprocessing layer is a Hadamard layer based on the Hadamard transform, which adaptively corrects the measured values corresponding to the positive and negative Hadamard base modes in the one-dimensional light intensity measurement sequence.
3. The deep learning single-pixel imaging method under a high dynamic scattering medium according to claim 2, characterized in that, The adaptive spectrum correction is achieved through the following steps: Let the measurements corresponding to the positive and negative base modes be respectively and The weighted difference is calculated using the following formula: in, These are trainable parameters; The original measured value is corrected using the weighted difference to obtain the corrected value, as shown in the formula: right and After performing difference calculations, an inverse Hadamard transform is performed to obtain intermediate data.
4. The deep learning single-pixel imaging method under a highly dynamic scattering medium according to claim 3, characterized in that, The trainable parameters The optimization objective is to adaptively correct spectral distortion caused by scattering from the imaging medium, and its optimization process is independent of the parameter optimization process of the image reconstruction network.
5. The deep learning single-pixel imaging method under a high dynamic scattering medium according to claim 1, characterized in that, The image reconstruction network is a neural network based on an encoder-decoder structure, and the downsampling and upsampling paths have been structurally improved to enhance the peak signal-to-noise ratio and structural similarity of the reconstructed image.
6. The deep learning single-pixel imaging method under a high dynamic scattering medium according to claim 5, characterized in that, The image reconstruction network is an improved Unet++ network.
7. The deep learning single-pixel imaging method under a high dynamic scattering medium according to claim 4, characterized in that, During model training, the trainable parameters The image reconstruction network is optimized using a first optimizer, and its parameters are optimized using a second optimizer.
8. The deep learning single-pixel imaging method under a high dynamic scattering medium according to claim 1, characterized in that, The dataset for the deep learning reconstruction model is constructed by collecting multiple sets of one-dimensional light intensity measurement sequences and their corresponding real target images in an imaging environment containing scattering media of different concentrations.
9. The deep learning single-pixel imaging method under a high dynamic scattering medium according to claim 8, characterized in that, The imaging environment of different concentrations of scattering media is simulated by adding the scattering media to the water and stirring it.
10. A single-pixel imaging system, characterized in that, include: A single-pixel imaging device used to acquire a one-dimensional light intensity measurement sequence of a target; A processor configured to perform the single-pixel imaging method as described in any one of claims 1 to 9 to reconstruct an image of the target from the one-dimensional light intensity measurement sequence.