Multi-input joint smart single-pixel imaging method and device thereof

Abstract

The present application relates to the technical field of stable detection in the ground security technology system, and particularly relates to a multi-input joint intelligent single-pixel imaging method and device thereof, which comprises: acquiring speckle patterns and corresponding light intensity values of light beams passing through objects; constructing a convolutional neural network based on a U-Net framework; training the convolutional neural network based on a designed loss function until convergence, and outputting a final image. The present application embeds a physical model into a neural network by using detected light intensity information and speckle information, optimizes network parameters by using intensity values collected by a detector as a reference standard, improves the generalization and interpretability of the network, optimizes parameters by combining one-dimensional input and two-dimensional input, and further improves imaging quality.

Description

Technical Field

[0001] This invention relates to the technical field of stable detection in near-site security technology systems, specifically to a multi-input joint intelligent single-pixel imaging method and device. Background Technology

[0002] Single-pixel imaging utilizes spatial light modulation technology to project structured light patterns onto the target scene, encoding the scene's spatial light information into a one-dimensional light intensity signal. A single-pixel detector collects this one-dimensional light signal containing the scene's spatial information, and the structured light pattern and the one-dimensional light intensity signal are jointly processed to reconstruct the object image. Single-pixel imaging breaks through the limitations of the traditional "point-to-point" imaging paradigm. Due to the excellent detection characteristics of single-pixel detectors, it has advantages in low-light conditions or in detecting weak signals. Furthermore, because the imaging system does not rely on the stability and consistency of the pixel array, it is more reliable and stable in special environments or conditions.

[0003] With the development of deep learning technology, convolutional neural networks have been widely used in the field of image processing. Because neural networks have powerful feature extraction capabilities, they can compensate for information loss during the acquisition of detection signals and are widely used to solve ill-conditioned inverse problems such as single-pixel imaging.

[0004] Existing data-driven single-pixel imaging methods require extensive sampling, are only applicable to similar imaging scenarios, exhibit poor generalization, and struggle to explain the imaging mechanism, resulting in poor interpretability. Neural networks, when processing one-dimensional signal inputs to generate two-dimensional images, suffer from significant randomness in the generated images. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a multi-input joint intelligent single-pixel imaging method. This method fully utilizes detected light intensity and speckle information, embeds a physical model into a neural network, and optimizes network parameters using the intensity values ​​collected by the detector as a reference standard, thereby improving the network's generalization and interpretability. The addition of two-dimensional input enhances the predictability and stability of the network's generated results, while the one-dimensional input contains prior image information. The joint optimization of parameters by one-dimensional and two-dimensional inputs further improves imaging quality.

[0006] The purpose of this invention is to provide a multi-input joint intelligent single-pixel imaging method, comprising:

[0007] Obtain the speckle pattern and the corresponding light intensity value of the beam passing through the object to form a one-dimensional light intensity sequence; obtain random images;

[0008] Convolutional neural networks based on the U-Net framework are constructed; one-dimensional light intensity sequences and random images are used as inputs to the convolutional neural network to generate their respective corresponding images;

[0009] Two generated images are obtained and speckle is calculated to obtain the corresponding intensity values; a loss function is designed based on the difference between the corresponding intensity value of the generated image and the light intensity value, as well as the difference in pixels between the two generated images.

[0010] Based on the designed loss function, the convolutional neural network is trained until it converges and outputs the final image.

[0011] Preferably, the speckle pattern is generated by modulating the light source with a spatial light modulator to produce a light pattern consistent with the speckle information.

[0012] More preferably, the light intensity value is the total light intensity of the modulated light pattern illuminating the object to be imaged, collected and recorded by a single-pixel bucket collector.

[0013] Preferably, the intensity values ​​corresponding to the two images are obtained according to the following steps:

[0014] Let a one-dimensional light intensity sequence be used as the input to a convolutional neural network, and the generated image be denoted as img. 1D ;

[0015] Let a random image be used as input to a convolutional neural network, and the generated image be denoted as img. 2D ;

[0016] Using the speckle pattern as the convolution kernel to... 1D and img 2D Perform convolution operations with a stride of 1 to obtain the corresponding intensity values.

[0017] More preferably, the loss function is calculated using the following formula:

[0018]

[0019] In the formula, α represents the weight, which gradually increases with the number of iterations to achieve the initial weight of I at the start of training. 1Di I 2Di and intensity value I i The difference is the primary reference, and in later stages, the img value is used as the main reference. 1D and img 2D The difference is the primary reference; I 1Di I 2Di These represent the generated image (img). 1D and img 2D The corresponding light intensity value I is obtained through speckle calculation. i The expression represents the light intensity of a beam passing through an object; mean() calculates the average.

[0020] mean(I 1Di -I i )2 The image (img) generated from a 1D input 1D The intensity sequence estimate I obtained by performing an inner product operation with speckle. 1Di and light intensity value I i The mean square error; mean(I 2Di -I i ) 2 The image (img) generated from a 2D input. 2D The intensity sequence estimate I obtained by performing an inner product operation with speckle. 2Di and light intensity value I i The mean square error.

[0021] Preferably, when training the convolutional neural network, the parameters of the convolutional neural network are adjusted by adjusting the weights in the loss function each time until convergence, and the image used in the last optimization is output. 1D or img 2D As the final image.

[0022] Preferably, when constructing a convolutional neural network based on the U-Net framework, the U-Net network includes 5 encoding layers and 5 decoding layers. The encoding layers consist of double convolution and pooling layers, and the decoding layers consist of transposed convolution and double convolution. The decoder layers are also connected to the features of the corresponding encoder layers to fuse semantic information from different layers.

[0023] Preferably, the one-dimensional light intensity sequence formed by the light intensity values ​​of the light beam passing through the object corresponding to the speckle pattern is obtained by receiving it through a single-pixel detector.

[0024] Preferably, the random image is a two-dimensional image in which each pixel is set to a random number between 0 and 1, and the random number is generated by the rand function.

[0025] A second objective of this invention is to provide a multi-input joint intelligent single-pixel imaging device, comprising:

[0026] The data acquisition module is used to acquire speckle patterns and the light intensity values ​​of corresponding beams passing through objects; acquire one-dimensional light intensity sequences transmitted or reflected by objects; and acquire random images.

[0027] The data processing module is used to construct a convolutional neural network based on the U-Net framework; it takes a one-dimensional light intensity sequence and a random image as inputs to the convolutional neural network to generate corresponding images; it obtains the corresponding intensity values ​​from the two generated images and speckle calculations; and it designs a loss function based on the difference between the intensity value of the generated image and the light intensity value, as well as the difference in pixels between the two generated images.

[0028] The image extraction module is used to train the convolutional neural network based on the designed loss function until it converges and outputs the final image.

[0029] Compared with the prior art, the present invention has at least the following beneficial effects:

[0030] This invention provides a multi-input joint intelligent single-pixel imaging method and apparatus, overcoming the problems of data-driven single-pixel imaging methods requiring extensive sampling, poor generalization and interpretability, and the randomness of image generation from one-dimensional signal input. This method fully utilizes detected light intensity and speckle information, embeds a physical model into a neural network, and optimizes network parameters using the intensity values ​​collected by the detector as a reference standard, improving the network's generalization and interpretability. The addition of two-dimensional input enhances the predictability and stability of the network's generated results, while the one-dimensional input contains prior image information. The joint optimization of parameters by one-dimensional and two-dimensional inputs further improves imaging quality. Attached Figure Description

[0031] Figure 1 This is a flowchart of a multi-input joint intelligent single-pixel imaging method according to the present invention. Detailed Implementation

[0032] In order to illustrate the technical means and effects adopted by the present invention in order to achieve the intended purpose of the invention, the following detailed description is provided in conjunction with the embodiments.

[0033] This invention provides a multi-input joint intelligent single-pixel imaging method, see [link to relevant documentation]. Figure 1 As shown, it includes:

[0034] S1. Obtain the speckle pattern and the corresponding light intensity value of the beam passing through the object to form a one-dimensional light intensity sequence; obtain a random image;

[0035] The speckle pattern is generated by modulating the light source using a spatial light modulator, resulting in speckle information P. i Consistent light pattern.

[0036] The light intensity value is the total light intensity of the object to be imaged, which is collected and recorded by a single-pixel bucket collector after modulation of the light pattern.

[0037] Specifically, the one-dimensional light intensity sequence, consisting of the light intensity values ​​of the beam passing through the object corresponding to the speckle pattern, is obtained by receiving data through a single-pixel detector. A random image is a two-dimensional image where each pixel is set to a random number between 0 and 1, generated by the rand function.

[0038] In this embodiment, in S1, speckle information P is imported. i And collect the light intensity I transmitted through the object. i ;

[0039] S1.1, Turn on the laser source to illuminate the DMD;

[0040] S1.2, based on speckle information P i Modulate light to generate beams of different shapes;

[0041] S1.3, the modulated light beam is directed onto the object, and the intensity I of the light beam passing through the object is collected. i ;

[0042] Where P i It is three-dimensional data, composed of (n, w, h). Here, n represents the number of speckle patterns, and w and h represent the width and height of a single speckle pattern, respectively. i It is one-dimensional data, composed of (n), where n represents the number of intensity values. The purpose of importing speckle information is for the image... 1D and img 2D The corresponding intensity value I is obtained through speckle calculation. 1Di I 2Di With the detected value I i This constitutes part of the joint loss.

[0043] S2. Construct a convolutional neural network based on the U-Net framework;

[0044] One-dimensional light intensity sequences and random images are used as inputs to a convolutional neural network to generate their respective corresponding images;

[0045] Two generated images are obtained and speckle is calculated to obtain the corresponding intensity values; a loss function is designed based on the difference between the corresponding intensity value of the generated image and the light intensity value, as well as the difference in pixels between the two generated images.

[0046] When constructing a convolutional neural network based on the U-Net framework, the U-Net network includes 5 encoding layers and 5 decoding layers. The encoding layer consists of double convolution and pooling layers, and the decoding layer consists of transposed convolution and double convolution. The decoder layer is also connected to the features of the corresponding encoder layer to fuse semantic information from different layers.

[0047] The intensity values ​​corresponding to the two images are obtained according to the following steps:

[0048] Let a one-dimensional light intensity sequence be used as the input to a convolutional neural network, and the generated image be denoted as img. 1D ;

[0049] Let a random image be used as input to a convolutional neural network, and the generated image be denoted as img. 2D ;

[0050] Using the speckle pattern as the convolution kernel to... 1D and img 2D Perform convolution operations with a stride of 1 to obtain the corresponding intensity values.

[0051] The loss function is calculated using the following formula:

[0052]

[0053] In the formula, α represents the weight, which gradually increases with the number of iterations to achieve the initial weight of I at the start of training. 1Di I 2Di and intensity value I i The difference is the primary reference, and in later stages, the img value is used as the main reference. 1D and img 2D The difference is the primary reference; mainly to achieve the desired result at the start of training using I. 1Di I 2Di and intensity value I i The difference is the primary reference, and in later stages, the img value is used as the main reference. 1D and img 2D The difference is used as the main reference for the process of optimizing parameters, which further improves the quality of the reconstructed image; the parameters of the convolutional neural network are adjusted by adjusting the weights in the loss function.

[0054] I 1Di I 2Di These represent the generated image (img). 1D and img 2D The light intensity value Ii obtained by speckle calculation represents the light intensity value of the beam passing through the object; mean() represents the average value.

[0055] mean(I 1Di -I i ) 2 The image (img) generated from a 1D input 1D The intensity sequence estimate I obtained by performing an inner product operation with speckle. 1Di and light intensity value I i The mean square error;

[0056] mean(I 2Di -I i ) 2 The image (img) generated from a 2D input. 2D The intensity sequence estimate I obtained by performing an inner product operation with speckle. 2Di and light intensity value I i The mean square error.

[0057] In this embodiment, the constructed convolutional neural network accepts the one-dimensional light intensity sequence I of S1 as input and the random image img as input, and generates their respective corresponding images.

[0058] img 1D =f cnn (I) (1)

[0059] img 2D =f cnn (img) (2)

[0060] speckle pattern P i As a convolution kernel for img 1D and img 2D Perform convolution operations separately with a stride of 1 to obtain the corresponding intensity values ​​I. 1Di and I 2Di .

[0061] I 1Di =img 1D *P i (3)

[0062] I 2Di =img 2D *P i (4)

[0063] S3. Based on the designed loss function, train the convolutional neural network until it converges and output the final image.

[0064] Specifically, when training a convolutional neural network, the parameters are adjusted by changing the weights in the loss function each time until convergence, and the final image (img) used in the last optimization is output. 1D or img 2D As the final image.

[0065] This invention provides a multi-input joint intelligent single-pixel imaging device, comprising:

[0066] The data acquisition module is used to obtain speckle patterns and the light intensity values ​​of the corresponding light beams passing through the object, forming a one-dimensional light intensity sequence; and to acquire random images.

[0067] The data processing module is used to construct a convolutional neural network based on the U-Net framework; it takes a one-dimensional light intensity sequence and a random image as inputs to the convolutional neural network to generate corresponding images; it obtains the corresponding intensity values ​​from the two generated images and speckle calculations; and it designs a loss function based on the difference between the intensity value of the generated image and the light intensity value, as well as the difference in pixels between the two generated images.

[0068] The image extraction module is used to train the convolutional neural network based on the designed loss function until it converges and outputs the final image.

[0069] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-input joint intelligent single-pixel imaging method, characterized in that, include: The speckle pattern and the corresponding light intensity value of the beam passing through the object are obtained to form a one-dimensional light intensity sequence; Get random images; Convolutional neural networks based on the U-Net framework are constructed; one-dimensional light intensity sequences and random images are used as inputs to the convolutional neural network to generate their respective corresponding images; The intensity values ​​are obtained by performing inner product operations between the two generated images and the speckle pattern; a loss function is designed based on the difference between the corresponding intensity value of the generated image and the light intensity value, as well as the difference in pixels between the two generated images. Based on the designed loss function, the convolutional neural network is trained until it converges and outputs the final image.

2. The multi-input joint intelligent single-pixel imaging method according to claim 1, characterized in that, The speckle pattern is generated by modulating the light source with a spatial light modulator to produce a light pattern consistent with the speckle information.

3. The multi-input joint intelligent single-pixel imaging method according to claim 1, characterized in that, The light intensity value is the total light intensity of the object to be imaged, which is collected and recorded by a single-pixel bucket collector after modulation of the light pattern.

4. The multi-input joint intelligent single-pixel imaging method according to claim 1, characterized in that, The intensity values ​​corresponding to the generated image are obtained according to the following steps: Let a one-dimensional light intensity sequence be used as the input to a convolutional neural network, and the generated image be denoted as... ; Let a random image be used as input to a convolutional neural network, and let the generated image be denoted as . ; The speckle pattern is used as the convolution kernel pair. and Perform convolution operations with a stride of 1 to obtain the corresponding intensity values.

5. The multi-input joint intelligent single-pixel imaging method according to claim 4, characterized in that, The loss function is calculated using the following formula: In the formula, The weights gradually increase with the number of iterations, to reach the initial weights at the start of training. , and intensity value The difference is the primary reference, and in the later stages... and The difference is the primary reference; , These represent the generated images. and The corresponding intensity values ​​are obtained by performing inner product calculations on the speckle pattern. This represents the light intensity value of the beam passing through the object; This indicates calculating the average. Indicates the generated image The intensity value is obtained by performing inner product calculation on the speckle pattern. and light intensity value The mean square error; Indicates the generated image The intensity value is obtained by performing inner product calculation on the speckle pattern. and light intensity value The mean square error.

6. The multi-input joint intelligent single-pixel imaging method according to claim 5, characterized in that, When training a convolutional neural network, the parameters are adjusted by changing the weights in the loss function each time until convergence, at which point the final image used for optimization is output. or As the final image.

7. The multi-input joint intelligent single-pixel imaging method according to claim 1, characterized in that, When constructing a convolutional neural network based on the U-Net framework, the U-Net network includes 5 encoding layers and 5 decoding layers. The encoding layers consist of double convolution and pooling layers, and the decoding layers consist of transposed convolution and double convolution. The decoding layers are also connected with the features of the corresponding encoding layers to fuse semantic information from different levels.

8. The multi-input joint intelligent single-pixel imaging method according to claim 1, characterized in that, The one-dimensional light intensity sequence, consisting of the light intensity values ​​of the beam passing through the object corresponding to the speckle pattern, is obtained by receiving the data through a single-pixel detector.

9. The multi-input joint intelligent single-pixel imaging method according to claim 1, characterized in that, A random image is a two-dimensional image where each pixel is set to a random number between 0 and 1, generated by the rand function.

10. An apparatus for the multi-input joint intelligent single-pixel imaging method according to claim 1, characterized in that, include: The data acquisition module is used to acquire the speckle pattern and the light intensity value of the corresponding beam passing through the object, forming a one-dimensional light intensity sequence; Get random images; The data processing module is used to construct a convolutional neural network based on the U-Net framework; it takes a one-dimensional light intensity sequence and a random image as inputs to the convolutional neural network to generate their respective corresponding images; it performs inner product operations with speckle on the two generated images to obtain the corresponding intensity values; and it designs a loss function based on the difference between the corresponding intensity value of the generated image and the light intensity value, as well as the difference in pixels between the two generated images. The image extraction module is used to train the convolutional neural network based on the designed loss function until it converges and outputs the final image.

Images