Intelligent optimization of speckle correlation imaging reconstruction and restoration method

Abstract

The present invention belongs to the technical field of stable detection in the on-site security technology system, and specifically relates to a method for intelligent optimization of speckle correlation imaging reconstruction and restoration. The method comprises the following steps: light emitted by a light source is projected onto a digital micromirror device (DMD), a series of speckle patterns are generated by the DMD, and projected onto a target, and the light intensity information projected or reflected by the target is obtained using a single-pixel detector. A convolution operation is used to extract information in the speckle that is beneficial to imaging, and the information is correlated with the light intensity collected by a single-pixel collector to obtain a target image. The difference between the light intensity of the speckle pattern and the light intensity collected by the single-pixel collector is used as a parameter of a constrained network to ensure that the network can better extract information in the speckle that is beneficial to imaging.

Description

Technical Field

[0001] The present invention belongs to the technical field of stable detection in a local security technology system, and specifically relates to a method for intelligently optimizing the correlation imaging reconstruction and restoration of speckles. Background Art

[0002] Optical imaging technology is an important means for humans to observe and perceive the objective world. Imaging systems and optical observation technologies are used to collect basic characteristic quantities in the process of free propagation of light beams and their interaction with media, so as to obtain information about the objective scene world that can be processed. Correlation imaging can obtain images in the presence of interference such as clouds, fog and smoke. It not only has new capabilities that surpass traditional imaging methods, but also provides a new solution to the problem of high-resolution imaging of dynamic scenes. Correlation imaging is a classic indirect light field correlation imaging technology. It obtains image information based on the second-order or high-order correlation operation of random light fields and light intensity detection sensors to detect light intensity. It is conducive to maintaining the fluctuation trend of the collected light intensity information and has unique advantages in achieving high-resolution imaging of large-scale dynamic scenes such as scattering medium imaging and long-distance detection. The stronger the algorithm capability, the higher the efficiency of target image reconstruction. With the development of deep learning technology, convolutional neural networks have been widely used in the field of image processing, especially in the understanding, restoration, and enhancement of images. Therefore, they have been successfully applied in the field of image processing.

[0003] Traditional correlation imaging methods lack the condensation of useful speckle information, and the second-order correlation operation of exploring all speckle information and light intensity introduces unnecessary noise into the generated image, affecting the quality of the generated image. The correlation imaging method combined with deep learning requires a large number of training samples to ensure the efficiency of the network, which greatly increases the cost of data collection. Summary of the invention

[0004] In order to overcome the problems of poor imaging quality of existing correlation imaging methods and difficulty in collecting training samples for neural networks, the present invention describes a correlation imaging reconstruction and restoration method for intelligent optimization of speckle. This method fully combines the advantages of traditional correlation imaging methods and deep learning, and uses neural networks to extract speckle information that is beneficial to imaging. The extracted speckle information and light intensity are used for correlation imaging calculations. With the help of neural networks, the characteristics of information can be efficiently explored, and the useful information in the speckle can be fully mined, so that the imaging quality of correlation imaging can be improved. At the same time, this method uses the difference between the intensity value of the generated image and the intensity value received by the detector as a reference standard to optimize the network parameters, without the need for additional training data sets, which greatly saves data acquisition costs.

[0005] The technical solution adopted by the present invention to solve the technical problem is: a method for intelligently optimizing the correlation imaging reconstruction and restoration of speckles, which is characterized by adopting the following steps:

[0006] Step 1: Import speckle information patterns.

[0007] Step 1.1, using a spatial light modulator to sequentially modulate the light source so as to generate a light pattern consistent with the speckle information patterns;

[0008] Step 1.2, using a bucket detector without spatial resolution to collect and record the total light intensity measurements of the modulated light pattern penetrating the object to be imaged;

[0009] Patterns is three-dimensional data, consisting of (n, w, h). n represents the number of speckle patterns, w and h represent the width and height of a single speckle, respectively. Measurements is one-dimensional data, consisting of n measured intensity values.

[0010] Step 2: Construct a neural network f to optimize speckle cnn (·).

[0011] Step 3: Use the constructed neural network f cnn (·) Optimize the speckle information patterns imported in step 1 to extract speckles that are more conducive to imaging. Speckle optimization function:

[0012] patterns=f cnn (patterns) (1)

[0013] Step 4: Use the speckle patterns optimized in step 3 and the intensity information measurements measured in step 1 to perform differential correlation imaging to form an image DGI. The differential correlation imaging formula is:

[0014] SI_aver=mean(patterns*measurements) (2)

[0015] B_aver=Mean(measurements) (3)

[0016] pattern=sum(sum(patterns)) (4)

[0017] R_aver=mean(patterns) (5)

[0018] RI_aver=mean(pattern*patterns) (6)

[0019] DGI=SI_aver-B_aver / R_aver*RI_aver (7)

[0020] Step 5, calculate the intensity value out_y of the image DGI obtained in step 4, design the loss function L according to the difference between the intensity value out_y of the generated image and the intensity value measurements detected in step 1, complete the parameter training process of the neural network, and output the DGI used in the last optimization as the final image, minimizing the objective function L:

[0021] out_y = DGI*patterns (8)

[0022] L=min(mean(out_y-measurements) 2 ) (9)

[0023] The beneficial effects of the present invention are as follows: the method uses deep learning to extract information from speckles that is helpful for imaging and ignores information from speckles that affects imaging, which can greatly improve imaging efficiency and enhance the quality of generated images. At the same time, the difference between the light intensity of the generated image and the light intensity received by the detector is used as a loss function, so that the generated image is infinitely close to the real image. Using such a loss function does not require the use of additional data sets as training sets, which reduces the cost of data acquisition.

[0024] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flow chart of a method for intelligently optimizing the associated imaging reconstruction and restoration of speckles of the present invention. DETAILED DESCRIPTION

[0026] Reference Figure 1 In this example, a method for intelligently optimizing the correlation imaging reconstruction and restoration of speckles is implemented, and the specific steps are as follows:

[0027] Step 1: Import speckle information patterns.

[0028] Step 1.1: Use a spatial light modulator to modulate the light source so as to generate a light pattern similar to the speckle information patterns;

[0029] Step 1.2 uses a bucket detector without spatial resolution to collect and record total light intensity measurements of the modulated light pattern passing through the object to be imaged;

[0030] Patterns is three-dimensional data, consisting of (w, h, n). n represents the number of speckle patterns, w and h represent the width and height of a single speckle, respectively. Measurements is one-dimensional data, consisting of n measured intensity values.

[0031] Step 2: Construct the speckle optimization neural network fcnn (·) and assigns it an initial weight value. cnn (·) A neural network consisting of 6 convolutional layers and 3 pooling layers. The first 3 convolutional layers increase the number of image channels to 16, 32, and 64, respectively, and the last 3 convolutional layers reduce the number of image channels to 32, 16, and 1. The first 3 convolutional layers are followed by a pooling layer to downsample the image, and the last 3 convolutional layers upsample the image while reducing the number of channels. The convolution kernels of all convolutional layers are set to 5*5.

[0032] Step 3: Use the constructed neural network f cnn (·) Optimize the speckle information patterns measured in step 1 to extract speckles that are more conducive to imaging. Speckle optimization function:

[0033] patterns=f cnn (patterns)

[0034] The specific process is:

[0035] Step 3.1, take the speckle information in step 1 as input, perform a convolution operation, the convolution kernel size is 5*5, the step length is 1, and the output feature layer is 16;

[0036] Step 3.2, pooling the output result of step 3.1 using the average pooling operation;

[0037] Step 3.3, perform a convolution operation on the output result in step 3.2, with a convolution kernel size of 5*5, a step size of 1, and an output feature layer of 32;

[0038] Step 3.4, pooling the output result in step 3.3 using an average pooling operation;

[0039] Step 3.5: Perform a convolution operation on the output result in step 3.4, with a convolution kernel size of 5*5, a step size of 1, and an output feature layer of 64;

[0040] Step 3.6, pooling the output result in step 3.5 using the average pooling operation;

[0041] Step 3.7, perform a convolution operation on the output result in step 3.6, with a convolution kernel size of 5*5, a step size of 1, an output feature layer of 32, and use a deconvolution operation to upsample the result;

[0042] Step 3.8, perform a convolution operation on the output result in step 3.7, with a convolution kernel size of 5*5, a step size of 1, an output feature layer of 16, and use a deconvolution operation to upsample the result;

[0043] Step 3.9, perform a convolution operation on the output result in step 3.8, with a convolution kernel size of 5*5, a step size of 1, an output feature layer of 1, and use a deconvolution operation to upsample the result.

[0044] Step 4: Use the speckle patterns optimized in step 3 and the intensity information measurements measured in step 1 to perform differential correlation imaging to form an image DGI.

[0045] Multiply all speckle information patterns and their corresponding intensity values ​​measurements and calculate their average value SI_aver

[0046] SI_aver=mean(patterns*measurements)

[0047] Wherein mean(·) represents the averaging function, which acts on n dimensions here and is used to find the average value of multiple speckle patterns. SI_aver is two-dimensional data consisting of width w and height h.

[0048] Calculate the average value B_aver of the total light intensity collected multiple times.

[0049] B_aver=Mean(measurements)

[0050] Where mean(·) represents the averaging function, which acts on n dimensions here to find the average value of multiple times. SI_aver is a two-dimensional data consisting of width w and height h.

[0051] Calculate the sum of all pixels in each speckle pattern patterns_s:

[0052] patterns_s=sum(sum(patterns))

[0053] Where sum(·) represents the sum function. Here, sum(·) is applied twice in the w dimension and the h dimension respectively to find the sum of each pixel in each speckle pattern. patterns_s is one-dimensional data, which records the sum of each pixel of n speckle patterns.

[0054] Calculate the average value R_aver of all speckle patterns' pixels and patterns_s:

[0055] R_aver=mean(patterns_s)

[0056] Wherein mean(·) represents the averaging function, which acts on n dimensions here and is used to find the average value R_aver of all speckle pattern images.

[0057] Calculate the sum of all pixel values ​​patterns_s of each speckle pattern and the product of the speckle pattern patterns, and find the overall average value RI_aver:

[0058] RI_aver=mean(patterns_s*patterns)

[0059] Using the SI_aver, B_ave, R_aver and RI_aver obtained above, differential correlation imaging is performed to obtain the image DGI:

[0060] DGI=SI_aver-B_aver / R_aver*RI_aver

[0061] Step 5, calculate the intensity value out_y of the image DGI obtained in step 4, design the loss function L according to the difference between the intensity value out_y of the generated image and the intensity value measurements detected in step 1, complete the parameter training process of the neural network, and output the DGI used in the last optimization as the final image, minimizing the objective function L:

[0062] out_y = DGI*patterns

[0063] L=min(mean(out_y-measurements) 2 )

[0064] The effect of the present invention is further illustrated by the following simulation experiment.

[0065] Simulation conditions: The present invention is simulated by using MATLAB software on an Intel(R) Core(TM) i7-6800K CPU@3.40GHz, 4G memory, and Ubuntu 14 operating system.

Claims

1. An intelligent optimization speckle correlation imaging reconstruction and restoration method, which is characterized by: The following steps are involved: Step 1, import speckle information patterns; Step 1.1, using a spatial light modulator to sequentially modulate the light source so as to generate a light pattern consistent with the speckle information patterns; Step 1.2, using a bucket detector without spatial resolution to collect and record the total light intensity measurements of the modulated light pattern penetrating the object to be imaged; Among them, patterns is three-dimensional data, composed of (n, w, h), n represents the number of speckle patterns, w and h represent the width and height of a single speckle, respectively; Measurements is one-dimensional data, composed of n measured intensity values; Step 2: Construct a neural network f to optimize speckle cnn (·); Step 3: Use the constructed neural network f cnn (·) Optimize the speckle information patterns imported in step 1 to extract speckles that are more conducive to imaging. The speckle optimization function is: patterns=f cnn (patterns) (1) Step 4: Use the speckle patterns optimized in step 3 and the intensity information measurements measured in step 1 to perform differential correlation imaging to form an image DGI; the differential correlation imaging formula is: SI_aver=mean(patterns*measurements) (2) B_aver=Mean(measurements) (3) pattern=sum(sum(patterns)) (4) R_aver=mean(patterns) (5) RI_aver=mean(pattern*patterns) (6) DGI=SI_aver-B_aver / R_aver*RI_aver (7) Step 5, calculate the intensity value out_y of the image DGI obtained in step 4, design the loss function L according to the difference between the intensity value out_y of the generated image and the intensity value measurements detected in step 1, complete the parameter training process of the neural network, and output the DGI used in the last optimization as the final image, minimizing the objective function L: out_y = DGI*patterns (8) L=min(mean(out_y-measurements) 2 ) (9)。

Images