Polarization differential imaging method based on twin neural network

Abstract

The application discloses a polarization differential imaging method based on a twin neural network and belongs to the field of computational imaging. The imaging quality of a traditional active polarization differential imaging method is limited due to the influence of background scattered light. The application introduces deep learning on the basis of a traditional active polarization differential imaging model and mainly optimizes a polarization imaging system through a convolutional neural network. The convolutional neural network discriminates the similarity of orthogonal polarized light images, extracts and fuses similar feature information through a core fusion algorithm, strengthens the features of target information obtained by a photoelectric sensor, and has some advantages in the accurate selection of orthogonal polarized light in the polarization imaging system. Compared with the traditional polarization differential imaging method, the method has higher imaging quality.

Description

Technical Field

[0001] This invention relates to a polarization difference imaging method based on a twin neural network, belonging to the field of computational imaging, and particularly to the field of underwater polarization imaging. Background Technology

[0002] Polarization imaging is a relatively direct method for imaging by acquiring polarization information. Compared to traditional imaging methods, it can obtain multi-dimensional information, extending visual information from two dimensions to higher dimensions, providing a new approach to acquiring target and environmental information. In the field of computational imaging, polarization imaging technology has attracted much attention due to its imaging mechanism utilizing the polarization characteristics of light. With the development of underwater polarization imaging technology, through the analysis of underwater polarization imaging algorithms, the introduction of deep learning methods in the image restoration process can better solve the problems of image detail loss and unclear imaging. However, there are two drawbacks in the process of obtaining target images: first, the transmission distance of polarized light is limited due to the light absorption problem of water particles; second, the image clarity is limited because of the influence of scattering particles, backscattered light obscures the target object, and forward scattered light loses target detail information.

[0003] To address the aforementioned problems with polarization imaging methods, this invention proposes a polarization differential imaging method based on Siamese neural networks. Summary of the Invention

[0004] A polarization differential imaging method based on Siamese neural networks is proposed, applicable to traditional active polarization differential imaging systems. Traditional active polarization differential imaging systems consist of three parts: a light source, an active polarization imaging physical model, and a computer. This invention uses a linearly polarized Gaussian beam as the active light source to illuminate a target object in a turbid medium that is obscured by scattering particles for imaging.

[0005] This method includes the following five steps:

[0006] S1: Using a linearly polarized Gaussian beam as the light source, an underwater active polarization imaging experimental system was built. For clear water and turbid solutions containing 5ml, 7ml and 10ml of milk, polarization intensity image datasets of the target and background at 45°, 0°±10° and 90°±10° were obtained by rotating the polarizer.

[0007] S2: Train a network model for orthogonal polarization intensity map feature learning and classification based on a Siamese neural network. Define the function `getSiameseBatch` to return a randomly selected batch of paired images. The function `getSiameseBatch` creates random pairs of similar and dissimilar images with orthogonal polarization intensity maps. The model gradient is evaluated using a model gradient function, and the loss function describes the matching degree, i.e., the similarity, between sample pairs. The loss function is expressed as follows: In the formula: y is the numerical value of the paired label, and d is the Euclidean distance between the two feature vectors.

[0008] S3: Test and use orthogonal polarization intensity map features to learn and classify the network model. The function getSiameseBatch creates random pairs of similar or dissimilar images.

[0009] S4: The two similar orthogonal polarization intensity maps obtained are fused using the core fusion strategy. The input image is decomposed into r fused depth detail images and one base image. The depth detail images are fused using a kernel norm fusion strategy, and the base image is fused using a weighted average fusion strategy.

[0010] Each detailed image is fused using a nuclear norm fusion strategy, and the representation is as follows: In the formula: R(·) is the operator for reconstructing the image of depth details. is the vector of the fused detail parts, and i is the decomposition level.

[0011] The deep detail components are fused using nuclear norm, while the basic components are fused using data averaging. Finally, the basic and deep detail components are combined to obtain the fused result.

[0012] S5: Obtain the target image using polarization inversion imaging.

[0013] Substituting the fused horizontally polarized light intensity and vertically polarized light intensity into the inversion imaging, the separated target light intensity information S' is represented as follows: In the formula: I P For horizontally polarized light intensity, I ⊥ For vertically polarized light intensity, p obj p represents the polarization degree of the target information. scat This represents the degree of polarization of the backscattered light. Attached Figure Description

[0014] Figure 1 The device diagram for the specific implementation is as follows: 1. Collimated 532nm wavelength laser source; 2. Polarizer 1; 3. Target in turbid medium; 4. Polarizer 2; 5. Image sensor.

[0015] Figure 2 Flowchart of polarization imaging based on twin neural network.

[0016] Figure 3(a) Siamese neural network framework; (b) similar and dissimilar pairs; (c) similarity graph of orthogonally polarized light.

[0017] Figure 4 Intensity diagram of orthogonally polarized light after fusion.

[0018] Figure 5 (a) Target image before processing; (b) Target image after processing. Detailed Implementation

[0019] To better explain the implementation process of the present invention, an embodiment will be used to further describe the present invention in detail below, but the present invention is not limited to this embodiment.

[0020] Example 1.

[0021] Figure 1 The apparatus diagram for a specific embodiment includes a 532nm laser (1), a polarizer 1 (2), a target in a turbid medium (3), a polarizer 2 (4), and an image sensor (5).

[0022] Combination Figure 1 The apparatus shown further illustrates the present invention.

[0023] S1: The experimental setup for underwater active polarization imaging with a linearly polarized Gaussian beam was constructed using a 100mW, 532nm wavelength MSI laser. The numerical aperture (NA) of the microscope objective was 0.1, and its dimensions were 35×20×22cm. 3 A transparent glass tank was filled with water, and the solution was made turbid by mixing clear water with milk. Since higher particle concentrations result in more scattering, the energy of the imaging beam is more dispersed, leading to more pronounced scattering blur. Therefore, datasets of polarization intensity maps of the target and background at 45°, 0°±10°, and 90°±10° were acquired using an image sensor by rotating a polarizer, for both clear water and turbid solutions containing 5ml, 7ml, and 10ml of milk.

[0024] S2: Figure 2 This is a flowchart of polarization imaging based on a Siamese neural network. Experiments were conducted with clear water and turbid solutions containing 5ml, 7ml, and 9ml of milk. The target was placed at water depths of 5cm, 7cm, and 10cm, respectively. Twelve sets of target polarization images and twelve sets of background polarization images were obtained for four concentrations and three distances. Each set of images contains 84 orthogonal polarization images and two 45° polarization intensity images, with an image size of 1024×1280×1. A network model for orthogonal polarization intensity image feature learning and classification based on a Siamese neural network was trained. Figure 3(a) shows the Siamese neural network framework. Training data consists of orthogonal polarization maps, with labels 1 and 2 embedded. The number of images corresponds to the number of channels. Random pairs of similar and dissimilar images are created using the function `getSiameseBatch`. Similarity is compared using a function corresponding to the labels, and the images are then labeled. Similar images are defined to have the same label, while dissimilar images have different labels. Figure 3 (b) Similar and dissimilar pairs of orthogonally polarized light obtained using the orthogonal polarization intensity map feature learning and classification network model. Create a function model gradient that returns the loss value and the gradient of the loss relative to the network's learnable parameters. The goal of the Siamese neural network is to output a feature vector for each image such that the feature vectors of similar images are similar, while the feature vectors of different images are significantly different. This allows the network to distinguish between two inputs. Find the output of the last fully connected layer, i.e., the feature vectors G from paired image 1 and paired image 2, respectively. W (X1) and feature G W Contrast loss between (X2).

[0025] The loss function is represented as follows: In the formula: y is the value of the paired label, and d is the Euclidean distance between the two feature vectors. d(X1,X2)=||G W (X1)-G W (X2)||2 The distance parameter d is used as a constraint: if a pair of images are dissimilar, then their distance should be at least the margin; otherwise, it will result in a loss. In the case of similar images, the first term can be non-zero, and this is achieved by reducing the image feature G. W (X1) and G W The distance between (X2) is minimized. In the case of dissimilar images, the second term can be non-zero and is minimized by increasing the distance between image features, at least to the edge. Figure 3 (c) are the two most similar images of orthogonally polarized light.

[0026] S3: Test and use an orthogonal polarization intensity map feature learning and classification network model. Input the orthogonal polarization intensity map obtained in the experiment. After converting the layered map into a dlnetwork object, train the network using a custom training loop with automatic discrimination enabled. Evaluate the model gradient using dlfeval and the model gradient function. Use the adaptive moment estimation (Adam) algorithm and update the network parameters using the adamupdate function. Train using a custom loop with 3000 iterations, iterating through the data and updating the network parameters. Based on the simplified feature map of the test data, finally obtain the two most similar orthogonal polarization intensity maps with the highest similarity.

[0027] S4: The two orthogonally polarized light similarity images with the highest similarity obtained in S3 are fused using a fusion algorithm. The input image is decomposed into r fused depth detail images and one base image. The depth detail images are fused using a kernel norm fusion strategy, and the base image is fused using a weighted average fusion strategy. The fused detail images are then processed. for: In the formula: R(·) is the operator for reconstructing the image of the depth details, and FS is the depth details fusion strategy;

[0028] Detail vectors The fused detail vectors are obtained through weighted summation. In the formula: the weight of each corresponding column is i represents the decomposition level; K represents the number of input images; ||·|| represents the nuclear norm; and re(·) is the operator for reconstructing image patches.

[0029] Each detailed image is fused using a nuclear norm fusion strategy: Basic Part Image I bf The formula for calculating (x,y) is: In the formula: ω b1 and ω b2 The basic weighting part. By combining the base and depth detail components, we can obtain the fused image I. f (x,y): The deep detail components are fused using nuclear norm, while the basic components are fused using data averaging. Finally, the basic and deep detail components are combined to obtain the fused result. Figure 4 This is the result of the fused orthogonal polarization intensity map.

[0030] S5: Obtain the target image using polarization inversion imaging. The intensity of the fused horizontally polarized light I P and vertically polarized light intensity I ⊥ Substituting the values ​​into the inversion imaging, the representation is as follows: I P =B P +S P I ⊥ =B ⊥ +S ⊥ In the formula: B is the background light and S is the target light. The polarization degree p of the target information obj The degree of polarization p of the backscattered light scat The representation of is as follows: By substituting the above formulas, we can obtain the following relationship: I ⊥ -I P =p scat B+p obj S

[0031] Because backscattering involves a large amount of polarization, the degree of polarization in backscattering can also be determined by... Indicates. A ∞ This represents the backscattered light intensity information of the target object after traveling an infinite distance from the target area in the water. Finally, by substituting the above formula, given the polarization degree of the target light and the polarization degree of the backscattered light, the target light intensity information S' can be separated from the two original images:

[0032] Figure 5 (a) is the target image without image processing. Figure 5 (b) is the target image processed by the Siamese neural network and fusion algorithm. The simulation results of this invention show higher image clarity and better imaging quality.

[0033] The embodiments described in the specific implementation details represent only one implementation of the present invention. Embodiment 1 is an implementation device and process designed according to the method proposed in this invention, and does not represent the scope of protection of this invention. All those skilled in the art can design various different embodiments according to the method proposed in this invention. All embodiments conforming to the claims of this invention are within the scope of protection of this patent.

Claims

1. A polarization difference imaging method based on Siamese neural network, characterized in that, This method includes the following five steps: S1: Using a linearly polarized Gaussian beam as the light source, an underwater active polarization imaging experimental system was built. This system was used to image the target and background at 45° angles by rotating a polarizer, targeting both clear water and turbid solutions containing 5ml, 7ml, and 10ml of milk. and Polarized light intensity image dataset; S2: Train an orthogonal polarization intensity map feature learning and classification network model based on a Siamese neural network; define the function `getSiameseBatch` to return a randomly selected batch of paired images, and create random pairs of similar and dissimilar images of orthogonal polarization intensity maps in the polarization intensity image dataset from step S1 using the `getSiameseBatch` function; evaluate the model gradient using the model gradient function; describe the matching degree, i.e., the similarity, between sample pairs using a loss function; the loss function is expressed as follows: In the formula: The numerical value of the paired tag. The Euclidean distance between two eigenvectors; S3: Test and use the orthogonal polarization intensity map feature learning and classification network model based on Siamese neural network in step S2, input the polarization intensity images of the target and background at 0°±10° and 90°±10° obtained by the underwater active polarization imaging experimental system in step S1, respectively, in clear water and turbid solutions injected with 5ml, 7ml and 10ml of milk, and perform similarity discrimination. S4: Using the core fusion strategy, fuse the two most similar images of all the orthogonal polarization intensity images obtained by similarity discrimination in step S3 respectively; S5: Substitute the fused orthogonally polarized light intensity obtained in step S4 into the polarization inversion imaging method to obtain the target image; the polarization inversion imaging method is: substituting the fused horizontally polarized light intensity... and vertically polarized light intensity Substitute the following formula into the inversion imaging to separate the target light intensity information. The representation is as follows: In the formula: As background light, For the target light, and , The degree of polarization of the target information. This represents the degree of polarization of the backscattered light.

2. The polarization difference imaging method based on Siamese neural network as described in claim 1, characterized in that, The Step S4 uses a core fusion strategy to fuse the depth details of two similar orthogonal polarization intensity maps. Perform multi-level decomposition on the input image to obtain The images are fused using a depth detail image and a base image. The depth detail images are fused using a kernel norm fusion strategy, and the base image is fused using a weighted average fusion strategy. The nuclear norm fusion strategy fusion algorithm is represented as follows: The deep detail components are fused using nuclear norm, while the basic components are fused using data averaging; finally, the basic and deep detail components are combined as the fused result.

Images