Polarization scattering imaging method for hierarchical polarization characteristic modulation

Abstract

The invention discloses a polarization scattering imaging method based on hierarchical polarization characteristic modulation. The method comprises the following steps: 1, establishing a neural network model which is based on singular value decomposition and can realize step-by-step regulation and control of polarization characteristics; 2, designing a dynamic loss function containing polarization information advantages; 3, adopting a local and overall common modulation training strategy; 4, shooting an s underwater scene data set for testing the feasibility and superiority of the framework; and 5, carrying out scattering removal processing on different scene pictures by utilizing the trained model. The method is applied to step-by-step local regulation and control of the polarization characteristics, flexible and sufficient analysis and extraction of the polarization characteristics are realized, the problem that a polarization scattering imaging algorithm based on deep learning cannot have better generalization in a continuously changing environment is solved, the degradation influence of the change of the scattering environment on the imaging effect is improved, and the imaging quality is improved. And the potential of promoting the development of the de-scattering research based on a dynamic and effective polarization characteristic analysis means is shown.

Description

Technical Field

[0001] The present invention belongs to the field of imaging, and in particular relates to a polarization scattering imaging method with hierarchical polarization feature modulation. Background Art

[0002] Polarization-scattering medium imaging based on deep learning has achieved many excellent results. Based on the multidimensional characteristics of polarization information, most deep learning-based polarization-scattering medium imaging methods directly increase the dimension and quantity of input data by directly inputting multidimensional polarization images to enhance the performance of the network. However, this will inevitably lead to information redundancy and insufficient utilization of polarization information. Therefore, in order to fully tap the advantages of polarization information, some studies have adopted effective data processing methods or designed special network architectures to enhance the stability and generalization ability of the model. Not only that, the analysis of polarization features is flexibly adjusted by dynamic fusion parameters and modulation of the loss function. However, these are all adjustment schemes that drive feature learning within the network model from a holistic level, and there is still a lack of relevant solutions for adjusting the interaction of features in the network model from a local perspective. Summary of the Invention

[0003] The present invention aims to address the shortcomings of the above-mentioned existing technologies and proposes a polarization scattering imaging method with hierarchical polarization feature modulation, in order to achieve flexible and comprehensive analysis and extraction of polarization features, thereby improving the degradation effect of changes in the scattering environment on the imaging effect, thereby improving the imaging quality and being able to flexibly adapt to scattering imaging in a variety of changing environments.

[0004] In order to achieve the above-mentioned object, the present invention adopts the following technical solutions:

[0005] The polarization scattering imaging method of the present invention with hierarchical polarization feature modulation is characterized in that it is performed according to the following steps:

[0006] Step 1: Get the polarization direction in the nth scene They are A set of polarization images ;in, Indicates that the polarization direction in the nth scene is The polarization image of ; N represents the total number of scenes;

[0007] Step 2: Get the clear image corresponding to the nth scene , ;

[0008] Step 3: A set of polarization images for the nth scene Calculate and get the linear polarization degree picture of the nth scene ;

[0009] Step 4: According to the threshold T, the clear image of the nth scene Perform singular value decomposition to obtain the singular value sub-atlas of the nth scene ,in, represents the t-th singular value subgraph of the n-th scene, The total number of singular value subgraphs representing the nth scene;

[0010] Step 5: Construct a hierarchical polarization feature modulation polarization scattering imaging network, including: a step-by-step adjustment module with P stages in parallel, a polarization analysis module, a fine-tuning module based on the VGG-16 network, and Processing is performed to obtain the global control characteristics of the nth scene ;

[0011] Step 6: Construct a multi-task loss with adaptive weights, including: level-by-level loss, polarization loss, and contrast loss;

[0012] Step 6: Use the gradient descent method to train the polarization scattering imaging network and minimize the multi-task loss in all scenarios to optimize the network parameters until the multi-task loss function converges, thereby obtaining the optimal polarization scattering imaging model for achieving descattering of the scattered image.

[0013] The polarization scattering imaging method with hierarchical polarization feature modulation according to the present invention is also characterized in that step 5 is performed as follows:

[0014] Step 5.1, The polarization characteristics of each stage are obtained by inputting them into the step-by-step adjustment module of each stage for processing. ,in, Indicates that the polarization direction in the nth scene is Polarization characteristics of the pth stage;

[0015] Will After element-by-element summation, the step-by-step polarization control characteristics in the nth scenario are obtained. ;

[0016] Step 5.2, Input to the polarization analysis module for processing, and output the polarization analysis characteristics under the nth scene ;

[0017] Step 5.3, The input is processed into the fine-tuning module based on the VGG-16 network, and the global control features of the nth scene are output. .

[0018] Furthermore, step 6 is performed as follows:

[0019] Step 6.1: Polarization characteristics of the pth stage in the nth scene As the feature map X, take the t-th singular value subgraph of the n-th scene As another feature map Y, the structural similarity loss between X and Y is calculated using formula (1) :

[0020] (1)

[0021] In formula (1), represents the mean of X, represents the mean of Y; represents the variance of X, represents the variance of Y, Represents the covariance of X and Y; C1 and C2 are two constants;

[0022] Step 6.2: Use formula (2) to construct the level-by-level loss for the nth scenario :

[0023] (2)

[0024] In formula (2), express The corresponding weight;

[0025] Step 6.3: Analyze the characteristics based on the polarization degree in the nth scene As the feature map X, take the linear polarization degree picture of the nth scene As another feature map Y, the polarization loss in the nth scenario is constructed using formula (1) ;

[0026] Step 6.4: Global control features of the nth scene As the third feature map F, the clear image corresponding to the nth scene As the fourth feature map G, use formula (4) to construct the contrast loss in the nth scene :

[0027] (4)

[0028] In formula (4), O represents the image with improved F brightness, and U represents the image with reduced F brightness; Represents the first layer, Indicates the total number of layers;

[0029] Step 6.5: Use formula (5) to construct the multi-task loss for the nth scenario :

[0030] (5)

[0031] In formula (5), , Represents 2 weights.

[0032] The electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the polarization scattering imaging method, and the processor is configured to execute the program stored in the memory.

[0033] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program executes the steps of the polarization scattering imaging method when executed by a processor.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. The present invention realizes step-by-step control of polarization information by utilizing singular value decomposition of the image, thereby enhancing the effective extraction of polarization features in the proposed framework and improving the descattering capability for complex scattering environments.

[0036] 2. To ensure the adequacy of target information extraction, the present invention uses contrastive learning as an overall adjustment method and designs a polarization loss function that incorporates polarization information, thereby improving the stability and generalization of the training process in a changing scattering medium environment.

[0037] 3. To ensure the proper assignment of multi-task loss function weights and the appropriateness of the weights used for each task, this paper employs a dynamic weight averaging method. This method adjusts task weights over time based on the rate of change of each task's loss. This prevents simpler scattering environments from dominating the training process, achieving the goal of assigning different training weights to different scattering environments.

[0038] 4. The present invention can effectively control the effects of polarization characteristics on various parts of the imaging process, thereby helping the network model to fully analyze and utilize the advantages of polarization information in the imaging process, such as stability and sensitivity, and improving the imaging quality and stability of this method in different scattering scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a network structure diagram of the present invention;

[0040] Figure 2A comparison of the clear images obtained by the present invention and those obtained by other methods;

[0041] Figure 3 A comparison of the clear images obtained by the present invention and those obtained by different ablation experiments;

[0042] Figure 4 This is a clear image obtained by the present invention in an underwater environment with varying turbidity;

[0043] Figure 5 These are clear images obtained by the present invention at different imaging distances. DETAILED DESCRIPTION

[0044] In this embodiment, a hierarchical polarization feature modulation polarization scattering imaging method utilizes singular value decomposition of the image to perform step-by-step local control of polarization features, enabling flexible and comprehensive analysis and extraction of polarization features. This method addresses the problem of deep learning-based polarization scattering imaging algorithms being unable to generalize well in changing environments, improves the degradation effect of changes in the scattering environment on imaging results, and demonstrates the potential of dynamic and effective polarization feature analysis to advance descattering research. Specifically, the method is performed in the following steps:

[0045] Step 1: Use a commercial focal plane polarization camera to obtain the polarization direction in the nth scene They are A set of polarization images ;in, Indicates that the polarization direction in the nth scene is The polarization image of ; N represents the total number of scenes.

[0046] Step 2: Get the clear image corresponding to the nth scene , ;

[0047] Step 3: A set of polarization images for the nth scene Calculate and get the linear polarization degree picture of the nth scene .

[0048] Step 4: According to the threshold T, the clear image of the nth scene Perform singular value decomposition to obtain the singular value sub-atlas of the nth scene ,in, represents the t-th singular value subgraph of the n-th scene, The total number of singular value subgraphs representing the nth scene.

[0049] Step 5: Construct a hierarchical polarization feature modulation polarization scattering imaging network, including: a step-by-step adjustment module with three parallel stages, a polarization analysis module, and a fine-tuning module based on the VGG-16 network. The network structure is as follows: Figure 1 As shown, SSAM stands for step-by-step adjustment module, PLM stands for polarization analysis module, and FTM stands for fine-tuning module based on VGG-16 network.

[0050] Step 5.1, The three stages of polarization characteristics are input into the step-by-step adjustment module for processing, and the polarization characteristics of the three stages are obtained accordingly. ,in, Indicates that the polarization direction in the nth scene is The polarization characteristics of the pth stage are obtained; each stage of the step-by-step adjustment module in the present invention is composed of a U-Net network. In specific implementations, the number of stage modules can be changed according to different situations. The verification experiment of the present invention uses three stage modules.

[0051] Will After element-by-element summation, the step-by-step polarization control characteristics in the nth scenario are obtained. ;

[0052] Step 5.2, Input to the polarization analysis module for processing, and output the polarization analysis characteristics under the nth scene ;

[0053] Step 5.3, The input is processed into the fine-tuning module based on the VGG-16 network, and the global control features of the nth scene are output. .

[0054] Step 6: Construct a multi-task loss with adaptive weights, including: level-by-level loss, polarization loss, and contrast loss:

[0055] Step 6.1: Polarization characteristics of the pth stage in the nth scene As the feature map X, take the t-th singular value subgraph of the n-th scene As another feature map Y, the structural similarity loss between X and Y is calculated using formula (1) :

[0056] (1)

[0057] In formula (1), represents the mean of X, represents the mean of Y; represents the variance of X, represents the variance of Y, Represents the covariance of X and Y; C1 and C2 are two constants.

[0058] Step 6.2: Use formula (2) to construct the level-by-level loss for the nth scenario :

[0059] (2)

[0060] In formula (2), express The corresponding weight;

[0061] Step 6.3: Analyze the characteristics based on the polarization degree in the nth scene As the feature map X, take the linear polarization degree picture of the nth scene As another feature map Y, the polarization loss in the nth scenario is constructed using formula (1) .

[0062] Step 6.4: Global control features of the nth scene As the third feature map F, the clear image corresponding to the nth scene As the fourth feature map G, use formula (4) to construct the contrast loss in the nth scene :

[0063] (4)

[0064] In formula (4), O represents the image with improved F brightness, and U represents the image with reduced F brightness; Represents the first layer, Indicates the total number of layers; in this embodiment, = {1, 3, 5, 9, 13}.

[0065] Step 6.5: Use formula (5) to construct the multi-task loss for the nth scenario :

[0066] (5)

[0067] In formula (5), , Represents 2 weights.

[0068] Step 6: Use the gradient descent method to train the polarization scattering imaging network and minimize the multi-task loss in all scenarios to optimize the network parameters until the multi-task loss function converges, and obtain the optimal polarization scattering imaging model to achieve the descattering effect on the scattered image. In this embodiment, the loss function weight is obtained by the dynamic average weight calculation method, that is, Figure 1 DWA in.

[0069] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0070] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are executed.

[0071] The method of the present invention is compared with other polarization scattering imaging methods based on deep learning. The comparison results are as follows: Figure 2 As shown in the figure, PDN refers to the Polarization Dense Network, PINet refers to the Polarization Fusion Network, SAMNet refers to the Self-Attention Multi-Scale Network, TIUNet refers to the Transformer and UNet combined network, PID2Net refers to the Polarization De-scattering and De-noising Network, and Ours refers to the method of the present invention. The comparison results show that the images obtained by the present invention have satisfactory imaging effects and detailed restoration for both planar and stereo objects.

[0072] The method of the present invention is compared with the results of different ablation experiments. Figure 3 As shown in the figure, GroundTruth refers to the original image, Wo-DWA refers to the network output without the dynamic average weight calculation, Wo-PLM refers to the network output without the polarization analysis module, Wo-ContraLoss refers to the network output without the fine-tuning module based on the VGG-16 network, W-RLoss refers to the network output using the proportional distribution of loss function weights, and Ours refers to the network output of the present invention. From the comparison results, it can be seen that the modules designed by the present invention have a positive effect on improving the effectiveness and superiority of the framework.

[0073] The scattering images obtained in water with different turbidity were input into the present invention for testing. The test results are as follows: Figure 4 As shown in the figure, the turbidity levels used are shown in the first column: 12.9 NTU, 29.4 NTU, 46.4 NTU, and 62.5 NTU; the peak signal-to-noise ratio (PSNR) calculated for the corresponding images is shown below the images. The test results show that the present invention achieves good imaging results and clarity in environments with increasing turbidity, demonstrating its ability to flexibly adapt to changing scattering environments.

[0074] The scattering images obtained at different scattering distances were input into the present invention for testing. The test results are as follows: Figure 5As shown, the scattering distances used are shown in the first row: 8cm, 13cm, 18cm, 20cm, 24cm, and 30cm; the peak signal-to-noise ratio (PSNR) calculated for the corresponding images is shown below. The test results show that the present invention achieves good imaging results and clarity at scalable imaging distances, demonstrating its strong generalization and robustness.

Claims

1. A polarization scattering imaging method with hierarchical polarization feature modulation, characterized in that: The steps are as follows: Step 1: Get the polarization direction in the nth scene They are A set of polarization images ;in, Indicates that the polarization direction in the nth scene is The polarization image of ; N represents the total number of scenes; Step 2: Get the clear image corresponding to the nth scene , ; Step 3: A set of polarization images for the nth scene Calculate and get the linear polarization degree picture of the nth scene ; Step 4: According to the threshold T, the clear image of the nth scene Perform singular value decomposition to obtain the singular value sub-atlas of the nth scene ,in, represents the t-th singular value subgraph of the n-th scene, The total number of singular value subgraphs representing the nth scene; Step 5: Construct a hierarchical polarization feature modulation polarization scattering imaging network, including: a step-by-step adjustment module with P stages in parallel, a polarization analysis module, a fine-tuning module based on the VGG-16 network, and Processing is performed to obtain the global control characteristics of the nth scene ; Step 6: Construct a multi-task loss with adaptive weights, including: level-by-level loss, polarization loss, and contrast loss; Step 7: Use the gradient descent method to train the polarization scattering imaging network and minimize the multi-task loss in all scenarios to optimize the network parameters until the multi-task loss function converges, thereby obtaining the optimal polarization scattering imaging model for achieving descattering of the scattered image.

2. The polarization scattering imaging method with hierarchical polarization feature modulation according to claim 1, characterized in that: Described step 5 is carried out as follows: Step 5.1, The polarization characteristics of each stage are obtained by inputting them into the step-by-step adjustment module of each stage for processing. ,in, Indicates that the polarization direction in the nth scene is Polarization characteristics of the pth stage; Will After element-by-element summation, the step-by-step polarization control characteristics in the nth scenario are obtained. ; Step 5.2, Input to the polarization analysis module for processing, and output the polarization analysis characteristics under the nth scene ; Step 5.3, The input is processed into the fine-tuning module based on the VGG-16 network, and the global control features of the nth scene are output. .

3. The polarization scattering imaging method with hierarchical polarization feature modulation according to claim 2, characterized in that: Described step 6 is carried out as follows: Step 6.1: Polarization characteristics of the pth stage in the nth scene As the feature map X, take the t-th singular value subgraph of the n-th scene As another feature map Y, the structural similarity loss between X and Y is calculated using formula (1) : (1) In formula (1), represents the mean of X, represents the mean of Y; represents the variance of X, represents the variance of Y, Represents the covariance of X and Y; C1 and C2 are two constants; Step 6.2: Use formula (2) to construct the level-by-level loss for the nth scenario : (2) In formula (2), express The corresponding weight; Step 6.3: Analyze the characteristics based on the polarization degree in the nth scene As the feature map X, take the linear polarization degree picture of the nth scene As another feature map Y, the polarization loss in the nth scenario is constructed using formula (1) ; Step 6.4: Global control features of the nth scene As the third feature map F, the clear image corresponding to the nth scene As the fourth feature map G, use formula (4) to construct the contrast loss in the nth scene : (4) In formula (4), O represents the image with improved F brightness, and U represents the image with reduced F brightness; Represents the first layer, Indicates the total number of layers; Step 6.5: Use formula (5) to construct the multi-task loss for the nth scenario : (5) In formula (5), , Represents 2 weights.

4. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the polarization scattering imaging method according to any one of claims 1 to 3, and the processor is configured to execute the program stored in the memory.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the polarization scattering imaging method according to any one of claims 1 to 3 are executed.

Images