An underwater polarized image and polarization information restoration method based on a double-flow diffusion model

Abstract

The application discloses a kind of underwater polarization image and polarization information recovery method based on double-flow diffusion model, belong to underwater image processing field, aim at solving the problem that underwater polarization imaging technology is difficult to consider image enhancement and polarization information recovery.This method includes:1) double-flow forward diffusion, same noise is added to underwater degraded polarization and true value image, and noise image sequence is generated;2) underwater degraded polarization image is reversedly denoised, and the last step noise image is output by constructing denoising network;3) scattering network bridge, and underwater degraded polarization image is enhanced by constructing scattering network;4) true value image is reversedly denoised, and recovered true value image is generated by fusing the output of denoising network and scattering network;5) iterative double-flow reversed denoising, and clear recovered polarization image is obtained by multiple iterations;6) composite loss function is designed to train network.The application can efficiently enhance image and accurately recover polarization information, significantly widen the practical application scene and range of underwater polarization imaging.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to an underwater polarization image and polarization information restoration method based on a dual-flow diffusion model. Background Technology

[0002] In underwater imaging scenarios, the scattering and absorption effects of water severely disrupt the propagation of light fields, making it difficult for traditional imaging techniques to obtain clear target information. Underwater polarization imaging technology, based on the differences in the polarization characteristics of light, effectively suppresses background scattering interference by capturing the inherent differences in polarization states (degree of polarization, polarization angle) between the target's reflected light and the water's scattered light. This provides a core technological path to overcome the bottleneck of underwater imaging. Furthermore, polarization information is correlated with the physical properties of objects, such as material and surface attributes. Based on the acquisition and interpretation of polarization information in underwater scenes, polarization imaging technology can achieve higher-quality underwater target recognition, material segmentation, autonomous driving, and other underwater tasks, serving as a crucial foundation for promoting the development of underwater detection technology towards higher precision and intelligence.

[0003] Currently, underwater polarization imaging technology is mainly divided into two categories: one is based on underwater imaging physical models, which establishes a mathematical propagation model of scattered light and target light, reverse-engineers and compensates for scattering interference to achieve image restoration; the other is based on deep learning, which utilizes the powerful feature learning capabilities of neural networks to learn the mapping relationship between scattering suppression and target enhancement from a large amount of underwater polarization image data, and quickly outputs high-quality imaging results. Both methods have achieved significant results in improving image intensity contrast and suppressing background noise, effectively improving the visual effect of underwater imaging. However, existing technologies generally have a key limitation: they only use polarization information as an auxiliary means for optimizing intensity images, ignoring the need to restore the polarization information itself. In fact, polarization information is the essential core of underwater polarization imaging technology. It contains unique information such as target material and surface condition, which cannot be replaced by traditional intensity images. If accurate restoration of polarization information can be achieved, it will provide deeper technical support for various underwater applications such as marine ranching, underwater facility maintenance, and marine engineering. Therefore, an effective underwater polarization imaging method is needed that can not only restore intensity images but also effectively recover polarization information. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and solve the problem that underwater polarization image methods can only restore light intensity information but cannot restore polarization information. This invention provides an underwater polarization image and polarization information restoration method based on a dual-flow diffusion model.

[0005] The objective of this invention is achieved through the following technical solution: A method for restoring underwater polarization images and polarization information based on a dual-flow diffusion model includes the following steps: Step 1. Dual-stream forward diffusion The underwater degraded polarization image and the corresponding ground truth image are respectively constructed based on Markov chains to construct forward diffusion. Noise is gradually added to the underwater degraded polarization image and the corresponding ground truth image. Each step is controlled by a timestamp parameter until the content of the underwater degraded polarization image and the corresponding ground truth image is covered by noise, generating a series of noisy underwater degraded polarization images and noisy ground truth images marked by timestamps. Step 2. Inverse denoising of underwater degraded polarization images A denoising network model is constructed. The underwater degraded polarization image with timestamp T obtained in step 1 is merged with the underwater degraded polarization image and input into the denoising network. The ground truth image with timestamp T-1 obtained in step 1 is input into the denoising network. The timestamp is embedded into the denoising network through sinusoidal encoding to obtain the underwater degraded polarization image with timestamp T-1 and the ground truth image with timestamp T-2. Step 3. Bridging the descattering network Construct a descattering network model, input the noisy underwater degraded polarization image with timestamp T-1 obtained in step 2 into the descattering network model, and obtain a descattered polarization image with timestamp T-1; Step 4. Inverse denoising of the ground truth image Add the noise ground truth image with timestamp T-1 obtained in step 1 to the descattered polarization image with timestamp T-1 obtained in step 3, then input it into the denoising network constructed in step 2, and embed the timestamp into the denoising network through sinusoidal encoding to obtain the noise ground truth image with timestamp T-2. Step 5. Iterative dual-stream reverse denoising and descattering Repeat steps 2, 3, and 4, decreasing the timestamp by 1 with each iteration. Stop the iteration when the timestamp is 1. This yields a noisy underwater degraded polarization image, a noise ground truth image, and a descattered polarization image with timestamp 1. Add the noise ground truth image and the descattered polarization image and then input them into the denoising network to obtain the noise ground truth image with timestamp 0, i.e., the restored polarization image. Simultaneously, the restored polarization information, including the degree of polarization and the polarization angle, can be calculated based on the polarization Stokes. Step 6. Design of loss function for polarization-constrained two-stream diffusion model Loss functions were designed for the denoising network and the descattering network respectively to constrain the dual-flow diffusion model to restore accurate image and polarization information.

[0006] Furthermore, the dual-flow forward diffusion described in step 1 is bridged by noise, which is controlled by a timestamp. The larger the value, the larger the noise variance. The underwater degradation polarization image of the noise with the same timestamp contains the same noise as the ground truth image of the noise.

[0007] Furthermore, the denoising network described in step 2 adopts the U-Net architecture, which includes a symmetrical encoder-decoder structure and skip connections between corresponding layers of the encoder and decoder, enabling it to simultaneously preserve low-level details and high-level semantic features; the timestamp t is embedded into the U-Net through sinusoidal position encoding, enabling the network to perceive noise features at different diffusion stages.

[0008] Furthermore, the descattering network described in step 3 consists of three parts: a polarization angle restoration network, a polarization degree restoration network, and a fusion network. The polarization degree restoration network and the polarization angle restoration network are connected through a polarization hybrid attention block, which guides the recovery of polarization information through polarization correlation. The polarization hybrid attention block optimizes polarization features through channel attention and spatial attention, and uses the polarization correlation between polarization degree and polarization angle to achieve feature calibration.

[0009] Furthermore, in the iterative dual-stream inverse denoising and descattering described in step 5, the dual-stream inverse denoising is bridged by the descattering network described in step 3. This network contains three sub-networks: a polarization degree complex atom network, a polarization angle complex atom network, and a fusion network. The polarization degree complex atom network and the polarization angle complex atom network adopt a U-shaped structure, including downsampling and upsampling layers to obtain polarization features at different scales. The fusion network uses a residual dense module, merging the restored polarization degree, polarization angle, and underwater degraded polarization image with noise as input to the fusion network. The network outputs a descattered polarization image. To better utilize polarization features, this invention incorporates the correlation between polarization angle and degree of polarization as an effective prior into the network. A polarization hybrid attention mechanism is designed within the descattering network, including a spatial polarization attention mechanism and a channel polarization attention mechanism. Specifically, the polarization angle feature map obtained from each layer in the upsampling layer of the polarization angle restoration network and the polarization degree feature map obtained from each layer in the downsampling layer of the polarization degree restoration network are respectively processed through channel and spatial max pooling to obtain initial spatial weights for polarization angle, initial channel weights for polarization angle, and initial spatial and channel weights for polarization degree. Then, the initial spatial weights for polarization angle and initial spatial weights for polarization degree are merged and passed through a convolutional layer to obtain the spatial polarization attention weights. The initial channel weights for polarization angle and initial channel weights for polarization degree are passed through a fully connected layer to obtain the channel polarization attention weights. Finally, the spatial polarization attention weights and channel polarization attention weights are adjusted in shape through a deformation layer and then added together. After activation, this is multiplied by the polarization degree feature map to obtain the polarization angle-guided polarization degree feature map.

[0010] Furthermore, in step 6, the loss function of the polarization-constrained dual-flow diffusion model includes two parts: denoising network loss and descattering network loss. The denoising network loss calculates the mean square error between the output of the denoising network and the noise added to the degraded underwater polarization image as the pixel loss. The descattering network loss first calculates the mean square error between the network output and the ground truth image at timestamps 0 and T, respectively. Then, it calculates the mean square error of the degree of polarization and the polarization angle between the restored polarization image and the corresponding ground truth polarization image as the polarization information loss. Finally, the pixel loss and the polarization information loss are weighted and added together to obtain the descattering network loss. The aforementioned denoising network loss function The training objective of the denoising network is to minimize the difference between the predicted noise and the actual noise, using L2 loss, as shown in the following formula:

[0011] In the formula, The noise predicted by the denoising network. The real Gaussian noise added in step 1, It is an L2 norm; The aforementioned descattering network loss function Includes pixel-level loss. Diffusion state loss With polarization information loss, where polarization information loss includes polarization angle loss and polarization loss The formula is as follows: .

[0012] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: 1. The underwater polarization image restoration method of the present invention is mainly based on a two-stream diffusion model. The diffusion model is applied to both the degraded underwater polarization image and the ground truth image, and a descattering network is used to bridge the two-stream diffusion model. Two-stream diffusion can better fit the data distribution transformation process between the degraded underwater polarization image and the ground truth image, achieving a more efficient image restoration effect.

[0013] 2. This invention relates to a descattering network, which mainly uses a polarization degree complex atom network, a polarization angle complex atom network, and a fusion network. By constraining the descattering network through polarization angle loss and polarization degree loss, it can guide the diffusion model to focus on polarization information during the reverse denoising process, realize the restoration of polarization information, and constrain the polarization information generated by diffusion to conform to physical laws.

[0014] 3. The method of the present invention relates to a polarization hybrid attention mechanism module, which takes the spatial correlation between polarization angle and polarization degree information as an image prior, and designs a cross-attention mechanism that mixes spatial domain and channel. The polarization angle features obtained by the polarization angle restoration network are fused with the polarization degree features of the polarization degree restoration network through the polarization hybrid attention mechanism module, which can enhance the utilization of polarization information and facilitate more accurate polarization information restoration. Attached Figure Description

[0015] Figure 1 The flowchart and schematic diagram of the descattering network model structure of the method of the present invention are shown below; Figure 2 This is a schematic diagram of the descattering network model structure of the present invention; Figure 3 This is a schematic diagram illustrating the enhanced effect of underwater polarization images and polarization information obtained from real sea areas according to the present invention.

[0016] Figure 4 This is a schematic diagram illustrating the application of the restored polarization information of this invention. Detailed Implementation

[0017] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The present invention is applicable to different color polarization imaging systems (amplitude division, focal plane division, etc.) and different types of polarization information imaging methods (Stokes, Mueller, etc.). The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0018] The following description, with reference to the accompanying drawings, further illustrates an underwater polarization image and polarization information restoration method based on a dual-flow diffusion model proposed in an embodiment of the present invention. The embodiment of the present invention takes the Stokes vector imaging method based on a focal plane polarization image as an example.

[0019] Figure 1 and Figure 2 These are a flowchart and a schematic diagram of a descattering network model, respectively, illustrating an underwater polarization image and polarization information restoration method based on a dual-flow diffusion model according to an embodiment of the present invention. The method of this embodiment includes: S1, Dual-flow forward diffusion In this step, the dual-stream forward diffusion is bridged by noise, which is controlled by timestamps. The larger the timestamp, the larger the noise variance. The underwater degraded polarization image with noise and the ground truth image with the same timestamp contain the same noise.

[0020] Specifically, this step constructs a forward diffusion process for the underwater degraded polarization image and its corresponding ground truth image using a Markov chain, gradually adding noise until the image content is covered by noise, generating a time-stamped sequence of noisy images. Let... This is the original degraded polarization image of water. for The corresponding truth image; , They represent , At timestamp t, in the diffusion state (i.e., the noisy image), the probability density of the forward diffusion process is defined by a Markov chain of length T, with a probability density function of q.

[0021] Generate a series of timestamped underwater degraded polarization images and noise ground truth images, and then... and Add the same Gaussian noise, and The closed-form expression is:

[0022] In the formula, To conform to Gaussian noise with mean 0 and variance , Let be the noise variance control parameter for timestamp t. It is the identity matrix. That is, the previous t steps The product of . Repeat this step until timestamp T, to obtain (Degraded polarization image underwater with noise) and (Noise-based image sequence).

[0023] The timestamp parameter .

[0024] S2, Inverse denoising of underwater degraded polarization images This step involves building a denoising network. Using the noisy degraded image generated in step 1 and the original degraded image as input, the constructed denoising network... Backdiffusion gradually removes noise, restoring the clarity of the degraded image.

[0025] Denoising Network Structure Design: Denoising Network The U-Net architecture is adopted, which contains a symmetrical encoder-decoder structure and skip connections between corresponding layers of the encoder and decoder (which can simultaneously preserve low-level details and high-level semantic features). The timestamp t is embedded into the U-Net through sinusoidal position encoding, enabling the network to perceive noise features at different diffusion stages.

[0026] Inverse denoising calculation: The trained denoising network can predict the noise distribution at timestamp t, and based on this prediction, from... (Current noisy and degraded image) Iterative generation (Previous step noise-degraded image), the formula is as follows:

[0027] In the formula, This refers to Gaussian noise during the sampling process. Indicates (Current image with degraded noise) The denoising network outputs (i.e., predicted noise) with inputs of (original degraded image) and t (timestamp). This is the variance parameter.

[0028] The noise-degraded image with timestamp T from step 1 Compared with the original degraded image Merged input denoising network Simultaneously, the sinusoidally encoded timestamp T is input, and the noise-degraded image with timestamp T-1 is calculated using formula (2). Simultaneously, the ground truth image with timestamp T from step 1 will be generated. Inputting this denoising network yields a ground truth image of noise at timestamp T. .

[0029] S3, Descattering Network Bridge To better utilize polarization features, this invention incorporates the correlation between polarization angle and degree of polarization as an effective prior into the network and designs a polarization hybrid attention mechanism in the descattering network, including a spatial polarization attention mechanism and a channel polarization attention mechanism. Specifically, the polarization angle feature map obtained from each layer in the upsampling layer of the polarization angle restoration network and the polarization degree feature map obtained from each layer in the downsampling layer of the polarization degree restoration network are respectively processed through channel and spatial max pooling to obtain initial spatial weights for polarization angle, initial channel weights for polarization angle, and initial spatial and channel weights for polarization degree. Then, the initial spatial weights for polarization angle and initial spatial weights for polarization degree are merged and passed through a convolutional layer to obtain the spatial polarization attention weights. The initial channel weights for polarization angle and initial channel weights for polarization degree are passed through a fully connected layer to obtain the channel polarization attention weights. Finally, the spatial polarization attention weights and channel polarization attention weights are adjusted in shape through a deformation layer and then added together. After activation, this is multiplied by the polarization degree feature map to obtain the polarization angle-guided polarization degree feature map.

[0030] Specifically, this step involves constructing a descattering network. The noise-degraded image is converted into a descattered polarization image to achieve preliminary elimination of degradation information, as detailed below: Descattering network Its function is to remove noise-degraded images Predicting non-degradable diffusion states (i.e., the non-degradable basis of the ground truth image with noise); due to and Sharing the same noise, the task can essentially be simplified to... predict Then add a noise reduction network Predicted noise.

[0031] Descattering network It consists of three parts: a polarization angle restoration network, a polarization degree restoration network, and a fusion network. The polarization degree restoration network and the polarization angle restoration network are connected through polarization mixing attention blocks, and polarization information is restored by guiding polarization correlation. The polarization hybrid attention block optimizes polarization features through channel attention and spatial attention, and achieves feature calibration by utilizing the polarization correlation between polarization degree and polarization angle (the DoLP of scattered light can be estimated by the spatial distribution of AoP).

[0032] Channel attention calculation: For the polarization degree restoration network Features of each convolutional block (C is the number of channels, H is the height, and W is the width) and characteristics of the polarization angle restoration network (n is the total number of convolutional blocks), spatial domain average pooling is used respectively. (deal with ) and max pooling (deal with Initial weights are generated, and then the channel attention map is output through a shared fully connected network (FC). :

[0033] Spatial attention calculation: for the polarization degree restoration network Features of each convolutional block Features of polarization angle restoration networks Average pooling is applied along the channel axis. With max pooling The results are then concatenated and processed through two layers of 3×3 dilated convolution ( Output spatial attention map (Diffuse convolution is used to expand the receptive field and utilize contextual information):

[0034] Feature Refinement: and All reshaped into After activation by the function, it is compared with the input features. Element-wise multiplication yields the refined DoLP features. :

[0035] In the formula, It is the sigmoid activation function.

[0036] The noise-degraded image with timestamp T-1 obtained in step 2 Input descattering network After the polarization angle is restored by the polarization angle restoration network and the polarization degree is restored by the polarization degree restoration network, the polarization degree and polarization angle are fused with the noise-degraded image by the fusion network. The characteristics of the output descattered polarization image with a timestamp of T-1 are used to determine the characteristics of the image.

[0037] S4, Truth Image Inverse Denoising This step combines the descattered polarization image from step 3 with the denoising network from step 2 to perform reverse denoising on the ground truth image, further optimizing the recovery accuracy of the ground truth image.

[0038] The ground truth image of the noise at time stamp T-1 in step 1 The descattered polarization image with timestamp T-1 obtained in step 3 is added element by element to obtain the noise ground truth image with fused descattered information.

[0039] The noise ground truth image obtained above with fused descattering information is input into the denoising network constructed in step 2. Simultaneously, the timestamp T-1 is embedded into the network using sinusoidal position encoding, and based on the same back-diffusion logic as in step 2, a noise ground truth image with timestamp T-2 is output. Its mathematical expression is:

[0040] In the formula, This is the descattered polarization image output by the descattering network.

[0041] S5, Iterative dual-stream reverse denoising and descattering In this iterative dual-stream inverse denoising and descattering step, the dual-stream inverse denoising is bridged by the descattering network from step 3. This network contains three sub-networks: a polarization degree complex atom network, a polarization angle complex atom network, and a fusion network. The polarization degree complex atom network and the polarization angle complex atom network adopt a U-shaped structure, including downsampling and upsampling layers to obtain polarization features at different scales. The fusion network uses a residual dense module, which combines the restored polarization degree, polarization angle, and underwater degraded polarization image with noise as input to the fusion network. The network output is a descattered polarization image.

[0042] Specifically, this step iteratively executes steps 2-4, gradually decreasing the timestamp t until t=1, ultimately generating the restored polarization image and polarization information.

[0043] Starting with timestamp T-1, the timestamp is decreased by 1 in each iteration (i.e., Repeat step 2 (calculate the noisy degraded image with timestamp t=1). Step 3 (Calculate the descattered polarization image at time t-1) and Step 4 (Calculate the noise ground truth image at time stamp t=1) (and so on) until t=1.

[0044] Since image superposition during the iteration process may cause deviations in the final restored image, it is necessary to analyze the ground truth image with time stamp t. Perform the distribution offset operation using the following formula:

[0045] In the formula, The median mean parameter is calculated using the following formula:

[0046] The iteration stops at t=1, and the ground truth image of the noise at this point is obtained. Add the descattered polarization image to the denoising network. The ground truth image with timestamp 0 is obtained. Then, according to formula (8), we get This refers to the final restored polarization image.

[0047] Based on restored polarization image By using the polarization Stokes vector calculation model, the polarization image is restored. The polarization information, including the degree of polarization and the polarization angle, is extracted and calculated.

[0048] S6, Design of loss function for polarization-constrained two-stream diffusion model This step designs loss functions for the denoising network and the descattering network separately, constraining the model to simultaneously restore accurate image and polarization information. The loss function of the polarization-constrained dual-stream diffusion model includes two parts: denoising network loss and descattering network loss. The denoising network loss calculates the mean square error between the output of the denoising network and the noise added to the degraded underwater polarized image as the pixel loss. The descattering network loss first calculates the mean square error between the network output and the ground truth image at timestamps 0 and T, respectively. Then, it calculates the mean square error of the degree of polarization and the polarization angle between the restored polarized image and the corresponding ground truth polarized image as the polarization information loss. Finally, the pixel loss and the polarization information loss are weighted and summed to obtain the descattering network loss.

[0049] Specifically, the loss function of the denoising network ( The training objective of the denoising network is to minimize the difference between the predicted noise and the actual noise, using L2 loss, as shown in the following formula: In the formula, The noise predicted by the denoising network. The real Gaussian noise added in step 1, It is an L2 norm.

[0050] Descattering network loss function ( The loss function of the descattering network includes pixel-level loss ( ), diffusion state loss ( ) and polarization information loss, where polarization information loss includes polarization angle loss ( ) and polarization loss ( The formula is as follows:

[0051] The definitions of each sub-loss in the formula are as follows: Pixel-level loss ( ): Ensure the truth image at t=0 (no diffusion state) With restored polarization image Pixel-level precision, using L2 loss:

[0052] In the formula, The output of the descattering network at t=0 (i.e., directly from...) predict ).

[0053] Diffusion state loss ( Since the denoising network is trained for both denoising and descattering tasks, it is necessary to minimize the time required for t steps. and The differences ensure consistency in the diffusion state:

[0054] Polarization angle loss ( To ensure consistency between the restored polarization angle and the true polarization angle, L2 loss is employed and a balance coefficient is introduced. :

[0055] In the formula, (Empirical balance coefficient) This is a function for calculating the polarization angle.

[0056] Polarization loss ( To ensure consistency between the restored DoLP and the real DoLP, L2 loss is used and a balance coefficient is introduced. :

[0057] In the formula, (Empirical balance coefficient) This is the function for calculating the degree of polarization.

[0058] In embodiments of the present invention, an underwater polarization image is processed using an underwater polarization image and polarization information restoration method based on a dual-flow diffusion model, resulting in restored intensity, degree of polarization, and polarization angle maps, as follows. Figure 3 As shown in the figure, the underwater polarization image and polarization information restoration method based on the dual-flow diffusion model proposed in this invention can effectively remove image blurring caused by scattering, and can well recover the polarization information of polarization degree images and polarization angle images from the information submerged by scattering blur. Figure 4 The invention demonstrates the application of underwater polarization information after restoration, where the degree of polarization can be used for camouflage target identification, material identification, etc., and the polarization angle can be applied to surface normal vector detection, transparent object detection, etc.

[0059] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0060] While this application provides method operation steps as shown in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive labor. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual device or client product execution, the method can be executed in the order shown in the embodiments or drawings or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0061] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0062] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0063] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0064] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0065] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention, and the resulting technical solutions are all within the protection scope of the present invention.

Claims

1. A method for restoring underwater polarization images and polarization information based on a dual-flow diffusion model, characterized in that, Includes the following steps: Step 1. Dual-stream forward diffusion The underwater degraded polarization image and its corresponding ground truth image are respectively constructed using a Markov chain for forward diffusion. Noise is then gradually added to the underwater degraded polarization image and its corresponding ground truth image, with each step using a timestamp parameter. Control until the content of the underwater degraded polarization image and the corresponding ground truth image is covered by noise, generating a series of noisy underwater degraded polarization images and noisy ground truth images marked with timestamps; Step 2. Inverse denoising of underwater degraded polarization images A denoising network model is constructed. The underwater degraded polarization image with timestamp T obtained in step 1 is merged with the underwater degraded polarization image and input into the denoising network. The ground truth image with timestamp T-1 obtained in step 1 is input into the denoising network. The timestamp is embedded into the denoising network through sinusoidal encoding to obtain the underwater degraded polarization image with timestamp T-1 and the ground truth image with timestamp T-1. Step 3. Bridge the descattering network Construct a descattering network model, input the noisy underwater degraded polarization image with timestamp T-1 obtained in step 2 into the descattering network model, and obtain a descattered polarization image with timestamp T-1; Step 4. Inverse denoising of the ground truth image Add the noise ground truth image with timestamp T-1 obtained in step 1 to the descattered polarization image with timestamp T-1 obtained in step 3, then input it into the denoising network constructed in step 2, and embed the timestamp into the denoising network through sinusoidal encoding to obtain the noise ground truth image with timestamp T-2. Step 5. Iterative dual-stream reverse denoising and descattering Repeat steps 2, 3, and 4, decreasing the timestamp by 1 with each iteration. Stop the iteration when the timestamp is 1. This yields a noisy underwater degraded polarization image, a noise ground truth image, and a descattered polarization image with timestamp 1. Add the noise ground truth image and the descattered polarization image and then input them into the denoising network to obtain the noise ground truth image with timestamp 0, i.e., the restored polarization image. Simultaneously, the restored polarization information, including the degree of polarization and the polarization angle, can be calculated based on the polarization Stokes. Step 6. Design of loss function for polarization-constrained two-stream diffusion model Loss functions were designed for the denoising network and the descattering network respectively to constrain the dual-flow diffusion model to restore accurate image and polarization information.

2. The method for restoring underwater polarization images and polarization information based on a dual-flow diffusion model according to claim 1, characterized in that, The dual-stream forward diffusion described in step 1 is bridged by noise, which is controlled by timestamps. The larger the timestamp, the larger the noise variance. The underwater degradation polarization image of the noise with the same timestamp contains the same noise as the ground truth image of the noise.

3. The underwater polarization image and polarization information restoration method based on the dual-flow diffusion model according to claim 1, wherein the denoising network in step 2 adopts the U-Net architecture, which contains a symmetrical encoder-decoder structure and skip connections between corresponding layers of the encoder and decoder, and can simultaneously preserve low-level details and high-level semantic features; the timestamp t is embedded into the U-Net through sinusoidal position encoding, so that the network can perceive the noise features of different diffusion stages.

4. The underwater polarization image and polarization information restoration method based on the dual-flow diffusion model according to claim 1, wherein the descattering network in step 3 consists of three parts: a polarization angle restoration network, a polarization degree restoration network, and a fusion network, and the polarization degree restoration network and the polarization angle restoration network are connected through a polarization hybrid attention block, which guides the recovery of polarization information through polarization correlation; the polarization hybrid attention block optimizes polarization features through channel attention and spatial attention, and realizes feature calibration by utilizing the polarization correlation between polarization degree and polarization angle.

5. The underwater polarization image and polarization information restoration method based on a dual-flow diffusion model according to claim 1, characterized in that, In step 5, the iterative dual-stream inverse denoising and descattering process involves bridging the descattering network described in step 3. This network contains three sub-networks: a polarization degree complex atom network, a polarization angle complex atom network, and a fusion network. The polarization degree complex atom network and the polarization angle complex atom network adopt a U-shaped structure, including downsampling and upsampling layers to obtain polarization features at different scales. The fusion network uses a residual dense module, which combines the restored polarization degree, polarization angle, and underwater degraded polarization image with noise as input to the fusion network. The network outputs a descattered polarization image.

6. The underwater polarization image and polarization information restoration method based on a dual-flow diffusion model according to claim 5, characterized in that, The correlation between polarization angle and polarization degree is incorporated into the network as an effective prior, and a polarization hybrid attention mechanism is designed in the descattering network, including a polarization spatial attention mechanism and a polarization channel attention mechanism. Specifically, the polarization angle feature map obtained by each layer in the upsampling layer of the polarization angle restoration network and the polarization degree feature obtained by each layer in the downsampling layer of the polarization degree restoration network are respectively obtained by max pooling in the channel and space to obtain the initial spatial weights of polarization angle, the initial channel weights of polarization angle, and the initial spatial weights and initial channel weights of polarization degree. Then, the initial spatial weights of polarization angle and the initial spatial weights of polarization degree are merged and passed through a convolutional layer to obtain the polarization spatial attention weights. The initial channel weights of polarization angle and the initial channel weights of polarization degree are passed through a fully connected layer to obtain the polarization channel attention weights. Finally, the polarization spatial attention weights and the polarization channel attention weights are added after adjusting their shape through a deformation layer, and after activation, multiplied by the polarization degree feature to obtain the polarization angle-guided polarization degree feature map.

7. The underwater polarization image and polarization information restoration method based on a dual-flow diffusion model according to claim 1, characterized in that, In step 6, the loss function of the polarization-constrained dual-stream diffusion model includes two parts: denoising network loss and descattering network loss. The denoising network loss calculates the mean square error between the output of the denoising network and the noise added to the degraded underwater polarization image as the pixel loss. The descattering network loss is first calculated by comparing the mean square error of the network output with the ground truth image at timestamps 0 and T. Then, the mean square error of the degree of polarization and the polarization angle of the restored polarization image with the corresponding ground truth polarization image are calculated as polarization information loss. Finally, the pixel loss and polarization information loss are weighted and summed to obtain the descattering network loss.

8. The underwater polarization image and polarization information restoration method based on a dual-flow diffusion model according to claim 7, in step 6, the denoising network loss function... The training objective of the denoising network is to minimize the difference between the predicted noise and the actual noise, using L2 loss, as shown in the following formula: In the formula, The noise predicted by the denoising network. The real Gaussian noise added in step 1, It is an L2 norm; The aforementioned descattering network loss function Includes pixel-level loss. Diffusion state loss With polarization information loss, where polarization information loss includes polarization angle loss and polarization loss The formula is as follows: 。

Images