An underwater image despeckling model construction method based on polarization imaging

Abstract

The application belongs to the technical field of polarization imaging, and particularly relates to a water image despeckling model construction method based on polarization imaging, which comprises the following steps: using a feature extraction encoder to map four polarization angle images and calculated intensity images to a low-dimensional latent feature space; introducing a diffusion model network, a forward process of which is used to gradually add Gaussian noise until the noise data conform to a standard Gaussian distribution, starting from clear latent features extracted based on a clear water target intensity image and a turbid water target polarization image; a reverse process of the diffusion model network is used to denoise the noise data under the guidance of polarization features of the turbid water target, generate clear latent features of the underwater target, and obtain clear latent feature estimation results; using a reconstruction module network, based on the estimation results and the intensity image of the underwater target in the corresponding sample, a corresponding clear underwater target image is generated. The application can adapt to scattering environments of water bodies with different turbidity, and has high generalization and high robustness.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of polarization imaging, and more particularly relates to a method for constructing an underwater image despeckling model based on polarization imaging. BACKGROUND

[0002] In an underwater imaging environment, scattering and absorption effects caused by suspended particles and dissolved substances can seriously degrade image quality. Among them, backscattering can cause color distortion and blurring of the collected image, reduce the image contrast, hinder the extraction of optical information of underwater targets, and seriously affect the application of fields such as seabed exploration and underwater rescue.

[0003] Underwater image despeckling is a highly ill-posed inverse problem. Although traditional optical underwater image despeckling methods have achieved certain results, they still have some bottlenecks. Among them, methods based on imaging models, image enhancement methods, and depth estimation methods rely on idealized imaging models, and empirical priors are difficult to adapt to complex underwater environments, and scene generalization is limited. Methods based on deep learning cannot separate backscattered light and target reflected light, and have no physical constraints, resulting in unstable results. Polarization imaging technology is an effective underwater despeckling method. It can effectively separate target signal light and backscattered light by using the polarization characteristic difference between the two, achieving the effect of suppressing scattering. Current underwater despeckling based on polarization imaging mostly focuses on combining polarization technology with physical models. Although this can improve image restoration quality, there are still some limitations. Patent CN 114494079B discloses a fast underwater image enhancement method that fuses polarization information. It uses Stokes vectors to convert the original optimal differential component problem, which is difficult to solve, into several easy-to-solve optimization problems, and obtains the enhanced image that meets the conditions through iteration. This method mainly relies on the idealized assumptions of the polarization imaging model, and has poor scene generalization for non-uniform scattering light distribution. The number of iterations under different turbidity water body scenes cannot be unified, making it difficult to adapt to different scenes. In recent years, methods based on deep learning have been gradually applied to underwater polarization imaging despeckling. Patent CN 114758030B proposes an underwater polarization imaging method that fuses physical models and deep learning. It combines an underwater polarization imaging model to construct a core network for feature extraction, obtains output polarization modulation parameters, and constructs an underwater imaging polarization despeckling correction model. The polarization modulation parameters are input into the underwater imaging polarization despeckling correction model to obtain the restored image. This method relies heavily on assumptions about the water body and scattering environment. Although it combines deep learning methods, the physical model is only used to construct input polarization modulation parameters, and the image restoration core process is still completed by an end-to-end CNN convolutional neural network. The local receptive field of the convolutional network has limitations and cannot model global scattering scenes, resulting in a significant decrease in performance in complex scenes such as high turbidity. At the same time, the CNN training process lacks interpretability. Although the literature "Transformer-based underwater polarization descattering in natural illumination" uses the self-attention mechanism of the Transformer to enhance the global association between multi-polarization features, its essence is still an end-to-end discriminative despeckling model. The feature extraction and despeckling process do not explicitly introduce polarization physical priors, so the model can only rely on data-driven mapping relationships to compensate for scattering.Most of the existing deep learning methods combining polarization information only take polarization images as input features, and do not construct a generative polarization prior to recover the target structure under clear water conditions from scattering interference. At the same time, the end-to-end network structure lacks physical constraints, and the training process has poor interpretability.

[0004] Therefore, in view of the limitations of the above methods, it is necessary to further improve the interpretability of the deep learning method, introduce a generative modeling mechanism, combine polarization information to construct a constrained feature model, adapt to the scattering environment of water bodies with different turbidity, and improve the generalization and robustness of the algorithm. SUMMARY

[0005] In view of the above defects or improvement needs of the prior art, the present application provides a water image despeckling model construction method based on polarization imaging, which aims to provide a high generalization and high robustness despeckling method that adapts to the scattering environment of water bodies with different turbidity.

[0006] To achieve the above-mentioned purpose, according to one aspect of the present application, a water image despeckling model construction method based on polarization imaging is provided, comprising:

[0007] Construct a training sample set, each training sample including an intensity image of a target under clear water and a plurality of polarization angle images, and an intensity image of a target under turbid water and a plurality of polarization angle images corresponding to one kind of turbid water, wherein the intensity image of the target under clear water is used as a true value label;

[0008] Construct a feature extraction encoder network for stacking the intensity image of the target under turbid water and the plurality of polarization angle images in each sample by channel, and performing feature extraction on the stacking result and the intensity image of the target under clear water in the sample to obtain clear latent features of the underwater target; at the same time, the stacking result is also separately subjected to feature extraction to obtain polarization features of the target under turbid water;

[0009] Construct a diffusion model network, the forward process part of which is used to start from the clear latent features output by the feature extraction encoder, gradually add Gaussian noise to the features at each time step, and convert the clear latent features into noise data conforming to a standard Gaussian distribution as the time step increases; the reverse process part is used to gradually denoise the noise data conforming to the Gaussian distribution based on a denoising network under the guidance of the polarization features output by the feature extraction encoder, to generate the clear latent features of the underwater target, and obtain a clear latent feature estimation result;

[0010] Construct a reconstruction module network for reconstructing a corresponding clear image of the underwater target based on the clear latent feature estimation result output by the diffusion model and the intensity image of the target under turbid water in the corresponding sample;

[0011] The loss is calculated based on the true value label and the generated clear intensity image of the underwater target by using the training sample set, and the feature extraction encoder network, the diffusion model network and the reconstruction module network are trained; the feature extraction encoder, the diffusion model inverse process part and the reconstruction module obtained by training constitute the de-scattering model, and the construction is completed.

[0012] Further, each image in each training sample is obtained by sequentially performing image normalization, random cropping and image enhancement on its original image.

[0013] Further, the feature extraction encoder network comprises:

[0014] A convolutional layer: containing a cascade of convolutional layers and GELU activation functions, used for extracting shallow features;

[0015] Residual convolutional block: containing a plurality of block residual convolutional blocks, each residual convolutional block being composed of a convolutional layer plus a corresponding GELU activation function, and each residual convolutional block being combined in a skip connection manner, and local spatial features being extracted step by step;

[0016] MLP module: used for global feature modeling, and finally generating clear latent features or polarization features.

[0017] Further, the main framework of the denoising network is a U-net structure.

[0018] Further, the reconstruction module network comprises:

[0019] Stacking unit: used for stacking the clear latent feature estimation results and the intensity images of the underwater target in the turbid water body in the corresponding sample according to the channel;

[0020] Input convolutional mapping layer: composed of a convolutional layer and a GELU activation function following it, used for preliminary mapping of the stacking results output by the stacking unit to obtain image reconstruction features;

[0021] Residual convolutional block group: containing a plurality of residual convolutional blocks connected in sequence, each residual convolutional block being composed of a convolutional layer plus a corresponding GELU activation function, and each residual convolutional block being combined in a skip connection manner, so as to recover image structure information and detail texture step by step;

[0022] Output reconstruction convolutional layer: containing a convolutional layer, the number of output channels of which is consistent with that of the intensity image of the underwater target in clear water, used for generating an output clear intensity image of the underwater target.

[0023] Further, in the training process, the loss function used to calculate the loss is is expressed as:

[0024]

[0025]

[0026] in, , , , These are the weight parameters for the prior-based simplified diffusion loss, pixel loss, gradient loss, and phase loss, respectively; the prior-based simplified diffusion loss represents the clear latent features of the target extracted by the feature extraction encoder network. Clear latent feature estimation results reconstructed during the inverse process The L1 norm loss and pixel loss are used to reconstruct the descattered image generated by the reconstruction module network. Intensity image of a target in clear water The pixel L1 norm loss and gradient loss are generated by the reconstruction module network. Intensity image of a target in clear water The gradient L1 norm loss and phase loss are used to reconstruct the descattered image generated by the reconstruction module network. Intensity image of a target in clear water Fourier transform Back phase angle The L1 norm loss.

[0027] According to another aspect of the present invention, an underwater image descattering method based on polarization imaging is provided, comprising:

[0028] Based on the intensity image and multiple polarization angle images of the blurred target in turbid water, the underwater image descattering model constructed by the underwater image descattering model construction method based on polarization imaging described above is used to obtain the descattered target object image.

[0029] According to another aspect of the present invention, an electronic device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the method described above.

[0030] According to another aspect of the invention, a computer-readable storage medium is provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is run by a processor, it controls the device where the storage medium is located to perform the steps of the method described above.

[0031] According to another aspect of the invention, a computer program product is provided, comprising a computer program or instructions that, when executed by a processor, implement the steps of the method described above.

[0032] Overall, compared with the prior art, the technical scheme provided by the present application has the following beneficial effects:

[0033] The application provides a polarized imaging-based underwater image despeckling model construction method, which maps four polarized angle images and calculated intensity images to a latent feature (i.e., a low-dimensional deep feature) space by using a feature extraction encoder, thereby constructing a diffusion model in a low-dimensional latent space, avoiding direct modeling of a complex diffusion model distribution in a high-dimensional image space, significantly reducing the difficulty of diffusion modeling, and improving training stability and generation quality. The clear latent feature of the underwater target is introduced into the diffusion model forward process by the encoder in the training stage, and the polarized feature of the underwater target in the turbid water body is introduced as conditional information (as a conditional constraint) in the diffusion reverse process, so that the distribution relationship between the polarized feature of the underwater target in the turbid water body and the latent feature of the corresponding underwater target in clear water can be modeled in the diffusion model, so that the trained diffusion model can realize explicit constraint of the polarized feature despeckling of the underwater target in the turbid water body in the reverse diffusion process, and the step-by-step generation process of the clear latent feature of the underwater target is guided by the prior of the underwater target image in clear water, the feature reconstruction process conforms to the polarized imaging mechanism, has high physical consistency, and can effectively suppress the target distortion and texture artifacts caused by scattering in the turbid water body. Then, the clear latent feature estimation result of the underwater target generated by the diffusion model is used as a guide, and the intensity image of the underwater target in the turbid water body is input into the despeckling model to reconstruct and generate a clear image of the underwater target, which can realize high consistency with the intensity image of the underwater target in clear water, has the characteristics of detail preservation, and achieves the purpose of underwater image despeckling. Therefore, the method can adapt to the scattering environment of water bodies with different turbidity, has high generalization and high robustness. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 A polarized imaging-based underwater image despeckling model construction method is provided for the embodiments of the application.

[0035] Figure 2 A training process schematic diagram of the polarized imaging prior and diffusion model guided underwater image despeckling network framework is provided for the embodiments of the application.

[0036] Figure 3 A structure diagram of the polarized imaging prior and diffusion model guided underwater image despeckling network framework is provided for the embodiments of the application.

[0037] Figure 4 An inference process schematic diagram of the polarized imaging prior and diffusion model guided underwater image despeckling network framework is provided for the embodiments of the application. DETAILED DESCRIPTION

[0038] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application. In addition, the technical features involved in the various embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0039] Embodiment one

[0040] A method for constructing an underwater image despeckling model based on polarization imaging, as shown in Figure 1 , comprising:

[0041] Construct a training sample set, each training sample comprising an intensity image of a target under clear water and a plurality of polarization angle images and an intensity image of a target under turbid water and a plurality of polarization angle images corresponding to one, wherein the intensity image of the target under clear water is used as a true value label;

[0042] Construct a feature extraction encoder network for stacking the intensity image of the target under turbid water and the plurality of polarization angle images in each sample by channel, and extracting features from the stacking result and the intensity image of the target under clear water in the sample to obtain clear latent features of the underwater target; at the same time, the stacking result is also extracted separately to obtain polarization features of the target under turbid water;

[0043] Construct a diffusion model network, the forward process part of which is used to start from the clear latent features output by the feature extraction encoder, gradually add Gaussian noise to the features at each time step, and convert the clear latent features into noise data conforming to a standard Gaussian distribution as the time step increases; the reverse process part thereof is used to gradually denoise the noise data conforming to the Gaussian distribution based on a denoising network under the guidance of the polarization features output by the feature extraction encoder to generate the clear latent features of the underwater target, and obtain clear latent feature estimation results;

[0044] Construct a reconstruction module network for generating corresponding clear underwater target images based on the clear latent feature estimation results output by the diffusion model and the intensity image of the target under turbid water in the corresponding sample;

[0045] Using the training sample set, calculate the loss based on the true value label and the clear intensity image of the underwater target generated by reconstruction, and train the feature extraction encoder network, the diffusion model network and the reconstruction module network; the trained feature extraction encoder, the reverse process part of the diffusion model and the reconstruction module constitute a despeckling model, and the construction is completed.

[0046] There are some deficiencies in the existing water scattering methods based on polarization imaging. For example, the method of combining polarization technology with a physical model mainly relies on the idealized assumptions of the polarization imaging model, and the scene generalization is difficult to adapt to different scenes. The end-to-end CNN convolutional neural network method lacks interpretability in the training process, and may have problems such as loss of details or color distortion.

[0047] The method provided by the embodiment maps the four polarization angle images and the calculated intensity images to the latent feature (i.e., low-dimensional deep feature) space by using a feature extraction encoder, thereby constructing a diffusion model in the low-dimensional latent space, avoiding direct modeling of a complex diffusion model distribution in a high-dimensional image space, significantly reducing the difficulty of diffusion modeling, and improving the training stability and generation quality.

[0048] It is further proposed that the feature extraction encoder extracts clear latent features (including polarization information and brightness information of the target) of the underwater target based on the intensity image and the multiple polarization angle images of the target under the turbid water and the intensity image of the target under the clear water, and uses the clear latent features as the feature prior of the ground truth image (i.e., the intensity image of the target under the clear water) of the underwater target, introduces the clear latent features into the diffusion model training stage as the input of the forward process of the diffusion model, and uses the clear latent features to generate a noise image containing the polarization feature information of the target under the turbid water in the forward process of the diffusion model training stage. Then, in the inverse process of the diffusion model, the clear latent features are recovered under the guidance of the polarization features of the target under the turbid water. Through training, the image result after denoising and reconstruction of the noise image by the inverse process of the diffusion model can more truly restore the clear latent features of the underwater target. In this way, a diffusion model capable of de-scattering the polarization features of the target under the turbid water is trained, and the clear latent features of the underwater target extracted by the encoder play a supervisory role in this process. The trained feature extraction encoder, diffusion model, and de-scattering model constitute a de-scattering model based on polarization imaging prior and diffusion model guidance.

[0049] In the method of the embodiment, the clear latent features of the underwater target extracted by the feature extraction encoder are introduced into the forward process of the diffusion model in the training stage, and the polarization features of the target under the turbid water are introduced as conditional information in the inverse process of the diffusion model, so that the distribution relationship between the polarization features of the target under the turbid water and the latent features of the corresponding target under the clear water can be modeled in the diffusion model. Therefore, the trained diffusion model can realize explicit constraints on the de-scattering of the polarization features of the target under the turbid water in the inverse diffusion process, the gradual correction and recovery process of the clear latent features of the underwater target is guided by the prior of the target image under the clear water, the feature reconstruction process conforms to the polarization imaging mechanism, has high physical consistency, and can effectively suppress the target distortion and texture artifacts caused by scattering in the turbid water.

[0050] Then, the estimation result of the clear potential features of the underwater target generated by the diffusion model is taken as a guide, and is input into a reconstruction module together with the intensity image of the target under the turbid water body to reconstruct a clear image of the underwater target. The clear image has high consistency with the intensity image of the clear underwater target in terms of texture and structure, and thus the method can realize detail fidelity and achieve the purpose of underwater image despeckling.

[0051] It should be noted that in the method of the embodiment, the multiple polarization angle images can include polarization images of four polarization angle directions of 0°, 45°, 90° and 135°.

[0052] The four polarization angle images are used to construct a polarization feature space in a channel cascading manner, which combines the polarization characteristics of the images of different polarization directions. Meanwhile, the intensity images are stacked in the channel in the encoder feature extraction stage to realize the fusion of the polarization information and the intensity information, so that the extracted features fuse the physical characteristics of different polarization directions and the brightness and texture information of the target, thereby enhancing the visibility of the target image and improving the retention ability of the texture details.

[0053] In general, the method of the embodiment can improve the explainability of the deep learning method. By introducing a generative modeling mechanism and combining polarization information to construct a constraintable feature modeling, the method can adapt to the scattering environment of different turbidity water bodies and improve the generalization and robustness of the algorithm.

[0054] Regarding the construction of the sample set, in specific implementation, an active illumination polarization imaging system can be built to construct a polarization underwater image dataset. Under laboratory conditions, active illumination is used, a focal plane polarization camera is used, a suitable exposure time is set, and the linear polarization image combination of the target in four polarization angle directions of 0°, 45°, 90° and 135° is captured. To construct the clear-turbid water body data pair, the clear polarization image (i.e., the linear polarization image combination) of the underwater target is collected under clear water, and different volumes of turbid medium are added to the same volume of clear water to simulate different turbidity water body environments to prepare turbid water bodies, and the polarization image (i.e., the linear polarization image combination) of the blurred target under the corresponding turbidity water body is collected. In this way, enough underwater polarization data of the target is collected to construct the polarization underwater image dataset.

[0055] The polarization underwater image dataset collected is combined (polarization angle images in four directions of 0°, 45°, 90° and 135°) to calculate the intensity image of the target , and the formula is as follows:

[0056]

[0057] wherein, , , , respectively represent the polarization angle images of four directions of 0°, 45°, 90° and 135°.

[0058] The constructed data set is divided into a training sample set, a validation sample set and a test sample set, wherein each group of data contains four polarization angle images, intensity images and corresponding four polarization angle images, intensity images collected under one kind of turbid water body.

[0059] As a preferred embodiment, the data set images are preprocessed, including image normalization. After image normalization, random cropping processing is performed (the specific uniform cropping size can be set according to actual requirements, for example, the size can be uniformly cropped to 512x512 for illustration), and image enhancement is performed on the images. The image enhancement operation includes rotation, translation, flipping, etc.

[0060] Regarding the feature extraction and encoding network, in the specific implementation, the 0°, 45°, 90° and 135° polarization angle images and the intensity images of the blurred target under the turbid water body are stacked by channel, and then the intensity images of the target under the clear water are extracted together to generate clear latent features of the underwater target (i.e. low-dimensional deep features containing physical characteristics of different polarization directions) for the subsequent forward process of the diffusion model. At the same time, the 0°, 45°, 90° and 135° polarization angle images and the intensity images of the blurred target under the turbid water body are stacked by channel and then extracted features separately to generate polarization features of the target under the turbid water body for the subsequent reverse process of the diffusion model.

[0061] The 0°, 45°, 90° and 135° polarization angle images and the intensity images of the blurred target under the turbid water body are stacked by channel, that is:

[0062]

[0063] wherein, , , , represents the 0°, 45°, 90° and 135° polarization angle images of the target under the turbid water body, represents the intensity image calculated from the four polarization images of the corresponding target under the turbid water body. represents the stacked image of the target under the turbid water body.

[0064] Then, the intensity images of the target under the clear water are extracted together to obtain clear latent features of the underwater target as prior features for training of the diffusion model, that is:

[0065] ​

[0066] wherein, denotes the feature extraction encoder network, is the feature stacked by channel for the multiple images under the turbid water, is the intensity image calculated for the four polarization angles of the corresponding clear underwater target image, is the clear latent feature of the underwater target extracted by the encoder.

[0067] The 0°, 45°, 90° and 135° polarization angle images and intensity images of the blurred target under the turbid water After stacking by channel, the features are extracted separately to generate the polarization features of the target under the turbid water for the subsequent diffusion model reverse process, that is,

[0068]

[0069] wherein, as the input condition of the diffusion model reverse process.

[0070] As a preferred implementation, the feature extraction encoder network includes a convolutional layer: for example, a convolutional layer with a 3x3 convolutional kernel size, followed by a GELU activation function, to extract shallow features; residual convolutional blocks: for example, 4 blocks of residual convolutional blocks. Each residual convolutional block is composed of (for example, 2 3x3) convolutional layers plus the corresponding GELU activation function. Each residual convolutional block is combined by a skip connection method, and local spatial features are extracted step by step; MLP module: the MLP module models the global features, and finally generates the latent feature prior or polarization feature for the subsequent diffusion model.

[0071] Regarding the diffusion model, in specific implementation, under the guidance of the polarization features of the target under the turbid water extracted by the feature extraction encoder, the distribution relationship between the polarization features of the target under the turbid water and the clear latent features of the corresponding underwater target is modeled, so as to reconstruct the clear latent features of the underwater target.

[0072] The polarization angle images (0°, 45°, 90°, and 135°) and intensity images of the target in turbid water are stacked by channel and output by a feature extraction encoder. The polarization features of the target are used as input to the inverse process of the diffusion model. The clear latent features of the underwater target extracted by the feature extraction encoder are used as input to the forward process of the diffusion model as a supervision signal. In the forward process, starting from the clear latent features of the underwater target extracted by the feature extraction encoder, Gaussian noise is gradually added to the latent features at each time step t. As the time step length increases sufficiently, it is converted into noise data conforming to a standard Gaussian distribution. In the inverse process, the polarization features of the target in turbid water extracted by the feature extraction encoder are used as input conditions. The denoising network gradually denoises the noise data conforming to the Gaussian distribution to recover (or generate) the prior features of the clear underwater target image.

[0073] Specifically, during the forward pass of the diffusion model, the feature extraction encoder extracts clear latent features of the underwater target. Used as input and as a supervisory signal to guide the model. From clear latent features Initially, Gaussian noise is gradually added at each time step t. As the time step T increases sufficiently, the latent features are... Convert to corresponding noise data conforming to a standard Gaussian distribution. The process is as follows:

[0074]

[0075] in, This represents the clear latent feature priors of the input diffusion model. , Indicates a Gaussian distribution. , , I is a variable used to control the variance of the noise added at time step t, where I represents the identity matrix.

[0076] Subsequently, during the inverse process of the diffusion model, the trained model is gradually denoised, gradually generating clear latent features for underwater targets. The estimation results can be expressed as follows:

[0077]

[0078] in, Indicates the predicted first Step characteristics The conditional mean, and the mean under the current state. Time step t and conditional input related, is a noise variance parameter related to the time step t in the diffusion process, used to control the randomness degree in the reverse sampling process. The conditional input is the polarization feature of the turbid underwater target, which is obtained by feature extraction by the feature extraction encoder.

[0079] After the reverse process is completed, the clear potential feature estimation result of the underwater target is output, which is used for the de-scattering network to reconstruct and generate a predicted clear image of the underwater target.

[0080] As a preferred implementation, the main framework of the denoising network is a U-net structure.

[0081] Regarding the reconstruction module network, in specific implementation, it is used to reconstruct the latent clear feature estimation result output by the diffusion model into a corresponding clear image. The reconstruction module network stacks and processes the clear latent feature estimation result of the underwater target output by the diffusion model and the intensity image of the underwater target in the turbid water body by channel, and reconstructs and generates a corresponding clear image of the underwater target, as follows:

[0082]

[0083] wherein, is the latent feature estimation result of the clear underwater target image recovered by the diffusion model, is the intensity image of the underwater target in the turbid water body, is the reconstruction module network, is the reconstructed clear image of the underwater target.

[0084] As a preferred implementation, the reconstruction module includes:

[0085] The input convolutional mapping layer is composed of a convolutional layer (for example, a convolutional kernel size of 3x3) and a GELU activation function following it, which is used to preliminarily map the clear latent feature estimation result to obtain image reconstruction features;

[0086] The residual convolutional block group includes multiple residual convolutional blocks (for example, four) connected in sequence, each residual convolutional block is composed of a convolutional layer (for example, two 3x3 sizes) plus a corresponding GELU activation function, and each residual convolutional block is combined in a skip connection manner, thereby recovering image structure information and detail texture step by step;

[0087] The output reconstruction convolutional layer includes a convolutional layer (for example, 3x3 size), the number of output channels of which is consistent with that of the target image, which is used to generate an output clear intensity image of the underwater target.

[0088] Regarding the loss function, the intensity image calculated from the four polarization images of the clear underwater target is the true value, which is obtained by extracting the clear latent feature of the underwater target by the feature extraction encoder , the clear feature estimation result of the underwater target estimated in the inverse process of the diffusion model is obtained, and a de-scattered image generated by the reconstruction module is obtained , the loss function of the underwater image de-scattering model network guided by the polarization imaging prior and the diffusion model is:

[0089]

[0090] wherein, , , , are weight parameters of the simplified diffusion loss based on the prior, the pixel loss, the gradient loss and the phase loss respectively. The simplified diffusion loss based on the prior is the L1 norm loss of the clear latent feature of the target extracted by the feature extraction encoder network and the clear latent feature estimation result estimated in the inverse process. The pixel loss is the pixel L1 norm loss of the de-scattered image generated by the reconstruction module network and the intensity image of the clear underwater target. The gradient loss is the gradient L1 norm loss of the de-scattered image generated by the reconstruction module network and the intensity image of the clear underwater target. The phase loss is the L1 norm loss of the phase angle after Fourier transform of the de-scattered image generated by the reconstruction module network and the intensity image of the clear underwater target.

[0091] In combination with the design of the loss function, the loss function of the network guided by the polarization imaging prior and the diffusion model is obtained, and back propagation is performed.

[0092] As an example, the training process is as shown in Figure 2 and Figure 3 , the total iteration of the training in the network framework is 300,000 times, the optimizer is AdamW, the initial learning rate is 0.0002, the learning rate scheduling strategy adopts a periodic annealing decay mode based on a cosine function, and the training process is divided into two stages. The learning rate remains unchanged in the first 100,000 iterations, and gradually decays to 1x10 -6 in the 100,000th to 300,000th iteration according to the cosine function rule, so as to ensure that the network has good convergence stability in the later training process; the diffusion model uses an 8-step linear noise scheduling strategy (the noise level is linearly increased from 0.1 to 0.99); the loss function hyperparameter is set to , , .

[0093] Embodiment Two

[0094] A polarization imaging-based underwater image de-scattering method, comprising:

[0095] Based on the intensity image and the plurality of polarization angle images of the blurred target under the turbid water body, the underwater image despeckling model constructed by the method described in the above embodiment is used to obtain the despeckled target object image, as shown in Figure 4

[0096] In the method of the embodiment, after the network is trained, in the inference mode, the four polarization angle images and the intensity image of the blurred underwater target under the turbid water body are stacked by the feature extraction encoder, and then the diffusion model inverse process part is used. The part first generates noise conforming to the standard Gaussian distribution by Gaussian noise initialization on the extracted features, and the four polarization angle images and the intensity image of the blurred underwater target under the turbid water body are used as the condition guide to generate the clear latent features of the underwater target from the noise step by step, and finally the clear image of the underwater target is reconstructed and generated by inputting the intensity image of the blurred underwater target into the despeckling network.

[0097] The related technical solutions are the same as above, and will not be repeated here.

[0098] Embodiment Three

[0099] The application also relates to an electronic device, which comprises a memory and a processor. The memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0100] The electronic device can be a desktop computer, a notebook computer, a palm computer, a cloud server and the like. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components and the like. The memory can be used to store computer programs and / or modules, and the processor can run or execute the computer programs and / or modules stored in the memory, and call data stored in the memory, so as to realize various functions of the electronic device.

[0101] The related technical solutions are the same as above, and will not be repeated here.

[0102] Embodiment Four

[0103] ​The application also relates to a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the above method.

[0104] Specifically, the memory can include a high-speed random access memory, and can also include a non-volatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash storage device, or other volatile solid-state storage devices.

[0105] The related technical solutions are the same as above, and will not be repeated here.

[0106] Embodiment five

[0107] The embodiment of the application provides a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps of the method of the above embodiment of the application.

[0108] The related technical solutions are the same as above, and will not be repeated here.

[0109] In general, the application provides a method for underwater image despeckling based on polarized imaging. By constructing an underwater image despeckling network framework based on polarized imaging prior and diffusion model guidance, clear potential features of underwater targets are extracted as prior features. Then, under the guidance of the polarized features of the blurred target in turbid water, a diffusion model is used to model the clear potential feature distribution of the underwater target image, to obtain an estimation result of the clear potential feature of the underwater target image for image reconstruction, and then a corresponding clear underwater target image is generated through a reconstruction module. By constructing a diffusion model in a low-dimensional latent space, the difficulty of diffusion modeling is significantly reduced, and the training stability is improved. The polarized imaging prior constraint is introduced, the polarized features of the target are used as the physical constraint condition of the diffusion process, the clear potential feature distribution of the underwater target image is guided and modeled, so that the physical consistency of the modeling process is improved, and the structural consistency and detail fidelity of the reconstruction result are improved. This method can effectively suppress the scattering interference in turbid water and improve the imaging clarity, which is conducive to widening the application of polarized imaging in the fields of ocean exploration, underwater operation and emergency rescue.

[0110] Those skilled in the art can easily understand that the above description is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for constructing an underwater image despeckling model based on polarization imaging, characterized in that, The method comprises the following steps: constructing a training sample set, each training sample comprising an intensity image and a plurality of polarization angle images of an underwater target in clear water and an intensity image and a plurality of polarization angle images of an underwater target in turbid water, wherein the intensity image of the underwater target in clear water is used as a true value label; constructing a feature extraction encoder network for stacking the intensity image and the plurality of polarization angle images of the underwater target in turbid water in each sample by channels, and performing feature extraction on the stacked result and the intensity image of the underwater target in clear water in the sample to obtain clear latent features of the underwater target; at the same time, the stacked result is also subjected to feature extraction to obtain polarization features of the underwater target in turbid water; constructing a diffusion model network, a forward process part of which is used to gradually add Gaussian noise to the clear latent features output by the feature extraction encoder at each time step, and convert the clear latent features into noise data conforming to a standard Gaussian distribution as the time step increases; a reverse process part of which is used to gradually denoise the noise data conforming to the Gaussian distribution based on a denoising network under the guidance of the polarization features output by the feature extraction encoder, to reconstruct the clear latent features of the underwater target to obtain clear latent feature estimation results; constructing a reconstruction module network for reconstructing a clear image of the corresponding underwater target based on the clear latent feature estimation results output by the diffusion model and the intensity image of the underwater target in turbid water in the corresponding sample; using the training sample set to calculate a loss based on the true value label and the clear intensity image of the underwater target reconstructed to train the feature extraction encoder network, the diffusion model network and the reconstruction module network; and training the feature extraction encoder, the reverse process part of the diffusion model and the reconstruction module to obtain a de-scattering model.

2. The method of claim 1, wherein, Each image in each training sample is obtained by sequentially performing image normalization, random cropping and image enhancement on the original image thereof.

3. The method of claim 1, wherein, The feature extraction encoder network comprises: a convolutional layer comprising a cascade of convolutional layers and GELU activation functions for extracting shallow features; a residual convolutional block comprising a plurality of block residual convolutional blocks, each block residual convolutional block comprising a convolutional layer and a corresponding GELU activation function, and each block residual convolutional block being combined by a skip connection to gradually extract local spatial features; an MLP module for global feature modeling to finally generate clear latent features or polarization features.

4. The method of claim 1, wherein, The main framework of the denoising network is a U-net structure.

5. The method of claim 1, wherein, The reconstruction module network comprises: a stacking unit for stacking the clear latent feature estimation results and the intensity image of the underwater target in turbid water in the corresponding sample by channels; an input convolutional mapping layer comprising a convolutional layer and a GELU activation function following the convolutional layer, for performing preliminary mapping on the stacked result output by the stacking unit to obtain image reconstruction features; a residual convolutional block group comprising a plurality of residual convolutional blocks connected in sequence, each residual convolutional block comprising a convolutional layer and a corresponding GELU activation function, and each residual convolutional block being combined by a skip connection to gradually recover image structure information and detail texture. An output reconstruction convolutional layer: it includes a convolutional layer, the number of output channels of which is consistent with the intensity image of the target under clear water, which is used to generate the output underwater target clear intensity image.

6. The method of claim 1, wherein, During training, a loss function for computing loss is represented as: in, , , , These are the weight parameters for the prior-based simplified diffusion loss, pixel loss, gradient loss, and phase loss, respectively; the prior-based simplified diffusion loss represents the clear latent features of the target extracted by the feature extraction encoder network. Clear latent feature estimation results reconstructed during the inverse process The L1 norm loss and pixel loss are used to reconstruct the descattered image generated by the reconstruction module network. Intensity image of a target in clear water The pixel L1 norm loss and gradient loss are generated by the reconstruction module network. Intensity image of a target in clear water The gradient L1 norm loss and phase loss are used to reconstruct the descattered image generated by the reconstruction module network. Intensity image of a target in clear water Fourier transform Back phase angle The L1 norm loss.

7. A method for underwater image despeckling based on polarization imaging, characterized in that, The method comprises the following steps: Based on the intensity image and the plurality of polarization angle images of the blurred target under the turbid water body, the underwater image despeckling model is obtained by using the underwater image despeckling model constructed by the method according to any one of claims 1 to 6. 8.An electronic device comprising a memory and a processor, the memory storing a computer program, wherein, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6 or the steps of the method according to claim 7.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises a stored computer program, wherein when the computer program is executed by a processor, the device where the storage medium is located is controlled to execute the steps of the method according to any one of claims 1 to 6 or the steps of the method according to claim 7.

10. A computer program product comprising computer programs or instructions, characterized in that, The computer program or the instructions are executed by the processor to implement the steps of the method according to any one of claims 1 to 6 or the steps of the method according to claim 7.

Images