Polarization image reconstruction method based on improved DeblurGAN-v2 model

Abstract

The application discloses a polarized image reconstruction method based on an improved DeblurGAN-v2 model, wherein in the generator of the improved DeblurGAN-v2 model, a coordinate attention module is introduced before a convolution layer of a feature extraction backbone network, and a BiFormer attention module is introduced after multi-scale feature splicing of a feature fusion module; a blurred polarized image and a corresponding clear polarized image under the same scene are collected, and the generator of the improved DeblurGAN-v2 model is trained; a blurred polarized image to be reconstructed is acquired and input into the trained generator of the improved DeblurGAN-v2 model, and a clear polarized reconstructed image is output. The application can effectively restore a polarized image under dynamic imaging and a non-uniform blurred scene, avoid distortion of polarized physical information, fully fuse multi-scale features, improve the quality of polarized image reconstruction, and improve the robustness of a polarized imaging system in a dynamic environment.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a polarization image reconstruction method based on an improved DeblurGAN-v2 model. It is suitable for clear reconstruction of blurred polarization images under dynamic imaging, carrier motion or camera shake conditions, and provides stable and reliable polarization image data for tasks such as polarization navigation orientation, target detection, and optical imaging quality improvement. Background Technology

[0002] Polarization imaging can record physical information such as polarization direction and degree of polarization in addition to light intensity, thus possessing significant application value in biomimetic polarization navigation, target detection in complex environments, remote sensing imaging, and underwater or low-visibility scene perception. In nature, organisms such as sand ants and bees can navigate using atmospheric polarization patterns. Based on this, biomimetic polarization navigation technology has been developed that does not rely on satellite signals, is unaffected by electromagnetic interference, and exhibits good autonomy and anti-interference capabilities. When acquiring scene polarization information, polarization cameras or polarization imaging sensors typically need to collect and calculate the light intensity in different polarization directions to obtain the polarization angle, degree of polarization, and related Stokes parameters.

[0003] However, during operation of drones, unmanned vehicles, handheld platforms, or other moving vehicles, polarization cameras are easily affected by platform vibration, rapid turning, attitude changes, and relative motion within the exposure time. This causes the incident light to form a continuous displacement integral on the sensor's imaging plane, resulting in motion blur. Motion blur not only degrades image edges, textures, and high-frequency details but also disrupts the spatial correspondence between polarization parameters, distorting the distribution of polarization angle and degree of polarization. This further affects the accuracy of subsequent directional calculations, feature point extraction, and target recognition.

[0004] Traditional image deblurring methods often rely on explicit degradation models, prior assumptions, or iterative optimization. While effective for uniform blurring or controlled scenes, they struggle to balance spatial detail restoration and polarization physical information preservation when dealing with non-uniform motion blur, complex dynamic scenes, and images with significant changes in polarization characteristics. With the development of deep learning, convolutional neural networks and generative adversarial networks (GANs) have been widely applied to image restoration tasks. DeblurGAN-v2, as a typical end-to-end motion deblurring model, achieves fast and high-quality image restoration through adversarial training between the generator and discriminator.

[0005] However, the original DeblurGAN-v2 model still has shortcomings when directly applied to polarization image reconstruction. On the one hand, parameters such as polarization angle are sensitive to spatial location and directional texture, and ordinary convolutional feature extraction lacks explicit encoding of horizontal and vertical positional information, which can easily lead to polarization edge localization errors. On the other hand, the original model mainly fuses multi-scale features in the feature pyramid structure through stitching, which makes it difficult to fully model the long-range dependencies between large-scale polarization regions, easily resulting in redundant feature accumulation or loss of key polarization correlations. Therefore, it is necessary to propose an improved DeblurGAN-v2 model oriented towards the characteristics of polarization images to enhance the representation of polarization spatial information and the ability to fuse multi-scale global features, thereby improving the robustness and accuracy of polarization image reconstruction in dynamic scenes. Summary of the Invention

[0006] Purpose of the invention: In order to solve the problem of poor polarization image restoration quality caused by motion blur in dynamic polarization imaging in the prior art, the present invention provides a polarization image reconstruction method based on the improved DeblurGAN-v2 model.

[0007] Technical solution: A polarization image reconstruction method based on an improved DeblurGAN-v2 model, comprising the following steps: Step 1: Construct an improved DeblurGAN-v2 model based on the DeblurGAN-v2 model. The improved DeblurGAN-v2 model includes a generator, which sequentially includes a feature extraction backbone network, a feature fusion module, and an image restoration module. The feature extraction backbone network introduces a coordinate attention module before the convolutional layer, and the feature fusion module introduces a BiFormer attention module after multi-scale feature concatenation. Step 2: Collect blurred polarization images and corresponding sharp polarization images in the same scene, perform spatial registration and standardization on the blurred and sharp polarization images to obtain standardized polarization images, use the standardized blurred polarization images as input to the generator, and use the corresponding standardized sharp polarization images as supervision targets, input to the generator of the improved DeblurGAN-v2 model for training; Step 3: Obtain the blurred polarization image to be reconstructed. Preprocess the blurred polarization image and input the preprocessed polarization image feature data into the generator of the trained improved DeblurGAN-v2 model. First, feature extraction is performed. During the feature extraction process, the coordinate attention module is used to jointly encode the spatial position information and channel feature information in the blurred polarization image to obtain the enhanced polarization feature map. Then, feature fusion is performed. During the feature fusion process, the BiFormer attention mechanism is used to dynamically filter and fuse polarization features at different scales and levels. Finally, the image restoration module outputs a clear polarization reconstructed image.

[0008] Further, in step three, during the feature extraction stage, the coordinate attention module receives the polarization feature map output by the preceding network and performs global average pooling on the polarization feature map along the height and width directions respectively to obtain a first feature vector containing horizontal position information and a second feature vector containing vertical position information. The first and second feature vectors are concatenated and input into a shared 1×1 convolutional layer and a nonlinear activation function for joint encoding to obtain intermediate features that fuse spatial coordinate information. Then, horizontal attention weights and vertical attention weights are generated by two parallel 1×1 convolutional layers respectively, and normalized by the Sigmoid function. Finally, the horizontal attention weights and vertical attention weights are multiplied element-wise with the polarization feature map output by the preceding network to obtain the enhanced polarization feature map.

[0009] Furthermore, in step three, during the feature fusion stage, the stitched multi-scale polarization feature map is divided into multiple non-overlapping local regions, and query vectors, key vectors, and value vectors are generated through linear mapping. The features within each local region are aggregated to obtain a region-level feature representation that represents the overall semantics and polarization distribution of the region. Adjacency relationships are constructed based on the correlation between region-level features, and several regions with the highest correlation to the current region are selected as attention calculation objects. Subsequently, a two-layer routing attention mechanism is used to perform feature interaction on the selected local regions to obtain fused features.

[0010] Furthermore, in step two, spatial registration and standardization processing includes: spatially registering the blurred polarized image and the clear polarized image to maintain a correspondence between the blurred polarized image and the clear polarized image in pixel coordinates; uniformly cropping or scaling the registered image to a preset resolution, and normalizing the pixel values ​​to obtain standardized polarized image data.

[0011] Furthermore, in step three, the preprocessing of the blurred polarized image to be reconstructed includes: resizing and pixel normalization of the blurred polarized image to be reconstructed.

[0012] Furthermore, in step one, the improved DeblurGAN-v2 model also includes a discriminator, which employs a dual-scale discrimination mechanism to discriminate the generated results at both the local image patch and overall image levels.

[0013] Compared with existing technologies, the polarization image reconstruction method based on the improved DeblurGAN-v2 model provided by this invention has the following beneficial effects: (1) Based on the DeblurGAN-v2 framework, this invention improves the traditional model structure to meet the needs of image reconstruction under dynamic polarization imaging conditions. It can effectively recover the texture, edge and spatial structure of polarization images in dynamic imaging and non-uniform blur scenes, avoid distortion of polarization physical information, fully integrate multi-scale features, and improve the quality of polarization image reconstruction.

[0014] (2) This invention introduces a coordinate attention module in the generator feature extraction stage, embedding spatial coordinate information into the channel weighting process, so that the network has a stronger position awareness while retaining the channel response capability, thereby enhancing the expression of key polarization edges, textures and blurred areas.

[0015] (3) This invention introduces the BiFormer attention mechanism in the multi-scale feature fusion stage, uses dynamic sparse attention and a two-layer routing mechanism to filter effective regions and model long-range dependencies, reduces information redundancy caused by traditional feature stitching, improves the utilization efficiency of multi-scale polarization features, and thus improves the quality of polarization image restoration.

[0016] (4) The present invention can improve the robustness of polarization imaging system in dynamic environment, and is especially suitable for clear reconstruction of blurred polarization images under dynamic imaging, carrier movement or camera shake conditions, providing clearer and more stable and reliable input images for subsequent polarization angle calculation, polarization navigation orientation, target detection and visual perception of complex scenes. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the structure of the improved DeblurGAN-v2 model proposed in this invention; Figure 2 This is a schematic diagram of the coordinate attention module structure introduced in this invention; Figure 3 This is a schematic diagram of the overall structure of the BiFormer attention module introduced in this invention; Figure 4 This is a schematic diagram of the structure of the dual-layer routing attention module introduced in this invention; Figure 5 This is a comparison chart showing the effects of the present invention on polarization image reconstruction compared to other traditional algorithms. Detailed Implementation

[0018] The present invention will be further explained and described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are only for illustrating the present invention and are not intended to limit the scope of protection of the present invention; those skilled in the art can make equivalent substitutions or improvements to the relevant structures, parameters or training methods without departing from the spirit and principles of the present invention.

[0019] This embodiment addresses the image reconstruction requirements under dynamic polarization imaging conditions, utilizing an improved DeblurGAN-v2 model to achieve end-to-end mapping from blurred polarized images to sharp polarized images. Due to the influence of motion blur, the spatial consistency of physical quantities such as Stokes parameters, polarization angle, and degree of polarization in polarized images is disrupted. Conventional deblurring networks lack specialized modeling of the spatial location and global dependencies of polarization features, easily leading to polarization edge artifacts and distortion in large-area polarization regions. Therefore, this embodiment introduces a coordinate attention module and a BiFormer attention mechanism into the original DeblurGAN-v2 generator, enabling the model to more fully utilize the spatial orientation information, multi-scale texture information, and long-range related information of the polarized image. The polarization image reconstruction method based on the improved DeblurGAN-v2 model includes the following steps: Step 1: Build an improved DeblurGAN-v2 model based on the DeblurGAN-v2 model, such as... Figure 1 As shown, the improved DeblurGAN-v2 model includes a generator, which sequentially comprises a feature extraction backbone network, a feature fusion module, and an image restoration module. The feature extraction backbone network extracts shallow texture, edge, and channel features from the input blurred polarized image. The feature fusion module fuses feature maps of different scales based on a feature pyramid structure to balance high-level semantic information and low-level structural information. The image restoration module gradually restores the spatial resolution of the image and outputs a clear polarized image through convolutional layers, upsampling layers, and skip connections. The feature extraction backbone network has a coordinate attention module connected in series before five 1×1 convolutional layers, and the feature fusion module introduces a BiFormer attention module after multi-scale feature concatenation. Step 2: Acquire blurred polarized images and their corresponding sharp polarized images from the same scene to construct sharp-blurred polarized image pairs as paired samples. Perform spatial registration and normalization on the blurred and sharp polarized images. First, spatially register the blurred and sharp polarized images to maintain a correspondence in pixel coordinates. Then, uniformly crop or scale the registered images to a preset resolution, such as 512×512, and normalize the pixel values ​​to obtain standardized polarized image data. Use the standardized blurred polarized image as input to the generator, and the corresponding standardized sharp polarized image as the supervision target. Input these into the generator of the improved DeblurGAN-v2 model for training to obtain the trained improved DeblurGAN-v2 model generator. Through the above image normalization preprocessing, the impact of image size differences, brightness variations, and inconsistent data distribution on model training and reconstruction results can be reduced, improving the stability of polarization image reconstruction.

[0020] Step 3: Obtain the blurred polarization image to be reconstructed. Preprocess the blurred polarization image to be reconstructed, including resizing and pixel normalization, to meet the input requirements of the improved DeblurGAN-v2 model. Input the preprocessed polarization image feature data into the generator of the trained improved DeblurGAN-v2 model. First, feature extraction is performed. During the feature extraction process, the coordinate attention module is used to jointly encode the spatial position information and channel feature information in the blurred polarization image to obtain the enhanced polarization feature map. Then, feature fusion is performed. During the feature fusion process, the BiFormer attention mechanism is used to dynamically filter and fuse polarization features at different scales and levels. Finally, the image restoration module outputs a clear polarization reconstructed image.

[0021] In the feature extraction stage, such as Figure 2 As shown, a coordinate attention module is introduced before multiple 1×1 convolutional layers of the generator's feature extraction backbone network. Let the size of the polarization feature map input to the coordinate attention module be H×W×C, where H, W, and C represent the feature map height, width, and number of channels, respectively. The coordinate attention module receives the polarization feature map output from the preceding network, performs global average pooling along the height direction to obtain a first feature vector reflecting horizontal positional information, and simultaneously performs global average pooling along the width direction to obtain a second feature vector reflecting vertical positional information. Unlike traditional channel attention, which only compresses the spatial dimension, the coordinate attention module can preserve the positional information of the feature response in both spatial directions. Subsequently, the first and second feature vectors are concatenated and input into a shared 1×1 convolutional layer and a nonlinear activation function for joint encoding, resulting in intermediate features that fuse spatial coordinate information and channel dependencies. These intermediate features are then passed through two parallel 1×1 convolutional layers to generate horizontal and vertical attention weights, which are then normalized using the Sigmoid function. Finally, the horizontal and vertical attention weights are element-wise multiplied with the polarization feature map output from the preceding network to obtain the enhanced polarization feature map. This step enables the model to simultaneously preserve spatial orientation information and channel dependencies when extracting texture, edge, and blurred region features from polarized images, thereby enhancing the model's ability to express the distribution patterns of edges, textures, blurred regions, and polarization features in polarized images, reducing high-frequency polarization texture aliasing caused by motion blur, and improving the model's ability to perceive polarization physical information.

[0022] In the feature fusion stage, such as Figure 3As shown, a BiFormer attention module is introduced after the four feature concatenation layers of the generator's feature pyramid structure. Since the feature pyramid structure concatenates features across different levels, traditional concatenation methods can easily lead to simple superposition of multi-scale features without effective filtering. The BiFormer attention mechanism first divides the concatenated multi-scale polarization feature map into multiple non-overlapping local regions and generates query vectors, key vectors, and value vectors through linear mapping. Then, it aggregates the features within each local region to obtain a region-level feature representation that represents the overall semantics and polarization distribution of that region.

[0023] In the region-level modeling process, the BiFormer attention mechanism constructs adjacency relationships based on the correlation between different region-level features, and selects several regions with the highest correlation to the current region as attention calculation objects. Through methods such as... Figure 4 The illustrated dual-layer routing attention mechanism allows the model to maintain detail recovery within a local window while simultaneously filtering effective polarization regions with long-range correlations to the current region globally. The network then calculates attention scores using collected key and value vectors and performs feature interactions, outputting a fused feature that combines local detail preservation with global dependency modeling. This mechanism avoids excessive involvement of irrelevant features and enhances the consistent representation of large-area polarization regions.

[0024] This step leverages its dynamic sparse attention and two-layer routing characteristics to model regional correlations and filter long-range dependencies in the stitched multi-source features, reducing redundant and ineffective features and strengthening the complementary relationship between key polarization features and blurred / degraded features. This results in a more detailed and structurally clearer polarization reconstruction image after restoration. By introducing the BiFormer attention mechanism after feature stitching, the problem of redundant information accumulation caused by the simple feature superposition in traditional stitching operations can be avoided, improving the utilization efficiency of multi-scale polarization features and enhancing the model's reconstruction robustness in non-uniform blur and dynamic imaging scenarios.

[0025] In the image restoration stage, the fused features enhanced by the BiFormer attention mechanism are fed into the generator's image restoration module. The image restoration module gradually restores the image's spatial resolution through upsampling layers and combines shallow structural information with deep semantic information using convolutional layers and skip connections, ultimately outputting a clear polarization reconstructed image. Because the coordinate attention module enhances the spatial localization capabilities of polarization textures and edges, and the BiFormer attention mechanism strengthens the dynamic fusion capabilities of cross-scale and cross-regional features, their synergistic effect enables the generator to more accurately restore polarization images damaged under motion blur conditions.

[0026] The improved DeblurGAN-v2 model may also include a discriminator, such as... Figure 1As shown on the right, the discriminator adopts a dual-scale discrimination mechanism, which discriminates the generated results from the local image patch level and the overall image level, thereby helping the generator to obtain a more realistic and detailed restored image.

[0027] In one specific embodiment, the experimental platform used a 64-bit Windows 11 operating system, an Intel Gold 6248R CPU @ 3.00GHz, an NVIDIA GeForce RTX 3090 GPU, Python 3.8 as the programming language, and PyTorch 2.1.0 and CUDA 11.8 as the deep learning framework. A self-built polarization image dataset was used for model training, comprising 2800 clear-blurred image pairs, with the image resolution uniformly converted to 512×512. Both the generator and discriminator used the Adam optimizer with an initial learning rate of 2×10^-4, a batch size of 16, and a total of 20000 iterations.

[0028] To verify the effectiveness of this embodiment, ablation and comparison experiments were conducted under consistent configuration and training parameters. The ablation experiments show that adding either a coordinate attention module or the BiFormer attention mechanism alone to the traditional DeblurGAN-v2 model can improve polarization image reconstruction results; when both are used together, the model achieves even more significant improvements in PSNR and SSIM metrics. Further comparison with the Richardson-Lucy algorithm and MSCNN algorithm shows that this method achieves a PSNR of 20.43 dB and an SSIM of 0.8223 on both synthetic and real polarization datasets, effectively improving the blurring and degradation of polarization images and enhancing the structural clarity and polarization feature preservation of the reconstructed images.

[0029] like Figure 5 The image shows a comparison of polarization image reconstruction results using this method and other methods. From top to bottom, the images show the results of polarization image reconstruction using the traditional Richardson-Lucy algorithm, the MSCNN algorithm, and this method. Visually, the traditional Richardson-Lucy algorithm tends to produce significant noise and feature point divergence in noisy or dynamically polarized images. While the MSCNN algorithm can improve image smoothness to some extent, it still suffers from insufficient edge localization accuracy and a wide feature distribution bandwidth. The polarization image reconstructed using this method has clearer texture edges and a more concentrated distribution of polarization feature points, providing higher-quality input data for subsequent tasks such as polarization angle calculation, polarization navigation orientation, or target recognition.

[0030] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Any modifications, equivalent substitutions, improvements, or parameter adjustments made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A polarization image reconstruction method based on an improved DeblurGAN-v2 model, characterized in that, Includes the following steps: Step 1: Construct an improved DeblurGAN-v2 model based on the DeblurGAN-v2 model. The improved DeblurGAN-v2 model includes a generator, which sequentially includes a feature extraction backbone network, a feature fusion module, and an image restoration module. The feature extraction backbone network introduces a coordinate attention module before the convolutional layer, and the feature fusion module introduces a BiFormer attention module after multi-scale feature concatenation. Step 2: Collect blurred polarization images and corresponding sharp polarization images in the same scene, perform spatial registration and standardization on the blurred and sharp polarization images to obtain standardized polarization images, use the standardized blurred polarization images as input to the generator, and use the corresponding standardized sharp polarization images as supervision targets, input to the generator of the improved DeblurGAN-v2 model for training; Step 3: Obtain the blurred polarization image to be reconstructed. Preprocess the blurred polarization image and input the preprocessed polarization image feature data into the generator of the trained improved DeblurGAN-v2 model. First, feature extraction is performed. During the feature extraction process, the coordinate attention module is used to jointly encode the spatial position information and channel feature information in the blurred polarization image to obtain the enhanced polarization feature map. Then, feature fusion is performed. During the feature fusion process, the BiFormer attention mechanism is used to dynamically filter and fuse polarization features at different scales and levels. Finally, the image restoration module outputs a clear polarization reconstructed image.

2. The polarization image reconstruction method based on the improved DeblurGAN-v2 model according to claim 1, characterized in that, In step three, during the feature extraction stage, the coordinate attention module receives the polarization feature map output by the preceding network and performs global average pooling on the polarization feature map along the height and width directions respectively to obtain a first feature vector containing horizontal position information and a second feature vector containing vertical position information. The first and second feature vectors are concatenated and input into a shared 1×1 convolutional layer and a nonlinear activation function for joint encoding to obtain intermediate features that fuse spatial coordinate information. Then, horizontal and vertical attention weights are generated by two parallel 1×1 convolutional layers, and normalized by the Sigmoid function. Finally, the horizontal and vertical attention weights are multiplied element-wise with the polarization feature map output by the preceding network to obtain the enhanced polarization feature map.

3. The polarization image reconstruction method based on the improved DeblurGAN-v2 model according to claim 1 or 2, characterized in that, In step three, during the feature fusion stage, the stitched multi-scale polarization feature map is divided into multiple non-overlapping local regions, and query vectors, key vectors, and value vectors are generated through linear mapping. The features within each local region are aggregated to obtain a region-level feature representation that represents the overall semantics and polarization distribution of the region. Adjacency relationships are constructed based on the correlation between region-level features, and several regions with the highest correlation to the current region are selected as attention calculation objects. Subsequently, a two-layer routing attention mechanism is used to perform feature interaction on the selected local regions to obtain fused features.

4. The polarization image reconstruction method based on the improved DeblurGAN-v2 model according to claim 1 or 2, characterized in that, In step two, spatial registration and standardization processing includes: spatially registering the blurred polarized image and the clear polarized image to maintain a correspondence between the blurred polarized image and the clear polarized image in pixel coordinates; uniformly cropping or scaling the registered image to a preset resolution, and normalizing the pixel values ​​to obtain standardized polarized image data.

5. The polarization image reconstruction method based on the improved DeblurGAN-v2 model according to claim 1 or 2, characterized in that, Step 3 involves preprocessing the blurred polarized image to be reconstructed, including resizing and pixel normalization.

6. The polarization image reconstruction method based on the improved DeblurGAN-v2 model according to claim 1 or 2, characterized in that, In step one, the improved DeblurGAN-v2 model also includes a discriminator, which adopts a dual-scale discrimination mechanism to discriminate the generated results from the perspectives of local image patches and the overall image.

Images