A Denoising Method for Color Polarized Images Based on 3D Convolutional Neural Networks

Abstract

This invention discloses a denoising method for color polarized images based on a three-dimensional convolutional neural network. The method includes: preprocessing a dataset of high-noise and low-noise color polarized images acquired by a color polarization camera; designing a three-dimensional convolutional neural network structure based on a residual dense architecture, where the network input is a low signal-to-noise ratio (SNR) color polarization image in five-dimensional tensor format, and the output is a high SNR five-dimensional tensor color polarization image; and designing a color polarization loss function based on physical constraints. This method utilizes the high-dimensionality of three-dimensional convolution, enabling simultaneous collaborative feature extraction and fusion across four dimensions: pixel length, pixel width, polarization, and color channels. It not only efficiently removes noise from color polarized images and accurately recovers the polarization information of the target from noisy images, but also corrects color cast and color difference issues in color images, thus improving the imaging quality of color polarization imaging technology in real-world noisy environments.

Description

Technical Field

[0001] This invention relates to the field of polarization imaging technology, and in particular to a deep learning-based method for denoising color polarized images based on a three-dimensional convolutional neural network. Background Technology

[0002] Color polarization imaging technology combines spectral and polarization characteristics, enabling the acquisition of more refined and richer red, green, and blue three-band polarization information than grayscale polarization images. Color polarization imaging is sensitive to differences in the spectral and polarization characteristics of targets and backgrounds. By comprehensively utilizing polarization information and its differences at different wavelengths for detection and identification, the target detection and identification capabilities of photoelectric imaging systems will be significantly improved. Therefore, color polarization imaging has potential application value in detecting stealth, camouflage, and false targets in harsh environments such as fog, smoke, and complex backgrounds. However, in practical applications, noise can significantly interfere with color polarization imaging, causing polarization information to be submerged in noise. Furthermore, noise can cause color distortion, thus affecting the imaging effect. Therefore, an effective method for denoising color polarization images is needed.

[0003] Existing image denoising techniques are mainly divided into two categories: data-driven and non-data-driven. In terms of denoising effect, the optimal non-data-driven method is the 3D similarity block matching method based on color space transformation. However, polarized images have physical correlations between pixels. If ordinary digital image denoising methods are directly applied to polarized images, polarization information cannot be fully utilized. Furthermore, non-data-driven methods are also ineffective for low signal-to-noise ratio images, and many parameters of these denoising algorithms require prior knowledge to determine, making it difficult to obtain the optimal solution. Data-driven deep learning methods outperform other methods in denoising performance, but these methods rely heavily on the dataset. Currently, synthetic images with Gaussian noise artificially added to noise-free images are commonly used as datasets. However, real-world noise distribution is more complex, and simulating real noise solely with Gaussian noise limits the algorithm's effectiveness in real-world noise scenarios. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and solve the problem that it is difficult to extract color polarization information from color polarized images due to noise, and to provide a denoising method for color polarized images based on a three-dimensional convolutional neural network.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A denoising method for color polarized images based on a three-dimensional convolutional neural network includes the following steps:

[0007] S1. Establish a dataset consisting of four-dimensional color polarization images. The data samples in the dataset are all captured by a color polarization camera in a real environment. The four-dimensional color polarization images with different signal-to-noise ratios paired in the dataset are obtained by adjusting the camera's exposure time and gain.

[0008] S2. Preprocess the dataset. First, use data augmentation methods to augment the dataset. Then, convert the four-dimensional color polarization image into a five-dimensional tensor format. Finally, randomly divide the dataset into training set, validation set, and test set according to the specified ratio to train, debug, and test the performance of the three-dimensional convolutional neural network.

[0009] S3. Construct a three-dimensional convolutional neural network based on a residual dense architecture; the three-dimensional convolutional neural network includes a shallow feature extraction module, a dense feature fusion module, and a deep feature fusion module, and the convolutional layer in each module is a three-dimensional convolutional layer, which can simultaneously extract information on pixel height, pixel width, polarization dimension, and color dimension to obtain a three-dimensional feature map;

[0010] S4. Input the dataset into the three-dimensional convolutional neural network for training, and use a color polarization loss function based on polarization information constraints for optimization, so as to guide the three-dimensional convolutional neural network to learn pixel information, polarization information and color information in a balanced way;

[0011] S5. Test the test set using the trained and debugged 3D convolutional neural network. Obtain the noise-removed color polarization image in 5D tensor format from the output layer of the 3D convolutional neural network. Utilize the polarization characteristics between the color polarization images to reduce the dimensionality of the 5D tensor format color polarization image, and obtain the color light intensity image, color polarization degree image, and color polarization angle image.

[0012] Furthermore, in step S1, the four-dimensional color polarized images in the dataset include four dimensions: pixel height, pixel width, color, and polarization, represented in order as H×W×(R,G,B)×(0°,45°,90°,135°). The four-dimensional color polarized images in the dataset are paired according to high signal-to-noise ratio (SNR) and low SNR. The paired four-dimensional color polarized images with different SNRs are obtained by the camera in a real environment, including indoor and outdoor non-digital environments. By setting the camera to a long exposure time and low gain, a high SNR image can be obtained, and by setting the camera to a short exposure time and high gain, a low SNR image can be obtained.

[0013] Furthermore, in step S2, the dataset is augmented using cropping, rotation, and flipping methods. The five dimensions in the five-dimensional tensor format are: batch dimension, polarization dimension, image height, image width, and color dimension. The batch dimension represents the size of the batch data processing, the polarization dimension represents different polarization angle components, the image height represents the image pixel height, the image width represents the image pixel width, and the color dimension represents the feature map channels.

[0014] Furthermore, in step S3, the shallow feature extraction module uses sequentially arranged three-dimensional convolutional layers to extract shallow polarization features; the dense feature fusion module uses sequentially stacked residual dense modules to fuse features. This module mainly consists of residual connections and dense connections of three-dimensional convolutional layers, and also includes three-dimensional merging layers, three-dimensional bottleneck layers, and local residual connections; the deep feature fusion module uses three-dimensional convolutional layers, three-dimensional merging layers, three-dimensional bottleneck layers, and global residual connections to perform deep feature fusion.

[0015] Furthermore, in step S4, the loss function includes light intensity loss, color loss, and polarization loss, which respectively constrain the light intensity, color, and polarization information in the training of the 3D convolutional neural network. The light intensity loss calculates the mean square error between the output image and the ground truth image of the 3D convolutional neural network; the color loss calculates the mean square error between the light intensity components of the output image and the ground truth image of the 3D convolutional neural network; and the polarization loss calculates the mean square error of the polarization angle and polarization degree components between the output image and the ground truth image of the 3D convolutional neural network. The proportions of each loss function are dynamically adjusted as the training process progresses through adaptive weights.

[0016] Furthermore, the convolution kernel of the three-dimensional convolutional layer is a cube with a shape of 3×3×3 pixels. During convolution, the convolution kernel is used to convolve the five-dimensional tensor format data in the pixel height, pixel width and polarization dimension, and then calculate and generate a three-dimensional feature map. Finally, the feature map is placed in the color dimension; each three-dimensional convolutional layer is followed by an activation function layer.

[0017] The shallow feature extraction module includes two three-dimensional convolutional layers. The first convolutional layer has a kernel size of 3×3×3 and 64 kernel channels. The second convolutional layer has a kernel size of 3×3×3 and 32 kernel channels. There is a global residual connection between the first and second three-dimensional convolutional layers that is directly connected to the output layer of the three-dimensional convolutional neural network. This connection is used to optimize the network training process and prevent gradient explosion.

[0018] The dense feature fusion module is configured with ten 3D convolutional residual dense modules. Each 3D convolutional residual dense module consists of six 3D convolutional layers, one 3D merging layer, and one 3D bottleneck layer. Each 3D convolutional layer can generate a 3D feature map, and the 3D convolutional layers are densely connected. The 3D convolutional residual dense modules are locally residual connected.

[0019] In the deep feature fusion module, the 3D merging layer acquires all the output feature maps of the dense feature fusion module and merges them in the color dimension; the 3D bottleneck layer can shrink the number of channels in the color dimension and fuse the 3D feature maps obtained by the 3D merging layer; the 3D convolutional layer of the deep feature fusion module further extracts features from the fused result to obtain the final output.

[0020] The present invention also provides a denoising device for color polarized images based on a three-dimensional convolutional neural network, comprising:

[0021] The data acquisition unit is used to acquire four-dimensional color polarization images through a camera to construct a dataset;

[0022] The preprocessing unit is used to preprocess the dataset and convert the images in the dataset into a five-dimensional tensor format;

[0023] Neural network units are used to obtain denoised output images.

[0024] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the denoising method for a color polarization image based on a three-dimensional convolutional neural network.

[0025] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the denoising method for color polarized images based on a three-dimensional convolutional neural network.

[0026] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0027] 1. This invention discloses a color polarization image denoising method based on a three-dimensional convolutional neural network, mainly based on a residual dense network structure to build a three-dimensional convolutional neural network. Three-dimensional convolution can simultaneously perform convolution operations on pixel height, pixel width, polarization dimension, and color dimension to extract three-dimensional feature maps and collaboratively process the relationships between multi-source features. Therefore, the network can not only further improve the image denoising effect but also restore polarization information and correct distorted color information.

[0028] 2. This invention discloses a loss function for a three-dimensional convolutional neural network suitable for color polarized images. It mainly employs intensity loss, polarization loss, and color loss, and uses an automatic adjustment mechanism to allocate weights for each component during training. Intensity loss provides basic constraints on the image, enabling effective training convergence; polarization loss effectively constrains the network's learning of polarization information; and color loss effectively constrains the network's learning of color information. Furthermore, the automatic adjustment mechanism automatically allocates weights, effectively resolving the coupling problem among multiple loss functions.

[0029] 3. This method utilizes the high-dimensionality of three-dimensional convolution, enabling simultaneous collaborative feature extraction and fusion of four dimensions: pixel height, pixel width, polarization, and color channel. It can not only efficiently remove noise from color polarized images and accurately recover the polarization information of the target from noisy images, but also correct color cast and color difference problems in color images. This is beneficial to improving the imaging quality of color polarization imaging technology in actual noisy environments and expanding its application scenarios and scope in low signal-to-noise ratio environments.

[0030] 4. This invention discloses a three-dimensional convolutional neural network based on a residual dense architecture, which effectively integrates three-dimensional convolution and residual dense modules. It can not only use three-dimensional convolution to extract features from high-dimensional images, but also use the residual dense architecture to efficiently fuse image features, thereby facilitating the extraction of effective information from noisy images and achieving better noise removal effects for color polarized images. Attached Figure Description

[0031] Figure 1 This is a flowchart of a color polarization image denoising method based on a three-dimensional convolutional neural network according to an embodiment of the present invention;

[0032] Figure 2 This is a schematic diagram illustrating the image shape transformation when a dataset is converted into a five-dimensional tensor according to an embodiment of the present invention.

[0033] Figure 3 A schematic diagram illustrating the structure of a three-dimensional convolutional neural network based on a residual dense architecture according to an embodiment of the present invention;

[0034] Figures 4a to 4i This is a schematic diagram illustrating the denoising and multi-source information restoration effect of a color polarized image according to an embodiment of the present invention. Detailed Implementation

[0035] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The present invention is applicable to different color polarization imaging systems (amplitude division, focal plane division, etc.) and different types of polarization information imaging methods (Stokes, Mueller, etc.). The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0036] The following describes a color polarization image denoising method based on a three-dimensional convolutional neural network according to an embodiment of the present invention, with reference to the accompanying drawings. Since color focal plane polarization image is a type of color polarization image, this embodiment takes the Stokes vector imaging method based on color focal plane polarization image as an example.

[0037] Figure 1 This is a flowchart of a color focal plane polarization image denoising method based on a three-dimensional convolutional neural network provided by the present invention. The image denoising method in this embodiment includes:

[0038] S1. Constructing a color polarization image dataset: First, set the color focal plane camera to high-gain mode and use a short exposure time. The images acquired under these conditions have extremely low signal-to-noise ratios (SNR) and serve as the noisy images input to the 3D convolutional neural network. Then, set the camera to low-gain mode and use a long exposure time. The images acquired under these conditions have low SNR and less noise. Finally, repeatedly acquire the high SNR images and calculate their average value to obtain images with minimal noise, which serve as the ground truth images for training. The ground truth image and the noisy image constitute an image pair. It is important to note that when acquiring the noisy image and the ground truth image, it is necessary to control the variation of ISO and exposure time and their complementary relationship to ensure that the light intensity values ​​of the image pair remain basically consistent.

[0039] S2, Preprocessing of the color polarized image dataset: First, for each pair of high- and low signal-to-noise ratio images, calculate their average pixel brightness. Then, calibrate the brightness of the two images based on their average pixel brightness to ensure they are the same. Next, perform data augmentation on the color focal plane images by cropping them into 64×64 image blocks and randomly flipping and rotating them. Finally, perform superpixel splitting and dimensionality transformation on the color focal plane images, such as... Figure 2As shown, specifically, for a color focal plane image with pixel size W×H, the RGB color superpixels are first split, and the image shape becomes W / 2×H / 2×3, where the last dimension 3 represents the three color channels R, G, and B. Then, the polarization superpixels of each color channel are split, and the image shape becomes 4×W / 4×H / 4×3, where the first dimension 4 represents the four polarization directions θ = 0°, 45°, 90°, and 135°. Finally, a batch dimension is added to represent the batch size in deep learning, thus constructing a five-dimensional tensor in the format {batch, polarization, pixel width, pixel height, channel}. Finally, the dataset is randomly split into training, validation, and test sets in a fixed ratio of 8:1:1, and the dataset preprocessing is complete.

[0040] Suppose that in each channel of the input image in the dataset, the noisy image is... The noise-free image is I θ The noise is n θ Then all three conditions are met:

[0041]

[0042] Where θ = 0°, 45°, 90°, 135°.

[0043] S3, Construct a 3D convolutional neural network based on a residual-dense structure, such as Figure 3 As shown, the three-dimensional convolutional neural network is mainly composed of multiple sequentially arranged three-dimensional convolutional residual dense modules.

[0044] 3.1 Configure the 3D convolutional layer. The convolution kernel of the 3D convolutional layer is a 3×3×3 pixel cube. During convolution, the kernel convolves the five-dimensional tensor data in the pixel height, pixel width, and polarization dimensions, then calculates and generates a 3D feature map, which is finally placed in the channel dimension. Additionally, the 3D convolutional layer is followed by the Leaky-ReLU activation function.

[0045] 3.2 Configure the 3D shallow feature extraction module. Specifically, the 3D shallow feature extraction module includes two 3D convolutional layers. The first convolutional layer has a kernel size of 3×3×3 and 64 channels. The second convolutional layer has a kernel size of 3×3×3 and 32 channels. In addition, there is a branch directly connecting the first and second 3D convolutional layers to the output layer. This branch is a global residual connection used to optimize the network training process and prevent gradient explosion.

[0046] 3.3 Configure the 3D dense feature fusion module. Specifically, in this embodiment, the module is configured with a total of 10 3D convolutional residual dense modules. Specifically, each 3D convolutional residual dense module consists of six 3D convolutional layers, six Leaky-ReLU activation function layers accompanying the six 3D convolutional layers, a 3D merging layer, a 3D bottleneck layer, and local residual connections. Specifically, each 3D convolutional layer generates a 3D feature map. Taking one 3D convolutional layer as an example, denoted as layer i (0 < i ≤ 6), and the subsequent convolutional layers as layers i+n (i < n ≤ 4), the feature map of layer i will be directly transmitted and merged into all subsequent convolutional layers i+n, and combined with the feature map generated by layer i+n. Figure 1 The input is fed into the i+n+1th 3D convolutional layer; this connection method is dense connection. Specifically, the input feature map of the 3D convolutional residual dense module is directly transmitted to the last layer of the module, and then added to the output feature map as the output of the entire 3D convolutional residual dense module; this connection method is local residual connection.

[0047] 3.3 Configure the 3D deep feature fusion module. Specifically, the feature fusion module includes a 3D merging layer, a 3D bottleneck layer, and a 3D convolutional layer. First, the 3D merging layer obtains all output feature maps from 10 3D convolutional residual dense modules, and then merges the feature maps in the color dimension; the 3D bottleneck layer can shrink the number of channels in the color dimension, fusing the large number of 3D feature maps obtained by the 3D merging layer; the 3D convolutional layer further extracts features from the fused result to obtain the final output.

[0048] 3.4 Configure the global residual connection structure. The global residual connection structure is a method that directly inputs the output feature map of the first 3D convolutional layer to the output feature map of the last 3D convolutional layer, and adds the two feature maps to obtain the final output result. This strategy can speed up training and improve the ability of 3D convolutional neural networks to remove image noise.

[0049] S4. Design a color polarization loss function. First, recover the light intensity information through light intensity loss and impose basic constraints on the training of the three-dimensional convolutional neural network. Then, recover the polarization information through polarization degree and polarization angle loss and constrain the three-dimensional convolutional neural network to learn polarization information during training.

[0050] Specifically, the pixel loss function L pixel The definition is as follows:

[0051]

[0052] Where N is the number of training samples. This represents a training pair consisting of a noisy image and a noise-free image. The image represents the denoised image output by the 3D convolutional neural network. θ = 0°, 45°, 90°, 135°, representing the polarization components at the four angles. ||·||2 represents the 2-norm.

[0053] Specifically, in one embodiment of the invention, polarization information is used to construct a loss function, including linear polarization degree (DoLP) and polarization angle (AoP). For polarization information, the Stokes vector S is typically used. T The polarization components are represented by a formula [S0, S1, S2, S3], where S0 represents the intensity component, S1 and S2 represent the linear polarization components, and S3 represents the circular polarization component, which is generally approximated as 0. Furthermore, DoLP and AoP can be calculated using Stokes vectors.

[0054]

[0055]

[0056] Where I θ θ = 0°, 45°, 90°, 135° represents the light intensity values ​​of images with different polarization components.

[0057] Furthermore, in one embodiment of the present invention, the polarization degree loss L DoLP and polarization angle loss L AoP The definition is as follows:

[0058]

[0059] DoLP(·) and AoP(·) represent the operators for linear polarization degree and polarization angle, respectively.

[0060] Furthermore, the polarization loss function is organically composed of polarization degree loss and polarization angle loss:

[0061] L polarization =ω DoLP L DoLP +φω AoP L AoP (6)

[0062] Where φ is the polarization angle adjustment factor, used to balance the weights of the loss function. The weights ω of each sub-function change with the loss function and are calculated as the ratio of the sub-loss function value to the total loss function value.

[0063] Specifically, in one embodiment of the present invention, a color loss function is used to constrain the network, enabling the network to correct color distortion. The color loss is obtained by calculating the mean square error of the S0 component, and is defined as follows:

[0064]

[0065] Furthermore, the color loss function L color The definition is as follows:

[0066]

[0067] S0(·) represents the operator for calculating the first component of the Stokes vector.

[0068] Specifically, the color polarization loss function consists of three parts: intensity loss, polarization loss, and color loss, and is defined as follows:

[0069] L=ω pixel L pixel +ω polarization L polarization +ω color L color (3)

[0070] Where ω is the weight of each sub-function, calculated as the ratio of the value of the sub-loss function to the total loss function. Therefore, during training, as the value of the loss function changes, the weights are constantly adjusted to automatically balance the proportions of each part and optimize the training process.

[0071] S5, in this embodiment of the invention, the parameters of the three-dimensional convolutional neural network model are manually adjusted according to the requirements of noise removal for color polarization images and the feedback of the effect of polarization information and color information restoration. These parameters include the initial learning rate and exponential decay rate of the learning rate, batch size, loss function adjustment factor φ, the number of three-dimensional convolutional residual dense modules, and the number of channels. Specifically, in this embodiment, the minimum batch size is 32 samples, the learning rate is initialized to 0.0001, an exponential decay learning rate method is used with a decay rate of 0.9, the training period is 70, the Adam algorithm is used to optimize the loss function with a loss function adjustment factor of 0.2, and there are a total of 10 three-dimensional convolutional residual dense modules, each module having 6 three-dimensional convolutional layers, and each convolutional layer having 32 channels.

[0072] In an embodiment of the present invention, a three-dimensional convolutional residual dense network is used to denoise a color polarization image, resulting in a reconstructed color image based on polarization information, a color linear polarization degree map, and a color polarization angle map. As shown in Figure 4, ... Figure 4a This is a color map of the noise. Figure 4b This is a noise-based color linear polarization map. Figure 4c This is a color polarization angle diagram for noise. Figure 4d e and f are respectively the color image, color polarization degree image and color polarization angle image reconstructed after denoising using the embodiments of the present invention; Figure 4gh and i represent the color image, color polarization degree image, and color polarization angle image reconstructed from the ground truth image, respectively. As can be seen from the figure, the color polarization image denoising algorithm based on a three-dimensional convolutional neural network proposed in this invention can effectively remove noise from images acquired by a color focal plane camera. Furthermore, for polarization degree and polarization angle images, which are extremely sensitive to noise, it can effectively recover polarization information from the noise-overwhelmed information, and it can also effectively correct color distortion caused by noise.

[0073] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0074] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for denoising color polarized images based on a three-dimensional convolutional neural network, characterized in that, Includes the following steps: S1. Establish a dataset consisting of four-dimensional color polarization images. The data samples in the dataset are all captured by a color polarization camera in a real environment. The four-dimensional color polarization images with different signal-to-noise ratios paired in the dataset are obtained by adjusting the camera's exposure time and gain. S2. Preprocess the dataset. First, use data augmentation methods to augment the dataset. Then, convert the four-dimensional color polarization image into a five-dimensional tensor format. Finally, randomly divide the dataset into training set, validation set, and test set according to the specified ratio to train, debug, and test the performance of the three-dimensional convolutional neural network. S3. Construct a three-dimensional convolutional neural network based on a residual dense architecture; the three-dimensional convolutional neural network includes a shallow feature extraction module, a dense feature fusion module, and a deep feature fusion module, and the convolutional layer in each module is a three-dimensional convolutional layer, which can simultaneously extract information on pixel height, pixel width, polarization dimension, and color dimension to obtain a three-dimensional feature map; S4. Input the dataset into the three-dimensional convolutional neural network for training, and use a color polarization loss function based on polarization information constraints for optimization, so as to guide the three-dimensional convolutional neural network to learn pixel information, polarization information and color information in a balanced way; S5. Test the test set using the trained and debugged 3D convolutional neural network. Obtain the noise-removed color polarization image in 5D tensor format from the output layer of the 3D convolutional neural network. Utilize the polarization characteristics between the color polarization images to reduce the dimensionality of the 5D tensor format color polarization image, and obtain the color light intensity image, color polarization degree image, and color polarization angle image.

2. The denoising method for color polarized images based on a three-dimensional convolutional neural network according to claim 1, characterized in that, In step S1, the four-dimensional color polarized images in the dataset include four dimensions: pixel height, pixel width, color, and polarization, represented in order as H×W×(R,G,B)×(0°,45°,90°,135°). The four-dimensional color polarized images in the dataset are paired according to high signal-to-noise ratio (SNR) and low SNR. The paired four-dimensional color polarized images with different SNRs are obtained by the camera in real-world environments, including indoor and outdoor non-digital environments. High SNR images can be obtained by setting the camera to a long exposure time and low gain, while low SNR images can be obtained by setting the camera to a short exposure time and high gain.

3. The denoising method for color polarized images based on a three-dimensional convolutional neural network according to claim 1, characterized in that, In step S2, the dataset is augmented using cropping, rotation, and flipping methods. The five dimensions in the five-dimensional tensor format are: batch dimension, polarization dimension, image height, image width, and color dimension. The batch dimension represents the size of the batch data processing, the polarization dimension represents the different polarization angle components, the image height represents the image pixel height, the image width represents the image pixel width, and the color dimension represents the feature map channels.

4. The denoising method for color polarized images based on a three-dimensional convolutional neural network according to claim 1, characterized in that, In step S3, the shallow feature extraction module uses sequentially arranged three-dimensional convolutional layers to extract shallow polarization features; the dense feature fusion module uses sequentially stacked residual dense modules to fuse features. This module mainly consists of residual connections and dense connections of three-dimensional convolutional layers, and also includes a three-dimensional merging layer, a three-dimensional bottleneck layer, and local residual connections. The deep feature fusion module uses a 3D merging layer, a 3D bottleneck layer, a 3D convolutional layer, and global residual connections to perform deep feature fusion.

5. The denoising method for color polarized images based on a three-dimensional convolutional neural network according to claim 1, characterized in that, In step S4, the loss function includes light intensity loss, color loss, and polarization loss, which respectively constrain the light intensity, color, and polarization information in the training of the 3D convolutional neural network. The light intensity loss calculates the mean square error between the output image and the ground truth image of the 3D convolutional neural network. The color loss calculates the mean square error between the light intensity components of the output image and the ground truth image of the 3D convolutional neural network. The polarization loss calculates the mean square error of the polarization angle and polarization degree components between the output image and the ground truth image of the 3D convolutional neural network. The proportions of each loss function are dynamically adjusted as the training process progresses through adaptive weights.

6. The denoising method for color polarized images based on a three-dimensional convolutional neural network according to claim 1, characterized in that, In step S2, the preprocessing process first calculates the average pixel brightness of each pair of high-low signal-to-noise ratio images in the dataset, and then calibrates the brightness of each pair of high-low signal-to-noise ratio images according to the average pixel brightness to make their average pixel brightness the same; then, the four-dimensional color polarization images in the dataset are augmented by cropping them into 64×64 image blocks and randomly flipping and rotating them; finally, the four-dimensional color polarization images are superpixel split and dimensional transformation is performed.

7. The method for denoising color polarized images based on a three-dimensional convolutional neural network according to claim 1 or 4, characterized in that, The convolution kernel of the three-dimensional convolutional layer is a cube with a shape of 3×3×3 pixels. During convolution, the convolution kernel is used to convolve the five-dimensional tensor format data in the pixel height, pixel width and polarization dimension, and then calculate and generate a three-dimensional feature map. Finally, the feature map is placed in the color dimension; each three-dimensional convolutional layer is followed by an activation function layer. The shallow feature extraction module includes two three-dimensional convolutional layers. The first convolutional layer has a kernel size of 3×3×3 and 64 kernel channels. The second convolutional layer has a kernel size of 3×3×3 and 32 kernel channels. There is a global residual connection between the first and second three-dimensional convolutional layers that is directly connected to the output layer of the three-dimensional convolutional neural network. This connection is used to optimize the network training process and prevent gradient explosion. The dense feature fusion module is configured with ten 3D convolutional residual dense modules. Each 3D convolutional residual dense module consists of six 3D convolutional layers, one 3D merging layer, and one 3D bottleneck layer. Each 3D convolutional layer can generate a 3D feature map, and the 3D convolutional layers are densely connected. The 3D convolutional residual dense modules are locally residual connected. In the deep feature fusion module, the 3D merging layer acquires all the output feature maps of the dense feature fusion module and merges them in the color dimension; the 3D bottleneck layer can shrink the number of channels in the color dimension and fuse the 3D feature maps obtained by the 3D merging layer; the 3D convolutional layer of the deep feature fusion module further extracts features from the fused result to obtain the final output.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the denoising method for color polarized images based on a three-dimensional convolutional neural network as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method for denoising color polarized images based on a three-dimensional convolutional neural network as described in any one of claims 1 to 5.

Images