Polarization restoration method under dynamic scattering environment driven by adaptive learning
Through adaptive learning-driven polarization descattering network, the problem of image restoration in dynamic scattering environment is solved, and high-quality image recovery effect is achieved, which is suitable for scenarios such as autonomous driving.
Patent Information
- Application Number
- CN202510423068.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-07
- Publication Date
- 2025-07-18
AI Technical Summary
In dynamic scattering environments, existing polarization imaging methods are difficult to effectively restore clear images, especially in non-uniform scattering media, which has limited effects, affecting the accuracy of applications such as autonomous driving.
Adaptive learning-driven polarization descattering network is designed, through polarization feature extraction, dense feature fusion and parameter fitting modules, the image restoration is used to use the physical information of the polarized image, and multi-scale feature recovery is performed by combining the encoder-decoder network and multi-hop connection.
High-quality image restoration is achieved in a dynamic non-uniform scattering environment, improving visual effects, and maintaining stable performance, suitable for image restoration under complex conditions.
Smart Images
Figure CN120339145A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of deep learning technology and polarization imaging technology, and particularly relates to a polarization restoration method in a dynamic scattering environment driven by adaptive learning. Background Art
[0002] Computer vision, as a key branch in the field of artificial intelligence, endows computers with the ability to analyze and understand image and video content similar to human eyes, and its research value is extremely significant. In many application scenarios, computer vision technology can automatically perform tasks such as quality inspection, monitoring, and navigation, effectively reducing manual participation and significantly improving work efficiency and accuracy.
[0003] Especially in the field of autonomous driving, computer vision is one of the core technologies for realizing autonomous driving vehicles. It enables vehicles to accurately identify road signs, pedestrians, other vehicles, and obstacles, thus ensuring safe driving. However, in computer vision tasks, the target objects are often degraded by scattering media such as aerosol and smoke, resulting in an unclear captured image. The obtained blurred and low-quality image is difficult to be used for other advanced vision tasks. Therefore, the research on image restoration in an inhomogeneous scattering environment has emerged.
[0004] Currently, image dehazing methods are mainly divided into two categories: physics-based methods and learning-based methods. Physics-based methods are based on the atmospheric scattering model, and estimate relevant information such as the concentration of the scattering medium by constructing a mathematical model, and then restore a clear image. This method has the characteristics of high efficiency and speed. The learning-based method mainly relies on the powerful mapping ability of the deep learning neural network, and constructs a model that can restore a turbid image to a clear image by training the neural network. Compared with the traditional physical method, although this method requires more data volume and computing resources, it shows better effects when processing hazy images in complex scenarios. Compared with the traditional intensity imaging technology, polarization imaging has unique advantages because it can additionally obtain physical information such as the polarization degree and polarization angle of the target, and has been applied in many fields such as image restoration, material and texture recognition, and image super-resolution. However, limited by factors such as complex scenarios and harsh environments, the effect of polarization imaging still has limitations. In the vast majority of scattering environments, the distribution of the scattering medium is not uniform. Therefore, there is an urgent need to study a polarization restoration imaging method with a wider applicability and higher image restoration quality. Summary of the Invention
[0005] To overcome the deficiencies in the above-mentioned prior art, the present invention provides a polarization restoration method in a dynamic scattering environment driven by adaptive learning. A dataset is established in a simulated environment, making full use of the physical information of polarization images to train a network so that it can output the corresponding clear image according to the input turbid polarization image. Using this network model to restore the turbid polarization image can obtain high-quality clear images, and good stability can be maintained in the scenario of non-uniform scattering media.
[0006] To achieve the above object, the technical solution adopted by the present invention is as follows: A polarization restoration method in a dynamic scattering environment driven by adaptive learning, comprising the following steps:
[0007] Step 1: Take clear polarization images of underwater objects and corresponding polarization images with different turbidity levels;
[0008] Step 2: Divide the captured polarization images into a training set, a validation set, and a test set according to the ratio of 7:2:1;
[0009] Step 3: According to the guidance of the physical polarization defogging model and the information carried by a sequence of consecutive frames of images, design an adaptive learning-driven polarization de-scattering network for adaptive image restoration in a dynamic turbid environment; Three consecutive time-series polarization turbid images sequentially pass through a polarization feature extraction module, a dense feature fusion module, a parameter fitting module, and a reconstruction module; First, the polarization feature extraction module obtains the Stokes vector of the input image through a convolutional layer and extracts polarization features from each polarization image; Then, the extracted polarization features are sent to a series of dense feature fusion modules to fuse the polarization features to enhance the target information; Next, the enhanced polarization features obtained from the dense feature fusion module will be sent to the parameter fitting module to fit the degradation parameters in the polarization defogging imaging model, which adaptively adjusts the parameter estimation at each pixel for image restoration under different scattering conditions; Finally, the clear de-scattered image is restored through the reconstruction module;
[0010] Step 4: Train the network, use the dataset obtained in Step 2 to train the dynamic de-scattering model constructed in Step 3 so that it can obtain a clear de-scattered image according to the input turbid image;
[0011] Step 5: Restore the turbid polarization image in a dynamic scattering environment, and finally obtain a restored image with significantly improved visual effects and stable performance under complex conditions.
[0012] Further, the clear polarization image of the underwater object taken in Step 1: The light beam emitted by the light source passes through the polarizer and beam expander of the polarization modulation system in sequence and then irradiates the object. After being reflected by the object, it reaches the split focal plane polarization camera, thereby obtaining the clear polarization image of the object.
[0013] Further, in step one, dynamic scattering polarization images under different turbidity levels are captured: scattering medium is gradually added to water, and a 5-million-pixel polarization Gigabit Ethernet industrial camera is used to capture the dynamic scattering polarization images of the target object under different turbidity concentration environments.
[0014] Further, a 532 nm blue-green laser is used as the light source.
[0015] Further, the polarization feature extraction module in step three extracts the Stokes vector from each polarization unit through a 2×2 convolution kernel, and by fixing the convolution kernel weights, it ensures consistent use of the same parameters to obtain the polarization image.
[0016] Further, the dense feature fusion module consists of an attention mechanism and residual learning; the attention mechanism includes a channel attention mechanism and a pixel attention mechanism. The channel attention mechanism emphasizes important channels, and the pixel attention mechanism focuses on different scattering effects of different pixel points.
[0017] Further, the parameter fitting module receives multi-level features based on an encoder-decoder network and multi-hop connections, and obtains multi-scale features through downsampling and upsampling; the feature downsampling block consists of a 2×2 convolutional layer and a LeakyRelu activation function, and the upsampling block gradually restores the feature PixelShuffle to the original size; the features obtained in each downsampling stage are optimized through skip connections during the upsampling process.
[0018] Further, the entire process does not require manual selection of the background area as in traditional algorithms, thus enabling de-scattering imaging of dynamic non-uniform scattering scenes.
[0019] Further, the target object is placed in a fiberglass tank filled with water to simulate the scattering environment.
[0020] Further, the fiberglass tank is made of PMMA material.
[0021] Compared with the prior art, the present invention has the following beneficial effects:
[0022] The polarization restoration method provided by the present invention effectively reduces the non-uniform scattering effect caused by suspended particles in the dynamic turbid underwater environment. Guided by the theoretical model of physical polarization dehazing, two polarization-related parameters at each pixel position are accurately estimated in the proposed neural network, and it has excellent convergence and restoration performance. In addition, this method proposes to fuse consecutive multi-frame polarization images to extract more effective target feature information for target image restoration. Description of the Drawings
[0023] Figure 1It is the overall process schematic diagram of the present invention;
[0024] Figure 2 It is the schematic diagram of the experimental device of the present invention;
[0025] Figure 3 It is the schematic diagram of the neural network model structure proposed in the present invention;
[0026] Figure 4 It is the model restoration effect diagram under the dynamic scattering environment of the present invention. (a) is the turbid polarization image, (b) is the image restored by the present invention, and (c) is the clear image;
[0027] Figure 5 It is the table of restoration effect evaluation indexes SSIM and PSNR.
[0028] In the figure: light source 1, polarizer 2, beam expander 3, glass cylinder 4, underwater environment 5, target object 6, split focal plane polarization camera 7. Specific implementation manners
[0029] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
[0030] As Figures 1 - 5 shown, the technical solution adopted by the present invention is as follows: This embodiment provides a polarization restoration method under a dynamic scattering environment driven by adaptive learning, including the following steps:
[0031] Step 1: Take clear polarization images of the underwater target object and corresponding polarization images with different turbidity levels;
[0032] Step 2: Divide the captured polarization images into a training set, a validation set, and a test set according to the ratio of 7:2:1;
[0033] Step 3: According to the guidance of the physical polarization dehazing model and the information carried by consecutive multi-frame image sequences, design an adaptive learning-driven polarization de-scattering network for adaptive image restoration in a dynamic turbid environment. Three consecutive time-series polarized turbid images sequentially pass through a polarization feature extraction module, a dense feature fusion module, a parameter fitting module, and a reconstruction module. First, the polarization feature extraction module obtains the Stokes vector of the input image through a convolutional layer and extracts polarization features from each polarized image. Then, the extracted polarization features are sent to a series of dense feature fusion modules to fuse the polarization features to enhance the target information. Next, the enhanced polarization features obtained from the dense feature fusion module will be fed into the parameter fitting module to fit the degradation parameters in the polarization dehazing imaging model, which adaptively adjusts the parameter estimation at each pixel for image restoration under different scattering conditions. Finally, a clear de-scattered image is restored through the reconstruction module.
[0034] Step 4: Train the network. Use the dataset obtained in Step 2 to train the dynamic de-scattering model constructed in Step 3 so that it can obtain a clear de-scattered image based on the input turbid image.
[0035] Step 5: Restore the turbid polarized image in a dynamic scattering environment to finally obtain a restored image with significantly improved visual effects and maintain stable performance under complex conditions.
[0036] Further, the clear polarized image of the underwater target object captured in Step 1: The light beam emitted by the light source 1 sequentially passes through the polarizer 2 and the beam expander 3 of the polarization modulation system and then irradiates the target object 6. After being reflected by the target object 6, it reaches the split focal plane polarization camera 7, thereby obtaining the clear polarized image of the target object 6.
[0037] Further, the dynamic scattering polarized images captured in different turbidity levels in Step 1: Gradually add a scattering medium (such as skim milk, etc.) to the water, and use a 5-million-pixel polarization gigabit Ethernet industrial camera to capture the dynamic scattering polarized images of the target object 6 at different turbidity levels.
[0038] Further, use a 532 nm blue-green laser as the light source 1.
[0039] Further, the polarization feature extraction module in Step 3 extracts the Stokes vector from each polarization unit through a 2×2 convolutional kernel, and by fixing the convolutional kernel weights, it ensures that the same parameters are consistently used to obtain the polarized image.
[0040] Further, the dense feature fusion module consists of an attention mechanism and residual learning. The attention mechanism includes a channel attention mechanism and a pixel attention mechanism. The channel attention mechanism emphasizes important channels, and the pixel attention mechanism focuses on different scattering effects of different pixel points.
[0041] Furthermore, the parameter fitting module receives multi-level features based on the encoder-decoder network and multi-hop connections, and obtains multi-scale features through downsampling and upsampling. The feature downsampling block consists of a 2×2 convolutional layer and a LeakyRelu activation function, and the upsampling block gradually restores the features to the original size by PixelShuffle. The features obtained in each downsampling stage are optimized through skip connections during the upsampling process.
[0042] Furthermore, the whole process does not require manual selection of the background area as in traditional algorithms, so that it is possible to realize de-scattering imaging of dynamic non-uniform scattering scenes.
[0043] Furthermore, the polarization de-scattering network described in step 3 describes the change relationship between the turbid image and the clear image.
[0044] Specifically, in step 1, an underwater active imaging system is used, and linear polarized light is used for active illumination to capture clear polarized images of the underwater target 6 and dynamic scattering turbid images at different concentrations. In this embodiment, a 532 nm blue-green laser is used as the light source 1, and a PMMA (polymethyl methacrylate) glass cylinder 4 is selected.
[0045] The light beam emitted by the light source 1 passes through the polarizer 2 and the beam expander 3 of the polarization modulation system in sequence and then irradiates the underwater target 6. After being reflected by the target 6, it reaches the split focal plane polarization camera 7 to capture clear polarized images of the underwater target 6. Scattering medium (such as skim milk, etc.) is gradually added to the water, and a 5-million-pixel polarization gigabit Ethernet industrial camera is used to capture polarized images of the target 6 at different turbidity levels. In this example, 10 groups of images are captured, with 20 different concentrations of milk added to each group to simulate the underwater environment 5 at 20 different concentrations. For each concentration, a polarized image of the target 6 (including polarization directions of 0 degrees, 45 degrees, 90 degrees, and 135 degrees) is captured according to the shooting method in step 1.
[0046] Step 2: Establish a data set; the images obtained in step 1 are enlarged by flipping, rotating, and cropping to expand the data set, and then divided into a training set, a validation set, and a test set according to a ratio of 7:2:1.
[0047] Step 3: According to the guidance of the physical polarization dehazing model and the information carried by consecutive multi-frame image sequences, design an adaptive learning-driven polarization de-scattering network for adaptive image restoration in dynamic turbid environments. Three consecutive time-series polarized turbid images are successively passed through a polarization feature extraction module, a dense feature fusion module, a parameter fitting module, and a reconstruction module. The polarization feature extraction module obtains the Stokes vector of the input image through a convolutional layer and extracts polarization features from each polarized image. Then, the extracted polarization features are sent to a series of dense feature fusion modules to fuse the polarization features to enhance the target information. The enhanced polarization features obtained from the dense feature fusion module are fed into the parameter fitting module to fit the degradation parameters in the polarization dehazing imaging model, which adaptively adjusts the parameter estimation at each pixel for image restoration under different scattering conditions. Finally, a clear de-scattered image is restored through the reconstruction module.
[0048] Step 4: Train the network. Use the dataset obtained in Step 2 to train the dynamic de-scattering model constructed in Step 3 so that it can obtain a clear de-scattered image based on the input turbid image.
[0049] Step 5: Restore the turbid polarized image in a dynamic scattering environment to finally obtain a restored image with significantly improved visual effects.
[0050] It can be seen from the experimental results that the present invention can effectively restore the polarized images taken underwater with high-concentration turbidity. Combining with the objective evaluation indexes SSIM (structural similarity) and PSNR (peak signal-to-noise ratio), the image restoration effect is remarkable.
[0051] Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions recorded in the foregoing embodiments or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.
Claims
1. An adaptive learning-driven polarization restoration method in a dynamic scattering environment, characterized in that It includes the following steps: Step 1: Take clear polarization images of the underwater target and polarization images corresponding to different turbidity levels. Step 2: Divide the captured polarization images into a training set, a validation set, and a test set according to the ratio of 7:2:
1. Step 3: According to the guidance of the physical polarization defogging model and the information carried by consecutive multi-frame image sequences, design an adaptive learning-driven polarization de-scattering network for adaptive image restoration in a dynamic turbid environment; three consecutive time-series polarization turbid images sequentially pass through a polarization feature extraction module, a dense feature fusion module, a parameter fitting module, and a reconstruction module; First, the polarization feature extraction module obtains the Stokes vector of the input image through a convolutional layer and acquires polarization features from each polarization image; Then, the extracted polarization features are sent to a series of dense feature fusion modules to fuse the polarization features to enhance the target information; Next, the enhanced polarization features obtained from the dense feature fusion module will be fed into the parameter fitting module to fit the degradation parameters in the polarization defogging imaging model, which adaptively adjusts the parameter estimation at each pixel for image restoration under different scattering conditions; Finally, a clear de-scattered image is restored through the reconstruction module. Step 4: Train the network. Use the data set obtained in Step 2 to train the dynamic de-scattering model constructed in Step 3 so that it can obtain a clear de-scattered image according to the input turbid image. Step 5: Restore the turbid polarization image in a dynamic scattering environment, and finally obtain a restored image with significantly improved visual effects and maintain stable performance under complex conditions.
2. The polarization restoration method in a dynamic scattering environment driven by adaptive learning according to claim 1, wherein: The clear polarization image of the underwater target taken in Step 1: The light beam emitted by the light source (1) sequentially passes through the polarizer (2) and the beam expander (3) of the polarization modulation system and then irradiates the target (6). After being reflected by the target (6), it reaches the split focal plane polarization camera (7), thereby obtaining the clear polarization image of the target (6).
3. The polarization restoration method in a dynamic scattering environment driven by adaptive learning according to claim 2, wherein: The dynamic scattering polarization images taken in Step 1 under different turbidity levels: Gradually add scattering medium into the water, and use a 5-million-pixel polarization gigabit Ethernet industrial camera to take the dynamic scattering polarization images of the target (6) in environments with different turbidity concentrations.
4. The polarization restoration method in a dynamic scattering environment driven by adaptive learning according to claim 2 or 3, characterized in that: Use a 532nm blue-green laser as the light source (1).
5. The polarization restoration method in a dynamic scattering environment driven by adaptive learning according to claim 4, wherein: In the polarization feature extraction module described in Step 3, the Stokes vector is extracted from each polarization unit through a 2×2 convolutional kernel, and the same parameters are consistently used by fixing the convolutional kernel weights to obtain the polarization image.
6. The polarization restoration method in a dynamic scattering environment driven by adaptive learning according to claim 4, characterized in that: The dense feature fusion module consists of an attention mechanism and residual learning; the attention mechanism includes a channel attention mechanism and a pixel attention mechanism. The channel attention mechanism emphasizes important channels, and the pixel attention mechanism focuses on different scattering effects of different pixel points.
7. The polarization restoration method in a dynamic scattering environment driven by adaptive learning according to claim 4, wherein: The parameter fitting module receives multi-level features based on an encoder-decoder network and multi-hop connections, and obtains multi-scale features through downsampling and upsampling. The feature downsampling block consists of a 2×2 convolutional layer and a LeakyRelu activation function, and the upsampling block gradually restores the features to the original size by PixelShuffle; the features obtained in each downsampling stage are optimized through skip connections during the upsampling process.
8. The polarization restoration method in a dynamic scattering environment driven by adaptive learning according to claim 4, characterized in that: The entire process does not require manual selection of the background area as in traditional algorithms, thus enabling de-scattering imaging of dynamic non-uniform scattering scenes.
9. The polarization restoration method in a dynamically scattering environment driven by adaptive learning according to claim 2 or 3, characterized in that: The object (6) is placed in a glass fiber reinforced plastic (4) filled with water to simulate the scattering environment (5).
10. The polarization restoration method in a dynamic scattering environment driven by adaptive learning according to claim 9, wherein: The glass fiber reinforced plastic (4) is made of PMMA material.