Restoration flame shielding imaging method based on large model and polarization frequency domain network

Abstract

The invention discloses a method for restoring flame shielding imaging based on a large model and a polarization frequency domain network, and the method is characterized in that the method comprises the following steps: 1, building a flame shielding target imaging system, and shooting a clear image of a target object without flame shielding; respectively shooting polarization images of the target object under the conditions of low flame shielding rate, high flame shielding rate and high-intensity flame shielding, and obtaining a low flame shielding rate data set, a high flame shielding rate data set and a high-intensity flame shielding rate data set; according to the restoration method provided by the invention, the problem of serious image degradation caused by strong radiation of flame and scattering of internal particles is effectively solved, and clear imaging of a target behind non-uniform flame shielding is realized. The method is based on an innovative double-branch deep network, and a polarization frequency domain branch accurately separates and extracts submerged target detail information by utilizing the difference between a target and flame in polarization characteristics and combining frequency domain analysis.

Description

Technical Field

[0001] The present invention relates to the fields of deep learning technology and polarization imaging technology, and in particular to a flame occlusion imaging restoration method based on a large model and a polarization frequency domain network. Background Art

[0002] Flames are a common and important natural and industrial phenomenon. The observation and analysis of targets within and behind flames plays a vital role in research in numerous fields, including fire rescue, combustion diagnosis, industrial process monitoring, and aerospace. However, flames themselves emit strong visible light radiation, and flames contain a large number of high-temperature particles (such as soot particles and unburned fuel droplets), which strongly scatter and absorb light. Due to the interference of this scattered light and the flame's own radiation, under traditional visible light imaging conditions, target information behind or within the flame is severely obscured, resulting in a sharp decline in image quality, such as extremely low target contrast, blurred details, or even complete invisibility. This makes conventional image-based restoration, recognition, and precise analysis difficult to effectively implement in flame environments.

[0003] Currently, imaging and information recovery methods for flame obstruction can be divided into two main categories: imaging methods based on specific band selection and traditional image enhancement methods. Band-selective methods, such as infrared thermal imaging, can partially penetrate smoke and flames, but they only capture temperature distribution information and have limited ability to recover detail and color information of the target itself. Traditional image enhancement methods, such as specialized filtering algorithms for flame images and contrast stretching, can improve visual effects to a certain extent, but often struggle to recover true details obscured by flame light and scattered light. Summary of the Invention

[0004] Polarization imaging technology, a new and promising optical detection method, has been discovered to be applicable to flame obstruction removal. Flame radiated light and light scattered by particles within the flame typically exhibit specific polarization characteristics, while light reflected from the target surface may also carry unique polarization information. By analyzing light intensity under different polarization states, polarization imaging can distinguish target reflected light from flame interference light, effectively suppressing the strong light interference and scattering effects of the flame. This allows the extraction of key information such as the outline, texture, and even material of the obscured target, further enhancing image restoration.

[0005] To address flame-obstructed scenes, this paper utilizes polarization imaging to eliminate interference from the flame's own radiation. In combination with deep learning techniques, a method for restoring flame-obstructed images based on a large visual model and a polarization frequency domain network is proposed. Using this network for restoration can effectively improve image quality.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows: a method for restoring flame obstruction imaging based on a large model and a polarization frequency domain network, characterized by comprising the following steps:

[0007] Step 1: Build a flame-obstructed target imaging system to capture clear images of the target without flame obstruction; capture polarization images of the target under low flame obstruction rate, high flame obstruction rate, and high-intensity flame obstruction rates, and obtain low flame obstruction rate dataset, high flame obstruction rate dataset, and high-intensity flame obstruction rate dataset;

[0008] Step 2: Randomly divide the low flame obstruction rate dataset, high flame obstruction rate dataset, and high-intensity flame obstruction rate dataset obtained in step 1 into training set, validation set, and test set according to the ratio of 75:5:20;

[0009] Step 3: All training sets in step 2 are rotated, flipped horizontally, and flipped vertically with a probability of 0.5 to achieve data enhancement and amplification;

[0010] Step 4. Construct a dual-branch network based on the guidance of the large visual model and the polarization frequency domain characteristics; the polarization frequency domain branch is based on U-Net, a network architecture composed of convolution, discrete wavelet transform fusion module, fast Fourier convolution and attention mechanism, and the input is the S1 and S2 images of size 1024*1024 in the Stokes vector; the ConvNeXt guided by the large visual model is based on U-Net, a network framework composed of downsampling, ConvNext module, large model guidance module, attention mechanism, and upsampling, and the input is the S0 image of size 1024*1024 in the Stokes vector; the output of the dual branches is then calculated through convolution and activation function to output the predicted image;

[0011] Step 5: Input 9 groups of images per batch for iterative training. Calculate the mean absolute error loss, multi-scale structural similarity loss, and distillation loss for the predicted image output in step 4 and the clear image without flame occlusion in step 1. Then backpropagate to update the parameters of the entire network.

[0012] Step 6: After training the low flame occlusion rate dataset, the high flame occlusion rate dataset, and the high-intensity flame occlusion rate dataset respectively, the trained network is used to predict the flame-occluded image, and finally the unoccluded restored image is obtained.

[0013] Furthermore, in step one, an LED light source and a linear polarizer are used to provide polarized illumination; the flame height of the flame generator or different flames are adjusted to achieve different occlusion rates of the flame on the target; and a split-focus plane polarization camera is used to capture polarized images of the target occluded by different flame states.

[0014] Furthermore, in step 2, random partitioning is performed using the random library in Python.

[0015] Furthermore, the discrete wavelet transform fusion module described in step 4 consists of two parts. The first part consists of 4*4 convolution, batch normalization layer, and Relu activation function; the second part first undergoes a two-dimensional discrete wavelet transform to output high-frequency features and low-frequency features. The high-frequency features will be jump-connected and common features will be obtained with the low-frequency features through the attention mechanism; finally, the outputs of the last two parts are jump-connected with the output of the second part and added together to reveal the frequency domain characteristics of the model and increase the detail representation capability.

[0016] Furthermore, in step 5, the Adam optimizer is used to update and optimize the parameters, with a learning rate of 0.0001, a hyperparameter weight of 1 for the mean absolute error loss, a hyperparameter weight of 0.2 for the multi-scale structural similarity loss, and a hyperparameter weight of 0.5 for the distillation loss, which together constitute the total loss.

[0017] Furthermore, the flame used alcohol and paraffin ( ) as fuel to realize the flame scene.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] The restoration method proposed in the present invention effectively overcomes the serious image degradation problem caused by the strong radiation of the flame itself and the internal particle scattering, and realizes clear imaging of the target behind the non-uniform flame. This method is based on an innovative dual-branch deep network, in which the polarization frequency domain branch uses the difference in polarization characteristics between the target and the flame, combined with frequency domain analysis to accurately separate and extract the detailed information of the submerged target; at the same time, the visual large model guidance branch uses powerful image prior knowledge to ensure the global structural authenticity of the restoration result. The synergistic effect of this dual constraint enables the method to achieve end-to-end automated high-fidelity restoration without the need for pre-segmentation or labeling of the flame area, significantly improving the imaging quality and robustness in complex dynamic scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0021] Figure 2 It is the flame obstruction target imaging system of the present invention;

[0022] Figure 3 It is a schematic diagram of the network architecture of the present invention;

[0023] Figure 4 Schematic diagram of the discrete wavelet transform fusion module of the present invention;

[0024] Figure 5 The network experiment effect diagram of the present invention is shown in FIG. (a) is the flame-blocked image, (b) is the restored image of the present invention, and (c) is the clear intensity image without flame blockage.

[0025] Figure 6 This is a table of restoration effect evaluation indicators SSIM and PSNR.

[0026] In the figure: LED light source 1, linear polarizer 2, flame generator 3, target 4, and focal plane polarization camera 5. DETAILED DESCRIPTION

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0028] like Figures 1-6 As shown, the technical solution adopted by the present invention is as follows: This embodiment provides a method for restoring flame obstruction imaging based on a large model and a polarization frequency domain network, comprising the following steps:

[0029] Step 1: Build a flame obstruction target imaging system to capture clear images of the target 4 without flame obstruction; capture polarized images of the target 4 under low flame obstruction rate, high flame obstruction rate, and high-intensity flame obstruction rates, and obtain a low flame obstruction rate dataset, a high flame obstruction rate dataset, and a high-intensity flame obstruction rate dataset;

[0030] Step 2: Randomly divide the low flame obstruction rate dataset, high flame obstruction rate dataset, and high-intensity flame obstruction rate dataset obtained in step 1 into training set, validation set, and test set according to the ratio of 75:5:20;

[0031] Step 3: All training sets in step 2 are rotated, flipped horizontally, and flipped vertically with a probability of 0.5 to achieve data enhancement and amplification;

[0032] Step 4: Figure 3As shown in the figure, a dual-branch network based on the guidance of a large visual model and polarization frequency domain characteristics is constructed; the polarization frequency domain branch is based on U-Net, a network architecture composed of convolution, discrete wavelet transform fusion module, fast Fourier convolution and attention mechanism, and the input is the S1 and S2 images of size 1024*1024 in the Stokes vector; the ConvNeXt guided by the large visual model is based on U-Net, a network architecture composed of downsampling, ConvNext module, large model guidance module, attention mechanism, and upsampling, and the input is the S0 image of size 1024*1024 in the Stokes vector; the output of the dual branches is then calculated through convolution and activation function to output the predicted image;

[0033] Step 5: Input 9 groups of images per batch for iterative training. Calculate the mean absolute error loss, multi-scale structural similarity loss, and distillation loss for the predicted image output in step 4 and the clear image without flame occlusion in step 1. Then backpropagate to update the parameters of the entire network.

[0034] Step 6: After training the low flame occlusion rate dataset, the high flame occlusion rate dataset, and the high-intensity flame occlusion rate dataset respectively, the trained network is used to predict the flame-occluded image, and finally the unoccluded restored image is obtained.

[0035] Furthermore, we regard flame as a non-uniform scattering medium with self-luminous properties. The experimental setup is as follows: Figure 2 As shown, in step 1, an LED light source 1 is used, with a linear polarizer 2 placed in front of it to provide polarized illumination. A split-focus plane polarization camera 5 is used to capture the target 4 obscured by the flame. The light source emits a divergent beam, which then passes through the polarizer to produce polarized light. The flame medium scatters the attenuated target light, and both the scattered light and the flame's spontaneous radiation are captured by the camera. The flame size (i.e., the flame height) of the flame generator 3 is adjusted to achieve different obscuration rates. Different flames are then used to achieve different intensities, achieving the desired image quality.

[0036] Furthermore, in step 2, random partitioning is performed using the random library in Python.

[0037] Further, such as Figure 4 As shown in the figure, the discrete wavelet transform fusion module described in step 4 consists of two parts. The first part is composed of 4*4 convolution, batch normalization layer, and Relu activation function; the second part first undergoes a two-dimensional discrete wavelet transform to output high-frequency features and low-frequency features. The high-frequency features will be jump-connected and common features will be obtained with the low-frequency features through the attention mechanism; finally, the outputs of the last two parts are jump-connected with the output of the second part and added together to reveal the frequency domain characteristics of the model and increase the detail representation capability.

[0038] Furthermore, the fast Fourier convolution described in step 4 combines fast Fourier transform and convolution to make up for the global frequency domain characteristics of convolution.

[0039] Furthermore, the large model guidance module described in step 4 inputs the intensity map of the flame occlusion into the trained SAM, performs feature extraction, calculates the distillation loss of features of the same size as the network, and utilizes the powerful feature extraction capability of the large model to perform feature distillation on the model, thereby reducing the number of iterations while avoiding falling into the local optimal solution.

[0040] Furthermore, in step 5, the Adam optimizer is used to update and optimize the parameters, with a learning rate of 0.0001, a hyperparameter weight of 1 for the mean absolute error loss, a hyperparameter weight of 0.2 for the multi-scale structural similarity loss, and a hyperparameter weight of 0.5 for the distillation loss, which together constitute the total loss.

[0041] Furthermore, the flame used alcohol and paraffin ( ) as fuel to realize the flame scene.

[0042] from Figure 5 and Figure 6 It can be seen from the experimental results that the present invention can effectively restore flames. Combining the objective evaluation criteria mean absolute error (MAE), structure similarity index measure (SSIM), peak signal to noise ratio (PSNR) and subjective feeling, the image restoration effect is significant.

[0043] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A flame occlusion imaging restoration method based on a large model and polarization frequency domain network, characterized in that: The following steps are involved: Step 1: Build a flame obstruction target imaging system to capture a clear image of the target (4) without flame obstruction; capture polarization images of the target (4) under low flame obstruction rate, high flame obstruction rate, and high-intensity flame obstruction respectively, and obtain a low flame obstruction rate data set, a high flame obstruction rate data set, and a high-intensity flame obstruction rate data set; Step 2: Randomly divide the low flame obstruction rate dataset, high flame obstruction rate dataset, and high-intensity flame obstruction rate dataset obtained in step 1 into training set, validation set, and test set according to the ratio of 75:5:20; Step 3: All training sets in step 2 are rotated, flipped horizontally, and flipped vertically with a probability of 0.5 to achieve data enhancement and amplification; Step 4: Construct a dual-branch network based on visual large model guidance and polarization frequency domain characteristics; The polarization frequency domain branch is based on U-Net, a network architecture consisting of convolution, discrete wavelet transform fusion module, fast Fourier convolution and attention mechanism. The input is S1 and S2 images of size 1024*1024 in Stokes vector. The ConvNeXt guided by the large visual model is based on U-Net, a network architecture consisting of downsampling, ConvNext module, large model guidance module, attention mechanism, and upsampling. The input is S0 image of size 1024*1024 in Stokes vector. The output of the two branches is then calculated through convolution and activation function to output the predicted image. Step 5: Input 9 groups of images per batch for iterative training. Calculate the mean absolute error loss, multi-scale structural similarity loss, and distillation loss for the predicted image output in step 4 and the clear image without flame occlusion in step 1. Then backpropagate to update the parameters of the entire network. Step 6: After training the low flame occlusion rate dataset, the high flame occlusion rate dataset, and the high-intensity flame occlusion rate dataset respectively, the trained network is used to predict the flame-occluded image, and finally the unoccluded restored image is obtained.

2. The flame occlusion imaging restoration method based on a large model and a polarization frequency domain network according to claim 1 is characterized in that: In step 1, polarized illumination is provided by using an LED light source (1) and a linear polarizer (2); the flame height of a flame generator (3) or different flames are adjusted to achieve different shielding rates of the flame on the target (4); and polarized images of the shielding of the target (4) by different flame states are captured by a split-focus plane polarization camera (5).

3. The flame occlusion imaging restoration method based on a large model and a polarization frequency domain network according to claim 1 is characterized by: In step 2, random partitioning is performed using the random library in Python.

4. The flame occlusion imaging restoration method based on a large model and a polarization frequency domain network according to claim 1 is characterized by: The discrete wavelet transform fusion module described in step 4 consists of two parts. The first part consists of 4*4 convolution, batch normalization layer, and Relu activation function; the second part first undergoes a two-dimensional discrete wavelet transform to output high-frequency features and low-frequency features. The high-frequency features will be jump-connected and common features will be obtained with the low-frequency features through the attention mechanism; finally, the outputs of the last two parts are jump-connected with the output of the second part and added together to reveal the frequency domain characteristics of the model and increase the detail representation capability.

5. The flame occlusion imaging restoration method based on a large model and a polarization frequency domain network according to claim 1 is characterized in that: In step 5, the Adam optimizer is used to update and optimize the parameters. The learning rate is 0.0001, the hyperparameter weight of the mean absolute error loss is 1, the hyperparameter weight of the multi-scale structural similarity loss is 0.2, and the hyperparameter weight of the distillation loss is 0.5, which together constitute the total loss.

6. The flame occlusion imaging restoration method based on a large model and a polarization frequency domain network according to claim 4 is characterized in that: The flame used alcohol and paraffin ( ) as fuel to realize the flame scene.

Images