A long-wave infrared polarization image enhancement method and system

Abstract

The application belongs to the technical field of polarized image processing and display, and in particular to a long-wave infrared polarized image enhancement method and system, which comprises the following: first, calculate the Stokes parameters to obtain intensity images and linear polarization degree images; perform a structure constraint information compensation operation to obtain a DoLP image that filters background noise and enhances polarization information; calculate the information difference of and to obtain a modal difference image; perform global average pooling on the basis of to obtain a feature description vector, perform linear transformation on the vector to obtain a vector Z, thereby calculating the corresponding weights and ; and the long-wave infrared polarized enhancement image is obtained by reconstructing the weighted calculation of the weights and and. The application adopts a composite loss function to constrain training from three aspects of structure, pixels and polarization information, and loss minimization is realized. The application can reasonably utilize the structural characteristics of infrared intensity images and the detailed texture characteristics of linear polarization degree images, and can improve the imaging effect of image targets, and has important application prospects for subsequent visual processing tasks.

Description

Technical Field

[0001] This invention belongs to the field of polarization image processing and display technology, and in particular to a method and system for enhancing long-wave infrared polarization images. Background Technology

[0002] Traditional visible light imaging can obtain high spatial resolution and rich texture details under sufficient lighting conditions. However, in complex environments such as night, backlight, rain, fog, and smoke, it is easily affected by factors such as changes in lighting and atmospheric scattering, resulting in a significant decrease in image quality. This manifests as reduced target contrast, loss of detail information, and structural blurring, making it difficult to meet the actual needs of stable perception in complex environments at all times and in all weather conditions.

[0003] Long-wave infrared imaging achieves imaging by receiving the thermal radiation information of the target itself. It does not depend on external lighting conditions and can obtain stable intensity information even in low-light or no-light environments. Therefore, it has important application value in target detection, environmental perception, and security monitoring. However, a single infrared intensity image only reflects the thermal radiation distribution characteristics of the target and lacks effective characterization of the object's surface material properties and structural information. In complex scenes, it is prone to problems such as insufficient contrast between the target and the background, loss of texture details, and significant reflection interference, thus limiting its application in fine perception tasks.

[0004] Polarization imaging technology, by acquiring polarization state information of radiated or reflected light, can characterize the surface material properties, geometric structure, and scattering characteristics of an object, providing an additional dimension of information for distinguishing targets from the background. Introducing polarization information into long-wave infrared imaging helps enhance target edge structures, suppress background interference, and improve the distinguishability between different objects. With the development of focal plane (DoFP) infrared polarization imaging technology, multi-directional polarization information can be acquired simultaneously in a single exposure, providing effective hardware support for real-time perception in complex environments.

[0005] However, due to limitations such as sensor noise, uneven spatial distribution of polarization information, and differences in multimodal information representation, long-wave infrared polarization images still suffer from insufficient contrast, significant noise interference, and inadequate utilization of polarization features in practical applications. Existing methods often struggle to effectively coordinate the complementary relationship between infrared intensity information and polarization features, resulting in deficiencies in structure preservation, detail enhancement, and noise suppression in the fusion results, making it difficult to fully leverage the advantages of polarization imaging.

[0006] Therefore, there is an urgent need to propose an effective long-wave infrared polarization image fusion method and system to achieve synergistic enhancement of multimodal information, improve the structural expression ability, detail clarity and overall readability of images, thereby meeting the practical needs of multi-scenario applications in complex environments. Summary of the Invention

[0007] (I) Solving technical problems

[0008] To address the shortcomings of existing technologies, this invention provides a long-wave infrared polarization image enhancement method and system, which solves the problems mentioned in the background section.

[0009] (II) Technical Solution

[0010] To achieve the above objectives, the present invention specifically adopts the following technical solution:

[0011] A method and system for enhancing long-wave infrared polarization images, comprising the following steps:

[0012] S1: Long-wave infrared polarization images are acquired using a split-plane polarization camera. The images are then depolarized using a mosaic operation to obtain polarization intensity maps for the four directions: 0°, 45°, 90°, and 135°. , , , The Stokes parameters are calculated based on the polarization intensity maps of the four directions mentioned above. , , and linear polarization image ;

[0013] S2: Calculated using S1 , The two images are then subjected to structural constraint information compensation and denoising operations to obtain a DoLP image that filters out background noise and enhances target polarization information. ;

[0014] S3: Based on calculations and The modal difference map between the two is obtained by calculating the information difference between them. ;

[0015] S4. Based on the difference diagram Global average pooling is used to obtain the feature description vector. , to carry out A linear transformation yields a vector Z, and the weights are calculated based on Z. and ;

[0016] S5: Calculate the obtained weights , , respectively with , The image is weighted and then input into the reconstructor to obtain a long-wave infrared polarization-enhanced image. .

[0017] In the implementation of this invention, a composite loss function is used to constrain the model training. This composite loss function guides the training process of the infrared polarization fusion network from different levels through three complementary loss terms: structural constraints, pixel constraints, and polarization constraints.

[0018] Furthermore, in S1, the... , , , Calculations were performed to obtain the Stokes parameters and the linear polarization image. The calculation formula is:

[0019]

[0020]

[0021] Furthermore, in S2, a structural constraint operation is performed, utilizing an intensity image with a high signal-to-noise ratio. As a guiding source, for linear polarization images Denoising processing is performed to preserve the structure, thereby obtaining Image. The calculation formula is as follows:

[0022] In local window Internally, solve for the linear coefficients. Thus filtering the output satisfy:

[0023]

[0024] And solve by minimizing the following energy function. and :

[0025]

[0026] The first term guarantees that the output is close to... The second regularization term prevents overfitting to noise. To smooth out control parameters, The larger the value, the smoother the output and the stronger the noise reduction. The smaller the value, the stronger the structural preservation. The coefficients are:

[0027]

[0028]

[0029]

[0030] In the formula Based on DoLP images guided by the structure The correlation coefficient is the local structural correlation coefficient. For local bias terms; This is a window centered at pixel k with a radius of r; , for Mean and variance within the window In order to be in The mean within the window; To calculate the obtained weights, for each pair of pixels i, aggregate the outputs of all windows covering it to obtain the final enhanced polarization image. This calculation method can effectively suppress... The background noise in the image is reduced while preserving the enhanced polarization characteristics consistent with the edge of the target's thermal radiation.

[0031] Furthermore, in S3, the modal difference map is calculated. The calculation formula is as follows:

[0032]

[0033] In the formula In order to local window centered The calculation formula is as follows:

[0034]

[0035] This item is subtracted to eliminate overall brightness difference and regional bias, retaining only the relatively abnormal difference response;

[0036] In the formula The standard deviation of differences within the same local window is calculated using the following formula:

[0037]

[0038] This term is used to characterize the stability of the differential scores within the region, and it varies with the magnitude of the differential scores;

[0039] In the formula To prevent extremely small positive numbers from being divided by zero;

[0040] Furthermore, the feature description vector described in S4 Vector Z, weight , The calculation formula is as follows:

[0041]

[0042]

[0043] In the formula This is a global average pooling operation. This is a linear transformation operation for fully connected layers.

[0044]

[0045]

[0046] Furthermore, in S5, the information obtained in S3 is utilized. , and , The enhanced infrared polarization image is obtained by weighting the images separately, and the calculation formula is as follows:

[0047]

[0048] In the implementation of this invention, a composite loss function is used to constrain the model training. This composite loss function guides the training process of the infrared polarization fusion network from different levels through three complementary loss terms: structural constraints, pixel constraints, and polarization constraints.

[0049] Structural constraint loss is primarily responsible for preserving the global spatial structure and edge contour information in infrared images, ensuring that the complete shape of high-temperature targets is not blurred or geometrically distorted after fusion. This is typically achieved through gradient difference or structural similarity metrics.

[0050]

[0051] Pixel constraint loss controls the deviation between the fused image and the input image at the pixel brightness level. It uses L1 or L2 norm to maintain true contrast and radiation characteristics, avoiding information loss caused by overexposure, underexposure, or dominance of a certain mode.

[0052]

[0053] in .

[0054] The polarization constraint loss is the core innovation of the entire composite function. It encodes the physical properties of polarized light (such as linear polarization degree and polarization angle) into the network, forces the fused image to retain the unique polarization characteristics of the material, and enables the network to distinguish different materials such as metal, plastic, glass, and water surface. At the same time, it suppresses specular reflection interference and guides the gradient direction to conform to the real physical laws.

[0055]

[0056] in To predict the fusion results.

[0057] When these three loss terms work together, structural and pixel constraints ensure that the fused image is first and foremost a normal infrared-enhanced image, while polarization constraints add material sensing capabilities, enabling the fused image to transcend the limitations of traditional thermal imaging. During training, the three constraints provide gradient signals from different dimensions, preventing the network from getting trapped in local optima and acting as a physical prior regularization, reducing dependence on labeled data and improving generalization ability. Ultimately, this composite loss function guides the network to generate an intelligent fused image that has a complete thermal target outline, true brightness and contrast, and can distinguish different material types.

[0058] (III) Beneficial Effects

[0059] Compared with the prior art, the present invention provides a long-wave infrared polarization image enhancement method and system, which has the following beneficial effects:

[0060] The method of this invention is suitable for linear polarization images with high noise levels. Structural constraint information compensation and noise reduction are performed to obtain a clearer structural texture. Then, based on the intensity map Compared with linear polarization diagram The modal difference map of the two inputs is calculated by adaptively assigning weights to the two inputs, fully considering the complementarity and conflict of the two modal information, and making reasonable use of infrared intensity information and polarization information to improve the salience of the target and the contrast between the target and the background. This is more in line with the visual habits of the observer, and the resulting infrared polarization enhanced image is more useful for subsequent visual processing tasks. It is of great significance for polarization imaging and camouflaged target detection. Attached Figure Description

[0061] Figure 1 This is a flowchart of a long-wave infrared polarization image enhancement method and system provided by the present invention;

[0062] Figure 2 This is a schematic diagram of a long-wave infrared polarization image enhancement method and system provided by the present invention;

[0063] Figure 3 This is the S0 constraint compensation denoising module provided in this invention example;

[0064] Figure 4 This is the modal difference calculation module provided in this invention example;

[0065] Figure 5 This is the weight calculation module provided in the example of the present invention;

[0066] Figure 6 The infrared intensity image provided in this invention example ;

[0067] Figure 7 This is a linear polarization image provided in an example of the present invention. ;

[0068] Figure 8 This is the denoised linear polarization image provided in this invention example. ;

[0069] Figure 9 These are differential images provided in the examples of this invention. Heatmap;

[0070] Figure 10 This is an example of an infrared polarization-enhanced image provided by the present invention. . Detailed Implementation

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] Example 1

[0073] like Figure 1-10 As shown, an embodiment of the present invention proposes a long-wave infrared polarization image enhancement method and system, which includes the following steps:

[0074] S1: Infrared polarization images are acquired using a split-plane polarization camera. The images are then de-mosaiced to obtain polarization intensity maps for the four directions: 0°, 45°, 90°, and 135°. , , , The Stokes parameters are calculated based on the polarization intensity maps of the four directions mentioned above. , , and linear polarization image Intensity map and linear polarization image like Figure 6 and Figure 7 As shown;

[0075] In S1, for , , , The formulas for calculating the Stock parameter and the DoLP image are as follows:

[0076]

[0077]

[0078] S2: This invention innovatively proposes a structural constraint compensation and denoising method for S0. It utilizes the values ​​calculated in S1... , By performing structural constraint compensation denoising on both images, a new DoLP image is obtained that filters out background noise and enhances target polarization information. ;image like Figure 8 As shown; the specific formula is as follows:

[0079]

[0080] Solve by minimizing the cost function:

[0081]

[0082] in To guide the image, For the input image, Here is the regularization parameter. The coefficients are:

[0083]

[0084]

[0085]

[0086] In the formula Based on DoLP images guided by the structure The correlation coefficient is the local structural correlation coefficient. For local bias terms; This is a window centered at pixel k with a radius of r; , for Mean and variance within the window In order to be in The mean within the window; To calculate the obtained weights, for each pair of pixels i, aggregate the outputs of all windows covering it to obtain the final enhanced polarization image. This calculation method can effectively suppress... The background noise in the image is reduced while preserving the enhanced polarization characteristics consistent with the edge of the target's thermal radiation.

[0087] S3: Based on calculations and The information difference between the two is calculated to obtain the information difference map. The calculation formula is as follows:

[0088]

[0089]

[0090]

[0091] In the formula The original pixel difference between the two images. This is the local mean of the original pixel difference image. Subtracting it is to eliminate the overall shift of the local background and only focus on abnormal differences relative to the background. This represents the local standard deviation of the original pixel difference image; To prevent the distribution from being 0.

[0092] S4: Based on the difference diagram Global average pooling is used to obtain the feature description vector. , to carry out A linear transformation yields a vector Z, and the weights are calculated based on Z. and The calculation formula is as follows:

[0093]

[0094]

[0095]

[0096] in r is the compression ratio. For ReLU activation function,

[0097]

[0098]

[0099] In the formula The original pixel difference between the two images. This is the local mean of the original pixel difference image. Subtracting it is to eliminate the overall shift of the local background and only focus on abnormal differences relative to the background. This represents the local standard deviation of the original pixel difference image; To prevent the distribution from being 0; in the formula This is a global average pooling operation. This is a linear transformation operation for fully connected layers.

[0100] S5: Using what was obtained in S4 , and , The enhanced infrared polarization image is obtained by performing weighted calculations and inputting the results into the reconstructor. The calculation formula is as follows:

[0101]

[0102] This invention innovatively proposes the aforementioned weight allocation method, the technical function of which is to calculate a dual-modal fusion weight allocation reference method, namely, a reference difference map. As shown in the figure, guided filtering is first used to denoise the image to obtain a linearly polarized enhanced image with clear structural texture. Next, the infrared intensity image was calculated. Image with linear polarization enhancement Difference diagram According to the difference diagram and The weights of the two inputs are calculated based on the information complementarity and information conflict, thus obtaining the final infrared polarization-enhanced image. Using the difference plot of two inputs This guides weight allocation, enabling a more reasonable balance between complementarity and conflict between the two modes, and provides appropriate guidance for infrared intensity maps. Thermal radiation differences are used to reflect the macroscopic contour information and linear polarization image of the target. The balance between unique physical properties such as surface roughness, material composition, and angle of incidence allows the two input characteristics to complement each other to achieve optimal performance.

[0103] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method and system for enhancing long-wave infrared polarization images, characterized in that: Includes the following steps: S1: Infrared polarization images are acquired using a split-focus plane polarization camera. The images are then de-mosaiced to obtain polarization intensity maps for the four directions: 0°, 45°, 90°, and 135°. , , , The Stokes parameters are calculated based on the polarization intensity maps of the four directions mentioned above. , , and linear polarization degree image . S2: Using the Swing Transformer decoder, respectively process the data in S1... , Feature extraction is performed, and then structural constraint information compensation and denoising are applied to the outputs of the two branches to obtain a DoLP image that filters out background noise and enhances target polarization information. . S3: Based on calculations and The modal difference map between the two is obtained by calculating the information difference between them. . S4. Based on the difference diagram Global average pooling is used to obtain the feature description vector. , to carry out A linear transformation yields a vector Z, and the weights are calculated based on Z. and . S5: Calculate the obtained weights , , respectively with , The weighted calculation is performed and input into the reconstructor to obtain a long-wave infrared polarization-enhanced image. . This invention employs a composite loss function to constrain model training from three aspects: structure, pixels, and polarization information, thereby minimizing the loss. This avoids the network getting trapped in local optima and also acts as a physical prior regularization, reducing dependence on labeled data and improving generalization ability.

2. The long-wave infrared polarization image enhancement method and system according to claim 1, characterized in that: In step S2, structural constraint compensation and noise reduction operations are performed to obtain the image. The formula is as follows: In the formula For based on DoLP images with preliminary structural guidance The correlation coefficient is the local structural correlation coefficient. For local bias terms; This is a window centered at pixel k with a radius of r; , for Mean and variance within the window for The mean within the window; The weights are calculated.

3. The long-wave infrared polarization image enhancement method and system according to claim 1, characterized in that: Image difference map in S3 The calculation formula is as follows: in, and These represent the mean and standard deviation of the input features, respectively.

4. The long-wave infrared polarization image enhancement method and system according to claim 1, characterized in that: The weight is calculated in S4. and The calculation formula is as follows: In the formula, r is the compression ratio. For ReLU activation function, , This is a global average pooling operation. For linear transformation operations in fully connected layers, To standardize operations.

5. In the implementation of this invention, a composite loss function is used to constrain model training. This composite loss function is achieved through structural constraints. Pixel constraints and polarization constraint Three complementary loss terms guide the training process of the infrared polarization fusion network from different levels. in .

6. A method and system for enhancing long-wave infrared polarization images as described in any one of claims 1-5, characterized in that: The system includes: Polarization calculation module: Infrared polarization images are acquired using a split-plane polarization camera. The images are then de-mosaiced to obtain polarization intensity maps in four directions: 0°, 45°, 90°, and 135°. , , , The Stokes parameters are calculated based on the polarization intensity maps of the four directions mentioned above. , , and linear polarization degree image ; S0 constraint compensation denoising module: used to determine... image, Image-guided denoising achieves new high quality image; Modal difference calculation module: used to calculate based on image, Image acquisition of two-modal difference maps ; Weight calculation module: used to calculate weights based on the difference plot. Get Images and Weights corresponding to the two modes of the image and ; Enhanced image calculation module: used to calculate based on the obtained weights and The images are weighted according to their corresponding modes and then input into the reconstructor to obtain the final enhanced image. .

Images