Infrared super-lens image quality enhancement method based on fusion of two-way decoding and physical perception prompt

Abstract

The invention relates to the technical field of image processing, and discloses an infrared super-lens image quality enhancement method based on fusion of double-path decoding and physical perception prompt, which comprises the following steps: S1, generating a physical parameter prompt, acquiring geometric and optical characteristic parameters of an infrared super-lens, and converting the parameters into textualized prompt vectors; s2, constructing a data set; s3, cross-modal feature coding and fusion are carried out; and S4, carrying out double-path decoding and reconstruction. According to the infrared super-lens image quality enhancement method based on the fusion of the two-way decoding and the physical perception prompt, by introducing physical parameter prompt and cross-modal feature fusion, the imaging physical prior of an infrared super-lens is effectively fused into a network, so that a model can understand a degradation source and imaging constraints; and deep collaboration of visual information and physical attributes is realized in a feature extraction process. The design not only enhances the adaptability of the model to a complex imaging system, but also improves the interpretability and robustness of an enhancement result.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an infrared superlens image quality enhancement method based on the fusion of dual-path decoding and physical perception cues. Background Technology

[0002] Infrared superlenses, as an emerging planar optical element, possess advantages such as small size and high integration, showing broad prospects in the field of infrared imaging. Currently, research on infrared image quality enhancement mainly falls into two categories: one is based on traditional image processing methods, such as dark channel priors, multi-scale Retinex, and histogram equalization. While these methods are computationally efficient, they rely on manually designed priors, have poor adaptability to complex degradation scenes, and struggle to effectively recover image information lost due to the physical properties of superlenses. The other category is based on deep learning methods, such as image restoration models based on convolutional neural networks or Transformers, which achieve image deblurring and denoising tasks through end-to-end learning. However, existing deep learning models are mostly designed for visible light images and do not fully consider the physical mechanisms of infrared superlens imaging, resulting in poor performance in infrared image enhancement tasks.

[0003] In recent years, with the development of generative artificial intelligence and visual pre-trained models, some studies have attempted to introduce prompt learning mechanisms into image restoration and enhancement tasks. By embedding degradation type cues, environmental semantic cues, or task-related cues into the network, the model can adaptively adjust feature extraction and restoration strategies according to different degradation types (such as noise, blur, uneven exposure, etc.), thus achieving good generalization performance in the natural image domain. However, these methods still have the following problems in infrared superlens image enhancement: First, the lack of modeling of the physical parameters of the superlens (such as structural size, operating wavelength, etc.) makes it difficult for the model to accurately understand the source of image degradation, thus limiting the realism and interpretability of the enhancement results; second, traditional prompt learning frameworks mainly focus on improving overall image quality, such as optimizing brightness, sharpness, or contrast, but neglect the spatial structural differences and semantic constraints between foreground targets and background environments in infrared images, resulting in insufficient performance in terms of local detail preservation, structural consistency, and target semantic continuity in the enhanced image. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an infrared superlens image quality enhancement method based on the fusion of dual-path decoding and physical perception cues, thus solving the problems mentioned in the background.

[0005] This invention provides the following technical solution: a method for enhancing the quality of infrared superlens images based on the fusion of dual-path decoding and physical perception cues, comprising the following steps: Step S1: Physical parameter prompt generation. Obtain the geometric and optical characteristic parameters of the infrared superlens and convert the parameters into a textual prompt vector. Step S2: Dataset construction. Collect infrared superlens image data containing multiple degradation types, construct samples in the form of triples {physical parameter hints, degraded infrared images, clear infrared images}, and divide them into training set and test set. Step S3: Cross-modal feature encoding and fusion. An image encoder is used to extract deep features from the input degraded infrared image to obtain visual features. Simultaneously, a physical parameter encoder is used to embed the generated textual cue vector to obtain physical features. The visual and physical features are then input into a cross-modal feature fusion module for fusion to obtain fused features. The cross-modal feature fusion module calculates spatial correlation through a positional coordinate attention mechanism, generates adaptive modulation weights, enhances key region features, and fuses the degraded cue features and physical cue features through a physical perception cue fusion module to generate a comprehensive cue vector for dynamically adjusting the distribution of multi-layer features. Step S4, Dual-path Decoding and Reconstruction: The obtained fused features are input into a dual-path collaborative decoding network, which includes a parallel global quality decoder and a spatial structure decoder. The global quality decoder is used to decode image domain features to improve the overall image quality index, and the spatial structure decoder is used to decode spectral domain or structural domain features to preserve the details and structure of the image. The outputs of the two decoders are fused to obtain an enhanced infrared superlens image.

[0006] Preferably, in step S1, the geometric and optical characteristic parameters include the diameter, focal length, operating wavelength, and numerical aperture of the infrared superlens.

[0007] Preferably, in step S3, the cross-modal feature fusion module includes multiple downsampling stages, each of which sequentially performs feature extraction, location coordinate attention calculation, physical perception cue fusion, and feature dimensionality reduction.

[0008] Preferably, the specific process of the position coordinate attention mechanism includes: encoding the input feature map along the height and width directions respectively to obtain feature maps that are perceived in two directions; concatenating, transforming and activating the feature maps in the two directions, and then splitting them into attention weights in the height and width directions; using the attention weights to spatially weight the input feature map to obtain the output feature map; The position coordinate attention mechanism is implemented through the following formula: Given input feature map First, use pooling kernels. and Encode along both the height and width directions to obtain feature maps for two-way perception. and ; Will and After concatenation, the data is transformed by convolution and activated by an activation function. Handling attention weights: ; Decompose f along the spatial dimension into and ; Calculate the attention maps for both directions separately: ; The final output feature map is obtained by positional attention weighting: ; This module uses position-keeping loss as a constraint: ; in and These represent the spatial location encodings of the output and input feature maps, respectively.

[0009] Preferably, the specific process of the physical perception prompt fusion module includes: extracting degenerate prompt features and physical prompt features through two lightweight prompt encoders respectively; splicing and fusing the two prompt features to generate comprehensive prompt features; calculating modulation parameters based on the comprehensive prompt features, and modulating the feature map of the corresponding level of the encoder.

[0010] Preferably, the physical perception prompt fusion module is implemented through the following process: Let the degradation-specific prompt be The physical parameters of the superlens are indicated as follows: ; Cue features are extracted using two lightweight cue encoders: ; The final prompt features are generated using a splicing and fusion strategy: ; Calculate modulation parameters using a linear layer: ; In the encoder's first l Layer, which modulates the feature map using cueing: ; This module optimizes by hinting at the fusion loss: ; in The true feature representation of the corresponding level is obtained through a pre-trained encoder; The encoder structure accurately captures spatial dependencies through a position coordinate attention mechanism, while using a physical perception cue fusion mechanism to inject the physical prior knowledge of the superlens into the feature extraction process, providing rich physical perception feature representations for the subsequent dual-path decoder.

[0011] Preferably, in step S4, the global quality decoder adopts a U-Net symmetric structure, includes multiple upsampling stages, and fuses features of the corresponding level of the encoder through skip connections, and internally adopts a channel attention mechanism; In the global quality decoder, the input physical perception cues are fused with the multi-scale features extracted by the encoder. First, feature enhancement is performed using the feature refinement module: ; in Representation layer normalization, Features corresponding to the encoder level; During the decoding process, a progressive upsampling strategy is adopted, and the output features of each layer are calculated as follows: ; Introducing channel attention mechanisms to enhance key features: ; The final output layer generates an enhanced image through convolution operations: ; The loss function of the global quality decoder includes L1 loss and perceptual loss: ; in This represents a pre-trained VGG network feature extractor.

[0012] Preferably, in step S4, the spatial structure decoder adopts a lightweight dense connection structure, which internally includes an edge perception module, a multi-scale convolution module, and a spatial attention mechanism. The spatial structure decoder recovers structural information through the following process: First, structural features are extracted using the edge sensing module: ; in These are shallow features from the encoder; Capturing structural information at different scales through multi-scale dilated convolution: ; The structural feature reconstruction process employs dense connections: ; ; Introducing spatial attention mechanisms to enhance important structural regions: ; The final structural enhancement output is: ; The loss function of the spatial structure decoder includes edge-preserving loss, structural similarity loss, and location segmentation loss: .

[0013] Preferably, in step S4, the global quality decoder and the spatial structure decoder interact and share information through a cross-attention mechanism; The outputs of the global quality decoder and the spatial structure decoder are adaptively fused using a learnable weighting strategy.

[0014] Preferably, the adaptive fusion is achieved in the following way: ; ; The final output is fused using learnable weights: ; Among them, the fusion weight Obtained through feature importance calculation: ; The total loss function for collaborative training is: ; Interactive loss ensures feature consistency between the two global quality decoders and the spatial structure decoder: ; The training process is constrained by a joint loss function, which includes spatial attention loss, cue fusion loss, global quality loss, and structure preservation loss.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention effectively integrates the imaging physics priors of infrared superlenses into the network by introducing physical parameter cues and cross-modal feature fusion. This enables the model to understand the sources of degradation and imaging constraints, achieving deep synergy between visual information and physical properties during feature extraction. This design not only enhances the model's adaptability to complex imaging systems but also improves the interpretability and robustness of the enhancement results.

[0016] 2. This invention achieves synergistic enhancement of "quality and structure" by employing a dual-path collaborative decoding network that runs the global quality decoder and spatial structure decoder in parallel and fuses their outputs through a feature interaction mechanism. The global quality decoder, combining multi-scale feature fusion and channel attention mechanisms, effectively improves the overall brightness, contrast, and sharpness of the image while suppressing blurring and noise degradation caused by superlens imaging. The spatial structure decoder, through edge perception, multi-scale convolution, and spatial attention mechanisms, strengthens image edges, textures, and structural information, ensuring the integrity of the enhanced image in terms of geometric structure and local details. The outputs of the two branches are adaptively fused through learnable weights to achieve a balance between global quality and local details.

[0017] 3. The network structure of this invention is optimized end-to-end under joint loss constraints, comprehensively considering global quality loss, structure preservation loss, and feature consistency loss, so that the enhanced image achieves an optimized state in terms of both overall visual effect and structural fidelity. This not only significantly improves the image clarity and structural accuracy of infrared superlens images, but also enhances the network's generalization ability to different degradation modes, making it suitable for various engineering applications such as scientific research, industrial inspection, and high-precision infrared imaging. Attached Figure Description

[0018] Figure 1 This is a flowchart of the present invention; Figure 2 This is a system architecture diagram of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please see Figure 1-2 An infrared superlens image quality enhancement method based on dual-path decoding and physical perception cues fusion includes the following steps: Step one involves designing a physical parameter prompting mechanism based on the research problem of this project. This includes describing the physical characteristics of the infrared superlens imaging system and numerically characterizing the metasurface structure parameters. Addressing the high coupling between structural parameters and spectral response in the infrared superlens imaging system, a physical parameter prompting generation mechanism is first designed. The input consists of the geometric and optical parameters of the infrared superlens, including diameter, focal length, operating wavelength, and numerical aperture. By converting the physical characteristic description into text prompts, the output is a semantic prompt vector to guide subsequent feature extraction.

[0021] Step two involves acquiring a dataset of infrared superlenses with multiple degradation types and dividing the data into training and testing sets. The dataset contains samples in the form of triplets: {physical parameter hints -- degraded infrared images -- clear infrared images}, corresponding to the physical characteristic description of the infrared superlens, the actually acquired degraded image, and the ideal clear image for that scene, respectively. The dimensions of both the degraded and clear infrared images are H×W×C, the physical parameter hint text length is N, where H, W, and C represent the image height, width, and number of channels, respectively, and N represents the maximum number of terms in the text description.

[0022] Step three involves the image encoder extracting visual features from the input infrared metalens image, capturing texture details and degradation patterns. The physical parameter encoder embeds physical parameters such as the metalens's diameter, focal length, and operating wavelength into a physical semantic representation. The cross-modal feature fusion module deeply fuses the physical semantic features with the visual features. Based on a positional coordinate attention mechanism, the module calculates the correlation between the two types of features, generating spatially adaptive modulation weights to achieve targeted enhancement of image features. This process enables the encoder to adaptively adjust the feature response according to the physical characteristics of the specific imaging system, accurately modeling image degradation caused by the metasurface. The final output joint feature representation includes both the original visual information and the physical parameters.

[0023] The encoder, composed of a cross-modal feature fusion module, comprises four downsampling stages. Each stage sequentially performs feature extraction, positional coordinate attention calculation, physical cue fusion, and feature downsampling. This structure employs joint optimization of positional coordinate attention loss and cue fusion loss to ensure both spatial accuracy and physical semantic consistency during feature extraction. Its main advantage lies in its ability to accurately capture long-range dependencies in infrared metalens images and incorporate the metalens's physical parameters as prior knowledge into the feature modulation process, thereby more accurately modeling complex degradation patterns induced by metasurfaces.

[0024] Location Coordinate Attention Module: This module captures long-distance dependencies and preserves location information by encoding features along both the horizontal and vertical directions. Given an input feature map... First, use pooling kernels. and Encode along both the height and width directions to obtain feature maps that are perceptible in two directions.

[0025] 1. After concatenating the features from two directions, the attention weights are obtained through convolution and activation functions: Will Split along spatial dimensions and Calculate the attention maps for both directions separately: The final output feature map is obtained by positional attention weighting: This module uses position-keeping loss as a constraint: in and These represent the spatial location encodings of the output and input feature maps, respectively.

[0026] 2. Physical Perception Cue Fusion Module: Building upon traditional degenerate specific cues, this method introduces the physical parameters of the superlens as additional cues. The specific process is as follows: Let the degradation-specific prompt be The physical parameters of the superlens are indicated as follows: .

[0027] Cue features are extracted using two lightweight cue encoders: The final prompt features are generated using a splicing and fusion strategy: Calculate modulation parameters using a linear layer: In the encoder's first Layer, which modulates the feature map using cueing: This module optimizes by hinting at the fusion loss: in The true feature representation for the corresponding level is obtained through a pre-trained encoder.

[0028] This encoder structure accurately captures spatial dependencies through a position coordinate attention mechanism, while simultaneously using a physical perception cue fusion mechanism to inject the physical prior knowledge of the superlens into the feature extraction process, providing rich physical perception feature representations for the subsequent dual-path decoder.

[0029] Step four: This method introduces a dual-path collaborative decoding network (DCDNet) to combine the advantages of global quality enhancement and local structure preservation. This network consists of two parallel branches: a global quality decoder and a spatial structure decoder, forming a collaborative enhancement mechanism of "quality-structure". The global quality decoder restores the overall contrast and sharpness of the image through multi-scale feature fusion and attention mechanisms; the spatial structure decoder focuses on edge preservation and texture detail reconstruction, ensuring the structural integrity of the enhanced image. The two decoders share information through a feature interaction module and are jointly optimized through a collaborative loss function, thereby improving the overall image quality while preserving rich detail information.

[0030] The global quality decoder employs a U-Net encoding / decoding architecture, comprising four upsampling stages. Each stage fuses features from the corresponding encoder level via skip connections. This decoder adaptively adjusts feature weights through a channel attention mechanism, focusing on restoring the overall contrast, brightness, and sharpness of the image while eliminating blur and noise degradation caused by superlensing.

[0031] The spatial structure decoder employs a lightweight, densely connected structure, focusing on the restoration of spatial details in images. This decoder includes an edge-aware module, a multi-scale convolution module, and a structure reconstruction module. Through position-sensitive convolution and spatial attention mechanisms, it emphasizes enhancing structural information such as edges and textures in the image, maintaining the geometric consistency of the enhanced image.

[0032] 1. The global quality decoder adopts an encoder-decoder symmetrical structure, and the specific process is as follows: Input physical perception prompts fused with multi-scale features extracted by the encoder First, feature enhancement is performed using the feature refinement module: in Representation layer normalization.

[0033] During the decoding process, a progressive upsampling strategy is adopted, and the output features of each layer are calculated as follows: Introducing channel attention mechanisms to enhance key features: The final output layer generates an enhanced image through convolution operations: The loss function of the global quality decoder includes L1 loss and perceptual loss: in This represents a pre-trained VGG network feature extractor.

[0034] 2. The spatial structure decoder focuses on the recovery of image structural information and employs a densely connected structure: First, structural features are extracted using the edge sensing module: Capturing structural information at different scales through multi-scale dilated convolution: The structural feature reconstruction process employs dense connections: Introducing spatial attention mechanisms to enhance important structural regions: The final structural enhancement output is: The loss function of the spatial structure decoder includes edge-preserving loss, structural similarity loss, and location segmentation loss: 3. The two decoders share information through a cross-attention mechanism: The final output is fused using learnable weights: Among them, the fusion weight Obtained through feature importance calculation: The total loss function for collaborative training is: Interactive loss ensures feature consistency between the two decoders: This dual-path collaborative decoding network achieves comprehensive quality improvement of infrared superlens images by organically combining global quality enhancement and spatial structure preservation, combined with the constraint of position segmentation loss. It enhances the overall contrast of the image while maintaining rich detail information and structural integrity.

[0035] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for enhancing the image quality of infrared superlens based on the fusion of dual-path decoding and physical perception cues, characterized in that, Includes the following steps: Step S1: Physical parameter prompt generation. Obtain the geometric and optical characteristic parameters of the infrared superlens and convert the parameters into a textual prompt vector. Step S2: Dataset construction. Collect infrared superlens image data containing multiple degradation types, construct samples in the form of triples {physical parameter hints, degraded infrared images, clear infrared images}, and divide them into training set and test set. Step S3: Cross-modal feature encoding and fusion. The image encoder is used to extract deep features from the input degraded infrared image to obtain visual features; at the same time, the physical parameter encoder is used to embed the generated textual cue vector to obtain physical features; the visual features and physical features are input into the cross-modal feature fusion module for fusion to obtain fused features. The cross-modal feature fusion module calculates spatial correlation through a position coordinate attention mechanism, generates adaptive modulation weights, enhances key region features, and fuses degraded cue features with physical cue features through a physical perception cue fusion module to generate a comprehensive cue vector for dynamically adjusting the distribution of multi-layer features. Step S4, Dual-path Decoding and Reconstruction: The obtained fused features are input into a dual-path collaborative decoding network, which includes a parallel global quality decoder and a spatial structure decoder. The global quality decoder is used to decode image domain features to improve the overall image quality index, and the spatial structure decoder is used to decode spectral domain or structural domain features to preserve the details and structure of the image. The outputs of the two decoders are fused to obtain an enhanced infrared superlens image.

2. The infrared superlens image quality enhancement method based on dual-path decoding and physical perception cues fusion as described in claim 1, characterized in that, In step S1, the geometric and optical characteristic parameters include the diameter, focal length, operating wavelength, and numerical aperture of the infrared superlens.

3. The infrared superlens image quality enhancement method based on dual-path decoding and physical perception cues fusion according to claim 2, characterized in that, In step S3, the cross-modal feature fusion module includes multiple downsampling stages, each of which sequentially performs feature extraction, location coordinate attention calculation, physical perception cue fusion, and feature dimensionality reduction.

4. The infrared superlens image quality enhancement method based on dual-path decoding and physical perception cues fusion according to claim 3, characterized in that, The specific process of the position coordinate attention mechanism includes: encoding the input feature map along the height and width directions respectively to obtain feature maps that are perceived in two directions; concatenating, transforming and activating the feature maps in the two directions, and then splitting them into attention weights in the height and width directions; using the attention weights to spatially weight the input feature map to obtain the output feature map; The position coordinate attention mechanism is implemented through the following formula: Given input feature map First, use pooling kernels. and Encode along both the height and width directions to obtain feature maps for two-way perception. and ; Will and After concatenation, the data is transformed by convolution and activated by an activation function. Handling attention weights: ； Decompose f along the spatial dimension into and ; Calculate the attention maps for both directions separately: ； The final output feature map is obtained by positional attention weighting: ； This module uses position-keeping loss as a constraint: ； in and These represent the spatial location encodings of the output and input feature maps, respectively.

5. The infrared superlens image quality enhancement method based on dual-path decoding and physical perception cues fusion according to claim 4, characterized in that, The specific process of the physical perception cue fusion module includes: extracting degenerate cue features and physical cue features through two lightweight cue encoders respectively; splicing and fusing the two cue features to generate comprehensive cue features; calculating modulation parameters based on the comprehensive cue features, and modulating the feature map of the corresponding level of the encoder.

6. The infrared superlens image quality enhancement method based on dual-path decoding and physical perception cues fusion according to claim 5, characterized in that, The physical perception prompt fusion module is implemented through the following process: Let the degradation-specific prompt be The physical parameters of the superlens are indicated as follows: ; Cue features are extracted using two lightweight cue encoders: ； The final prompt features are generated using a splicing and fusion strategy: ； Calculate modulation parameters using a linear layer: ； In the encoder's first l Layer, which modulates the feature map using cueing: ； This module optimizes by hinting at the fusion loss: ； in The true feature representation of the corresponding level is obtained through a pre-trained encoder; The encoder structure accurately captures spatial dependencies through a position coordinate attention mechanism, while using a physical perception cue fusion mechanism to inject the physical prior knowledge of the superlens into the feature extraction process, providing rich physical perception feature representations for the subsequent dual-path decoder.

7. The infrared superlens image quality enhancement method based on dual-path decoding and physical perception cues fusion according to claim 6, characterized in that, In step S4, the global quality decoder adopts a U-Net symmetric structure, which includes multiple upsampling stages and fuses the features of the corresponding level of the encoder through skip connections. It also employs a channel attention mechanism internally. In the global quality decoder, the input physical perception cues are fused with the multi-scale features extracted by the encoder. First, feature enhancement is performed using the feature refinement module: ； in Representation layer normalization, Features corresponding to the encoder level; During the decoding process, a progressive upsampling strategy is adopted, and the output features of each layer are calculated as follows: ； Introducing channel attention mechanisms to enhance key features: ； The final output layer generates an enhanced image through convolution operations: ； The loss function of the global quality decoder includes L1 loss and perceptual loss: ； in This represents a pre-trained VGG network feature extractor.

8. The infrared superlens image quality enhancement method based on dual-path decoding and physical perception cues fusion according to claim 7, characterized in that, In step S4, the spatial structure decoder adopts a lightweight dense connection structure, which includes an edge perception module, a multi-scale convolution module and a spatial attention mechanism. The spatial structure decoder recovers structural information through the following process: First, structural features are extracted using the edge sensing module: ； in These are shallow features from the encoder; Capturing structural information at different scales through multi-scale dilated convolution: ； The structural feature reconstruction process employs dense connections: ； ； Introducing spatial attention mechanisms to enhance important structural regions: ； The final structural enhancement output is: ； The loss function of the spatial structure decoder includes edge-preserving loss, structural similarity loss, and location segmentation loss: 。 9. The infrared superlens image quality enhancement method based on dual-path decoding and physical perception cues fusion according to claim 8, characterized in that, In step S4, the global quality decoder and the spatial structure decoder interact and share information through a cross-attention mechanism. The outputs of the global quality decoder and the spatial structure decoder are adaptively fused using a learnable weighting strategy.

10. The infrared superlens image quality enhancement method based on dual-path decoding and physical perception cues fusion according to claim 9, characterized in that, The adaptive fusion is achieved in the following way: ； ； The final output is fused using learnable weights: ； Among them, the fusion weight Obtained through feature importance calculation: ； The total loss function for collaborative training is: ； Interactive loss ensures feature consistency between the two global quality decoders and the spatial structure decoder: ； The training process is constrained by a joint loss function, which includes spatial attention loss, cue fusion loss, global quality loss, and structure preservation loss.

Images