Infrared image stripe removal method and system based on physical prior

Abstract

This invention discloses a method and system for destriating infrared images based on physical priors. The invention incorporates an asymmetric receptive field module at the network head, employing parallel 7×1 vertical convolution and 3×3 spatial convolution paths to extract and fuse stripe and spatial features. By improving column attention, it generates column weights by coupling mean and extreme value statistics along the height dimension using double pooling, accurately correcting column noise. The network only predicts the stripe noise map, restoring the clean image based on a physical subtraction model. A weighted joint loss function is constructed, including noise loss, column consistency constraint loss, and reconstruction loss, constraining the noise to conform to the physical characteristics of vertical distribution. This invention is equipped with a dedicated processing system and hardware. It aligns with the stripe generation mechanism, significantly improving the accuracy of infrared image destriating, eliminating lateral artifacts, and finely preserving image details. It is suitable for image preprocessing in various infrared detection devices.

Description

Technical Field

[0001] This invention relates to a method and system for removing stripes from infrared images based on physical priors. More specifically, it relates to a method, system, and hardware device for removing vertical stripes from infrared images based on physical prior constraints of infrared imaging. It can be applied to imaging equipment of cooled and uncooled infrared detectors and is widely applicable to infrared image preprocessing operations in scenarios such as photoelectric detection, infrared security monitoring, industrial infrared non-destructive testing, and airborne infrared reconnaissance. It belongs to the field of infrared image processing and deep learning. Background Technology

[0002] When acquiring scene information, infrared imaging systems inevitably generate vertical stripe noise due to the non-uniformity of detector response. Most existing denoising methods based on convolutional neural networks (CNNs) treat this as a "black box" mapping problem, ignoring the specific physical form of the stripe noise.

[0003] The shortcomings of existing technologies are: 1) The receptive field is isotropic, making it impossible to focus on the directional distribution of vertical stripes; 2) The attention mechanism in the feature fusion stage is redundant, resulting in the loss of the most critical one-dimensional column prior information; 3) The model inference logic usually adopts residual addition, which is not compatible with the physical process of stripe superposition, leading to insufficient stripe removal purity and easy artifacts.

[0004] To address the shortcomings of existing technologies, this invention proposes the PASCNet network, which improves upon existing technologies in four dimensions: network structure design, attention mechanism reconstruction, physical imaging model implementation, and multi-constraint loss function design. It achieves targeted denoising by relying on the physical prior of stripe generation. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing infrared destriating networks, such as non-directional receptive field, loss of prior information due to attention, reconstruction logic that violates the physical laws of infrared imaging, and easy loss of details in image processing. Based on the physical prior of non-uniform noise of the detector, the overall architecture of PASCNet is designed. Under the premise of completely removing vertical stripe noise in the image, it retains the infrared image edges, small heat source targets and background radiation details to the maximum extent, thereby improving the peak signal-to-noise ratio (PSNR) of the denoised image.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solution: This invention provides a method for destriping infrared images based on physical priors, comprising: S1. Acquire noisy infrared images collected on-site. H is the image height in pixels, and W is the image width in pixels. The image is input into the PASCNet model that has been trained offline in advance. S2. Asymmetric head parallel KX1 vertical convolution and 3X3 spatial convolution, where K takes the value of an odd number in the range [5, 15]. Vertical convolution captures vertical stripe features, and spatial convolution extracts spatial details of the image, resulting in: ; The two features are concatenated after BN and ReLU, and then fused using a 1x1 convolution. ;in, The stripe feature map output by the vertical convolution branch; The output is a spatial detail feature map from a 3×3 convolution; BN is a batch normalization operation. This involves concatenating features along the channel dimension. This is the initial comprehensive feature map after the fusion of the two-branch features; For activation functions; S3. Use Haar wavelet DWT for multi-scale coding to obtain feature maps. C represents the number of channels in the feature map; after a one-dimensional column prior attention module performs double global pooling along the height, 1X1 convolution couples statistical features, and a bottleneck structure generates a column-oriented attention weight vector. The column-oriented noise feature correction is completed by multiplying the weights by the feature map X; where, Use the Sigmoid activation function; Activate ReLU; Z represents the convolution weights; Z is the coupling vector. S4. The decoder is upsampled by the PixelShuffle module and features are concatenated across layers. The network only outputs a stripe noise map. From physical models A clean image is obtained by subtracting pixels one by one. The result is a clean infrared image after noise reduction; S5. A joint loss model is used to train the model to preserve the original details of the denoised clean infrared image: ;in, This represents the noise fitting loss; For column consistency loss; To preserve the fidelity of image reconstruction; This represents the total training loss of the model.

[0007] Preferably, the vertical convolution uses a 7x1 convolution kernel, and each branch output undergoes batch normalization (BN) and ReLU activation before channel splicing.

[0008] Preferably, the Haar wavelet DWT splits the features into one low-frequency component and three high-frequency components and splices them together. The shallow features are downsampled by a step size of 2 and then fused with the encoded features by adding the residuals.

[0009] The preferred formula for calculating noise loss is: In the formula, N represents the total number of pixels in the entire image; i is the pixel number. Predict the noise value for the model at the i-th pixel; The actual labeled noise value corresponding to the i-th pixel; For absolute value operations, For summation operations.

[0010] Preferably, the column consistency loss is calculated first by taking the mean of each column: ; Then calculate the loss: In the formula, c is the feature channel number. Image column number, The pixel index in the height direction of the image; For the c-th channel The average pixel value of the column; This represents the expected value calculated across all channels and all columns; H is the total height of the image in pixels.

[0011] Preferably, reconstruction loss In the formula, N is the total number of pixels in the image. A true, clean infrared tag image without stripes. For absolute value operations, This represents the summation operation for all pixels.

[0012] Preferably, the decoder uses the PixelShuffle module for subpixel upsampling to eliminate checkerboard artifacts, and the encoder uses multi-layer shallow feature skipping splicing followed by 1X1 convolution dimensionality reduction before being fed into the decoding enhancement module CNCM.

[0013] The present invention provides an infrared image destriping system based on physical priors. The system includes an asymmetric feature extraction module for signal connections, a column prior attention modulation module, and a physical image reconstruction module. The asymmetric feature extraction module is used to acquire the infrared image to be processed. It uses parallel strip-shaped vertical convolutional layers and standard spatial convolutional layers to extract vertical long-range dependency features and spatial structure features respectively, and performs splicing and weighted fusion in the channel dimension to obtain comprehensive features containing stripe prior information. The column prior attention modulation module is used to perform global pooling on the comprehensive features along the height dimension, extract the mean and maximum features, learn the column-directed noise correlation, generate the column-directed attention weight vector, and perform column-level correction on the feature map to obtain the corrected feature map. The physical image reconstruction module is used to extract and process features from the corrected feature map using a backbone coding network, predict the stripe noise map, and subtract the stripe noise map from the original input image based on the inverse logic of the physical additive model to obtain the final stripe-free clean image.

[0014] The present invention provides a hardware device for removing stripes from infrared images, which implements the method described above. The device includes an infrared imaging system, a data processing center, a memory, and a display. The infrared imaging system is communicatively connected to the data processing center, which in turn is communicatively connected to the memory and the display. The infrared imaging system includes an optical lens and an infrared detector, used to acquire infrared radiation signals from the scene and convert them into raw infrared image frames with stripe noise. The data processing center is internally configured with a high-performance GPU server. The memory is used to store various types of image data and model files. The display is used to show the processed infrared images in real time.

[0015] Preferably, the memory is a high-speed solid-state drive for persistent storage of the original noisy image, the ground truth image, the PASCNet model weight file, and the processed stripe-free image; the GPU server is equipped with the CUDA parallel computing framework, which accelerates convolution operations, wavelet transforms, attention weight calculations, and backpropagation of loss function gradients in hardware parallel.

[0016] Beneficial effects: Excellent image processing accuracy: In the verification test on the general infrared stripe dataset, the peak signal-to-noise ratio (PSNR) of the output image of the algorithm of this invention can reach 43.94dB, which is better than the existing mainstream SOTA stripe removal algorithm. Vertical stripes are completely removed and the residual noise in the image is extremely low. Strong physical interpretability: The overall network architecture and loss function can be designed based on the physical prior of the non-uniform stripes of the infrared detector, thus getting rid of the drawbacks of black box fitting. It has excellent generalization performance and can stably complete denoising in different types of infrared detectors and indoor and outdoor multi-modal imaging scenarios, with excellent generalization and interpretability. Outstanding detail preservation: Asymmetric convolution directional recognition of stripes, column attention only corrects column noise, and reconstruction loss constrains image fidelity. It can effectively distinguish high-frequency stripe noise from high-frequency edges of objects in the image, avoid the problem of loss of small targets and contour details caused by excessive smoothing of conventional algorithms, and generate no lateral artifacts. Lightweight and computationally efficient: StripeCALayer uses full-channel feature coupling instead of traditional attention feature decoupling and segmentation, which simplifies the number of model parameters and computation. Combined with sub-pixel upsampling and lightweight wavelet decomposition, it balances image processing speed and imaging accuracy. It can be deployed on embedded GPU hardware to achieve real-time online noise reduction for infrared devices. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the asymmetric vertical sensing head module structure of the present invention.

[0018] Figure 2This is a diagram of the architecture of the Physical Prior Infrared Image Destriking Network (PASCNet) model of this invention.

[0019] Figure 3 This is the overall architecture diagram of the non-uniform correction module of the present invention.

[0020] Figure 4 This is a diagram of the internal structure of the one-dimensional column prior attention module (StripeCALayer) of this invention.

[0021] Figure 5 This is a schematic diagram of the basic submodule (CSSC) structure of the Column Constraint Features of this invention.

[0022] Figure 6 This is a schematic diagram comparing the image processing effects of the present invention. Detailed Implementation

[0023] The following will refer to the accompanying drawings in the embodiments of the present invention. Figures 1 to 6 The technical solutions in the embodiments of the present invention are clearly and completely described herein. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0024] Figure 1 In the input image, the noisy infrared original image is divided into two parallel convolutional branches: one branch uses a 7×1 vertical strip convolutional kernel to extract vertical stripe noise features; the other branch uses a 3×3 square spatial convolutional kernel to extract infrared image contour and detail texture features. The outputs of both convolutional branches are connected to the L (LeakyReLU activation function) to introduce nonlinear transformation. Finally, the features from the two branches are combined and output to separate stripe noise from effective image details from the input image by channel. The physical prior of stripe distribution is embedded in the front end of the network.

[0025] Figure 2 Chinese: Front-end integration Figure 1 The system employs a dual-branch asymmetric feature extraction structure. The encoder consists of a three-level cascaded residual Haar discrete wavelet transform (RHDWT) and a column non-uniformity correction module (CNCM) to extract deep features layer by layer at multiple scales. The decoder uses sub-pixel upsampling (PixelShuffle) to restore feature resolution. Shallow features from each layer of the encoder are fused across layers via C (channel stitching) to the decoder. At the end of the decoder, multi-level convolution and T (Tanh activation) are used to generate the stripe noise map of the network's predicted output. Finally, using an infrared imaging physical model, the predicted noise map is subtracted pixel by pixel from the input original image to obtain a denoised and clean infrared image.

[0026] Figure 3The module internally consists of two sets of cascaded branches. Each branch sequentially passes through a 3×3 convolution, LeakyReLU activation, and an improved column constraint feature submodule (RCSSC). The branch outputs are then concatenated and fused via the C channel. The fused features are then further processed through a combination of 1×1 and 3×3 convolutions and LeakyReLU activation. Finally, element-wise addition of residuals is used to achieve short-circuit connections, fusing the original input features with the processed features. This module leverages the column constraint submodule to enhance the directional correlation of feature columns, ensuring that the constrained features conform to the vertical stripe distribution pattern and suppressing the generation of horizontal artifacts.

[0027] Figure 4 In the process, the input features are first compressed along the image height dimension using column-wise average pooling (CAP) and column-wise max pooling (CMP). The two pooling results are then concatenated via the C channel. Subsequently, the features are transformed by sequentially performing 1×1 convolution, batch normalization (BN), LeakyReLU, and multi-level 1×1 convolution + ReLU. After transformation, the features are multiplied element-wise with the original input, and then sequentially connected to two auxiliary attention branches: a spatial attention layer (SALayer) and an auxiliary channel attention layer (SCLayer). Finally, the features from multiple branches are concatenated and fused, and the residual is output. Column-wise dual pooling accurately captures the statistical information of single-column pixels, generating column-wise attention weights to achieve noise correction.

[0028] Figure 5 In: Figure 4 An improved column constraint feature submodule (RCSSC) is added as an auxiliary branch above the basic column attention module. The RCSSC branch consists of a 3×3 convolution, a basic column constraint submodule, and a residual structure. The branch output is processed by global average pooling (GAP) and global max pooling (GMP) to extract global statistical features, generating global attention weights. These weights are fused with the column attention features of the main branch, and further optimized using SALayer spatial attention and SCLayer auxiliary channel attention. This structure simultaneously constrains features from both global and single-column dimensions, further improving the accuracy of vertical stripe recognition.

[0029] Figure 6 In the image: Noise (noisy image): the original infrared input image with superimposed vertical stripe noise; GT (denoised pure infrared image, Ground Truth benchmark): a standard reference image without any stripe noise; Result (final output image after stripe removal): the output image processed by the PASCNet algorithm of this invention, which can completely eliminate all vertical stripes in the image, and the details and textures of clouds, cyclones, etc. are highly consistent with the ground truth image, with no loss of detail and no new horizontal artifacts.

[0030] Technical Solution / Content: This invention provides a physical prior-based infrared image stripe removal method, supporting system, and hardware processing device. The overall solution is based on the infrared imaging physical additive model design, that is, the original noisy infrared image is composed of a noise-free clean image superimposed with column-directed vertical stripe noise. The core technologies include five main parts: an asymmetric vertical sensing head module, a Haar wavelet multi-scale coding module, a StripeCALayer one-dimensional column prior attention module, a sub-pixel decoding and physical image reconstruction module, and a multi-weighted joint loss function.

[0031] I. Asymmetric Vertical Sensing Head Module Abandoning the traditional design of a single 3×3 convolution in the first layer of the network, a dual-branch parallel asymmetric convolution architecture is adopted. The output data of both convolutions undergo batch normalization (BN) and ReLU activation processing before channel concatenation. 1.1 Vertical Stripe Perception Branch: A k×1 long vertical convolution is selected, where k is an odd number between [5, 15]. A 7×1 convolution kernel is preferred. Relying on the ultra-long vertical receptive field, a long-range pixel dependency relationship in the image height dimension is established to accurately capture the non-uniform response deviation of cross-row pixels and prioritize the extraction of vertical stripe features. ; 1.2 Spatial Detail Perception Branch: A standard 3×3 isotropic convolution is used to extract the basic spatial contours, edges, and target detail features of the image. ; 1.3 Feature Fusion Processing: The two features are concatenated along the channel dimension, and cross-channel information fusion and dimensionality reduction are achieved through 1×1 convolution. The fusion formula is as follows: ;in, The stripe feature map output by the vertical convolution branch; The output is a spatial detail feature map from a 3×3 convolution; BN is a batch normalization operation. This involves concatenating features along the channel dimension. This is the initial comprehensive feature map after the fusion of the two-branch features; This is the activation function.

[0032] II. Multi-scale coding module based on Haar discrete wavelet DWT Two-dimensional Haar discrete wavelet transform is used to refine the initial feature map (initial synthesized feature map). Multi-scale frequency domain decomposition is performed. A single DWT transform splits the feature map into one low-frequency contour component and three high-frequency detail components (horizontal, vertical, and diagonal). These four components are then concatenated along the channel dimension to achieve frequency domain reconstruction, separating high-frequency stripe noise from low-frequency background information. The reconstructed features are fed into a convolutional coding layer to complete deep nonlinear feature mapping. Simultaneously, local residual skip connections are implemented. Before encoding, shallow features are downsampled and convolved with a stride of 2 for size alignment, and then added element-wise with the encoded output features. This avoids gradient vanishing and loss of shallow background details during deep network training, ultimately outputting a deep feature map. C represents the number of feature channels, H is the number of pixels in the image height, and W is the number of pixels in the image width.

[0033] III. One-Dimensional Column Prior Attention Module (StripeCALayer) By leveraging the physical characteristics of infrared stripes—namely, "high correlation of pixels within a single column and statistical independence between columns"—the traditional channel attention mechanism is reconstructed. This reconstructs the image by compressing only the height dimension and retaining the width dimension to generate one-dimensional column features, achieving precise column-level noise correction. 3.1 Two-Dimensional Global Pooling: For the input feature map X, global average pooling and global max pooling are performed along the image height H to extract the average radiance of each column of pixels and the extreme value of the noise peak in each column, respectively, resulting in two sets of... A one-dimensional column statistical vector of specifications; 3.2 Statistical Feature Coupling: Two types of statistical vectors are concatenated along the channel dimension to form... The composite vector is obtained by compressing the channels through a 1×1 convolution. The coupling vector Z enables deep coupling of two types of statistical information: column mean and column extreme value, and uncovers the global and local nonlinear correlations of noise. 3.3 Bottleneck Structure for Attention Weight Generation: The coupling vector Z is fed into a bottleneck structure consisting of two 1×1 convolutions. The first 1×1 convolution compresses the channels to 1 / r of the original number of channels, where r is the channel reduction rate. One-dimensional column-directed convolutions are used to smooth the local information exchange between adjacent columns of pixels. After ReLU activation, the second 1×1 convolution restores the original number of channels, and then Sigmoid activation and normalization generate the column-directed attention weight vector. In the formula, It is the Sigmoid activation function. It is the ReLU activation function. These are the weight parameters for the two convolutional layers in the bottleneck structure, respectively. 3.4 Feature Modulation: Weighting Expand the image along the height direction to the same size as the feature map X, multiply it pixel by pixel with the original feature map X, amplify the feature weights of the columns containing noise, and suppress the interference of the noiseless background columns to obtain the corrected feature map.

[0034] IV. Subpixel upsampling decoding and physical noise reconstruction module 4.1 Decoding Upsampling: The decoder uses the PixelShuffle module (sub-pixel convolution) to achieve feature upsampling. It relies on channel recombination to improve feature resolution and avoids the chessboard artifacts and edge blurring caused by deconvolution and bilinear interpolation from the root. 4.2 Cross-layer global skip connection (SkipConnection): The shallow detail features retained by each layer of the encoder and the decoding features are concatenated (Concat) in the channel dimension. The redundant features after concatenation are compressed by 1×1 convolution and then sent to the CNCM decoding enhancement module. 4.3 Noise Output: A 1×1 convolutional layer with Tanh activation is configured at the end of the network. The network only outputs a pure stripe noise map of the entire image. ; 4.4 Physical Image Reconstruction: Strictly follows the infrared imaging physical subtraction model for pixel-by-pixel calculation. ;in, To input a noisy infrared image, To obtain a clean infrared image after denoising; abandoning the black-box approach of directly outputting a clean image, simplifying the network fitting task to single noise prediction, reducing the difficulty of model training, and achieving physical inverse decoupling between scene content and stripe noise.

[0035] V. Joint Constraint Loss Function for Three Objectives To constrain the output noise shape of the model to conform to the physical property of vertical stripes during the training phase, a weighted composite total loss function is designed: Where 0.1 is the column consistency loss weighting coefficient, 0.5 is the image reconstruction loss weighting coefficient, and the three loss terms work together to optimize the network parameters. 5.1 Noise Fitting Loss (Main Loss Term): The predicted noise is constrained by L1 mean absolute error to approximate the actual labeled noise. The calculation formula is as follows: N represents the total number of pixels in the entire image, and i is the pixel index. The model predicts the noise value at the i-th pixel. The actual labeled noise value corresponding to the i-th pixel. For absolute value operations, For summation operations; this term dominates the network and allows for rapid convergence, locking in the main distribution pattern of the stripes; 5.2 Column Consistency Loss (Physical Constraints): By minimizing the variance of pixels in a single column, the noise generated by the model is forced to have consistent values ​​within the same column, ensuring that the noise has a long, vertical strip shape and suppressing horizontal artifacts; first, the mean value of pixels in a single column is calculated: ; Next, calculate the expected variance of all channels and all columns: In the formula, c is the feature channel number, w is the image column number, and h is the pixel number in the image height direction. The average pixel value in the w-th column of the c-th channel. This represents the expected value calculated across all channels and all columns, where H is the total height of the image in pixels. 5.3 Image Reconstruction Loss (Fidelity constraint): Generate a restored image based on a physical model, and a stripe-free, realistic, and clean infrared tag image. Apply L1 loss constraints, formula: N is the total number of pixels in the image. A true, clean infrared tag image without stripes. For absolute value operations, This represents the summation of all pixels; this prevents the network from over-predicting noise and causing background details to be incorrectly removed.

[0036] During the model training phase, the Adam optimizer was used, and the convolutional weights and bias parameters of the entire PASCNet network were iteratively updated by backpropagation of the total loss to complete the model convergence training.

[0037] VI. Supporting Systems and Hardware Devices 6.1 Infrared Image Destriping System: The system consists of an asymmetric feature extraction module, a column prior attention modulation module, and a physical image reconstruction module, all interconnected by signals. The asymmetric feature extraction module acquires the infrared image to be processed, extracts vertical long-range dependency features and spatial structure features using parallel strip-shaped vertical convolutional layers and standard spatial convolutional layers, respectively, and performs concatenation and weighted fusion along the channel dimension to obtain a comprehensive feature containing stripe prior information. The column prior attention modulation module performs global pooling on the comprehensive feature along the height dimension, extracts mean and maximum features, performs column-wise noise correlation learning, generates a column-wise attention weight vector, and performs column-level correction on the feature map to obtain a corrected feature map. The physical image reconstruction module uses a backbone coding network to extract and process features from the corrected feature map, predicts the stripe noise map, and subtracts the stripe noise map from the original input image based on the inverse logic of the physical additive model to obtain the final destriped clean image.

[0038] 6.2 Hardware Device for Infrared Image Destriking: The device includes an infrared imaging system, a data processing center, a memory, and a display. The infrared imaging system is communicatively connected to the data processing center, which in turn is communicatively connected to the memory and the display. The infrared imaging system includes an optical lens and an infrared detector, used to acquire infrared radiation signals from the scene and convert them into raw infrared image frames with stripe noise. The data processing center is equipped with a high-performance GPU server. The memory is used to store various image data and model files. The display is used to show the processed infrared images in real time. The memory uses a high-speed solid-state drive for persistent storage of the raw noisy image, ground truth labeled image, PASCNet model weight file, and the processed destriking image. The GPU server is equipped with the CUDA parallel computing framework, which accelerates convolution operations, wavelet transforms, attention weight calculations, and backpropagation of loss function gradients in hardware parallel.

[0039] Example 1: Specific steps of the physical prior-based infrared image destriping (PASCNet) method proposed in this invention: Step 101) Acquire noisy infrared images and perform front-end feature perception.

[0040] This step acquires the raw, noisy infrared image using a sensor. Addressing the inherently strong directionality of vertical stripe noise generated by the non-uniformity of infrared detectors, and overcoming the limitation of conventional isotropic convolutions in distinguishing between "inter-column noise response" and "scene structure information" during feature extraction, this invention abandons the conventional design of using a single 3x3 convolution as the starting layer and innovatively constructs an Asymmetric Vertical Perception Head. This perception head simulates the preprocessing mechanism of human vision for directional noise, working collaboratively through the following parallel dual-branch mechanism: (1.1) Branch A (Directional Vertical Perception Branch): Using an asymmetric convolution kernel (such as 7X1 or 5X1, this invention uses a 7X1 convolution kernel in this step), the statistical correlation of pixels within a column is directly established in the shallowest layer of the image by utilizing its long receptive field in the vertical dimension. This branch aims to perform low-pass smoothing on the background texture while accurately focusing on the high-frequency stripe features in the vertical direction, thereby achieving preliminary noise pattern capture.

[0041] (1.2) Branch B (Multidimensional Spatial Perception Branch): Employs a standard 3x3 convolution kernel and is responsible for extracting the basic spatial structure features and detailed edges of the image. This branch serves as an auxiliary branch to ensure that the inherent topological relationships and target semantic information of the image are fully preserved during noise recognition.

[0042] Feature fusion and dimensionality reduction: To deeply integrate the two types of dissimilar features, the system concatenates the column prior features of branch A with the spatial context features of branch B in the channel dimension. Subsequently, a 1x1 convolutional layer is introduced as a non-linear fusion operator to perform cross-channel information interaction and feature dimensionality reduction, thereby eliminating feature redundancy and obtaining an initial feature map (initial integrated feature map) with anisotropic perception capabilities. Its mathematical expression model is as follows: ; ; ;in, This indicates a splicing operation. This represents the activation function. Through this design, the network possesses "prior sensitivity" to stripe noise at the input, laying the foundation for subsequent deep feature extraction.

[0043] Step 102) Perform multi-scale frequency domain deconstruction and depth coding on the feature map.

[0044] To effectively separate stripe noise from background structure in infrared images, this invention employs a multi-scale downsampling and residual fusion mechanism based on Discrete Wavelet Transform (DWT). The specific process is as follows: (2.1) Frequency band decomposition and reconstruction: Using a two-dimensional discrete wavelet transform (DWT) module based on Haar wavelets, the input feature map is decomposed losslessly into one low-frequency component and three high-frequency detail components. This operation expands the network's receptive field without losing the energy of the original feature signal. Subsequently, these four frequency band components are concatn in the channel dimension to achieve spatial deconstruction of stripe noise with specific frequency domain morphology (especially in the high-frequency vertical direction); (2.2) Deep convolution feature extraction: The recombined feature map is fed into a single-layer or double convolutional coding module to perform high-dimensional nonlinear feature mapping and extract deep semantic information; (2.3) Local residual skip connections: To prevent the attenuation of basic background information caused by deep networks, local residual connections are introduced. The shallow features before encoding are spatially aligned by a downsampling convolutional layer with a stride of 2, and then added element by element with the features output by the above encoding module to generate a deep feature stream containing multi-scale context, which is then fed into the subsequent enhancement module (CNCM).

[0045] Step 103) In a specific feature layer, reconstruct the vertical weights using an improved specialized column attention mechanism (StripeCALayer).

[0046] To address the problem that conventional two-dimensional spatial attention mechanisms easily lead to dimensional information redundancy and lateral artifacts when processing highly directional vertical stripes, this invention proposes a one-dimensional attention module specialized for stripe morphology. The specific steps are as follows: (3.1) Calculate the one-dimensional column multi-source descriptor. Assume the input feature map is... This invention utilizes both the Adaptive Average Pooling operator (AdaptiveAvgPool) and the Adaptive Max Pooling operator (AdaptiveMaxPool) to forcibly perform global compression along the height direction (compressing the size to...). These two pooling features are concatenated along the channel dimension to simultaneously capture the "global response mean" and "local extrema (striated strong noise response)" along the column dimension, generating a high-fidelity initial column descriptor.

[0047] (3.2) Feature fusion and dimensionality alignment. The concatenated column descriptors are input into a 1X1 convolutional layer, a batch normalization layer (BatchNorm), and a LeakyReLU activation function for cross-channel fusion and dimensionality reduction to obtain smoothed one-dimensional features. .

[0048] (3.3) Extract column prior weights. (This involves extracting the prior weights of the features.) The input is fed into a nonlinear bottleneck structure consisting of two 1x1 convolutional layers (first reducing the number of channels to a lower dimension, then increasing the dimension to restore it), and the final column prior weight vector is extracted. Its mathematical expression is: ;in It is the Sigmoid activation function. It is the ReLU activation function. and This represents the weights of the convolutional layer.

[0049] (3.4) Aggregation and Modulation Output. The generated column prior weight vector is broadened in the height direction and multiplied pixel by pixel with the original feature map X to achieve feature modulation. Through the above-described extremely simple forced locking design, the network can directly focus on and amplify specific column energy levels in the fringe set, fundamentally eliminating gradient interference in the height direction.

[0050] Step 104) Perform multi-scale image decoding and stripe noise map regression.

[0051] In the upsampling and reconstruction stage of network features, pure stripe features are regressed through structural reparameterization and multi-level fusion: (4.1) Subpixel lossless upsampling: The PixelShuffle module (subpixel convolution) is used to replace the traditional interpolation or deconvolution algorithm to increase the resolution of the feature map. This method avoids the checkerboard effect during image reconstruction by rearranging the pixels in the channel dimension, and eliminates channel artifacts through subsequent smooth convolution layers.

[0052] (4.2) Global cross-layer feature bridging: Through global skip connections, the shallow high-frequency detail features retained by each layer of the encoder are precisely concatenated with the current decoding features in the channel dimension. The concatenated redundant features are then compressed by a specific 1X1 convolutional layer and sent to the decoder enhancement module for processing.

[0053] (4.3) Noise Mapping and Global Residual Regression: After the feature flow reaches the end of the network, a mapping tail layer consisting of 1X1 convolutions and the Tanh function outputs the regressed stripe components. Finally, using a global residual learning mechanism, this component is mapped to the original noisy input image. By combining and separating / regressing, a clean, restored image and a stripe noise map are obtained.

[0054] Step 105) Constrain and optimize the network based on the physical subtraction model.

[0055] Breaking away from the traditional "black box" mapping logic of networks directly outputting clean images, this invention strictly defines the network inference target as a pure noise term.

[0056] Perform rigorous physical subtraction and reconstruction loss calculation. Subtract the original noisy image... Subtract the fringe noise map predicted by the network This yields the final clean restored image. : ; Step 106) Collaborative optimization of loss function based on multi-objective constraints.

[0057] This invention constructs a three-in-one joint loss function. By balancing explicit noise supervision, physical morphological constraints, and image reconstruction, the network is guided to accurately remove stripe noise. The specific optimization steps are described below: (6.1) Calculate the noise reconstruction loss (Dominant Loss). This invention directly defines the network output target as the stripe noise map. .use Norm computation network predicts noise With tag noise The mean absolute error between the two. Since the noise distribution is sparse relative to the entire image, this loss term enables the network to converge quickly and capture the main energy of the stripes: ; (6.2) Calculate the prior loss for column consistency (Physical Constraints). To suppress potential lateral artifacts during denoising, the noise generated by the network is forced to have physical consistency in the vertical direction. Apply variance constraints in the column direction: Calculate column mean descriptor: Calculate predicted noise plot Along the height dimension The average value is obtained. : ; Calculate column variance loss: Calculate the dispersion of each pixel in the noise map relative to the mean of its column. By minimizing this variance, the noise is forced to exhibit a strictly vertical stripe distribution. ; (6.3) Calculate the global image reconstruction loss Based on the physical subtraction model Derive the restored image. Calculate the label of the restored image compared to the original clean image. Between Loss. This term serves as a global constraint to prevent the network from overestimating noise and causing a loss of background details. ; (6.4) Weighted Fusion of Joint Loss Function. To achieve optimal destriating effect while preserving the background, this invention introduces weight coefficients of 0.1 and 0.5 to linearly weight and fuse the above three losses, forming the total loss function: Using this joint loss function, the network parameters are backpropagated and updated through the Adam optimizer, so that the network can preserve the original details of the infrared image to the maximum extent while ensuring the purity of stripe removal.

[0058] Example 2: Offline infrared image destriping implementation (The specific parameters involved in this example are only a specific description of the technical solution and should not be construed as a limitation on the technical solution) S1. Dataset Preprocessing: A publicly available infrared non-uniform stripe dataset was selected and divided into a training set and a validation set; each sample set included a noisy input image. Truth Value Clean Image Manually annotated real stripe noise map The image pixels are uniformly normalized to the [0,1] interval, and the image size is standardized. ; S2, PASCNet Network Construction: 2.1 Asymmetric Head: The vertical branch uses 7×1 convolution, and the spatial branch uses 3×3 convolution. The output channels of both branches are uniformly set to 32. After the branch outputs are processed by BN+ReLU, the channels are concatenated to obtain 64-channel features. These features are then compressed to 32 channels by 1×1 convolution to obtain the initial feature map (initial comprehensive feature map). ; 2.2 Encoding Stage: The Haar-DWT wavelet module is fed into the Haar-DWT wavelet module to complete a single multi-scale decomposition. The four sub-band feature channels are spliced ​​together and then fed into a two-layer CNCM convolutional coding module to output 64 feature channels and 128X128 feature size. 2.3 StripeCALayer configuration: Channel reduction rate r=4, bottleneck middle one-dimensional column convolution kernel is 3, double global pooling is used to obtain column statistical features, after coupling, attention weights are generated by two layers of 1×1 convolution, and feature modulation is completed column by column; 2.4 Decoding Stage: Multi-level PixelShuffle subpixel upsampling; the three shallow features of the encoder are spliced ​​to the corresponding decoding layers in a skip manner; the final 1×1 convolution combined with Tanh activation outputs a single-channel noise map. ; S3. Model Training Configuration: Adam optimizer is selected, initial learning rate 1x10⁻¹⁰. -4 Single training batch size = 8; total loss coefficient is fixed. Weight 0.1 The weight is 0.5; the loss values ​​are calculated according to the corresponding formulas for the three losses, and the weighted sum is then backpropagated to update the network parameters. The model converges after 80 iterations, and the optimal model weight file is saved. S4. Image Inference Test: Input the noisy infrared image to be tested into the trained PASCNet model, and the network outputs a noisy image through forward inference. ,according to The denoised image was obtained by pixel-by-pixel operation; the average PSNR of the test dataset reached 43.94dB, the vertical stripes of the image were completely eliminated, the edges of small targets such as buildings and heat sources were completely preserved, and there were no horizontal stripe artifacts.

[0059] Example 3: Online real-time image processing using hardware devices (The specific product specifications involved in this example are merely a specific description of the technical solution and should not be construed as a limitation on the technical solution). Hardware components: uncooled optical lens, infrared detector, RTX3090 GPU server, SSD high-speed solid-state drive, industrial LCD display; The infrared imaging system acquires infrared video streams of outdoor scenes upon power-up, and the infrared detector outputs single-channel noisy infrared image frames in real time, which are then transmitted to the GPU server via the network port. The GPU server loads the PASCNet weight file trained in Example 1, and accelerates convolution, wavelet transform, and attention operations in parallel using the CUDA framework. The processing time for a single frame image is less than 15ms, which meets the real-time imaging processing requirements of the device. The de-striped image generated by the physical subtraction model is stored in an SSD solid-state drive for archiving and saved in one path, and pushed to an industrial LCD monitor in real time for real-time preview. Continuous 24-hour field environment testing showed that column stripes could be stably removed under scenarios with varying day-night temperature differences and random switching between near and far targets. Gradual stripes caused by temperature drift of the low-temperature environment detector were also effectively suppressed.

[0060] Finally, it should be noted that the present invention is not limited to the above embodiments, and many variations are possible. All variations that can be directly derived or conceived by those skilled in the art from the disclosure of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for removing stripes from infrared images based on physical priors, characterized in that, include: S1. Acquire noisy infrared images collected on-site. H is the image height in pixels, and W is the image width in pixels. The image is input into the PASCNet model that has been trained offline in advance. S2, Asymmetric head parallel KX1 vertical convolution and 3X3 spatial convolution, where the value of K is an odd number in the range of [5,15]. Vertical convolution captures vertical stripe features, and spatial convolution extracts spatial details of the image. S3. Use Haar wavelet DWT for multi-scale coding to obtain feature maps. C represents the number of channels in the feature map; after a one-dimensional column prior attention module performs double global pooling along the height, 1X1 convolution couples statistical features, and a bottleneck structure generates a column-oriented attention weight vector. The column-oriented noise feature correction is completed by multiplying the weights by the feature map X; where, Use the Sigmoid activation function; Activate ReLU; Z represents the convolution weights; Z is the coupling vector. S4. After upsampling and cross-layer feature concatenation, the decoder outputs only a stripe noise map. From physical models A clean image is obtained by subtracting pixels one by one. This is the clean infrared image after noise reduction; S5. A joint loss model is used to train the model to preserve the original details of the denoised clean infrared image: ;in, This represents the noise fitting loss; For column consistency loss; To preserve the fidelity of image reconstruction; This represents the total training loss of the model.

2. The infrared image destriping method based on physical priors according to claim 1, characterized in that: The vertical convolution uses a 7x1 convolution kernel, and each branch output undergoes batch normalization (BN) and ReLU activation before channel splicing.

3. The infrared image destriping method based on physical priors according to claim 1, characterized in that: Haar wavelet DWT splits the features into one low-frequency component and three high-frequency components and concatenates the channels. The shallow features are downsampled by a step size of 2 and then fused with the encoded features by adding the residuals.

4. The infrared image destriping method based on physical priors according to claim 1, characterized in that: Noise loss calculation formula: In the formula, N represents the total number of pixels in the entire image; i is the pixel index. Predict the noise value for the model at the i-th pixel; The actual labeled noise value corresponding to the i-th pixel; For absolute value operations, For summation operations.

5. The infrared image destriping method based on physical priors according to claim 1, characterized in that: Column consistency loss is first calculated by taking the mean of each column: ; Then calculate the loss: In the formula, c is the feature channel number. Image column number, The pixel index in the height direction of the image; For the c-th channel The average pixel value of the column; This represents the expected value calculated across all channels and all columns; H is the total height of the image in pixels.

6. The infrared image stripe removal method based on physical priors according to claim 1, characterized in that: Reconstruction losses In the formula, N is the total number of pixels in the image. A true, clean infrared tag image without stripes. For absolute value operations, This represents the summation operation for all pixels.

7. The infrared image stripe removal method based on physical priors according to claim 1, characterized in that: The decoder uses the PixelShuffle module to eliminate checkerboard artifacts by subpixel upsampling, and the encoder uses multi-layer shallow features to skip and stitch together, then performs 1X1 convolution dimensionality reduction before feeding them into the decoding enhancement module CNCM.

8. A physical prior-based infrared image destriping system, characterized in that, The system for performing the method according to any one of claims 1-7 includes an asymmetric feature extraction module for signal connections, a column prior attention modulation module, and a physical image reconstruction module; The asymmetric feature extraction module is used to acquire the infrared image to be processed. It uses parallel strip-shaped vertical convolutional layers and standard spatial convolutional layers to extract vertical long-range dependency features and spatial structure features respectively, and performs splicing and weighted fusion in the channel dimension to obtain comprehensive features containing stripe prior information. The column prior attention modulation module is used to perform global pooling on the comprehensive features along the height dimension, extract the mean and maximum features, learn the column-directed noise correlation, generate the column-directed attention weight vector, and perform column-level correction on the feature map to obtain the corrected feature map. The physical image reconstruction module is used to extract and process features from the corrected feature map using a backbone coding network, predict the stripe noise map, and subtract the stripe noise map from the original input image based on the inverse logic of the physical additive model to obtain the final stripe-free clean image.

9. A hardware device for removing stripes from infrared images, characterized in that, The apparatus for implementing the method of any one of claims 1-7 comprises an infrared imaging system, a data processing center, a memory, and a display; The infrared imaging system is communicatively connected to the data processing center, which in turn is communicatively connected to the memory and the display. The infrared imaging system includes an optical lens and an infrared detector, used to acquire infrared radiation signals from the scene and convert them into raw infrared image frames with stripe noise. The data processing center is internally configured with a high-performance GPU server. The memory is used to store various types of image data and model files. The display is used to show the processed infrared images in real time.

10. The apparatus according to claim 9, characterized in that: The memory is a high-speed solid-state drive, used for persistent storage of the original noisy image, ground truth labeled image, PASCNet model weight file, and processed stripe-free image; the GPU server is equipped with the CUDA parallel computing framework, which accelerates convolution operations, wavelet transforms, attention weight calculations, and backpropagation of loss function gradients in hardware parallel.

Images