An infrared image processing method and system combining denoising and super-resolution

Abstract

This invention discloses a method and system for combined denoising and super-resolution infrared image processing. The method includes: determining the characterization of various types of infrared physical noise; identifying various types of noise; constructing an infrared image dataset for training based on the characterization of the various types of infrared physical noise and the various types of noise; training an initial neural network based on the infrared image dataset to obtain a target neural network; acquiring the infrared image to be processed, and processing the infrared image to be processed through the target neural network to obtain a denoised target image. This invention generates a high-fidelity, large-scale infrared image dataset and achieves denoising and super-resolution of infrared images through a target neural network, reducing costs and improving image resolution.

Description

Technical Field

[0001] This invention relates to the field of infrared image processing technology, and in particular to an infrared image processing method and system that combines denoising and super-resolution. Background Technology

[0002] Infrared imaging technology receives electromagnetic waves in the infrared band emitted by a target through detection equipment and obtains an infrared image by processing the difference in infrared radiation between the target and the background. Infrared images can reflect the intensity distribution of infrared radiation on an object's surface, allowing people to perceive infrared radiation spectra that are difficult to observe with the naked eye. This technology has wide applications in fields such as healthcare, remote sensing, surveillance, transportation, and smart homes. However, infrared images suffer from drawbacks such as complex noise sources, low resolution, low contrast, and low signal-to-noise ratio, which severely affect image quality. Therefore, obtaining high-resolution infrared images is a pressing problem that needs to be solved.

[0003] In existing technologies, high-resolution infrared images can be obtained by developing high-density infrared focal plane arrays with small pixel sizes. However, reducing the size of the infrared focal plane array will cause severe diffraction, making it impossible to obtain high-resolution infrared images. Infrared crystal materials with good uniformity are expensive, resulting in high costs. Furthermore, image processing techniques can be used to denoise and perform super-resolution processing on infrared images. However, compared to visible light images, infrared images usually have problems such as insufficient high-frequency information such as edge textures, unclear image grayscale levels, and poor noise suppression capabilities. In addition, infrared imaging will have different degrees of complex noise under different conditions, resulting in low resolution of the processed image. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] To address this, the present invention proposes a combined denoising and super-resolution infrared image processing method. Based on the characterization of various types of infrared physical noise and various types of noise, an infrared image dataset for training can be constructed. The target neural network can be trained using the infrared image dataset, thereby generating a high-fidelity large-scale infrared image dataset. Furthermore, the infrared image denoising and super-resolution are achieved through the target neural network, reducing costs and improving image resolution.

[0006] Another objective of this invention is to propose an infrared image processing system that combines denoising and super-resolution.

[0007] To achieve the above objectives, the present invention proposes a method for infrared image processing that combines denoising and super-resolution, the method comprising:

[0008] Characterization of various types of infrared physical noise;

[0009] Identify the various different types of noise;

[0010] Based on the characterization of the various types of infrared physical noise and the various types of noise, an infrared image dataset for training is constructed.

[0011] The initial neural network is trained based on the infrared image dataset to obtain the target neural network;

[0012] An infrared image to be processed is acquired, and the infrared image to be processed is processed through the target neural network to obtain a denoised target image.

[0013] An infrared image processing method combining denoising and super-resolution according to an embodiment of the present invention may also have the following additional technical features:

[0014] In one embodiment of the present invention, the various types of infrared physical noise include:

[0015] Pixel non-uniformity noise;

[0016] Read out the noise;

[0017] Read out non-uniform stripe noise;

[0018] Time stripe noise.

[0019] In one embodiment of the present invention, determining the plurality of different types of noise includes:

[0020] Acquire a sequence of infrared images with noise under uniform radiation;

[0021] The infrared noisy image sequence is analyzed based on spatial and frequency domains to obtain various types of noise.

[0022] In one embodiment of the present invention, the analysis of the infrared noisy image sequence based on spatial and frequency domains yields various types of noise, including:

[0023] Determine the time mean in the time domain and the row and column mean in the frequency domain of the infrared noisy image sequence;

[0024] The pixel non-uniform noise is obtained based on the time mean and the row and column mean;

[0025] The temporal noise is obtained based on the infrared noisy image sequence and the time mean.

[0026] The time noise is subjected to rectangular filtering to obtain the readout noise and the time stripe noise;

[0027] Based on the infrared noisy image sequence and the pixel non-uniform noise, readout non-uniform stripe noise is obtained.

[0028] In one embodiment of the present invention, the step of constructing an infrared image dataset for training based on the characterization of the various types of infrared physical noise and the various types of noise includes:

[0029] Based on the various types of noise, determine the noise parameters for each noise type;

[0030] Based on the noise parameters of each noise type and the characterization of the various types of infrared physical noise, the correction parameters for each noise type are determined;

[0031] The correction parameter is added to each pixel of the noise-free image to obtain an infrared image dataset for training.

[0032] In one embodiment of the present invention, the target neural network includes a shallow feature extraction module, a deep feature fusion module, and an image reconstruction module; the step of processing the infrared image to be processed through the target neural network to obtain a denoised target image includes:

[0033] The feature extraction module performs shallow feature extraction on the infrared image to be processed through convolution to obtain a first feature tensor.

[0034] The deep feature fusion module extracts deep features from the first feature tensor using a residual Swing Transformer block to obtain a second feature tensor.

[0035] The image reconstruction module performs feature upsampling and reconstruction on the first feature tensor and the second feature tensor through subsampling convolutional layers to obtain a denoised and super-resolution reconstructed target image.

[0036] To achieve the above objectives, another aspect of the present invention proposes an infrared image processing system that combines denoising and super-resolution, the system comprising:

[0037] The first determining module is used to determine the characterization of various different types of infrared physical noise;

[0038] The second determining module is used to determine the various different types of noise;

[0039] A construction module is used to construct an infrared image dataset for training based on the characterization of the various types of infrared physical noise and the various types of noise.

[0040] The training module is used to train the initial neural network based on the infrared image dataset to obtain the target neural network;

[0041] The processing module is used to acquire the infrared image to be processed and process the infrared image to be processed through the target neural network to obtain the denoised target image.

[0042] An infrared image processing method combining denoising and super-resolution according to an embodiment of the present invention can construct an infrared image dataset for training based on the characterization of various types of infrared physical noise and various types of noise, and use the infrared image dataset to train a target neural network, thereby generating a high-fidelity large-scale infrared image dataset, and achieving denoising and super-resolution of infrared images through the target neural network, reducing costs and improving image resolution.

[0043] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0044] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0045] Figure 1 This is a flowchart of an infrared image processing method combining denoising and super-resolution according to an embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram comparing a simulated synthesized noisy image with a real image according to an embodiment of the present invention;

[0047] Figure 3 This is a schematic diagram of the target neural network according to an embodiment of the present invention;

[0048] Figure 4 This is a structural diagram of an infrared image processing system combining denoising and super-resolution according to an embodiment of the present invention. Detailed Implementation

[0049] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0050] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0051] A method and system for combined denoising and super-resolution infrared image processing according to an embodiment of the present invention will now be described with reference to the accompanying drawings.

[0052] Figure 1 This is a flowchart of an infrared image processing method combining denoising and super-resolution according to an embodiment of the present invention.

[0053] like Figure 1 As shown, the method includes:

[0054] S1, characterizes various types of infrared physical noise;

[0055] In one embodiment of the present invention, an infrared physical noise model can be constructed, and the characterization of various different types of infrared physical noise can be determined through the infrared physical noise model.

[0056] In one embodiment of the present invention, after constructing an infrared physical noise model, the infrared physical noise generated by each device in the infrared thermal imaging system during the imaging process and the characterization of each type of infrared physical noise can be analyzed through human experience, based on the structure and imaging principle of the infrared thermal imaging system.

[0057] Furthermore, in one embodiment of the present invention, the aforementioned various types of infrared physical noise may include:

[0058] Pixel non-uniformity noise;

[0059] Read out the noise;

[0060] Read out non-uniform stripe noise;

[0061] Time stripe noise.

[0062] Specifically, in one embodiment of the present invention, when a pixel on the focal plane array absorbs infrared radiation, the temperature of the thermistor material rises, causing the pixel resistance to decrease. At this time, by applying a bias voltage, the thermal radiation flux is converted into a voltage difference across the pixel thermistors. Furthermore, since the thermistor material, pixel resistance, and thermistors cannot be made identical during manufacturing, a pixel non-uniformity noise that is different on each pixel is generated.

[0063] In one embodiment of the present invention, pixel non-uniformity noise generated in the infrared focal plane array can be determined based on the principle of infrared radiation reception by the infrared focal plane array. The first characteristic is:

[0064]

[0065] in, and These represent the gain and offset values ​​of pixel non-uniformity noise for each pixel in the focal plane array, respectively. This indicates a noise-free image.

[0066] Furthermore, in one embodiment of the present invention, the column strip noise of the infrared readout non-uniform stripe noise of the readout circuit is determined by a linear relationship. Parallel strip noise The second characterization between the data and the actual infrared radiation data is as follows:

[0067]

[0068]

[0069] in, and Representing pixels Gain and offset of line stripe noise at the location, and Representing pixels Gain and offset of column stripe noise at the location.

[0070] Furthermore, in one embodiment of the present invention, during analog-to-digital conversion (ADC), the time variation of dark current caused by thermally induced electron fluctuations generates time-varying fringe noise, referred to as time fringe noise. .in, Appearing in one direction, can be accessed and Determine the third characterization of time stripe noise for:

[0071]

[0072] in, This represents the gain value of each row of time stripe noise at different times. This represents the offset value of each line of the time stripe noise at different times.

[0073] Furthermore, in one embodiment of the present invention, the dark current noise that may be generated by the infrared focal plane array, and the thermal noise, amplifier noise, and digitization noise that may be generated by the readout circuit are unified into a Gaussian distributed readout noise. And read out noise The fourth characteristic is:

[0074]

[0075] in, The standard deviation is the mean Gaussian noise model.

[0076] S2, identifies various different types of noise;

[0077] In one embodiment of the present invention, after determining the characterization of different types of infrared physical noise through the above steps, it is necessary to obtain the parameters of the mathematical model of each noise source in order to determine each type of noise.

[0078] Specifically, in one embodiment of the present invention, the method for determining multiple different types of noise described above may include the following steps:

[0079] S21. Obtain a sequence of infrared images with noise under uniform radiation;

[0080] S22. Based on spatial and frequency domain analysis, infrared noisy image sequences are obtained, yielding various types of noise.

[0081] In one embodiment of the present invention, a lens cap can be used to cover the lens to obtain an infrared image sequence under uniform radiation, and two segments of infrared images under uniform radiation can be used. and Obtain different types of noise, among which, and These represent the number of images in the two infrared image sequences acquired.

[0082] Furthermore, in one embodiment of the present invention, the method for analyzing infrared noisy image sequences based on spatial and frequency domains to obtain various types of noise may include the following steps:

[0083] S221. Determine the time mean and row and column mean of the infrared noisy image sequence in the time domain;

[0084] S222. Pixel non-uniform noise is obtained based on time mean and row and column mean.

[0085] S223. Based on the infrared noisy image sequence and the time mean, obtain the temporal noise;

[0086] S224. Perform rectangular filtering on the time noise to obtain readout noise and time stripe noise;

[0087] S225. Based on the infrared noisy image sequence and pixel non-uniform noise, readout non-uniform stripe noise is obtained.

[0088] Specifically, in one embodiment of the present invention, the average value of the original images in each image sequence for:

[0089]

[0090] in, This represents the number of images in the infrared image sequence.

[0091] In one embodiment of the present invention, the average time of the infrared noisy image sequence in the time domain can be calculated to obtain the time mean T.

[0092] Furthermore, in one embodiment of the present invention, each noise source can be separated by calculating a fixed row mean and a fixed column mean. The fixed column mean can be calculated using a first formula. The first formula is:

[0093]

[0094] Where W is the number of columns in each infrared image.

[0095] Furthermore, in one embodiment of the present invention, the fixed row mean can be calculated using a second formula. The second formula is:

[0096]

[0097] Where H is the number of rows in each infrared image.

[0098] Furthermore, in one embodiment of the present invention, the following can be removed: Extracting fixed stripe noise separately The pixel value is obtained by subtracting the row and column mean from the time mean of the image sequence. Noise map of generated pixel non-uniformity noise and .

[0099] Furthermore, in one embodiment of the present invention, the temporal mean can be subtracted from the infrared noisy image sequence to obtain the temporal noise N. TS And time noise N TS Rectangular filtering is performed to obtain readout noise. and time stripe noise .

[0100] Furthermore, in one embodiment of the present invention, this can be achieved by analyzing the original infrared noisy image sequence.

[0101] Remove from To obtain an image containing a smooth temperature field (low-frequency true signal) and read out non-uniform stripe noise from the high-frequency signal. .

[0102] S3, based on the characterization of various types of infrared physical noise and various types of noise, constructs an infrared image dataset for training;

[0103] In one embodiment of the present invention, after obtaining the characterizations of various types of infrared physical noise and various types of noise through the above steps, an infrared image dataset for training can be constructed based on the characterizations of various types of infrared physical noise and various types of noise.

[0104] Specifically, in one embodiment of the present invention, a method for constructing an infrared image dataset for training based on the characterization of various types of infrared physical noise and various types of noise may include the following steps:

[0105] S31, based on multiple different types of noise, determine the noise parameters for each noise type;

[0106] S32, Based on the noise parameters of each noise type and the characterization of various types of infrared physical noise, determine the correction parameters for each noise type;

[0107] S33, add correction parameters to each pixel of the noiseless image to obtain the infrared image dataset for training.

[0108] In one embodiment of the present invention, after determining various types of noise through the above steps, it is necessary to determine the noise parameters in the corresponding characterization for each type of noise so as to subsequently determine the correction parameters for each type of noise.

[0109] Specifically, in one embodiment of the present invention, by and noise graph and The corresponding noise parameters are determined using the third formula. and The third formula is:

[0110]

[0111]

[0112]

[0113] .

[0114] Furthermore, in one embodiment of the present invention, a normality test can be used for detection. The distribution and the parameters of the zero-mean Gaussian distribution are obtained. .

[0115] Furthermore, in one embodiment of the present invention, the readout non-uniform stripe noise of the low-frequency signal and the high-frequency signal is obtained through the above steps. Then, a linear model can be built to fit the relationship between the above signals to obtain the corresponding noise parameters. For example, the least squares method is used to fit a linear model to obtain readout non-uniform fringe noise. corresponding , , , The value of .

[0116] Furthermore, in one embodiment of the invention, through time stripe noise Obtain the corresponding noise parameters and The method is the same as the method described above for obtaining the noise parameters corresponding to the readout non-uniform stripe noise, and will not be repeated here in the embodiments of the present invention.

[0117] It should be noted that, in one embodiment of the present invention, the noise parameters of different pixel sources corresponding to each noise type can be obtained through the above steps.

[0118] Furthermore, in one embodiment of the present invention, after obtaining the noise parameters for each noise type through the above steps, the noise parameters for each noise type can be substituted into the characterization of the corresponding type of noise to determine the correction parameters for different pixel sources corresponding to each noise type.

[0119] Furthermore, in one embodiment of the present invention, after obtaining the correction parameters for each noise type through the above steps, the correction parameters of the corresponding pixel sources of the different noise types can be randomly added to the pixels of the noise-free image to obtain an infrared image dataset of the corresponding noise type, and thus an infrared image dataset for training. For example, as shown... Figure 2 As shown, this is a schematic diagram comparing the simulated noisy image with the real image.

[0120] S4, the initial neural network is trained based on the infrared image dataset to obtain the target neural network;

[0121] In one embodiment of the present invention, after obtaining the infrared image dataset through the above steps, the initial neural network can be trained based on the infrared image dataset to obtain the target neural network.

[0122] Furthermore, in one embodiment of the present invention, the initial neural network may include a shallow feature extraction module, a deep feature fusion module, and an image reconstruction module.

[0123] Furthermore, in one embodiment of the present invention, the method of training the initial neural network based on the infrared image dataset to obtain the target neural network is the same as the prior art, and specific details can be found in the prior art. The embodiments of the present invention will not be elaborated here.

[0124] S5: Acquire the infrared image to be processed, and process the infrared image to be processed through the target neural network to obtain the denoised target image.

[0125] In one embodiment of the present invention, after obtaining the target neural network through the above steps, an infrared image to be processed can be acquired, and the infrared image to be processed can be processed through the target neural network to obtain a denoised target image.

[0126] Specifically, in one embodiment of the present invention, the method for processing the infrared image to be processed using a target neural network to obtain a denoised target image may include the following steps:

[0127] S51, the feature extraction module performs shallow feature extraction on the infrared image to be processed by convolution to obtain the first feature tensor;

[0128] S52, the deep feature fusion module extracts deep features from the first feature tensor through the residual Swing Transformer block to obtain the second feature tensor;

[0129] S53, the image reconstruction module performs feature upsampling and reconstruction on the first feature tensor and the second feature tensor through subsampling convolutional layers to obtain the denoised and super-resolution reconstructed target image.

[0130] In one embodiment of the present invention, the feature extraction module can perform shallow feature extraction on the infrared image to be processed by 3×3 convolution to obtain a first feature tensor.

[0131] Furthermore, in one embodiment of the present invention, the deep feature fusion module can extract details at different image scales through the hierarchical structure of multiple residual SwinTransformer blocks and the sliding window mechanism to perform deep feature extraction on the first feature tensor to obtain the second feature tensor, thereby improving the performance of the model when processing high-resolution images while controlling the computational cost.

[0132] Based on the above description Figure 3 This is a schematic diagram of a target neural network proposed in this invention. Figure 3 As shown, after the target neural network acquires the infrared image to be analyzed, it can perform shallow feature extraction through the feature extraction module of the target neural network to obtain the first feature tensor, and then perform deep feature extraction on the first feature tensor through the deep feature fusion module to obtain the second feature tensor. Finally, the image reconstruction module performs feature upsampling and reconstruction on the first and second feature tensors to obtain the denoised and super-resolution reconstructed target image.

[0133] According to an embodiment of the present invention, an infrared image processing method combining denoising and super-resolution can construct an infrared image dataset for training based on the characterization of various types of infrared physical noise and various types of noise, and use the infrared image dataset to train a target neural network, thereby generating a high-fidelity large-scale infrared image dataset. Furthermore, the target neural network achieves denoising and super-resolution of infrared images, reducing costs and improving image resolution.

[0134] To achieve the above embodiments, such as Figure 4 As shown, this embodiment also provides an infrared image processing system 10 that combines denoising and super-resolution. The system includes a first determining module 401, a second determining module 402, a construction module 403, a training module 404, and a processing module 405.

[0135] The first determining module 401 is used to determine the characterization of various types of infrared physical noise;

[0136] The second determining module 402 is used to determine multiple different types of noise;

[0137] Module 403 is used to construct an infrared image dataset for training based on the characterization of various types of infrared physical noise and various types of noise.

[0138] Training module 404 is used to train the initial neural network based on the infrared image dataset to obtain the target neural network;

[0139] The processing module 405 is used to acquire the infrared image to be processed and process the infrared image to be processed through the target neural network to obtain the denoised target image.

[0140] Furthermore, the aforementioned various types of infrared physical noise include:

[0141] Pixel non-uniformity noise;

[0142] Read out the noise;

[0143] Read out non-uniform stripe noise;

[0144] Time stripe noise.

[0145] Furthermore, the aforementioned second determining module 402 is specifically used for:

[0146] Acquire a sequence of infrared images with noise under uniform radiation;

[0147] Based on spatial and frequency domain analysis, various types of noise were obtained from the infrared noisy image sequences.

[0148] Furthermore, the second determining module 402 described above is also used for:

[0149] Determine the time mean in the time domain and the row and column mean in the frequency domain of the infrared noisy image sequence;

[0150] Pixel non-uniform noise is obtained based on time mean and row and column mean;

[0151] Temporal noise is obtained based on the infrared noisy image sequence and the time mean;

[0152] Rectangular filtering is applied to the time noise to obtain readout noise and time stripe noise;

[0153] Based on the infrared noisy image sequence and pixel non-uniform noise, readout non-uniform stripe noise is obtained.

[0154] Furthermore, the aforementioned building module 403 is specifically used for:

[0155] Based on various different types of noise, determine the noise parameters for each noise type;

[0156] Based on the noise parameters for each noise type and the characterization of various types of infrared physical noise, correction parameters for each noise type are determined;

[0157] By adding correction parameters to each pixel of the noise-free image, an infrared image dataset is obtained for training.

[0158] Furthermore, the aforementioned target neural network includes a shallow feature extraction module, a deep feature fusion module, and an image reconstruction module; the aforementioned processing module 405 is specifically used for:

[0159] The feature extraction module performs shallow feature extraction on the infrared image to be processed by convolution to obtain the first feature tensor;

[0160] The deep feature fusion module extracts deep features from the first feature tensor using the residual Swin Transformer block to obtain the second feature tensor;

[0161] The image reconstruction module performs feature upsampling and reconstruction on the first and second feature tensors through subsampling convolutional layers to obtain the denoised and super-resolution reconstructed target image.

[0162] According to an embodiment of the present invention, the infrared image processing system for joint denoising and super-resolution can construct an infrared image dataset for training based on the characterization of various types of infrared physical noise and various types of noise, and use the infrared image dataset to train a target neural network, thereby generating a high-fidelity large-scale infrared image dataset. Furthermore, the system achieves denoising and super-resolution of infrared images through the target neural network, thereby reducing costs and improving image resolution.

[0163] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0164] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A method for combined denoising and super-resolution infrared image processing, characterized in that, include: Characterization of various types of infrared physical noise; The process of identifying various types of noise, including: pixel non-uniformity noise; readout noise; readout non-uniform stripe noise; and temporal stripe noise, comprises: acquiring a noisy infrared image sequence under uniform radiation; analyzing the noisy infrared image sequence in the spatial and frequency domains to obtain the various types of noise, including: determining the time mean in the time domain and the row and column mean in the frequency domain of the noisy infrared image sequence; obtaining the pixel non-uniformity noise based on the time mean and the row and column mean; obtaining the temporal noise based on the noisy infrared image sequence and the time mean; performing rectangular filtering on the temporal noise to obtain the readout noise and the temporal stripe noise; and obtaining the readout non-uniform stripe noise based on the noisy infrared image sequence and the pixel non-uniformity noise. Based on the characterization of the various types of infrared physical noise and the various types of noise, an infrared image dataset for training is constructed. The initial neural network is trained based on the infrared image dataset to obtain the target neural network; An infrared image to be processed is acquired, and the infrared image to be processed is processed through the target neural network to obtain a denoised target image; Among them, pixel non-uniformity noise The first characteristic is: in, and These represent the gain and offset values ​​of pixel non-uniformity noise for each pixel in the focal plane array, respectively. Represents a noise-free image; Column strip noise of non-uniform stripe noise Parallel strip noise The second characterization between the data and the actual infrared radiation data is as follows: in, and Representing pixels Gain and offset of line stripe noise at the location, and Representing pixels Gain and offset of column stripe noise at the location; Determine time stripe noise The third characteristic is: in, This represents the gain value of each row of time stripe noise at different times. This represents the offset value of each line of the time stripe noise at different times; Readout noise The fourth characteristic is: in, The standard deviation is the mean Gaussian noise model.

2. The method according to claim 1, characterized in that, The infrared image dataset for training is constructed based on the characterization of the various types of infrared physical noise and the various types of noise, including: Based on the various types of noise, determine the noise parameters for each noise type; Based on the noise parameters of each noise type and the characterization of the various types of infrared physical noise, the correction parameters for each noise type are determined; The correction parameter is added to each pixel of the noise-free image to obtain an infrared image dataset for training.

3. The method according to claim 1, characterized in that, The target neural network includes a shallow feature extraction module, a deep feature fusion module, and an image reconstruction module; the process of processing the infrared image to be processed through the target neural network to obtain a denoised target image includes: The feature extraction module performs shallow feature extraction on the infrared image to be processed through convolution to obtain a first feature tensor. The deep feature fusion module extracts deep features from the first feature tensor using a residual Swing Transformer block to obtain a second feature tensor. The image reconstruction module performs feature upsampling and reconstruction on the first feature tensor and the second feature tensor through subsampling convolutional layers to obtain a denoised and super-resolution reconstructed target image.

4. A combined denoising and super-resolution infrared image processing system, characterized in that, include: The first determining module is used to determine the characterization of various different types of infrared physical noise; The second determining module is used to determine the various types of noise, including: pixel non-uniformity noise; readout noise; readout non-uniform stripe noise; and temporal stripe noise. The process of determining these various types of noise includes: acquiring a noisy infrared image sequence under uniform radiation; analyzing the noisy infrared image sequence based on spatial and frequency domains to obtain the various types of noise, including: determining the time mean in the time domain and the row and column mean in the frequency domain of the noisy infrared image sequence; obtaining the pixel non-uniformity noise based on the time mean and the row and column mean; obtaining temporal noise based on the noisy infrared image sequence and the time mean; performing rectangular filtering on the temporal noise to obtain the readout noise and the temporal stripe noise; and obtaining the readout non-uniform stripe noise based on the noisy infrared image sequence and the pixel non-uniformity noise. A construction module is used to construct an infrared image dataset for training based on the characterization of the various types of infrared physical noise and the various types of noise. The training module is used to train the initial neural network based on the infrared image dataset to obtain the target neural network; The processing module is used to acquire the infrared image to be processed and process the infrared image to be processed through the target neural network to obtain the denoised target image; Among them, pixel non-uniformity noise The first characteristic is: in, and These represent the gain and offset values ​​of pixel non-uniformity noise for each pixel in the focal plane array, respectively. Represents a noise-free image; Column strip noise of non-uniform stripe noise Parallel strip noise The second characterization between the data and the actual infrared radiation data is as follows: in, and Representing pixels Gain and offset of line stripe noise at the location, and Representing pixels Gain and offset of column stripe noise at the location; Determine time stripe noise The third characteristic is: in, This represents the gain value of each line of the time stripe noise at different times. This represents the offset value of each line of the time stripe noise at different times; Readout noise The fourth characteristic is: in, The standard deviation is the mean Gaussian noise model.

5. An electronic device, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-3.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-3.

Images