Denoising and super-resolution combined infrared image processing method and system

Abstract

The invention discloses a denoising and super-resolution combined infrared image processing method and system, and the method comprises the steps: determining the characterization of a plurality of different types of infrared physical noises; determining a plurality of different types of noise; constructing an infrared image data set for training based on the characterization of the infrared physical noise of various different types and the noise of various different types; training the initial neural network based on the infrared image data set to obtain a target neural network; and obtaining a to-be-processed infrared image, and processing the to-be-processed infrared image through the target neural network to obtain a denoised target image. According to the method, the high-fidelity large-scale infrared image data set is generated, denoising and super-resolution of the infrared image are realized through the target neural network, the cost is reduced, and the image resolution is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of infrared image processing, and in particular to an infrared image processing method and system for joint denoising and super-resolution. Background Art

[0002] Infrared imaging technology receives electromagnetic waves in the infrared band radiated by a target through a detection device, and processes the infrared difference between the target and the background to obtain an infrared image. Among them, the infrared image can reflect the intensity distribution of the infrared radiation on the surface of an object, so that people can perceive the infrared radiation spectrum that is difficult to observe with the naked eye, and can be widely used in fields such as medical and health, remote sensing detection, monitoring and inspection, transportation, and home intelligence. However, infrared images have the disadvantages of complex noise sources, low resolution, low contrast, and low signal-to-noise ratio, which seriously affect the quality of the images. Based on this, how to obtain high-resolution infrared images is an urgent problem to be solved.

[0003] In the prior art, high-resolution infrared images can be obtained by developing infrared focal plane arrays with high density and small pixel sizes. However, reducing the size of the infrared focal plane array will cause serious diffraction phenomena and high-resolution infrared images cannot be obtained; infrared crystal materials with good uniformity are expensive and have high costs; and image processing techniques can be used to perform denoising and super-resolution processing on infrared images. However, compared with visible light images, infrared images usually have problems such as too little high-frequency information such as edge textures, unclear gray levels of the images, poor noise suppression ability, and different degrees of complex noise in different infrared imaging situations, resulting in low resolution of the processed images. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems in the related art to some extent.

[0005] To this end, the present invention proposes an infrared image processing method for joint denoising and super-resolution, which can construct an infrared image dataset for training based on the characteristics of multiple different types of infrared physical noise and multiple different types of noise, and use the infrared image dataset for training to obtain a target neural network, so as to generate a high-fidelity large-scale infrared image dataset, and realize denoising and super-resolution of infrared images through the target neural network, reducing costs and improving image resolution.

[0006] Another object of the present invention is to propose an infrared image processing system for joint denoising and super-resolution.

[0007] To achieve the above object, on the one hand, the present invention proposes an infrared image processing method for joint denoising and super-resolution, and the method includes: Determine the characteristics of multiple different types of infrared physical noise; Determine the multiple different types of noise; Based on the characterization of the multiple different types of infrared physical noise and the multiple different types of noise, construct an infrared image dataset for training; Train an initial neural network based on the infrared image dataset to obtain a target neural network; Obtain an infrared image to be processed, and process the infrared image to be processed through the target neural network to obtain a denoised target image.

[0008] An infrared image processing method for joint denoising and super-resolution according to an embodiment of the present invention may further have the following additional technical features: In an embodiment of the present invention, the multiple different types of infrared physical noise include: Pixel non-uniformity noise; Readout noise; Readout non-uniform stripe noise; Temporal stripe noise.

[0009] In an embodiment of the present invention, the determining the multiple different types of noise includes: Obtain an infrared noisy image sequence under uniform radiation; Analyze the infrared noisy image sequence based on the spatial domain and the frequency domain to obtain the multiple different types of noise.

[0010] In an embodiment of the present invention, the analyzing the infrared noisy image sequence based on the spatial domain and the frequency domain to obtain the multiple different types of noise includes: Determine the temporal mean value of the infrared noisy image sequence in the time domain and the row-column mean value in the frequency domain; Based on the temporal mean value and the row-column mean value, obtain the pixel non-uniformity noise; Based on the infrared noisy image sequence and the temporal mean value, obtain the temporal noise; Perform rectangular filtering on the temporal noise to obtain the readout noise and the temporal stripe noise; Based on the infrared noisy image sequence and the pixel non-uniformity noise, obtain the readout non-uniform stripe noise.

[0011] In an embodiment of the present invention, the constructing an infrared image dataset for training based on the characterization of the multiple different types of infrared physical noise and the multiple different types of noise includes: Based on the multiple different types of noise, determine the noise parameters of each noise type; Determine the correction parameters for each type of noise based on the noise parameters for each type of noise and the characterization of the multiple different types of infrared physical noise; Add the correction parameters to each pixel of the noise-free image to obtain an infrared image dataset for training.

[0012] In an embodiment of the present invention, the target neural network includes a shallow feature extraction module, a deep feature fusion module, and an image reconstruction module; the process of obtaining the denoised target image by processing the infrared image to be processed through the target neural network includes: The feature extraction module performs shallow feature extraction on the infrared image to be processed through convolution to obtain a first feature tensor; The deep feature fusion module performs deep feature extraction on the first feature tensor through residual Swin Transformer blocks to obtain a second feature tensor; The image reconstruction module performs feature upsampling and reconstruction on the first feature tensor and the second feature tensor through a subsampling convolutional layer to obtain a denoised and super-resolution reconstructed target image.

[0013] To achieve the above object, on the other hand, the present invention proposes an infrared image processing system for joint denoising and super-resolution, the system includes: A first determination module, configured to determine the characterization of multiple different types of infrared physical noise; A second determination module, configured to determine the multiple different types of noise; A construction module, configured to construct an infrared image dataset for training based on the characterization of the multiple different types of infrared physical noise and the multiple different types of noise; A training module, configured to train an initial neural network based on the infrared image dataset to obtain a target neural network; A processing module, configured to obtain an infrared image to be processed, and process the infrared image to be processed through the target neural network to obtain a denoised target image.

[0014] An infrared image processing method for joint denoising and super-resolution according to an embodiment of the present invention can construct an infrared image dataset for training based on the characterization of multiple different types of infrared physical noise and multiple different types of noise, and use the infrared image dataset for training to obtain a target neural network, so as to generate a high-fidelity large-scale infrared image dataset, and realize denoising and super-resolution of infrared images through the target neural network, reducing costs and improving image resolution.

[0015] Additional aspects and advantages of the present invention will be given in part in the following description, become apparent in part from the following description, or be understood through the practice of the present invention. Brief Description of the Drawings

[0016] The above and / or additional aspects and advantages of the present invention will become apparent and be readily understood from the following description of embodiments in conjunction with the accompanying drawings, where: Figure 1 is a flowchart of an infrared image processing method for joint denoising and super-resolution according to an embodiment of the present invention; Figure 2 is a comparison schematic diagram of a simulated synthetic noise image and a real image according to an embodiment of the present invention; Figure 3 is a schematic diagram of a target neural network according to an embodiment of the present invention; Figure 4 is a structural diagram of an infrared image processing system for joint denoising and super-resolution according to an embodiment of the present invention. Detailed Embodiments

[0017] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0018] In order to enable those skilled in the art of the present technology to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0019] A method and system for infrared image processing for joint denoising and super-resolution according to an embodiment of the present invention will be described below with reference to the accompanying drawings.

[0020] Figure 1 is a flowchart of an infrared image processing method for joint denoising and super-resolution according to an embodiment of the present invention.

[0021] As Figure 1 shown, the method includes: S1, determining the representations of multiple different types of infrared physical noise; In an embodiment of the present invention, an infrared physical noise model may be constructed, and through the infrared physical noise model, the representations of multiple different types of infrared physical noise are determined.

[0022] Among them, in an embodiment of the present invention, after constructing the infrared physical noise model, according to the composition and imaging principle of the infrared thermal imaging system, each device in the infrared thermal imaging system can be analyzed through artificial experience to generate various different types of infrared physical noise and the characterization of each type of infrared physical noise during the imaging process.

[0023] And, in an embodiment of the present invention, the above-mentioned various different types of infrared physical noise may include: Pixel non-uniformity noise; Readout noise; Readout non-uniform stripe noise; Temporal stripe noise.

[0024] Specifically, in an embodiment of the present invention, when the pixels on the focal plane array absorb infrared radiation, the temperature of the thermosensitive material rises, causing the pixel resistance to decrease. At this time, by applying a bias voltage, the thermal radiation flux is converted into a voltage difference across the pixel thermistor. And, because the thermosensitive material, pixel resistance, and thermistor cannot be kept the same during the manufacturing process, based on this, a pixel non-uniformity noise that is different for each pixel will be generated.

[0025] Among them, in an embodiment of the present invention, the first characterization of the pixel non-uniform noise generated in the infrared focal plane array can be determined according to the principle of the infrared focal plane receiving infrared radiation is:

[0026] Among them, and respectively represent the gain value and offset value of the pixel non-uniform noise of each pixel in the focal plane array, represents the noise-free image.

[0027] And, in an embodiment of the present invention, the column stripe noise and row stripe noise of the infrared readout non-uniform stripe noise of the readout circuit are determined through a linear relationship, and the second characterization with the infrared true radiation data is:

[0028]

[0029] Among them, and respectively represent the gain and offset of the row stripe noise at the pixel location, and respectively represent the gain and offset of the column stripe noise at the pixel location.

[0030] Furthermore, in an embodiment of the present invention, during the analog-to-digital conversion (ADC) process, the time variation of the dark current caused by the fluctuations of thermally induced electrons generates time-varying stripe noise, referred to as time stripe noise. . Among them, Appearing in one direction, the third characterization of the time stripe noise can be determined by and as: where,

[0031] where, represents the gain value of each row of the time stripe noise at different times, represents the offset value of each row of the time stripe noise at different times.

[0032] Furthermore, in an embodiment of the present invention, the dark current noise that may be generated by the infrared focal plane array, and the thermal noise, amplifier noise, and digitalization noise that may be generated by the readout circuit are unified into readout noise with a Gaussian distribution , and the fourth characterization of the readout noise is:

[0033] where, is the standard deviation of the Gaussian noise model with the mean value.

[0034] S2. Determine various different types of noise; In an embodiment of the present invention, after determining the characterizations of different types of infrared physical noise through the above steps, it is necessary to obtain the parameters of the mathematical model of each noise source to determine each type of noise.

[0035] Specifically, in an embodiment of the present invention, the method for determining various different types of noise may include the following steps: S21. Obtain an infrared noisy image sequence under uniform radiation; S22. Analyze the infrared noisy image sequence based on the spatial domain and the frequency domain to obtain various different types of noise.

[0036] Among them, in an embodiment of the present invention, the lens can be covered with a lens cap to obtain an infrared image sequence under uniform radiation, and two infrared images under uniform radiation and are used to obtain different types of noise, where, and are respectively the numbers of the two obtained infrared image sequences.

[0037] Moreover, in an embodiment of the present invention, the method for analyzing an infrared noisy image sequence in the spatial domain and the frequency domain to obtain various different types of noise may include the following steps: S221. Determine the time mean value in the time domain and the row-column mean value in the frequency domain of the infrared noisy image sequence; S222. Obtain pixel non-uniformity noise based on the time mean value and the row-column mean value; S223. Obtain time noise based on the infrared noisy image sequence and the time mean value; S224. Perform rectangular filtering on the time noise to obtain the noise of the readout noise and the time stripe noise; S225. Obtain the readout non-uniform stripe noise based on the infrared noisy image sequence and the pixel non-uniformity noise.

[0038] Specifically, in an embodiment of the present invention, the average value of the original images in each image sequence is:

[0039] wherein, is the number of infrared image sequences.

[0040] In an embodiment of the present invention, the average value of the time in the infrared noisy image sequence may be obtained to obtain the time mean value T.

[0041] Moreover, in an embodiment of the present invention, each noise source may be separated by calculating the fixed row mean value and the fixed column mean value. Among them, the fixed column mean value may be calculated by the first formula , and the first formula is:

[0042] wherein, W is the number of columns of each infrared image.

[0043] Moreover, in an embodiment of the present invention, the fixed row mean value may be calculated by the second formula , and the second formula is:

[0044] wherein, H is the number of rows of each infrared image.

[0045] Furthermore, in an embodiment of the present invention, the fixed stripe noise in may be removed and the separate may be extracted, and the noise map of the pixel non-uniformity noise generated by the pixels is obtained by subtracting the row-column mean value from the time mean value of the image sequence and .

[0046] Further, in an embodiment of the present invention, the temporal mean can be subtracted from the infrared noisy image sequence to obtain the temporal noise N TS , and the temporal noise N TS is subjected to rectangular filtering to obtain the read noise and the temporal stripe noise .

[0047] Further, in an embodiment of the present invention, by removing from the original infrared noisy image sequence

[0048] , an image containing a smooth temperature field (low-frequency true signal) and high-frequency signal read non-uniform stripe noise is obtained .

[0049] S3. Based on the characterizations of various different types of infrared physical noises and various different types of noises, construct an infrared image dataset for training; Among them, in an embodiment of the present invention, after obtaining the characterizations of various different types of infrared physical noises and various different types of noises through the above steps, an infrared image dataset for training can be constructed based on the characterizations of various different types of infrared physical noises and various different types of noises.

[0050] Specifically, in an embodiment of the present invention, the method for constructing an infrared image dataset for training based on the characterizations of various different types of infrared physical noises and various different types of noises may include the following steps: S31. Based on various different types of noises, determine the noise parameters of each noise type; S32. Based on the noise parameters of each noise type and the characterizations of various different types of infrared physical noises, determine the correction parameters of each noise type; S33. Add the correction parameters to each pixel of the noise-free image to obtain the infrared image dataset for training.

[0051] Among them, in an embodiment of the present invention, after determining various different types of noises through the above steps, it is necessary to determine the noise parameters in the corresponding characterizations through each type of noise, so as to subsequently determine the correction parameters of each noise type.

[0052] Specifically, in an embodiment of the present invention, through and the noise map and , the corresponding noise parameters and are determined using the third formula, where the third formula is:

[0053]

[0054]

[0055] 。

[0056] Also, in an embodiment of the present invention, normality test can be used to detect the distribution and obtain the parameters of a Gaussian distribution with zero mean 。

[0057] Furthermore, in an embodiment of the present invention, after obtaining the read non-uniform stripe noise of the low-frequency signal and the high-frequency signal through the above steps , a linear model can be established to fit the relationship between the above signals to obtain the corresponding noise parameters. For example, the least squares method is used to fit the linear model to obtain the read non-uniform stripe noise corresponding , , , values.

[0058] Furthermore, in an embodiment of the present invention, the method of obtaining the corresponding noise parameters through the temporal stripe noise is the same as the method of obtaining the noise parameters corresponding to the read non-uniform stripe noise above, and the embodiments of the present invention will not elaborate here. and

[0059] It should be noted that, in an embodiment of the present invention, the noise parameters corresponding to different pixel sources for each noise type can be obtained through the above steps.

[0060] Also, in an embodiment of the present invention, after obtaining the noise parameters of each noise type through the above steps, the noise parameters of each noise type can be substituted into the characterization of the corresponding type of noise to determine the correction parameters corresponding to different pixel sources for each noise type.

[0061] Furthermore, in an embodiment of the present invention, after obtaining the correction parameters of each noise type through the above steps, the correction parameters corresponding to the pixel sources in the above different noise types can be randomly added to the pixels of the noise-free image to obtain an infrared image dataset corresponding to the corresponding noise type, and an infrared image dataset for training can be obtained. For example, as Figure 2 shown, a comparison schematic diagram of the simulated synthetic noise image and the real image.

[0062] S4. Train the initial neural network based on the infrared image dataset to obtain the target neural network; Among them, in an embodiment of the present invention, after obtaining the infrared image dataset through the above steps, the initial neural network can be trained based on the infrared image dataset to obtain the target neural network.

[0063] In addition, in an embodiment of the present invention, the above initial neural network may include a shallow feature extraction module, a deep feature fusion module, and an image reconstruction module.

[0064] Furthermore, in an embodiment of the present invention, the method of training the initial neural network based on the infrared image dataset to obtain the target neural network is the same as the prior art. Specifically, reference can be made to the prior art, and the embodiments of the present invention will not be elaborated herein.

[0065] S5. Obtain the infrared image to be processed, and process the infrared image to be processed through the target neural network to obtain the denoised target image.

[0066] Among them, in an embodiment of the present invention, after obtaining the target neural network through the above steps, the infrared image to be processed can be obtained, and the infrared image to be processed can be processed through the target neural network to obtain the denoised target image.

[0067] Specifically, in an embodiment of the present invention, the method of processing the infrared image to be processed through the target neural network to obtain the denoised target image may include the following steps: S51. The feature extraction module performs shallow feature extraction on the infrared image to be processed through convolution to obtain the first feature tensor; S52. The deep feature fusion module performs deep feature extraction on the first feature tensor through the residual Swin Transformer block to obtain the second feature tensor; S53. The image reconstruction module performs feature upsampling and reconstruction on the first feature tensor and the second feature tensor through the subsampling convolutional layer to obtain the denoised and super-resolution reconstructed target image.

[0068] Among them, in an embodiment of the present invention, the feature extraction module may perform shallow feature extraction on the infrared image to be processed through 3×3 convolution to obtain the first feature tensor.

[0069] In addition, in an embodiment of the present invention, the deep feature fusion module can extract details at different image scales through the hierarchical structure and sliding window mechanism of multiple residual Swin Transformer blocks to perform deep feature extraction on the first feature tensor to obtain the second feature tensor, thereby improving the performance of the model when processing high-resolution images while controlling the computational cost.

[0070] Based on the above description, Figure 3 is a schematic diagram of a target neural network proposed by the present invention. As Figure 3 shown, after the target neural network obtains the infrared image to be analyzed, it can perform shallow feature extraction on the infrared image through the feature extraction module of the target neural network to obtain the first feature tensor, and perform deep feature extraction on the first feature tensor through the deep feature fusion module to obtain the second feature tensor, and then perform feature upsampling and reconstruction on the first feature tensor and the second feature tensor through the image reconstruction module to obtain the target image of denoising and super-resolution reconstruction.

[0071] According to an infrared image processing method for joint denoising and super-resolution according to an embodiment of the present invention, the method can construct an infrared image dataset for training based on the representations of multiple different types of infrared physical noises and multiple different types of noises, and use the infrared image dataset to train a target neural network, so as to generate a high-fidelity large-scale infrared image dataset, and realize denoising and super-resolution of infrared images through the target neural network, reducing costs and improving image resolution.

[0072] To implement the above embodiment, as Figure 4 shown, in this embodiment, an infrared image processing system 10 for joint denoising and super-resolution is further provided. The system includes a first determination module 401, a second determination module 402, a construction module 403, a training module 404, and a processing module 405; The first determination module 401 is configured to determine the representations of multiple different types of infrared physical noises; The second determination module 402 is configured to determine multiple different types of noises; The construction module 403 is configured to construct an infrared image dataset for training based on the representations of multiple different types of infrared physical noises and multiple different types of noises; The training module 404 is configured to train an initial neural network based on the infrared image dataset to obtain a target neural network; The processing module 405 is configured to obtain an infrared image to be processed, and process the infrared image to be processed through the target neural network to obtain a denoised target image.

[0073] Furthermore, the above-mentioned multiple different types of infrared physical noises include: Pixel non-uniformity noise; Readout noise; Readout non-uniform stripe noise; Temporal stripe noise.

[0074] Furthermore, the above-mentioned second determination module 402 is specifically configured to: Obtain an infrared noisy image sequence under uniform radiation; Analyze the infrared noisy image sequence in the spatial domain and frequency domain to obtain various different types of noise.

[0075] Furthermore, the above-mentioned second determination module 402 is further configured to: Determine the time mean in the time domain and the row-column mean in the frequency domain of the infrared noisy image sequence; Based on the time mean and the row-column mean, obtain pixel non-uniformity noise; Based on the infrared noisy image sequence and the time mean, obtain time noise; Perform rectangular filtering on the time noise to obtain readout noise and time stripe noise; Based on the infrared noisy image sequence and the pixel non-uniformity noise, obtain readout non-uniform stripe noise.

[0076] Furthermore, the above-mentioned construction module 403 is specifically configured to: Based on various different types of noise, determine the noise parameters of each noise type; Based on the noise parameters of each noise type and the characterization of various different types of infrared physical noise, determine the correction parameters of each noise type; Add the correction parameters to each pixel of the noise-free image to obtain an infrared image dataset for training.

[0077] Furthermore, the above-mentioned target neural network includes a shallow feature extraction module, a deep feature fusion module, and an image reconstruction module; the above-mentioned processing module 405 is specifically configured to: The feature extraction module performs shallow feature extraction on the infrared image to be processed through convolution to obtain a first feature tensor; The deep feature fusion module performs deep feature extraction on the first feature tensor through a residual Swin Transformer block to obtain a second feature tensor; The image reconstruction module performs feature upsampling and reconstruction on the first feature tensor and the second feature tensor through a subsampling convolutional layer to obtain a target image for denoising and super-resolution reconstruction.

[0078] According to the infrared image processing system for joint denoising and super-resolution of the embodiments of the present invention, the system can construct an infrared image dataset for training based on the characterization of various different types of infrared physical noise and various different types of noise, and use the infrared image dataset for training to obtain a target neural network, so as to generate a high-fidelity large-scale infrared image dataset, and realize denoising and super-resolution of infrared images through the target neural network, reducing costs and improving image resolution.

[0079] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0080] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of the present invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise specifically defined.

Claims

1. A method for processing infrared images by combining denoising and super-resolution, characterized in that: include: Determine the characterization of many different types of infrared physical noise; determining the plurality of different types of noise; Based on the representations of the multiple different types of infrared physical noise and the multiple different types of noise, construct an infrared image dataset for training; Training an initial neural network based on the infrared image data set to obtain a target neural network; An infrared image to be processed is acquired, and the infrared image to be processed is processed by the target neural network to obtain a denoised target image.

2. The method according to claim 1, characterized in that The various types of infrared physical noise include: Pixel non-uniformity noise; Readout noise; Reading out non-uniform fringe noise; Temporal streak noise.

3. The method according to claim 2, characterized in that The determining of the multiple different types of noises comprises: Obtain infrared noisy image sequences under uniform radiation; The infrared noisy image sequence is analyzed based on the spatial domain and the frequency domain to obtain the multiple different types of noise.

4. The method according to claim 3, characterized in that The infrared noisy image sequence is analyzed based on the spatial domain and the frequency domain to obtain the multiple different types of noise, including: Determine the time mean of the infrared noisy image sequence in the time domain and the row and column mean in the frequency domain; Based on the time mean and the row and column mean, the pixel non-uniform noise is obtained; Based on the infrared noisy image sequence and the time mean, obtaining time noise; Performing rectangular filtering on the temporal noise to obtain the readout noise and the temporal fringe noise; Based on the infrared noisy image sequence and the pixel non-uniform noise, readout non-uniform stripe noise is obtained.

5. The method according to claim 1, characterized in that The step of constructing an infrared image data set for training based on the characterization of the multiple different types of infrared physical noise and the multiple different types of noise includes: Based on the multiple different types of noise, determining a noise parameter for each noise type; Determining a correction parameter for each noise type based on the noise parameter of each noise type and the representations of the multiple different types of infrared physical noises; The correction parameter is added to each pixel of the noise-free image to obtain an infrared image data set for training.

6. The method according to claim 1, characterized in that The target neural network includes a shallow feature extraction module, a deep feature fusion module and an image reconstruction module; the target neural network is used to process the infrared image to be processed to obtain a denoised target image, including: The feature extraction module performs shallow feature extraction on the infrared image to be processed through convolution to obtain a first feature tensor; The deep feature fusion module performs deep feature extraction on the first feature tensor through a residual Swin Transformer block to obtain a second feature tensor; The image reconstruction module performs feature upsampling and reconstruction on the first feature tensor and the second feature tensor through a subsampling convolution layer to obtain a denoised and super-resolution reconstructed target image.

7. A combined denoising and super-resolution infrared image processing system, characterized in that: include: A first determination module is used to determine the representations of multiple different types of infrared physical noise; A second determination module, configured to determine the multiple different types of noise; A construction module, configured to construct an infrared image data set for training based on the representations of the multiple different types of infrared physical noise and the multiple different types of noise; A training module, used for training an initial neural network based on the infrared image data set to obtain a target neural network; The processing module is used to obtain the infrared image to be processed, and process the infrared image to be processed through the target neural network to obtain a denoised target image.

8. The system according to claim 7, characterized in that The various types of infrared physical noise include: Pixel non-uniformity noise; Readout noise; Reading out non-uniform fringe noise; Temporal streak noise.

9. An electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images