Infrared image non-uniformity correction method based on global and local features

By adopting the point-by-point superposition method of global and local feature extraction layers in the infrared image non-uniformity correction method, the problems of high computational complexity and poor image detail retention ability in the prior art are solved, and high efficiency and low complexity are achieved Inhomogeneity correction effect.

CN119006771APending Publication Date: 2024-11-22BEIJING INST OF REMOTE SENSING INFORMATION
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411123771.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing infrared image non-uniformity correction method based on deep learning has problems such as high computational complexity and large model scale, making it difficult to implement engineering applications, and the image details retention ability of small models is poor, and there are deformation problems.

Method used

The infrared image non-uniformity correction method based on global and local features is adopted. The feature extraction layer is divided into global grayscale residual and row and column grayscale residual feature extraction blocks. The feature map is fused by point-by-point superposition to reduce the complexity of the model. And improve the non-uniformity correction efficiency.

Benefits of technology

It effectively reduces the complexity of the model, improves the inhomogeneity correction efficiency and image detail retention ability, avoids image loss of detail information, and improves the corrected image structure similarity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006771A_ABST
    Figure CN119006771A_ABST
Patent Text Reader

Abstract

The invention provides an infrared image non-uniformity correction method based on global and local features, and belongs to the field of image processing. The problems that a small model image is poor in detail retention capacity and has deformation, and a complex model is large in scale, complex in calculation and difficult to apply in an engineering mode are solved. A point-by-point superposition mode is used to replace a convolution mode to fuse features, and an interpolation mode is used to replace a deconvolution mode to perform up-sampling, so that the model complexity is effectively reduced, and the non-uniformity correction efficiency is improved. The projection space of each level of feature extraction layer is unified, mutual redundant representation of feature maps is avoided, the number of model channels is increased in a disguised manner, the network detail retention capability is further effectively improved, and the situation that the image loses detail information after non-uniformity correction is avoided; the feature extraction layer can separate stripe noise based on a residual structure, texture details are extracted from a relatively smooth image, the stripe noise is prevented from interfering and defining a real edge, and the image after non-uniformity correction has higher structural similarity with an actual image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of image processing, and in particular relates to an infrared image non-uniformity correction method based on global and local features. Background Art

[0002] In recent years, with the continuous development of deep learning, the infrared image non-uniformity correction method based on deep learning can effectively overcome problems such as inconsistent response of infrared detectors and changes in temperature characteristics over time. It shows excellent robustness and non-uniformity correction capabilities, and has gradually become a key research area for experts and scholars.

[0003] However, although the non-uniformity correction method based on deep learning has strong generalization ability and non-uniformity correction performance, it often has the problem of high computational complexity. The image quality after non-uniformity correction of large models is high, but it has the disadvantages of large model scale and complex calculation, which is often difficult to achieve engineering application, and the single frame processing speed cannot meet the requirements of engineering fast processing. Although small models are easy to deploy and can meet engineering requirements, they have poor non-uniformity correction capabilities and are prone to blurred image details or even deformation. Therefore, studying how to balance the complexity of the model with the non-uniformity correction performance and designing a deep learning model with low complexity, fast processing speed and strong non-uniformity correction performance is an important problem to be solved in this field.

[0004] This method removes stripe noise through a feature extraction layer composed of row and column grayscale residuals and global grayscale feature extraction blocks to achieve low-cost, high-quality texture extraction, fuses multi-scale features in a point-by-point linear manner, and outputs a corrected image. While enhancing the model's detail retention capability, it greatly reduces the model complexity and achieves efficient, high-quality non-uniformity correction. Summary of the invention

[0005] (I) Purpose of the invention

[0006] The purpose of the present invention is to provide an infrared image non-uniformity correction method based on global and local features to solve the problem that small model images have poor detail retention and deformation, while complex models are large in scale and complex in calculation, making them difficult to apply in engineering.

[0007] (II) Technical solution

[0008] In order to achieve the above objectives and solve the above technical problems, the technical solutions of the present invention are as follows:

[0009] A method for correcting infrared image non-uniformity based on global and local features comprises the following steps:

[0010] Step 1: Perform a 3×3 convolution on the image I of size H×W×1 to initially extract image features and obtain a feature map I0 of size H×W×4;

[0011] Step 2: Input the feature map I0 into the feature extraction layer to obtain the H×W×4 feature map I0′;

[0012] Step 3: Downsample the feature map I0′ through convolution to output the feature map I1 with a size of H / 2×W / 2×8;

[0013] Step 4: Input the feature map I1 into the feature extraction layer to obtain the feature map I1′ of H / 2×W / 2×8;

[0014] Step 5: Downsample the feature map I1′ through convolution to output the feature map I2 with a size of H / 4×W / 4×16;

[0015] Step 6: Input the feature map I2 into the feature extraction layer to obtain the feature map I2′ of H / 4×W / 4×16;

[0016] Step 7: Downsample the feature map I2′ through convolution to output the feature map I3 with a size of H / 8×W / 8×32;

[0017] Step 8: Input the feature map I3 into the feature extraction layer to obtain the feature map I3′ of H / 8×W / 8×32;

[0018] Step 9: Upsample the feature map I3′ to output a feature map I4 with a size of H / 4×W / 4×16;

[0019] Step 10: Fuse the feature maps I4 and I3′ to output the feature map I4′ with a size of H / 4×W / 4×16;

[0020] Step 11: Input the feature map I4′ into the feature extraction layer to obtain the feature map I4″ of H / 4×W / 4×16;

[0021] Step 12: Upsample the feature map I4″ to output a feature map I5 with a size of H / 2×W / 2×8;

[0022] Step 13: Fuse the feature maps I5 and I2′ to output a feature map I5′ with a size of H / 2×W / 2×8;

[0023] Step 14: Input the feature map I5′ into the feature extraction layer to obtain the feature map I5″ of H / 2×W / 2×8;

[0024] Step 15: Upsample the feature map I5″ to output a feature map I6 with a size of H×W×4;

[0025] Step 16: Fuse the feature map I6 with I1′ to output the feature map I6′ with a size of H×W×4;

[0026] Step 17: Input the feature map I6′ into the feature extraction layer to obtain a feature map I6″ of H×W×4;

[0027] Step 18: I6″ is passed through a 3×3 convolution to output a rectified image I with a size of H×W×1 out .

[0028] Among them, in steps 2, 4, 6, 8, 11, 14, and 17, the feature extraction layer includes row, column grayscale, and global grayscale residual feature extraction blocks. The feature extraction layer is divided into two branches, one for global grayscale residual, and the other for row and column grayscale residual feature extraction modules connected in series. The feature extraction layer fuses the feature maps of the two branches by point-by-point superposition.

[0029] The column grayscale residual feature extraction module is divided into two branches. One is the residual block, which sums the feature matrix along the height direction of the feature map and then calculates the average, and then up-scales the dimension by copying to obtain a feature map with the same size as the input. The other branch is composed of 3×3 convolution, Relu activation function and 3×3 convolution cascade. Finally, the feature maps of the two branches are fused by point-by-point superposition.

[0030] The row grayscale residual feature extraction module is divided into two branches. One is the residual block, which sums the feature matrix along the width of the feature map and then calculates the mean, and then up-scales the dimension by copying to obtain a feature map with the same size as the input. The other branch is composed of 3×3 convolution, Relu activation function and 3×3 convolution cascade. Finally, the feature maps of the two branches are fused by point-by-point superposition.

[0031] The global grayscale residual feature extraction module sums the feature matrix along the channel direction of the feature map and then calculates the average, and then upscales the dimension by copying to obtain a feature map with the same size as the input.

[0032] Among them, in steps 3, 5, and 7, the downsampling method uses a 2×2 convolution method with a step size of 2 to downsample the feature map to half of the original size and double the number of channels.

[0033] Among them, in steps 9, 12, and 15, the upsampling method divides every four channels of the feature map into a group, and interlaces the feature values ​​of different channels to form a feature map with a scale increased by 1 times and the number of channels reduced by 4 times.

[0034] Among them, in steps 10, 13, and 16, the feature integration method sums the deep feature map in the channel direction of the feature map and then calculates the average, and then upgrades the dimension by copying to obtain a feature map with the same size as the shallow feature map, and fuses the feature maps by point-by-point superposition.

[0035] (III) Effective income

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. The present invention uses point-by-point superposition instead of convolution to fuse features, and uses interpolation instead of deconvolution to upsample, which can effectively reduce model complexity and improve the efficiency of non-uniformity correction. At the same time, it can unify the projection space of feature extraction layers at all levels, avoid redundant representation of feature maps, increase the number of model channels in disguise, and effectively improve the network detail retention ability, avoiding the loss of detail information of the image after non-uniformity correction;

[0038] 2. The feature extraction layer in the present invention can separate stripe noise based on the residual structure, so that the model can extract texture details from a relatively smooth image, avoid the interference of stripe noise in defining the real edge, and make the image after non-uniformity correction have a higher structural similarity with the actual image. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a structural diagram of the infrared image non-uniformity correction method based on global and local features of the present invention.

[0040] Figure 2 This is a structural diagram of the grayscale residual feature extraction module;

[0041] Figure 3 It is the structural diagram of the row grayscale residual feature extraction module;

[0042] Figure 4 It is the global grayscale residual structure and feature fusion structure diagram;

[0043] Figure 5 It is a schematic diagram of the upsampling method;

[0044] Figure 6 It is a diagram of the upsampling feature fusion process. DETAILED DESCRIPTION

[0045] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0046] like Figure 1 As shown, the present invention provides an infrared image non-uniformity correction method based on global and local features. The deployment process of the model is as follows.

[0047] S1, construct an infrared image non-uniformity correction method based on global and local features as a deep learning network model;

[0048] S2, training the model by using multiple groups of non-uniform noise images and corresponding noise-free infrared images as training sets to obtain a trained model;

[0049] The training process is continuously iterated, and the similarity between the model output and the noise-free image is evaluated through the loss function Loss, and then the model weights are adjusted to optimize the model. The loss function of the model is as follows:

[0050]

[0051] Among them I out with I true They represent the model output image and the noise-free image respectively, H and W represent the height and width of the image respectively, and the subscripts i and j represent the coordinates of the corresponding pixel positions in the image.

[0052] S3, input any non-uniform noise infrared image into the trained network model, and use the output of the model as the image non-uniform correction result.

[0053] Assume that the input image I is an infrared non-uniform image with a width and height of 1024 pixels and a channel number of 1.

[0054] The denoising process of the deployed and trained model is as follows:

[0055] Step 1: The input image I is subjected to a 3×3 convolution with a step size of 1 to initially extract image features, and a feature map I0 with a size of 1024×1024×4 is obtained;

[0056] Step 2: If Figure 4 , input the feature map I0 into the feature extraction layer to obtain a 1024×1024×4 feature map I0′;

[0057] Step 2.1: Figure 4 , input the feature map I0 into the global grayscale residual block, that is, sum the feature matrix along the channel direction of the feature map and then calculate the average value to obtain a feature map of size 1024×1024×1, and copy it along the channel direction to output a feature map I of size 1024×1024×4 0-0 ;

[0058] Step 2.2: If Figure 4 , input the feature map I0 into the row and column grayscale feature extraction modules in turn, and obtain the feature map I with a size of 1024×1024×4 0-1 ;

[0059] Among them, Figure 2, the feature map I0 is processed synchronously in two ways in the row grayscale feature extraction module. One way converts the feature map I0 into a 1×1024×4 feature map by taking the average value along the high direction, and copies it along the high direction to obtain a 1024×1024×4 feature map. The other way passes the feature map I0 through a 3×3 convolution with a step size of 1, a Relu activation function, and a 3×3 convolution with a step size of 1 to obtain a 1024×1024×4 feature map. The two branches are performed synchronously, and the two feature maps are superimposed point by point to obtain a 1024×1024×4 feature map.

[0060] Furthermore, if Figure 3 , the feature map The grayscale feature extraction module is divided into two processing paths. The average value is converted into a 1024×1×4 feature map along the width direction, and then copied along the width direction to obtain a 1024×1024×4 feature map. The 1024×1024×4 feature map is obtained by sequentially performing a 3×3 convolution with a step size of 1, a Relu activation function, and a 3×3 convolution with a step size of 1. The two branches are performed simultaneously, and the two feature maps are superimposed point by point to obtain a 1024×1024×4 feature map I 0-1 .

[0061] Step 2.1 and step 2.2 are performed simultaneously.

[0062] Step 2.3: If Figure 4 , the feature map I 0-0 with I 0-1 Superimpose point by point to obtain the feature map I0′;

[0063] Step 3: Downsample the feature map I0′ to a feature map I1 of size 512×512×8 through a 2×2 convolution with a stride of 2;

[0064] Step 4: Input the feature map I1 into the feature extraction layer to obtain a 512×512×8 feature map I1′;

[0065] Step 4.1: Figure 4 , input the feature map I1 into the global grayscale residual block, that is, sum the feature matrix along the channel direction of the feature map and then calculate the average value to obtain a feature map of size 512×512×1, and copy it along the channel direction to output a feature map I of size 512×512×8 1-0 ;

[0066] Step 4.2: If Figure 4 , input the feature map I1 into the row and column grayscale feature extraction modules in turn, and obtain a feature map I with a size of 512×512×8 1-1 ;

[0067] Among them, Figure 2 , the feature map I1 is processed synchronously in two ways in the row grayscale feature extraction module. One way converts the feature map I1 into a 1×512×8 feature map by taking the average value along the high direction, and copies it along the high direction to obtain a 512×512×8 feature map. The other way passes the feature map I1 through a 3×3 convolution with a step size of 1, a Relu activation function, and a 3×3 convolution with a step size of 1 to obtain a 512×512×8 feature map. The two branches are performed synchronously, and the two feature maps are superimposed point by point to obtain a 512×512×8 feature map.

[0068] Furthermore, if Figure 3 , the feature map The grayscale feature extraction module is divided into two processing paths. The average value is converted into a 512×1×8 feature map along the width direction, and then copied along the width direction to obtain a 512×512×8 feature map. The 512×512×8 feature map is obtained by sequentially performing a 3×3 convolution with a step size of 1, a Relu activation function, and a 3×3 convolution with a step size of 1. The two branches are performed simultaneously, and the two feature maps are superimposed point by point to obtain a 512×512×8 feature map I 1-1 .

[0069] Step 4.1 and step 4.2 are performed simultaneously.

[0070] Step 4.3: If Figure 4 , the feature map I 1-0 with I 1-1 Superimpose point by point to obtain the feature map I1′;

[0071] Step 5: Downsample the feature map I1′ to a feature map I2 of size 256×256×16 through a 2×2 convolution with a stride of 2;

[0072] Step 6: Input the feature map I2 into the feature extraction layer to obtain a 256×256×16 feature map I2′;

[0073] Step 6.1: Figure 4 , input the feature map I2 into the global grayscale residual block, that is, sum the feature matrix along the channel direction of the feature map and then calculate the average value to obtain a feature map of size 256×256×1, and copy it along the channel direction to output a feature map I of size 256×256×16 2-0 ;

[0074] Step 6.2: If Figure 4 , input the feature map I2 into the row and column grayscale feature extraction modules in turn, and obtain a feature map I with a size of 256×256×162-1 ;

[0075] Among them, Figure 2 , the feature map I2 is processed synchronously in two ways in the row grayscale feature extraction module. One way converts the feature map I2 into a 1×256×16 feature map by taking the average value along the high direction, and copies it along the high direction to obtain a 256×256×16 feature map. The other way passes the feature map I2 through a 3×3 convolution with a step size of 1, a Relu activation function, and a 3×3 convolution with a step size of 1 to obtain a 256×256×16 feature map. The two branches are performed synchronously, and the two feature maps are superimposed point by point to obtain a 256×256×16 feature map.

[0076] Furthermore, if Figure 3 , the feature map The grayscale feature extraction module is divided into two processing paths. The average value is converted into a 256×1×16 feature map along the width direction, and then copied along the width direction to obtain a 256×256×16 feature map. The 256×256×16 feature map is obtained by sequentially performing a 3×3 convolution with a step size of 1, a Relu activation function, and a 3×3 convolution with a step size of 1. The two branches are performed simultaneously, and the two feature maps are superimposed point by point to obtain a 256×256×16 feature map I 2-1 .

[0077] Step 6.1 and step 6.2 are performed simultaneously.

[0078] Step 6.3: If Figure 4 , the feature map I 2-0 with I 2-1 Superimpose point by point to obtain the feature map I2′;

[0079] Step 7: Downsample the feature map I2′ to a feature map I3 of size 128×128×32 through a 2×2 convolution with a stride of 2;

[0080] Step 8: Input the feature map I3 into the feature extraction layer to obtain a 128×128×32 feature map I3′;

[0081] Step 8.1: Figure 4 , input the feature map I3 into the global grayscale residual block, that is, sum the feature matrix along the channel direction of the feature map and then calculate the average value to obtain a feature map of size 128×128×1, and copy it along the channel direction to output a feature map I of size 128×128×32 3-0 ;

[0082] Step 8.2: If Figure 4, input the feature map I3 into the row and column grayscale feature extraction modules in turn, and obtain the feature map I with a size of 128×128×32 3-1 ;

[0083] Among them, Figure 2 , the feature map I3 is processed synchronously in two ways in the row grayscale feature extraction module. One way converts the feature map I3 into a 1×128×32 feature map by taking the average value along the high direction, and copies it along the high direction to obtain a 128×128×32 feature map. The other way passes the feature map I3 through a 3×3 convolution with a step size of 1, a Relu activation function, and a 3×3 convolution with a step size of 1 to obtain a 128×128×32 feature map. The two branches are performed synchronously, and the two feature maps are superimposed point by point to obtain a 128×128×32 feature map.

[0084] Furthermore, if Figure 3 , the feature map The grayscale feature extraction module is divided into two processing paths. The average value is converted into a 128×1×32 feature map along the width direction, and then copied along the width direction to obtain a 128×128×32 feature map. The 128×128×32 feature map is obtained by sequentially performing a 3×3 convolution with a step size of 1, a Relu activation function, and a 3×3 convolution with a step size of 1. The two branches are performed simultaneously, and the two feature maps are superimposed point by point to obtain a 128×128×32 feature map I 3-1 .

[0085] Step 8.1 and step 8.2 are performed simultaneously.

[0086] Step 8.3: If Figure 4 , the feature map I 3-0 with I 3-1 Superimpose point by point to obtain the feature map I3′;

[0087] Step 9: Figure 5 , the feature map I3′ is grouped into four channels and upsampled by interleaving interpolation to a feature map I4 of size 256×256×16;

[0088] Step 10: Figure 5 , the feature map I4 is averaged along the channel direction to obtain a feature map of size 256×256×1, and is copied along the channel direction to output a feature map of size 256×256×16. Finally, it is superimposed point by point with I3′ to form a feature map I4′ of size 256×256×16;

[0089] Step 11: Input the feature map I4′ into the feature extraction layer to obtain a 256×256×16 feature map I4″;

[0090] Step 11.1: Figure 4 , input the feature map I4′ into the global grayscale residual block, that is, sum the feature matrix along the channel direction of the feature map and then calculate the average value to obtain a feature map of size 256×256×1, and copy it along the channel direction to output a feature map I of size 256×256×16 4-0 ;

[0091] Step 11.2: If Figure 4 , input the feature map I4′ into the row and column grayscale feature extraction modules in turn, and obtain a feature map I with a size of 256×256×16 4-1 ;

[0092] Among them, Figure 2 , the feature map I4′ is processed synchronously in two ways in the row grayscale feature extraction module. One way converts the feature map I4′ into a 1×256×16 feature map by taking the average value along the high direction, and copies it along the high direction to obtain a 256×256×16 feature map. The other way passes the feature map I4′ through a 3×3 convolution with a step size of 1, a Relu activation function, and a 3×3 convolution with a step size of 1 to obtain a 256×256×16 feature map. The two branches are performed synchronously, and the two feature maps are superimposed point by point to obtain a 256×256×16 feature map I4.

[0093] Furthermore, if Figure 3 , the feature map The grayscale feature extraction module is divided into two processing paths. The average value is converted into a 256×1×16 feature map along the width direction, and then copied along the width direction to obtain a 256×256×16 feature map. The 256×256×16 feature map is obtained by sequentially performing a 3×3 convolution with a step size of 1, a Relu activation function, and a 3×3 convolution with a step size of 1. The two branches are performed simultaneously, and the two feature maps are superimposed point by point to obtain a 256×256×16 feature map I 4-1 .

[0094] Step 11.1 and step 11.2 are performed simultaneously.

[0095] Step 11.3: If Figure 4 , the feature map I 4-0 with I 4-1 Overlay point by point to obtain feature map I4″;

[0096] Step 12: The feature map I4″ is grouped into four channels and upsampled by interleaving interpolation to a feature map I5 of size 512×512×8;

[0097] Step 13: Figure 5 , the feature map I5 is averaged along the channel direction to obtain a feature map of size 512×512×1, and is copied along the channel direction to output a feature map of size 512×512×8, and finally superimposed with I2′ point by point to form a feature map I5× with a size of 512×512×8;

[0098] Step 14: Input the feature map I5′ into the feature extraction layer to obtain a 512×512×8 feature map I5″;

[0099] Step 14.1: Figure 4 , input the feature map I5′ into the global grayscale residual block, that is, sum the feature matrix along the channel direction of the feature map and then calculate the average value to obtain a feature map of size 512×512×1, and copy it along the channel direction to output a feature map I of size 512×512×8 5-0 ;

[0100] Step 14.2: If Figure 4 , input the feature map I5′ into the row and column grayscale feature extraction modules in turn, and obtain a feature map I with a size of 512×512×8 5-1 ;

[0101] Among them, Figure 2 , the feature map I5′ is processed synchronously in two ways in the row grayscale feature extraction module. One way converts the feature map I5′ into a 1×512×8 feature map by taking the average value along the high direction, and copies it along the high direction to obtain a 512×512×8 feature map. The other way passes the feature map I5′ through a 3×3 convolution with a step size of 1, a Relu activation function, and a 3×3 convolution with a step size of 1 to obtain a 512×512×8 feature map. The two branches are performed synchronously, and the two feature maps are superimposed point by point to obtain a 512×512×8 feature map.

[0102] Furthermore, if Figure 3 , the feature map The grayscale feature extraction module is divided into two processing paths. The average value is converted into a 512×1×8 feature map along the width direction, and then copied along the width direction to obtain a 512×512×8 feature map. The 512×512×8 feature map is obtained by sequentially performing a 3×3 convolution with a step size of 1, a Relu activation function, and a 3×3 convolution with a step size of 1. The two branches are performed simultaneously, and the two feature maps are superimposed point by point to obtain a 512×512×8 feature map I 5-1 .

[0103] Step 14.1 and step 14.2 are performed simultaneously.

[0104] Step 14.3: If Figure 4 , the feature map I 5-0 with I 5-1 Superimpose point by point to obtain the feature map I5″;

[0105] Step 15: The feature map I5″ is grouped into four channels and upsampled by interleaving interpolation to a feature map I6 of size 1024×1024×4;

[0106] Step 16: Figure 5 , the feature map I6 is averaged along the channel direction to obtain a feature map of size 1024×1024×1, and is copied along the channel direction to output a feature map of size 1024×1024×4, and finally superimposed point by point with I1′ to form a feature map I6′ of size 1024×1024×4;

[0107] Step 17: Input the feature map I6′ into the feature extraction layer to obtain a feature map I6″ of H×W×4;

[0108] Step 17.1: Figure 4 , input the feature map I6′ into the global grayscale residual block, that is, sum the feature matrix along the channel direction of the feature map and then calculate the average value to obtain a feature map of size 1024×1024×1, and copy it along the channel direction to output a feature map I of size 1024×1024×4 6-0 ;

[0109] Step 17.2: If Figure 4 , input the feature map I6′ into the row and column grayscale feature extraction modules in turn, and obtain a feature map I with a size of 1024×1024×4 6-1 ;

[0110] Among them, Figure 2 , the feature map I6′ is processed synchronously in two ways in the row grayscale feature extraction module. One way converts the feature map I6′ into a 1×1024×4 feature map by taking the average value along the high direction, and copies it along the high direction to obtain a 1024×1024×4 feature map. The other way passes the feature map I6′ through a 3×3 convolution with a step size of 1, a Relu activation function, and a 3×3 convolution with a step size of 1 to obtain a 1024×1024×4 feature map. The two branches are performed synchronously, and the two feature maps are superimposed point by point to obtain a 1024×1024×4 feature map.

[0111] Furthermore, if Figure 3 , the feature map The grayscale feature extraction module is divided into two processing paths. The average value is converted into a 1024×1×4 feature map along the width direction, and then copied along the width direction to obtain a 1024×1024×4 feature map. The 1024×1024×4 feature map is obtained by sequentially performing a 3×3 convolution with a step size of 1, a Relu activation function, and a 3×3 convolution with a step size of 1. The two branches are performed simultaneously, and the two feature maps are superimposed point by point to obtain a 1024×1024×4 feature map I 6-1 .

[0112] Step 17.1 and step 17.2 are performed simultaneously.

[0113] Step 17.3: If Figure 4 , the feature map I 6-0 with I 6-1 Overlay point by point to obtain feature map I6″;

[0114] Step 18: I6″ is subjected to a 3×3 convolution with a stride of 1 to output a rectified image I with a size of 1024×1024×1 out , corrected image I out This is the resulting image after non-uniform correction.

[0115] The above specific examples are used to illustrate the content of the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art of the present invention, according to the idea of ​​the present invention, several simple deductions, modifications or substitutions can be made.

[0116] The following is an experiment to illustrate the effect of the infrared image non-uniformity correction based on global and local features.

[0117] 1. Experimental conditions:

[0118] The experiment is based on Windows 10 operating system, 13th Gen Intel(R) Core(TM) i7-13700KF processor, 32G memory, Pytorch 1.13.1 framework, and trained by NVIDIA GeForce RTX 4090 graphics card.

[0119] The number of training iterations is 300, the training optimizer is Adam, the initial learning rate of the network is 0.001, and the learning rate adjustment strategy adopts the ReduceLROnPlateau adaptive method. When the network loss does not change for 50 rounds, the learning rate is reduced to half of the original value, and the minimum learning rate for the learning rate reduction is 0.0001.

[0120] 2. Experimental content and results:

[0121] Table 1 shows the comparison results of various non-uniformity correction algorithms.

[0122] Table 1 Performance comparison of non-uniformity correction algorithms

[0123]

[0124] The results in the table show that the non-uniformity correction performance of the traditional algorithm BM3D is the worst, with a PSNR of only 14.2dB and poor image quality after correction. The other three types of non-uniformity correction methods have stronger performance, with PSNRs all above 30dB. Among them, the correction method based on global and local features proposed in the present invention has the best performance, with a PSNR 13.2dB higher than the U-Net model and 2.9dB higher than SU-Net. The SSIM is comparable to SU-Net and 0.04 higher than U-Net. At the same time, the present invention also has advantages in terms of complexity. SU-Net has no engineering application value because its parameter volume is as high as 95.1M and the amount of calculation is nearly 60 times that of the present invention. Since the present invention abandons convolution and adopts superposition to fuse features, the parameter volume and calculation volume are significantly reduced compared to U-Net. Its parameter volume is about one-fifth of U-Net, and the calculation volume is reduced by about one-third. Therefore, the present invention has the advantages of strong non-uniformity correction performance of large models and low complexity of small models, and has high engineering application value.

[0125] The above contents are further detailed descriptions of the present invention in combination with specific implementation methods, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for correcting infrared image non-uniformity based on global and local features, characterized in that: The following steps are involved: Step 1: Perform a 3×3 convolution on the image I of size H×W×1 to initially extract image features and obtain a feature map I0 of size H×W×4; Step 2: Input the feature map I0 into the feature extraction layer to obtain the H×W×4 feature map I0′; Step 3: Downsample the feature map I0′ through convolution to output the feature map I1 with a size of H / 2×W / 2×8; Step 4: Input the feature map I1 into the feature extraction layer to obtain the feature map I1′ of H / 2×W / 2×8; Step 5: Downsample the feature map I1′ through convolution to output the feature map I2 with a size of H / 4×W / 4×16; Step 6: Input the feature map I2 into the feature extraction layer to obtain the feature map I2′ of H / 4×W / 4×16; Step 7: Downsample the feature map I2′ through convolution to output the feature map I3 with a size of H / 8×W / 8×32; Step 8: Input the feature map I3 into the feature extraction layer to obtain the feature map I3′ of H / 8×W / 8×32; Step 9: Upsample the feature map I3′ to output a feature map I4 with a size of H / 4×W / 4×16; Step 10: Fuse the feature maps I4 and I3′ to output the feature map I4′ with a size of H / 4×W / 4×16; Step 11: Input the feature map I4′ into the feature extraction layer to obtain the feature map I4″ of H / 4×W / 4×16; Step 12: Upsample the feature map I4″ to output a feature map I5 with a size of H / 2×W / 2×8; Step 13: Fuse the feature maps I5 and I2′ to output a feature map I5′ with a size of H / 2×W / 2×8; Step 14: Input the feature map I5′ into the feature extraction layer to obtain the feature map I5″ of H / 2×W / 2×8; Step 15: Upsample the feature map I5″ to output a feature map I6 with a size of H×W×4; Step 16: Fuse the feature map I6 with I1′ to output the feature map I6′ with a size of H×W×4; Step 17: Input the feature map I6′ into the feature extraction layer to obtain a feature map I6″ of H×W×4; Step 18: I6″ is passed through a 3×3 convolution to output a rectified image I with a size of H×W×1 out .

2. The infrared image non-uniformity correction method based on global and local features according to claim 1, characterized in that: In steps 2, 4, 6, 8, 11, 14, and 17, the feature extraction layer includes row grayscale, column grayscale, and global grayscale residual feature extraction blocks; the feature extraction layer is divided into two branches, one is the global grayscale residual, and the other is composed of a row grayscale residual feature extraction module and a column grayscale residual feature extraction module connected in series; the feature extraction layer fuses the feature maps of the two branches by point-by-point superposition.

3. The infrared image non-uniformity correction method based on global and local features according to claim 2, characterized in that: The column grayscale residual feature extraction module is divided into two branches; One path is the residual block, which sums the feature matrix along the height direction of the feature map and then calculates the average, and then upscales the dimension by copying to obtain a feature map with the same size as the input; The other route is composed of 3×3 convolution, ReLU activation function and 3×3 convolution cascade. Finally, the feature maps of the two branches are fused by point-by-point superposition.

4. The infrared image non-uniformity correction method based on global and local features according to claim 2, characterized in that: The row grayscale residual feature extraction module is divided into two branches; One path is the residual block, which sums the feature matrix along the width of the feature map and then calculates the average, and then upscales the dimension by copying to obtain a feature map with the same size as the input; The other route is composed of 3×3 convolution, ReLU activation function and 3×3 convolution cascade. Finally, the feature maps of the two branches are fused by point-by-point superposition.

5. The infrared image non-uniformity correction method based on global and local features according to claim 2, characterized in that: The global grayscale residual feature extraction module sums the feature matrix along the feature map channel direction and then calculates the average, and then upgrades the dimension in a copying manner to obtain a feature map consistent with the input size.

6. The infrared image non-uniformity correction method based on global and local features according to claim 1, characterized in that: In the steps 3, 5, and 7, the downsampling method uses a 2×2 convolution method with a step size of 2 to downsample the feature map to half of the original size and double the number of channels.

7. The infrared image non-uniformity correction method based on global and local features according to claim 1, characterized in that: In steps 9, 12, and 15, the upsampling method divides every four channels of the feature map into a group, and the feature values ​​of different channels are interlaced and spliced ​​to form a feature map with a scale increased by 1 times and the number of channels reduced by 4 times.

8. The infrared image non-uniformity correction method based on global and local features according to claim 1, characterized in that: In the steps 10, 13, and 16, the feature integration method sums the deep feature map in the channel direction of the feature map and then calculates the average, and then upgrades the dimension by copying to obtain a feature map with the same size as the shallow feature map, and fuses the feature maps by point-by-point superposition.