Method and device for correcting non-uniform noise of image

Abstract

The embodiment of the invention discloses a method and a device for correcting non-uniform noise of an image. The method comprises the following steps: acquiring an original image acquired by an infrared detector; de-noising the original image based on a preset de-noising algorithm to obtain a standard image, and taking the standard image as a label graph of the original image to generate a pairing data set; and inputting the paired data set into an infrared image non-uniformity correction algorithm, performing multi-scale feature extraction, fusion and residual calculation on the paired data set by using the infrared image non-uniformity correction algorithm to obtain an output corrected image, and performing algorithm optimization on the infrared image non-uniformity correction algorithm by using the corrected image and the paired data set. According to the infrared image non-uniformity correction algorithm, multi-scale feature extraction and fusion are carried out on the image, complex non-uniformity noise can be processed, in addition, the algorithm is optimized through a paired data set, and the calibration effect of the corrected image is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and particularly to a method and device for correcting non-uniform noise in images. Background Art

[0002] Nowadays, infrared imaging technology has been widely applied in fields such as medical diagnosis, environmental monitoring, security monitoring, etc. However, during the infrared detection and imaging process, even when receiving the same intensity of infrared radiation, temperature fluctuations of the detector, changes in drive signals and voltage signals, and non-uniformity of charge transfer efficiency will all cause inconsistent output responses of the detection units on the infrared detector focal plane array. This non-uniformity caused by the response of the detection image units will result in infrared image degradation, affecting the visual effect of the infrared image and the accuracy of subsequent detection tasks.

[0003] Therefore, researching infrared non-uniformity correction technology is of great significance for improving infrared imaging effects, preparing for downstream tasks of infrared detection, and promoting the development of infrared imaging technology.

[0004] Currently, there are some limitations in infrared non-uniformity correction methods. In a wide temperature range and complex scenarios, the correction effect is limited. Therefore, the existing infrared non-uniformity correction technology has the problem of poor correction effect. Summary of the Invention

[0005] Based on this, it is necessary to propose a method and device for correcting non-uniform noise in images to solve the above problems and achieve a better non-uniform image correction effect.

[0006] To achieve the above object, the first aspect of the present application provides a method for correcting non-uniform noise in images, the method comprising:

[0007] Obtain an original image collected by an infrared detector;

[0008] Denoise the original image based on a preset denoising algorithm to obtain a standard image, and use the standard image as the label map of the original image to generate a paired data set;

[0009] Input the paired data set into an infrared image non-uniformity correction algorithm to perform multi-scale feature extraction, fusion, and residual calculation on the paired data set using the infrared image non-uniformity correction algorithm, obtain an output corrected image, and use the corrected image and the paired data set to optimize the infrared image non-uniformity correction algorithm.

[0010] Further, the performing multi-scale feature extraction, fusion, and residual calculation on the paired data set using the infrared image non-uniformity correction algorithm to obtain an output corrected image specifically includes:

[0011] Performing multi-scale feature extraction and feature fusion on the original images in the paired dataset based on the infrared image non-uniform correction algorithm to obtain an aggregated feature map;

[0012] Calculating the residual based on the aggregated feature map and the original image to obtain the output corrected image.

[0013] Furthermore, the infrared image non-uniform correction algorithm includes U-Net, a multi-scale large kernel attention module, and a gated spatial attention unit;

[0014] Then, performing multi-scale feature extraction and feature fusion on the original image based on the infrared image non-uniform correction algorithm to obtain an aggregated feature map specifically includes:

[0015] Using the encoder of U-Net to perform shallow feature extraction on the original image to obtain an image feature matrix;

[0016] Based on the multi-scale large kernel attention module, performing multi-scale deep feature extraction and feature fusion on the image feature matrix to obtain an initial fused feature image;

[0017] Based on the gated spatial attention unit, performing context information aggregation according to the initial fused feature image to obtain an aggregated feature map.

[0018] Furthermore, based on the multi-scale large kernel attention module, performing multi-scale deep feature extraction and feature fusion on the image feature matrix to obtain an initial fused feature image specifically includes:

[0019] The multi-scale large kernel attention module uses depthwise convolution and dilated depthwise convolution to perform multi-scale feature extraction on the image feature matrix to obtain multi-scale feature information;

[0020] Using pointwise convolution to perform feature fusion on the multi-scale feature information to obtain an initial fused feature image.

[0021] Furthermore, the initial fused feature image is calculated by the following formula:

[0022]

[0023] In the formula, F MAB represents the initial fused feature image, PWConv represents pointwise convolution, DWDConv represents dilated depthwise separable convolution, DWConv represents depthwise separable convolution, represents the element-wise multiplication operation, n is the number of scales, Σ represents channel concatenation, k i represents the convolution kernel size of different scales, F input represents the image feature matrix.

[0024] Further, calculating a residual between the aggregated feature map and the original image to obtain an output corrected image, specifically including:

[0025] Decoding and reconstructing the aggregated feature map using the decoder of U-Net to obtain loss features;

[0026] Performing multi-scale depth feature fusion based on the loss features to obtain fused intermediate features;

[0027] Performing convolutional residual fusion based on the intermediate features to obtain fused target loss features;

[0028] Performing residual processing based on the target loss features and the original image to obtain an output corrected image.

[0029] Further, the intermediate features are calculated by the following formula:

[0030]

[0031] f(·) = UP(BN(Conv(Down(F1))))

[0032] In the formula, F2 represents the intermediate features, F1 represents the loss features, f1, f2, and f3 are network branches at scales of 2×2, 4×4, and 8×8 respectively, Up represents the upsampling operation, BN represents the batch normalization operation, Conv represents the convolution operation, Down represents the downsampling operation, represents channel concatenation.

[0033] Further, the target loss features are calculated by the following formula:

[0034] F3 = ResCBS(F2)

[0035]

[0036] In the formula, F3 represents the target loss features, ResCBS represents consecutive residual convolutional blocks, F2 represents the intermediate features, SE represents the self-attention mechanism, BN represents the batch normalization operation, and Conv represents the convolution operation.

[0037] Further, optimizing the infrared image non-uniform correction algorithm using the corrected image and the paired dataset specifically includes:

[0038] Calculating a residual between the standard image and the original image to obtain a residual label;

[0039] Calculating a loss function based on the corrected image, the target loss features, and the residual label;

[0040] Based on the backpropagation algorithm, the infrared image non-uniform correction algorithm is optimized using the loss function to obtain an optimized infrared image non-uniform correction algorithm;

[0041] The loss function is calculated according to the following formula:

[0042] L = αSSIM(F3, I gt1 ) + βMSE(I out , I gt2 )

[0043] In the formula, L represents the loss function, α and β are preset weight coefficients, I gt1 represents the residual label, F3 represents the target loss feature, I out represents the corrected image, and I gt2 represents the standard image.

[0044] To achieve the above object, a second aspect of the present application provides a device for correcting non-uniform noise of an image. The device includes: an image acquisition unit, an image processing unit, and an image correction unit;

[0045] The image acquisition unit is configured to acquire an original image collected by an infrared detector;

[0046] The image processing unit is configured to denoise the original image based on a preset denoising algorithm to obtain a standard image, and use the standard image as a label map of the original image to generate a paired data set;

[0047] The image correction unit is configured to input the paired data set into the infrared image non-uniform correction algorithm, so as to perform multi-scale feature extraction, fusion, and residual calculation on the paired data set using the infrared image non-uniform correction algorithm to obtain an output corrected image, and use the corrected image and the paired data set to optimize the infrared image non-uniform correction algorithm.

[0048] Adopting the embodiments of the present invention has the following beneficial effects:

[0049] An embodiment of the present invention provides a method for correcting non-uniform noise in images. The method includes: obtaining an original image collected by an infrared detector; denoising the original image based on a preset denoising algorithm to obtain a standard image, and using the standard image as a label map of the original image to generate a paired data set; inputting the paired data set into an infrared image non-uniform correction algorithm to perform multi-scale feature extraction, fusion, and residual calculation on the paired data set using the infrared image non-uniform correction algorithm to obtain an output corrected image, and using the corrected image and the paired data set to optimize the infrared image non-uniform correction algorithm. The infrared image non-uniform correction algorithm used in the present invention can process complex non-uniform noise by performing multi-scale feature extraction and fusion on the image. In addition, the algorithm is optimized using the paired data set, greatly improving the calibration effect of the corrected image. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0051] Among them:

[0052] Figure 1 is a schematic flowchart of the method for correcting non-uniform noise in images according to the embodiment of the present invention;

[0053] Figure 2 is the overall network structure diagram in the embodiment of the present invention;

[0054] Figure 3 is the structure diagram of the multi-scale large kernel attention module and the gated spatial attention unit in the embodiment of the present invention;

[0055] Figure 4 is the structure diagram of the fusion network in the embodiment of the present invention;

[0056] Figure 5 is the algorithm edge deployment process in the embodiment of the present invention;

[0057] Figure 6 is the structural block diagram of the device for correcting non-uniform noise in images according to the embodiment of the present invention;

[0058] Figure 7 is the internal structure diagram of the computer device in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0060] For an infrared detector, the output signal of the infrared detector is affected by the incident infrared radiation intensity, the gain coefficient, and the bias coefficient of the detection pixel. As shown in formula (1)

[0061] I(x,y) = G(x,y)·φ(x,y) + O(x,y) (1)

[0062] In formula (1), the output signal of the infrared detector is I(x,y), (x,y) represents the position of the pixel, G(x,y) represents the gain coefficient of the pixel, φ(x,y) represents the incident infrared radiation intensity, and O(x,y) represents the bias coefficient of the pixel.

[0063] Due to the differences in manufacturing processes and material characteristics, there are differences in the gain coefficient and bias coefficient of pixels among different pixels, resulting in non-uniformity of the output signal.

[0064] Based on this, the embodiments of the present invention propose a method for correcting image non-uniform noise. This method uses models such as convolutional neural networks based on deep learning to automatically and efficiently learn the complex non-linear response characteristics of the detector, providing higher correction accuracy and robustness. Especially for different large-scale data and complex scenarios, the deep learning model can extract features from large-scale and high-dimensional data sets, thereby improving the correction accuracy and reducing the correction residue. Reference can be made to Figure 1 , Figure 1 is a schematic flow chart of the method for correcting image non-uniform noise in the embodiments of the present invention. The method includes:

[0065] Step 110, obtain the original image collected by the infrared detector.

[0066] In the embodiments of the present invention, the original image is the image data collected by the infrared detector.

[0067] Before using the infrared image non-uniform correction algorithm to correct the original image, the infrared image non-uniform correction algorithm can be optimized using the original picture. To ensure the correction performance of the infrared image non-uniform correction algorithm, in the embodiments of the present invention, a series of image data of a blackbody with continuously changing temperature can be collected in a constant temperature laboratory environment as the original image. Since the temperature of the uncooled infrared detector will increase with the number of uses and time during operation, which will affect the imaging of the infrared detector. And if there is a large difference in the camera environment temperature, it will also affect the imaging of the infrared detector, and thus the non-uniform noise will also be affected by the temperature. Therefore, in order to enrich the training data to achieve data coverage and improve the correction performance, the image data of the blackbody with continuously changing temperature is selected to optimize the training of the infrared image non-uniform correction algorithm. For example, the temperature of the blackbody is at least 10 °C, and the temperature change step size can be 1 °C, 5 °C or 10 °C.

[0068] Step 120: Denoise the original image based on a preset denoising algorithm to obtain a standard image, and use the standard image as the label map of the original image to generate a paired data set.

[0069] In the embodiments of the present invention, one or more denoising algorithms can be used to remove the non-uniform noise in the original image to obtain a standard image, and the denoising algorithm can select an algorithm with better non-uniform correction effect.

[0070] Specifically, remove the non-uniform noise from all the collected original images to obtain the standard images corresponding to each original image. Then, using indicators such as PSNR and UR, select the standard image with the best correction effect. Each standard image with the best correction effect is the label map of the original image corresponding to it. An original image and the label map of this original image form a paired data. Input the paired data set into the infrared image non-uniform correction algorithm to perform image correction and algorithm optimization.

[0071] Step 130: Input the paired data set into the infrared image non-uniform correction algorithm to perform multi-scale feature extraction, fusion and residual calculation on the paired data set using the infrared image non-uniform correction algorithm to obtain the output corrected image, and use the corrected image and the paired data set to optimize the infrared image non-uniform correction algorithm.

[0072] In an embodiment of the present invention, an infrared image non-uniform correction algorithm is selected to perform non-uniform correction on the input image. The infrared image non-uniform correction algorithm adopts a multi-scale large kernel attention mechanism to capture global and local features of the input image and perform feature fusion to process complex non-uniform noise.

[0073] Specifically, the original image is input into the infrared image non-uniform correction algorithm. The infrared image non-uniform correction algorithm performs multi-scale feature extraction and feature fusion on the original image, performs residual processing based on the fused features and the original image features, and obtains and outputs the corrected image. In addition, the infrared image non-uniform correction algorithm is further optimized through the original image and the corresponding label map by using a paired data set, so as to perform image correction based on an algorithm with better calibration effect in the subsequent stage.

[0074] The image non-uniform noise correction method proposed by the present invention adapts to the non-uniform noise in complex scenes by performing multi-scale feature extraction and fusion on the image, and effectively improves the image correction effect.

[0075] In an embodiment of the present invention, a specific implementation manner of image correction by the infrared image non-uniform correction algorithm is proposed. The specific step 130 is to perform multi-scale feature extraction, fusion, and residual calculation on the paired data set by using the infrared image non-uniform correction algorithm to obtain the output corrected image, which specifically includes:

[0076] Step310: Perform multi-scale feature extraction and feature fusion on the original image based on the infrared image non-uniform correction algorithm to obtain an aggregated feature map.

[0077] Specifically, any one of the original images is selected for multi-scale feature extraction, and the extracted features are fused to obtain a fused aggregated feature map.

[0078] In an embodiment of the present invention, the infrared image non-uniform correction algorithm includes U-Net, a multi-scale large kernel attention module, a gated spatial attention unit, and a fusion network, which can be referred to Figure 2 , Figure 2 which is the overall network structure diagram in the embodiment of the present invention. Based on this network structure, Step310: Perform multi-scale feature extraction and feature fusion on the original image in the paired data set based on the infrared image non-uniform correction algorithm to obtain an aggregated feature map, which specifically includes:

[0079] Step311: Use the encoder of U-Net to perform shallow feature extraction on the original image to obtain an image feature matrix.

[0080] In the embodiment of the present invention, U-Net includes an encoder and a decoder, and the upsampling and downsampling operations of the codec implement the coupling of U-Net and the multi-scale large kernel attention module.

[0081] When inputting the original image into the infrared image non-uniformity correction algorithm, first, the encoder of U-Net is used to perform downsampling on the original image, gradually separating the structural information and noise information components of the original image to obtain the shallow features of the original image, and outputting an image feature matrix containing the shallow features. In an embodiment of the present invention, the encoder can be set to 3 levels, and there are multiple convolution activation normalization pairs in each layer, such as 3 pairs in each layer. The method of selecting the maximum pooling of the image features is used for downsampling, and the size can be set to 2×2.

[0082] Since the non-uniformity noise is usually closely related to the local details of the image, the U-Net structure can effectively separate the non-uniformity noise from the clean information of the image, providing a clear input for subsequent correction; in addition, infrared non-uniform images usually have complex noise patterns, which are likely to cause model overfitting, while the symmetric encoder-decoder structure of U-Net can effectively balance the feature extraction and image reconstruction processes, reducing the risk of overfitting.

[0083] Step312. Based on the multi-scale large kernel attention module, perform multi-scale deep feature extraction and feature fusion on the image feature matrix to obtain an initial fusion feature image.

[0084] Because infrared non-uniformity noise usually exhibits different characteristics at different scales, traditional single-scale correction methods are difficult to simultaneously process these noises at different scales, resulting in unsatisfactory correction effects. Therefore, the embodiment of the present invention effectively simulates the physical response mapping curves of infrared detectors at different scales that are lacking through the multi-scale large kernel attention module to extract the noise and important non-uniform features, and fuses the extracted features, so that the obtained initial fusion feature image contains richer information.

[0085] In an embodiment of the present invention, the multi-scale large kernel attention module may include a depth convolution kernel, a dilated depth convolution kernel, and a pointwise convolution kernel, which can be referred to Figure 3 , Figure 3 is the structural diagram of the multi-scale large kernel attention module and the gated spatial attention unit of the embodiment of the present invention. Based on Figure 3 the structure of the multi-scale combined attention module, Step312. Based on the multi-scale large kernel attention module, perform multi-scale deep feature extraction and feature fusion on the image feature matrix to obtain an initial fusion feature image, specifically including:

[0086] Step3121. The multi-scale large kernel attention module uses depth convolution and dilated depth convolution to perform multi-scale feature extraction on the image feature matrix to obtain multi-scale feature information.

[0087] Specifically, deep convolutions (such as 5×5, 7×7, 9×9) have a larger receptive field and can capture non-uniform noise patterns over a larger range. Based on the fact that deep convolutions can effectively extract large-scale noise features in images, more comprehensive information is provided for subsequent correction. Dilated convolutions further expand the receptive field without increasing the computational cost by introducing a dilation rate (such as 2, 3, 4), and can capture the context information of non-uniform noise over a larger range, which is suitable for dealing with complex noise patterns in infrared images, such as stripe noise and block noise.

[0088] Through the above feature fusion mechanism, non-uniform noise at different scales can be processed simultaneously, avoiding the limitations of single-scale correction.

[0089] Step3122. Use pointwise convolution to perform feature fusion on multi-scale feature information to obtain an initial fused feature image.

[0090] Specifically, pointwise convolution can effectively integrate feature information at different scales to generate a more representative feature image.

[0091] In the embodiment of the present invention, through the combination of deep convolution, dilated deep convolution, and pointwise convolution, non-uniform noise at large scales and small scales can be processed simultaneously, ensuring that the non-uniform correction algorithm for infrared images has good performance at different scales.

[0092] Based on Figure 3 the multi-scale large kernel attention module structure in, an embodiment of the present invention also proposes a generation method for the initial fused feature image. Specifically, the initial fused feature image is calculated through formulas (2) and (3):

[0093]

[0094] In the formula, F MAB represents the initial fused feature image, PWConv represents pointwise convolution, DWDConv represents dilated depthwise separable convolution, DWConv represents depthwise separable convolution, represents the element-wise multiplication operation, n is the number of scales, Σ represents channel concatenation, k i represents the convolution kernel size of different scales, and F input represents the image feature matrix.

[0095] Step313. Based on the gated spatial attention unit, perform context information aggregation according to the initial fused feature image to obtain an aggregated feature map.

[0096] The gated spatial attention unit is used to adaptively aggregate the context information of the initial fused feature image, capture the local and global dependencies in the initial fused feature image, dynamically adjust the fusion weights of features at each scale, achieve the plasticity of information transmission, and further enhance the expressiveness of the image. For reference, see Figure 3 the structure of the gated spatial attention unit in

[0097] In an embodiment of the present invention, the aggregated feature map is calculated by formula (4):

[0098]

[0099] In the formula, F GSAU represents the aggregated feature map, PWConv represents pointwise convolution, DWDConv represents depthwise separable convolution with dilation, DWConv represents depthwise separable convolution, and F MAB represents the initial fused feature.

[0100] The infrared non-uniformity noise is not only related to the response characteristics of individual pixels, but also closely related to the context information of the surrounding pixels. For example, the non-uniformity noise in some areas may appear as local stripes or blocky noise, and these noise patterns can be effectively corrected through context information. Therefore, by focusing on the context information of the image to further optimize the feature map, the image correction effect of the algorithm can be effectively improved.

[0101] In an embodiment of the present invention, the multi-scale large kernel attention module and the gated spatial attention unit form the MAB module, and the MAB module can be used in multiple series in the network structure. For example, 2 MABs are stacked, and the actual number of uses can be adjusted according to the strength of the actual non-uniform features.

[0102] Step320: Calculate the residual based on the aggregated feature map and the original image to obtain the output corrected image.

[0103] By performing residual processing on the aggregated feature map obtained by feature fusion and the original image, a corrected image with better effect can be obtained.

[0104] In an embodiment of the present invention, the network structure of the infrared image non-uniformity correction algorithm also includes a fusion network. The fusion network can utilize the existing feature information to deeply improve the self-adaptability and effectiveness of feature fusion through repeated residual operations. For specific reference, see Figure 4 , Figure 4 is the structure diagram of the fusion network in an embodiment of the present invention. The fusion network includes a multi-scale residual fusion module and a residual convolution block. Based on Figure 4The fusion network structure, Step 320, perform residual calculation based on the aggregated feature map and the standard image to obtain the output corrected image, specifically including:

[0105] Step 321, use the decoder of U-Net to decode and reconstruct the aggregated feature map to obtain loss features.

[0106] In the embodiment of the present invention, the decoder of U-Net is used to perform transposed convolution upsampling operation to gradually restore the spatial noise information of the image from the abstract separated features of the aggregated feature map. The decoder maps the deep features back to the shallow space and restores the detailed features of the image, that is, decodes and reconstructs the feature vectors of the aggregated feature map, restores the aggregated feature map to the original size, and the restored image features are the loss features.

[0107] During the decoding process, there are skip connections with feature filtering modules starting from the encoder of U-Net to separate the structural information and noise components to ensure the purity of cross-layer feature fusion. Among them, the feature filtering module is a group of convolutional blocks.

[0108] Step 322, perform multi-scale depth feature fusion according to the loss features to obtain the fused intermediate features.

[0109] Specifically, input the loss features into the multi-scale residual fusion network in the fusion network for depth feature fusion, which can refer to Figure 4 the multi-scale residual fusion network in. This network has three scales and a residual connection, and each layer includes upsampling, downsampling, and convolutional blocks. Based on this multi-scale residual fusion network, the intermediate features can be represented by formulas (5) and (6):

[0110]

[0111] f(·) = UP(BN(Conv(Down(F1)))) (6)

[0112] In the formula, F2 represents the intermediate features, F1 represents the loss features, f1, f2, and f3 are the network branches at scales of 2×2, 4×4, and 8×8 respectively, Up represents the upsampling operation, the upsampling operation can use nearest neighbor interpolation, BN represents the batch normalization operation, Conv represents the convolutional operation, Down represents the downsampling operation, and the downsampling operation can be the average pooling method, represents channel concatenation.

[0113] Step 323, perform convolutional residual fusion according to the intermediate features to obtain the fused target loss features.

[0114] Specifically, input the intermediate features into the residual convolutional block for residual fusion to obtain the fused target loss features. InFigure 4 In the residual convolution block, 6 convolutional residual fusion blocks are stacked. Based on this, the target loss feature can be expressed by formulas (7) and (8):

[0115] F3 = ResCBS(F2) (7)

[0116]

[0117] In the formula, F3 represents the target loss feature, ResCBS represents consecutive residual convolution blocks, F2 represents intermediate features, SE represents the self-attention mechanism, BN represents batch normalization operation, Conv represents convolution operation, and each convolution is followed by activation and batch normalization operations. The activation function usually selects Relu.

[0118] Step324. Perform residual processing on the target loss feature and the standard image to obtain the output corrected image.

[0119] In the embodiment of the present invention, after the input based on the original image, the corrected image output by the infrared image non-uniform correction algorithm is calculated by the residual method, that is, by performing residual processing on the target loss feature and the original image.

[0120] In order to achieve the optimal correction effect, an embodiment of the present invention uses the weighted sum of the mean square error and the structural similarity index as the loss function, and optimizes the infrared image non-uniform correction algorithm based on the loss function. Among them, the mean square error points to the corrected image, and the structural similarity points to the non-uniform noise, so that the loss function not only ensures the pixel-level accuracy of the corrected image, but also retains the structural information of the image, greatly improving the correction effect of the optimized infrared image non-uniform correction algorithm. Specifically, step 130. Use the corrected image and the paired dataset to optimize the infrared image non-uniform correction algorithm, specifically including:

[0121] step510. Calculate the residual according to the standard image and the original image to obtain the residual label.

[0122] step520. Calculate the loss function according to the corrected image, the target loss feature and the paired dataset.

[0123] In the embodiment of the present invention, the loss function is calculated according to formula (9):

[0124] L = αSSIM(F3, I gt1 ) + βMSE(I out , I gt2 )

[0125] In the formula, L represents the loss function, α and β are preset weight coefficients, I gt1Indicates the residual label, which is obtained based on the residual between the original image and the standard image. F3 represents the target loss feature, and I out Indicates the corrected image, and I gt2 Indicates the standard image.

[0126] Step 530: Optimize the non-uniform correction algorithm for infrared images using the loss function based on the backpropagation algorithm to obtain the optimized non-uniform correction algorithm for infrared images.

[0127] Specifically, the network parameters in the non-uniform correction algorithm for infrared images are optimized using the backpropagation algorithm. The loss function is minimized through optimization algorithms such as gradient descent, and the network parameters are continuously adjusted to improve the correction effect of the non-uniform correction algorithm for infrared images. In the embodiments of the present invention, the Adam optimizer can be used to optimize the non-uniform correction algorithm for infrared images, and its learning rate is set to 0.001. During the optimization process, the network parameters are adjusted through methods such as cross-validation to prevent overfitting and ensure the generalization ability and correction effect of the non-uniform correction algorithm for infrared images.

[0128] After completing the training and optimization of the non-uniform correction algorithm for infrared images, the algorithm model can be deployed. The trained and optimized algorithm model is configured for the host environment from the original format to the domestic processing board model. First, during model training, the torch.save statement is used to save the network structure and training weights in a.pth file at the same time. Second, after completing the model training, the onnx.export function provided by the pytorch library is called to convert the pth model into an onnx model. Third, the relevant environment is configured with reference to the manual of the domestic processor, and the onnx model is quantized and converted to obtain the processor model, which is finally deployed on the domestic processor platform.

[0129] Reference can be made to Figure 5 , Figure 5 which is the algorithm edge deployment process of the embodiments of the present invention, specifically including:

[0130] (1) Install the vmare virtual machine and load the ubuntu20 image;

[0131] (2) Install anaconda in the virtual machine;

[0132] (3) Create a virtual environment with python = 3.8 and install the relevant dependent libraries.

[0133] Finally, for the upper computer display, load the relevant tool libraries of the edge deployment platform (for example, the rknn-toolkit2 library can be loaded for the rk3588 edge computing platform), implement the algorithm call according to the official manual, and design the algorithm display interface using c++Qt, including displaying the images before and after correction, relevant processing metrics, and algorithm execution parameters, etc.

[0134] Through the above embodiments, the present invention realizes an efficient and accurate infrared image non-uniformity correction method. The method makes full use of the feature extraction of multi-scale large kernel attention and the deep learning ability of the U-net structure. Through the multi-layer optimization of the fusion mechanism, the correction effect is significantly improved, and the problems of insufficient correction accuracy and non-uniformity noise residue in traditional methods are solved.

[0135] In the embodiments of the present invention, a correction device for image non-uniform noise is also proposed. Refer to Figure 6 , Figure 6 which is the structural block diagram of the correction device for image non-uniform noise in the embodiments of the present invention. The device includes: an image acquisition unit 601, an image processing unit 602, and an image correction unit 603.

[0136] The image acquisition unit 601 is used to acquire the original image collected by the infrared detector.

[0137] The image processing unit 602 is used to denoise the original image based on a preset denoising algorithm to obtain a standard image, and use the standard image as the label map of the original image to generate a paired data set.

[0138] The image correction unit 603 is used to input the paired data set into the infrared image non-uniformity correction algorithm for image correction, so as to perform multi-scale feature extraction, fusion, and residual calculation on the paired data set using the infrared image non-uniformity correction algorithm to obtain loss features, and perform residual processing based on the loss features and the paired data set to obtain the output corrected image.

[0139] The infrared image non-uniformity correction algorithm used by the correction device for image non-uniform noise proposed in the embodiments of the present invention can process complex non-uniformity noise through multi-scale feature extraction and fusion of the image. In addition, the algorithm is optimized through the paired data set, effectively improving the calibration effect of the corrected image.

[0140] Figure 7 shows the internal structure diagram of a computer device in an embodiment of the present invention. The computer device can specifically be a terminal or a system. As Figure 7 shown, the computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and can also store a computer program. When the computer program is executed by the processor, the processor can implement each step in the above method embodiments. The internal memory can also store a computer program. When the computer program is executed by the processor, the processor can execute each step in the above method embodiments. Those skilled in the art can understand, Figure 7The structure shown is only a block diagram of some of the structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different component layout.

[0141] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor is caused to execute each step in the above method embodiment.

[0142] In one embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by the processor, the processor is caused to execute each step in the above method embodiment.

[0143] Those of ordinary skill in the art can understand that all or part of the processes of implementing the above method embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it may include the processes of the above method embodiments. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in this application may include non-volatile and / or volatile memories. Non-volatile memories may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0144] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0145] The above embodiments only illustrate several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation to the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several variations and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A method for correcting non-uniform noise in an image, characterized in that, The method includes: Obtaining the original image collected by an infrared detector; Denosing the original image based on a preset denoising algorithm to obtain a standard image, and using the standard image as the label map of the original image to generate a paired dataset; Inputting the paired dataset into an infrared image non-uniformity correction algorithm to perform multi-scale feature extraction, fusion, and residual calculation on the paired dataset using the infrared image non-uniformity correction algorithm to obtain an output corrected image, and using the corrected image and the paired dataset to optimize the infrared image non-uniformity correction algorithm.

2. The method according to claim 1, wherein The performing multi-scale feature extraction, fusion, and residual calculation on the paired dataset using the infrared image non-uniformity correction algorithm to obtain an output corrected image specifically includes: Performing multi-scale feature extraction and feature fusion on the original image in the paired dataset based on the infrared image non-uniformity correction algorithm to obtain an aggregated feature map; Performing residual calculation based on the aggregated feature map and the original image to obtain an output corrected image.

3. The method according to claim 2, characterized in that, The infrared image non-uniformity correction algorithm includes U-Net, a multi-scale large kernel attention module, and a gated spatial attention unit; Then the performing multi-scale feature extraction and feature fusion on the original image based on the infrared image non-uniformity correction algorithm to obtain an aggregated feature map specifically includes: Using the encoder of U-Net to perform shallow feature extraction on the original image to obtain an image feature matrix; Based on the multi-scale large kernel attention module, performing multi-scale deep feature extraction and feature fusion on the image feature matrix to obtain an initial fused feature image; Based on the gated spatial attention unit, performing context information aggregation according to the initial fused feature image to obtain an aggregated feature map.

4. The method according to claim 3, wherein The performing multi-scale deep feature extraction and feature fusion on the image feature matrix based on the multi-scale large kernel attention module to obtain an initial fused feature image specifically includes: The multi-scale large kernel attention module uses depth convolution and dilated depth convolution to perform multi-scale feature extraction on the image feature matrix to obtain multi-scale feature information; Using pointwise convolution to perform feature fusion on the multi-scale feature information to obtain an initial fused feature image.

5. The method according to claim 4, wherein The initial fused feature image is calculated by the following formula: Where, F MAB represents the initial fused feature image, PWConv represents pointwise convolution, DWDConv represents depthwise separable convolution with dilation, DWConv represents depthwise separable convolution, represents an element-wise multiplication operation, n is the number of scales, Σ represents channel concatenation, k i represents the convolution kernel sizes of different scales, and F input represents the image feature matrix.

6. The method according to claim 2, wherein The performing residual calculation based on the aggregated feature map and the original image to obtain an output corrected image specifically includes: Using the decoder of U-Net to decode and reconstruct the aggregated feature map to obtain a loss feature; Performing multi-scale depth feature fusion according to the loss feature to obtain a fused intermediate feature; Performing convolutional residual fusion according to the intermediate feature to obtain a fused target loss feature; Performing residual processing based on the target loss feature and the original image to obtain an output corrected image.

7. The method according to claim 6, wherein The intermediate feature is calculated by the following formula: f(·) = UP(BN(Conv(Down(F1)))) Wherein, F2 represents the intermediate feature, F1 represents the loss feature, f1, f2, and f3 are network branches at scales of 2×2, 4×4, and 8×8 respectively, Up represents the upsampling operation, BN represents the batch normalization operation, Conv represents the convolution operation, Down represents the downsampling operation, and ⊕ represents channel concatenation.

8. The method according to claim 6, wherein The target loss feature is calculated by the following formula: F3 = ResCBS(F2) Wherein, F3 represents the target loss feature, ResCBS represents a continuous residual convolution block, F2 represents the intermediate feature, SE represents the self-attention mechanism, BN represents the batch normalization operation, and Conv represents the convolution operation.

9. The method according to claim 6, wherein The algorithm optimization of the infrared image non-uniform correction algorithm using the corrected image and the paired dataset specifically includes: Calculating the residual based on the standard image and the original image to obtain a residual label; Calculating a loss function based on the corrected image, the target loss feature, and the residual label; Optimizing the infrared image non-uniform correction algorithm using the loss function based on the backpropagation algorithm to obtain an optimized infrared image non-uniform correction algorithm; The loss function is calculated by the following formula: L = αSSIM(F3, I gt1 ) + βMSE(I out , I gt2 ) Wherein, L represents a loss function, α and β are preset weight coefficients, I gt1 represents the residual label, F3 represents the target loss feature, I out represents the corrected image, I gt2 represents the standard image.

10. An apparatus for correcting non-uniform noise of an image, characterized in that, The device includes: an image acquisition unit, an image processing unit, and an image correction unit; The image acquisition unit is used to acquire the original image collected by the infrared detector; The image processing unit is used to denoise the original image based on a preset denoising algorithm to obtain a standard image, and use the standard image as the label map of the original image to generate a paired dataset; The image correction unit is used to input the paired dataset into the infrared image non-uniform correction algorithm to perform multi-scale feature extraction, fusion, and residual calculation on the paired dataset using the infrared image non-uniform correction algorithm to obtain an output corrected image, and use the corrected image and the paired dataset to perform algorithm optimization on the infrared image non-uniform correction algorithm.

Images