A method and device for correcting non-uniform noise of infrared images
Through the adaptive parameter update processing and the DeNU and sigmaN parameter adjustment methods, the problems of inhomogeneous noise correction accuracy and efficiency in infrared image processing are solved, and efficient noise removal and image quality optimization of infrared images are achieved.
Patent Information
- Application Number
- CN202411349908.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-26
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-09-26
AI Technical Summary
The existing infrared image processing methods have low accuracy and efficiency when processing non-uniform noise, which affects the image's clarity, contrast and target recognition capabilities.
Preprocessing by acquiring infrared images, including format conversion and data amplification, and then adaptive parameter update processing based on frame count (such as DeShading, DeFPN, DeStrip parameter update), combined with DeNU parameter update and sigmaN parameter adjustment, finally data reduction Steps to complete non-uniform noise correction.
It realizes accurate matching of noise characteristics under different scenes and lighting conditions, effectively removes fixed mode noise and dynamically changing striped noise in the image, optimizes the overall brightness and contrast of the image, and improves the stability, accuracy and efficiency of the correction effect. .
Smart Images

Figure CN118864304B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the technical field of image processing, and in particular to a method and device for correcting non-uniform noise of infrared images. Background Art
[0002] In the field of infrared imaging, infrared detectors often have significant non-uniform noise problems in the collected infrared images due to differences in production processes, material properties, and readout circuit design. These noises are mainly manifested as fixed pattern noise, stripe noise (such as vertical stripe noise), and irregular speckle noise on the image. Non-uniform noise seriously affects the quality of infrared images, reducing image clarity, contrast, and target recognition capabilities, especially in low contrast and complex backgrounds, where this effect is more obvious.
[0003] Traditional infrared image processing methods often focus on a single type of noise suppression, such as reducing random noise through simple filtering techniques, but these methods have limited effects when dealing with non-uniform noise, and have low processing accuracy and efficiency. Summary of the invention
[0004] The embodiments of the present application provide a method and device for correcting non-uniform noise of infrared images, so as to solve the technical problems of low accuracy and efficiency of existing methods for correcting non-uniform noise of infrared images.
[0005] In a first aspect, an embodiment of the present application provides a method for correcting non-uniform noise of an infrared image, comprising: acquiring an infrared image, and preprocessing the infrared image to obtain a preprocessed infrared image, wherein the preprocessing includes: format conversion and data amplification; based on the frame number of the infrared image, performing target processing on the infrared image to obtain a first infrared image, wherein the target processing includes: adaptive parameter update processing or non-parameter update processing, and the adaptive parameter update processing includes one of the following: DeShading parameter update processing, DeFPN parameter update processing and DeStrip parameter update processing; based on the frame number of the infrared image, performing DeNU parameter update processing on the first infrared image to obtain a second infrared image; performing sigmaN parameter update processing on the second infrared image to obtain a third infrared image; performing data reduction on the third infrared image to obtain an infrared image that has completed non-uniform noise correction.
[0006] Furthermore, the target processing is a DeShading parameter update processing, then based on the frame number of the infrared image, the infrared image is subjected to target processing to obtain a first infrared image, including: filtering and cropping the infrared image to obtain a target infrared image; calculating the dot product between the target infrared image and a preprocessing matrix to obtain a first matrix, wherein the preprocessing matrix includes: a resShading matrix and a resShading4 matrix, and the value range of the resShading matrix and the resShading4 matrix are both (-2 13 , 2 13 ); summing the first matrices to obtain a summation result; based on the summation result and a preset constant, updating the adaptive parameter alphaShading and the adaptive parameter alphaShShading4 to obtain the first infrared image.
[0007] Further, if the target processing is DeFPN parameter update processing, the infrared image is subjected to target processing based on the frame number of the infrared image to obtain a first infrared image, including: performing mean filtering and cropping without zero-padding edges on the infrared image to obtain a second matrix and a third matrix, wherein the second matrix and the third matrix have the same size; subtracting the second matrix from the third matrix to obtain a fourth matrix; performing dot multiplication and summing of the fourth matrix and the FP_DeTrnd matrix to obtain a target constant, wherein the data range of the FP_DeTrnd matrix is (−2 12 ,2 12 ); based on the target constant, the adaptive parameter FPN is updated to obtain the first infrared image.
[0008] Further, if the target processing is DeStrip parameter update processing, the infrared image is subjected to target processing based on the frame number of the infrared image to obtain a first infrared image, including: subtracting a first zero matrix from the infrared image to obtain a fifth matrix; performing Wiener filtering on the fifth matrix to obtain a sixth matrix, and subtracting the sixth matrix from the infrared image to obtain a seventh matrix; calculating the median of each row and column in the seventh matrix, and assigning the values in each row and column to the corresponding medians to obtain an eighth matrix and a ninth matrix; calculating the sum of the eighth matrix and the ninth matrix to obtain a tenth matrix, and calculating the difference between the means of the tenth matrix and the tenth matrix to obtain an eleventh matrix; based on the eleventh matrix, iteratively updating the adaptive parameter Strip until the number of iterations reaches a first preset number of times to obtain the first infrared image.
[0009] Further, the first infrared image is subjected to DeNU parameter update processing to obtain a second infrared image, including: using the processed infrared image to subtract the adaptive parameter NU to obtain a first calculation result; performing Gaussian filtering on the first calculation result to obtain a first filtering result, and using the processed infrared image to subtract the first filtering result to obtain a second calculation result; performing Gaussian filtering on the absolute value of the second calculation result to obtain a second filtering result; performing equal-interval sampling on the second filtering result to obtain a first array, and determining the static variables corresponding to the sampling array; performing weighted summation on the first infrared image and the second zero matrix to obtain a twelfth matrix; calculating the absolute value of the processed infrared image minus the twelfth matrix, and performing weighted summation on the absolute value and the third zero matrix to obtain a thirteenth matrix; performing equal-interval sampling on the thirteenth matrix to obtain a second array; iteratively updating the adaptive parameter NU based on the first array and the second array until the number of iterations reaches a second preset number of times to obtain the second infrared image.
[0010] Furthermore, the second infrared image is subjected to sigmaN parameter update processing to obtain a third infrared image, including: performing Gaussian filtering on the second infrared image to obtain a third filtering result, and subtracting the third filtering result from the second infrared image to obtain a third calculation result; performing Gaussian filtering on the absolute value of the third calculation result to obtain a fourth filtering result; subtracting the adaptive parameter sigmaN from the square value of the fourth filtering result to obtain a fourth calculation result, and determining the larger value between the fourth calculation result and 0; calculating the dot product between the larger value and the third filtering result to obtain a fifth calculation result; calculating the dot division between the fifth calculation result and the square value of the fourth filtering result to obtain a fifth calculation result; assigning the sum of the fifth calculation result and the third filtering result to the second infrared image to obtain the third infrared image.
[0011] Furthermore, the method also includes: counting the median of the square values of the fourth filtering results; determining the histogram statistical range based on the median of the square values of the fourth filtering results; determining the horizontal coordinate of the highest column within the histogram statistical range; and updating the adaptive parameter sigmaN based on the horizontal coordinate.
[0012] In a second aspect, an embodiment of the present application provides a device for correcting non-uniform noise of an infrared image, comprising: an acquisition unit, for acquiring an infrared image, and preprocessing the infrared image to obtain a preprocessed infrared image, wherein the preprocessing includes: format conversion and data amplification; a first processing unit, for performing target processing on the infrared image based on the frame number of the infrared image to obtain a first infrared image, wherein the target processing includes: adaptive parameter update processing or non-parameter update processing, and the adaptive parameter update processing includes one of the following: DeShading parameter update processing, DeFPN parameter update processing and DeStrip parameter update processing; a second processing unit, for performing DeNU parameter update processing on the first infrared image based on the frame number of the infrared image to obtain a second infrared image; a third processing unit, for performing sigmaN parameter update processing on the second infrared image to obtain a third infrared image; and a fourth processing unit, for performing data reduction on the third infrared image to obtain an infrared image that has completed non-uniform noise correction.
[0013] In a third aspect, an embodiment of the present application provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement the method described in the first aspect above.
[0014] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing a computer program, which, when executed by a computer, implements the method described in the first aspect above.
[0015] In the embodiment of the present invention, the inter-frame changes of infrared images are comprehensively considered, and the accurate matching of noise characteristics under different scenes and lighting conditions is achieved through adaptive parameter update processing (such as DeShading, DeFPN, and DeStrip parameter updates). Furthermore, combined with the DeNU parameter update and sigmaN parameter adjustment, it is not only possible to effectively remove the fixed pattern noise and dynamically changing stripe noise in the image, but also to optimize the overall brightness and contrast of the image while maintaining the image details. Finally, through the data reduction step, the high fidelity and low-latency output of the processed image are guaranteed, which is suitable for various high-performance infrared imaging applications, and can adapt to a wider range of image conditions, thereby improving the stability, accuracy, and efficiency of the correction effect.
[0016] These and other aspects of the present application will become more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0018] Figure 1 A flowchart of a method for correcting non-uniform noise of an infrared image provided by an embodiment of the present application;
[0019] Figure 2 A flowchart of the DeShading parameter update process provided in an embodiment of the present application;
[0020] Figure 3 A flowchart of the DeFPN parameter update process provided in an embodiment of the present application;
[0021] Figure 4 A flowchart of the DeStrip parameter update process provided in an embodiment of the present application;
[0022] Figure 5 A flowchart of the DeNU parameter update process provided in an embodiment of the present application;
[0023] Figure 6 A flowchart of the sigmaN parameter update process provided in an embodiment of the present application;
[0024] Figure 7 A schematic diagram of a non-uniform noise correction device for infrared images provided in an embodiment of the present application;
[0025] Figure 8 A schematic diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0027] Embodiment 1:
[0028] According to an embodiment of the present invention, an embodiment of a method for correcting non-uniform noise of an infrared image is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0029] Figure 1 FIG. 1 is a flow chart of a method for correcting non-uniform noise of an infrared image according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:
[0030] Step S102, acquiring an infrared image, and preprocessing the infrared image to obtain a preprocessed infrared image, wherein the preprocessing includes: format conversion and data amplification;
[0031] In the embodiment of the present invention, taking the infrared image as a 14-bit unsigned integer as an example, dr(dynamicrange)=14, the data range is [0, 2 dr ).
[0032] First, enlarge the infrared image times, of which , the data range becomes After all modules have been processed, the data is reduced The infrared image after non-uniform noise correction is finally of the same magnitude as the acquired infrared image. Because the magnification and reduction multiples are powers of 2, left shift is used to achieve magnification, and right shift is used to achieve reduction.
[0033] Step S104, performing target processing on the infrared image based on the frame number of the infrared image to obtain a first infrared image, wherein the target processing includes: adaptive parameter update processing or parameter non-update processing, and the adaptive parameter update processing includes one of the following: DeShading parameter update processing, DeFPN parameter update processing and DeStrip parameter update processing;
[0034] Step S106, performing DeNU parameter update processing on the first infrared image based on the frame number of the infrared image to obtain a second infrared image;
[0035] Step S108, performing sigmaN parameter update processing on the second infrared image to obtain a third infrared image;
[0036] Step S110, performing data reduction on the third infrared image to obtain an infrared image after non-uniform noise correction is completed.
[0037] In the embodiment of the present invention, firstly, a judgment is made according to the number of frames of the infrared image, thereby completing four options for adaptive parameter updating: DeShading parameter updating processing, DeFPN parameter updating processing, DeStrip parameter updating processing and non-parameter updating processing.
[0038] It should be noted that if the judgment result is that any two or three parameter update processes in the adaptive parameter update process need to be executed, any two or three parameter update processes can be executed in parallel.
[0039] For example, for adaptive parameter update processing, every 12 frames of infrared images are a cycle, the DeShading parameters are updated in the second frame, the DeFPN parameters are updated in the third frame, the DeStrip parameters are updated in the fifth frame, and the parameters are not updated in the fifth, seventh, eighth, ninth, tenth, and twelfth frames.
[0040] The DeNU parameter update also relies on the frame count of the infrared image to determine whether the corresponding adaptive parameters need to be updated. If necessary, they can also be updated in parallel.
[0041] For example, the DeNU parameters are updated once every five infrared image frames are acquired, that is, the DeNU parameters are updated for infrared image frames numbered 1, 6, 11, 16, ...
[0042] The sigmaN parameter is updated and can also be updated in parallel. After all the above modules are processed, the non-uniform noise correction of the infrared image is completed.
[0043] In an embodiment of the present invention, if the target processing is a DeShading parameter update processing, target processing is performed on the infrared image based on the number of frames of the infrared image to obtain a first infrared image, including:
[0044] Filtering and cropping the infrared image to obtain a target infrared image;
[0045] Calculate the dot product between the target infrared image and the preprocessing matrix to obtain a first matrix, wherein the preprocessing matrix includes: a resShading matrix and a resShading4 matrix, and the value range of the resShading matrix and the resShading4 matrix is (-2 13 , 2 13 );
[0046] Summing the first matrix to obtain a summation result;
[0047] Based on the summation result and the preset constant, the adaptive parameter alphaSh and the adaptive parameter alphaSh4 are updated to obtain the first infrared image.
[0048] Specifically, the DeShading parameter update processing block is responsible for updating the adaptive parameters for removing pot cover noise. The shift amplification parameters involved in the DeShading module are set as follows: nucShift=4, varShift=10, matShift=20, ShShift=13, Sh4Shift=13.
[0049] There are two pre-processing matrices and three constants that need to be initialized in the DeShading parameter update process, namely the resShading matrix and the resShading4 matrix, as well as the preset constants sh_sh, sh_sh4, and sh4_sh4. The original floating point result of sh_sh is 0.7606804296, sh_sh4 is 0.5223224804, and sh4_sh4 is 0.4147914436. These three variables are uniformly scaled up. The maximum absolute value of matrix resShading is 0.0043669081; the maximum absolute value of matrix resShading4 is 0.0043982514; for the amplification of matrix values, the solution adopted is to enlarge them to retain the sixth decimal place, so the appropriate amplification factor is = 1048576. The maximum values are 4579 and 4612, both smaller than , that is, resShading and resShading4 matrices are enlarged The data range after multiple is .
[0050] Initialization of Shading and Shading4 matrices. Their original data are all decimals. At the same time, when calibrating the pot lid, the value range of alphaSh*Shading and alphaSh4*Shading4 is In order to ensure that the int32 data type does not overflow, the maximum times. This is because the alphaShading and alphaShading4 calculated in DeShading are both magnified. , so Shading and Shading4 can only enlarge The details of the variables processed by the DeShading parameter update are shown in Table 1:
[0051]
[0052] Table 1
[0053] Combine the following Figure 2The DeShading parameter update process is described in detail.
[0054] For example, the infrared image is recorded as ORG (640x512);
[0055] Filter ORG (filter kernel is {-1,1}) and crop it to obtain the target infrared image resORG (539x412);
[0056] The target infrared image resORG is dot-multiplied with the preprocessing matrix resShading and resShading4 (also called denoising template), and the result is recorded as the first matrix tmpSh (corresponding to m and n in the flowchart, where m and n are the dot-multiplied results of the target infrared image resORG and the preprocessing matrix resShading and resShading4). Because the range of resShading and reShading is , target infrared image resORG zoom The value range after multiplication is , so the value of the first matrix tmpSh of the dot product result is less than (Negative numbers are greater than , not listed separately, the same below), does not exceed the data range that can be represented by int32. Then the tmpSh matrix is summed, recorded as shSum, and the matrix size is , so the data range of shSum is , int64 can represent the range. At this time, the magnification of shSum is , in order to control the magnification of the subsequent division operation result , it is necessary to control the magnification of shSum to be consistent with the three preprocessing constants (sh_sh, sh_sh4, sh4_sh4), so shSum is reduced The final magnification is .
[0057] Calculate the key parameters beta and alpha, which will be amplified times, and then rely on them to complete the update of the adaptive parameters alphaShading and alphaShShading4.
[0058] Among them, beta=(sh_sh*m- sh_sh4*n)* scale / (sh_sh*sh_sh4- sh_sh4* sh_sh4);
[0059] alpha=(n-beta*sh_sh4)*scale / sh_sh;
[0060] alphaShading=(alphaShading*8+ alpha*2) / 10;
[0061] alphaShShading4=(alphaShading4*8+beta*2) / 10;
[0062] Among them, scale represents the intensity of shadow processing, which is a scaling factor that can be adjusted to control the effect of denoising, making the impact of shadows stronger or weaker.
[0063] Specifically, the details of each variable updated by the DeShading parameter are shown in Table 2.
[0064]
[0065] Table 2
[0066] The shading noise removal part is to subtract alphaSh*Shading and alphaSh4*Shading4 from the matrix data to be processed (denoted as NUC), so the subtrahend and the minuend magnification need to be kept consistent. Because NUC, alphaSh and alphaSh4 are all shifted left by nucShift, and Shading and Shading4 are shifted left by ShShift and ShShift4, the product of alphaSh*Shading and alphaSh4*Shading4 can be right-shifted by ShShift and ShShift4, and the final result is magnified. times, which is consistent with NUC.
[0067] In the embodiment of the present invention, if the target processing is a DeFPN parameter update process, target processing is performed on the infrared image based on the frame number of the infrared image to obtain a first infrared image, including:
[0068] Performing mean filtering and cropping without zero-padding edges on the infrared image to obtain a second matrix and a third matrix, wherein the second matrix and the third matrix have the same size;
[0069] Subtract the second matrix from the third matrix to obtain a fourth matrix;
[0070] The fourth matrix and the FP_DeTrnd matrix are point-multiplied and summed to obtain the target constant, where the data range of the FP_DeTrnd matrix is (−2 12 , 2 12 );
[0071] The adaptive parameter FPN is updated based on the target constant to obtain the first infrared image.
[0072] In the embodiment of the present invention, the DeFPN parameter update process is responsible for updating the adaptive parameters for removing fixed pattern noise. The shift amplification parameters involved in the DeFPN parameter update process are set as follows: nucShift=4, fpShift=11, fpSumShift=4.
[0073] The DeFPN parameter update process requires the initialization of one matrix and one constant in advance. Initialization depends on the auxiliary correction image FP (usually obtained by taking a photo with the infrared camera closed). Because the FP image is normalized and the mean is subtracted, the value range is (-1,1), and the range is less than 1. First, the FP is mean filtered with a filter kernel of size 201x201, and only the calculated filter data without zero-filled edges is returned to obtain FP_BLUR, which is 440x312 in size ( =131072<440 x 312 = 137280< ), then cut FP and subtract FP_BLUR to get FP_DeTrnd, so the value range is (-1,1), then FP_DeTrnd minus its own mean, the value range becomes (-2,2). Square FP_DrTrnd and perform matrix summation (the range after squaring becomes (0,4), the data range after summation is , and zoom in times and then rounded to get SumFPTrnd, which is less than . At the same time, enlarge the FP_DeTrnd matrix The fixed-point FP_DeTrnd matrix is obtained by rounding off after multiples. The reason for choosing to shift left by 11 bits is that the matrix dot multiplication operation of FP_DeTrnd is involved in the formal processing flow of DeFPN to prevent the result from exceeding the representation range of int32. The details of the preprocessing variables for the DeFPN parameter update process are shown in Table 3.
[0074]
[0075] Table 3
[0076] Combine the following Figure 3 The above-mentioned DeFPN parameter update process is described in detail.
[0077] When the infrared image is an ORG matrix, the ORG matrix is subjected to mean filtering without zero-padding edges (filter kernel size is 201x201) and cropping (the size after cropping is the same as the size after mean filtering, and the cropping is based on the center of the matrix) to obtain the second matrix Y_BLUR and the third matrix Y_DeTrnd(1), and then Y_DeTrnd(1) is subtracted from Y_BLUR to update the fourth matrix Y_DeTrnd(2). Because the data range of the input matrix ORG is enlarged times, so the value range of Y_BLUR is also , and the range in the ORG matrix is less than , so the value range of Y_BLUR is also , because the range is less than , so the value range of the matrix Y_DeTrnd(2) obtained after subtraction is also .
[0078] The fourth matrix Y_DeTrnd(2) is multiplied by the FP_DeTrnd matrix and the sum is calculated to obtain the target constant sum_Y_FP_DeTrnd. The data range of FP_DeTrnd is Therefore, the numerical range of the result of multiplying Y_DeTrnd by FP_DeTrnd is (within the range of int32 representation). In extreme cases, the value range after matrix summation obviously exceeds the range of int32 representation, so the sum_Y_FP_DeTrnd variable uses the int64 data type, and its maximum value is less than .
[0079] Determine whether the current adaptive parameter FPN update is the first update. If so, sum_Y_FP_DeTrnd / SumFPDeTrnd, and then reduce The magnification of alphaFP is ; If it is not the first update, the weighted addition update is performed based on the previous alphaFP value. The details of the DeFPN parameter update processing variables are shown in Table 4.
[0080]
[0081] Table 4
[0082] DeFPN parameter update process FPN noise removal is to subtract alphaFP*FP from the matrix data to be processed (recorded as NUC), so it is necessary to keep the subtrahend and the minuend magnification consistent. Because NUC and alphaFP are shifted left by nucShift positions, and FP is shifted left by fpShift positions, the product of alphaFP*FP can be shifted right by fpShift positions, and the final result is magnified. times, which is consistent with NUC.
[0083] In an embodiment of the present invention, if the target processing is a DeStrip parameter update processing, target processing is performed on the infrared image based on the number of frames of the infrared image to obtain a first infrared image, including:
[0084] Subtracting the first zero matrix from the infrared image to obtain a fifth matrix;
[0085] Performing Wiener filtering on the fifth matrix to obtain a sixth matrix, and subtracting the sixth matrix from the infrared image to obtain a seventh matrix;
[0086] Calculate the median of each row and each column in the seventh matrix, and assign the values in each row and each column to the corresponding median, to obtain an eighth matrix and a ninth matrix;
[0087] Calculate the sum of the eighth matrix and the ninth matrix to obtain a tenth matrix, and calculate the difference between the tenth matrix and the mean of the tenth matrix to obtain an eleventh matrix;
[0088] Based on the eleventh matrix, the adaptive parameter Strip is iteratively updated until the number of iterations reaches a first preset number, thereby obtaining the first infrared image.
[0089] In the embodiment of the present invention, the DeStrip parameter update process is responsible for updating the adaptive parameters for removing stripe noise. The DeStrip module does not need to amplify the data additionally, and the data after left shifting by nucShift bits is sufficient. In addition, the DeStrip module does not need to perform preprocessing.
[0090] Combine the following Figure 4 The process of DeStrip parameter update processing is described in detail.
[0091] When the infrared image is an ORG matrix, ORG minus the first zero matrix S2(1) obtains the fifth matrix tmp(1), the fifth matrix tmp(1) is subjected to Wiener filtering to obtain the sixth matrix tmp(2), and then ORG minus the sixth matrix tmp(2) obtains the seventh matrix tmp(3).
[0092] The median of each row and column of the seventh matrix tmp(3) is calculated respectively, and all the values of the row / column are assigned to the median of the corresponding row / column, thereby obtaining the eighth tmp(4) and the ninth matrix S2(2). The tenth matrix S2(3) is obtained by adding the two matrices. S2(3) is then subtracted from its own mean to obtain the eleventh matrix S2(4).
[0093] Determine whether 9 Wiener filters have been performed. If not, repeat the previous steps and iterate until 9 Wiener filters have been performed cumulatively. After the iteration is completed, the adaptive parameter Strip can be updated. First, S2(1) is thresholded in the range of -std(S2(1))~std(S2(1)) to obtain S2(4). Then determine whether it is the first time to update the adaptive parameter Strip. If so, assign S2(4) to Strip. If not, update the adaptive parameter Strip based on the weighted addition of the current adaptive parameter Strip and S2(4).
[0094] The specific process of Wiener filtering is described below:
[0095] After Gaussian filtering the fifth matrix tmp(1) of the Wiener filter input, the local_mean matrix is obtained, and then the residual matrix is obtained by subtracting local_mean from the fifth matrix tmp(1). According to the algorithm principle, the value range of the fifth matrix tmp(1) is , but the difference between the maximum and minimum values in tmp does not exceed , so the value range of residual is still .
[0096] The absolute value of the residual matrix is taken and assigned to tmp(2). Gaussian filtering is performed on tmp(2) to obtain tmp(3), which is then squared to obtain tmp(4). From the previous step, we know that the value range of tmp(2) is , so the value range of tmp(4) is , then it is obvious that 2*dr+2*nucShift=2*14+2*4=36>31, which exceeds the representation range of int32, so before opening the operation, reduce the value range of tmp(3) to , so as to ensure that the range of tmp(4) is , so as not to exceed the representation range of int32, tmp(3) is shifted right by (dr+nucShift-15) bits before the square operation is performed.
[0097] Calculate the median of tmp(4), and determine the statistical range of the histogram based on the median. Then, obtain the value sigmaN2 from the statistical result of the histogram. Subtract the constant sigmaN2 from the tmp(4) matrix to obtain the matrix sigmaU2(1). Perform a thresholding operation on sigmaU2 and perform the max(sigmaU2(1),0) operation to obtain sigmaU2(2). Then, multiply sigmaU2(2) by residual to obtain sigmaU2(3). Then, divide sigmaU2(3) by tmp(4) to obtain tmp(5). Finally, add local_mean to tmp(5) to obtain the result of the Wiener filter and assign it to tmp. It should be noted that the numerical range of sigmaU2(2) is the same as that of tmp(4). , and the value range of residual is , we can combine the dot multiplication and dot division operations into one step in the loop, so no additional memory is required to store sigmU2(3). In addition, it is very important to add 1 to all positions in the tmp(4) matrix to avoid 0 in the matrix because it involves the dot division operation. The details of some variables in the Wiener filter are shown in Table 5.
[0098]
[0099] Table 5
[0100] In the embodiment of the present invention, performing DeNU parameter updating processing on the first infrared image to obtain a second infrared image includes:
[0101] Subtracting the adaptive parameter NU from the processed infrared image to obtain a first calculation result;
[0102] Performing Gaussian filtering on the first calculation result to obtain a first filtering result, and subtracting the first filtering result from the processed infrared image to obtain a second calculation result;
[0103] Performing Gaussian filtering on the absolute value of the second calculation result to obtain a second filtering result;
[0104] Performing equal-interval sampling on the second filtering result to obtain a first array, and determining a static variable corresponding to the sampling array;
[0105] Performing weighted summation on the first infrared image and the second zero matrix to obtain a twelfth matrix;
[0106] calculating an absolute value of the processed infrared image minus the twelfth matrix, and performing a weighted summation of the absolute value and a third zero matrix to obtain a thirteenth matrix;
[0107] Performing equal-interval sampling on the thirteenth matrix to obtain a second matrix;
[0108] The adaptive parameter NU is iteratively updated based on the first array and the second array until the number of iterations reaches a second preset number, thereby obtaining the second infrared image.
[0109] In the embodiment of the present invention, the DeNU parameter update process is responsible for updating the adaptive parameters for removing ghost noise. The shift amplification parameter involved in the DeNU parameter update process is set as follows: nuShift=4. Because the amplification factors of all key variables are the same, they are not marked separately.
[0110] The preprocessing of DeNU parameter update is relatively simple. Just create two zero matrices: thetaUT and sigmaUT. In fact, the data is multiplied by a certain amount, but because the initial value is 0, no processing is required. In addition, there is a value that needs to be calculated in advance, namely mu, whose value depends on the dynamic range of the image data. The details of the DeNU preprocessing variables are shown in Table 6.
[0111]
[0112] Table 6
[0113] All key data magnifications in the DeNU parameter update process are Assume that the input image of the DeNU parameter update processing module is an ORG matrix.
[0114] Combine the following Figure 5 The DeNU parameter update process is described in detail.
[0115] The first infrared image ORG is subtracted from the adaptive parameter NU to obtain the first calculation result local_m(1), and then the first filtering result local_mean(2) is obtained through Gaussian filtering. The first infrared image ORG is subtracted from the first filtering result local_m(2) to obtain the second calculation result sigmaU_s(1), and the absolute value is taken and then high filtering is performed to obtain the second filtering result sigmaU_s(2). Then, the sigmaU_s(2) matrix is sampled at equal intervals with 501 data to form an array first array S, and S is sorted.
[0116] The first infrared image ORG and the old second zero matrix theatUT(1) are weightedly summed to obtain the twelfth matrix thetaUT(2). Then, the absolute value of the result of subtracting the twelfth matrix thetaUT(2) from the first infrared image ORG is taken and weightedly summed with the third zero matrix sigmaUT(1) to obtain the thirteenth matrix sigmaUT(2). Then, 501 data of the thirteenth matrix sigmaUT(2) are sampled at equal intervals to form a second array T, and T is sorted.
[0117] Determine whether flag is greater than 10 (flag is a static variable, initially 0). If not, add 1. If yes, proceed to the next step. Calculate the rank rnk based on arrays S and T, and determine whether the value of rnk is less than the threshold we set. If yes, update NU. The specific formula for updating NU is as follows: , so the magnification of NU is consistent with mu, and the original data for calculating mu is magnified times, mu is magnified again in the DeNUDeNU parameter update process times, the final magnification of NU is .
[0118] In the embodiment of the present invention, performing sigmaN parameter updating processing on the second infrared image to obtain a third infrared image includes:
[0119] Performing Gaussian filtering on the second infrared image to obtain a third filtering result, and subtracting the third filtering result from the second infrared image to obtain a third calculation result;
[0120] Performing Gaussian filtering on the absolute value of the third calculation result to obtain a fourth filtering result;
[0121] Subtracting the adaptive parameter sigmaN from the square value of the fourth filtering result to obtain a fourth calculation result, and determining a larger value between the fourth calculation result and 0;
[0122] Calculate the dot product between the larger value and the third filtering result to obtain a fifth calculation result;
[0123] Calculate the dot division between the fifth calculation result and the square value of the fourth filtering result to obtain a fifth calculation result;
[0124] The sum of the fifth calculation result and the third filtering result is assigned to the second infrared image to obtain the third infrared image.
[0125] In the embodiment of the present invention, the sigmaN parameter update process is responsible for updating the adaptive parameters for removing random noise, and the sigmaN parameter update process does not require additional data amplification, and only requires data that is left-shifted by nucShift bits. In addition, the DeNoise module does not require preprocessing.
[0126] Combine the following Figure 5 The sigmaN parameter update process is described in detail.
[0127] When the second infrared image is an ORG matrix, the second infrared image ORG is Gaussian filtered to obtain a third filtering result local_m, the second infrared image ORG is subtracted from the third filtering result local_m to obtain a third calculation result res, and then the absolute value is taken to obtain sigmaV(1), and then Gaussian filtered to obtain a fourth filtering result sigmaV(2).
[0128] Then the fourth filtering result sigmaV(2) is squared. It should be noted that the value range of the fourth filtering result sigmaV(2) is , the value after square may exceed the representation range of int32, so the result is temporarily stored as int64, and then reduced to the range of int32, that is, right shifted (dr+nucShift)*2-31 bits, and the final result is stored in the fourth calculation result sigmaV(3).
[0129] Subtract the adaptive parameter sigmaN from the fourth calculation result sigmaV(3) to obtain the fourth calculation result sigmaU(1), and then compare it with 0, take the larger value, and obtain the larger value sigmaU(2). Then multiply the larger value simgaU(2) by the third filter result local_m, and then divide the fourth calculation result sigmaV(3) to obtain the fifth calculation result. Note that the result of the dot multiplication operation may exceed the representation range of int32, so the intermediate result of the dot multiplication is stored in int64, and the result of the dot division can be stored as int32. Finally, the fifth calculation result plus the third filter result local_m is assigned to the second infrared image ORG to obtain the third infrared image.
[0130] It should be noted that the sigmaN parameter needs to be updated, and its update frequency is the same as the frequency of the adaptive parameter update process. First, the median of the square value of the fourth filter result is counted, and the histogram statistical range is determined by the median, and then histVal (the horizontal coordinate of the highest column in the histogram statistical result) is obtained from the histogram statistical result. Finally, the weighted sum of histVal and the old adaptive parameter sigmaN is used to update the adaptive parameter sigmaN.
[0131] The histogram calculation process is described below.
[0132] In histogram statistics, it is necessary to divide the data into 100 segments for statistics. If it is a floating point number, decimals can exist in the 100 segments and the data can be divided normally. However, the problem faced by fixed-point operations is that in extreme cases, the data range is less than 100, so it cannot be divided into 100 segments for statistics. For the above extreme cases, the strategy adopted is to determine whether the range of the data (that is, the statistical range) is less than 100. If it is less than 100, it is divided into data segments with the same range. If it is greater than 100, the range is divided by 100 and rounded up as the segment length.
[0133] Gaussian filtering is an important step in the non-uniform noise correction method of infrared images. The following will explain the two filtering processes in detail.
[0134] In the embodiment of the present invention, most Gaussian filters can be calculated based on int32, but sometimes, in order to ensure the calculation accuracy, a small number of Gaussian filter calculations need to be implemented based on int64. Because the Gaussian filter kernel usually appears in the form of a decimal, the key to the fixed-point implementation of Gaussian filtering is to amplify the filter kernel value and finally reduce the filter result.
[0135] Int32-based Gaussian filtering, the number of left shifts is set to g32Shift=12. The value range of int32 is Because our data is initialized and simply processed in the range of ,enlarge After times , so now you can zoom in up to Because the sum of the Gaussian filter kernels is 1, the filter result is also magnified by the same multiple as the Gaussian filter kernel. Therefore, the maximum magnification of the Gaussian filter kernel is Therefore, there is no data overflow in this scheme.
[0136] Gaussian filter based on int64. The number of left shifts is set to g64Shift=25. The value range of int32 is , the calculation steps based on the int64 type appear in DeNU. From the introduction of DeNU parameter update processing, we know that the Gaussian filter object is enlarged times, so the result of Gaussian filtering before reducing the data range should be ,
[0137] And dr+nucShift+nuShift+g64Shift=14+4+4+25=47<63, so the data range of int64 is sufficient for storage, but int32 is not up to the task. The reason why these calculations use int64 type Gaussian filtering is that experiments have shown that using int32 type Gaussian filtering in this part will cause large errors in fixed-point number and floating-point number algorithms. Therefore, in order to ensure the accuracy of the algorithm, a small number of operations are implemented based on int64 data type.
[0138] Embodiment 2:
[0139] An embodiment of the present invention also provides a non-uniform noise correction device for infrared images, which is used to execute the non-uniform noise correction method for infrared images provided in the above content of the embodiment of the present invention. The following is a specific introduction to the non-uniform noise correction device for infrared images provided in the embodiment of the present invention.
[0140] like Figure 2 As shown, Figure 2 Schematic diagram of the above-mentioned infrared image non-uniform noise correction device, the infrared image non-uniform noise correction device comprises:
[0141] The acquisition unit 10 is used to acquire an infrared image and preprocess the infrared image to obtain a preprocessed infrared image, wherein the preprocessing includes: format conversion and data amplification;
[0142] A first processing unit 20 is used to perform target processing on the infrared image based on the frame number of the infrared image to obtain a first infrared image, wherein the target processing includes: adaptive parameter update processing or parameter non-update processing, and the adaptive parameter update processing includes one of the following: DeShading parameter update processing, DeFPN parameter update processing and DeStrip parameter update processing;
[0143] A second processing unit 30, configured to perform a DeNU parameter update process on the first infrared image based on the frame number of the infrared image to obtain a second infrared image;
[0144] A third processing unit 40 is used to perform sigmaN parameter update processing on the second infrared image to obtain a third infrared image;
[0145] The fourth processing unit 40 is used to perform data reduction on the third infrared image to obtain an infrared image after non-uniform noise correction is completed.
[0146] In the embodiment of the present invention, the inter-frame changes of infrared images are comprehensively considered, and the accurate matching of noise characteristics under different scenes and lighting conditions is achieved through adaptive parameter update processing (such as DeShading, DeFPN, and DeStrip parameter updates). Furthermore, combined with the DeNU parameter update and sigmaN parameter adjustment, it is not only possible to effectively remove the fixed pattern noise and dynamically changing stripe noise in the image, but also to optimize the overall brightness and contrast of the image while maintaining the image details. Finally, through the data reduction step, the high fidelity and low-latency output of the processed image are guaranteed, which is suitable for various high-performance infrared imaging applications, and can adapt to a wider range of image conditions, thereby improving the stability, accuracy, and efficiency of the correction effect.
[0147] Embodiment three:
[0148] The embodiment of the present invention further provides a computing device for executing a program of the method described in the first embodiment, such as Figure 3 As shown, the computing device may include a storage component 41 and a processing component 42;
[0149] The storage component 41 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 42 .
[0150] The processing component 42 may include one or more processors to execute computer instructions to complete all or part of the steps in the method described in Example 1. Of course, the processing component may also be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.
[0151] The storage component 41 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0152] The display component 43 may be an electroluminescent (EL) element, a liquid crystal display or a micro display having a similar structure, or a retinal direct display or a similar laser scanning display.
[0153] Of course, a computing device may also include other components, such as input / output interfaces, communication components, etc.
[0154] The input / output interface provides an interface between the processing component and the peripheral interface module, which may be an output device, an input device, etc.
[0155] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.
[0156] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.
[0157] Embodiment 4:
[0158] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The method of the illustrated embodiment.
[0159] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0160] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0161] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for correcting non-uniform noise of infrared images, characterized in that: include: Acquire an infrared image, and preprocess the infrared image to obtain a preprocessed infrared image, wherein the preprocessing includes: format conversion and data amplification; Based on the frame number of the infrared image, target processing is performed on the infrared image to obtain a first infrared image, wherein the target processing includes: adaptive parameter update processing or parameter non-update processing, and the adaptive parameter update processing includes at least one of the following: DeShading parameter update processing, DeFPN parameter update processing and DeStrip parameter update processing; Based on the frame number of the infrared image, performing DeNU parameter update processing on the first infrared image to obtain a second infrared image; performing a sigmaN parameter update process on the second infrared image to obtain a third infrared image; Performing data reduction on the third infrared image to obtain an infrared image that has completed non-uniform noise correction; Wherein, if the target processing is a DeShading parameter update processing, then based on the frame number of the infrared image, target processing is performed on the infrared image to obtain a first infrared image, including: Filtering and cropping the infrared image to obtain a target infrared image; Calculate the dot product between the target infrared image and the preprocessing matrix to obtain a first matrix, wherein the preprocessing matrix includes: a resShading matrix and a resShading4 matrix, and the value range of the resShading matrix and the resShading4 matrix is (-2 13 , 2 13 ); Summing the first matrix to obtain a summation result; Based on the summation result and the preset constant, the adaptive parameter alphaShShading and the adaptive parameter alphaShShading4 are updated to obtain the first infrared image; Wherein, if the target processing is a DeFPN parameter update processing, target processing is performed on the infrared image based on the frame number of the infrared image to obtain a first infrared image, including: Performing mean filtering and cropping without zero-padding edges on the infrared image to obtain a second matrix and a third matrix, wherein the second matrix and the third matrix have the same size; Subtract the second matrix from the third matrix to obtain a fourth matrix; The fourth matrix and the FP_DeTrnd matrix are point-multiplied and summed to obtain the target constant, where the data range of the FP_DeTrnd matrix is (−2 12 , 2 12 ); The adaptive parameter FPN is updated based on the target constant to obtain the first infrared image; If the target processing is a DeStrip parameter update processing, then based on the number of frames of the infrared image, target processing is performed on the infrared image to obtain a first infrared image, including: Subtracting the first zero matrix from the infrared image to obtain a fifth matrix; Performing Wiener filtering on the fifth matrix to obtain a sixth matrix, and subtracting the sixth matrix from the infrared image to obtain a seventh matrix; Calculate the median of each row and each column in the seventh matrix, and assign the values in each row and each column to the corresponding median, to obtain an eighth matrix and a ninth matrix; Calculate the sum of the eighth matrix and the ninth matrix to obtain a tenth matrix, and calculate the difference between the tenth matrix and the mean of the tenth matrix to obtain an eleventh matrix; Based on the eleventh matrix, iteratively update the adaptive parameter Strip until the number of iterations reaches a first preset number, thereby obtaining the first infrared image; The step of performing DeNU parameter updating processing on the first infrared image to obtain a second infrared image includes: Subtracting the adaptive parameter NU from the processed infrared image to obtain a first calculation result; Performing Gaussian filtering on the first calculation result to obtain a first filtering result, and subtracting the first filtering result from the processed infrared image to obtain a second calculation result; Performing Gaussian filtering on the absolute value of the second calculation result to obtain a second filtering result; Performing equal-interval sampling on the second filtering result to obtain a first array, and determining a static variable corresponding to the sampling array; Performing weighted summation on the first infrared image and the second zero matrix to obtain a twelfth matrix; Calculating an absolute value of the processed infrared image minus the twelfth matrix, and performing a weighted summation of the absolute value and a third zero matrix to obtain a thirteenth matrix; Performing equal-interval sampling on the thirteenth matrix to obtain a second matrix; Iteratively updating the adaptive parameter NU based on the first array and the second array until the number of iterations reaches a second preset number, thereby obtaining the second infrared image; The step of performing sigmaN parameter updating processing on the second infrared image to obtain a third infrared image includes: Performing Gaussian filtering on the second infrared image to obtain a third filtering result, and subtracting the third filtering result from the second infrared image to obtain a third calculation result; Performing Gaussian filtering on the absolute value of the third calculation result to obtain a fourth filtering result; Subtract sigmaN from the square value of the fourth filtering result to obtain a fourth calculation result, and determine a larger value between the fourth calculation result and 0; Calculate the dot product between the larger value and the third filtering result to obtain a fifth calculation result; Calculate the dot division between the fifth calculation result and the square value of the fourth filtering result to obtain a fifth calculation result; The sum of the fifth calculation result and the third filtering result is assigned to the second infrared image to obtain the third infrared image.
2. The method according to claim 1, characterized in that The method further comprises: Counting the median of the square values of the fourth filtering result; Determining a histogram statistical range based on a median of the square values of the fourth filtering result; Determine the horizontal coordinate of the highest column within the statistical range of the histogram; Based on the horizontal coordinate, the adaptive parameter sigmaN is updated.
3. A non-uniform noise correction device for infrared images, characterized in that: A method for correcting non-uniform noise of infrared images according to any one of claims 1 to 2, comprising: An acquisition unit, used for acquiring an infrared image and preprocessing the infrared image to obtain a preprocessed infrared image, wherein the preprocessing includes: format conversion and data amplification; A first processing unit is used to perform target processing on the infrared image based on the frame number of the infrared image to obtain a first infrared image, wherein the target processing includes: adaptive parameter update processing or parameter non-update processing, and the adaptive parameter update processing includes one of the following: DeShading parameter update processing, DeFPN parameter update processing and DeStrip parameter update processing; a second processing unit, configured to perform a DeNU parameter update process on the first infrared image based on the frame number of the infrared image to obtain a second infrared image; A third processing unit, configured to perform sigmaN parameter updating processing on the second infrared image to obtain a third infrared image; The fourth processing unit is used to perform data reduction on the third infrared image to obtain an infrared image that has completed non-uniform noise correction.
4. A computing device, characterized in that It comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement the method as described in any one of claims 1 to 2.
5. A computer storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a computer, the method according to any one of claims 1 to 2 is implemented.
Citation Information
Patent Citations
Method for correcting infrared focal plane heterogeneity based on sigma filter
CN102968765A
Linear filter and non-linear filter combined heterogeneity correction method
CN102968776A
