Preprocessing method for infrared focal plane image sequence

Abstract

The invention relates to a preprocessing method for an infrared focal plane image sequence, which comprises the following steps of: averaging each pixel point of an image in a time domain to obtain a time domain mean value; then local nonlinear filtering is used, the maximum value and the minimum value in a window are removed in the sliding window of the image, dynamic threshold detection is carried out on the time domain mean value of each point, the local mean value of the window and the maximum value and the minimum value in the window, and pixels exceeding the threshold are marked as blind pixels; after median filtering compensation is carried out on blind pixels, a time-space domain difference operator non-uniformity correction model is established for an infrared image sequence, and the infrared image sequence is solved by using an alternating direction operator method, so that the infrared image sequence is pre-processed in a self-adaptive manner. When the method is used for preprocessing, the accuracy of blind pixel compensation is high, the non-uniformity noise correction effect is good, and image artifacts are effectively reduced.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a preprocessing method for infrared focal plane image sequences. Background Art

[0002] Since the number of detection units distributed on the surface of the infrared focal plane varies, usually there are tens of thousands. Due to the limitations of processing technology, manufacturing materials and R&D level, different pixels on the infrared focal plane have inconsistent responses to the same uniform radiation, resulting in blind pixel noise and non-uniform noise in the infrared focal plane array.

[0003] When the satellite is in orbit, it is affected by factors such as photon effect, temperature change, and time drift, and the response characteristics of the infrared focal plane array will drift. This will cause deviations when using the calibration data obtained from the ground to correct the non-uniformity of the infrared focal plane array of the infrared imaging system in the satellite environment, resulting in more serious blind pixels and non-uniform noise of the infrared focal plane array. The readout circuit of the traditional detector only reads one line, so there is only horizontal or vertical stripe noise. However, due to the inconsistent response between pixels between the array infrared focal plane detectors, changes in the readout circuit, and temperature fluctuations in the optical lens, the non-uniform noise of the array infrared focal plane detector will have block non-uniform noise and tilted stripe noise superimposed, which seriously affects the quality of the infrared image. Most traditional methods correct the horizontal or vertical stripe non-uniform noise of the infrared image, and are not completely applicable to the tilted stripe noise and fixed pattern noise caused by optics. It brings greater difficulties to the subsequent detection, tracking, and recognition of targets. Summary of the invention

[0004] The purpose of the present invention is to provide a preprocessing method for infrared focal plane image sequences, which solves the problem that most of the existing technologies correct the horizontal or vertical stripe non-uniform noise of infrared images, but are not fully applicable to inclined stripe noise and fixed pattern noise caused by optics.

[0005] To achieve the above object, the technical solution of the present invention is:

[0006] A preprocessing method for an infrared focal plane image sequence, characterized in that: blind pixel correction and non-uniformity correction are performed on the infrared focal plane image sequence, wherein the blind pixel correction includes blind pixel compensation for the image; and the method comprises the following steps:

[0007] Blind pixel correction and non-uniformity correction are performed on the infrared focal plane image sequence, and the blind pixel correction includes blind pixel compensation for the image; the steps include:

[0008] Step S100, inputting an image;

[0009] Step S200, establishing a time domain value model; taking an average of each pixel point of the image in the time domain to obtain a time domain mean;

[0010] Step S300, local nonlinear filtering; in the sliding window of the image, the maximum value and the minimum value in the window are removed, and the local average value of the window is subjected to dynamic threshold detection with the maximum value and the minimum value in the original window;

[0011] Step S400, determining blind pixels by threshold comparison; performing dynamic threshold detection on the time domain mean of each point, the local mean of the window, the maximum value and the minimum value in the window, and recording pixels exceeding the threshold as blind pixels;

[0012] If it is a blind pixel, a blind pixel table is established, blind pixel compensation is performed, and the image is output to proceed to the next step; if it is not a blind pixel, the image is directly output to proceed to the next step;

[0013] Step S500, establish a time-space domain difference operator non-uniformity correction model; after median filtering compensation for blind pixels, establish a time-space domain difference operator non-uniformity correction model for the infrared image sequence, using L 2，1 The norm constrains the group sparsity of the denoised image, and two one-way total variation regularization methods are used to enhance the smoothness of the noisy image in the horizontal and temporal directions;

[0014] Step S600, decomposing the alternating operator method into sub-problems; solving the sub-problems using the alternating direction operator method, thereby adaptively preprocessing the infrared image sequence;

[0015] Step S700, iteratively optimizing sub-problems;

[0016] Step S800, threshold judgment, judge whether convergence; if the conclusion is no, after updating the intermediate variables, return to step S700, iterate the optimization sub-problem again, and enter this step; if the conclusion is yes, directly output the real image.

[0017] Step S200 is for the infrared image sequence f(i,j,t), where t is the frame number, (i,j) represent the horizontal and vertical coordinates of the focal plane pixel, i = 1,2,…,m, j = 1,2,…,n, t = 1,2,…,T; m and n represent the number of rows and columns of each image, respectively, and T represents the total number of frames in the infrared image sequence; first, the time domain mean is taken for each pixel point An infrared image sequence is calculated as a time-domain mean image.

[0018] Step S300, perform local nonlinear filtering on the time domain mean image, specifically, B(i,j) represents the maximum and minimum values ​​of the time domain mean image in the square neighborhood centered at (i,j) minus the average grayscale value in the spatial domain, N is the side length of the square neighborhood, the unit is pixel, N is 3 pixels, B max (i±p,j±q) is the maximum value in a square neighborhood with sides p and q, B min (i±p,j±q) is the minimum value in a square neighborhood with sides of p and q;

[0019]

[0020] Step S400, determining blind pixels by threshold comparison, the specific steps are:

[0021] Step S401, using the mean value B(i,j) calculated in step S300 to calculate the fixed blind pixel of the image, the threshold is related to the temporal mean of the current point, the local mean value of the window, the maximum temporal mean value of the window, and the minimum temporal mean value of the window. Wherein, B(i,j) is the gray value of the pixel (i,j), and the thresholds delta1 and delta2 are respectively the ratio of the difference between the local mean value of the window and the maximum and minimum values ​​of the window relative to the local mean value;

[0022]

[0023] Step S402, compare the thresholds delta1 and delta2 with the ratio a, if they are greater than a, then the point (i, j) is determined to be a blind pixel, where a is an adjustable parameter and the range of a is between 0.1-0.5;

[0024]

[0025] Step S500 is to model the non-uniform noise of the infrared image sequence to form a time-space domain differential operator non-uniformity correction model, and regard a single infrared image as a two-dimensional variation function; the gain coefficient is regarded as 1, and the mathematical expression of the non-uniform noise model is as follows f=u+b, where f is the observed image, u is the desired noise-free image, and b is the non-uniform bias noise image. Each image is stacked into a long column vector of size mnT×1, f=[vec(f 1 );vec(f 2 );…,vec(f T )]∈R mnT ×1 , where vec(f t )(t=1,2,…,T) represents the observed image of the tth frame; four constraints are proposed for the non-uniformity correction model based on the spatiotemporal domain, as follows:

[0026] Condition 1: Group sparsity of the expected noise-free image: For video images, the expected noise-free image is continuous in the spatial dimension and also continuous in the temporal dimension, so it has the prior characteristics of spatiotemporal continuity. The expected noise-free image contains dynamic images. Although it is not low-rank, it uses L because of its weak correlation in spatiotemporal continuity. 2，1 Regularization constrains the group sparsity of the desired noise-free image and promotes the separation of noisy images and the desired noise-free images.

[0027] Condition 2: Smoothness of the noise image along the horizontal direction: Under normal circumstances, assuming that the direction of the readout circuit stripe noise is horizontal, if vertical stripes appear, the image is rotated. The derivatives of the stripe noise and the expected noise-free image along the horizontal direction are different, that is, the derivative of the stripe noise along the horizontal direction is sparser than the derivative of the expected noise-free image. Therefore, we use L 1 D x (uf) norm to enhance the smoothness of noise along the horizontal direction.

[0028] Condition 3, smoothness of the expected noise-free image along the vertical direction: The expected noise-free image is piecewise smooth, which indicates that the derivatives of each frame in the infrared video image in the vertical and horizontal directions are not dense. The horizontal stripes destroy the smoothness along the vertical direction. Compared with the derivatives of the noisy image, the derivatives of the expected noise-free image are sparse along the vertical direction. Therefore, the derivatives of the stripe noise along the horizontal direction are dense. Therefore, we use L 1 D y The u-norm is used to enhance the smoothness of the desired noise-free image along the vertical direction.

[0029] Condition 4: Smoothness of non-uniform noise image along the time direction: Since non-uniform noise is additive noise, the derivative of the noise image is sparse along the time direction. The derivative of the noise image along the time direction is sparse, while the derivative of the expected noise-free image is not sparse. Therefore, using L 1 D t (uf) norm is used to enhance the smoothness of the noisy image along the time direction.

[0030] in,

[0031] D x , D y , D t are the corresponding linear operators of the horizontal, vertical and time domain first-order difference operators, so the non-uniformity correction model is established as follows:

[0032]

[0033] In step S600, let dx =D x (uf),d y =D y u,d t =D t (uf),d 4 =ufThe above formula becomes

[0034]

[0035] std x =D x (uf),d y =D y u,d t =D t (uf),d 4 =u.

[0036] Step S700, using Bregman iteration to further transform the above non-uniformity correction model into an unconstrained minimization problem;

[0037]

[0038] In the above formula, α 1 , α 2 , α 3 , α 4 is the Bregman penalty coefficient, variable b x 、b y 、b t 、b 4 is an intermediate variable determined by Bregman iteration.

[0039] Step S800 is to use

[0040]

[0041] An ADMM (Alternating Direction Multiplier Method) was developed to optimize the solution. One variable was optimized iteratively while other variables were fixed. The non-uniformity correction model can be divided into five minimization sub-problems, as follows:

[0042] Step S801, the sub-questions related to u are as follows:

[0043]

[0044] Since it is a convex function, it is equal to the following linear system:

[0045] (α 1 D x T D x +α 2D y T D y +α 3 D t T D t +α 4 ) k+1 =α 1 D x T (d x k +D x fb x k )+α 2 D y T (d y k -b y k )+α 3 D t T (d t k +D t fb t k )+α 4 *d 4 k -b 4 k );

[0046] The superscript k is the number of iterations, the superscript T is the matrix transposition factor, and the closed-form solution of the fast Fourier transform (FFT) is used to solve the above equation;

[0047]

[0048] where Ι is the identity matrix, is the fast Fourier transform, is the inverse fast Fourier transform;

[0049] Step S802, with d x The relevant sub-questions are as follows:

[0050]

[0051] It can be calculated by the soft threshold operator,

[0052]

[0053] where Shrink(·) is a soft threshold operation,

[0054]

[0055] Similarly, with d y d t Related sub-questions are related to d x Same, so

[0056]

[0057] Among them, b x k+1 、b y k+1 、b t k+1 Update with the following formula

[0058]

[0059] Step S803, with d 4 The relevant sub-questions are as follows:

[0060]

[0061] in, From step S802, it can be seen that

[0062] Iterate u k+1 , b x k+1 、b y k+1 、b t k+1 、b 4 k+1 Calculate the value of ε, set the parameter ε, when When , the iteration stops, norm(·) is the normalization function; output u at this time k+1 , as the desired noise-free image.

[0063] The advantages of the present invention are: 1. Solving the problem of superposition of tilted stripe non-uniformity and optically induced fixed pattern noise; 2. Reducing the afterimage problem left by other algorithms in low-contrast images; 3. Forming a time-space domain differential operator non-uniformity correction model, using the alternating operator method to decompose the model into four sub-problems, and performing iterative optimization and solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a preprocessing flow chart of the present invention.

[0065] Figure 2 is a simulated image to be partially denoised according to the present invention, wherein a is the 1st frame, b is the 30th frame, c is the 60th frame, d is the 90th frame, e is the 120th frame, and f is the 150th frame;

[0066] Figure 3 The grayscale image of the present invention is partially preprocessed by the method of the present invention, wherein a is the 1st frame, b is the 30th frame, c is the 60th frame, d is the 90th frame, e is the 120th frame, and f is the 150th frame;

[0067] Figure 4 A peak signal-to-noise ratio statistical diagram of an image preprocessed and simulated using the present invention;

[0068] Figure 5 A statistical diagram of structural similarity of images preprocessed and simulated using the present invention; DETAILED DESCRIPTION

[0069] The present invention is further described below in conjunction with the accompanying drawings, which are only used for exemplary description and cannot be understood as limiting the present invention.

[0070] In order to more concisely describe the present embodiment, some parts known to those skilled in the art but not related to the main content of the present invention are omitted in the drawings or descriptions. In addition, for the convenience of description, some parts in the drawings are omitted, enlarged or reduced, but they do not represent the size or entire structure of the actual product.

[0071] The present invention discloses a preprocessing method for infrared focal plane image sequences, such as Figure 1 As shown, blind pixel correction and non-uniformity correction are performed on the infrared focal plane image sequence, and the blind pixel correction includes blind pixel compensation for the image; the steps include:

[0072] Step S100, inputting an image;

[0073] Step S200, establishing a time domain value model; taking an average of each pixel point of the image in the time domain to obtain a time domain mean;

[0074] For the infrared image sequence f(i,j,t), where t is the frame number, (i,j) represents the horizontal and vertical coordinates of the focal plane pixel, i = 1,2,…,m, j = 1,2,…,n, t = 1,2,…,T; m and n represent the number of rows and columns of each image, respectively, and T represents the total number of frames in the infrared image sequence; first, take the time domain mean for each pixel An infrared image sequence is calculated as a time-domain mean image.

[0075] Step S300, local nonlinear filtering; in the sliding window of the image, the maximum value and the minimum value in the window are removed, and the local average value of the window is subjected to dynamic threshold detection with the maximum value and the minimum value in the original window;

[0076] B(i,j) represents the maximum and minimum values ​​of the cumulative time domain mean image minus the spatial domain average grayscale value in the square neighborhood centered at (i,j). N is the side length of the square neighborhood in pixels. N is 3 pixels. max (i±p,j±q) is the maximum value in a square neighborhood with sides p and q, B min (i±p,j±q) is the minimum value in a square neighborhood with sides of p and q;

[0077]

[0078] Step S400, determining blind pixels by threshold comparison; performing dynamic threshold detection on the time domain mean of each point, the local mean of the window, the maximum value and the minimum value in the window, and recording pixels exceeding the threshold as blind pixels;

[0079] If it is a blind pixel, a blind pixel table is established, blind pixel compensation is performed, and the image is output to proceed to the next step; if it is not a blind pixel, the image is directly output to proceed to the next step; the specific steps are:

[0080] Step S401, using the mean value B(i,j) calculated in step S300 to calculate the fixed blind pixel of the image, the threshold is related to the temporal mean of the current point, the local mean value of the window, the maximum temporal mean value of the window, and the minimum temporal mean value of the window. Wherein, B(i,j) is the gray value of the pixel (i,j), and the thresholds delta1 and delta2 are respectively the ratio of the difference between the local mean value of the window and the maximum and minimum values ​​of the window relative to the local mean value;

[0081]

[0082] Step S402, compare the thresholds delta1 and delta2 with the ratio a, if they are greater than a, then the point (i, j) is determined to be a blind pixel, where a is an adjustable parameter and the range of a is between 0.1-0.5;

[0083]

[0084] Step S500, establish a time-space domain difference operator non-uniformity correction model; after median filtering compensation for blind pixels, establish a time-space domain difference operator non-uniformity correction model for the infrared image sequence, using L 2，1 The norm constrains the group sparsity of the denoised image, and two one-way total variation regularization methods are used to enhance the smoothness of the noisy image in the horizontal and temporal directions;

[0085] The non-uniform noise of the infrared image sequence is modeled to form a time-space domain differential operator non-uniformity correction model, and a single infrared image is regarded as a two-dimensional variation function; generally speaking, the gain coefficient does not affect the estimated value of the real infrared image, so it is usually regarded as 1, and the bias coefficient and stripe noise of the infrared image are often additive.

[0086] Therefore, the mathematical expression of the non-uniform noise model is as follows: f = u + b, where f is the observed image, u is the expected noise-free image, and b is the non-uniform bias noise image. Each image is stacked into a long column vector of size mnT × 1, for example, f = [vec(f 1 );vec(f 2 );…,vec(f T )]∈R mnT×1 , where vec(f t )(t=1,2,…,T) represents the observed image of the tth frame; four constraints are proposed for the non-uniformity correction model based on the spatiotemporal domain, as follows:

[0087] Condition 1: Group sparsity of the expected noise-free image: For video images, the expected noise-free image is continuous in the spatial dimension and also continuous in the temporal dimension, so it has the prior characteristics of spatiotemporal continuity. The expected noise-free image contains dynamic images. Although it is not low-rank, it uses L because of its weak correlation in spatiotemporal continuity. 2，1 Regularization constrains the group sparsity of the desired noise-free image and promotes the separation of noisy images and the desired noise-free images.

[0088] Condition 2: Smoothness of the noise image along the horizontal direction: Under normal circumstances, assuming that the direction of the readout circuit stripe noise is horizontal, if vertical stripes appear, the image is rotated. The derivatives of the stripe noise and the expected noise-free image along the horizontal direction are different, that is, the derivative of the stripe noise along the horizontal direction is sparser than the derivative of the expected noise-free image. Therefore, we use L 1 D x (uf) norm to enhance the smoothness of noise along the horizontal direction.

[0089] Condition 3, smoothness of the expected noise-free image along the vertical direction: The expected noise-free image is piecewise smooth, which indicates that the derivatives of each frame in the infrared video image in the vertical and horizontal directions are not dense. The horizontal stripes destroy the smoothness along the vertical direction. Compared with the derivatives of the noisy image, the derivatives of the expected noise-free image are sparse along the vertical direction. Therefore, the derivatives of the stripe noise along the horizontal direction are dense. Therefore, we use L 1 D y The u-norm is used to enhance the smoothness of the desired noise-free image along the vertical direction.

[0090] Condition 4: Smoothness of non-uniform noise image along the time direction: Since non-uniform noise is additive noise, the derivative of the noise image is sparse along the time direction. The derivative of the noise image along the time direction is sparse, while the derivative of the expected noise-free image is not sparse. Therefore, using L 1 D t (uf) norm is used to enhance the smoothness of the noisy image along the time direction.

[0091] in,

[0092] D x , D y , D t are the corresponding linear operators of the horizontal, vertical and time domain first-order difference operators, so the non-uniformity correction model is established as follows:

[0093]

[0094] Step S600, decomposing the alternating operator method into sub-problems; solving the sub-problems using the alternating direction operator method, thereby adaptively preprocessing the infrared image sequence;

[0095] Let d x =D x (uf),d y =d y u,d t =D t (uf),d 4 =ufThe above formula becomes

[0096]

[0097] std x =D x (uf),d y =D y u,d t =D t (uf),d 4 =u.

[0098] Step S700, iteratively optimizing sub-problems;

[0099] The above non-uniformity correction model is further transformed into an unconstrained minimization problem using Bregman iteration;

[0100]

[0101] In the above formula, α 1 , α 2 , α 3 , α 4is the Bregman penalty coefficient, variable b x 、b y 、b t 、b 4 is an intermediate variable determined by Bregman iteration.

[0102] Step S800, threshold judgment, judging whether convergence; if the conclusion is no, after updating the intermediate variables, return to step S700, iterate the optimization sub-problem again, and enter this step; if the conclusion is yes, directly output the real image.

[0103]

[0104] An ADMM (Alternating Direction Multiplier Method) was developed to optimize the solution. One variable was optimized iteratively while other variables were fixed. The non-uniformity correction model can be divided into five minimization sub-problems, as follows:

[0105] Step S801, the sub-questions related to u are as follows:

[0106]

[0107] Since it is a convex function, it is equal to the following linear system:

[0108] (α 1 D x T D x +α 2 D y T D y +α 3 D t T D t +α 4 ) k+1 =α 1 D x T (d x k +D x fb x k )+α 2 D y T (d y k -b y k )+α 3 D t T (d t k +D t fb tk )+α 4 *d 4 k -b 4 k );

[0109] The superscript k is the number of iterations, the superscript T is the matrix transposition factor, and the closed-form solution of the fast Fourier transform (FFT) is used to solve the above equation;

[0110]

[0111] where Ι is the identity matrix, is the fast Fourier transform, is the inverse fast Fourier transform;

[0112] Step S802, with d x The relevant sub-questions are as follows:

[0113]

[0114] It can be calculated by the soft threshold operator,

[0115]

[0116] where Shrink(·) is a soft threshold operation,

[0117]

[0118] Similarly, with d y d t Related sub-questions are related to d x Same, so

[0119]

[0120] Among them, b x k+1 、b y k+1 、b t k+1 Update with the following formula

[0121]

[0122] Step S803, with d 4 The relevant sub-questions are as follows:

[0123]

[0124] in, From step S802, it can be seen that

[0125] Iterate u k+1 , b x k+1 、b y k+1 、b t k+1 、b 4 k+1 Calculate the value of ε, set the parameter ε, when When , the iteration stops, norm(·) is the normalization function; output u at this time k+1 , as the desired noise-free image.

[0126] Using the public infrared data set, the noise image is simulated. Figure 2-3 As shown, a random stripe noise with an intensity of 25 is used, and it is rotated 60 degrees and 120 degrees and superimposed as stripe non-uniformity noise. The center point of the image is used as the starting point, a Gaussian distribution probability density function with a standard deviation of 25 is used, and the distance between other pixels and the center point of the image is used as the random variable of the Gaussian distribution probability density function, which is set as the fixed pattern noise caused by optics. After superimposing it with the stripe non-uniformity noise, a simulated image data set is obtained.

[0127] For blind pixel correction, the threshold a is set to 0.1. For delta1 and delta1, points greater than 0.1 are considered blind pixels. For the established non-uniform noise model, we set α 1 , α 2 , α 3 , α 4 Set to 50, 50, 50, 50, and set λ 1 , 2 , 3 , 4 Set to 100, 5, 100, 100, set ε to 0.005, and k to 500. When the optimization loop Or when k is greater than 500, the iteration is terminated and u is output k+1 , as the real image.

[0128] The noisy images and simulated noisy images using the algorithm are evaluated, e.g. Figure 4-5 As shown, its peak signal-to-noise ratio increased by 20% and its structural similarity increased by about 3 times.

[0129] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of the present invention. That is, any equivalent changes and modifications made according to the content of the patent application scope of the present invention should be within the technical scope of the present invention.

Claims

1. A preprocessing method for infrared image sequences, characterized in that: Blind pixel correction and non-uniformity correction are performed on the infrared image sequence, and the blind pixel correction includes blind pixel compensation for the image; the steps include: Step S100, inputting an image; Step S200, establishing a time domain value model; taking an average of each pixel point of the image in the time domain to obtain a time domain mean; Step S300, local nonlinear filtering; in the sliding window of the image, the maximum value and the minimum value in the window are removed, and the local average value of the window is subjected to dynamic threshold detection with the maximum value and the minimum value in the original window; Step S400, determining blind pixels by threshold comparison; performing dynamic threshold detection on the time domain mean of each point, the local mean of the window, the maximum value and the minimum value in the window, and recording pixels exceeding the threshold as blind pixels; If it is a blind pixel, a blind pixel table is established, blind pixel compensation is performed, and the image is output to proceed to the next step; if it is not a blind pixel, the image is directly output to proceed to the next step; Step S500, establish a time-space domain difference operator non-uniformity correction model; after median filtering compensation for blind pixels, establish a time-space domain difference operator non-uniformity correction model for the infrared image sequence, using L 2，1 The norm constrains the group sparsity of the denoised image, and two one-way total variation regularization methods are used to enhance the smoothness of the noisy image in the horizontal and temporal directions; Step S600, decomposing the alternating operator method into sub-problems; solving the sub-problems using the alternating direction operator method, thereby adaptively preprocessing the infrared image sequence; Step S700, iteratively optimizing sub-problems; Step S800, threshold judgment, judge whether convergence; if the conclusion is no, after updating the intermediate variables, return to step S700, iterate the optimization sub-problem again, and enter this step; if the conclusion is yes, directly output the real image.

2. The pretreatment method according to claim 1, characterized in that: Step S200 is for the infrared image sequence f(i,j,t), where t is the frame number, (i,j) represent the horizontal and vertical coordinates of the focal plane pixel, i = 1,2,…,m, j = 1,2,…,n, t = 1,2,…,T; m and n represent the number of rows and columns of each image, respectively, and T represents the total number of frames in the infrared image sequence; first, the time domain mean is taken for each pixel point An infrared image sequence is calculated as a time-domain mean image.

3. The pretreatment method according to claim 2, characterized in that: Step S300, perform local nonlinear filtering on the time domain mean image, specifically, B(i,j) represents the maximum and minimum values ​​of the time domain mean image in the square neighborhood centered at (i,j) minus the average grayscale value in the spatial domain, N is the side length of the square neighborhood, the unit is pixel, N is 3 pixels, B max (i±p,j±q) is the maximum value in a square neighborhood with sides p and q, B min (i±p,j±q) is the minimum value in a square neighborhood with sides of p and q; 4. The pretreatment method according to claim 3, characterized in that: Step S400, determining blind pixels by threshold comparison, the specific steps are: Step S401, using the mean value B(i,j) calculated in step S300 to calculate the fixed blind pixel of the image, the threshold is related to the temporal mean of the current point, the local mean value of the window, the maximum temporal mean value of the window, and the minimum temporal mean value of the window. Wherein, B(i,j) is the gray value of the pixel (i,j), and the thresholds delta1 and delta2 are respectively the ratio of the difference between the local mean value of the window and the maximum and minimum values ​​of the window relative to the local mean value; Step S402, compare the thresholds delta1 and delta2 with the ratio a, if they are greater than a, then the point (i, j) is determined to be a blind pixel, where a is an adjustable parameter and the range of a is between 0.1-0.5; 5. The pretreatment method according to claim 4, characterized in that: Step S500 is to model the non-uniform noise of the infrared image sequence to form a time-space domain differential operator non-uniformity correction model, and regard a single infrared image as a two-dimensional variation function; the gain coefficient is regarded as 1, and the mathematical expression of the non-uniform noise model is as follows: f = u + b, f is the observed image, u is the expected noise-free image, b is the non-uniform bias noise image, and each image is stacked into a long column vector of size mnT×1, f = [vec(f1); vec(f2); ..., vec(f T )]∈R mnT×1 , where vec(f t )(t=1,2,…,T) represents the observed image of the tth frame; four constraints are proposed for the non-uniformity correction model based on the spatiotemporal domain, as follows: Condition 1: Group sparsity of the expected noise-free image: For video images, the expected noise-free image is continuous in the spatial dimension and also continuous in the temporal dimension, so it has the prior characteristics of spatiotemporal continuity. The expected noise-free image contains dynamic images. Although it is not low-rank, it uses L because of its weak correlation in spatiotemporal continuity. 2，1 Regularization constrains the group sparsity of the desired noise-free image and promotes the separation of noisy images and the desired noise-free images. Condition 2: Smoothness of the noise image along the horizontal direction: Under normal circumstances, assuming that the direction of the readout circuit stripe noise is horizontal, if vertical stripes appear, the image is rotated. The derivatives of the stripe noise and the expected noise-free image along the horizontal direction are different, that is, the derivative of the stripe noise along the horizontal direction is sparser than the derivative of the expected noise-free image. Therefore, we use D of L1 x (uf) norm to enhance the smoothness of noise along the horizontal direction. Condition 3, smoothness of the expected noise-free image along the vertical direction: The expected noise-free image is piecewise smooth, which indicates that the derivatives of each frame in the infrared video image in the vertical and horizontal directions are not dense. The horizontal stripes destroy the smoothness along the vertical direction. Compared with the derivatives of the noisy image, the derivatives of the expected noise-free image are sparse along the vertical direction. Therefore, the derivatives of the stripe noise along the horizontal direction are dense. Therefore, we use D y The u-norm is used to enhance the smoothness of the desired noise-free image along the vertical direction. Condition 4: Smoothness of non-uniform noise image along the time direction: Since non-uniform noise is additive noise, the derivative of the noise image is sparse along the time direction. The derivative of the noise image along the time direction is sparse, while the derivative of the desired noise-free image is not sparse. Therefore, using L1 D t (uf) norm is used to enhance the smoothness of the noisy image along the time direction. in, D x , D y , D t are the corresponding linear operators of the horizontal, vertical and time domain first-order difference operators, so the non-uniformity correction model is established as follows:

6. The pretreatment method according to claim 5, characterized in that: In step S600, let d x =D x (uf),d y =D y u,d t =D t (uf), d4=ufThe above formula becomes std x =D x (uf),d y =D y u,d t =D t (uf),d4u.

7. The pretreatment method according to claim 6, characterized in that: Step S700, using Bregman iteration to further transform the above non-uniformity correction model into an unconstrained minimization problem; In the above formula, α1, α2, α3, and α4 are Bregman penalty coefficients, and variable b x , b y , b4, b4 are intermediate variables determined by Bregman iteration.

8. The pretreatment method according to claim 7, characterized in that: Step S800 is to use An ADMM (Alternating Direction Multiplier Method) was developed to optimize the solution. One variable was optimized iteratively while other variables were fixed. The non-uniformity correction model can be divided into five minimization sub-problems, as follows: Step S801, the sub-problems related to u are as follows: Since it is a convex function, it is equal to the following linear system: (α1D x T D x +α2D y T D y +α3D t T D t +α4)u k+1 =α1D x T (d x k +D x fb x k )+α2D y T (d y k -b y k )+α3D t T (d t k +D t fb t k )+α4*d4 k -b4 k ); The superscript k is the number of iterations, the superscript T is the matrix transposition factor, and the closed-form solution of the fast Fourier transform (FFT) is used to solve the above equation; where Ι is the identity matrix, is the fast Fourier transform, is the inverse fast Fourier transform; Step S802, with d x The relevant sub-questions are as follows: It can be calculated by the soft threshold operator, where Shrink(·) is a soft threshold operation, Similarly, with d y ,d t Related sub-questions are related to d x Same, so Among them, b x k+1 , b y k+1 , b t k+1 Update with the following formula Step S803, the sub-questions related to d4 are as follows: in, From step S802, it can be seen that 9. The pretreatment method according to claim 8, characterized in that: Iterate u k+1 , b x k+1 , b y k+1 , b t k+1 、b4 k+1 Calculate the value of ε, set the parameter ε, when When , the iteration stops, norm(·) is the normalization function; output u at this time k+1 , as the desired noise-free image.

Images