Infrared small target detection method, device and storage medium in complex noise scene

Abstract

The present invention discloses a method, device and storage medium for detecting small infrared targets in complex noise scenes. A deep denoising prior is introduced through a plug-and-play structure to help better denoise and retain image detail information in complex noise scenes. At the same time, the weighted sum of tensor nuclear norms is used to more accurately evaluate the background. The present invention adopts an alternating direction multiplier method to solve this non-convex problem. Experimental results on different scenes show that the method proposed by the present invention has stronger robustness and higher detection rate in complex noise scenes.

Description

Technical Field

[0001] The present invention relates to infrared small target detection technology, in particular to an infrared small target detection method, device and storage medium in a complex noise scene. Background Art

[0002] The research on infrared small target detection has a wide range of application value. However, due to the long imaging distance of the infrared detection system, the target usually lacks shape information and texture information. In addition, the background clutter and noise interference in the imaging system are strong, and the small target is usually buried in the complex noise background with low signal-to-noise ratio. Therefore, infrared small target detection is still a challenging task and has attracted widespread attention.

[0003] In the past few decades, many infrared small target detection methods have been proposed. These methods are divided into model-based and deep learning-based methods. Among them, low-rank sparse decomposition methods are one of the classic model-based methods. Low-rank sparse decomposition methods have achieved good detection results in uniform scenes, but poor performance in noisy scenes. The main reason is that they ignore the impact of noise on small target detection tasks. Therefore, many scholars have introduced noise terms in low-rank sparse decomposition methods. These methods assume that under ideal conditions, the noise obeys Gaussian independent and identical distribution, so the Frobenius norm is used to constrain the noise. In fact, infrared images often contain complex noise. In order to better remove noise, the literature [X.Wang, Z.Peng, D.Kong, P.Zhang, and Y.He, “Infrared dim target detection based on totalvariation regularization and principal component pursuit,” Image and VisionComputing, vol.63, pp.1–9, 2017.] introduced the total variation (TV) regularization term in the low-rank model. The above methods improve the performance of small target detection in noisy scenes, but the detection results are not ideal in complex noise scenes. The main reason is that the assumptions about noise in the above methods are relatively simple. Therefore, these methods cannot match the complex noise in real scenes well, so the detection results of infrared small targets in real scenes are not ideal.

[0004] Considering the complex noise scene, the literature [C.Gao, L.Wang, Y.Xiao, Q.Zhao, and D.Meng, "Infrared small-dim target detection based on markov random field guidednoise modeling," Pattern Recognition, vol.76, pp.463–475, 2018.] proposed a mixed Gaussian noise model guided by Markov random field. Although this method has achieved good performance, it ignores the fact that the noise between different frames in the real scene does not obey the Gaussian independent and identical distribution. Therefore, in some complex noise scenes, the detection results of this method will be affected. Furthermore, the literature [Y.Sun, J.Yang, and W.An, "Infrared smalltarget detection based on reweighted infrared patch-image model and totalvariation regularization," in Image and Signal Processing for Remote Sensing XXV, vol. 11155. International Society for Optics and Photonics, 2019, p. 111551F.] combines the improved Markov random field and non-independent Gaussian distribution for infrared small target detection. This method has good detection performance in complex noise scenes. This shows that the noise in real scenes is complex, and using only the Frobenius norm or total variation regularization is not enough to describe the noise distribution of real scenes, resulting in problems such as low detection rate and high false alarm rate. Therefore, it is necessary to find an effective denoiser to improve the performance of infrared small target detection in complex noise scenes.

[0005] Invention patent application CN113256585A provides a real-time detection method for small moving targets in infrared video based on spatiotemporal tensor decomposition. When constructing the infrared detection model, it is assumed that the noise in the infrared image is Gaussian independent and identically distributed, so only the Frobenius norm is used to constrain the noise term. The above method can obtain good detection results in simple scenes. However, the noise contained in real infrared images is very complex, and the above method will have problems such as low detection rate and high false alarm rate in real scenes. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a method, device and storage medium for detecting small infrared targets in complex noise scenes in view of the shortcomings of the existing technology, so as to improve the accuracy of small infrared target detection.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for detecting small infrared targets in complex noise scenes, comprising the following steps:

[0008] Acquire infrared images in complex noise scenes;

[0009] The infrared image is divided into a plurality of image blocks, all the image blocks are stacked, and an infrared block tensor is constructed;

[0010] Using the infrared block tensor, the following optimization model is established:

[0011]

[0012] in, They represent infrared image block tensor, background block tensor and target block tensor respectively; represents the weighting constant, ||B|| WSTNN Represents the background patch tensor The weighted sum of the tensor nuclear norms of , ⊙ represents the Hadamard product, is an implicit regularization term, k 1 , k 2 denotes the modal number, λ and λ 1 represents the regularization parameter; W is the local weight map; l represents the target block tensor 1 norm; the implicit regularization term is the infrared block tensor obtained by removing the noise in the infrared block tensor using the denoising network;

[0013] A target block tensor is obtained by calculating the above optimization model, and the target block tensor is reconstructed into a target image;

[0014] Detecting small infrared targets from the target image.

[0015] The present invention introduces a deep denoising prior into a low-rank sparse decomposition model through a PnP (Plug-and-play) structure. The deep denoising prior has a stronger denoising ability and better image detail protection ability for the noise constraint, which can help to better denoise and retain image detail information in complex noise scenes. Therefore, the method of the present invention can better describe the noise in real scenes, greatly improve the accuracy of infrared small target detection, and reduce the high false alarm rate of infrared small target detection.

[0016] The denoiser is FFDNet, which is used to remove noise in the background.

[0017] In the present invention, the specific implementation process of calculating the target block tensor through the optimization model includes:

[0018] make To fix the constant, solve the following submodel:

[0019] but, Among them, Th γ (X) = sign(X)max(X|-γ,0), μ t represents the penalty coefficient for the tth iteration, represents the Lagrange multiplier of the t-th iteration, represents the background patch tensor of the t-th iteration, Represents the target block tensor for the tth iteration.

[0020] The present invention obtains the target block tensor Th by a soft threshold method [reference A.Beck and M.Teboulle, "A fast iterative shrinkage-thresholding algorithm for linear inverse problems," SIAM journal on imaging sciences, vol.2, no.1, pp.183-202, 2009.] γ (X)=sign(X)max(X|-γ,0) represents the operation of soft threshold.

[0021] The background patch tensor for the tth iteration The calculation process includes:

[0022] Let the target block tensor be To fix the constant, solve the following submodel:

[0023] but, represents the Lagrange multiplier of the t-1th iteration, μ t-1 represents the penalty parameter for the t-1th iteration, k represents the background patch tensor 1 k 2 modal, Represents the auxiliary variable for the t-1th iteration.

[0024] The present invention obtains the background block tensor by using the gradient descent method [reference Ruder S. An overview of gradient descent optimization algorithms [J]. arXiv preprint arXiv: 1609.04747, 2016.] The calculation process is simple.

[0025] The iterative update expression of the Lagrange multiplier is:

[0026]

[0027] in, Represents t-SVT operation [Reference Chen L, Liu Y, Zhu C. Iterative block tensor singular value thresholding for extraction oflowrank component of image data [C] / / 2017IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2017:1862-1866.], t-fold(X,k 1 ,k 2 ) means according to k 1 ,k 2 Rearrange the dimensions of X in the specified order; FFDNet represents the deep denoising network, μ t+1 =min(ρμ t ,μ max ), μ max represents the penalty coefficient μ t The maximum value of , ρ is a constant, Indicates the maximum value of the penalty coefficient.

[0028] The present invention converts the constrained convex optimization problem into an unconstrained convex optimization problem by introducing Lagrange multipliers [reference Z.Lin, M.Chen, and Y.Ma, "The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices," arXiv preprint arXiv:1009.5055, 2010.] and auxiliary variables. Then, the problem is solved iteratively by the alternating direction multiplier method (ADMM) [reference S.Boyd, N.Parikh, and E.Chu, Distributed optimization and statistical learning via the alternating direction method of multipliers. Now Publishers Inc, 2011.] algorithm. ADMM is an important method for solving separable convex optimization problems. Due to its fast processing speed and good convergence performance, the ADMM algorithm is widely used in the fields of image processing, machine learning, etc.

[0029] As an inventive concept, the present invention also provides a terminal device, which includes a processor and a memory; the memory stores a computer program / instructions; the processor executes the computer program / instructions stored in the memory; the computer program / instructions are configured to implement the steps of the above-mentioned method of the present invention.

[0030] As an inventive concept, the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon; the computer program / instruction is executed by a processor to perform the steps of the above method of the present invention.

[0031] Compared with the prior art, the present invention has the following beneficial effects: the present invention introduces a deep denoising prior into the existing low-rank sparse decomposition model through a PnP structure, which can better remove noise and retain image detail information, thereby better solving the problems of low detection rate and high false alarm rate caused by complex noise scenes. In order to more accurately evaluate the background, the present invention replaces the nuclear norm with the WSTNN (weighted sum of tensor nuclear norm) norm, and organically combines the deep denoising prior and the low-rank prior, so as to obtain a clearer background image, thereby further improving the detection performance of small infrared targets in complex noise scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0033] Figure 2 Constructing a graph for a block tensor of an embodiment of the present invention;

[0034] Figure 3(a) to Figure 3(c) 3(a) is a ROC curve diagram for scenario 1, FIG3(b) is a ROC curve diagram for scenario 2, and FIG3(c) is a ROC curve diagram for scenario 3;

[0035] Figure 4 The figures are experimental results of the method according to the embodiment of the present invention and other methods in three scenarios. DETAILED DESCRIPTION

[0036] like Figure 1 As shown, the embodiment of the present invention uses MATLAB as an implementation platform to design an infrared small target detection method in a complex noise scene, including the following steps:

[0037] S1, obtain a real infrared image as the original input image;

[0038] S2, infrared block tensor is obtained by block tensor construction method;

[0039] The implementation method of step S2 is as follows: Assuming that the input infrared image is O, a sliding window of a fixed size is used to traverse the entire image from the upper left to the lower right, and then the obtained image blocks are stacked into a block tensor See the detailed process Figure 2 In the embodiment of the present invention, the sliding window size can be set according to actual conditions.

[0040] S3, establish infrared small target detection model;

[0041] (1) Analyze the characteristics of infrared images:

[0042] Because the infrared background changes slowly, low-rank tensors are usually used to describe it in low-rank sparse decomposition methods. In addition, due to the long imaging distance of the infrared system, small infrared targets only occupy a few pixels, which means that relative to the entire infrared image, is a sparse block tensor.

[0043] (2) Analysis of basic model:

[0044] Based on the above features, the infrared small target detection model can be written as:

[0045]

[0046] Solving the above model is an NP problem. Therefore, the nuclear norm is used as a convex approximation of the rank function, and the above formula is changed as follows:

[0047]

[0048] The above model works well in simple uniform scenes, but has problems such as low detection rate and high false alarm rate in complex noise scenes. Therefore, a noise term is introduced into the above model, and a total variation regularization term constraint is added to the background. The above model changes to the following form:

[0049]

[0050] The above model works well in simple noise scenes, but not in real scenes. This is mainly because the noise in real scenes is very complex, and only using the Frobenius norm or total variation regularization is not enough to describe the noise distribution in real scenes, resulting in problems such as low detection rate and high false alarm rate. In addition, the nuclear norm will over-shrink the singular value, which cannot accurately describe the low-rank characteristics of the background, and thus cannot accurately evaluate the background.

[0051] (3) Establish a new optimization model:

[0052] Based on the analysis of the above classic model, the following improvements are made:

[0053] 1) Introducing deep denoising prior into the existing model through the PnP structure, which can help better remove noise and retain image detail information.

[0054] 2) Use the WSTNN norm instead of the nuclear norm for more accurate background assessment.

[0055] At this point, the optimization model is established as follows:

[0056]

[0057] in, and Respectively represent the infrared image block tensor, background block tensor and target block tensor, represents the weighting constant and satisfies B WSTNN Represents the background patch tensor The weighted sum of the tensor nuclear norms of . ⊙ denotes the Hadamard product. It is an implicit regularization term, which can be selected from many denoisers through a PnP structure. The embodiment of the present invention selects a fast and flexible denoising network (FFDNet) to remove noise in the background. It is an implicit regularization term, which is mainly used to remove the noise of the input background block tensor and maintain the detail information in the background block. When constructing the infrared detection model, the existing technology assumes that the noise in the infrared image conforms to the Gaussian independent and identical distribution. Therefore, only the Frobenius norm or total variation regularization is used to constrain the noise term. This type of method can obtain better detection results in simple noise scenes. However, the noise contained in the real infrared image is very complex, so only using the Frobenius norm or total variation regularization is not enough to describe the noise distribution of the real scene, resulting in problems such as low detection rate and high false alarm rate. Therefore, it is necessary to find an effective denoiser to improve the infrared small target detection performance in complex noise scenes. In view of the above problems, considering the powerful ability of deep learning denoising, the embodiment of the present invention introduces a fast and flexible denoising network (FFDNet) through a PnP structure to remove noise in the background. Compared with the existing methods, the embodiment of the present invention can better combine deep prior and low-rank prior to remove complex noise in real scenes, thereby greatly improving the accuracy of infrared small target detection and reducing the high false alarm rate of infrared small target detection. W is a local weight map used to simultaneously encode relevant information between the background and the target. l represents the tensor block target 1 Norm.

[0058] In order to solve the model, auxiliary variables are introduced and The model is rewritten as follows:

[0059]

[0060] The model is solved by the alternating direction multiplier method (ADMM) as follows:

[0061] in, is the Lagrange multiplier, and μ is the penalty coefficient, is the Frobenius norm, L(.) represents the augmented Lagrangian function, which is obtained by introducing Lagrangian multipliers and auxiliary variables to transform the established infrared small target detection model from a constrained convex optimization problem to an unconstrained convex optimization problem. In order to solve the variable Etc., separate different variables from L(.), and then use ADMM to iteratively solve each variable.

[0062] S4, optimize the infrared background block tensor The method is:

[0063] make is a fixed constant, solving the background The sub-models are as follows:

[0064]

[0065] The updated expression is as follows:

[0066]

[0067] S5, optimize and solve infrared target block tensor The method is:

[0068] Let the infrared background block tensor is a fixed constant, solving the target The sub-models are as follows:

[0069]

[0070] The updated expression is as follows:

[0071]

[0072] Among them, Th γ (X) = sign(X)max(|X|-γ,0).

[0073] S6, the optimization solution method for the remaining parameters is:

[0074] In order to optimize the solution model, the alternating direction multiplier method is used to transform the constrained problem into an unconstrained problem. In the embodiment of the present invention, the optimization solution method of introducing parameters is as follows:

[0075] To facilitate model solving, auxiliary variables are introduced Optimization solution:

[0076] Let the infrared background block tensor Infrared target block tensor For fixed constants, solve the submodel:

[0077]

[0078] The updated expression is as follows:

[0079]

[0080] in, Represents the t-SVT operation. t-fold(X,k 1 ,k 2 ) means according to k1 ,k 2 Rearrange the dimensions of X in the specified order.

[0081] To facilitate model solving, auxiliary variables are introduced Optimization solution:

[0082]

[0083] The updated expression is as follows:

[0084]

[0085] in, In the embodiment of the present invention, in order to better solve the infrared background block tensor Introduced auxiliary variables Make Thus we get

[0086] Introducing Lagrange multipliers Updated as follows:

[0087]

[0088] Introduce the optimization solution of parameter μ:

[0089] μ t+1 =min(ρμ t ,μ max );

[0090] μ max represents the maximum value of the penalty parameter, μ max =1e10.

[0091] Introduce the optimization solution of parameter β:

[0092]

[0093] ρ is a constant. In the embodiment of the present invention, ρ=1.5, β max represents the maximum value of the penalty parameter, β max =1e10.

[0094] The background in the infrared image is low-rank and the target is sparse. The embodiment of the present invention establishes an infrared small target detection model based on low-rank sparse tensor decomposition; in order to obtain a more accurate background assessment, the embodiment of the present invention uses the sum of the weighted nuclear norm of the tensor to constrain the background. Considering that the infrared image contains complex noise, in order to better remove the noise and protect the detailed information of the image, the embodiment of the present invention introduces a deep denoising prior in the low-rank model through a PnP structure. By combining the deep denoising prior and the low-rank prior, the present invention can better solve the problem of infrared small target detection in complex noise scenes.

[0095] After solving the background block tensor subproblem, the target block tensor subproblem and other parameter subproblems, the uniform average of estimators reprojection scheme (UAE) in the literature [J.Salmon and Y.Strozecki, “Patch reprojections for non-local methods,” SignalProcessing, vol.92, no.2, pp.477–489, 2012.] is used to reconstruct the solved background block tensor and target block tensor into the background image and target image.

[0096] Finally, the embodiment of the present invention uses a threshold segmentation method [C. Gao, D. Meng, Y. Yang, Y. Wang, X. Zhou, and A. G. Hauptmann, “Infrared patch-image model for small target detection in a single image,” IEEE Transactions on Image Processing, vol. 22, no. 12, pp. 4996–5009, 2013.] to detect infrared small targets from the obtained target image, that is, to achieve infrared small target detection in complex noise scenes.

[0097] The embodiment of the present invention randomly adds mixed noise to three real infrared scenes to simulate the complex noise in real scenes. And conducts comparative experiments with 6 infrared small target detection methods to verify the strong robustness and high detectability of the infrared small target detection method provided by the present invention in complex noise scenes. The steps are as follows:

[0098] (1) Six infrared small target detection methods are selected, namely:

[0099] Method 1: Background suppression-based method [J.-F.Rivest and R.Fortin, “Detection of dimtargets in digital infrared imagery by morphological image processing,” Optical Engineering, vol.35, no.7, pp.1886–1893, 1996.], referred to as Top-hat method.

[0100] Method 2: Local contrast-based method [J.Han, S.Moradi, I.Faramarzi, H.Zhang, Q.Zhao, X.Zhang, and N.Li, “Infrared small target detection based on the weighted strengthened local contrast measure,” IEEE Geoscience and Remote Sensing Letters, 2020.], referred to as WSLCM method.

[0101] Method 3: Method based on matrix low-rank sparse decomposition [C.Gao,D.Meng,Y.Yang,Y.Wang,X.Zhou,and AGHauptmann,“Infrared patch-image model for small target detection in a single image,”IEEE Transactions on Image Processing,vol.22,no.12,pp.4996–5009,2013.], referred to as IPI method.

[0102] Method 4: Method based on tensor low-rank sparse decomposition [Z.Cao, X.Kong, Q.Zhu, S.Cao, and Z.Peng, “Infrared dim target detection via mode-k1k2 extension tensor tubal rankunder complex ocean environment,” ISPRS Journal of Photogrammetry and RemoteSensing, vol.181, pp.167–190, 2021.], referred to as METTR method.

[0103] Method 5: Tensor low-rank sparse decomposition method with TV regularization term [T.Liu, J.Yang, B.Li, C.Xiao, Y.Sun, Y.Wang, and W.An, “Non-convex tensor low-rank approximation for infraredsmall target detection,” arXiv preprint arXiv:2105.14974, 2021.], referred to as NMOG-MFD method.

[0104] Method 6: Markov random field based method [Y.Sun, J.Yang, and W.An, “Infrared smalltarget detection based on reweighted infrared patch-image model and totalvariation regularization,” in Image and Signal Processing for Remote Sensing XXV, vol. 11155. International Society for Optics and Photonics, 2019, p. 111551F.], referred to as ASTTV-NTLA method.

[0105] Method 7: A method for detecting small infrared targets in complex noise scenes (method of an embodiment of the present invention)

[0106] (2) In order to quantitatively evaluate the effectiveness of the method, the above six infrared small target detection methods are used to conduct experiments on three synthetic noise infrared sequence images, and the feature indicators are selected as: detection rate and false alarm rate, and ROC curves are drawn, as shown in the figure. Figure 3(a) to Figure 3(c) shown.

[0107] (3) For the excellence of qualitative evaluation methods, Figure 4 The test results of the above 6 methods and the embodiments of the present invention are shown in FIG. Figure 4 shown. Figure 3(a) to Figure 3(c) , Figure 4 The experimental results show that the method of the embodiment of the present invention has more advantages than the other six infrared small target detection models in both quantitative and qualitative evaluation, with higher detection rate and lower false alarm rate. Therefore, the solution of the embodiment of the present invention can effectively solve the problem of detecting small infrared small targets in complex noise scenes.

Claims

1. A method for detecting small infrared targets in complex noise scenes, characterized in that: The following steps are involved: Acquire infrared images in complex noise scenes; The infrared image is divided into a plurality of image blocks, all the image blocks are stacked, and an infrared block tensor is constructed; Using the infrared block tensor, the following optimization model is established: in, They represent infrared image block tensor, background block tensor and target block tensor respectively; represents the weighting constant, ||B|| WSTNN Represents the background block tensor The weighted sum of the tensor nuclear norms of , ⊙ represents the Hadamard product, is an implicit regularization term, k1 and k2 represent the modal numbers, λ and λ1 represent the regularization parameters; W is the local weight map; represents the l1 norm of the target block tensor; the implicit regularization term is the infrared block tensor obtained by removing the noise in the infrared block tensor using the denoising network; A target block tensor is obtained by calculating the above optimization model, and the target block tensor is reconstructed into a target image; Detecting a small infrared target from the target image; The specific implementation process of calculating the target block tensor through the optimization model includes: make To fix the constant, solve the following submodel: but, Among them, Th g (X) = sign(X)max(X|-g,0), μ t represents the penalty coefficient for the tth iteration, represents the Lagrange multiplier of the t-th iteration, represents the background patch tensor of the t-th iteration, represents the target block tensor of the tth iteration; The background patch tensor for the tth iteration The calculation process includes: Let the target block tensor be To fix the constant, solve the following submodel: but, represents the Lagrange multiplier of the t-1th iteration, μ t-1 represents the penalty parameter for the t-1th iteration, represents the auxiliary variable for the t-1th iteration, Represents the k1k2 mode of the background patch tensor.

2. The infrared small target detection method in complex noise scene according to claim 1 is characterized in that: The denoiser is FFDNet.

3. The infrared small target detection method in complex noise scene according to claim 1 is characterized in that: The iterative update expression of the Lagrange multiplier is: in, Represents the t-SVT operation, t-fold(X1,k1,k2) means rearranging the dimensions of X in the order specified by k1 and k2; FFDNet represents the deep denoising network, μ t+1 =min(rm t ,μ max ), μ max represents the penalty coefficient μ t The maximum value of , ρ is a constant, Indicates the maximum value of the penalty coefficient.

4. A terminal device, characterized in that: The invention comprises a processor and a memory; the memory stores a computer program / instruction; the processor executes the computer program / instruction stored in the memory; the computer program / instruction is configured to implement the steps of the method according to any one of claims 1 to 3.

5. A computer-readable storage medium having a computer program / instruction stored thereon; characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

Images