An iterative-based single-frame infrared image small target detection method and device

Abstract

The application discloses a kind of based on iteration's single-frame infrared image small target detection method and device. Including: (1) the infrared frame to be detected and its adjacent front and rear frame, construct as three-dimensional space-time tensor;(2) establish the model based on low-rank sparse tensor decomposition;(3) by the kernel information of background tensor and the spatial information of frontal slice, define the tensor kernel norm based on kernel information and log calculation;(4) based on structure tensor theory, extract the target saliency information tensor of current iteration round, weight subsequent round of image data to be decomposed;(5) establish the model based on low-rank sparse tensor decomposition of iteration;6) solve the model based on low-rank sparse tensor decomposition of iteration, extract the frontal slice in the middle of target tensor, as the detection result of infrared frame to be detected, realize the single-frame infrared small target detection based on iteration.The infrared small target detection model and solving algorithm based on iteration of the application can effectively improve the detection performance of infrared small target.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to a single-frame infrared image small target detection method and device based on iteration. BACKGROUND

[0002] Currently, the infrared search and tracking (IRST) system has a wide range of applications, such as mine detection, low-altitude safety, missile tracking, and other military and civilian fields. Among them, the infrared small target detection technology plays a crucial role in IRST. However, due to the influence of complex background, strong clutter and noise, and other factors, the existing thermal infrared small target detection task mainly faces the following challenges:

[0003] 1) Due to long-distance imaging and complex background, the signal-to-noise ratio of infrared images is relatively low, making it easy for infrared small targets to disappear in the surrounding environment, which poses a challenge to effective infrared small target detection tasks;

[0004] 2) Infrared small target detection will be affected by various disturbances such as inherent noise and structural noise, which will significantly cause false alarms and reduce the background suppression ability of the detection method;

[0005] 3) The infrared search and tracking (IRST) system has real-time requirements for infrared small target detection. Therefore, the infrared small target detection method not only needs to improve the detection accuracy, but also needs to optimize the operation efficiency to meet the requirements of practical applications.

[0006] Generally, infrared small target detection methods are mainly divided into data-driven based methods and model-driven based methods. Data-driven based methods use deep learning models as the framework, and design various networks to complete the infrared small target detection task. Research results show that deep learning models have strong adaptive ability, can adapt to different infrared scenes and target changes by learning the distribution and features of data, and deep learning models can learn more rich and abstract feature representations through large-scale data set training. However, data-driven based methods require a large amount of labeled data for training, and for the infrared small target detection task, labeled data is often difficult to obtain, which will limit the application of deep learning methods in infrared small target detection; deep learning models are usually black box models, and it is difficult to explain the decision-making process and reasoning results of the model, which may be limited in some application scenarios that require high interpretability, such as military and security fields.

[0007] The model-driven based methods can be further divided into background estimation filter based methods, local feature representation based methods, and component decomposition based methods. The background estimation filter based methods consider that the background part has high correlation, and achieve the purpose of background suppression and target extraction by designing spatial filter, frequency filter, etc. However, in the process of infrared small target detection in complex scenes, these filters not only enhance the infrared small target, but also highlight the saliency of the background texture, such as heavy cloud clutter and significant building edges, causing serious false alarms.

[0008] The local feature representation based methods aim to simulate the mechanism of the human visual system to encode the contrast features in the flow, and consider that the infrared small target usually shows the statistical characteristics of gray scale distribution which is easy to distinguish from the background clutter and noise, and a specific local contrast descriptor can be designed to describe the difference in characteristics. Chen et al. first proposed a descriptor called local contrast measure (LCM) for infrared small target detection. Later, a series of LCM variants were proposed, such as tri-layer window LCM (TLLCM), weighted strengthened LCM (WSLCM), weighted three-layer window LCM (WTLLCM), and halo structure prior (HSP)-based LCM (HSPLCM). However, in high-brightness complex backgrounds, such descriptors are prone to difficulties, because the clutter may be similar to the target, resulting in serious false alarms and affecting the effectiveness of small target detection.

[0009] The component decomposition based method is based on the fact that the background component of the infrared image presents a low-rank characteristic because most of the background parts are occupied by similar image blocks; the target presents a sparse characteristic because the target pixels only occupy a small number of pixels in the image. According to different image data forms (for example, two-dimensional matrix, three-dimensional spatial tensor, three-dimensional space-time tensor and four-dimensional space-time tensor), the component decomposition based method converts the infrared target detection task into a low-rank sparse component decomposition problem. For example, Pang et al. proposed a three-dimensional space-time tensor and tensor spectral k support norm minimization model for detecting infrared dim small targets with extremely sparse characteristics. Liu et al. proposed a non-convex tensor low-rank approximation (NTLA) method to adaptively assign different weights to different singular values to achieve background estimation, and designed an asymmetric space-time total variation (ASTTV) regularization based on NTLA. The proposed low-rank sparse decomposition based model achieved relatively excellent results on the test data set. Wang et al. established a non-overlapping space-time image block tensor (NPSTT) model, and on this basis, introduced TCNN regularization to improve the accuracy of background tensor estimation. Based on NPSTT and TCNN, an infrared small target detection model was designed, which achieved good detection results in complex scenes. Compared with the infrared small target detection algorithms based on background estimation filtering and local feature representation, the component decomposition based algorithm can better eliminate clutter, separate and enhance small targets from complex backgrounds. However, some sharp background edges may be decomposed into the sparse target component, so the small target detection method based on component decomposition needs to solve the problem of being sensitive to strong interference signals.

[0010] The existing infrared small target detection methods still cannot meet the demand of practical application in terms of target detection ability in complex scenes, background suppression ability and comprehensive effectiveness of the detector, and challenge the real-time performance. Therefore, it is necessary to further study the related strategies to improve the performance of infrared small target detection. SUMMARY

[0011] In view of the deficiencies in the prior art, the present application aims to provide an iterative-based single-frame infrared image small target detection method and device, which fully utilizes the space-time information carried by adjacent frames of images, and for the infrared image frame to be detected, uses the adjacent previous and subsequent frames of images for auxiliary detection and constructs a three-dimensional space-time tensor; from the data decomposition perspective, establishes a target detection model based on low-rank sparse tensor decomposition, and by mining the intrinsic space-time intrinsic information of the background component, defines a tensor kernel norm based on kernel information and log calculation, and improves the accuracy of background estimation; in addition, from the perspective of iterative solution, a saliency information extraction method is designed based on structure tensor theory, and based on the target detection model based on low-rank sparse tensor decomposition, an iterative-based low-rank sparse tensor decomposition model is established, which effectively improves the quality of the image data to be decomposed, enhances the target saliency of the decomposed data, and suppresses the background component of the decomposed data; through model iterative solution, the effectiveness of the target detection result of the infrared frame image to be detected is improved, the infrared small target detection is realized, and the target detection capability and background suppression capability of the method proposed by the present application are effectively verified.

[0012] To achieve the above object, the present application provides the following technical scheme:

[0013] The present application discloses an iterative-based single-frame infrared image small target detection method, characterized in that it comprises the following steps:

[0014] Step 1): using the time and space information carried by the thermal infrared image sequence, the current infrared frame image to be detected and its adjacent previous and subsequent frame images are constructed into a three-dimensional space-time tensor

[0015] Step 2): based on the characteristics of background, target and noise, a model based on low-rank sparse tensor decomposition is established;

[0016] Step 3): based on the kernel information of the background tensor and the space information carried by each frontal slice of the background tensor, a tensor kernel norm based on kernel information and log calculation is defined;

[0017] Step 4): in the process of iterative solution of the model based on low-rank sparse tensor decomposition, based on the structure tensor theory, the saliency information tensor of the temporary target tensor obtained by the current iteration is extracted and weighted to the image data to be decomposed in the next iteration round;

[0018] Step 5): based on the three-dimensional space-time tensor of step 1), combined with steps 2) to 4), an iterative-based low-rank sparse tensor decomposition model is established;

[0019] Step 6): the target tensor is obtained by solving the iteration-based low-rank sparse tensor decomposition model by using the ADMM algorithm, the central frontal slice of the target tensor is extracted as the infrared small target detection result of the infrared frame image to be detected, and the iteration-based single-frame infrared image small target detection is realized.

[0020] The application further discloses an iteration-based single-frame infrared image small target detection device for implementing the method.

[0021] The three-dimensional space-time tensor establishing module constructs the current infrared frame image to be detected and its adjacent front and rear frame images into a three-dimensional space-time tensor, so as to lay a data foundation for subsequent decomposition.

[0022] The low-rank sparse tensor decomposition model establishing module establishes a model based on low-rank sparse tensor decomposition according to the characteristics of the background, the target and the noise.

[0023] The low-rank background estimation norm design module defines a tensor kernel norm based on kernel information and log calculation according to the kernel information of the background tensor and the spatial information carried by each frontal slice of the background tensor.

[0024] The saliency information extraction module designs a saliency information tensor extraction method based on the structure tensor theory, and is used for extracting a saliency information tensor from a temporary target tensor obtained in the current iteration and weighting the saliency information tensor to image data to be decomposed in the next iteration round.

[0025] The iteration-based low-rank sparse tensor decomposition model establishing module establishes an iteration-based low-rank sparse tensor decomposition model based on the three-dimensional space-time tensor and the low-rank sparse tensor decomposition model, the low-rank background estimation norm and the low-rank background estimation norm.

[0026] The solving optimization model module solves the iteration-based low-rank sparse tensor decomposition model by using the ADMM algorithm, obtains the target tensor, and extracts the central frontal slice of the target tensor as the infrared small target detection result of the infrared frame image to be detected.

[0027] The target detection result output module is used for outputting the infrared small target detection result of the single-frame image.

[0028] Compared with the prior art, the application has the beneficial effects including:

[0029] 1) The present application makes full use of the rich time-space information carried by the infrared image sequence. For the infrared image frame to be detected, the adjacent front and rear frame images are used for auxiliary detection, and a three-dimensional space-time tensor is constructed. On this basis, an infrared small target detection model based on low-rank sparse tensor decomposition is established. At the same time, the kernel information of the background tensor and the spatial information carried by each positive slice of the background tensor are combined to define the tensor kernel norm based on the kernel information and log calculation, which improves the accuracy of low-rank background estimation.

[0030] 2) The present application proposes a saliency information extraction method based on structure tensor theory. Based on the target detection model based on low-rank sparse tensor decomposition, an iterative low-rank sparse tensor decomposition model is established. In the iterative solution process, the saliency information tensor is extracted from the temporary target tensor obtained by the current iteration and weighted to the image data to be decomposed in the next iteration, so as to enhance the target saliency of the decomposed data and suppress the background component of the decomposed data. The ADMM algorithm is used to solve the iterative low-rank sparse tensor decomposition model, which improves the effectiveness of the target detection result of the infrared frame image to be detected. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 The present application is a three-dimensional space-time tensor construction process diagram. The red image frame is the current infrared frame image to be detected, and the blue image frame is the image frame for auxiliary detection.

[0032] Figure 2 The present application is a three-dimensional tensor through t-SVD.

[0033] Figure 3 The present application is a kernel tensor The present application is a kernel matrix expanded according to the time dimension The present application is a schematic diagram.

[0034] Figure 4 The present application is a structure diagram of the infrared small target detection device.

[0035] Figure 5 The present application is an example frame image of a thermal infrared image sequence for experimental test.

[0036] Figure 6 The present application is the thermal infrared small target detection result of the example frame of the thermal infrared image sequence obtained by the method proposed in the present application.

[0037] Figure 7 The present application is the infrared small target detection result of the example frame of the thermal infrared image sequence and the infrared small target detection result after the proposed method, ELUM, RRIPT, METTR, ASTTV-NTLA, TCNN-NPSTT, TSPK, SRSTT and RCTVW detection. DETAILED DESCRIPTION

[0038] In order to make the objects, technical solutions and advantages of the present application clearer, the following will describe the present application in detail with specific embodiments. The following describes specific embodiments to simplify the present application. However, it should be appreciated that the present application is not limited to the described embodiments, and various modifications of the present application are possible without departing from the basic principles, and these equivalent forms also fall within the scope defined by the appended claims of the present application.

[0039] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the accompanying drawings in the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of protection of the present application.

[0040] The basic steps of the present application based on an iterative single-frame infrared image small target detection method mainly include:

[0041] Step 1: Using the time and space information carried by the thermal infrared image sequence, the current infrared frame image to be detected and its adjacent front and rear frame images are constructed into a three-dimensional space-time tensor

[0042] Specifically, as shown in Figure 1 , using the time and space information of the thermal infrared image, the current infrared frame image D t to be detected and its adjacent front and rear frame images D t-1 , D t-2 , …, D t-f , D t+1 , D t+2 , …, D t+f are stacked in the time order of the image frames to construct a three-dimensional space-time tensor with the size of n1×n2×n3, n1 represents the width of the frontal slice of D , n2 represents the height of the frontal slice of D , n3 represents the thickness of D , and n3=2f+1; wherein, D t-1 represents the first frame image before D t , D t-2 represents the second frame image before D t , D t-f represents the fth frame image before D t , D t+1 represents the first frame image after D t , D t+2 represents the second frame image after D t , D t+f represents the fth frame image after Dt The image of the fth frame after;

[0043] In the specific implementation example, f is set to 3, and the constructed three-dimensional spacetime tensor The dimensions are 256×256×7.

[0044] Step 2: Based on the characteristics of the background, target, and noise, establish a model based on low-rank sparse tensor decomposition;

[0045] Specifically, a single-frame infrared image D is modeled as a linear combination of a low-rank background matrix B, a sparse target matrix T, and a noise component matrix N. The corresponding model based on low-rank sparse matrix decomposition is established as shown in formula (1):

[0046]

[0047] Where rank(·) represents the operator for the background rank, and ||·|0 represents the l0 norm for estimating the sparsity of the target. λ1 and λ2 represent the square of the Frobenius norm that measures random noise, and the weighting coefficients.

[0048] For the three-dimensional spacetime tensor constructed in step 1), It preserves the characteristics of background, target, and noise components, thereby transforming the three-dimensional spatiotemporal tensor. Modeled as a low-rank background tensor Sparse target tensor and noise component tensor A linear combination of these elements is used to establish a model based on low-rank sparse tensor decomposition, as shown in equation (2):

[0049]

[0050] Where, ||·||1 represents the l1 norm used for the optimal approximation of the l0 norm.

[0051] Step 3: Define the tensor kernel norm based on kernel information and logarithmic calculation, using the kernel information of the background tensor and the spatial information carried by each frontal slice of the background tensor.

[0052] Specifically, for the background tensor The k-th frontal slice The front slice contains R singular values. Calculate σ for each singular value. r The corresponding log value is obtained R represents the total number of singular values ​​in the frontal slice, from which the tensor nuclear norm ‖·‖ calculated based on logarithm is defined. LgTNN As shown in formula (4):

[0053]

[0054] Where ε1 represents a positive constant, and log(·) is the log operation operator;

[0055] like Figure 2 As shown, for the background tensor Performing t-SVD operations, there are Two orthogonal tensors are obtained and background tensor kernel tensor Kernel Tensor The size is n1×n2×n3, where n1 represents The width, n2 represents The high, n3 represents thickness, kernel tensor Its size is consistent with that of the three-dimensional spacetime tensor, and it contains rich spacetime feature information, such as... Figure 3 As shown, the kernel tensor Expanded into a kernel matrix according to the time dimension Γ(·) represents the operator for expanding the kernel matrix into a matrix, and n3 represents the kernel matrix. The height of the matrix High numerical values ​​and kernel tensors The thickness n3 is the same, and min{n1,n2} represents the kernel matrix. The width is the smaller of n1 and n2; thus, the nuclear norm of the kernel matrix calculated based on logarithm is defined as ||·||. LgNN As shown in formula (5):

[0056]

[0057] Where ε2 represents a positive constant, Represents the kernel matrix The l-th singular value, l = 1, 2, ..., L, where L represents the kernel matrix. The total number of singular values;

[0058] Combining formulas (4) and (5), we obtain the tensor nuclear norm ||| based on kernel information and logarithmic calculation. L-ITNN As shown in formula (6):

[0059]

[0060] Step 4: In the iterative solution of the model based on low-rank sparse tensor decomposition, the saliency information tensor of the temporary target tensor obtained in the current iteration is extracted based on the structural tensor theory and weighted into the image data to be decomposed in the next iteration.

[0061] Specifically, the solution process of model (2) based on low-rank sparse tensor decomposition is an iterative process. For the current iteration i, the calculated target tensor is denoted as... Target Tensor Each frontal slice is denoted as For each element (x, y) in the matrix, the corresponding structure tensor matrix is: Where G is the Gaussian kernel function, I x yes Horizontal gradient I y yes vertical gradient The eigenvalues ​​of the structure tensor matrix J for each position element (x, y) are α1 and α2. For pixels in flat regions, α1 ≈ α2 ≈ 0; for pixels in corner regions, α1 ≥ α2 >> 0; and for pixels in edge-line regions, α1 >> α2 ≈ 0. Therefore, the eigenvalues ​​α1 and α2 of the structure tensor J are used to extract... The saliency information W at each position element (x, y) (i) (x,y) is

[0062]

[0063] Where ε3 is a positive constant, W (i) (x,y) represents the target tensor in the current iteration i. The saliency information corresponding to the position element (x,y) of the current front slice;

[0064] From formula (7), we can obtain the target tensor in the current iteration i. Current front slice The corresponding saliency information matrix W (i) For each frontal slice, a corresponding saliency information matrix is ​​extracted and normalized. Therefore, in the current iteration i, the saliency information matrices corresponding to n3 frontal slices can be obtained. By stacking all the saliency information matrices in order, the target tensor can be obtained. saliency information tensor

[0065] The process of extracting saliency information tensors is represented as follows: Represents the target tensor Operators for extracting saliency information; saliency information tensor Tensor data to be decomposed in the next weighted iteration round i+1 As shown in formula (8):

[0066]

[0067] wherein, ⊙ denotes Hadamard product, denotes the saliency information tensor iteratively updating to obtain the to-be-decomposed image data in the i+1 iteration round denotes the saliency information tensor the weighted to-be-decomposed tensor, thereby obtaining the weighted to-be-decomposed tensor data

[0068] Step 5: based on the three-dimensional space-time tensor of step 1), combined with steps 2) to 4), an iterative low-rank sparse tensor decomposition model is established;

[0069] Specifically, in order to speed up the model solving speed, a sparsity reweighting strategy is introduced into formula (2), and the target tensor obtained in the current iteration round i is the sparsity reweighting weight of the target in the next iteration round i+1 is calculated As shown in formula (9):

[0070]

[0071] wherein, ε4 denotes a normal number, (·) -1 denotes the element inverse operation, and |·| denotes the absolute value operation;

[0072] Combined with steps 2) to 4), an iterative low-rank sparse tensor decomposition model is established, as shown in formula (10):

[0073]

[0074] Step 6: using the ADMM algorithm to solve the iterative low-rank sparse tensor decomposition model, obtaining the target tensor, and extracting the middle frontal slice of the target tensor as the infrared small target detection result of the to-be-detected infrared frame image, realizing the iterative single-frame infrared image small target detection;

[0075] Specifically, the ADMM algorithm is used to solve the iterative low-rank sparse tensor decomposition model (10), in order to make each variable in (10) solvable, the auxiliary variable is introduced respectively The formula (11) is as follows:

[0076]

[0077] The corresponding augmented Lagrangian function is

[0078]

[0079] where, M3 is the Lagrange multiplier, μ is the penalty term; each variable is solved by alternating iteration, and the subproblem corresponding to each variable is as follows:

[0080] 1) -Subproblem

[0081] Fixing other parameters in the i-th iteration, the iterative formula of variable is

[0082]

[0083] where, argmin represents obtaining the parameter value corresponding to the minimum value of the function, represents

[0084] The solving formula equivalent to formula (13) is:

[0085]

[0086] where, represents the tensor singular value decomposition, represents the orthogonal tensor after the tensor singular value decomposition, represents the transpose of the tensor , * represents the tensor product operator, represents the singular value tensor of the tensor , D (·) (·) is the singular value shrinkage operator, σ is the singular value, is the soft threshold, diag[·] is used to obtain the element value on the diagonal line of the matrix, τ + = max{τ, 0} represents the maximum value operator;

[0087] 2) -Subproblem

[0088] Fixing other parameters in the i-th iteration, the iterative formula of variable is

[0089]

[0090] Similar to formula (15), the solving formula of formula (15) is

[0091]

[0092] where, represents the matrix singular value decomposition, σ is the singular value in the singular value matrix S;

[0093] 3) -Subproblem

[0094] Fixing other parameters in the i-th iteration, the variable is updated by the iteration formula

[0095]

[0096] Transforming formula (17) into the matrix domain for solving:

[0097]

[0098] where W (i) denote the tensor the corresponding matrix after mode-3 expansion;

[0099] Solving formula (18) gives

[0100]

[0101] Then we have

[0102]

[0103] where fold3(·) denotes the operator that converts a matrix to a three-dimensional tensor according to mode-3;

[0104] 4) -Subproblem

[0105] Fixing other parameters in the i-th iteration, the variable is updated by the iteration formula

[0106]

[0107] where Th (·) (·) is a soft threshold shrinkage operator, Th (τ) (x) = sign(x) · max(|x| - τ, 0), sign(·) is a sign function, and τ is a soft threshold;

[0108] 5) -Subproblem

[0109] Fixing other parameters in the i-th iteration, the variable is updated by the iteration formula

[0110]

[0111] 6) -Subproblem

[0112] By get The iteration formula of

[0113]

[0114] 7) Lagrange multiplier - sub-problem

[0115] Fixing other parameters in the i-th round of iteration, the iteration formula of Lagrange multiplier The iteration formula of M3 is

[0116]

[0117] 8) penalty term - sub-problem

[0118] Fixing other parameters in the i-th round of iteration, the iteration formula of penalty term μ is

[0119] μ (i+1) = min (γμ (i) , μ max ) (28) Wherein, γ represents an update coefficient, μ max represents the maximum value of μ;

[0120] The iteration stopping condition is: when the relative error is less than the error δ, the iteration is terminated; when the l0 norm of the target tensor no longer changes, the iteration is stopped; when the adjacent iteration , the iteration is stopped; when the adjacent iteration , the iteration is stopped;

[0121] Further, the middle frontal slice of the target tensor is extracted as the infrared small target detection result image T t corresponding to the infrared frame image D t , realizing the small target detection of the single frame infrared image based on iteration.

[0122] Specifically in the embodiments, λ2=50λ1, λ L =1.6, μ max =100, γ=1.2, μ (0) =0.005, δ=0.0001;

[0123] Corresponding to the foregoing one embodiment of the small target detection method of the single frame infrared image based on iteration, the application further provides one embodiment of the small target detection device of the single frame infrared image based on iteration.

[0124] Figure 4A block diagram of an infrared small target detection device based on posterior information according to an exemplary embodiment is shown in FIG. 1. As shown in FIG. 1, the device comprises: Figure 4

[0125] A three-dimensional space-time tensor establishing module constructs the current infrared frame image to be detected and its adjacent front and rear frame images into a three-dimensional space-time tensor, laying a data foundation for subsequent decomposition;

[0126] A low-rank sparse tensor decomposition model establishing module establishes a model based on low-rank sparse tensor decomposition according to the characteristics of background, target and noise;

[0127] A low-rank background estimation norm design module defines a tensor kernel norm based on kernel information and log calculation, in combination with the kernel information of the background tensor and the spatial information carried by each frontal slice of the background tensor;

[0128] A saliency information extraction module designs a saliency information tensor extraction method based on structure tensor theory, for extracting a saliency information tensor from the temporary target tensor obtained in the current iteration and weighting it to the image data to be decomposed in the next iteration round, in an iterative solving process;

[0129] A low-rank sparse tensor decomposition model establishing module based on iteration establishes a low-rank sparse tensor decomposition model based on iteration, in combination with the low-rank background estimation norm, the low-rank background estimation norm, the three-dimensional space-time tensor and the low-rank sparse tensor decomposition model;

[0130] A solving optimization model module solves the low-rank sparse tensor decomposition model based on iteration by using an ADMM algorithm, obtains a target tensor, and extracts the middle frontal slice of the target tensor as the infrared small target detection result of the infrared frame image to be detected;

[0131] A target detection result output module is configured to output the infrared small target detection result image of the single frame image.

[0132] As to the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment relating to the method, and will not be described in detail here.

[0133] ​For the device embodiment, since it basically corresponds to the method embodiment, the relevant part is described in the method embodiment. The device embodiment described above is only illustrative, and each module in the device is a logical functional division. In actual implementation, another division mode can be used, for example, a plurality of modules can be combined or integrated into another unit. In addition, the connection between the displayed or discussed modules can be a communication connection through some interfaces, which can be electrical or other forms. According to actual needs, part or all of the modules can be selected to achieve the purpose of the scheme of the present application. Those skilled in the art can understand and implement without creative labor. The specific implementation is described below with the disclosed real thermal infrared image sequence as an example to embody the technical effect of the present application, and the specific steps in the embodiment are not described again.

[0134] Embodiment

[0135] The drawings shown in the embodiment of the present application can make the purpose, technical scheme and advantage of the present application more clear and understandable. It should be noted that the specific embodiments described herein are only used to explain the present application and not to limit the present application. Any equivalent replacement, improvement, etc. within the method idea and principle provided by the present application should be included in the protection scope of the present application.

[0136] In the embodiment, the effectiveness of the thermal infrared small target detection algorithm is verified through a disclosed infrared image sequence. The disclosed infrared image sequence is described as follows:

[0137] The thermal infrared image sequence contains 100 real infrared frame images, and the frame image size is 256x256. There are a large number of strong clutter and noise in the background, and there are high-brightness cloud layer textures. In the air-ground complex background, there is a slow flying unmanned aerial vehicle passing through the cloud layer. Figure 5 An example frame of the infrared image sequence is shown in FIG. 1. Figure 6 The target detection result image obtained by detecting the example frame of the infrared image sequence by the method of the present application is shown in FIG. 2. As shown in the detection result image, the background and noise are completely suppressed, the small target is completely detected, and the saliency of the target is significantly enhanced. To quantitatively evaluate the effectiveness of the proposed iterative-based single infrared image small target detection method, from the qualitative point of view, the intuitive target detection result image of an example frame of the infrared image sequence is used for evaluation; from the quantitative point of view, the 3D-ROC evaluation index system is used to evaluate the detection performance of the algorithm. The 3D-ROC evaluation index system contains three types of indexes, including target detection capability (TD), background suppression capability (BS) and comprehensive effectiveness evaluation index of the detector, which are specifically introduced as follows:

[0138] 1) Target detection capability

[0139] AUC(D,τ) Indicates (P) D The AUC value of the (τ) curve can measure the target detection capability of the detector.

[0140] 0≤AUC (D,τ) ≤1

[0141] AUC TD The joint detection capability of the detectors is defined as follows:

[0142] 0≤AUC TD =AUC (D,F) +AUC (D,τ) ≤2

[0143] 2) Background detection capability

[0144] AUC (F,τ) Indicates (P) D The AUC value of the curve (τ) characterizes the background suppression capability of the detector, and is defined as 0 ≤ AUC. (F,τ) ≤1

[0145] AUC BS The joint background suppression capability of the detector is characterized by, as defined

[0146] -1≤AUC BS =AUC (D,F) -AUC (F,τ) ≤1

[0147] AUC SNPR The signal-to-noise ratio (SNR) characterizing the detector is defined as follows:

[0148]

[0149] 3) Overall effectiveness of the detector

[0150] AUC TDBS The combined target detection and background suppression capability of a characterizing detector is defined as follows:

[0151] -1≤AUC TDBS =AUC (D,τ) -AUC (F,τ) ≤1

[0152] AUC (D,F) Characterization (P) D ,P F The AUC value of the curve characterizes the effectiveness of the detector and is defined as follows:

[0153] 0≤AUC (D,F) ≤1

[0154] AUC ODP The total detection probability of the characterizing detector is defined as

[0155] -1 ≤ AUC ODP = AUC (D,F) + AUC (D,τ) - AUC (F,τ) ≤ 2

[0156] Table 1 gives the 3D-ROC evaluation system index of the small target detection results of the thermal infrared image sequence using ELUM,RIPT,METTR,ASTTV-NTLA,TCNN-NPSTT,TSPK,SRSTT,RCTVW and the method of the present application respectively. The bold and underlined values represent the optimal AUC value and the suboptimal AUC value respectively. Among them, ELUM is a small target detection algorithm based on local contrast measure, RIPT, METTR is a small target detection algorithm based on single-frame low-rank sparse tensor decomposition, ASTTV-NTLA, TCNN-NPSTT, TSPK, SRSTT, RCTVW is a small target detection algorithm based on multi-frame low-rank sparse tensor decomposition.

[0157] ELUM comes from Zhao, Erwei, et al. "Infrared Small Target Detection Using Local Component Uncertainty Measure With Consistency Assessment." IEEE Geoscience and Remote Sensing Letters 19 (2022): 1-5.

[0158] RIPT comes from Dai, Yimian, and Yiquan Wu. "Reweighted infrared patch-tensor model with both nonlocal and local priors for single-frame small target detection." IEEE journal of selected topics in applied earth observations and remote sensing 10.8 (2017): 3752-3767.

[0159] METTR from Cao, Zhaoyang, et al. "Infrared dim target detection via mode-k1k2 extension tensor tubal rank under complex ocean environment." ISPRS Journal of Photogrammetry and Remote Sensing 181 (2021): 167-190.

[0160] ASTTV-NTLA from Liu, Ting, et al. "Nonconvex tensor low-rank approximation for infrared small target detection." IEEE Transactions on Geoscience and Remote Sensing 60 (2021): 1-18.

[0161] TCNN-NPSTT from Wang, Guanghui, et al. "Infrared small target detection using nonoverlapping patch spatial–temporal tensor factorization with capped nuclear norm regularization." IEEE Transactions on Geoscience and Remote Sensing 60 (2021): 1-17.

[0162] TSPK from Pang, Dongdong, et al. "Tensor Spectral k-support Norm Minimization for Detecting Infrared Dim and Small Target against Urban Backgrounds." IEEE Transactions on Geoscience and Remote Sensing (2023).

[0163] SRSTT comes from Li, Jie, et al. "Sparse Regularization-Based Spatial-Temporal Twist Tensor Model for Infrared Small Target Detection." IEEE Transactions on Geoscience and Remote Sensing 61 (2023): 1-17.

[0164] RCTVW comes from Liu, Ting, et al. "Representative Coefficient Total Variation for Efficient Infrared Small Target Detection." IEEE Transactions on Geoscience and Remote Sensing (2023).

[0165] Table 1 Quantitative indicators of detection results of the present method and ELUM, RlPT, METTR, ASTTV-NTLA, TCNN-NPSTT, TSPK, SRSTT, RCTVW on a sequence of thermal infrared image examples

[0166] Method AUC (D,F) ]]> AUC (D,τ) ]]> AUC (F,τ) ]]> AUC TD ]] AUC BS ]]> AUC SNPR ]]> AUC TDBS ]] AUC ODP ]] The present invention 1.000 1.000 2.372e-5 2.000 1.000 4.216e4 1.000 2.000 ELUM 1.000 0.964 8.946e-4 1.964 0.999 1.078e3 0.964 1.964 RIPT 1.000 0.970 3.016e-4 1.970 1.000 3.215e3 0.969 1.969 METTR 1.000 0.985 2.727e-1 1.985 0.727 3.612 0.712 1.712 ASTTV-NTLA 1.000 0.995 5.486e-2 1.995 0.945 1.813e1 0.940 1.940 TCNN-NPSTT 1.000 0.965 6.060e-5 1.965 1.000 1.592e4 0.965 1.965 TSPK 0.940 0.938 1.212e-3 1.878 0.939 7.741e2 0.937 1.877 SRSTT 1.000 1.000 8.383e-5 2.000 1.000 1.192e4 0.999 1.999 RCTVW 0.990 0.989 7.978e-2 1.979 0.910 1.239e1 0.909 1.899

[0167] Figure 7 Fig. 6 shows the detection results of the present method and ELUM, RlPT, METTR, ASTTV-NTLA, TCNN-NPSTT, TSPK, SRSTT, RCTVW on a raw image of a thermal infrared image example frame. From left to right, the first column is the raw image, the second column is the detection result of ELUM, the third column is the detection result of RlPT, the fourth column is the detection result of METTR, the fifth column is the detection result of ASTTV-NTLA, the sixth column is the detection result of TCNN-NPSTT, the seventh column is the detection result of TSPK, the eighth column is the detection result of SRSTT, the ninth column is the detection result of RCTVW, and the tenth column is the detection result of the present method. Figure 7 As can be seen from the qualitative results of Fig. 6, the selected comparison methods can all detect infrared small targets, but have different background suppression capabilities. The detection results of RlPT, METTR, and TCNN-NPSTT have residual clutter, and the background suppression capabilities of RCTVW and ASTTV-NTLA are weak, and the residual background energy in the target detection image is still high. In contrast, the detection method proposed in the present application can effectively suppress the background and enhance the target saliency. As can be seen from the statistical indicator results shown in Table 1, the method proposed in the present application achieves the optimal value on all eight 3D-ROC evaluation indicators. Although the background suppression indicator AUC (F,τ) = 6.060e-5 of TCNN-NPSTT exceeds that of other comparison methods, the detection capability is sacrificed, with AUC (D,τ) = 0.96, AUC TD=1.965; similarly, the target detection capability index of ASTTV-NTLA exceeds other comparative methods, but the background suppression capability is sacrificed, AUC (F,τ) =5.486e-2; and the small target detection method proposed by the present method achieves the optimal value in all indexes. Through comprehensive qualitative analysis and quantitative analysis, the thermal infrared small target detection method proposed by the present application has superior target detection capability, background suppression capability and comprehensive effectiveness.

[0168] The drawings of the embodiments of the present application can make the purpose, technical scheme and advantage of the present application more clear and understandable. It should be noted that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. Any equivalent replacement, improvement, etc. within the method idea and principle provided by the present application should be included in the protection scope of the present application.

Claims

1. A method for small target detection based on iterative single-frame infrared images, characterized in that, Includes the following steps: Step 1): Utilizing the temporal and spatial information carried by the thermal infrared image sequence, construct a three-dimensional spatiotemporal tensor from the current infrared frame image to be detected and its adjacent preceding and following frames. Step 2): Based on the characteristics of the background, target, and noise, establish a model based on low-rank sparse tensor decomposition; Step 3): Define the tensor kernel norm based on kernel information and logarithmic calculation, using the kernel information of the background tensor and the spatial information carried by each frontal slice of the background tensor. Step 3) specifically refers to: For the background tensor The k-th frontal slice The front slice contains R singular values, and the σ of each singular value is calculated. r The corresponding log value is obtained R represents the total number of singular values ​​in the frontal slice, from which the tensor nuclear norm ‖·‖ calculated based on logarithm is defined. LgTNN As shown in formula (4): Where ε1 represents a positive constant, and log(·) is the log operation operator; For background tensor Performing t-SVD operations, there are Two orthogonal tensors are obtained and background tensor kernel tensor n1 represents The width, n2 represents The high, n3 represents thickness, kernel tensor Containing rich spatiotemporal feature information, the kernel tensor Expanded into a kernel matrix according to the time dimension Γ(·) represents the kernel tensor Operators that expand to matrices, where n3 represents the kernel matrix. The height of the matrix High values ​​and The thickness n3 is the same, and min{n1,n2} represents the kernel matrix. The width is the smaller of n1 and n2; thus, the nuclear norm of the kernel matrix calculated based on logarithm is defined as ||·||. LgNN As shown in formula (5): Where ε2 represents a positive constant, Represents the kernel matrix The l-th singular value, l = 1, 2, ..., L, where L represents the kernel matrix. The total number of singular values; Combining formulas (4) and (5), we obtain the tensor nuclear norm ||| based on kernel information and logarithmic calculation. L-ITNN As shown in formula (6): Step 4): In the process of iteratively solving the model based on low-rank sparse tensor decomposition, the saliency information tensor of the temporary target tensor obtained in the current iteration is extracted based on the structural tensor theory and weighted into the tensor data to be decomposed in the next iteration. Step 5): Based on the three-dimensional spacetime tensor of Step 1), and combined with Steps 2) to 4), establish a model based on iterative low-rank sparse tensor decomposition. Step 6): Solve the model based on iterative low-rank sparse tensor decomposition using the ADMM algorithm to obtain the target tensor, and extract the front slice in the middle of the target tensor as the infrared small target detection result of the infrared frame image to be detected, thus realizing the small target detection of a single frame infrared image based on iteration.

2. The method for small target detection based on iterative single-frame infrared images according to claim 1, characterized in that, Step 1) specifically refers to: Using the temporal and spatial information of thermal infrared images, the current infrared frame image to be detected, D t The adjacent frames D t-1 D t-2 ,…,D t-f D t+1 D t+2 ,…,D t+f By stacking 2f+1 consecutive images in chronological order, a three-dimensional spacetime tensor is constructed. The size is n1×n2×n3, where n1 represents The width of the front slice, n2 represents The height of the front slice, n3 represents The thickness is given by n3 = 2f + 1; where D t-1 Indicates that it is located at D t The first frame image before, D t-2 Indicates that it is located at D t The second frame image before, D t-f Indicates that it is located at D t The f-th frame image before, D t+1 Indicates that it is located at D t The first frame image after D t+2 Indicates that it is located at D t The second frame image after D t+f Indicates that it is located at D t The f-th frame image after that.

3. The method for small target detection based on iterative single-frame infrared images according to claim 1, characterized in that, Step 2) specifically refers to: A single-frame infrared image D is modeled as a linear combination of a low-rank background matrix B, a sparse target matrix T, and a noise component matrix N. The corresponding model based on low-rank sparse matrix decomposition is shown in Equation (1): Where rank(·) represents the operator for background rank, and |·|0 represents the operator for estimating the sparsity of the target. Norm, λ1 and λ2 represent the square of the Frobenius norm that measures random noise, and the weighting coefficients. For the three-dimensional spacetime tensor constructed in step 1), It preserves the characteristics of background, target, and noise components, thereby transforming the three-dimensional spatiotemporal tensor. Modeled as a low-rank background tensor Sparse target tensor and noise component tensor A linear combination of these elements is used to establish a model based on low-rank sparse tensor decomposition, as shown in equation (2): Where ||·||1 represents the value used for optimal approximation. norm Norm.

4. The method for small target detection based on iterative single-frame infrared images according to claim 1, characterized in that, Step 4) specifically refers to: The solution process of model (2) based on low-rank sparse tensor decomposition is an iterative process. For the current iteration i, the calculated target tensor is denoted as . Target Tensor Each frontal slice is denoted as For each element (x, y) in the matrix, the corresponding structure tensor matrix is: Where G is the Gaussian kernel function, I x yes Horizontal gradient I y yes vertical gradient The eigenvalues ​​of the structure tensor matrix J for each position element (x, y) are α1 and α2. For pixels in flat regions, α1 ≈ α2 ≈ 0; for pixels in corner regions, α1 ≥ α2 >> 0; and for pixels in edge-line regions, α1 >> α2 ≈ 0. Therefore, the eigenvalues ​​α1 and α2 of the structure tensor J are used to extract... The saliency information W at each position element (x, y) (i) (x,y) is Where ε3 is a positive constant, W (i) (x,y) represents the target tensor in the current iteration i. The saliency information corresponding to the position element (x,y) of the current front slice; From formula (7), we can obtain the target tensor in the current iteration i. Current front slice The corresponding saliency information matrix W (i) For each frontal slice, a corresponding saliency information matrix is ​​extracted and normalized. Therefore, in the current iteration i, saliency information matrices corresponding to n3 frontal slices can be obtained. All saliency information matrices are stacked sequentially to obtain the target tensor. saliency information tensor The process of extracting saliency information tensors is represented as follows: Represents the target tensor Operators for extracting saliency information; saliency information tensor Tensor data to be decomposed in the next weighted iteration round i+1 As shown in formula (8): Where ⊙ represents the Hadamarda product, Representing the use of saliency information tensors The image data to be decomposed in iteration i+1 is obtained through iterative updates. Represents the saliency information tensor The weighted tensor to be decomposed is thus obtained.

5. The method for small target detection in a single frame infrared image based on iteration according to claim 1, characterized in that, Step 5) specifically refers to: To accelerate the model solution speed, a sparsity reweighting strategy is introduced into formula (2), where the target tensor obtained from the current iteration i is used. Calculate the reweighted sparsity of the target in the next iteration round i+1. As shown in formula (9): Where ε4 represents a positive constant, (·) -1 The symbol represents the inverse operation, and |·| represents the absolute value operation. Combining steps 2) to 4), an iterative low-rank sparse tensor decomposition model is established, as shown in formula (10). As shown:

6. The method for small target detection based on iterative single-frame infrared images according to claim 1, characterized in that, Step 6) specifically refers to: The ADMM algorithm is used to solve the iterative low-rank sparse tensor decomposition model (10). To ensure that each variable in (10) is solvable, the following are introduced respectively: Auxiliary variables Formula (11) is obtained as follows: The corresponding augmented Lagrangian function is in, M3 is the Lagrange multiplier, and μ is the penalty term; each variable is solved through alternating iterations, and the subproblems corresponding to each variable are as follows: 1) - Subproblem Fix other parameters in the i-th iteration, variables The iterative formula is Here, argmin represents the parameter value corresponding to the minimum value of the function. In the (i+1)th iteration The equivalent solution formula to formula (13) is: in, This represents the singular value decomposition of the tensor. This represents the orthogonal tensor after singular value decomposition of the tensor. Tensor The transpose of , * denotes the tensor product operator, Tensor The singular value tensor, D (·) (·) is the singular value contraction operator. σ is a singular value. As a soft threshold, diag[·] is used to retrieve the values ​​of the elements on the diagonal of the matrix, τ + =max{τ,0} represents the maximum value operator; 2) - Subproblem Fix other parameters in the i-th iteration, variables The iterative formula is Similar to formula (15), the solution formula for formula (15) is as follows: in, Represents the singular value decomposition of a matrix. σ represents the singular values ​​in the singular value matrix S; 3) - Subproblem Fix other parameters in the i-th iteration, variables The iterative formula is Equation (17) is solved in the matrix domain: in, B (3) , W (i) Represent tensors respectively The corresponding matrix after expansion according to mode-3; Solving formula (18) yields Then there is Here, fold3(·) represents the operator that converts a matrix into a three-dimensional tensor according to mode -3; 4) - Subproblem Fix other parameters in the i-th iteration, variables The iterative formula is Among them, Th (·) (·) is the soft threshold shrinkage operator, Th (τ) (x) = sign(x)·max(|x|-τ,0), where sign(·) is the sign function and τ is the soft threshold; 5) - Subproblem Fix other parameters in the i-th iteration, variables The iterative formula is 6) - Subproblem Depend on get The iterative formula is 7) Lagrange multiplier-subproblem With other parameters fixed in the i-th iteration, the Lagrange multipliers The iterative formula for M3 is: 8) Penalty Item - Subproblem m (i+1) =min(gm (i) ,m max ) (28) Where γ represents the update coefficient, μ max This represents the maximum value of μ; The iteration stopping condition is: when the relative error... The iteration terminates when the error is less than δ; when the target tensor When the norm no longer changes, stop iterating; when adjacent iterations Stop iteration when the time is right; stop iteration when the time is right in an adjacent iteration. Stop iterating when the time comes; Furthermore, extract the target tensor. The central front slice is used as the infrared frame image D to be detected. t The corresponding infrared small target detection result image T t This enables small target detection based on iterative single-frame infrared images.

7. A device for detecting small targets in a single frame infrared image based on an iterative method according to claim 1, characterized in that, include: The three-dimensional space-time tensor construction module constructs a three-dimensional space-time tensor from the current infrared frame image to be detected and its adjacent previous and next frame images, laying the data foundation for subsequent decomposition. The low-rank sparse tensor decomposition model building module establishes a model based on low-rank sparse tensor decomposition by taking into account the characteristics of the background, target, and noise. The low-rank background estimation norm design module combines the kernel information of the background tensor with the spatial information carried by each frontal slice of the background tensor to define the tensor kernel norm based on kernel information and logarithmic calculation. The saliency information extraction module, based on structural tensor theory, designs a saliency information tensor extraction method to extract saliency information tensors from the temporary target tensor obtained in the current iteration during the iterative solution process, and weights them into the image data to be decomposed in the next iteration. The module for establishing an iterative low-rank sparse tensor decomposition model is based on three-dimensional spacetime tensor and low-rank sparse tensor decomposition model, combined with low-rank background estimation norm and low-rank background estimation norm, to establish an iterative low-rank sparse tensor decomposition model. The optimization model module solves the iterative low-rank sparse tensor decomposition model using the ADMM algorithm to obtain the target tensor, and extracts the front slice in the middle of the target tensor as the infrared small target detection result of the infrared frame image to be detected. The target detection result output module is used to output the infrared small target detection result map of a single frame image.

Images