An infrared small target detection method and device based on five-dimensional space-time domain

Abstract

The application discloses an infrared small target detection method and device based on five-dimensional space-time domain. The method comprises the following steps: (1) for an infrared image sequence, a five-dimensional space-time block tensor is constructed according to a given rule; (2) data whitening is performed on the five-dimensional space-time block tensor to obtain a five-dimensional whitened space-time block tensor; (3) a five-dimensional kernel norm of an estimated background tensor is defined, and a five-dimensional sparsity reweighting strategy is introduced to estimate a sparse target tensor; (4) a five-dimensional tensor completion model based on low-rank sparsity is established; (5) the five-dimensional tensor completion model based on low-rank sparsity is solved based on an ADMM algorithm; and (6) a target tensor output by the model is reconstructed into a sequence T with the same size as the input infrared image sequence, and the sequence T is taken as an infrared small target detection result, so that the infrared small target detection based on the five-dimensional space-time domain is realized. The low-rank sparse tensor completion model based on space-time information and the solving algorithm can effectively improve the detection performance of the infrared small target.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a method and apparatus for detecting small infrared targets based on five-dimensional spatiotemporal domain. Background Technology

[0002] In recent years, numerous infrared small target detection methods have emerged, including traditional detection methods and deep learning-based methods. Traditional detection methods can be broadly categorized into three types: methods based on the background consistency assumption, methods based on the human visual system (HVS), and methods based on component analysis. Methods based on the background consistency assumption typically involve designing different filters to separate the target from the background; however, in complex dynamic scenes, these filters often produce false alarms. Inspired by the human visual system, HVS-based methods aim to design different local contrast descriptors to achieve significant target enhancement and background suppression. However, these contrast descriptors are highly sensitive to salient structures and interference, easily misdetecting bright clutter structures as targets in complex scenes, resulting in false alarms. The basic theory of component analysis-based methods is low-rank sparse decomposition. Based on the data structure constructed from infrared image sequences, these methods can be divided into methods based on single-frame matrices, single-frame tensors, multi-frame matrices, multi-frame three-dimensional tensors, and multi-frame four-dimensional tensors. How to correctly describe the characteristics of each component and accurately measure them is crucial for achieving accurate separation of the target, background, and noise. This is also the challenge currently faced by component analysis-based methods.

[0003] Currently, deep learning-based methods for infrared small target detection are attracting increasing attention. Numerous researchers have proposed various network models, including Moderately Dense Adaptive Feature Fusion Network (MDAFNet), Dense Nested Attention Network, Sliced ​​Spatiotemporal Network (SSTNet), Multiscale Progressive Fusion Filter Network, Spatiotemporal Transformer (ST-Trans), and Attention-Guided Pyramid Context Network (AGPCNet). However, as data-driven methods, most deep learning-based approaches suffer from limitations such as high data requirements, insufficient generalization ability, and poor network interpretability.

[0004] In real-world scenarios, the task of detecting small infrared targets is often performed in complex and dynamically changing environments such as sea, air, and ground. Existing algorithms are often ill-suited to achieving efficient thermal infrared small target detection in complex scenarios. Traditional methods struggle to accurately capture the dynamic spatiotemporal information carried in sequences, while deep learning methods, limited by their generalization capabilities, also lose their superior performance when faced with target detection tasks in complex scenarios.

[0005] In conclusion, considering the significant application value of infrared small target detection technology in military, security, aerospace and other fields, further in-depth research is still needed to improve its detection accuracy and robustness to meet practical application requirements. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the present invention aims to provide a method and apparatus for infrared small target detection based on five-dimensional spatiotemporal domain. This method constructs a five-dimensional spatiotemporal block tensor from a thermal infrared image sequence, fully utilizing and expressing the local and global spatiotemporal information of the infrared image sequence. Furthermore, a data whitening operation removes second-order statistics that characterize Gaussian noise and part of the background, which affect subsequent low-rank sparse analysis. This lays a solid data foundation for subsequent component analysis. Different constraints are designed for different components: for the five-dimensional background tensor... A five-dimensional nuclear norm for the background is proposed; for sparse targets, a five-dimensional sparsity reweighting strategy is introduced to accelerate sparse target estimation; thereby, a five-dimensional tensor completion model based on low-rank sparsity and an efficient optimization solution algorithm are established; through examples, the performance of the infrared small target detection method based on the five-dimensional spatiotemporal domain is comprehensively verified, improving the target detection capability, background suppression capability and overall performance of infrared small target detection.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] This invention discloses a method for detecting small infrared targets based on five-dimensional spatiotemporal domain, characterized by comprising the following steps:

[0009] Step 1): Using the original thermal infrared image sequence D, in the t-th frame image D... t The window of fixed size is slidable with a fixed sliding step size, and the s-th local image patch D is obtained. t,s Constructed as a three-dimensional block tensor Stack all the obtained 3D block tensors sequentially along the fourth dimension to achieve the effect of transforming the image D... t Constructed as a four-dimensional block tensor Each frame of image is sequentially constructed as a four-dimensional spatial block tensor, and the time dimension is taken as the fifth dimension. The four-dimensional spatial block tensors corresponding to each frame of image are stacked sequentially in the fifth dimension to obtain the five-dimensional spatiotemporal block tensor corresponding to the original thermal infrared image sequence D.

[0010] Step 2): Apply the five-dimensional spacetime block tensor obtained in Step 1) Whitening is performed to remove second-order statistics that interfere with target detection, resulting in a five-dimensional whitened spatiotemporal block tensor.

[0011] Step 3): Define the estimated background tensor The five-dimensional nuclear norm is calculated, and the target tensor is estimated. To improve the sparsity of sparse target tensors, a five-dimensional sparsity reweighting strategy is introduced to accelerate the estimation of sparse target tensors.

[0012] Step 4): Transform the infrared small target detection task into a component decomposition problem based on the five-dimensional space-time domain. From Step 1) to Step 3), establish a five-dimensional tensor completion model based on low-rank sparsity.

[0013] Step 5): Design and optimize the solution framework to solve the five-dimensional tensor completion model based on low-rank sparsity, and obtain the target tensor.

[0014] Step 6): Solve the target tensor obtained in Step 5). The sequence is reconstructed into a sequence T with the same size as the input image sequence D, which is used as the infrared small target detection result, thus realizing infrared small target detection based on the five-dimensional spatiotemporal domain.

[0015] The present invention also discloses a five-dimensional spatiotemporal infrared small target detection device for implementing the method, comprising:

[0016] The five-dimensional spacetime block tensor construction module constructs a five-dimensional spacetime block tensor from the infrared image sequence according to a given rule.

[0017] The five-dimensional spacetime block tensor whitening module suppresses the second-order statistics of the five-dimensional spacetime block tensor and achieves five-dimensional tensor data whitening.

[0018] The low-rank background estimation module defines the estimated background tensor. The five-dimensional nuclear norm;

[0019] The sparse target tensor estimation module performs low-rank estimation of sparse targets and introduces a five-dimensional sparsity reweighting strategy to accelerate sparse target estimation.

[0020] The tensor completion model building module is based on the five-dimensional white block space-time block tensor. It uses a low-rank background estimation strategy and a sparse target tensor estimation strategy to build a five-dimensional tensor completion model based on low-rank sparsity.

[0021] The model optimization and solution module designs an optimization and solution framework based on the ADMM algorithm to solve the five-dimensional tensor completion model, obtain the target tensor, and reconstruct it into a sequence of infrared small target detection results.

[0022] The target detection result output module is used to output the infrared small target detection result map for each frame of infrared image.

[0023] Compared with the prior art, the beneficial effects of the present invention include:

[0024] 1) This invention innovatively proposes a tensor construction method that constructs the infrared image frames to be detected as four-dimensional spatial block tensors, and then sequentially stacks the four-dimensional spatial block tensors corresponding to each frame of infrared image in the time dimension, thereby constructing a five-dimensional spatiotemporal block tensor for the thermal infrared image sequence. This tensor construction method, through the combination of different dimensions, fully expresses the local spatial information, local spatiotemporal information, and global information of the infrared image sequence, which is beneficial for the characterization and extraction of background, target, and other components. On this basis, a data whitening operation is performed on the five-dimensional spatiotemporal block tensor to remove second-order noise that can characterize Gaussian noise and part of the background. Statistics improve data quality and lay a solid data foundation for subsequent component analysis. In addition, unlike existing two-dimensional, three-dimensional and four-dimensional tensor structures, the five-dimensional tensor constructed in this invention is the optimal structure, with excellent dimensional combination, optimal information synthesis and optimal structure. Constructing tensors with higher dimensions loses its meaning. As the tensor dimension increases, the amount of information that needs to be stored and processed in each dimension also increases, which leads to a significant increase in computational complexity. Moreover, as the tensor dimension increases, the interpretation and understanding of the information carried by the tensor also becomes more difficult.

[0025] 2) For the five-dimensional background tensor A five-dimensional nuclear norm for the background is proposed. For sparse targets, a five-dimensional sparsity reweighting strategy is introduced to accelerate sparse target estimation. Based on this, a five-dimensional tensor completion model based on low-rank sparse decomposition is established. Especially in complex and dynamically changing scenes, this tensor completion model based on low-rank sparse decomposition demonstrates excellent component feature representation capabilities. Furthermore, an efficient optimization algorithm based on ADMM is designed, which can more accurately extract target components, obtain small target detection results for the infrared image sequence to be detected, and improve the target detection capability, background suppression capability, and overall performance of infrared small target detection. Attached Figure Description

[0026] Figure 1 This is a schematic diagram illustrating the process of constructing a frame of thermal infrared image into a four-dimensional spatial block tensor in this invention.

[0027] Figure 2This is a schematic diagram illustrating the process of constructing a thermal infrared image sequence into a five-dimensional spacetime block tensor in this invention.

[0028] Figure 3 This is a schematic diagram of the structure of the infrared small target detection device based on the five-dimensional space-time domain in this invention;

[0029] Figure 4 Example frame images of a thermal infrared image sequence used for experimental testing;

[0030] Figure 5 The thermal infrared small target detection results are obtained by applying the method proposed in this invention to an example frame of a thermal infrared image sequence.

[0031] Figure 6 The image shows the original image of a thermal infrared image example frame and the thermal infrared small target detection results obtained by the methods of this invention, GSWLCM, FAMSIS, IPCE, RCTVW, STT-TRNR, TSPK, SSTTC and AGPCNet. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention is described in detail below with reference to specific embodiments. Specific embodiments are described below to simplify the invention. However, it should be understood that the invention is not limited to the described embodiments, and various modifications are possible without departing from the basic principles; these equivalent forms also fall within the scope defined by the appended claims.

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0034] The basic steps of the five-dimensional spatiotemporal infrared small target detection method of the present invention in this embodiment mainly include:

[0035] Step 1: Using the original thermal infrared image sequence D, in the t-th frame image D t The window of fixed size is slidable with a fixed sliding step size, and the s-th local image patch D is obtained. t,s Constructed as a three-dimensional block tensor Stack all the obtained 3D block tensors sequentially along the fourth dimension to achieve the effect of transforming the image D... t Constructed as a four-dimensional block tensor Each frame of image is sequentially constructed as a four-dimensional spatial block tensor, and the time dimension is taken as the fifth dimension. The four-dimensional spatial block tensors corresponding to each frame of image are stacked sequentially in the fifth dimension to obtain the five-dimensional spatiotemporal block tensor corresponding to the original thermal infrared image sequence D.

[0036] Specifically, such as Figure 1 As shown, using the original infrared image sequence D, in the t-th frame image D... t A fixed-size window slides with a fixed sliding step size; image D t The size is n1×n2, the sliding step is 3ps, the size of the sliding window is 3ps×3ps, and the number of image blocks obtained is I4; the obtained s-th image block D t,s The image is further divided into 3×3 sub-image blocks, and these 9 sub-image blocks are stacked sequentially along the third dimension to construct a three-dimensional block tensor. The dimensions are I1×I2×I3, where I1 and I2 have the same values ​​as ps, and I3 has the value of 9. I4 image blocks can be used to construct I4 three-dimensional block tensors. These I4 three-dimensional block tensors are then stacked sequentially along the fourth dimension to achieve the transformation of image D... t Constructed as a four-dimensional block tensor

[0037] like Figure 2 As shown, each frame of image is constructed as a four-dimensional spatial block tensor, and the time dimension is taken as the fifth dimension. The four-dimensional spatial block tensors corresponding to each frame of infrared image are stacked sequentially in the fifth dimension to obtain the five-dimensional spatiotemporal block tensor corresponding to the original thermal infrared image sequence D. The dimensions are I1×I2×I3×I4×I5, where the value of I5 is the same as the number of image frames in the infrared image sequence D.

[0038] In the specific implementation, I5 is set to 7, ps is set to 20, and the constructed five-dimensional spacetime block tensor The dimensions are 20×20×9×25×7.

[0039] Step 2: Process the five-dimensional spacetime block tensor obtained in Step 1 Whitening is performed to remove second-order statistics that interfere with target detection, resulting in a five-dimensional whitened spatiotemporal block tensor.

[0040] Specifically, the five-dimensional spacetime block tensor obtained in step 1... Perform data whitening: First, along the time dimension... Expand into a two-dimensional matrix Represents a two-dimensional matrix D seqThe height is N, which is the same as I5, and the width is M, which is the same as the product of I1, I2, I3, and I4; calculate D. seq The covariance matrix C, the eigenvalue matrix and eigenvector matrix of the covariance matrix C are ∑ and V respectively; for D seq Perform whitening based on principal component analysis; the corresponding expression is:

[0041]

[0042] Where "*" represents the matrix multiplication operation, D pcaw D represents seq Principal component analysis whitening matrix, ε represents a positive constant to avoid zero denominator, V T The transpose of the eigenvector matrix;

[0043] In the specific embodiment, ε is set to 0.000001;

[0044] The whitening matrix D of principal component analysis pcaw By mapping the zero-phase component analysis whitening operation onto the original feature space, we obtain the zero-phase component analysis whitening matrix D. zcaw The corresponding expression is

[0045]

[0046] D zcaw Reconstructed as a five-dimensional spacetime block tensor Scale-consistent five-dimensional whitened spacetime block tensor

[0047] Step 3: Define the estimated background tensor The five-dimensional nuclear norm is calculated, and the target tensor is estimated. To improve the sparsity of sparse target tensors, a five-dimensional sparsity reweighting strategy is introduced to accelerate the estimation of sparse target tensors.

[0048] Specifically, the five-dimensional whitened spacetime block tensor constructed in step 2 The second-order statistics containing Gaussian noise and part of the background that interfere with target detection are removed, and the low-rank characteristics of the background component, the sparsity of the target component, and the characteristics of the noise component are enhanced.

[0049] For low-rank five-dimensional background tensor Design to estimate the five-dimensional background tensor The five-dimensional tensor kernel norm As shown in formula (3):

[0050]

[0051] in, Represents the 3D block background tensor tensor kernel norm, Represents the five-dimensional background tensor Image D in frame t t The s-th image patch D t,s The corresponding three-dimensional block tensor, and 1≤s≤I4, 1≤t≤I5;

[0052] The sparsity of the sparse objective is estimated using the l0 norm, and the l1 norm is used to replace the l0 norm for further estimation. The expression is as follows: Introducing a five-dimensional sparsity reweighting strategy to accelerate the estimation of sparse target tensors Five-dimensional reweighted weight tensor in each iteration The target tensor obtained from the previous iteration Determined; in the (i+1)th iteration, the five-dimensional reweighted weight tensor of the target. As shown in formula (4):

[0053]

[0054] Where ε0 represents a positive constant, (·) -1 The operation represents calculating the reciprocal of an element, and |·| represents calculating the absolute value.

[0055] In the specific embodiment, ε0 is set to 0.01.

[0056] Step 4: Transform the infrared small target detection task into a component decomposition problem based on the five-dimensional space-time domain. From Step 1) to Step 3), establish a five-dimensional tensor completion model based on low-rank sparsity.

[0057] Specifically, in the five-dimensional whitened spacetime block tensor The second-order statistics that represent Gaussian noise and part of the background have been removed. Therefore, the infrared small target detection task is transformed into a component decomposition problem based on the five-dimensional spatiotemporal domain, and the five-dimensional whitened spatiotemporal block tensor is... Modeled as a low-rank background tensor and sparse target tensor A linear combination of the five-dimensional tensor nuclear norm from step 3). Sparse estimation of the target and the five-dimensional reweighted weight tensor A five-dimensional tensor completion model based on low-rank sparsity is established, as shown in Equation (5):

[0058]

[0059] Where, ‖·‖1 represents the l1 norm of the optimal approximation l0 norm, rank(·) represents the rank estimator, λ represents the positive coordination parameter, and ⊙ represents the Hadamard product.

[0060] Step 5: Design and optimize the solution framework to solve the five-dimensional tensor completion model based on low-rank sparsity, and obtain the target tensor.

[0061] Specifically, the ADMM algorithm is used to solve the five-dimensional tensor completion model based on low-rank sparsity (5). To ensure that each variable to be solved in (5) is solvable, the following is introduced: Auxiliary variables Formula (6) is obtained as follows:

[0062]

[0063] The corresponding augmented Lagrangian function is

[0064]

[0065] in, Let μ be a Lagrange multiplier and μ be a penalty term; each variable is solved through alternating iterations, and the subproblems corresponding to each variable are as follows:

[0066] 1) The iterative formula is:

[0067] Will The update iteration is decomposed into sequential updates subtensor And there are express Given a three-dimensional subtensor with the fourth dimension value s and the fifth dimension value t, then we have: The iterative formula is:

[0068]

[0069] Where i represents the iteration number. This represents the result obtained in the (i+1)th iteration. In the i-th iteration The fourth dimension is s, and the fifth dimension is the subtensor corresponding to t, that is... And we have 1≤s≤I4, 1≤t≤I5; This represents the singular value decomposition of the tensor. For singular value tensors, and Let be the unitary tensor obtained after singular value decomposition of the tensor, (·). T For the transpose operation of a tensor, Th(·) (·) denotes the singular value contraction operator, and has

[0070] 2) The iterative formula is:

[0071]

[0072] in, This represents the result obtained in the (i+1)th iteration.

[0073] 3) The iterative formula is:

[0074]

[0075] in, This represents the result obtained in the (i+1)th iteration.

[0076] 4) The iterative formula is:

[0077]

[0078] in, This represents the result obtained in the (i+1)th iteration.

[0079] 5) Lagrange multipliers The iterative formula is:

[0080]

[0081]

[0082] in, They represent the results obtained in the (i+1)th iteration. And we have 1≤s≤I4, 1≤t≤I5;

[0083] 6) The iterative formula for the penalty term μ is:

[0084]

[0085] Where, μ (i+1) μ represents the value obtained in the (i+1)th iteration, γ represents the iteration update coefficient, and μ max This represents the maximum value of μ;

[0086] The iteration stopping condition of formula (7) is: 1) when the relative error The iteration terminates when the error is less than δ; 2) When the five-dimensional target tensor in two adjacent iterations... 3) Stop iterating when the l0 norm no longer changes;

[0087] This completes the iterative solution of the five-dimensional tensor completion model (5) based on low-rank sparsity in step 4), and yields the five-dimensional target tensor.

[0088] Specifically, in the embodiments... λ L =15, μ max =100, γ=1.2, μ (0) =0.008,

[0089] Step 6: Solve the target tensor obtained in Step 5 The sequence is reconstructed into a sequence T with the same size as the input image sequence D, which is used as the infrared small target detection result, thus realizing infrared small target detection based on the five-dimensional spatiotemporal domain.

[0090] Specifically, the target tensor obtained in step 5) The sequence is reconstructed into a sequence T with the same size as the input image sequence D. Sequence T contains I5 target detection images T, with the size of each image being n1×n2. Sequence T serves as the infrared small target detection result of the original infrared thermal infrared image sequence, realizing infrared small target detection based on the five-dimensional spatiotemporal domain.

[0091] Corresponding to the aforementioned embodiment of an infrared small target detection method based on five-dimensional space-time domain, the present invention also provides an embodiment of an infrared small target detection device based on five-dimensional space-time domain.

[0092] Figure 3 This is a block diagram illustrating an infrared small target detection device based on a five-dimensional spatiotemporal domain, according to an exemplary embodiment. Figure 3 As shown, the device includes:

[0093] The five-dimensional spacetime block tensor construction module constructs a five-dimensional spacetime block tensor from the infrared image sequence according to a given rule.

[0094] The five-dimensional spacetime block tensor whitening module suppresses the second-order statistics of the five-dimensional spacetime block tensor and achieves five-dimensional tensor data whitening.

[0095] The low-rank background estimation module defines the estimated background tensor. The five-dimensional nuclear norm;

[0096] The sparse target tensor estimation module performs low-rank estimation of sparse targets and introduces a five-dimensional sparsity reweighting strategy to accelerate sparse target estimation.

[0097] The tensor completion model building module is based on the five-dimensional white block space-time block tensor. It uses a low-rank background estimation strategy and a sparse target tensor estimation strategy to build a five-dimensional tensor completion model based on low-rank sparsity.

[0098] The model optimization and solution module designs an optimization and solution framework based on the ADMM algorithm to solve the five-dimensional tensor completion model, obtain the target tensor, and reconstruct it into a sequence of infrared small target detection results.

[0099] The target detection result output module is used to output the infrared small target detection result map for each frame of infrared image.

[0100] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0101] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the description of the method embodiments. The device embodiments described above are merely illustrative; the various modules in the device represent a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another unit. Furthermore, the connections between the displayed or discussed modules may be communication connections through some interfaces, which may be electrical or other forms. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort. The following uses a publicly available real thermal infrared image sequence as an example to illustrate specific implementation methods to demonstrate the technical effects of the present invention; specific steps in the embodiments will not be repeated.

[0102] Example

[0103] The accompanying drawings illustrating the embodiments of the present invention will make the objectives, technical solutions, and advantages of the present invention clearer. It should be noted that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention. All equivalent substitutions and improvements made within the framework of the methods and principles provided by the present invention should be included within the scope of protection of the present invention.

[0104] In this embodiment, the effectiveness of the thermal infrared small target detection algorithm will be verified using a publicly available infrared image sequence. The publicly available infrared image sequence is described as follows:

[0105] The publicly released thermal infrared image sequence contains 100 real thermal infrared images, each with a width and height of 256 pixels. These images were taken in a complex and dynamic scene, mainly consisting of dense forest scenes and bright linear structures. The background contains a large amount of strong clutter and noise, which increases the difficulty of image analysis. The bright cloud texture occupies an important position in the image, adding more detail to the air-to-ground background. Among them, a small aircraft slowly traverses the forest and clouds, and its movement trajectory above the forest is clearly visible, becoming a small target in the whole scene. Figure 4 Here is an example frame from this infrared image sequence. Figure 5 The image shows the target detection results obtained by applying the five-dimensional spatiotemporal infrared small target detection method of this invention to an example frame of an infrared image sequence. The detection results visually demonstrate that complex backgrounds and noise in the image are well suppressed, and the detected small targets are significantly enhanced. To more accurately and objectively evaluate the effectiveness of the proposed five-dimensional spatiotemporal infrared small target detection method, from a qualitative perspective, the visual target detection results of the example frame of this infrared image sequence are used for comparative analysis and evaluation; from a quantitative perspective, the detection performance of the algorithm is evaluated using the 3D-ROC evaluation index system, which was first proposed by Chein-I Chang, originating from Chang, Chein-I. "An effective evaluation tool for hyperspectral target detection: 3D receiver operating characteristic curve analysis." IEEE Transactions on Geoscience and Remote Sensing 59.6(2020):5131-5153.

[0106] Table 1 presents the 3D-ROC evaluation results of small target detection using GSWLCM, FAMSIS, IPCE, RCTVW, STT-TRNR, TSPK, SSTTC, AGPCNet, and the method of this invention for the thermal infrared image sequence. The bold and underlined values ​​represent the corresponding optimal and suboptimal AUC values, respectively.

[0107] GSWLCM is from Qiu, Zhaobing, et al. "Global sparsity-weighted local contrast measure for infrared small target detection." IEEE Geoscience and Remote Sensing Letters 19(2022):1-5.

[0108] FAMSIS is from Chen, Yaohong, et al. "Small infrared target detection based on fast adaptive masking and scaling with iterative segmentation." IEEE Geoscience and Remote Sensing Letters 19(2021):1-5.

[0109] IPCE is from Zhang, Chunmin, et al. "Infrared small target detection via interpatch correlation enhancement and joint local visual saliency prior." IEEE Transactions on Geoscience and Remote Sensing 60(2021):1-14.

[0110] RCTVW is from Liu, Ting, et al. "Representative Coefficient Total Variation for Efficient Infrared Small Target Detection." IEEE Transactions on Geoscience and Remote Sensing(2023).

[0111] STT-TRNR is from Yi, Haiyang, et al. "Spatial-Temporal Tensor Ring Norm Regularization for Infrared Small Target Detection." IEEE Geoscience and Remote Sensing Letters 20(2023):1-5.

[0112] TSPK comes from Pang, Dongdong, et al. "Tensor Spectral k-support NormMinimization for Detecting Infrared Dim and Small Target against UrbanBackgrounds." IEEE Transactions on Geoscience and Remote Sensing (2023).

[0113] SSTTC comes from Xia, Chaoqun, et al. "Separable Spatial-Temporal Patch-TensorPair Completion for Infrared Small Target Detection." IEEE Transactions onGeoscience and Remote Sensing (2024).

[0114] AGPCNet comes from Zhang, Tianfang, et al. "AGPCNet: Attention-guided pyramidcontext networks for infrared small target detection." arXiv preprint arXiv:2111.03580(2021).

[0115] Figure 6 This image shows the original frame of a thermal infrared image example and the thermal infrared small target detection results obtained using the methods of this invention, GSWLCM, FAMSIS, IPCE, RCTVW, STT-TRNR, TSPK, SSTTC, and AGPCNet. Figure 6Qualitative results show that the background suppression capability of AGPCNet, based on deep learning, is too excellent, leading to target loss. Traditional infrared small target detection methods such as GSWLCM, FAMSIS, IPCE, and STT-TRNR all exhibit target loss, and highlighted non-target structures negatively impact GSWLCM, FAMSIS, and IPCE, causing false alarms. Furthermore, RCTVW, TSPK, and SSTTC detection results retain significant clutter. Conversely, the five-dimensional spatiotemporal infrared small target detection method proposed in this invention effectively suppresses background and noise, detecting small targets. According to the 3D-ROC evaluation metrics in Table 1, AGPCNet performs the worst overall, followed by IPCE and FAMSIS. Although RCTVW's background suppression capability surpasses that of the proposed method, its target detection capability does not meet the requirements of practical applications. Overall, the proposed method outperforms all compared methods in all eight 3D-ROC evaluation metrics, achieving optimal or near-optimal values. Based on a combination of qualitative and quantitative analysis, the thermal infrared small target detection method proposed in this invention exhibits superior target detection capability, background suppression capability, and overall effectiveness in complex and dynamically changing scenarios.

[0116] Table 1. Quantitative indicators of the detection results of thermal infrared image example sequences using GSWLCM, FAMSIS, IPCE, RCTVW, STT-TRNR, TSPK, SSTTC, AGPCNet, and the method of this invention.

[0117]

[0118] The accompanying drawings illustrating the embodiments of the present invention will make the objectives, technical solutions, and advantages of the present invention clearer. It should be noted that the specific embodiments described herein are merely illustrative and not intended to limit the scope of the invention. All equivalent substitutions and improvements made within the framework of the methods and principles provided by the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting small infrared targets based on five-dimensional spatiotemporal domain, characterized in that, Includes the following steps: Step 1): Using the original thermal infrared image sequence D, in the t-th frame image... The window of a fixed size is slidable with a fixed sliding step size, and the obtained first... Local image patch Constructed as a three-dimensional block tensor All the obtained 3D block tensors are stacked sequentially along the fourth dimension to achieve image... Constructed as a four-dimensional block tensor ; Each frame of image is sequentially constructed as a four-dimensional spatial block tensor, and the time dimension is taken as the fifth dimension. The four-dimensional spatial block tensors corresponding to each frame of image are stacked sequentially in the fifth dimension to obtain the five-dimensional spatiotemporal block tensor corresponding to the original thermal infrared image sequence D. ; Step 1) Specifically: Using the original infrared image sequence D, in the t-th frame image A fixed-size window slides with a fixed sliding step size, and the image... The size is The sliding step size is 3ps, the sliding window size is 3ps × 3ps, and the number of image patches obtained is The obtained first Image blocks Further divided into Nine sub-image blocks are stacked sequentially along the third dimension to construct a three-dimensional block tensor. The size is ,in, and The value is the same as in ps. The value is 9; The image blocks are constructed to obtain A three-dimensional block tensor, the obtained The three-dimensional block tensors are stacked sequentially along the fourth dimension to realize the image Constructed as a four-dimensional block tensor ; Each frame of image is constructed as a four-dimensional spatial block tensor, and the time dimension is taken as the fifth dimension. The four-dimensional spatial block tensors corresponding to each frame of infrared image are stacked sequentially in the fifth dimension to obtain the five-dimensional spatiotemporal block tensor corresponding to the original thermal infrared image sequence D. , The size is ,in, The value is the same as the number of image frames in infrared image sequence D; Step 2): The five-dimensional spacetime block tensor obtained in Step 1). Whitening is performed to remove second-order statistics that interfere with target detection, resulting in a five-dimensional whitened spatiotemporal block tensor. ; Step 3): Define the estimated background tensor The five-dimensional nuclear norm is calculated, and the target tensor is estimated. To improve the sparsity of sparse target tensors, a five-dimensional sparsity reweighting strategy is introduced to accelerate the estimation of sparse target tensors. Step 4): Transform the infrared small target detection task into a component decomposition problem based on the five-dimensional space-time domain. From Step 1) to Step 3), establish a five-dimensional tensor completion model based on low-rank sparsity. Step 5): Design and optimize the solution framework to solve the five-dimensional tensor completion model based on low-rank sparsity, and obtain the target tensor. ; Step 6): Solve the target tensor obtained in Step 5). The sequence is reconstructed into a sequence T with the same size as the input image sequence D, which is used as the infrared small target detection result, thus realizing infrared small target detection based on the five-dimensional spatiotemporal domain.

2. The infrared small target detection method based on five-dimensional spatiotemporal domain according to claim 1, characterized in that, Step 2) specifically refers to: The five-dimensional spacetime block tensor obtained in step 1) Perform data whitening: First, along the time dimension... Expand into a two-dimensional matrix , Representing a two-dimensional matrix The height, numerically related to same, Width, numerically related to The product results are the same; calculate covariance matrix covariance matrix The eigenvalue matrix and eigenvector matrix are respectively and ;right Perform whitening based on principal component analysis; the corresponding expression is: （1） in," " indicates the matrix multiplication operation. express Principal component analysis whitening matrix, To represent a positive integer, The transpose of the eigenvector matrix; Whitening matrix of principal component analysis The zero-phase component analysis whitening matrix is ​​obtained by mapping the whitening operation onto the original feature space. The corresponding expression is （2） in, Represents the eigenvector matrix; Will Reconstructed as a five-dimensional spacetime block tensor Scale-consistent five-dimensional whitened spacetime block tensor .

3. The infrared small target detection method based on five-dimensional spatiotemporal domain according to claim 1, characterized in that, Step 3) specifically refers to: The five-dimensional whitened spacetime block tensor constructed in step 2) It removes the second-order statistics containing Gaussian noise and part of the background that interfere with target detection, and enhances the low-rank characteristics of the background component, the sparsity characteristics of the target component, and the characteristics of the noise component. For low-rank five-dimensional background tensor Design to estimate the five-dimensional background tensor The five-dimensional tensor kernel norm As shown in formula (3): （3） in, Represents the 3D block background tensor tensor kernel norm, Represents the five-dimensional background tensor Middle Frame Image The Image blocks The corresponding 3D block background tensor, and has , ; use Norm estimates the sparsity of sparse targets, and uses Norm substitution Norms, used to estimate sparse targets, are expressed as follows: A five-dimensional sparsity reweighting strategy is introduced to accelerate the estimation of sparse target tensors. The five-dimensional reweighted weight tensor in each iteration process The target tensor obtained from the previous iteration Confirmed; the first In each iteration, the target's five-dimensional reweighted weight tensor As shown in formula (4): （4） in, Represents positive numbers. This represents the operation of calculating the reciprocal of an element. This indicates the operation of calculating the absolute value.

4. The infrared small target detection method based on five-dimensional spatiotemporal domain according to claim 2, characterized in that, Step 4) specifically refers to: Five-dimensional whitened spacetime block tensor The second-order statistics that represent Gaussian noise and part of the background have been removed. Therefore, the infrared small target detection task is transformed into a component decomposition problem based on the five-dimensional spatiotemporal domain, and the five-dimensional whitened spatiotemporal block tensor is... Modeled as a low-rank background tensor and sparse target tensor A linear combination of the five-dimensional tensor nuclear norm from step 3). sparse estimation of the target and the five-dimensional reweighted weight tensor A five-dimensional tensor completion model based on low-rank sparsity is established, as shown in formula (5): （5） in, Describing the optimal approximation norm Norm, This represents the rank estimation operator. Indicates a positive coordination parameter. It represents the Hadamardi (or Hadama) stack.

5. The infrared small target detection method based on five-dimensional spatiotemporal domain according to claim 4, characterized in that, Step 5) specifically refers to: The ADMM algorithm is used to solve the five-dimensional tensor completion model based on low-rank sparsity (5). To ensure that each variable to be solved in (5) is solvable, the following is introduced: Auxiliary variables Formula (6) is obtained as follows: （6） The corresponding augmented Lagrangian function is （7） in, For Lagrange multipliers, This is a penalty term; each variable is solved through alternating iterations, and the subproblems corresponding to each variable are as follows: 1) The iterative process is as follows: Will The update iteration is decomposed into sequential updates subtensor And there are ,express The value of the fourth dimension is The fifth dimension value is The three-dimensional subtensor then has The iterative formula is: （8） in, Indicates the number of iterations. Indicates the first Round iterations obtained , Indicates the first In round of iteration The fourth dimension is The fifth dimension is The corresponding subtensor, i.e. And there are , ; This represents the singular value decomposition of the tensor. For singular value tensors, and The unitary tensor obtained after singular value decomposition of the tensor. For the transpose operation of a tensor, To represent the singular value contraction operator, we have: , ; 2) The iterative formula is （9） in, Indicates the first Round iterations obtained ; 3) The iterative formula is （10） in, Indicates the first Round iterations obtained ; 4) The iterative formula is （11） in, Indicates the first Round iterations obtained ; 5) Lagrange multipliers The iterative formula is （12） （13） in, They represent the first Round iterations obtained And there are , ; 6) Penalty Items The iterative formula is （20） in, Indicates the first Round iterations obtained , Indicates the iterative update coefficients. express The maximum value; The iteration stopping condition of formula (7) is: 1) when the relative error Less than the error When the iteration terminates; 2) when the five-dimensional target tensor in two adjacent iterations of 3) Stop iteration when the norm no longer changes; This completes the iterative solution of the five-dimensional tensor completion model (5) based on low-rank sparsity in step 4), and yields the five-dimensional target tensor. .

6. The infrared small target detection method based on five-dimensional spatiotemporal domain according to claim 1, characterized in that, Step 6) specifically refers to: The target tensor obtained in step 5) Reconstructed into a sequence T of the same size as the input image sequence D, the sequence T contains Zhang target detection image The image size is The sequence T is used as the infrared small target detection result of the original infrared thermal infrared image sequence to realize infrared small target detection based on the five-dimensional spatiotemporal domain.

7. A five-dimensional spatiotemporal infrared small target detection device implementing the method of claim 1, characterized in that, include: The five-dimensional spacetime block tensor construction module constructs a five-dimensional spacetime block tensor from the infrared image sequence according to a given rule. The five-dimensional spacetime block tensor whitening module suppresses the second-order statistics of the five-dimensional spacetime block tensor and achieves five-dimensional tensor data whitening. The low-rank background estimation module defines the estimated background tensor. The five-dimensional nuclear norm; The sparse target tensor estimation module performs low-rank estimation of sparse targets and introduces a five-dimensional sparsity reweighting strategy to accelerate sparse target estimation. The tensor completion model building module is based on the five-dimensional white block space-time block tensor. It uses a low-rank background estimation strategy and a sparse target tensor estimation strategy to build a five-dimensional tensor completion model based on low-rank sparsity. The model optimization and solution module designs an optimization and solution framework based on the ADMM algorithm to solve the five-dimensional tensor completion model, obtain the target tensor, and reconstruct it into a sequence of infrared small target detection results. The target detection result output module is used to output the infrared small target detection result map for each frame of infrared image.

Images