A method and device for infrared small target detection based on representation learning

Abstract

The present invention discloses a method and device for detecting small infrared targets based on representation learning. The method comprises: (1) constructing a non-overlapping block spatiotemporal tensor from the original thermal infrared image sequence; (2) considering the characteristics of background, target and noise, building an architecture based on low-rank sparse tensor representation learning; (3) using the fully connected tensor ring decomposition to define the fully connected tensor nuclear norm and establish a low-rank background representation learning model; (4) designing a three-dimensional spatiotemporal weighted total variation regularization term to protect the edge structure of different background regions and remove noise, thereby reducing the impact of the background on target detection; (5) on the basis of the above steps, establishing a spatiotemporal tensor model based on low-rank sparse representation learning; (6) using the ADMM algorithm to optimize the solution model, obtain the target detection result, and realize infrared weak small target detection. The present invention can effectively realize the detection of infrared weak small targets by using the designed low-rank sparse representation learning model and the ADMM optimization algorithm.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method and device for detecting small infrared targets based on representation learning. Background Art

[0002] Infrared small target detection is an important part of automatic missile tracking, buried mine detection, and infrared search and tracking systems. Infrared small targets are usually characterized by small size and low signal noise. Small targets lack texture and shape information due to interference from long-distance imaging and complex environments, and are easily submerged in complex scenes. Therefore, the study of infrared small target detection has great research value and significance.

[0003] At present, infrared small target detection methods mainly include traditional detection methods and deep learning-based methods. Deep learning-based infrared small target detection algorithms include CPNet, CBP-Net, ALCNet, etc. However, these algorithms require a large amount of sample data to ensure the effectiveness and robustness of the detection algorithm, and require a large amount of computing resources. In addition, the black box characteristics of deep learning make the mechanism of small target detection difficult to reason and understand.

[0004] Traditional thermal infrared small target detection methods are mainly divided into methods based on background estimation filtering, local feature representation, and low-rank sparse decomposition and representation. Methods based on background estimation filtering include two-dimensional least mean square (TDLMS) filtering, top-hat filtering, Max-Median / Max-Mean filtering, etc. However, infrared images are usually complex and contain a lot of random noise and clutter. Methods based on background estimation filtering are prone to serious misdetection. Given that small targets are more visually significant than their surrounding backgrounds, methods based on local feature representation have been proposed one after another, including the local contrast metric (LCM) method and its variant algorithms, including TLLCM, MDWCM, WSLCM, MDWCM, etc. According to the contrast difference between the target and the background, the local contrast metric criterion is designed to suppress the background and enhance the target saliency, but the detection ability of the detector is easily affected by the background. In infrared images, the background exhibits low-rank characteristics and the target exhibits sparse characteristics. The method based on low-rank sparse decomposition extracts the target from the original image by adding constraints on the target and background. Typical methods include IPI, LogTFNN, IMNN-LWEC, etc. However, due to the inaccurate constraints imposed on the target and background, the existing algorithms are prone to problems such as background shrinkage and target leakage. The method based on low-rank sparse representation believes that the background or target in the infrared image comes from one or more subspaces, and small target detection is achieved by learning the representation of the background or target. Classic small target detection methods based on representation learning include multi-subspace learning (SMSL) model, low-rank and sparse representation (LRSR) model, low-rank representation (LRR) model, etc. The construction of representation learning model plays a key role in the detection of infrared weak small targets. At present, the main work is still concentrated on the basis of infrared images in matrix form. Few research works are on the low-rank sparse representation of tensor structure data of infrared images to achieve infrared weak small target detection. Summary of the invention

[0005] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a method and device for infrared small target detection based on representation learning, which organically combines the low-rank representation learning of the background, the sparsity of the target and the three-dimensional weighted total variation regularization term, and establishes a spatiotemporal tensor model based on low-rank sparse representation learning, which can better decompose the original infrared image into background components, target components and noise components. The proposed infrared small target detection method based on representation learning can effectively suppress the background, enhance the target, improve the detection performance of thermal infrared small targets, and realize infrared small target detection.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] On one hand, the present invention discloses an infrared small target detection method based on representation learning, which is characterized by comprising the following steps:

[0008] Step 1): For the original thermal infrared image sequence D, stack the image frames in the order of the image frames to realize the transformation from the original image sequence D to the spatiotemporal tensor 's transformation;

[0009] Step 2): Consider the background tensor Target Tensor and the noise tensor Characteristics, construct background dictionary and the coefficient tensor Background tensor Perform low-rank representation on the target tensor Perform sparse estimation and build an architecture based on low-rank sparse tensor representation learning;

[0010] Step 3): For the coefficient tensor Perform full connection tensor ring decomposition to obtain coefficient tensor factor set It contains three coefficient tensor factors and The non-convex low-rank estimation norm, i.e. the fully connected tensor nuclear norm, is constructed by taking the weighted sum of the tensor nuclear norms of all coefficient tensor factors. The coefficient tensor is measured The low-rank nature of , and establish a low-rank background representation learning model;

[0011] Step 4): Design weighted isotropic and anisotropic total variation regularization terms to construct the three-dimensional spatiotemporal weighted total variation norm of the background tensor Protect the edge structures of different areas in the infrared image background, remove noise, and reduce the impact of the background on target detection;

[0012] Step 5): Combining step 3), step 4) and the sparsity weighting strategy, on the basis of the architecture based on low-rank sparse tensor representation learning in step 2), a spatiotemporal tensor model based on low-rank sparse representation learning is established;

[0013] Step 6): Solve the spatiotemporal tensor model based on low-rank sparse representation learning to obtain the target tensor And reconstruct the infrared target detection result map to obtain the small target detection result map sequence T, realizing thermal infrared small target detection.

[0014] The present invention also discloses an infrared small target detection device based on representation learning for implementing the method, which comprises:

[0015] The spatiotemporal tensor construction module is used to construct the original infrared image sequence into a spatiotemporal tensor, laying the data foundation for subsequent low-rank sparse representation learning;

[0016] The low-rank sparse tensor representation learning framework building module uses the low-rank characteristics of the background tensor and the sparse characteristics of the target tensor to build an architecture based on low-rank sparse tensor representation learning;

[0017] The low-rank background representation learning module is used to perform a fully connected tensor ring decomposition on the representation coefficient tensor, construct a non-convex low-rank estimation norm to measure the low-rank property of the coefficient tensor, and establish a low-rank background representation learning model to characterize the background component;

[0018] The three-dimensional spatiotemporal weighted total variation norm module designs weighted isotropic and anisotropic total variation regularization terms for background components to protect the edge structures of different areas in the infrared image background, remove noise, and reduce the impact of background components on target detection.

[0019] The spatiotemporal tensor model building module based on low-rank sparse representation learning combines the low-rank background representation learning model, the sparse estimation of the target tensor, the sparsity weighting strategy and the three-dimensional spatiotemporal weighted total variation norm of the background tensor to establish a spatiotemporal tensor model based on low-rank sparse representation learning;

[0020] The representation learning model solving module solves the representation learning model and obtains the target tensor;

[0021] The target detection result output module is used to output the infrared small target detection result image sequence.

[0022] The beneficial effects of the present invention are:

[0023] 1) The present invention fully considers the spatiotemporal dimensional characteristics of the background components, represents the background in a low-rank manner, decomposes the coefficient tensor using a fully connected tensor ring, and proposes a new non-convex low-rank estimation norm, namely the fully connected tensor nuclear norm, which can more realistically measure the low-rank coefficient tensor and thus more accurately represent the low-rank background, which is conducive to separating the target and the background and reducing the influence of the background component on the target component.

[0024] 2) Inspired by the influence of noise in different image regions on small thermal infrared targets, the present invention proposes a three-dimensional weighted total variation regularization term from the perspective of spatiotemporal dimensions and applies it to the background tensor. The spatiotemporal anisotropy in the regularization term can remove noise and retain edge information in the background, and the isotropy can retain more details of flat areas while removing noise during the denoising process. In this way, the influence of background components and target components on small target detection can be further reduced, and the false alarm rate can be reduced.

[0025] 3) The present invention combines the low-rank representation learning of the background, the sparsity of the target and the three-dimensional weighted total variation regularization term to establish a spatiotemporal tensor model based on low-rank sparse representation learning, which can better decompose the original infrared image into background components, target components and noise components, and to a certain extent alleviate the contradiction between the detector's target detection performance and background suppression performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of the present invention for detecting small targets in thermal infrared images;

[0027] Figure 2 It is a structural schematic diagram of the infrared small target detection device of the present invention;

[0028] Figure 3 It is a schematic diagram of the construction of the space-time tensor of the infrared image of the present invention;

[0029] Figure 4 It is a schematic diagram of decomposition of a fully connected tensor ring of the present invention;

[0030] Figure 5 This is a sample frame image of a thermal infrared image sequence used for experimental testing;

[0031] Figure 6 The thermal infrared small target detection result of the example frame of the thermal infrared image sequence;

[0032] Figure 7 The detection result image and the corresponding 3D grayscale visualization image of the example frames of the thermal infrared image sequence after being detected by the proposed method, FAMSIS, SRSTT, STT-TRNR and MSLSTIPT algorithms;

[0033] Figure 8 The detection result image and the corresponding three-dimensional grayscale visualization image of the example frames of the thermal infrared image sequence after detection by TCNN-NPSTT, ASTTV-NTLA, ECA-STT and WSNM-STIPT algorithms. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention is described in detail below in conjunction with specific embodiments. Specific embodiments are described below to simplify the present invention. However, it should be recognized that the present invention is not limited to the illustrated embodiments, and various modifications of the present invention are possible without departing from the basic principles, and these equivalent forms also fall within the scope defined by the claims attached to the present application.

[0035] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] like Figure 1 As shown, it is a basic step flow chart of the infrared small target detection method based on representation learning in this embodiment of the present invention, which mainly includes:

[0037] Step 1: For the original thermal infrared image sequence D, stack the image frames in sequence to realize the transformation from the original image sequence D to the spatiotemporal tensor 's transformation;

[0038] like Figure 3 As shown, from the original infrared image sequence D, I3 image frames are obtained each time, and the image frames According to the frame order of the image frames, stack them one by one to construct the space-time tensor I1 and I2 represent the length and width of the image frame respectively. In the embodiment of the present invention, I3 is set to 16;

[0039] Step 2: Consider the background tensor Target Tensor and the noise tensor Characteristics, construct background dictionary and the coefficient tensor Background tensor Perform low-rank representation on the target tensor Perform sparse estimation and build an architecture based on low-rank sparse tensor representation learning;

[0040] Specifically, the space-time tensor Considered as background tensor Target Tensor and the noise tensor Considering the low-rank nature of the background and the sparsity of the target, the spatiotemporal tensor can be Modeled as:

[0041]

[0042] Among them, rank(·) represents the rank calculation operator, r represents the rank of the background tensor, ||·||0 represents the l0 norm, τ represents the sparsity of the target, and ||·|| F represents the Frobenius norm, and κ represents the intensity of the noise.

[0043] For the background tensor It can be considered to come from a single subspace or multiple low-rank subspaces. Since the representation learning model can represent uniform background and highly heterogeneous background, by using the background dictionary Low-rank representation can be used to represent background components.

[0044]

[0045] in, is the coefficient tensor, and “*” is the tensor product operator; considering the tensor In the spacetime tensor, most of them are low-rank backgrounds. Therefore, As a background dictionary The background components are represented by a low rank. Although the rank function is non-convex, the calculation of tensor rank is still an NP-hard problem, so the coefficient tensor needs to be Perform low-rank estimation.

[0046] For the target tensor The l0 norm can be used to estimate its sparsity, but directly solving the l0 norm is also an NP-difficult problem, so the l1 norm is used to best approximate the l0 norm.

[0047] Therefore, the space-time tensor Model (1) can establish an architecture based on low-rank sparse tensor representation learning:

[0048]

[0049] Among them, ||·||1 represents the l1 norm, and λ1 and λ2 are both coordination parameters. Specifically in the embodiment, λ2=0.1λ1,λ L Take it as 0.0503.

[0050] Step 3: Coefficient Tensor Perform full connection tensor ring decomposition to obtain coefficient tensor factors It contains three coefficient tensor factors and The non-convex low-rank estimation norm, i.e. the fully connected tensor nuclear norm, is constructed by taking the weighted sum of the tensor nuclear norms of all coefficient tensor factors. Metric coefficient tensor The low-rank nature of , and establish a low-rank background representation learning model;

[0051] Specifically, Figure 4 As shown, for the coefficient tensor Perform full connection tensor ring decomposition to obtain 3 coefficient tensor factors and The set of all coefficient tensor factors is denoted by The calculation formula is:

[0052]

[0053] Among them, Ψ(·) represents the fully connected tensor ring decomposition operator, R 1,2 , R 2,3 , I3 represents the coefficient tensor factor Scale of The scales of the first and second dimensions are The scale of the second and first dimensions is consistent, The third dimension of the scale and The scale of the second dimension of The scale of the first dimension is The scale of the third dimension is consistent, I2 is the width of the image frame, and I3 is the number of consecutive image frames;

[0054] More specifically, the element-by-element form of the fully connected tensor ring decomposition can be expressed as:

[0055]

[0056] in, Represents the coefficient tensor The element at position (i,j,k) in Represents the factor located in the coefficient tensor (i,r 1,2 ,r 1,3 ) position element, Represents the factor located in the coefficient tensor Middle (r 1,2 ,j,r 2,3 ) position element, Represents the factor located in the coefficient tensor Middle (r 1,3 ,r 2,3 , the element at position k), i∈[1,I2], j∈[1,I2], k∈[1,I3], “·” represents the element multiplication symbol;

[0057] By the coefficient tensor The three coefficient tensor factors obtained by decomposing the fully connected tensor ring and The weighted sum of the tensor nuclear norms of , defining the fully connected tensor nuclear norm ||·|| FCTNN , the calculation formula is:

[0058]

[0059] Among them, ωm Represents the mth coefficient tensor factor The weight of the tensor nuclear norm, is the coefficient tensor factor The i-th frontal slice after fast Fourier transformation, R m,3 ,m=1,2 represents the mth coefficient tensor factor The third dimension of the scale, ||·|| * represents a nuclear norm calculation operator; specifically in the embodiment, ω is taken as 4.

[0060] Therefore, the low-rank background representation learning model is:

[0061]

[0062] Wherein, λ is a positive balance parameter. Specifically in the embodiment, λ is set to 1.

[0063] Step 4: Design weighted isotropic and anisotropic total variation regularization terms to construct the three-dimensional spatiotemporal weighted total variation norm of the background tensor Protect the edge structures of different areas in the infrared image background, remove noise, and reduce the impact of the background on target detection;

[0064] Specifically, considering the three-dimensional spatiotemporal form of the background tensor in model (1), in order to remove image noise and retain edge information from the spatiotemporal dimension, the two-dimensional anisotropy of the background tensor is extended to a three-dimensional form. The norm of its three-dimensional anisotropy is ||·|| ani for:

[0065]

[0066] Among them, H h ,H v ,H z Respectively represent the linear horizontal and vertical difference operators along the spatial dimension and the linear difference operator along the time dimension, defined as in, Represents the background tensor For the element at position (i, j, k) in , α is a positive balance parameter; specifically in the embodiment, α is taken as 0.5.

[0067] Isotropic filtering makes it easier to smooth details and edge areas during the denoising process, and can remove noise while retaining more details in flat areas. At the same time, the norm of isotropic filtering is in the form of ||·|| iso for

[0068]

[0069] Therefore, we design the weighted isotropic and anisotropic total variation regularization terms and construct the three-dimensional spatiotemporal weighted total variation norm of the background tensor for:

[0070]

[0071] Wherein, β is a positive balance parameter. Specifically in the embodiment, β is set to 1.

[0072] Step 5: Combining the low-rank representation and sparsity weighting strategy of the background in step 3) and the three-dimensional spatiotemporal weighted total variation of the background in step 4), a spatiotemporal tensor model based on low-rank sparse representation learning is established on the basis of the architecture based on low-rank sparse tensor representation learning in step 2);

[0073] Specifically, in order to improve the efficiency of model solving, a sparsity weighting strategy is introduced, and the sparsity enhancement weight The iterative calculation formula is:

[0074]

[0075] Wherein, a is a positive constant, and ε is a positive constant to avoid the denominator being zero. Specifically in the embodiment, a is 2, and ε is 0.01.

[0076] The space-time tensor The model combines the low-rank representation learning of the background, the sparsity weighting strategy and the three-dimensional spatiotemporal weighted total variation regularization of the background, and the low-rank sparse representation learning model of the thermal infrared image can be obtained as follows:

[0077]

[0078] Among them, ⊙ is the Hadamard product.

[0079] Step 6: Solve the spatiotemporal tensor model based on low-rank sparse representation learning to obtain the target tensor And reconstruct the infrared target detection result map to obtain the small target detection result map sequence T, realizing thermal infrared small target detection.

[0080] Specifically, the alternating direction method of multipliers (ADMM) algorithm is used to solve the optimization model, and the auxiliary variables are introduced and The equivalent model of the spatiotemporal tensor model based on low-rank sparse representation learning is obtained as:

[0081]

[0082] The augmented Lagrangian function corresponding to the equivalent model for:

[0083]

[0084] Among them, μ is the penalty coefficient, is the Lagrange multiplier.

[0085] The problem of solving model (14) is decomposed into several sub-problems, and the variables are updated alternately:

[0086] -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is:

[0087]

[0088] in, represents the pseudo-inverse operator, Representing a tensor The pseudo-inverse calculation of represents the unit tensor, Represents the background dictionary The transpose of , t represents the number of iterations;

[0089] -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is:

[0090]

[0091] in, and Respectively represent the auxiliary tensors in the tth generation and Perform the mode-k expansion of the tensor to obtain the matrix, represents the Lagrange multiplier of the tth generation The matrix obtained by performing the mode-k expansion of the tensor is defined as Represents the kth tensor factor k∈{1,2,3} does not participate in the fully connected tensor ring decomposition, Represents the tth generation According to [m 1:N-1 ;n 1:N-1 ] is a matrix obtained by sequentially expanding the dimensions of , N = 3, and there is

[0092]

[0093] The calculation formula for solving formula (14) is as follows:

[0094]

[0095] Furthermore, for the matrix Merge and reconstruct the coefficient tensor factor

[0096] -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is:

[0097]

[0098] in, represents the singular value contraction operator,

[0099] -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is:

[0100]

[0101] in, represents the inverse fast Fourier transform operator;

[0102] -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is:

[0103]

[0104] -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is:

[0105]

[0106]

[0107] in, represents the soft threshold shrinkage operator, and

[0108] -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is:

[0109]

[0110] -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is:

[0111]

[0112] -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is:

[0113]

[0114] -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is:

[0115]

[0116] -Subproblem: Fix other parameters of the tth generation and iteratively update the calculation formula of the Lagrange multiplier:

[0117]

[0118]

[0119]

[0120]

[0121]

[0122]

[0123]

[0124] μ,R-subproblem: The iterative calculation formula of μ,R is:

[0125] μ (t+1) =min(ημ (t+1) ,μ max ) (35)

[0126]

[0127] Among them, η is the amplification factor, the penalty parameter μ of the t+1th generation is η times that of the tth generation, and μ max and (R i,j ) max is the maximum value set. Specifically in the embodiment, μ (0) Take it as 0.1, μ max is taken as 1e10, η is taken as 1.4, Take it as 3, (R i,j ) max Take it as 20.

[0128] When the relative error When the error is less than δ=1e-4, the iteration ends. The number of non-zero elements in No longer changes, stop iteration. Finally, the target tensor component will be obtained Reconstruct the target detection result image sequence T to realize the detection of small infrared targets.

[0129] Corresponding to the aforementioned embodiment of an infrared small target detection method based on representation learning, the present invention also provides an embodiment of an infrared small target detection device based on representation learning.

[0130] Figure 2 FIG. 1 is a block diagram of an infrared small target detection device based on representation learning according to an exemplary embodiment. Figure 2 As shown, the device comprises:

[0131] The spatiotemporal tensor construction module is used to construct the original infrared image sequence into a spatiotemporal tensor, laying the data foundation for subsequent low-rank sparse representation learning;

[0132] The low-rank sparse tensor representation learning framework building module uses the low-rank features of the background and the sparse characteristics of the target to build an architecture based on low-rank sparse tensor representation learning;

[0133] The low-rank background representation learning module is used to perform a fully connected tensor ring decomposition on the representation coefficient tensor, construct a non-convex low-rank estimation norm to measure the low-rank property of the coefficient tensor, and establish a low-rank background representation learning model to characterize the background component;

[0134] Three-dimensional weighted total variation regularization module, for background components, designs weighted isotropic and anisotropic total variation regularization terms to protect the edge structure of different areas in the infrared image background, remove noise, and reduce the impact of background components on target detection;

[0135] The spatiotemporal tensor model building module based on low-rank sparse representation learning combines the low-rank representation learning of the background, the sparse estimation of the target component, the sparsity weighting strategy and the three-dimensional spatiotemporal weighted total variation of the background to establish a spatiotemporal tensor model based on low-rank sparse representation learning;

[0136] The representation learning model solving module solves the representation learning model based on the ADMM algorithm and obtains the target detection tensor;

[0137] The target detection result output module is used to output the infrared small target detection result image sequence.

[0138] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0139] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic. The various modules in the device are a kind of logical function division. There may be other division methods in actual implementation. For example, multiple modules can be combined or integrated into another unit. Another point is that the connection between the modules shown or discussed can be a communication connection through some interfaces, which can be electrical or other forms. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present application. Ordinary technicians in this field can understand and implement it without paying creative labor. The following takes the disclosed real thermal infrared image sequence as an example to illustrate the specific implementation method to reflect the technical effect of the present invention. The specific steps in the embodiment will not be repeated.

[0140] Example

[0141] The accompanying drawings shown in the embodiments of the present invention can make the purpose, technical solutions and advantages of the present invention more clearly described. It should be noted that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. All equivalent replacements, improvements, etc. made within the method ideas and principles provided by the present invention should be included in the protection scope of the present invention.

[0142] Next, a public infrared image sequence is used as the research object to verify the thermal infrared small target detection method. This data set contains 100 frames of thermal infrared images with a frame size of 256×256. Specifically, the air-ground background contains strong noise and clutter, and there is a bright background structure. The flying object moves slowly, and the contrast between it and the local background is not significant, and there are many non-target suspected objects. In order to evaluate the effectiveness of the algorithm proposed in the present invention from the perspective of intuitive visual and quantitative evaluation, the performance of the thermal infrared small target detection method is evaluated by using the intuitive result graph of small target detection and the corresponding three-dimensional visualized grayscale image, 3D-ROC and its derived evaluation index system. In 3D-ROC and its derived evaluation index system, the target detection capability (TD) indicators include AUC (D,τ) , AUC TD ; Background suppression (BS) indicators include AUC (F,τ) , AUC BS , AUC SNPR , and the comprehensive effectiveness evaluation indicators of the detector include AUC TDBS , AUC (D,F) , AUC ODP , specifically:

[0143] 1) AUC (D,τ) Indicates (P D,τ) curve, which represents the target detection ability of the detector

[0144] 0≤AUC (D,τ) ≤1 (37)

[0145] 2) AUC TD Characterizing the joint detection capabilities of detectors

[0146] 0≤AUC TD =AUC (D,F) +AUC (D,τ) )≤2 (38)

[0147] 3) AUC (F,τ) Indicates (P D ,τ) curve, which represents the background suppression ability of the detector.

[0148] 0≤AUC (F,τ) ≤1 (39)

[0149] 4) AUC BS Characterizing the joint background suppression capabilities of detectors

[0150] -1≤AUC BS =AUC (D,F) -AUC (F,τ) )≤1 (40)

[0151] 5) AUC SNPR Characterizing the detector's signal-to-noise ratio

[0152]

[0153] 6) AUC TDBS Characterizes the detector's combined target detection and background suppression capabilities

[0154] -1≤AUC TDBS =AUC (D,τ) -AUC (F,τ) )≤1 (42)

[0155] 7) AUC (D,F) Characterization (P D ,P F ) curve, indicating the effectiveness of the detector

[0156] 0≤AUC (D,F) ≤1 (43)

[0157] 8) AUC ODP Characterizes the total detection probability of the detector

[0158] -1≤AUC ODP =AUC (D,F)+AUC (D,τ) -AUC (F,τ) ≤2 (44)

[0159] Figure 3 is a sample frame of an infrared image sequence, such as Figure 4 The corresponding thermal infrared small target detection result diagram obtained by the method of the present invention. The visualization result intuitively shows that the complex background and noise are greatly suppressed, and the significance of the small target is enhanced, well enhanced and highlighted. Table 1 shows the quantitative indicators of the small target detection results of the thermal infrared image example sequence using FAMSIS, SRSTT, STT-TRNR, MSLSTIPT, TCNN-NPSTT, ASTTV-NTLA, ECA-STT, WSNM-STIPT and the method proposed in the present invention. The bold and underlined values ​​represent the corresponding optimal AUC results and suboptimal AUC results, respectively.

[0160] The sources of the comparison methods are as follows:

[0161] The FAMSIS method comes from Y.Chen, G.Zhang, Y.Ma, JUKang, and C.Kwan, "Small infraredtarget detection based on fast adaptive masking and scaling with iterativesegmentation," IEEE Geosci.Remote Sens.Lett., vol.19, pp.1–5, 2021.

[0162] The SRSTT method comes from J.Li, P.Zhang, L.Zhang, and Z.Zhang, "Sparse regularization-based spatial-temporal twist tensor model for infrared small target detection," IEEE Trans.Geosci.Remote Sens., 2023.

[0163] The STT-TRNR method comes from H.Yi, C.Yang, R.Qie, J.Liao, F.Wu, T.Pu, and Z.Peng, "Spatial-temporal tensor ring norm regularization for infrared small target detection," IEEE Geosci.Remote Sens.Lett., 2023.

[0164] The MSLSTIPT method is from Y. Sun, J. Yang, and W. An, “Infrared dim and small target detection via multiple subspace learning and spatial-temporal patch-tensor model,” IEEE Trans. Geosci. Remote Sens., vol. 59, no. 5, pp. 3737 - 3752, 2020.

[0165] The TCNN-NPSTT method is from G. Wang, B. Tao, X. Kong, and Z. Peng, “Infrared small target detection using nonoverlapping patch spatial-temporal tensor factorization with capped nuclear norm regularization,” IEEE Trans. Geosci. Remote Sens., vol. 60, pp. 1 - 17, 2021.

[0166] The ASTTV-NTLA method is from T. Liu, J. Yang, B. Li, C. Xiao, Y. Sun, Y. Wang, and W. An, “Nonconvex tensor low-rank approximation for infrared small target detection,” IEEE Trans. Geosci. Remote Sens., vol. 60, pp. 1 - 18, 2021.

[0167] The ECA-STT method is from P. Zhang, L. Zhang, X. Wang, F. Shen, T. Pu, and C. Fei, “Edge and corner awareness-based spatial-temporal tensor model for infrared small-target detection,” IEEE Trans. Geosci. Remote Sens., vol. 59, no. 12, pp. 10708 - 10724, 2020.

[0168] The WSNM-STIPT method comes from Y.Sun, J.Yang, M.Li, and W.An, "Infrared small target detection via spatial-temporal infrared patch-tensor model and weightedschatten pnorm minimization," Infrared Phys.Technol., vol.102, p.103050, 2019.

[0169] Table 1 Thermal infrared image example sequences using FAMSIS, SRSTT, STT-TRNR, MSLSTIPT, TCNN-NPSTT,

[0170] Quantitative indicators of the detection results of ASTTV-NTLA, ECA-STT, WSNM-STIPT and the proposed method

[0171] method <![CDATA[AUC (D，F) ]]> <![CDATA[AUC (D，τ) ]]> <![CDATA[AUC (F，τ) ]]> <![CDATA[AUC TD ]]> <![CDATA[AUC BS ]]> <![CDATA[AUC SNPR ]]> <![CDATA[AUC TDBS ]]> <![CDATA[AUC ODP <!-- 11 -->]]> The present invention 1.0000 0.9990 1.6044e-5 1.9990 1.0000 6.2265e4 0.9990 1.9990 FAMSIS 0.9999 0.9960 5.7976e-5 1.9959 <![CDATA[ 0.9999 ]]> 1.7179e4 <![CDATA[ 0.9959 ]]> <![CDATA[ 1.9959 ]]> SRSTT 1.0000 0.9810 8.3335e-5 1.9810 <![CDATA[ 0.9999 ]]> 1.1772e4 0.9809 1.9809 STT-TRNR <![CDATA[ 0.9867 ]]> 0.9867 2.1147e-2 1.9733 0.9655 4.6658e1 0.9655 1.9522 MSLSTIPT 1.0000 0.9922 3.4283e-1 1.9922 0.6572 2.8941e0 0.6494 1.6494 TCNN-NPSTT 1.0000 0.8461 <![CDATA[ 4.2574e-5 ]]> 1.8461 0.9999 <![CDATA[ 1.9873e4 ]]> 0.8460 1.8460 ASTTV-NTLA 1.0000 0.9900 9.4075e-2 1.9900 0.9059 1.0524e1 0.8959 1.8959 ECA-STT 1.0000 <![CDATA[ 0.9987 ]]> 3.1984e-3 <![CDATA[ 1.9987 ]]> 0.9968 3.1226e2 0.9955 1.9955 WSNM-STIPT <![CDATA[ 0.9867 ]]> 0.9185 1.3351e-4 1.9052 0.9865 6.8798e3 0.9184 1.9051

[0172] Figure 5 and Figure 6 The result images of thermal infrared small target detection using the present invention, FAMSIS, SRSTT, STT-TRNR, MSLSTIPT, TCNN-NPSTT, ASTTV-NTLA, ECA-STT, and WSNM-STIPT for the thermal infrared image example sequence are shown. From the qualitative results, it can be seen that although these comparison methods can detect small targets, the background suppression ability is not strong, and there are still many clutters and false alarms; the spatiotemporal tensor model based on low-rank sparse representation learning proposed in the present invention can completely suppress the background and enhance the significance of small targets. Combined with the quantitative results of the 8 3D-ROC evaluation indicators shown in Table 1, the method proposed in the present invention is superior to other existing algorithms in terms of target detection ability, background suppression ability and comprehensive effectiveness of the detector. In summary, the thermal infrared small target detection method proposed in the present invention has certain advantages in terms of target detection ability and background suppression ability.

[0173] The accompanying drawings shown in the embodiments of the present invention can make the purpose, technical solutions and advantages of the present invention more clearly described. It should be noted that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. All equivalent replacements, improvements, etc. made within the method ideas and principles provided by the present invention should be included in the protection scope of the present invention.

Claims

1. A small infrared target detection method based on representation learning, characterized in that: The steps include: Step 1): For the original thermal infrared image sequence D, stack the image frames in the order of the image frames to realize the transformation from the original image sequence D to the spatiotemporal tensor 's transformation; Step 2): Consider the background tensor Target Tensor and the noise tensor Characteristics, construct background dictionary and the coefficient tensor Background tensor Perform low-rank representation on the target tensor Perform sparse estimation and build an architecture based on low-rank sparse tensor representation learning; Step 3): For the coefficient tensor Perform full connection tensor ring decomposition to obtain coefficient tensor factor set It contains three coefficient tensor factors and The non-convex low-rank estimation norm, i.e. the fully connected tensor nuclear norm, is constructed by taking the weighted sum of the tensor nuclear norms of all coefficient tensor factors. The coefficient tensor is measured The low-rank nature of , and establish a low-rank background representation learning model; The step 3) is specifically as follows: Coefficient Tensor Perform full connection tensor ring decomposition to obtain 3 coefficient tensor factors and The set of all coefficient tensor factors is denoted by The calculation formula is: Among them, Ψ(·) represents the fully connected tensor ring decomposition operator, R 1,2 , R 1,3 , R 2,3 is the scale of the coefficient tensor factor; The scales of the first and second dimensions are The scale of the second and first dimensions is consistent, The third dimension of the scale and The scale of the second dimension of The scale of the first dimension is The scale of the third dimension is consistent, I2 is the width of the image frame, and I3 is the number of consecutive image frames; More specifically, the element-wise form of the fully connected tensor ring decomposition is expressed as: in, Represents the coefficient tensor The element at position (i,j,k) in Represents the factor located in the coefficient tensor (i,r 1,2 ,r 1,3 ) position element, Represents the factor located in the coefficient tensor Middle (r 1,2 ,j,r 2,3 ) position element, Represents the factor located in the coefficient tensor Middle (r 1,3 ,r 2,3 , the element at position k), i∈[1,I2], j∈[1,I2], k∈[1,I3], "·" represents the element multiplication symbol; By coefficient tensor The three coefficient tensor factors obtained by decomposing the fully connected tensor ring and The weighted sum of the tensor nuclear norms of , defining the fully connected tensor nuclear norm ||·|| FCTNN , the calculation formula is: Among them, ω m Represents the mth coefficient tensor factor The weight of the tensor nuclear norm, is the coefficient tensor factor The i-th frontal slice after fast Fourier transformation, R m,3 ,m=1,2 represents the mth coefficient tensor factor The third dimension of the scale, ||·|| * represents the nuclear norm calculation operator; Therefore, the low-rank background representation learning model is: Among them, λ is the positive balance parameter; Step 4): Design weighted isotropic and anisotropic total variation regularization terms to construct the three-dimensional spatiotemporal weighted total variation norm of the background tensor Protect the edge structures of different areas in the infrared image background, remove noise, and reduce the impact of the background on target detection; Step 5): Combining step 3), step 4) and the sparsity weighting strategy, on the basis of the architecture based on low-rank sparse tensor representation learning in step 2), a spatiotemporal tensor model based on low-rank sparse representation learning is established; Step 6): Solve the spatiotemporal tensor model based on low-rank sparse representation learning to obtain the target tensor And reconstruct the infrared target detection result map to obtain the small target detection result map sequence T, realizing thermal infrared small target detection.

2. The infrared small target detection method based on representation learning according to claim 1 is characterized in that: In the step 1), I3 image frames are obtained from the original infrared image sequence D each time, and each image frame According to the frame order of the image frames, stack them one by one to construct the space-time tensor and The i-th frontal slice of is the i-th complete image frame, I1 and I2 represent the length and width of the image frame respectively, and I3 represents the number of continuous image frames acquired each time.

3. The infrared small target detection method based on representation learning according to claim 1 is characterized in that: The step 2) is specifically as follows: The space-time tensor Considered as background tensor Target Tensor and the noise tensor Considering the low-rank characteristics of the background tensor and the sparse characteristics of the target tensor, the spatial-temporal tensor Modeled as Among them, rank(·) represents the rank calculation operator, r represents the rank of the background tensor, and ||·||0 represents norm, τ represents the sparsity of the target, ||·|| F represents the Frobenius norm, κ represents the intensity of the noise; For the background tensor It is believed that it comes from a single subspace or multiple low-rank subspaces. Therefore, by Perform a low-rank representation to represent the background tensor, in, is the coefficient tensor, "*" is the tensor product operator; considering the tensor In the spacetime tensor, most of them are low-rank backgrounds. Therefore, As a background dictionary A low-rank representation of the background component is required; and the coefficient tensor Perform low-rank estimation; For the target tensor use The sparsity is estimated by using Norm Best Approximation norm; Therefore, the space-time tensor Model (1) builds an architecture based on low-rank sparse tensor representation learning: Among them, ||·||1 means norm, λ1 and λ2 are coordination parameters.

4. The infrared small target detection method based on representation learning according to claim 3 is characterized in that: The step 4) is specifically as follows: Considering the three-dimensional spatiotemporal form of the background tensor in model (1), in order to remove image noise and retain edge information from the spatiotemporal dimension, the two-dimensional anisotropy of the background tensor is extended to a three-dimensional form. The norm of its three-dimensional anisotropy is ||·|| ani for: Among them, H h ,H v ,H z Respectively represent the linear horizontal and vertical difference operators along the spatial dimension and the linear difference operator along the time dimension, defined as in, Represents the background tensor The element at position (i, j, k) in , α is a positive balance parameter; The isotropic norm of the background tensor is in the form ||·|| iso for: Therefore, we design the weighted isotropic and anisotropic total variation regularization terms and construct the three-dimensional spatiotemporal weighted total variation norm of the background tensor for: Among them, β is a positive balance parameter.

5. The infrared small target detection method based on representation learning according to claim 4 is characterized in that: The step 5) is specifically as follows: In order to improve the efficiency of model solving, a sparsity weighting strategy is introduced, and the sparsity enhances the weight tensor The calculation formula is: Among them, a is a positive constant, ε is a positive constant to avoid the denominator being zero, Based on the architecture based on low-rank sparse tensor representation learning, combined with the low-rank background representation learning model, the three-dimensional spatiotemporal weighted total variation norm of the background tensor and formula (11), the spatiotemporal tensor model based on low-rank sparse representation learning is obtained: Among them, ⊙ is the Hadamard product.

6. The infrared small target detection method based on representation learning according to claim 5 is characterized in that: The step 6) is specifically as follows: The alternating direction method of multipliers (ADMM) algorithm is used to solve the optimization model and introduce auxiliary variables and The equivalent model of the spatiotemporal tensor model based on low-rank sparse representation learning is obtained as The augmented Lagrangian function corresponding to the equivalent model for: Among them, μ is the penalty coefficient, is the Lagrange multiplier; Decompose the problem of solving formula (14) into several sub-problems and update the variables alternately; During the iteration process, when the relative error When the error is less than the set error δ, the iteration terminates; when the target tensor The number of non-zero elements in No longer changes, stop iteration; the target tensor will be obtained in the end Reconstructed into infrared target detection result map, the small target detection result map sequence T is obtained to realize thermal infrared small target detection.

7. The infrared small target detection method based on representation learning according to claim 6 is characterized in that: The problem of solving formula (14) is decomposed into several sub-problems, and the variables are updated alternately, specifically: -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is: in, represents the pseudo-inverse operator, Representing a tensor The pseudo-inverse calculation of represents the unit tensor, Represents the background dictionary The transpose of , t represents the number of iterations; -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is: in, and Respectively represent the auxiliary tensors in the tth generation and Perform the mode-k expansion of the tensor to obtain the matrix, represents the Lagrange multiplier of the tth generation The matrix obtained by performing the mode-k expansion of the tensor is defined as Represents the kth tensor factor k∈{1,2,3} does not participate in the fully connected tensor ring decomposition, Represents the tth generation According to [m 1:N-1 ;n 1:N-1 ] is a matrix obtained by sequentially expanding the dimensions of , N = 3, and there is The calculation formula for solving formula (14) is as follows: Furthermore, for the matrix Merge and reconstruct the coefficient tensor factor -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is: in, represents the singular value contraction operator, -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is: in, represents the inverse fast Fourier transform operator; -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is: -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is: in, represents the soft threshold shrinkage operator, and -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is: -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is: -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is: -Subproblem: Fix other parameters of the tth generation and iteratively update them to obtain The calculation formula is: -Subproblem: Fix other parameters of the tth generation and iteratively update the calculation formula of the Lagrange multiplier: μ,R-subproblem: The iterative calculation formula of μ,R is: m (t+1) =min(day (t+1) ,m max ) (35) Among them, η is the amplification factor, the penalty parameter μ of the t+1th generation is η times that of the tth generation, and μ max and (R i,j ) max is the set maximum value.

8. An infrared small target detection device based on representation learning for implementing the method of claim 1, characterized in that: include: The spatiotemporal tensor construction module is used to construct the original infrared image sequence into a spatiotemporal tensor, laying the data foundation for subsequent low-rank sparse representation learning; The low-rank sparse tensor representation learning framework building module uses the low-rank characteristics of the background tensor and the sparse characteristics of the target tensor to build an architecture based on low-rank sparse tensor representation learning; The low-rank background representation learning module is used to perform a fully connected tensor ring decomposition on the representation coefficient tensor, construct a non-convex low-rank estimation norm to measure the low-rank property of the coefficient tensor, and establish a low-rank background representation learning model to characterize the background component; The three-dimensional spatiotemporal weighted total variation norm module designs weighted isotropic and anisotropic total variation regularization terms for background components to protect the edge structures of different areas in the infrared image background, remove noise, and reduce the impact of background components on target detection. The spatiotemporal tensor model building module based on low-rank sparse representation learning combines the low-rank background representation learning model, the sparse estimation of the target tensor, the sparsity weighting strategy and the three-dimensional spatiotemporal weighted total variation norm of the background tensor to establish a spatiotemporal tensor model based on low-rank sparse representation learning; The representation learning model solving module solves the representation learning model and obtains the target tensor; The target detection result output module is used to output the infrared small target detection result image sequence.

Images