An Infrared Small Target Detection Method and Device Based on Orthogonal Subspace Projection

Abstract

The present invention discloses an infrared small target detection method and device based on orthogonal subspace projection. The method includes: (1) constructing a three-dimensional spatio-temporal tensor from a thermal infrared image sequence to be detected; (2) establishing a data component analysis model based on the three-dimensional spatio-temporal tensor, and performing principal component analysis on the three-dimensional spatio-temporal tensor to obtain the number of principal components and the corresponding principal component matrix; (3) establishing a subspace and an orthogonal subspace corresponding to the principal components based on the principal component matrix, and projecting the three-dimensional spatio-temporal tensor onto the orthogonal subspace; (4) designing an information filter to suppress the remaining background and noise in the data and retain the infrared small target to obtain a target component tensor; (5) reconstructing the target component tensor into a target detection result sequence T to achieve thermal infrared small target detection. By using the designed orthogonal subspace projection operator and information filter, the present invention can effectively suppress the background, enhance the target, and achieve the detection of infrared weak and small targets.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and particularly to an infrared small target detection method and device based on orthogonal subspace projection. Background Art

[0002] Infrared detection and tracking systems have good concealment and strong anti-interference capabilities, and are widely used in military and civilian fields. Infrared small target detection is an important part of infrared detection and tracking systems. Due to the long distance between the target and the infrared sensor, infrared targets usually exhibit characteristics such as small size, weak thermal radiation, and small difference from the surrounding environment. Therefore, the detection and recognition of infrared small targets (such as long-range missiles, unmanned aerial vehicles, etc.) face the following unique challenges:

[0003] 1) The size of infrared small targets is usually between 2×2 and 9×9 pixels, so it is difficult to extract important features such as the specific shape, structure, and texture of the target, which greatly limits target detection and positioning;

[0004] 2) Infrared small targets are easily interfered by complex background conditions, including factors such as clouds, atmospheric changes, and illumination; in addition, the complexity of non-stationary backgrounds in infrared sequences, as well as various environmental noises and inherent sensor noises, will affect target detection;

[0005] 3) The real-time performance of infrared small target detection technology is another key factor determining the practicality of infrared detection and tracking systems. Target detection technology must optimize computational efficiency to meet the requirements of practical applications.

[0006] Generally, there are two different categories in the field of infrared small target detection, including traditional methods and deep learning-based detection methods. However, as data-driven methods, deep learning-based infrared small target detection methods require a large number of samples for effective feature learning; given the limited physical interpretability of neural networks, it will be challenging to solve specific problems in infrared small target detection; moreover, the designed networks often lack robustness in different scenarios. Therefore, traditional methods still have certain advantages and are worthy of further research.

[0007] Specifically, traditional methods can be roughly divided into methods based on the background consistency assumption, methods based on the human visual system, and methods based on low-rank sparse decomposition. For methods based on the background consistency assumption, various specific filters are usually designed for background suppression, and the detection results are easily affected by complex background textures. Methods based on the human visual system mainly use the contrast mechanism and design local contrast descriptors to achieve target enhancement and background suppression. However, the descriptors are sensitive to background regions and disturbances with significant contrast changes, resulting in false alarms. In addition, when the target-background contrast is low, the possibility of target omission increases. For methods based on low-rank sparse decomposition, they are usually designed and optimized under specific datasets or specific application scenarios, and their applicability may be limited by factors such as data distribution, noise models, and target characteristics; and methods based on low-rank sparse decomposition usually involve the selection of some parameters, which need to be adjusted in different scenarios, bringing certain difficulties to the use and popularization of this method; in addition, a large amount of calculation and iteration is usually required during the solution process, especially when dealing with high-dimensional data, which will significantly increase the complexity of the algorithm, resulting in low efficiency of the method in actual application. Summary of the Invention

[0008] Aiming at the deficiencies in the prior art, the purpose of the present invention is to provide a thermal infrared small target detection method and device based on orthogonal subspace projection. From the perspective of data components, the present invention makes full use of the data distribution differences between the background and the target, represents the background with the main components, and represents the target with the secondary components. Considering that the unique statistical distribution characteristics of the target and the background allow the construction of their corresponding subspaces, and project the secondary component and the main component into their corresponding subspaces respectively. Thus, the main components and the corresponding main component matrix are extracted through principal component analysis (PCA), and the corresponding subspaces and orthogonal subspaces are established, and the target components and some residual background components are projected into the orthogonal subspaces to achieve target enhancement and background suppression. Further, in order to filter out the residual background components and noise components, an information filter is designed, which can quickly detect thermal infrared small targets, realize the detection of infrared small targets, and the target detection ability, background suppression ability, and overall performance ability of this algorithm have been effectively verified.

[0009] To achieve the above object, the present invention provides the following technical solutions:

[0010] On the one hand, the present invention discloses an infrared small target detection method based on orthogonal subspace projection, including the following steps:

[0011] Step 1): Utilize the spatio-temporal information of the thermal infrared image, and stack the image frames in the thermal infrared image sequence in chronological order to construct a three-dimensional spatio-temporal tensor D;

[0012] Step 2): Based on the three-dimensional spatio-temporal tensor D, establish a data component analysis model, and perform principal component analysis on the three-dimensional spatio-temporal tensor D to obtain m principal components and the corresponding principal component matrix;

[0013] Step 3): Based on the m principal components and the corresponding principal component matrix, establish the subspace and orthogonal subspace corresponding to the principal components, project the three-dimensional spatio-temporal tensor D onto the orthogonal subspace, and obtain the projection tensor D_(V_m^"PCA")^⊥ of tensor D in the orthogonal subspace;

[0014] Step 4): Design an information filter to suppress the background components and noise components remaining in the projection tensor D_(V_m^"PCA")^⊥, and retain the infrared small targets to obtain the target component tensor T;

[0015] Step 5): Reconstruct the target component tensor T into the target detection result sequence T to achieve the detection of infrared small targets.

[0016] The present invention also discloses an infrared small target detection device based on orthogonal subspace projection for implementing the above method, which includes:

[0017] An infrared image reconstruction module, which stacks the image frames in the original infrared image sequence in chronological order to construct a three-dimensional spatio-temporal tensor, laying a data foundation for subsequent principal component extraction and orthogonal subspace construction;

[0018] A principal component determination module, which based on the three-dimensional spatio-temporal tensor, establishes a data component analysis model, and performs principal component analysis on the three-dimensional spatio-temporal tensor to obtain m principal components and the corresponding principal component matrix;

[0019] An orthogonal subspace construction module, which uses the number of principal components and the principal component matrix to establish the subspace and orthogonal subspace corresponding to the principal components, and obtains the projection tensor of the three-dimensional spatio-temporal tensor in the orthogonal subspace to enhance the target saliency and suppress the background components corresponding to the principal components;

[0020] An information filter design module, which is used to remove the noise and background components remaining in the data after orthogonal subspace projection, retain the target component data, and reconstruct the target component data into the corresponding infrared small target detection result sequence;

[0021] A target detection result output module, which is used to output the infrared small target detection result map.

[0022] The beneficial effects of the present invention are as follows:

[0023] 1) The present invention makes full use of the data distribution differences between the background components and the target components, uses principal component analysis to extract the principal components in the data, and establishes the subspace and orthogonal subspace corresponding to the principal components. By projecting the data into the orthogonal subspace, the extraction of target and noise components and background suppression are realized;

[0024] 2) The present invention designs an efficient information filter, which can effectively filter the remaining background and noise components after the original data is projected by the orthogonal subspace projection, realize the detection of small thermal infrared targets, and the background suppression ability, target detection ability and comprehensive effectiveness of the present invention have been effectively verified. Description of the Drawings

[0025] Figure 1 is the flow chart for the present invention to carry out the detection of small thermal infrared targets based on orthogonal subspace projection;

[0026] Figure 2 is the structural schematic diagram of the infrared small target detection device of the present invention;

[0027] Figure 3 is an example frame image of the infrared image sequence for experimental testing;

[0028] Figure 4 is the detection result of the small infrared target corresponding to the example frame of the thermal infrared image sequence;

[0029] Figure 5 is the detection result graph after the example frame of the thermal infrared image sequence is detected by WSLCM, GSWLCM, FAMSIS, METTR, LogTFNN, ASTTV-NTLA, 4D-TR, RCTVW and the proposed method. Detailed Description of the Invention

[0030] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be described in detail below in conjunction with specific embodiments. Specific embodiments are described below to simplify the present invention. However, it should be recognized that the present invention is not limited to the described embodiments, and various modifications of the present invention are possible without departing from the basic principles, and these equivalent forms also fall within the scope defined by the appended claims of this application.

[0031] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0032] As Figure 1 shown, it is the basic step flow chart of the infrared small target detection method based on orthogonal subspace projection of the present invention in this embodiment, mainly including:

[0033] Step 1: Utilize the spatio-temporal information of the thermal infrared image, and stack the image frames in a thermal infrared image sequence in chronological order to construct a three-dimensional spatio-temporal tensor

[0034] Specifically, using the spatio-temporal information of the thermal infrared images, a sequence containing N infrared image frames is stacked in chronological order to form a frame image D t , t = 1, 2, …, N, to construct a three-dimensional spatio-temporal tensor with a size of n 1 ×n 2 ×n 3 ; n 1 represents the width of the front slice of 2 represents the height of the front slice of 3 represents the thickness of 3 and n 3 is taken as 15;

[0035] Step 2: Based on the three-dimensional spatio-temporal tensor establish a data component analysis model, and perform principal component analysis on the three-dimensional spatio-temporal tensor to obtain m principal components and the corresponding principal component matrix;

[0036] Specifically, from the perspective of data components, the three-dimensional spatio-temporal tensor in step 1) can be modeled as a combination of m principal components and minor components, as shown in formula (1):

[0037]

[0038] where represents the three-dimensional spatio-temporal tensor, represents m principal components, represents minor components;

[0039] Perform principal component analysis on the three-dimensional spatio-temporal tensor to determine the number of principal components m:

[0040] Specifically, expand along the time dimension to obtain an expansion matrix which can be expressed as reshape(·) represents a dimension adjustment operator that adjusts a tensor D with a size of n 1 ×n 2 ×n 3 to an expansion matrix with a size of n 1 n 2 ×n 3 as where n 1 n 2 represents the expansion matrix Width, n 3 Denote the unfolded matrix Height;

[0041] To determine the number of principal components m, perform singular value decomposition on the unfolded matrix as shown in formula (2):

[0042]

[0043] where svd(·) represents the singular value decomposition operator, Denote n 1 n 2 ×n 1 n 2 Unitary matrix of order n, Denote n 1 n 2 ×n 3 Singular value matrix of order n, Denote n 3 ×n 3 Unitary matrix of order n; the number of principal components m is determined by the singular value σ in, and the elements on the diagonal of the singular value matrix are arranged in descending order. When the condition σ i+1 > ησ i is satisfied, then m is set to i, where η represents a positive coefficient, σ i represents the singular value with index i, and σ i+1 represents the singular value with index i + 1;

[0044] Three-dimensional spatio-temporal tensor The principal component matrix corresponding to the m principal components of is determined as follows:

[0045] Center the unfolded matrix and perform eigenvalue decomposition on the covariance matrix Cov corresponding to the centered data as shown in formula (3):

[0046] [V, E] = eig(Cov) (3)

[0047] where eig(·) represents the eigenvalue decomposition operator, the obtained V represents the eigenvector matrix, and E represents the eigenvalue matrix; thus, extract the first m eigenvectors from the eigenvector matrix V as the m principal components, denoted as the principal component matrix PC m ; thus define the matrix Represents the matrix containing m eigenvectors obtained by principal component analysis.

[0048] Specifically in the embodiment, η determines the number of principal components m, and η is taken as 0.4;

[0049] Step 3: Based on the m principal components and the corresponding principal component matrix, establish the subspaces and orthogonal subspaces corresponding to the principal components, and project the three-dimensional spatio-temporal tensor onto the orthogonal subspace to obtain the tensor of the projection tensor in the orthogonal subspace

[0050] Specifically, according to the principal component matrix PC m obtained in step 2), construct the subspaces corresponding to the m principal components, and the projection matrix corresponding to this subspace is shown in formula (4):

[0051]

[0052] where (·) T represents the matrix transpose operator, and (·) -1 represents the matrix inverse operator;

[0053] Construct the orthogonal subspace corresponding to the subspaces of the m principal components from formula (4), and the projection matrix corresponding to this orthogonal subspace is shown in formula (5):

[0054]

[0055] where I represents the identity matrix;

[0056] According to formula (5), project the expansion matrix onto the orthogonal subspace to obtain the projected matrix

[0057]

[0058] where the projected matrix represents the projection of the reconstruction matrix in the orthogonal subspace corresponding to the matrix containing m eigenvectors obtained by principal component analysis;

[0059] Adjust the size of the projected matrix to an n 1 × n 2 × n 3 tensor to obtain the projection tensor in the orthogonal subspace

[0060]

[0061] where reshape(·) represents the dimension adjustment operator Represents a three-dimensional spatio-temporal tensor The tensor obtained by projection in the orthogonal subspace corresponding to the principal component subspace;

[0062] Step 4: Design an information filter to suppress the small amount of background components and noise components remaining in the projected tensor Retain the infrared small target to obtain the target component tensor

[0063] Specifically, design an information filter. For each positive slice of the orthogonal subspace projection tensor Retain the top k largest element values, where k is a positive constant set to distinguish the target components from the remaining background and noise components, and set the remaining element values to 0 to obtain the target component tensor For each positive slice in the target component tensor As shown in formula (8): For each positive slice in As shown in formula (8):

[0064]

[0065] Among them, Represents the i-th positive slice of the tensor , Represents the designed information filter, which projects the elements with index values belonging to the set Ω In the positive slice 1:k To itself, and projects all other elements to zero, Represents The index value set Ω of the top k largest element values in 1:k , and 1 ≤ i ≤ n 3 ;

[0066] Specifically in the embodiment, k is taken as 2;

[0067] Step 5: Reconstruct the target component tensor Into the target detection result sequence T to achieve the detection of thermal infrared small targets;

[0068] Specifically, extract the i-th positive slice of the target component tensor obtained by formula (8) as the thermal infrared small target detection result T Of the i-th original image frame, thus obtaining the target detection result sequence T containing n i Thermal infrared small target detection result images to achieve the detection of thermal infrared small targets. 3

[0069] ​Corresponding to the embodiment of the foregoing infrared small target detection method based on orthogonal subspace projection, the present invention also provides an embodiment of an infrared small target detection device based on orthogonal subspace projection.

[0070] Figure 2 As shown in the block diagram of an infrared small target detection device based on orthogonal subspace projection according to an exemplary embodiment, Figure 2 as shown, the device includes:

[0071] A reconstructed infrared image module that stacks the image frames in the original infrared image sequence in chronological order to construct a three-dimensional spatio-temporal tensor, laying a data foundation for subsequent principal component extraction and orthogonal subspace construction;

[0072] A principal component determination module that establishes a data component analysis model based on the three-dimensional spatio-temporal tensor and performs principal component analysis on the three-dimensional spatio-temporal tensor to obtain m principal components and corresponding principal component matrices;

[0073] An orthogonal subspace construction module that uses the number of principal components and the principal component matrix to establish the subspace and orthogonal subspace corresponding to the principal components, obtaining the projection tensor of the three-dimensional spatio-temporal tensor in the orthogonal subspace to enhance target saliency and suppress the background components corresponding to the principal components;

[0074] An information filter design module for removing noise and background components remaining in the data after orthogonal subspace projection, retaining the target component data, and reconstructing the target component data into a corresponding infrared small target detection result sequence;

[0075] A target detection result output module for outputting an infrared small target detection result map.

[0076] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0077] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial descriptions of the method embodiments. The device embodiments described above are merely illustrative. The various modules in the device are a logical function division. In actual implementation, there may be other division methods. For example, multiple modules can be combined or integrated into another unit. Another point is that the connections between the modules shown or discussed can be communication connections through some interfaces, which can be electrical or other forms. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this application. A person of ordinary skill in the art can understand and implement it without creative work. Taking the publicly available real thermal infrared image sequence as an example below to illustrate the specific implementation manner to reflect the technical effects of the present invention, the specific steps in the embodiments will not be described in detail again.

[0078] Embodiment

[0079] The accompanying drawings description shown in the embodiments of the present invention can make the purpose, technical solutions and advantages of the present invention introduced more clearly and clearly. It should be noted that the specific embodiments described here are only used to explain the present invention and are not used to limit the present invention. Any equivalent replacement, improvement, etc. made within the method ideas and principles provided by the present invention should be included in the protection scope of the present invention.

[0080] In this embodiment, the effectiveness verification of the thermal infrared small target detection algorithm will be carried out through a publicly available infrared image sequence. From a qualitative perspective, the intuitive result graphs of target detection of an example frame of the infrared image sequence by the proposed method and the comparative method are used to evaluate the algorithm effectiveness; from a quantitative perspective, the 3D-ROC evaluation index system is used to evaluate the target detection ability, background suppression ability and comprehensive effectiveness performance of the algorithm. The 3D-ROC evaluation index system is derived from C.-I. Chang, "An Effective Evaluation Tool for Hyperspectral Target Detection: 3D Receiver Operating Characteristic Curve Analysis," in IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 6, pp. 5131 - 5153, June 2021, doi: 10.1109 / TGRS.2020.3021671.

[0081] It includes the target detection ability related index (TD), the background suppression ability related index (BS) and the comprehensive effectiveness related index of the detector. The specific introductions of each index are as follows in the table:

[0082] Table 1 3D-ROC Evaluation Index System

[0083]

[0084] For an open and real thermal infrared image sequence containing 100 frames, the size of all frame images is 256×256. Specifically, it shows an air-ground complex background with slow background transformation, low target movement speed, and contains highlighted buildings and iron towers with relatively complex structures. In addition, there are strong noises and clutters in the image frames, which cause certain interference to the detection of small infrared targets. Figure 3 It is an example frame of this infrared image sequence. Figure 4 It is the target detection result map of this example frame detected by the method of the present invention. From the detection result map, it can be seen that the highlighted buildings and iron towers with complex structures in the air-ground background are completely suppressed, and the saliency of the infrared targets is enhanced. To quantitatively evaluate the effective performance of the proposed algorithm, Table 2 gives the index situation of the 3D-ROC evaluation system for the small target detection results of this thermal infrared image sequence using WSLCM, GSWLCM, FAMSIS, METTR, LogTFNN, ASTTV-NTLA, 4D-TR, RCTVW and the proposed method respectively. The bold and underlined values represent the sum of the corresponding optimal AUC value and the sub-optimal AUC value respectively. Among them, WSLCM, GSWLCM and FAMSIS are infrared small target detection algorithms based on local contrast measurement, and METTR, LogTFNN, ASTTV-NTLA, 4D-TR and RCTVW are infrared small target detection algorithms based on low-rank sparse decomposition.

[0085] The sources of the above comparison algorithms are as follows:

[0086] WSLCM is from J.Han et al., "Infrared Small Target Detection Based on the Weighted Strengthened Local Contrast Measure," in IEEE Geoscience and Remote Sensing Letters, vol.18, no.9, pp.1670-1674, Sept.2021, doi:10.1109 / LGRS.2020.3004978.

[0087] GSWLCM is from Z. Qiu, Y. Ma, F. Fan, J. Huang and L. Wu, "Global Sparsity-Weighted Local Contrast Measure for Infrared Small Target Detection," in IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1-5, 2022, Art no. 7507405, doi: 10.1109 / LGRS.2022.3196433.

[0088] FAMSIS is from Y. Chen, G. Zhang, Y. Ma, J. U. Kang and C. Kwan, "Small Infrared Target Detection Based on Fast Adaptive Masking and Scaling With Iterative Segmentation," in IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1-5, 2022, Art no. 7000605, doi: 10.1109 / LGRS.2020.3047524.

[0089] METTR is from Cao, Zhaoyang, et al. "Infrared dim target detection via mode-k1k2 extension tensor tubal rank under complex ocean environment." ISPRS Journal of Photogrammetry and Remote Sensing, vol. 181, pp. 167-190, 2021, doi: 10.1016 / j.isprsjprs.2021.09.007.

[0090] LogTFNN is from X. Kong, C. Yang, S. Cao, C. Li and Z. Peng, "Infrared Small Target Detection via Nonconvex Tensor Fibered Rank Approximation," in IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1 - 21, 2022, Art no. 5000321, doi: 10.1109 / TGRS.2021.3068465.

[0091] ASTTV - NTLA is from T. Liu et al., "Nonconvex Tensor Low - Rank Approximation for Infrared Small Target Detection," in IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1 - 18, 2022, Art no. 5614718, doi: 10.1109 / TGRS.2021.3130310.

[0092] 4D - TR is from F. Wu, H. Yu, A. Liu, J. Luo and Z. Peng, "Infrared Small Target Detection Using Spatiotemporal 4 - D Tensor Train and Ring Unfolding," in IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1 - 22, 2023, Art no. 5002922, doi: 10.1109 / TGRS.2023.3288024.

[0093] RCTVW is from T. Liu, J. Yang, B. Li, Y. Wang and W. An, "Representative Coefficient Total Variation for Efficient Infrared Small Target Detection," in IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1-18, 2023, Art no. 5622918, doi: 10.1109 / TGRS.2023.3324821.

[0094] Table 2 Quantitative metrics of the detection results of the thermal infrared image instance sequence using WSLCM, GSWLCM, FAMSIS, METTR, LogTFNN, ASTTV-NTLA, 4D-TR, RCTVW and the proposed method

[0095] Method <![CDATA[AUC (D,F) > <![CDATA[AUC (D,τ) > <![CDATA[AUC (F,τ) > <![CDATA[AUC TD > <![CDATA[AUC BS > <![CDATA[AUC SNPR > <![CDATA[AUC TD-BS > <![CDATA[AUC ODP > The present invention 1.0000 0.9960 0 1.9960 1.0000 ∞ 0.9960 1.9960 WSLCM 1.0000 0.9780 5.5884e-5 1.9780 0.9999 1.7500e4 0.9779 1.9779 GSWLCM 1.0000 0.4678 3.4228e-5 1.4678 0.9999 1.3667e4 0.4678 1.4677 FAMSIS 0.9999 0.9960 1.0156e-4 1.9959 0.9998 9.8070e3 0.9959 1.9958 METTR 1.0000 0.9952 1.8162e-1 1.9952 0.8184 5.4794 0.8135 1.8135 LogTFNN 1.0000 0.9987 3.0099e-3 1.9987 0.9970 3.3180e2 0.9957 1.9957 ASTTV-NTLA 1.0000 0.9859 1.1332e-1 1.9859 0.8867 8.7001 0.8726 1.8726 4D-TR 1.0000 0.9960 4.0568e-2 1.9960 0.9594 2.4552e1 0.9554 1.9554 RCTVW 0.9196 0.9084 1.9849e-2 1.8280 0.8998 4.5764e1 0.8885 1.8081

[0096] Figure 5 Fig. 10 is the small target detection result diagram of a thermal infrared image example frame by the present invention, WSLCM, GSWLCM, FAMSIS, METTR, LogTFNN, ASTTV-NTLA, 4D-TR and RCTVW. It can be seen from the qualitative results that except for GSWLCM, the selected comparison algorithms can basically detect small targets, but the background suppression ability is weak, and there are some clutters and noises remaining. Especially for METTR and LogTFNN, they cannot suppress the highlighted buildings. In addition, in the target detection diagram after ASTTV-NTLA detection, the background energy is still high, reflecting its weak background suppression ability. For the method proposed in the present invention, it can completely suppress the background and enhance the target saliency. It is also reflected from the quantitative metric results shown in Table 2 that the background suppression ability of the method proposed in the present invention is better than all the comparison methods, and it can completely achieve the suppression of the background and noise components. For example, AUC (F,τ) reaches 0, AUC BS reaches 1, AUC SNPR reaches ∞, indicating that the background suppression ability is very excellent; although the metrics related to the target detection ability are slightly lower than those of LogTFNN, the excellent target detection ability of LogTFNN is at the cost of sacrificing the background suppression ability, but it is still better than the vast majority of comparison algorithms. Based on the above qualitative and quantitative analyses, the infrared small target detection method based on orthogonal subspace decomposition proposed in the present invention has superior target detection ability, background suppression ability and comprehensive effectiveness.

[0097] The accompanying drawings described in the embodiments of the present invention can make the objectives, technical solutions and advantages of the present invention more clearly understood. It should be noted that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Any equivalent replacement, improvement, etc. made within the method ideas and principles provided by the present invention shall be included in the protection scope of the present invention.

Claims

1. An infrared small target detection method based on orthogonal subspace projection, characterized in that, it includes the following steps: Step 1): Utilize the spatio-temporal information of the thermal infrared images to stack the image frames in the thermal infrared image sequence in chronological order to construct a three-dimensional spatio-temporal tensor Using the spatio-temporal information of thermal infrared images, stack the sequence of N infrared image frames in chronological order to obtain the frame image D t , where t = 1, 2, …, N, to construct a three-dimensional spatio-temporal tensor with a size of n 1 ×n 2 ×n 3 , where n 1 represents the width of the front slice of 2 and n represents 3 the height of the front slice of , and n 3 = N; Step 2): Based on the three-dimensional spatio-temporal tensor build a data component analysis model, and perform principal component analysis on the three-dimensional spatio-temporal tensor to obtain m principal components and the corresponding principal component matrix; Step 3): Based on the m principal components and the corresponding principal component matrix, establish the subspaces and orthogonal subspaces corresponding to the principal components, and project the three-dimensional spatio-temporal tensor onto the orthogonal subspace to obtain the tensor the projected tensor of The specific content of step 3) is: According to the principal component matrix PC obtained in step 2) m , construct the subspaces corresponding to the m principal components, and the projection matrix corresponding to this subspace is shown in formula (4): Among them, (*) T represents the matrix transpose operator, (·) -1 represents the matrix inverse operator; Construct the orthogonal subspace corresponding to the subspace of m principal components from formula (4), and the projection matrix corresponding to this orthogonal subspace As shown in formula (5): where I represents the identity matrix; According to formula (5), the three-dimensional spatio-temporal tensor The expansion matrix obtained by expanding according to the time dimension is projected into the orthogonal subspace to obtain the projected matrix Among them, the projected matrix represents the expansion matrix projected onto the orthogonal subspace corresponding to the matrix containing m eigenvectors obtained by principal component analysis; The projected matrix is resized to an n 1 × n 2 × n 3 tensor, resulting in the projection tensor in the orthogonal subspace where reshape(*) represents a dimensionality adjustment operator, represents a three-dimensional spatio-temporal tensor which is the tensor obtained by projection in the orthogonal subspace corresponding to the principal component subspace; Step 4): Design an information filter to suppress the background components and noise components remaining in the projection tensor and retain the infrared small targets to obtain the target component tensor Step 5): Reconstruct the target component tensor into the target detection result sequence T to achieve small thermal infrared target detection.

2. The infrared small target detection method based on orthogonal subspace projection according to claim 1, characterized in that, the specific content of step 2) is: 2.1) From the perspective of data components, it is possible to model the three-dimensional spatio-temporal tensor in step 1) as a combination of m principal components and minor components, as shown in formula (1): ​ Among them, represents a three-dimensional spatio-temporal tensor, represents m principal components, represents secondary components; 2.2) Perform principal component analysis on the three-dimensional spatio-temporal tensor to determine the number m of principal components: Specifically, expand along the time dimension to obtain an expansion matrix denoted as reshape(*) represents a dimension adjustment operator that adjusts a tensor of size n 1 ×n 2 ×n 3 to an expansion matrix of size n n 1 n 2 ×n 3 where n represents the width of the expansion matrix 1 n 2 and n represents the height of the expansion matrix 3 ; ​ To determine the number \(m\) of the principal components, perform singular value decomposition on the unfolded matrix as shown in formula (2): Among them, svd(·) represents the singular value decomposition operator, denotes an 1 n 2 ×n 1 n 2 -order unitary matrix, denotes an 1 n 2 ×n 3 -order singular value matrix, denotes an 3 ×n 3 -order unitary matrix; the number of principal components m is determined by the singular values in, and the singular values in the singular value matrix are arranged in descending order. When the condition σ i+1 > ησ i is satisfied, then m is set to i, where η represents a positive coefficient, σ i represents the singular value with index i, and σ i+1 represents the singular value with index i + 1; 2.3) Three-dimensional spatio-temporal tensor The principal component matrix corresponding to the m principal components is determined as follows: Unfolded matrix is centralized, and the covariance matrix Cov corresponding to the centralized data is subjected to eigenvalue decomposition as shown in formula (3): [V, E] = eig(Cov) (3) Among them, eig(·) represents the eigenvalue decomposition operator, the obtained V represents the eigenvector matrix, and E represents the eigenvalue matrix; thus, the first m eigenvectors are extracted from the eigenvector matrix V as m principal components, denoted as the principal component matrix PC m , and thus the matrix is defined represents the matrix containing m eigenvectors obtained by principal component analysis.

3. The infrared small target detection method based on orthogonal subspace projection according to claim 1, characterized in that, the specific content of step 4) is: Design information filter for orthogonal subspace projection tensor For each positive slice Keep the top k largest element values, where k is a positive constant set to distinguish residual background and noise components from target components, and set the remaining element values to 0 to obtain the target component tensor Target component tensor For each positive slice As shown in formula (8): Among them, represents the i-th front slice of the tensor , represents the designed information filter, which projects the index values belonging to the set Ω in the front slice 1:k to themselves, and projects all other elements to zero, represents the set Ω of the index values of the top k largest element values in 1:k , and 1 ≤ i ≤ n 3 .

4. The infrared small target detection method based on orthogonal subspace projection according to claim 3, characterized in that, the specific content of step 5) is: The target component tensor obtained from formula (8) Extract the i-th positive slice as the thermal infrared small target detection result T of the i-th frame of the original image i , from which n 3 thermal infrared small target detection result maps are obtained to form the target detection result sequence T, realizing the detection of thermal infrared small targets 5. An infrared small target detection device based on orthogonal subspace projection for implementing the method according to claim 1, characterized in that, it includes: A reconstructed infrared image module that stacks the image frames in the original infrared image sequence in chronological order to construct a three-dimensional spatio-temporal tensor, laying a data foundation for subsequent principal component extraction and orthogonal subspace construction; A principal component determination module that establishes a data component analysis model based on the three-dimensional spatio-temporal tensor and performs principal component analysis on the three-dimensional spatio-temporal tensor to obtain m principal components and the corresponding principal component matrix; An orthogonal subspace construction module that uses the number of principal components and the principal component matrix to establish the subspace and orthogonal subspace corresponding to the principal components, obtaining the projection tensor of the three-dimensional spatio-temporal tensor in the orthogonal subspace to enhance target saliency and suppress the background components corresponding to the principal components; An information filter design module for removing the noise and background components remaining in the data after orthogonal subspace projection, retaining the target component data, and reconstructing the target component data into a corresponding infrared small target detection result sequence; A target detection result output module for outputting an infrared small target detection result map.

Images