Time-sensitive small target detection method and device for infrared scene based on space-time tensor

Abstract

The present invention discloses a method and device for detecting time-sensitive small infrared targets based on space-time tensors, which can solve the technical problem of time-sensitive small infrared target detection in complex scenes that do not meet the low-rank assumption. The method includes: constructing a first problem model for solving low-rank background component information and sparse target component information of infrared images; determining the prior weight of the infrared image based on the corner information and edge information of the infrared image, and converting the first problem model into a second problem model; constructing a low-rank sparse matrix by using space-time local sampling to obtain the tensor of the original infrared image corresponding to the second problem model; substituting the tensor of the infrared image to be processed into the second problem model to separate the background component represented by the low-rank matrix information of the infrared image to be processed and the target component represented by the sparse component information; taking the target component as the target image, processing the target image to obtain the detection result.

Description

Technical Field

[0001] The present invention relates to the field of infrared imaging, and in particular to a method and device for detecting time-sensitive small targets in infrared scenes based on space-time tensors. Background Art

[0002] Due to the uniqueness of the wavelength band used by infrared imaging, its operation mode is unique. Compared with visible light imaging, infrared imaging is not restricted by the environment, is easy to conceal and can work all day long. These advantages make infrared imaging of great significance in applications such as early warning and space target detection, and infrared target detection is a key technology.

[0003] However, existing infrared target detection technology still faces huge challenges, especially in the detection of weak infrared targets. The difficulties mainly include: (1) The infrared imaging distance is long, and the target is mostly a weak point target after imaging, resulting in the lack of obvious and analyzable texture and geometric structure information of the target. (2) Infrared imaging is easily affected by the environment, and the target is often submerged in the clutter generated by scenes such as the sea of ​​clouds. At the same time, after the target has traveled a long distance, the image is weak, resulting in a low signal-to-noise ratio of the target, making subsequent detection more difficult. (3) Objects on the earth's surface that also have high radiation, such as cumulus clouds, forest fires, and frozen water bodies, may be imaged together with the target, which will cause a large number of false alarms and also put higher requirements on the detection algorithm. (4) The target is not a static target. The result obtained by continuous sampling of the camera is usually a series of multi-frame images, that is, the target and the background may move at the same time. In addition, the movement of the target and the background in the image generally does not have obvious regularity.

[0004] Generally speaking, infrared target detection methods can be divided into two categories: detect-before-track (DBT) and track-before-detect (TBD). DBT focuses on the information in a single frame. Its idea is to detect the target from each frame image, thereby detecting the target of the entire data set. TBD focuses on the information combination between multiple frames. However, the TBD method requires the accumulation of information from multiple frames, has low timeliness, and has relatively high requirements for hardware equipment, so it is less practical. At present, the DBT method has higher computational efficiency and stronger applicability, and is the focus of research by scholars from various countries. DBT methods can be mainly divided into the following three categories: (1) methods based on the background consistency assumption; (2) methods based on the human visual system (HVS) to extract salient features; (3) methods based on sparse and low-rank matrix recovery. The method based on the background consistency assumption assumes that the background area of ​​the image is uniform, regards the target as a point or block pixel that destroys the background uniformity, and separates the background and the target by filtering. This method is simple in principle and fast in speed, but has low detection accuracy. The HVS-based method is inspired by the contrast mechanism of the human visual system. It uses a sliding window to calculate the contrast of each pixel, obtains the local contrast saliency map of the entire image, and obtains the final detection result through a simple filtering method. The HVS-based method has high computational efficiency and a simple and easy-to-understand principle, but it is difficult to accurately detect the target when facing a complex background, and is very sensitive to noise points, resulting in a high false alarm rate in the detection result. The method based on sparse and low-rank matrix recovery is based on the characteristics of infrared images. It assumes that the image background is low-rank and the target is sparse, and establishes that the image is composed of noise, background, and target components. The final detection result is obtained by iteratively solving the optimization algorithm. In recent years, the concept of tensor has also been applied to infrared small target detection, so that the image information can be better applied to the matrix recovery process, improving the detection accuracy and computational efficiency.

[0005] Recently, Zhang et al. proposed an infrared small target detection method based on partial sum of the tensor nuclear norm (PSTNN), which has greatly improved accuracy, robustness, and time. In order to solve the problem that the target is prone to excessive shrinkage and possible disappearance, the algorithm introduces a local prior mapping into the model, and then uses tensor singular value decomposition (T-SVD) and alternating direction method of multipliers (ADMM) to optimize and solve the detection results. The algorithm flow is as follows: Figure 1 shown.

[0006] However, most detection algorithms, including PSTNN, currently use only the information of a single frame image, and most of them are oriented to relatively simple scenes. In complex scenes with simultaneous target movement, the detection results still need to be improved. In addition, the defects of PSTNN are: (1) The prior weight calculation method of PSTNN works better for images with simple backgrounds, but it cannot effectively suppress the background in complex scenes, as shown in Figure 4(B). The false alarm rate of the detection results is high and the accuracy is low. (2) PSTNN is oriented to the situation of a single frame with a relatively simple background. The tensor it constructs only uses spatial information and does not use temporal information. This makes it unable to meet the constraint assumption of low background rank when facing complex scenes, and thus cannot obtain good detection results, as shown in Figure 5(B). (3) The singular values ​​of the matrix represent different components in the image, and their weights are not the same. However, PSTNN assumes that all singular values ​​have the same weight and uses the sum of some singular values ​​to directly approximate the rank of the background, which is not very reasonable. In addition, PSTNN truncates the rank by a fixed energy ratio, which makes its scene generalization ability weak. (4) Since the inter-frame information is not used, the tensor model constructed by PSTNN still has a lot of background components after solving complex scenes. At the same time, the restored target has a certain degree of loss, so its contrast with the background components is not high, resulting in a large number of false alarms in the post-processing results directly using threshold processing, making the detection results relatively poor. Summary of the invention

[0007] In view of this, the present invention provides a method and device for time-sensitive small target detection in infrared scenes based on space-time tensor, which can solve the technical problem of time-sensitive small target detection in complex scenes that do not meet the low-rank assumption.

[0008] In order to solve the above technical problems, the present invention is implemented as follows.

[0009] A method for detecting time-sensitive small targets in infrared scenes based on space-time tensors, wherein the time-sensitive small targets refer to targets with pixels in the range of 2x2 to 9x9, or targets with pixel areas not exceeding 0.15% of the image, and time-sensitive targets refer to targets whose states are changing and whose attack time is very sensitive, or targets whose targets change rapidly over time and have strong mobility. The method comprises the following steps:

[0010] Step S1: constructing an infrared image model, wherein the infrared image model is used to characterize the background and target of the infrared image; based on the infrared image model, constructing a problem model for solving low-rank background component information and sparse target component information of the infrared image, which is recorded as the first problem model;

[0011] Step S2: based on the corner point information and edge information of the infrared image, assigning different weights to the corner point information and the edge information to determine the prior weight of the infrared image; extracting prior weight information based on the prior weight, and converting the first problem model into a second problem model;

[0012] Step S3: obtaining a sequence of infrared images to be processed, and for each infrared image in the sequence of infrared images to be processed: taking the infrared image as an original infrared image, and obtaining a tensor of the original infrared image corresponding to the second problem model by adopting a spatiotemporal local sampling method based on the original infrared image and the saliency image after assigning different weights to the original infrared image;

[0013] Step S4: for each infrared image in the infrared image sequence to be processed: substituting the tensor of the infrared image to be processed into the second problem model, optimizing and solving the second problem model, and separating the background component represented by the low-rank matrix information of the infrared image to be processed and the target component represented by the sparse component information;

[0014] Step S5: taking the target component as a target image, processing the target image to obtain a detection result.

[0015] Preferably, the infrared image model is

[0016] D=B+T+N (Formula 1)

[0017] Among them, D, B, T, N∈R m*n*k , respectively represent the input image tensor, background tensor, target tensor and noise tensor, m is the size of the first dimension, n is the size of the second dimension, k is the size of the third dimension, and R is a real number set;

[0018] Based on the infrared image model, a problem model for solving low-rank background component information and sparse target component information of the infrared image is constructed, which is recorded as the first problem model. The first problem model is:

[0019]

[0020] Among them, rank is the rank of the tensor, that is, the number of non-zero singular values, λ is the penalty factor, which is a positive number, and ||T||0 is the L0 norm of the target.

[0021] Preferably, the local prior weight W P (x,y) is:

[0022]

[0023] Among them, a is the weight of the corner indicator, b is the weight of the edge indicator, (x, y) is the coordinate of the pixel in the image, C is the corner indicator, and E is the edge indicator;

[0024] The normalized local prior weight is

[0025] Among them, w min 、w max Represents all local prior weights W P Minimum and maximum values ​​in (x,y);

[0026] The weight W is:

[0027] W=W sw *W rec (Formula 8)

[0028] Among them, W rec is the reciprocal of the corresponding element of the normalized local prior weight, c is a non-negative constant, ε is a positive number less than a preset threshold, and T represents the target component of the kth decomposition;

[0029] Then the second problem model is:

[0030]

[0031] in, represents Hadamard, ||B|| LPTNN is an approximate substitute for the background rank based on the Laplace operator, λ is the penalty factor, is the Hadamard product of the target image and the weighted W.

[0032] Preferably, the step S3, wherein the tensor of the original infrared image corresponding to the second problem model is obtained by adopting a spatiotemporal local sampling method based on the original infrared image and the saliency image after assigning different weights to the original infrared image, comprises:

[0033] The original infrared image is determined as the current frame, and n frames immediately preceding the current frame and n frames immediately following the current frame in the infrared image sequence to be processed are obtained as adjacent frames; if the number of frames preceding the current frame is less than n, the n frames following the current frame are taken as adjacent frames for the current frame; if the number of frames following the current frame is less than n, the n frames preceding the current frame are taken as adjacent frames for the current frame;

[0034] In the current frame, the entire original infrared image is used as a slider sliding area, and a slider is set to obtain a block image of the slider area. The slider has a fixed size and a sliding step length. The slider size is m*n, the sliding step length is d, and the sliding mode is sliding from the upper left corner of the original infrared image to the lower right corner;

[0035] For each sliding position of the slider in the entire original infrared image of the current frame: obtain the position of the slider in the entire original infrared image of the current frame, determine the neighborhood around the slider position, and determine the point point q slider sizes away from the center point of the slider in the neighborhood pos,num , with point pos,num As the center point, get the slider block with the same size and direction as the slider pos,num ; In adjacent frames and block pos,num Corresponding positions, get the slider blocks of each adjacent frame image,pos,num ; Wherein, 1≤pos≤POSITION, POSITION is the number of slider positions, 1≤num≤NUM, NUM is the number of sliders corresponding to the sliding position pos, 1≤image≤NUM1, NUM1 is the number of adjacent frames;

[0036] For each frame of the adjacent frame: move the slider block image,pos,num Arrange them in order from left to right and from top to bottom to form a block image of the frame; obtain position information corresponding to the current frame and the block image to form a block image of the current frame;

[0037] Obtain the block images corresponding to the current frame and each adjacent frame, and arrange the block images in the time sequence of their corresponding frames to form a tensor D.

[0038] A time-sensitive small target detection device for infrared scenes based on space-time tensor, wherein the time-sensitive small target refers to a target with pixels in the range of 2x2 to 9x9, or a target with a pixel area not exceeding 0.15% of the image, and the time-sensitive target refers to a target whose state is changing and the time to attack it is very sensitive, or a target that changes rapidly over time and has strong mobility, and the device comprises:

[0039] A first model building module: configured to build an infrared image model, wherein the infrared image model is used to characterize the background and target of the infrared image; based on the infrared image model, a problem model for solving low-rank background component information and sparse target component information of the infrared image is built, which is recorded as a first problem model;

[0040] The second model building module is configured to assign different weights to the corner point information and the edge information of the infrared image based on the corner point information and the edge information, and determine the prior weight of the infrared image; extract the prior weight information based on the prior weight, and convert the first problem model into a second problem model;

[0041] A tensor acquisition module is configured to acquire a sequence of infrared images to be processed, and for each infrared image in the sequence of infrared images to be processed: the infrared image is used as an original infrared image, and a tensor of the original infrared image corresponding to the second problem model is obtained by using a spatiotemporal local sampling method based on the original infrared image and a saliency image after different weights are assigned to the original infrared image;

[0042] A decomposition module: configured to, for each infrared image in the infrared image sequence to be processed: substitute the tensor of the infrared image to be processed into the second problem model, and optimize and solve the second problem model to separate the background component represented by the low-rank matrix information of the infrared image to be processed and the target component represented by the sparse component information;

[0043] Detection module: configured to take the target component as a target image, process the target image, and obtain a detection result.

[0044] Preferably, the infrared image model is

[0045] D=B+T+N (Formula 1)

[0046] Among them, D, B, T, N∈R m*n*k , respectively represent the input image tensor, background tensor, target tensor and noise tensor, m is the size of the first dimension, n is the size of the second dimension, k is the size of the third dimension, and R is a real number set;

[0047] Based on the infrared image model, a problem model for solving low-rank background component information and sparse target component information of the infrared image is constructed, which is recorded as the first problem model. The first problem model is:

[0048]

[0049] Among them, rank is the rank of the tensor, that is, the number of non-zero singular values, λ is the penalty factor, which is a positive number, and ||T||0 is the L0 norm of the target.

[0050] Preferably, the local prior weight W P (x,y) is:

[0051]

[0052] Among them, a is the weight of the corner indicator, b is the weight of the edge indicator, (x, y) is the coordinate of the pixel in the image, C is the corner indicator, and E is the edge indicator;

[0053] The normalized local prior weight is

[0054] Among them, w min、w max Represents all local prior weights W P Minimum and maximum values ​​in (x,y);

[0055] The weight W is:

[0056] W=W sw *W rec (Formula 8)

[0057] Among them, W rec is the reciprocal of the corresponding element of the normalized local prior weight, c is a non-negative constant, ε is a positive number less than a preset threshold, and T represents the target component of the kth decomposition;

[0058] Then the second problem model is:

[0059]

[0060] in, represents Hadamard, ||B|| LPTNN is an approximate substitute for the background rank based on the Laplace operator, λ is the penalty factor, is the Hadamard product of the target image and the weighted W.

[0061] Preferably, the obtaining of the tensor of the original infrared image corresponding to the second problem model by adopting a spatiotemporal local sampling method based on the original infrared image and the saliency image after assigning different weights to the original infrared image comprises:

[0062] The original infrared image is determined as the current frame, and n frames immediately preceding the current frame and n frames immediately following the current frame in the infrared image sequence to be processed are obtained as adjacent frames; if the number of frames preceding the current frame is less than n, the n frames following the current frame are taken as adjacent frames for the current frame; if the number of frames following the current frame is less than n, the n frames preceding the current frame are taken as adjacent frames for the current frame;

[0063] In the current frame, the entire original infrared image is used as a slider sliding area, and a slider is set to obtain a block image of the slider area. The slider has a fixed size and a sliding step length. The slider size is m*n, the sliding step length is d, and the sliding mode is sliding from the upper left corner of the original infrared image to the lower right corner;

[0064] For each sliding position of the slider in the entire original infrared image of the current frame: obtain the position of the slider in the entire original infrared image of the current frame, determine the neighborhood around the slider position, and determine the point point q slider sizes away from the center point of the slider in the neighborhood pos,num , with point pos,numAs the center point, get the slider block with the same size and direction as the slider pos,num ; In adjacent frames and block pos,num Corresponding positions, get the slider blocks of each adjacent frame image,pos,num ; Wherein, 1≤pos≤POSITION, POSITION is the number of slider positions, 1≤num≤NUM, NUM is the number of sliders corresponding to the sliding position pos, 1≤image≤NUM1, NUM1 is the number of adjacent frames;

[0065] For each frame of the adjacent frame: move the slider block image,pos,num Arrange them in order from left to right and from top to bottom to form a block image of the frame; obtain position information corresponding to the current frame and the block image to form a block image of the current frame;

[0066] Obtain the block images corresponding to the current frame and each adjacent frame, and arrange the block images in the time sequence of their corresponding frames to form a tensor D.

[0067] Beneficial effects:

[0068] (1) The present invention adopts a non-convex approximation model based on the Laplace operator, which can better approximate the rank of the tensor represented by the L0 norm compared to the L1 norm, which is beneficial to better approximate the background component of the image. Compared with the norm of the log operator, the Laplace operator can better approximate the L0 norm when the singular value is small.

[0069] (2) The prior weights of the present invention adopt the weighted geometry of edges and corners, which is beneficial to highlighting the target and suppressing the background. It overcomes the problem that the existing technology only considers the characteristics of corner points to enhance the contrast of corner points, but causes missed detection when the target corner points are around the edge of the image.

[0070] (3) The present invention chooses to adopt a space-time joint method, adopts a multi-frame local slider sampling method, divides the entire image into multiple block images, and then combines the multi-frame sliders around each image block to form a tensor, thereby improving the low rank and thus improving the high false alarm rate problem of the previous method. It overcomes the problem in the prior art that for some infrared images with complex backgrounds and small targets, especially for the situation where the position of the target is difficult for the human eye to distinguish, the non-local similarity of the background is not particularly high, and the matrix or tensor obtained by the slider method of a single image cannot meet the low rank requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 It is a flow chart of the PSTNN algorithm in the prior art;

[0072] Figure 2 It is a schematic diagram of sampling of multi-frame image blocks;

[0073] Figure 3 It is a flow chart of a method for detecting time-sensitive small targets in infrared scenes based on space-time tensor;

[0074] FIG4(A) is a schematic diagram of a first original image;

[0075] FIG4(B) is a schematic diagram of image saliency obtained by PSTNN weight calculation;

[0076] FIG4(C) is a schematic diagram of image saliency obtained by calculating using only edge indicators;

[0077] FIG4(D) is a schematic diagram of image saliency obtained by calculating using only corner point indicators;

[0078] FIG4(E) is a schematic diagram of image saliency obtained by the weighted calculation method of the present invention;

[0079] FIG5(A) is a schematic diagram of a second original image;

[0080] FIG5(B) is a schematic diagram of a single frame detection result;

[0081] FIG. 5(C) is a schematic diagram of the detection results of an embodiment of the present invention. DETAILED DESCRIPTION

[0082] definition:

[0083] Complex infrared scenes: mainly refer to complex imaging environments and complex motion states. In many application scenarios in the military field, natural and human factors often make the imaging environment complex. Complex motion states: Targets in the military field often have the characteristics of high speed and strong maneuverability, and infrared detectors may also be installed on various motion platforms (ships, aircraft, satellites). Therefore, after imaging, the targets and background in the image may move along irregular trajectories.

[0084] Time-sensitive weak and small targets: The so-called weak and small targets refer to the small size of the targets in the image due to the long imaging distance; considering the noise of the imaging system and the interference of background clutter, the signal-to-clutter ratio of the targets in the image is low; therefore, the targets appear as weak and small targets after long-distance infrared imaging. The main manifestations are few features and low intensity. The time-sensitive characteristic means that the target state is moving, and the time to attack it is very sensitive, or it is highly mobile and changes very quickly over time. The time-sensitive weak and small targets, where weak and small targets refer to targets with pixels in the range of 2x2 to 9x9, or targets with a pixel area not exceeding 0.15% of the image, time-sensitive targets refer to targets whose state is changing, and the time to attack it is very sensitive, or targets that change rapidly over time and are highly mobile.

[0085] The present invention is described in detail below in conjunction with the accompanying drawings and embodiments.

[0086] The present invention proposes a method for detecting time-sensitive weak small targets in infrared scenes based on space-time tensors. The method uses the time domain information of sequence images to construct a tensor that meets the low-rank assumption, and then optimizes and solves it. At the same time, suitable prior weights are established for complex scenes to improve the results of low-rank sparse decomposition. In addition, multi-scale contrast filtering is introduced to address the problem of target loss after image decomposition, and the contrast between the target and the residual background components is enhanced, thereby improving the final detection result after threshold segmentation. For time-sensitive moving targets in complex scenes, the present invention changes the tensor construction method on the basis of the PSTNN model, and uses the information between adjacent frames of the sequence image to obtain more spatiotemporal information to solve the problem when the background is complex and does not meet the low-rank assumption. At the same time, a prior weight suitable for complex scenes is constructed, and a non-convex approximation model based on the Laplace operator is introduced to weight the singular values, thereby improving the adaptability of the PSTNN algorithm to complex scenes. In addition, in the post-processing process, the contrast of the image decomposition result is firstly improved, and then adaptive binarization is performed to obtain the final detection result, thereby improving the loss problem of the low-rank sparse decomposition algorithm for the target.

[0087] like Figure 3 As shown, the present invention provides a time-sensitive small target detection method for infrared scenes based on space-time tensors, wherein the time-sensitive small target refers to a target with pixels in the range of 2x2 to 9x9, or a target with a pixel area not exceeding 0.15% of the image, and the time-sensitive target refers to a target whose state is changing and the time to attack it is very sensitive, or a target that changes rapidly over time and has strong mobility. The method comprises the following steps:

[0088] Step S1: constructing an infrared image model, wherein the infrared image model is used to characterize the background and target of the infrared image; based on the infrared image model, constructing a problem model for solving low-rank background component information and sparse target component information of the infrared image, which is recorded as the first problem model;

[0089] Step S2: based on the corner point information and edge information of the infrared image, assigning different weights to the corner point information and the edge information to determine the prior weight of the infrared image; extracting prior weight information based on the prior weight, and converting the first problem model into a second problem model;

[0090] Step S3: obtaining a sequence of infrared images to be processed, and for each infrared image in the sequence of infrared images to be processed: taking the infrared image as an original infrared image, constructing a low-rank sparse matrix by using a spatiotemporal local sampling method based on the original infrared image and the saliency image after assigning different weights to the original infrared image, and obtaining a tensor of the original infrared image corresponding to the second problem model;

[0091] The spatiotemporal local sampling is:

[0092] The original infrared image is determined as the current frame, and n frames immediately preceding the current frame and n frames immediately following the current frame in the infrared image sequence to be processed are obtained as adjacent frames; if the number of frames preceding the current frame is less than n, the n frames following the current frame are taken as adjacent frames for the current frame; if the number of frames following the current frame is less than n, the n frames preceding the current frame are taken as adjacent frames for the current frame;

[0093] In the current frame, the entire original infrared image is used as a slider sliding area (the edge portion copies the nearest neighboring image), and a slider is set to obtain a block image of the slider area. The slider has a fixed size and a sliding step. Assume that the slider size is m*n, the sliding step is d, and the sliding mode is from the upper left corner of the original infrared image to the lower right corner;

[0094] For each sliding position of the slider in the entire original infrared image of the current frame: obtain the position of the slider in the entire original infrared image of the current frame, determine the neighborhood around the slider position, and determine the point point q slider sizes away from the center point of the slider in the neighborhood pos,num , with point pos,num As the center point, get the slider block with the same size and direction as the slider pos,num ; In adjacent frames and block pos,num Corresponding positions, get the slider blocks of each adjacent frame image,pos,num ; Wherein, 1≤pos≤POSITION, POSITION is the number of slider positions, 1≤num≤NUM, NUM is the number of sliders corresponding to the sliding position pos, 1≤image≤NUM1, NUM1 is the number of adjacent frames;

[0095] For each frame of the adjacent frame: move the slider block image,pos,num Arrange them in order from left to right and from top to bottom to form a block image of the frame; obtain position information corresponding to the current frame and the block image to form a block image of the current frame;

[0096] Obtain the block images corresponding to the current frame and each adjacent frame, and arrange the block images in the time sequence of their corresponding frames to form a tensor D.

[0097] Step S4: for each infrared image in the infrared image sequence to be processed: substituting the tensor of the infrared image to be processed into the second problem model, optimizing and solving the second problem model, and separating the background component represented by the low-rank matrix information of the infrared image to be processed and the target component represented by the sparse component information;

[0098] Step S5: taking the target component as a target image, processing the target image to obtain a detection result.

[0099] In step S1, the infrared image model is D=B+T+N (Formula 1)

[0100] Among them, D, B, T, N∈R m*n*k , respectively represent the input image tensor, background tensor, target tensor and noise tensor, m is the size of the first dimension, n is the size of the second dimension, k is the size of the third dimension, and R is a real number set;

[0101] Based on the infrared image model, a problem model for solving low-rank background component information and sparse target component information of the infrared image is constructed, which is recorded as the first problem model. The first problem model is:

[0102]

[0103] Among them, rank is the rank of the tensor, that is, the number of non-zero singular values, λ is the penalty factor, which is a positive number, and ||T||0 is the L0 norm of the target.

[0104] In this embodiment, based on the assumption that the background is low-rank and the target is sparse, the target tensor T is actually a sparse tensor, that is, it satisfies ||T||0<K, where K is a constant determined by the complexity of the image and represents the degree of sparseness. At the same time, in infrared imaging, it is generally assumed that the noise is additive Gaussian white noise, that is, it satisfies ||N|| F <δ,||N|| F represents the Frobenius norm, and for δ>0, we have ||DBT|| F <δ. Therefore, the solution to the image background and target components can be described as the tensor robust principle component analysis (TRPCA) problem shown in Formula 2, that is, the low-rank and sparse components of the image are decomposed to obtain Formula 2.

[0105] The step S2, based on the corner point information and edge information of the infrared image, assigns different weights to the corner point information and the edge information to determine the priori weight of the infrared image, including: C is a corner point indicator, that is, the weight assigned by the corner point information, that is, C=W cs (Formula 4), the weight diagram is shown in Figure 4 (D), E is the edge indicator, that is, the weight given by the edge information, E (x, y) = λ1-λ2, the weight diagram is shown in Figure 4 (C), and the prior weight of the infrared image of this method is determined as The weight diagram is shown in Figure 4(E); where W P is the weighted weight, a is the weight of the corner indicator, b is the weight of the edge indicator, (x, y) is the coordinate of the pixel in the image, C is the corner indicator, E is the edge indicator, λ1 and λ2 are the eigenvalues ​​contained in the structure tensor of each pixel position in the infrared image.

[0106] The step S2, extracting prior weight information based on the prior weight, and converting the first problem model into a second problem model, includes:

[0107] In this embodiment, the traditional tensor nuclear norm (TNN) and sum of nuclear norms (SNN) consider that the weights of singular values ​​are the same, which is unreasonable. In addition, the error of using 1 norm to approximate the rank is relatively large. Therefore, the present invention uses a more approximate tensor fiber nuclear norm non-convex approximation model based on Laplace operation to approximate the low-rank part of the image, that is,

[0108]

[0109] Among them, σ j (X i ) is the singular value of the jth frontal slice, p is the number of singular values, ε is a small positive number to ensure the operation, and n3 is the size of the third dimension of the tensor. Secondly, in order to solve the problem of unbalanced coefficients of singular values, the present invention adopts a re-weighted solution:

[0110]

[0111] Among them, W sw is the weighting coefficient, is the weighted coefficient value obtained after the k+1th iteration, c is a non-negative constant, ε is a positive number less than the preset threshold value to avoid the denominator being 0, k represents the kth iteration, and T represents the target component of the kth decomposition. At the same time, in order to solve the problem that there is still some background in the decomposed target component, the present invention establishes appropriate weights to enhance the recovery effect of sparse components based on prior information. The structure tensor of each pixel position in the image contains two eigenvalues ​​λ1 and λ2. When the pixel is located in the corner area, the eigenvalue satisfies λ1>>λ2≈0, when the pixel is located in the edge area, it satisfies λ1≥λ2≈0; when the pixel is located in the flat area, it satisfies λ1≈λ2≈0. Therefore, the intensity function of the corner point in the image can be expressed as:

[0112]

[0113] Where (x, y) represents the position of the pixel in the image, det represents the determinant of the matrix, tr represents the trace of the matrix, and ST represents the result tensor. At the same time, the edge strength function is represented by E = λ1-λ2. Therefore, the local prior weight established by the present invention is

[0114]

[0115] The normalized weight is

[0116]

[0117] Among them, w min 、w max Represents W P The minimum and maximum values ​​in (x, y). The prior weights used in the present invention can better highlight the target and better suppress the edge corners in the background.

[0118] Combining the two weights, the weighting established by the present invention is:

[0119] W=W sw *W rec (Formula 8)

[0120] Among them, W rec It is W P The reciprocal of the corresponding element of (x,y), Then the first problem model is converted into the second problem model, and the second problem model is:

[0121]

[0122] in, represents Hadamard, ||B|| LPTNN is an approximate substitute for the background rank based on the Laplace operator, λ is the penalty factor, is the Hadamard product of the target image and W.

[0123] In this embodiment, two eigenvalues ​​corresponding to the position of each pixel are calculated, and the prior weight of each pixel is calculated by formula (6) to obtain a saliency map as shown in FIG4(E).

[0124] In step S3, in this embodiment, in order to solve the problem of detecting time-sensitive targets in complex scenes, the previous method of obtaining the final tensor by using a slider of a single frame image is changed, and the n frames before and after the image of the current frame are taken, that is, a total of (2*n+1) frames of images are used, and the current slider is taken as the main body, and sliders with a distance of q sliders around the current slider are taken to obtain multiple sliding windows, and at the same time, the same number of sliders are taken at the same position of the previous and next multiple frames to obtain the tensor of the current slider of the final current frame, such as Figure 2 As shown, in the present invention, n is 2 and p is 1. Figure 2 In the example, the image in the middle is the current frame, the solid color filled part is the current slider, and the front and back correspond to adjacent frames. In the present invention, a total of 45 block images are obtained to form the tensor of the current slider, taking the current slider as the main body.

[0125] The step S4, wherein the second problem model is decomposed into two sub-problems:

[0126]

[0127]

[0128] Where k is the kth iteration, W k is the kth iteration of weight W, B k+1 is the kth iteration of the background component, y k is the kth iteration of the Lagrange multiplier, u k is the kth iteration of the trade-off factor, T k+1 is the kth iteration of the target component.

[0129] When the iteration reaches the preset number of iterations N or the difference between the solution results of the previous and next two iterations does not exceed the preset threshold, the iteration is stopped.

[0130] The background components represented by the low-rank matrix information and the target components represented by the sparse component information of the infrared image to be processed are solved in an iterative manner, and the detection result is shown in FIG5(C).

[0131] In this embodiment, in step S4, the present invention adopts the cross-directional multiplier method to solve the established model, thereby obtaining the background and target components of the image. First, the problem is written as an augmented Lagrangian function, that is:

[0132]

[0133] Among them, y is the Lagrange multiplier, <·> represents the inner product of two tensors, and μ>0 is the penalty factor. The cross multiplication method is to fix one variable while solving the other variable, that is, the problem can be decomposed into two sub-problems:

[0134]

[0135]

[0136] Through continuous iteration, the low-rank and sparse components of the infrared image to be processed are solved, that is, the infrared image to be processed is decomposed into background and target.

[0137] In the step S5, most of the current methods use simple threshold segmentation to segment the decomposed target components and obtain the final detection result, and the detection result obtained in this way will cause the target to be lost. Therefore, the present invention first performs high-boost-based multiscale local contrast measure (HBMLCM) on the iteratively solved target image to improve the contrast between the target and the background, and then combines the threshold segmentation to achieve the purpose of more complete retention of the target.

[0138] Furthermore, other local contrast enhancement operators can be used instead of HBMLCM, such as multiscale patch-based contrast measure (MPCM).

[0139] The present invention also provides a time-sensitive small target detection device for infrared scenes based on space-time tensors, wherein the time-sensitive small target refers to a target with pixels in the range of 2x2 to 9x9, or a target with a pixel area not exceeding 0.15% of the image, and the time-sensitive target refers to a target whose state is changing and the time to attack it is very sensitive, or a target that changes rapidly over time and has strong mobility. The device comprises:

[0140] A first model building module: configured to build an infrared image model, wherein the infrared image model is used to characterize the background and target of the infrared image; based on the infrared image model, a problem model for solving low-rank background component information and sparse target component information of the infrared image is built, which is recorded as a first problem model;

[0141] The second model building module is configured to assign different weights to the corner point information and the edge information of the infrared image based on the corner point information and the edge information, and determine the prior weight of the infrared image; extract the prior weight information based on the prior weight, and convert the first problem model into a second problem model;

[0142] A tensor acquisition module is configured to acquire a sequence of infrared images to be processed, and for each infrared image in the sequence of infrared images to be processed: the infrared image is used as an original infrared image, and a low-rank sparse matrix is ​​constructed by using a spatiotemporal local sampling method based on the original infrared image and a saliency image after different weights are assigned to the original infrared image, so as to obtain a tensor of the original infrared image corresponding to the second problem model;

[0143] A decomposition module: configured to, for each infrared image in the infrared image sequence to be processed: substitute the tensor of the infrared image to be processed into the second problem model, and optimize and solve the second problem model to separate the background component represented by the low-rank matrix information of the infrared image to be processed and the target component represented by the sparse component information;

[0144] Detection module: configured to take the target component as a target image, process the target image, and obtain a detection result.

[0145] The above specific embodiments only describe the design principle of the present invention. The shapes and names of the components in the description may be different and are not limited. Therefore, those skilled in the art in the field of the present invention may modify or replace the technical solutions recorded in the above embodiments; and these modifications and replacements do not deviate from the creative purpose and technical solutions of the present invention and should all fall within the protection scope of the present invention.

Claims

1. A method for detecting time-sensitive small targets in infrared scenes based on space-time tensors, wherein the time-sensitive small targets refer to targets with pixels in the range of 2x2 to 9x9, or targets with a pixel area not exceeding 0.15% of the image, and the time-sensitive targets refer to targets whose states are changing, characterized in that: The method comprises the following steps: Step S1: constructing an infrared image model, wherein the infrared image model is used to characterize the background and target of the infrared image; based on the infrared image model, constructing a problem model for solving low-rank background component information and sparse target component information of the infrared image, which is recorded as the first problem model; Step S2: based on the corner point information and edge information of the infrared image, assigning different weights to the corner point information and the edge information to determine the prior weight of the infrared image; extracting prior weight information based on the prior weight, and converting the first problem model into a second problem model; Step S3: obtaining a sequence of infrared images to be processed, and for each infrared image in the sequence of infrared images to be processed: taking the infrared image as an original infrared image, and obtaining a tensor of the original infrared image corresponding to the second problem model by adopting a spatiotemporal local sampling method based on the original infrared image and the saliency image after assigning different weights to the original infrared image; Step S4: for each infrared image in the infrared image sequence to be processed: substituting the tensor of the infrared image to be processed into the second problem model, optimizing and solving the second problem model, and separating the background component represented by the low-rank matrix information of the infrared image to be processed and the target component represented by the sparse component information; Step S5: taking the target component as a target image, processing the target image to obtain a detection result; The infrared image model is D=B+T+N Among them, D, B, T, N∈R m*n*k , respectively represent the input image tensor, background tensor, target tensor and noise tensor, m is the size of the first dimension, n is the size of the second dimension, k is the size of the third dimension, and R is a real number set; Based on the infrared image model, a problem model for solving low-rank background component information and sparse target component information of the infrared image is constructed, which is recorded as the first problem model. The first problem model is: Among them, rank is the rank of the tensor, that is, the number of non-zero singular values, λ is the penalty factor, which is a positive number, and ||T||0 is the L0 norm of the target; Local prior weight W P (x,y) is: Among them, a is the weight of the corner indicator, b is the weight of the edge indicator, (x, y) is the coordinate of the pixel in the image, C is the corner indicator, and E is the edge indicator; The normalized local prior weight is Among them, w min 、w max Represents all local prior weights W P Minimum and maximum values ​​in (x,y); The weight W is: W=W sw *IN rec Among them, W rec is the reciprocal of the corresponding element of the normalized local prior weight, c is a non-negative constant, ε is a positive number less than a preset threshold, and T represents the target component of the kth decomposition; Then the second problem model is: in, represents Hadamard, ||B|| LPTNN is an approximate substitute for the background rank based on the Laplace operator, λ is the penalty factor, is the Hadamard product of the target image and the weighted W.

2. The method according to claim 1, characterized in that The step S3, wherein the tensor of the original infrared image corresponding to the second problem model is obtained by using a spatiotemporal local sampling method based on the original infrared image and the saliency image after assigning different weights to the original infrared image, comprises: The original infrared image is determined as the current frame, and n frames immediately preceding the current frame and n frames immediately following the current frame in the infrared image sequence to be processed are obtained as adjacent frames; if the number of frames preceding the current frame is less than n, the n frames following the current frame are taken as adjacent frames for the current frame; if the number of frames following the current frame is less than n, the n frames preceding the current frame are taken as adjacent frames for the current frame; In the current frame, the entire original infrared image is used as a slider sliding area, and a slider is set to obtain a block image of the slider area. The slider has a fixed size and a sliding step length. The slider size is m*n, the sliding step length is d, and the sliding mode is sliding from the upper left corner of the original infrared image to the lower right corner; For each sliding position of the slider in the entire original infrared image of the current frame: obtain the position of the slider in the entire original infrared image of the current frame, determine the neighborhood around the slider position, and determine the point point q slider sizes away from the center point of the slider in the neighborhood pos,num , with point pos,num As the center point, get the slider block with the same size and direction as the slider pos,num ; In adjacent frames and block pos,num Corresponding positions, get the slider blocks of each adjacent frame image,pos,num ; Wherein, 1≤pos≤POSITION, POSITION is the number of slider positions, 1≤num≤NUM, NUM is the number of sliders corresponding to the sliding position pos, 1≤image≤NUM1, NUM1 is the number of adjacent frames; For each frame of the adjacent frame: move the slider block image,pos,num Arrange them in order from left to right and from top to bottom to form a block image of the frame; obtain position information corresponding to the current frame and the block image to form a block image of the current frame; Obtain the block images corresponding to the current frame and each adjacent frame, and arrange the block images in the time sequence of their corresponding frames to form a tensor D.

3. A time-sensitive small target detection device for infrared scenes based on space-time tensor, wherein the time-sensitive small target refers to a target with pixels in the range of 2x2 to 9x9, or a target with a pixel area not exceeding 0.15% of the image, and the time-sensitive target refers to a target whose state is changing, characterized in that: The device comprises: A first model building module: configured to build an infrared image model, wherein the infrared image model is used to characterize the background and target of the infrared image; based on the infrared image model, a problem model for solving low-rank background component information and sparse target component information of the infrared image is built, which is recorded as a first problem model; The second model building module is configured to assign different weights to the corner point information and the edge information of the infrared image based on the corner point information and the edge information, and determine the prior weight of the infrared image; extract the prior weight information based on the prior weight, and convert the first problem model into a second problem model; A tensor acquisition module is configured to acquire a sequence of infrared images to be processed, and for each infrared image in the sequence of infrared images to be processed: the infrared image is used as an original infrared image, and a tensor of the original infrared image corresponding to the second problem model is obtained by using a spatiotemporal local sampling method based on the original infrared image and a saliency image after different weights are assigned to the original infrared image; A decomposition module: configured to, for each infrared image in the infrared image sequence to be processed: substitute the tensor of the infrared image to be processed into the second problem model, and optimize and solve the second problem model to separate the background component represented by the low-rank matrix information of the infrared image to be processed and the target component represented by the sparse component information; Detection module: configured to take the target component as a target image, process the target image, and obtain a detection result; The infrared image model is D=B+T+N Among them, D, B, T, N∈R m*n*k , respectively represent the input image tensor, background tensor, target tensor and noise tensor, m is the size of the first dimension, n is the size of the second dimension, k is the size of the third dimension, and R is a real number set; Based on the infrared image model, a problem model for solving low-rank background component information and sparse target component information of the infrared image is constructed, which is recorded as the first problem model. The first problem model is: Among them, rank is the rank of the tensor, that is, the number of non-zero singular values, λ is the penalty factor, which is a positive number, and ||T||0 is the L0 norm of the target; Local prior weight W P (x,y) is: Among them, a is the weight of the corner indicator, b is the weight of the edge indicator, (x, y) is the coordinate of the pixel in the image, C is the corner indicator, and E is the edge indicator; The normalized local prior weight is Among them, w min 、w max Represents all local prior weights W P Minimum and maximum values ​​in (x,y); The weight W is: W=W sw *IN rec Among them, W rec is the reciprocal of the corresponding element of the normalized local prior weight, c is a non-negative constant, ε is a positive number less than a preset threshold, and T represents the target component of the kth decomposition; Then the second problem model is: in, represents Hadamard, ||B|| LPTNN is an approximate substitute for the background rank based on the Laplace operator, λ is the penalty factor, is the Hadamard product of the target image and the weighted W.

4. The device according to claim 3, characterized in that The tensor acquisition module, wherein the tensor of the original infrared image corresponding to the second problem model is obtained by using a spatiotemporal local sampling method based on the original infrared image and the saliency image after assigning different weights to the original infrared image, comprises: The original infrared image is determined as the current frame, and n frames immediately preceding the current frame and n frames immediately following the current frame in the infrared image sequence to be processed are obtained as adjacent frames; if the number of frames preceding the current frame is less than n, the n frames following the current frame are taken as adjacent frames for the current frame; if the number of frames following the current frame is less than n, the n frames preceding the current frame are taken as adjacent frames for the current frame; In the current frame, the entire original infrared image is used as a slider sliding area, and a slider is set to obtain a block image of the slider area. The slider has a fixed size and a sliding step length. The slider size is m*n, the sliding step length is d, and the sliding mode is sliding from the upper left corner of the original infrared image to the lower right corner; For each sliding position of the slider in the entire original infrared image of the current frame: obtain the position of the slider in the entire original infrared image of the current frame, determine the neighborhood around the slider position, and determine the point point q slider sizes away from the center point of the slider in the neighborhood pos,num , with point pos,num As the center point, get the slider block with the same size and direction as the slider pos,num ; In adjacent frames and block pos,num Corresponding positions, get the slider blocks of each adjacent frame image,pos,num ; Wherein, 1≤pos≤POSITION, POSITION is the number of slider positions, 1≤num≤NUM, NUM is the number of sliders corresponding to the sliding position pos, 1≤image≤NUM1, NUM1 is the number of adjacent frames; For each frame of the adjacent frame: move the slider block image,pos,num Arrange them in order from left to right and from top to bottom to form a block image of the frame; obtain position information corresponding to the current frame and the block image to form a block image of the current frame; Obtain the block images corresponding to the current frame and each adjacent frame, and arrange the block images in the time sequence of their corresponding frames to form a tensor D.

Images