An Infrared Small Target Detection Method Based on Phase Registration and Consistency Analysis

Abstract

The present invention discloses an infrared small target detection method based on phase registration and consistency analysis, comprising the following steps: Step S1, input an infrared image sequence; Step S2, image registration; Step S3, calculate a consistency image; Step S4, extract candidate targets; Step S5, disappearance tolerance mechanism and static tolerance mechanism; Step S6, target screening and tracking; Step S7, strengthen the weight of the target area. The present invention uses a differential consistency detection algorithm enhanced by gradients to effectively suppress background interference and improve the detection accuracy of a single frame; by introducing a static tolerance mechanism and a disappearance tolerance mechanism, the detection method has extremely strong robustness to cope with irregular target motion and occlusion problems; through a modular framework and a dynamic weight adjustment mechanism, efficient detection is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of infrared image target detection, and more specifically, to an infrared small target detection method based on phase registration and consistency analysis. Background Art

[0002] Currently, single-frame methods include techniques such as local contrast, top-hat filters, low-rank and sparse decomposition, deep learning, and background suppression noise processing. These methods mainly rely on the information of a single image frame for target detection. Multi-frame methods have significant advantages in accuracy compared to single-frame methods because they can improve the stability and accuracy of target detection and tracking through the accumulation of information in the time dimension. The fundamental reason is that multi-frame methods can utilize the target motion information and time series relationships in consecutive images, effectively reduce noise interference, reduce false alarms and missed detections, and improve performance under low signal-to-noise ratio and complex background conditions. At the same time, the accumulation of multi-frame data enhances the target signal and improves the accuracy of target state estimation, especially suitable for cases where the target moves quickly or the signal is weak. Currently, common multi-frame methods are as follows:

[0003] (1) Traditional filtering methods

[0004] In the research of the TBD (Tracking Before Detection) algorithm, traditional filtering methods laid the foundation for target detection and tracking. Kalman et al. [1] first proposed the Kalman Filter for motion estimation of targets under linear state space models, but it has limitations in dealing with complex backgrounds and non-linear motions. For this reason, Julier et al. [2] proposed the Unscented Kalman Filter (UKF), which effectively addressed the non-linear problem. Subsequently, Gordon et al. [3] proposed the Particle Filter, which simulates the target state through a large number of particles and is suitable for more complex dynamic environments. To improve the efficiency of the Particle Filter, Doucet et al. [4] further proposed the Adaptive Particle Filter (APF), which improves the accuracy of target detection and tracking by dynamically adjusting the number and weights of particles.

[0005] (2) Tensor decomposition methods

[0006] With the increase in data dimension, tensor decomposition techniques have gradually occupied an important position in the TBD algorithm. Xu et al. [5] first introduced Low-rank Tensor Decomposition, which effectively separated small targets from complex backgrounds by representing sequential images as a tensor structure. Based on this research, researchers proposed Spatio-temporal Tensor Decomposition, which was further optimized by Liu et al. [6]. This method utilized temporal information to enhance the detection effect of small targets. In addition, Zhao et al. [7] proposed Tensor Sparse Decomposition, which combined sparse representation techniques to further improve the object detection performance in complex scenarios.

[0007] (3) Network structure method

[0008] In recent years, with the rise of deep learning, the TBD algorithm has obtained new development. LeCun et al. [8] proposed the Convolutional Neural Network (CNN), which provided a powerful tool for feature extraction of small targets. However, CNN has certain limitations in processing sequential images. For this reason, Hochreiter and Schmidhuber [9] proposed the Long Short-Term Memory Network (LSTM), which solved the problem of temporal information processing by introducing memory units. Further, Tran et al.

[10] proposed the 3D Convolutional Neural Network (3D CNN), which captured the motion features between multiple frames by performing convolutional operations in both the time and space dimensions. Recently, Goodfellow et al.

[11] introduced the Generative Adversarial Network (GAN), which enhanced the detection effect in low signal-to-noise ratio environments by generating a realistic target motion trajectory model.

[0009] Currently, the multi-frame method has the following defects:

[0010] (1) Most of the existing algorithms rely on the accuracy improvement brought by the analysis of multi-frame time series data, resulting in a large time overhead for the algorithms. The fundamental reason is that traditional methods often need to process long time series data, thus accumulating a high computational complexity in the time dimension;

[0011] (2) When the existing algorithms face scenarios with complex environments and severe noise, small targets often get submerged in heavy noise, leading to the loss of targets. The fundamental reasons are as follows. First, single-frame detection algorithms are difficult to effectively suppress strong object edges. Second, image noise, lens sensor noise, and deviations caused by motion can result in the failure to accurately capture or identify the motion features of small targets, thus destroying the motion laws of targets and further affecting the effect of the detection algorithm;

[0012] (3) Existing algorithms often rely on the assumption of motion continuity and cannot solve the problems of target tracking loss caused by rapid target movement, excessively slow target speed, and target occlusion. Summary of the Invention

[0013] The object of the present invention is to overcome the deficiencies of the above-mentioned existing technologies and provide an infrared small target detection method based on phase registration and consistency analysis.

[0014] To achieve the above object, the present invention adopts the following technical solutions:

[0015] An infrared small target detection method based on phase registration and consistency analysis, comprising the following steps:

[0016] Step S1, input an infrared image sequence, read and sort the sequence to be detected containing K consecutive infrared images, and initialize the displacement threshold, registration threshold, and regional area threshold;

[0017] Step S2, image registration, select the first frame as the reference image, calculate the phase correlation transformation matrix between the subsequent frames and the reference frame, perform inter-frame registration, and if the registration effect is not good, use the transformation matrix of the previous frame for correction;

[0018] Step S3, consistency image calculation, use the registered images to calculate the inter-frame consistency image, and process it based on the set threshold to suppress the noise area and enhance the possible target points;

[0019] Step S4, candidate target extraction, screen the target candidate regions from the consistency image through connected region analysis, eliminate the false alarm points with too large area or close to the edge, and sort the qualified regions according to the maximum peak value, and select the top ten candidate target points;

[0020] Step S5, disappearance tolerance mechanism, if the target is not detected within a short time but there are still possible targets near the predicted position, maintain its tracking state; static tolerance mechanism, if the target is still changing within the allowable number of frames, it is determined as a real target;

[0021] Step S6, target screening and tracking, combine the target positions of the previous few frames, calculate the motion trajectories of the candidate points, eliminate the target points deviating from the reasonable motion trajectories, and finally confirm the real target;

[0022] Step S7, target area weight enhancement, predict the position where the real target appears in the next frame, and enhance and strengthen the area range of the predicted appearance position.

[0023] Further, in step S2, the phase correlation feature in the frequency domain is used to estimate the translational displacement between image frames to achieve pixel-level alignment operation.

[0024] Further, the phase difference between two images is calculated by Fourier transform to obtain the formula for translational phase registration:

[0025] (1)

[0026] In formula (1), and respectively represent the Fourier transforms of two frames of images; is 's conjugate complex number; represents the inverse Fourier transform.

[0027] Further, in step S3, by calculating the difference consistency in multiple directions around the target point, redundant information is suppressed and the target edge features are highlighted.

[0028] Further, step S3 includes the following steps:

[0029] Step S301, calculate the pixel difference value using a predefined pixel range R and 8 directions; in each direction, the difference calculation formula is:

[0030] (2)

[0031] In formula (2), represents the difference value in the r-th range and the d-th direction; represents the pixel value of the target point, with coordinates p and q; and respectively represent the pixel offsets in different directions; K is a correction factor used to adjust the difference value;

[0032] Step S302, calculate the sum of differences within the continuous pixel range R, and multiply the sum of differences in different directions to obtain the value of the consistency coefficient C:

[0033] (3)

[0034] In formula (3), represents the difference value in the r-th range and the d-th direction.

[0035] Further, in step S5, the static tolerance mechanism includes the following steps:

[0036] Step S501, assume the displacement of the target between consecutive frames is:

[0037] (4)

[0038] In formula (4), represents the coordinate of the target in the t-th frame, Indicates the coordinates of the target in the (t - 1)-th frame;

[0039] Step S502, if the displacement of the target satisfies the following conditions:

[0040] (5)

[0041] In formula (5), is the distance threshold;

[0042] When the displacement of the target between two frames is less than , the target is considered stationary. At this time, the stationary tolerance is updated to:

[0043] (6)

[0044] Step S503, when the stationary tolerance exceeds the set maximum stationary tolerance , the target is considered stationary, but it will not be lost and will continue to be kept in the target list.

[0045] Furthermore, in step S5, the disappearance tolerance mechanism includes:

[0046] Step S504, let the disappearance tolerance of the target in the t-th frame be , and its update rule is as follows:

[0047] If the target is not matched, then ;

[0048] If the target is matched, then ;

[0049] Step S505, when the disappearance tolerance exceeds the set maximum disappearance tolerance , the target is judged to be lost; in the frames where the target is not matched, its position is predicted according to the historical displacement information of the target; let the position of the target in the (t - 1)-th frame be ,

[0050] its displacement is ,

[0051] the predicted position of the target in the t-th frame is (7).

[0052] Furthermore, step S7 includes:

[0053] Assume that the displacement of the target between two adjacent frames conforms to uniform linear motion; calculate the displacement of the target between the current frame and the previous frame to predict the position of the target in the next frame; the vicinity of the predicted position will be weighted, and the weight coefficient is determined according to the speed and direction of the target's movement. The weighting operation formula is as follows:

[0054] (8)

[0055] In formula (8), are the pixel coordinates in the image; is the predicted target position; is the weight coefficient; is the distance threshold, representing the distance range from the predicted position.

[0056] The beneficial effects of the present invention are as follows:

[0057] 1. The present invention uses the gradient-enhanced difference consistency detection algorithm to effectively suppress background interference and improve the single-frame detection accuracy.

[0058] 2. By introducing the static tolerance mechanism and disappearance tolerance mechanism, the detection method of the present invention has extremely strong robustness to cope with the problems of irregular target movement and occlusion.

[0059] 3. Through the modular framework and dynamic weight adjustment mechanism, the present invention realizes efficient detection. Description of the Drawings

[0060] Figure 1 is a schematic diagram of the overall framework of the infrared small target detection method based on phase registration and consistency analysis in this embodiment;

[0061] Figure 2 is a flowchart of the infrared small target detection method based on phase registration and consistency analysis in this embodiment. Detailed Embodiments

[0062] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0063] Embodiment: An infrared small target detection method based on phase registration and consistency analysis, as Figure 1 、 Figure 2 shown, includes the following steps:

[0064] Step S1: Input the infrared image sequence, read and sort the sequence to be detected containing K consecutive infrared images, ensure that the image sizes are consistent, and initialize key parameters such as the displacement threshold, registration threshold, and area threshold of the region.

[0065] Step S2: Image registration. Select the first frame as the reference image, calculate the phase correlation transformation matrix between the subsequent frames and the reference frame for inter-frame registration. If the registration effect is not good, use the transformation matrix of the previous frame for correction.

[0066] Furthermore, use the phase correlation characteristics in the frequency domain to efficiently estimate the translational displacement between image frames, thereby realizing pixel-level alignment operations and improving the detection stability of subsequent algorithms. By combining phase registration technology, inter-frame displacement correction is realized to enhance detection robustness and solve the influence of small lens movements on the detection of infrared small targets. Specifically:

[0067] Calculate the phase difference between two images through Fourier transform, and thus obtain the formula for translational phase registration:

[0068] (1)

[0069] In formula (1), and respectively represent the Fourier transforms of two frames of images; is 's conjugate complex number; represents the inverse Fourier transform.

[0070] By locating the peak position of the phase correlation function the translational displacement amount of the image can be accurately restored, thereby completing image alignment.

[0071] Step S3: Calculate the consistency image. Use the registered images to calculate the inter-frame consistency image and process it based on the set threshold to suppress the noise area and enhance the possible target points.

[0072] Furthermore, by calculating the difference consistency in multiple directions around the target point, redundant information is suppressed and the target edge features are highlighted. By calculating the difference consistency between the target point and the surrounding pixels, background interference is suppressed and target features are highlighted for effective analysis of local features and edge information in infrared images. Specifically:

[0073] Step S301: Calculate the pixel difference value using the predefined pixel range R and 8 directions; in each direction, the difference calculation formula is:

[0074] (2)

[0075] In formula (2), Denote the difference value in the r-th range and the d-th direction; Denote the pixel value of the target point with coordinates p and q; and Denote the pixel offsets in different directions respectively; K is a correction factor used to adjust the difference value;

[0076] For pixels outside the boundary, compensation measures are adopted, that is, the pixel value of the target point itself is used instead.

[0077] Step S302. To obtain the overall consistency feature, calculate the sum of differences within the continuous pixel range R and multiply the sum of differences in different directions to obtain the value of the consistency coefficient C:

[0078] (3)

[0079] In formula (3), Denote the difference value in the r-th range and the d-th direction.

[0080] The smaller the value of the consistency coefficient C, the higher the consistency between the target point and the surrounding pixels, thus reflecting its edge saliency in the image. Through this consistency analysis, small targets in the image can be effectively highlighted, background interference can be suppressed, and the target detection effect can be improved.

[0081] Step S4. Candidate target extraction. Screen the target candidate regions from the consistency image through connected component analysis, eliminate the false alarm points with too large area or close to the edge, and sort the qualified regions according to the maximum peak value, and select the top ten candidate target points;

[0082] Step S5. Disappearance tolerance mechanism. If the target is not detected in a short time but there may still be a target near the predicted position, maintain its tracking state; Static tolerance mechanism. If the target is still changing within the allowable number of frames, it is determined as a real target, reducing false detection and missed detection;

[0083] Among them, the static tolerance mechanism allows the detection of low-speed targets to be maintained, avoiding misjudging the disappearance of the target due to too slow speed; The disappearance tolerance mechanism is based on the assumption of the target's uniform motion in a short time, predicts the possible position after its disappearance and rematches. By introducing the disappearance tolerance mechanism and the static tolerance mechanism, the robustness of infrared small target detection is further improved.

[0084] Both the static tolerance mechanism and the disappearance tolerance mechanism are based on the assumption that the target moves in a uniform straight line in a short period of time, corresponding to the situations of target stillness and target temporary disappearance respectively, thus improving the stability of target detection in complex environments. The static tolerance is used to handle the situation where the target is misjudged as disappeared due to no significant displacement in a short period of time, while the disappearance tolerance is used to solve the problem of loss caused by target temporary occlusion or image noise. By reasonably setting these two tolerances, it is possible to better cope with the temporary stillness or disappearance of the target, avoid tracking failure, and thus achieve more stable and reliable target detection and tracking. Specifically:

[0085] The static tolerance mechanism includes the following steps:

[0086] Step S501, let the displacement of the target between consecutive frames be:

[0087] (4)

[0088] In formula (4), represents the coordinates of the target in the t-th frame, represents the coordinates of the target in the (t - 1)-th frame;

[0089] Step S502, if the displacement of the target satisfies the following condition:

[0090] (5)

[0091] In formula (5), is the distance threshold;

[0092] When the displacement of the target between two frames is less than the target is considered to be static. At this time, the static tolerance is updated to:

[0093] (6)

[0094] Step S503, when the static tolerance exceeds the set maximum static tolerance the target is considered to be static, but it will not be lost and will continue to be kept in the target list to avoid being misjudged as disappeared due to short-term stillness.

[0095] In the actual scenario, the target may not be detected for several frames due to factors such as occlusion or image noise. To cope with this situation, the disappearance tolerance is introduced. By recording the number of times the target is not matched in consecutive frames, the target is allowed to be tracked even after a short-term disappearance.

[0096] Step S504, let the disappearance tolerance of the target in the t-th frame be and its update rule is as follows:

[0097] If the target is not matched, then ;

[0098] If the target is matched, then ;

[0099] Step S505, when the disappearance tolerance exceeds the set maximum disappearance tolerance the target is determined to be lost; in the frames where the target is not matched, predict its position according to the historical displacement information of the target; assume the position of the target in the (t - 1)-th frame is ,

[0100] its displacement (i.e., velocity) is ,

[0101] the predicted position of the target in the t-th frame is (7).

[0102] Through this mechanism, tracking can still be maintained when the target disappears or is occluded for a short time, and tracking can be restored when the target reappears, thereby improving the robustness of the tracking algorithm.

[0103] Step S6, target screening and tracking, combine the target positions in the previous few frames, calculate the motion trajectories of the candidate points, eliminate the target points that deviate from the reasonable motion trajectories, and finally confirm the real target;

[0104] Step S7, target area weight strengthening, predict the position where the real target will appear in the next frame, and strengthen and enhance the area range of the predicted appearance position.

[0105] Image alignment and dynamic compensation are achieved through phase registration, providing a stable basis for target detection; combining gradient analysis and weight enhancement techniques, the pixel characteristics of the target area are strengthened, making the response of small targets more prominent in the consistent image. Subsequently, connected component analysis is performed on the weight-enhanced consistent image, and through the dual constraints of area and margin, the candidate areas that meet the target characteristics are accurately screened. These candidate areas are sorted according to the peak intensity, the centroid is extracted as the candidate target point, and the attributes such as the motion state and matching information are recorded in a structured manner, laying a data foundation for the running tracking and target management of the candidates.

[0106] Specifically, assume that the displacement of the target between two adjacent frames conforms to uniform linear motion; by calculating the displacement amount between the current frame and the previous frame of the target, the position of the target in the next frame is speculated; the area near the predicted position will be weighted, and the weight coefficient is determined according to the speed and direction of the target motion. The weighting operation formula is as follows:

[0107] (8)

[0108] In formula (8), are the pixel coordinates in the image; is the predicted target position; is the weight coefficient; is the distance threshold, representing the distance range from the predicted position. If the distance between a certain pixel and the predicted position is within the threshold then the weight of this point will be enhanced, thereby improving the detection priority of the target.

[0109] This embodiment adopts a modular framework structure. First, by selecting the first ten peaks in each frame of the image as candidate target points, it can select the most representative targets from multiple potential regions, thereby avoiding the misdetection problem that may be caused by a single peak, enhancing the accuracy of target localization, and performing more robustly especially in complex backgrounds. Secondly, when no valid target points are detected, all candidate points are regarded as possible target points for re-analysis. This strategy effectively avoids the situation of missed detection, ensuring that even in the case of rapid target changes or complex backgrounds, potential targets can still be found. In addition, the introduction of the area threshold and the margin threshold further reduces the probability of misdetection by filtering out regions that are too close to the edge and small irrelevant targets, ensuring the reliability of the target area. Based on the above strategies, the detection process is made more flexible and accurate, capable of adapting to different environmental changes, and has extremely strong robustness.

[0110] The above is only the preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art, several improvements and refinements made without departing from the principle of the present invention should also be regarded as within the protection scope of the present invention.

Claims

1. A method for detecting small infrared targets based on phase registration and consistency analysis, characterized in that: The steps include: Step S1, inputting an infrared image sequence, reading and sorting a sequence to be detected containing K frames of continuous infrared images, and initializing a displacement threshold, a registration threshold, and a region area threshold; Step S2, image registration, select the first frame as the reference image, calculate the phase correlation transformation matrix between the subsequent frames and the reference frame, and perform inter-frame registration. If the registration effect is not good, use the transformation matrix of the previous frame for correction; Step S3, consistency image calculation, using the registered image to calculate the inter-frame consistency image, and processing based on the set threshold to suppress the noise area and enhance the possible target points; Step S4, candidate target extraction, screen the target candidate areas from the consistency image through connected region analysis, remove the false alarm points that are too large or close to the edge, and sort the qualified areas by the maximum peak value to select the top ten candidate target points; Step S5, disappearance tolerance mechanism, if the target is not detected within a short period of time but there is still a possible target near the predicted position, then maintain its tracking state; static tolerance mechanism, if the target is still changing within the allowable frame number range, then it is determined to be a real target; Step S6, target screening and tracking, combining the target positions of the previous frames, calculating the motion trajectory of the candidate points, eliminating the target points that deviate from the reasonable motion trajectory, and finally confirming the real target; Step S7, the target area weight is enhanced, the position where the real target will appear in the next frame is predicted, and the predicted position area is enhanced.

2. The infrared small target detection method based on phase registration and consistency analysis according to claim 1 is characterized in that: In step S2, the phase correlation features in the frequency domain are used to estimate the translation displacement between image frames to achieve pixel-level alignment operation.

3. The infrared small target detection method based on phase registration and consistency analysis according to claim 2 is characterized in that: The phase difference between the two images is calculated by Fourier transform, and the formula for phase registration of the translation amount is obtained: (1) In formula (1), and Represent the Fourier transform of two frames of images respectively; for The complex conjugate of represents the inverse Fourier transform.

4. The infrared small target detection method based on phase registration and consistency analysis according to claim 1 is characterized in that: In step S3, the difference consistency between the target point and its surroundings in multiple directions is calculated to suppress redundant information and highlight the target edge features.

5. The infrared small target detection method based on phase registration and consistency analysis according to claim 4 is characterized in that: Step S3 includes the following steps: Step S301, using a predefined pixel range R and 8 directions to calculate pixel difference values; in each direction, the difference calculation formula is: (2) In formula (2), It represents the difference value in the dth direction within the rth range; Represents the pixel value of the target point, with coordinates p and q; and They represent pixel offsets in different directions respectively; K is a correction factor used to adjust the difference value; Step S302, calculate the sum of differences within the continuous pixel range R, and multiply the sum of differences in different directions to obtain the value of the consistency coefficient C: (3) In formula (3), Indicates the difference value in the dth direction within the rth range.

6. The infrared small target detection method based on phase registration and consistency analysis according to claim 1 is characterized in that: In step S5, the inactivity tolerance mechanism includes the following steps: Step S501, assuming that the displacement of the target between consecutive frames is: (4) In formula (4), represents the coordinates of the target in the tth frame, Represents the coordinates of the target in the t-1th frame; Step S502: If the displacement of the target meets the following conditions: (5) In formula (5), is the distance threshold; When the displacement of the target between two frames is less than , the target is considered to be stationary. At this time, the stationary tolerance Updated to: (6) Step S503: When the static tolerance Exceeding the maximum static tolerance setting , the target is considered stationary, but is not lost and remains in the target list.

7. The infrared small target detection method based on phase registration and consistency analysis according to claim 1 is characterized in that: In step S5, the disappearance tolerance mechanism includes: Step S504: Set the disappearance tolerance of the target in the tth frame to , and its update rules are as follows: If the target is not matched, then ; If the target is matched, then ; Step S505: when the tolerance level disappears Exceeding the maximum disappearance tolerance When , the target is judged to be lost; in the frame where the target is not matched, its position is predicted based on the historical displacement information of the target; let the position of the target in the t-1 frame be , Its displacement is , The predicted position of the target in the tth frame is (7).

8. The infrared small target detection method based on phase registration and consistency analysis according to claim 1 is characterized in that: Step S7 includes: Assume that the displacement of the target between two adjacent frames conforms to uniform linear motion; calculate the displacement of the target between the current frame and the previous frame to infer the position of the target in the next frame; the area near the predicted position will be weighted, and the weight coefficient is determined according to the speed and direction of the target movement. The weighted operation formula is as follows: (8) In formula (8), is the pixel coordinate in the image; is the predicted target position; is the weight coefficient; is the distance threshold, which represents the distance range from the predicted location.

Images