Infrared small target detection method based on phase registration and consistency analysis

Abstract

The invention discloses an infrared small target detection method based on phase registration and consistency analysis. The method comprises the following steps: S1, inputting an infrared image sequence; s2, carrying out image registration; s3, calculating a consistency image; step S4, candidate target extraction; step S5, a disappearance tolerance mechanism and a static tolerance mechanism; step S6, target screening and tracking; and S7, reinforcing the weight of the target area. According to the invention, a gradient enhanced difference consistency detection algorithm is utilized, background interference is effectively suppressed, and single-frame detection precision is improved; a static tolerance mechanism and a disappearance tolerance mechanism are introduced, so that the detection method has extremely high robustness to deal with the problems of irregular target movement and shielding; and efficient detection is realized through a modular framework and a dynamic weight adjustment mechanism.

Description

Technical Field

[0001] The present invention relates to the technical field of infrared image target detection, and more specifically, to an infrared small target detection method based on phase registration and consistency analysis. Background Art

[0002] At present, single-frame methods include local contrast, top-hat filters, low-rank and sparse decomposition, deep learning, and background suppression noise processing. These methods mainly rely on the information of a single image frame for target detection. Compared with the single-frame method, the multi-frame method has a significant advantage in accuracy because it can improve the stability and accuracy of target detection and tracking through information accumulation in the time dimension. The fundamental reason is that the multi-frame method can utilize the target motion information and time series relationship in continuous images to effectively reduce noise interference, reduce false alarms and missed detections, and improve performance under low signal-to-noise ratio and complex background conditions. At the same time, multi-frame data accumulation enhances the target signal and improves the accuracy of target state estimation, especially for situations where the target moves quickly or the signal is weak. The current common multi-frame methods are:

[0003] (1) Traditional filtering method

[0004] In the study of TBD (Tracking Before Detection) algorithm, traditional filtering methods have laid the foundation for target detection and tracking. Kalman et al. [1] first proposed the Kalman filter for target motion estimation under linear state space model, but it has limitations in dealing with complex background and nonlinear motion. To this end, Julier et al. [2] proposed the Unscented Kalman Filter (UKF), which effectively dealt with nonlinear problems. Subsequently, Gordon et al. [3] proposed the Particle Filter, which simulates the target state through a large number of particles and is suitable for more complex dynamic environments. In order to improve the efficiency of the particle filter, Doucet et al. [4] further proposed the Adaptive Particle Filter (APF), which improves the accuracy of target detection and tracking by dynamically adjusting the number and weight of particles.

[0005] (2) Tensor decomposition method

[0006] As the data dimension increases, tensor decomposition technology gradually occupies an important position in TBD algorithms. Xu et al. [5] first introduced low-rank tensor decomposition, which effectively separated small targets from complex backgrounds by representing sequence images as tensor structures. Based on this study, researchers proposed spatio-temporal tensor decomposition, which was further optimized by Liu et al. [6]. This method uses temporal information to enhance the detection effect of small targets. In addition, Zhao et al. [7] proposed tensor sparse decomposition, which combined with sparse representation technology further improved the target detection performance in complex scenes.

[0007] (3) Network structure method

[0008] In recent years, with the rise of deep learning, TBD algorithms have made new progress. LeCun et al. [8] proposed a convolutional neural network (CNN), which provides a powerful tool for feature extraction of small targets. However, CNN has certain limitations when processing sequential images. To this end, Hochreiter and Schmidhuber [9] proposed a long short-term memory network (LSTM), which solves the problem of temporal information processing by introducing memory units. Furthermore, Tran et al.

[10] proposed a three-dimensional convolutional neural network (3D CNN), which captures motion features between multiple frames by performing convolution operations simultaneously in time and space dimensions. Recently, Goodfellow et al.

[11] introduced a generative adversarial network (GAN), which enhances the detection effect in low signal-to-noise ratio environments by generating realistic target motion trajectory models.

[0009] Currently, the multi-frame method has the following defects:

[0010] (1) Most existing algorithms rely on the accuracy improvement brought by the analysis of multi-frame time series data, which leads to a large time overhead of the algorithm. The fundamental reason is that traditional methods often need to process long time series data, thus accumulating high computational complexity in the time dimension;

[0011] (2) When the existing algorithms face scenes with complex environments and severe noise, small targets are often submerged in heavy noise, resulting in target loss. The root causes are the following two points: First, it is difficult for single-frame detection algorithms to effectively suppress strong object edges. Second, image noise, lens sensor noise, and motion-induced deviations will cause the motion features of small targets to be unable to be accurately captured or identified, thereby destroying the target's motion pattern and affecting the effectiveness of the detection algorithm;

[0012] (3) Existing algorithms are often based on the assumption of motion continuity and cannot solve the problem of target tracking loss caused by fast target movement, slow target movement, or target occlusion. Summary of the invention

[0013] The purpose of the present invention is to overcome the deficiencies of the prior art and provide an infrared small target detection method based on phase alignment and consistency analysis.

[0014] In order to achieve the above object, the present invention adopts the following technical solution:

[0015] An infrared small target detection method based on phase registration and consistency analysis includes the following steps:

[0016] Step S1, inputting an infrared image sequence, reading and sorting a sequence to be detected containing K frames of continuous infrared images, and initializing a displacement threshold, a registration threshold, and a region area threshold;

[0017] Step S2, image registration, select the first frame as the reference image, calculate the phase correlation transformation matrix between the subsequent frames and the reference frame, and perform inter-frame registration. If the registration effect is not good, use the transformation matrix of the previous frame for correction;

[0018] Step S3, consistency image calculation, using the registered image to calculate the inter-frame consistency image, and processing based on the set threshold to suppress the noise area and enhance the possible target points;

[0019] Step S4, candidate target extraction, screen the target candidate areas from the consistency image through connected region analysis, remove the false alarm points that are too large or close to the edge, and sort the qualified areas by the maximum peak value to select the top ten candidate target points;

[0020] Step S5, disappearance tolerance mechanism, if the target is not detected within a short period of time but there is still a possible target near the predicted position, then maintain its tracking state; static tolerance mechanism, if the target is still changing within the allowable frame number range, then it is determined to be a real target;

[0021] Step S6, target screening and tracking, combining the target positions of the previous frames, calculating the motion trajectory of the candidate points, eliminating the target points that deviate from the reasonable motion trajectory, and finally confirming the real target;

[0022] Step S7, the target area weight is enhanced, the position where the real target will appear in the next frame is predicted, and the predicted position area is enhanced.

[0023] Furthermore, in step S2, the phase correlation feature in the frequency domain is used to estimate the translation displacement between image frames to achieve pixel-level alignment operation.

[0024] Furthermore, the phase difference between the two images is calculated by Fourier transform, and the formula for translation phase registration is obtained:

[0025] (1)

[0026] In formula (1), and Represent the Fourier transform of two frames of images respectively; for The complex conjugate of represents the inverse Fourier transform.

[0027] Furthermore, in step S3, the difference consistency between the target point and its surroundings in multiple directions is calculated to suppress redundant information and highlight the target edge features.

[0028] Further, step S3 includes the following steps:

[0029] Step S301, using a predefined pixel range R and 8 directions to calculate pixel difference values; in each direction, the difference calculation formula is:

[0030] (2)

[0031] In formula (2), It represents the difference value in the dth direction within the rth range; Represents the pixel value of the target point, with coordinates p and q; and They represent pixel offsets in different directions respectively; K is a correction factor used to adjust the difference value;

[0032] Step S302, calculate the sum of differences within the continuous pixel range R, and multiply the sum of differences in different directions to obtain the value of the consistency coefficient C:

[0033] (3)

[0034] In formula (3), Indicates the difference value in the dth direction within the rth range.

[0035] Furthermore, in step S5, the inactivity tolerance mechanism includes the following steps:

[0036] Step S501, assuming that the displacement of the target between consecutive frames is:

[0037] (4)

[0038] In formula (4), represents the coordinates of the target in the tth frame, Represents the coordinates of the target in the t-1th frame;

[0039] Step S502: If the displacement of the target meets the following conditions:

[0040] (5)

[0041] In formula (5), is the distance threshold;

[0042] When the displacement of the target between two frames is less than , the target is considered to be stationary. At this time, the stationary tolerance Updated to:

[0043] (6)

[0044] Step S503: When the static tolerance Exceeding the maximum static tolerance setting , the target is considered stationary, but is not lost and remains in the target list.

[0045] Furthermore, in step S5, the disappearance tolerance mechanism includes:

[0046] Step S504: Set the disappearance tolerance of the target in the tth frame to , and its update rules are as follows:

[0047] If the target is not matched, then ;

[0048] If the target is matched, then ;

[0049] Step S505: when the tolerance level disappears Exceeding the maximum disappearance tolerance When , the target is judged to be lost; in the frame where the target is not matched, its position is predicted based on the historical displacement information of the target; let the position of the target in the t-1 frame be ,

[0050] Its displacement is ,

[0051] The predicted position of the target in the tth frame is (7).

[0052] Further, step S7 includes:

[0053] Assume that the displacement of the target between two adjacent frames conforms to uniform linear motion; calculate the displacement of the target between the current frame and the previous frame to infer the position of the target in the next frame; the area near the predicted position will be weighted, and the weight coefficient is determined according to the speed and direction of the target movement. The weighted operation formula is as follows:

[0054] (8)

[0055] In formula (8), is the pixel coordinate in the image; is the predicted target position; is the weight coefficient; is the distance threshold, which represents the distance range from the predicted location.

[0056] The beneficial effects of the present invention are:

[0057] 1. The present invention utilizes a gradient-enhanced difference consistency detection algorithm to effectively suppress background interference and improve single-frame detection accuracy.

[0058] 2. The present invention introduces a static tolerance mechanism and a disappearance tolerance mechanism to make the detection method extremely robust to cope with irregular target motion and occlusion problems.

[0059] 3. The present invention achieves efficient detection through a modular framework and a dynamic weight adjustment mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 Schematic diagram of an overall architecture of the infrared small target detection method based on phase registration and consistency analysis in this embodiment;

[0061] Figure 2 This is a flow chart of the infrared small target detection method based on phase alignment and consistency analysis in this embodiment. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] Embodiment: A method for detecting small infrared targets based on phase registration and consistency analysis, such as Figure 1 , Figure 2 As shown, the following steps are included:

[0064] Step S1, inputting an infrared image sequence, reading and sorting a sequence to be detected containing K frames of continuous infrared images, ensuring that the image sizes are consistent, and initializing key parameters such as displacement threshold, registration threshold, and region area threshold;

[0065] Step S2, image registration, select the first frame as the reference image, calculate the phase correlation transformation matrix between the subsequent frames and the reference frame, and perform inter-frame registration. If the registration effect is not good, use the transformation matrix of the previous frame for correction;

[0066] Furthermore, the phase correlation characteristics in the frequency domain are used to efficiently estimate the translation displacement between image frames, thereby achieving pixel-level alignment operations and improving the detection stability of subsequent algorithms. By combining phase registration technology, inter-frame displacement correction is achieved to enhance detection robustness and solve the impact of small lens movement on infrared small target detection. Specifically:

[0067] The phase difference between the two images is calculated by Fourier transform, thus obtaining the formula for translation phase registration:

[0068] (1)

[0069] In formula (1), and Represent the Fourier transform of two frames of images respectively; for The complex conjugate of represents the inverse Fourier transform.

[0070] By locating the phase correlation function The peak position of the image can be accurately restored to complete the image alignment.

[0071] Step S3, consistency image calculation, using the registered image to calculate the inter-frame consistency image, and processing based on the set threshold to suppress the noise area and enhance the possible target points;

[0072] Furthermore, by calculating the difference consistency between the target point and its surroundings in multiple directions, redundant information can be suppressed and the target edge features can be highlighted. By calculating the difference consistency between the target point and the surrounding pixels, background interference can be suppressed and the target features can be highlighted, which is used to effectively analyze the local features and edge information in the infrared image. Specifically:

[0073] Step S301, using a predefined pixel range R and 8 directions to calculate pixel difference values; in each direction, the difference calculation formula is:

[0074] (2)

[0075] In formula (2), It represents the difference value in the dth direction within the rth range; Represents the pixel value of the target point, with coordinates p and q; and They represent pixel offsets in different directions respectively; K is a correction factor used to adjust the difference value;

[0076] Compensation measures are taken for pixels outside the boundary, that is, the pixel value of the target point itself is used to replace it.

[0077] Step S302, to obtain the overall consistency feature, the difference sum in the continuous pixel range R is calculated, and the difference sum in different directions is multiplied to obtain the value of the consistency coefficient C:

[0078] (3)

[0079] In formula (3), Indicates the difference value in the dth direction within the rth range.

[0080] The smaller the value of the consistency coefficient C, the higher the consistency between the target point and the surrounding pixels, which reflects its edge significance in the image. Through this consistency analysis, small targets in the image can be effectively highlighted, background interference can be suppressed, and target detection effects can be improved.

[0081] Step S4, candidate target extraction, screen the target candidate areas from the consistency image through connected region analysis, remove the false alarm points that are too large or close to the edge, and sort the qualified areas by the maximum peak value to select the top ten candidate target points;

[0082] Step S5, disappearance tolerance mechanism, if the target is not detected in a short time but there is still a possible target near the predicted position, then maintain its tracking state; static tolerance mechanism, if the target is still changing within the allowable frame number range, it is determined to be a real target, reducing false detection and missed detection;

[0083] Among them, the static tolerance mechanism allows the detection of slow targets to be maintained, avoiding the misjudgment of target disappearance due to slow speed; the disappearance tolerance mechanism is based on the assumption that the target moves at a uniform speed in a short period of time, predicts its possible position after disappearance and re-matches it. By introducing the disappearance tolerance mechanism and the static tolerance mechanism, the robustness of infrared small target detection is further improved.

[0084] Both the static tolerance mechanism and the disappearance tolerance mechanism are based on the assumption that the target moves in a straight line at a uniform speed in a short period of time, corresponding to the situations where the target is static and the target disappears temporarily, respectively, thereby improving the stability of target detection in complex environments. Static tolerance is used to deal with situations where the target is misjudged as disappeared because it has not moved significantly in a short period of time, while disappearance tolerance is used to solve the problem of loss caused by temporary occlusion of the target or image noise. By setting these two tolerances reasonably, it is possible to better cope with the temporary stillness or disappearance of the target, avoid tracking failure, and achieve more stable and reliable target detection and tracking. Specifically:

[0085] The inactivity tolerance mechanism includes the following steps:

[0086] Step S501, assuming that the displacement of the target between consecutive frames is:

[0087] (4)

[0088] In formula (4), represents the coordinates of the target in the tth frame, Represents the coordinates of the target in the t-1th frame;

[0089] Step S502: If the displacement of the target meets the following conditions:

[0090] (5)

[0091] In formula (5), is the distance threshold;

[0092] When the displacement of the target between two frames is less than , the target is considered to be stationary. At this time, the stationary tolerance Updated to:

[0093] (6)

[0094] Step S503: When the static tolerance Exceeding the maximum static tolerance setting When the target is considered to be stationary, it will not be lost but will continue to be kept in the target list to avoid being misjudged as disappeared due to short-term stillness.

[0095] In actual scenarios, the target may not be detected in several frames due to factors such as occlusion or image noise. To deal with this situation, disappearance tolerance is introduced. By recording the number of times the target is not matched in consecutive frames, the target can still be tracked after disappearing for a short time.

[0096] Step S504: Set the disappearance tolerance of the target in the tth frame to , and its update rules are as follows:

[0097] If the target is not matched, then ;

[0098] If the target is matched, then ;

[0099] Step S505: when the tolerance level disappears Exceeding the maximum disappearance tolerance When , the target is judged to be lost; in the frame where the target is not matched, its position is predicted based on the historical displacement information of the target; let the position of the target in the t-1 frame be ,

[0100] Its displacement (i.e. velocity) is ,

[0101] The predicted position of the target in the tth frame is (7).

[0102] Through this mechanism, tracking can be maintained even when the target disappears or is blocked for a short time, and tracking can be resumed when the target reappears, thereby improving the robustness of the tracking algorithm.

[0103] Step S6, target screening and tracking, combining the target positions of the previous frames, calculating the motion trajectory of the candidate points, eliminating the target points that deviate from the reasonable motion trajectory, and finally confirming the real target;

[0104] Step S7, the target area weight is enhanced, the position where the real target will appear in the next frame is predicted, and the predicted position area is enhanced.

[0105] Phase registration is used to achieve image alignment and dynamic compensation, providing a stable foundation for target detection. Gradient analysis and weight enhancement techniques are combined to enhance the pixel characteristics of the target area, making the response of small targets more prominent in the consistency image. Subsequently, the weight-enhanced consistency image is subjected to connected region analysis, and candidate regions that meet the target characteristics are accurately screened out through the dual constraints of area and margin. The peak intensities of these candidate regions are sorted, and the centroids are extracted as candidate target points. The motion status, matching information and other attributes are recorded in a structured manner, laying a data foundation for candidate operation tracking and target management.

[0106] Specifically, it is assumed that the displacement of the target between two adjacent frames conforms to uniform linear motion; the displacement of the target between the current frame and the previous frame is calculated to infer the position of the target in the next frame; the area near the predicted position will be weighted, and the weight coefficient is determined according to the speed and direction of the target movement. The weighted operation formula is as follows:

[0107] (8)

[0108] In formula (8), is the pixel coordinate in the image; is the predicted target position; is the weight coefficient; is the distance threshold, which indicates the distance range from the predicted position. If the weight of the point is will be enhanced, thereby increasing the detection priority of the target.

[0109] This embodiment adopts a modular framework structure. First, by selecting the first ten peaks in each frame of the image as candidate target points, it is possible to select the most representative target from multiple potential areas, thereby avoiding the problem of false detection that may be caused by a single peak, enhancing the accuracy of target positioning, and being more robust in complex backgrounds. Secondly, when no valid target point can be detected, all candidate points are re-analyzed as possible target points. This strategy effectively avoids missed detections and ensures that potential targets can be found even when the target changes rapidly or the background is complex. In addition, the introduction of area thresholds and margin thresholds further reduces the probability of false detection and ensures the reliability of the target area by filtering out areas that are too close to the edge and small irrelevant targets. Based on the above strategy, the detection process is more flexible and accurate, can adapt to different environmental changes, and has extremely strong robustness.

[0110] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for detecting small infrared targets based on phase registration and consistency analysis, characterized in that: The steps include: Step S1, inputting an infrared image sequence, reading and sorting a sequence to be detected containing K frames of continuous infrared images, and initializing a displacement threshold, a registration threshold, and a region area threshold; Step S2, image registration, select the first frame as the reference image, calculate the phase correlation transformation matrix between the subsequent frames and the reference frame, and perform inter-frame registration. If the registration effect is not good, use the transformation matrix of the previous frame for correction; Step S3, consistency image calculation, using the registered image to calculate the inter-frame consistency image, and processing based on the set threshold to suppress the noise area and enhance the possible target points; Step S4, candidate target extraction, screen the target candidate areas from the consistency image through connected region analysis, remove the false alarm points that are too large or close to the edge, and sort the qualified areas by the maximum peak value to select the top ten candidate target points; Step S5, disappearance tolerance mechanism, if the target is not detected within a short period of time but there is still a possible target near the predicted position, then maintain its tracking state; static tolerance mechanism, if the target is still changing within the allowable frame number range, then it is determined to be a real target; Step S6, target screening and tracking, combining the target positions of the previous frames, calculating the motion trajectory of the candidate points, eliminating the target points that deviate from the reasonable motion trajectory, and finally confirming the real target; Step S7, the target area weight is enhanced, the position where the real target will appear in the next frame is predicted, and the predicted position area is enhanced.

2. The infrared small target detection method based on phase registration and consistency analysis according to claim 1 is characterized in that: In step S2, the phase correlation features in the frequency domain are used to estimate the translation displacement between image frames to achieve pixel-level alignment operation.

3. The infrared small target detection method based on phase registration and consistency analysis according to claim 2 is characterized in that: The phase difference between the two images is calculated by Fourier transform, and the formula for phase registration of the translation amount is obtained: (1) In formula (1), and Represent the Fourier transform of two frames of images respectively; for The complex conjugate of ; represents the inverse Fourier transform.

4. The infrared small target detection method based on phase registration and consistency analysis according to claim 1 is characterized in that: In step S3, the difference consistency between the target point and its surroundings in multiple directions is calculated to suppress redundant information and highlight the target edge features.

5. The infrared small target detection method based on phase registration and consistency analysis according to claim 4 is characterized in that: Step S3 includes the following steps: Step S301, using a predefined pixel range R and 8 directions to calculate pixel difference values; in each direction, the difference calculation formula is: (2) In formula (2), It represents the difference value in the dth direction within the rth range; Represents the pixel value of the target point, with coordinates p and q; and They represent pixel offsets in different directions respectively; K is a correction factor used to adjust the difference value; Step S302, calculate the sum of differences within the continuous pixel range R, and multiply the sum of differences in different directions to obtain the value of the consistency coefficient C: (3) In formula (3), Indicates the difference value in the dth direction within the rth range.

6. The infrared small target detection method based on phase registration and consistency analysis according to claim 1 is characterized in that: In step S5, the inactivity tolerance mechanism includes the following steps: Step S501, assuming that the displacement of the target between consecutive frames is: (4) In formula (4), represents the coordinates of the target in the tth frame, Represents the coordinates of the target in the t-1th frame; Step S502: If the displacement of the target meets the following conditions: (5) In formula (5), is the distance threshold; When the displacement of the target between two frames is less than , the target is considered to be stationary. At this time, the stationary tolerance Updated to: (6) Step S503: When the static tolerance Exceeding the maximum static tolerance setting , the target is considered stationary, but is not lost and remains in the target list.

7. The infrared small target detection method based on phase registration and consistency analysis according to claim 1 is characterized in that: In step S5, the disappearance tolerance mechanism includes: Step S504: Set the disappearance tolerance of the target in the tth frame to , and its update rules are as follows: If the target is not matched, then ; If the target is matched, then ; Step S505: when the tolerance level disappears Exceeding the maximum disappearance tolerance When , the target is judged to be lost; in the frame where the target is not matched, its position is predicted based on the historical displacement information of the target; let the position of the target in the t-1 frame be , Its displacement is , The predicted position of the target in the tth frame is (7).

8. The infrared small target detection method based on phase registration and consistency analysis according to claim 1 is characterized in that: Step S7 includes: Assume that the displacement of the target between two adjacent frames conforms to uniform linear motion; calculate the displacement of the target between the current frame and the previous frame to infer the position of the target in the next frame; the area near the predicted position will be weighted, and the weight coefficient is determined according to the speed and direction of the target movement. The weighted operation formula is as follows: (8) In formula (8), is the pixel coordinate in the image; is the predicted target position; is the weight coefficient; is the distance threshold, which represents the distance range from the predicted location.

Images