An infrared target detection method and a computer readable storage medium

Abstract

The application provides an infrared target detection method and a computer storage medium, and specifically comprises the following steps: inputting continuous video frame data; correcting lens shaking and background changes of the continuous video frame data by using an image registration method based on a foreground mask algorithm; associating adjacent frame targets in the continuous video frame data by using time and space constraints; combining a KCF algorithm to further track the associated targets to obtain tracking results; and finally determining whether the targets are moving targets according to the number of times of target association and the tracking results. According to the infrared target detection method provided by the application, the shaking and background changes are corrected by using KLT-based feature point matching, the adjacent frame targets are associated by using time and space constraints, and then the KCF-based kernel correlation filtering tracking method is combined to perform multi-dimensional target association and joint detection of moving targets, so that the infrared dim and small target detection rate under the airborne forward-looking and downward-looking conditions is improved, and the false alarm rate is reduced.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to an infrared target detection method and a computer-readable storage medium. Background Technology

[0002] Detection and tracking of small targets in airborne forward-looking infrared imaging is an extremely difficult problem. In airborne environments, infrared imaging systems experience severe shaking and background changes, resulting in low resolution of infrared images. In addition, the small size of the target and atmospheric disturbances make target detection very difficult.

[0003] Infrared target detection plays a crucial role in infrared imaging technology, enabling the detection, identification, and early warning of targets at long distances. Currently, various algorithms exist for infrared target detection, including methods based on the human visual system, pipeline filtering, morphological filtering, and deep learning.

[0004] The detection and tracking performance of airborne forward-looking infrared (FLIR) targets with small targets largely depends on the correction of lens shake, the suppression of ground background, and the correlation between targets in adjacent frames and the constraints of time and space on the targets. Summary of the Invention

[0005] The technical problem to be solved by the present invention is how to improve the detection rate of infrared weak targets, and to provide an infrared target detection method and a computer-readable storage medium.

[0006] According to the infrared target detection method provided by the present invention, the method includes:

[0007] Input continuous video frame data;

[0008] The image registration method based on the foreground mask algorithm is used to correct lens shake and background changes in the continuous video frame data.

[0009] Utilize temporal and spatial constraints to associate adjacent frame targets in the continuous video frame data;

[0010] By combining the KCF algorithm, we can further track related targets to obtain tracking results;

[0011] The final determination of whether a target is a moving target is based on the number of times the target is associated and the tracking results.

[0012] The infrared target detection method provided by this invention corrects for swaying and background changes by feature point matching based on KLT, uses temporal and spatial features to constrain the association of targets in adjacent frames, and combines a kernel correlation filter tracking method based on KCF to perform multi-dimensional target association and jointly detect moving targets. This improves the detection rate of weak infrared targets under airborne forward-looking and downward-looking conditions and reduces the false alarm rate.

[0013] In some embodiments of the present invention, a foreground mask-based image registration method is used to correct lens shake and background changes, specifically including:

[0014] For three consecutively input images in the continuous video frame data, with the last image as a reference, candidate regions are given;

[0015] The offset of the candidate regions in the other two frames of images was calculated using a feature point matching algorithm based on the KLT corner detection method, and the image offset of the three frames of images was corrected.

[0016] According to some embodiments of the present invention, the method further includes:

[0017] Foreground mask data is obtained from three frames of images after image offset correction using the three-frame difference method;

[0018] The foreground image is finally obtained by post-processing the foreground mask image.

[0019] In some embodiments of the present invention, the post-processing includes normalization, erosion, dilation, and binarization.

[0020] According to some embodiments of the present invention, the method of using temporal and spatial constraints to correlate adjacent frame targets in video frame data specifically includes:

[0021] Based on the foreground image, segment the target and extract its location and image information;

[0022] For targets in adjacent frames, similarity is calculated based on location information to perform target association.

[0023] In some embodiments of the present invention, for targets in adjacent frames, similarity is calculated based on location information to perform target association, specifically as follows:

[0024] If the Euclidean distance between targets in adjacent frames is less than a certain threshold, they are considered potential targets. The smaller the distance, the higher the similarity. When the similarity meets the preset conditions, the targets in adjacent frames are associated.

[0025] According to some embodiments of the present invention, the further association and confirmation of moving targets based on the KCF algorithm specifically includes:

[0026] Targets with a number of associations exceeding a preset value are designated as highly suspicious targets.

[0027] A regressor is trained based on the KCF algorithm, and then used to track highly suspicious targets.

[0028] In some embodiments of the present invention, the use of a regressor to track highly suspicious targets specifically involves:

[0029] The response values ​​of the latest frame and the previous frame of the highly suspicious target are obtained by the regressor. When the distance between the latest frame and the previous frame and the response value meet the preset conditions, the targets of the latest frame and the previous frame of the highly suspicious target are associated as the same target.

[0030] According to some embodiments of the present invention, the infrared target detection method further includes: collecting positive and negative samples in the area surrounding the target using a cyclic matrix, and training a regressor using ridge regression.

[0031] The present invention also provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to perform the method steps as described in some embodiments of the present invention. Attached Figure Description

[0032] Figure 1 This is a flowchart of an infrared target detection method according to an embodiment of the present invention;

[0033] Figure 2 This is a flowchart of an infrared target detection method according to a specific embodiment of the present invention;

[0034] Figure 3 This is a schematic diagram illustrating the effect of an infrared target detection method according to a specific embodiment of the present invention. Detailed Implementation

[0035] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments.

[0036] The steps described in the specification and the flowcharts in the accompanying drawings of this invention are not necessarily to be strictly followed according to the step numbers; the execution order of the steps can be changed. Furthermore, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be broken down into multiple steps.

[0037] The infrared target detection method provided by the present invention, such as Figure 1 As shown, the method includes:

[0038] S100: Input continuous video frame data.

[0039] S200: Corrects lens shake and background changes in continuous video frame data using an image registration method based on a foreground masking algorithm.

[0040] S300: Uses temporal and spatial constraints to associate adjacent frame targets in continuous video frame data.

[0041] S400: Combines the KCF algorithm to further track associated targets in order to obtain tracking results.

[0042] S500: Based on the number of times the target is associated and the tracking results, it is finally determined whether the target is a moving target.

[0043] The infrared target detection method provided by this invention corrects for swaying and background changes by feature point matching based on KLT, uses temporal and spatial features to constrain the association of targets in adjacent frames, and combines a kernel correlation filter tracking method based on KCF to perform multi-dimensional target association and jointly detect moving targets. This improves the detection rate of weak infrared targets under airborne forward-looking and downward-looking conditions and reduces the false alarm rate.

[0044] In some embodiments of the present invention, S200: Correcting lens shake and background changes using an image registration method based on a foreground mask algorithm specifically includes: for three consecutively input images in continuous video frame data, using the last image as a reference, providing candidate regions; calculating the offsets of the candidate regions in the other two images using a feature point matching algorithm based on the KLT corner detection method, and correcting the image offset of the three images.

[0045] Specifically, KLT corner detection states that if two points match in images I and J, then there exists a minimal squared difference ε in gray levels within a window centered at the two points and defined by W, as follows:

[0046]

[0047] Where coordinates offset Change formula (1) to

[0048]

[0049] To obtain the minimum matching, minimizing the value of ε, the above equation is applied at point... Perform a Taylor expansion at the point, truncating higher-order terms and retaining linear terms.

[0050]

[0051] in Using the above formula, let respectively and Substitute into the above equation: Let Know Substituting into the above equation, we get:

[0052]

[0053] Therefore, there is.

[0054]

[0055] Substituting (2) into (3) approximately, we get:

[0056]

[0057] make We can obtain:

[0058]

[0059] Equivalent to

[0060] in The optimal offset position is determined according to formula (5).

[0061] According to some embodiments of the present invention, the method further includes:

[0062] Foreground mask data is obtained from three frames of images after image offset correction using the three-frame difference method;

[0063] The foreground image is finally obtained by post-processing the foreground mask image.

[0064] In some embodiments of the present invention, post-processing includes normalization, erosion, dilation, and binarization.

[0065] According to some embodiments of the present invention, targets in adjacent frames of video frame data are associated using temporal and spatial constraints. Specifically, the targets are segmented based on the foreground image, and location information and image information are extracted. For targets in adjacent frames, similarity is calculated based on the location information to perform target association.

[0066] In some embodiments of the present invention, the similarity of targets in adjacent frames is calculated based on location information for target association. Specifically, targets in adjacent frames with a Euclidean distance less than a certain threshold are set as potential targets. The smaller the distance, the higher the similarity. When the similarity meets a preset condition, the targets in adjacent frames are associated.

[0067] The process of associating targets in adjacent frames specifically includes: 1. Since the target area is not very large, large-area targets are first filtered out from the foreground mask obtained by the foreground mask detection algorithm. 2. Target segmentation is performed, and location and image information, including position, size, and image template, are collected for each target. 3. Targets in adjacent frames whose Euclidean distance is less than a certain threshold are designated as potential targets; the smaller the distance, the higher the similarity. 4. As the number of associated targets increases, the probability of the target being a moving target increases. For example, suspicious targets with more than 5 associated targets can be designated as highly suspicious targets. 5. Tracking algorithms are used to track highly suspicious targets.

[0068] According to some embodiments of the present invention, in conjunction with the KCF algorithm, moving targets are further associated and confirmed. Specifically, targets with more than a preset number of associations are set as highly suspicious targets, a regressor is trained according to the KCF algorithm, and the regressor is used to track highly suspicious targets.

[0069] Specifically, KCF, as a discriminative tracking method, typically trains a target detector during the tracking process. This detector is then used to detect whether the predicted location in the next frame is the target. The new detection results are then used to update the training set and subsequently the target detector. By collecting positive and negative samples using a circulant matrix of the region surrounding the target, and training the target detector using ridge regression, KCF successfully leverages the diagonalizability of circulant matrices in Fourier space to transform matrix operations into vector Hadamad products (elemental dot products), significantly reducing computational complexity and increasing speed.

[0070] By searching the region Z, and using the objective function f(z) = w T The result of z is used to determine the target's position and minimize the objective function.

[0071]

[0072] in, Let yi be the i-th training sample, and yi be the pixel value of the center point of the corresponding region. Taking the derivative of formula (6)w and finding that it equals 0, we can obtain:

[0073] w = (X H X+λI) -1 X H y (7)

[0074] Where matrix X is formed by the target vector x = [x1, x2, ..., x...]. n The resulting cyclic matrix is:

[0075]

[0076] A circular matrix can be diagonalized in Fourier space using a discrete Fourier matrix, i.e.:

[0077]

[0078] This represents performing a Fourier transform on vector x, i.e. in,

[0079]

[0080] Substituting (8) into (7), we simplify:

[0081]

[0082]

[0083]

[0084]

[0085] in, This represents the element-wise dot product of two vectors. After introducing the kernel function, the objective function becomes:

[0086]

[0087] Where k represents the kernel function, its definition and operation are as follows:

[0088]

[0089] The final solution is:

[0090]

[0091] Generally, k is chosen as the Gaussian kernel function:

[0092]

[0093] Using formula (9), a regressor can be trained to search for the target's location. The location regression formula is as follows:

[0094]

[0095] Generally, during training, X is taken as the region where the target is located. During location regression, the response value... The position of the maximum value is the target's moving position.

[0096] In some embodiments of the present invention, the use of a regressor to track highly suspicious targets specifically involves:

[0097] The response values ​​of the latest frame and the previous frame of the highly suspicious target are obtained by the regressor. When the distance between the latest frame and the previous frame and the response value meet the preset conditions, the targets of the latest frame and the previous frame of the highly suspicious target are associated as the same target.

[0098] When the regressor obtained by the KCF algorithm tracks the target, it can be seen from formula (10) that when the target x in the previous frame is similar to the target z in the next frame, the response value It will exhibit a good Gaussian distribution. Therefore, the response value can be determined based on the result of formula (10). To determine whether two targets are similar, the response value is taken. The standard deviation threshold σ = 0.05. When σ ≥ 0.05, the two targets are considered similar. In addition, due to the temporal and spatial constraints, the displacement between target frames will not be too large. The target distance threshold d = 15 is taken. When the distance d > 15 and σ > 0.05, the two targets can be associated as the same target.

[0099] According to some embodiments of the present invention, the infrared target detection method further includes: collecting positive and negative samples in the area surrounding the target using a cyclic matrix, and training a regressor using ridge regression. When training the regressor, the target area is generally selected as the positive sample, and the area surrounding the target as the negative sample; naturally, the closer the area is to the target, the greater the probability that it is a positive sample.

[0100] The present invention also provides a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the method steps as described in some embodiments of the present invention. The difference is that, in engineering implementation, this embodiment can be implemented by means of software plus necessary general-purpose hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0101] Based on this understanding, the method of the present invention can be embodied in the form of a computer software product, which is stored in a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a device (which may be a computer or other device with data processing and graphics processing functions) to execute the method of the embodiments of the present invention.

[0102] The infrared target detection method according to the present invention will now be described in detail with reference to the accompanying drawings and a specific embodiment. It is to be understood that the following description is merely exemplary and not intended to limit the scope of the invention.

[0103] This invention discloses an infrared weak target detection method based on feature point matching and kernel correlation filtering. It solves the problem of detecting and tracking moving weak targets under airborne downward-looking conditions by performing image correction through KLT-based feature point matching and target association through a KCF-based kernel correlation filtering algorithm. The method achieves good detection and tracking performance even under complex ground conditions. Similar feature point matching methods include, but are not limited to, feature point matching based on grayscale images, feature point matching based on contour curves, feature point matching based on binarization, and feature point matching based on phase. Kernel correlation filtering methods include, but are not limited to, CSK-based kernel correlation filtering, STC-based kernel correlation filtering, and KCF-based kernel correlation filtering.

[0104] The proposed algorithm first employs a Harris-based corner detection method for multi-point correction, then uses the KLT optical flow method to determine the inter-frame background image offset. Simultaneously, multi-scale median filtering is performed on the current frame and the two preceding frames. Subsequently, a three-frame difference pair is calculated after offset compensation using either frame skipping or a fixed frame interval. After inputting three images and obtaining two difference images, an AND operation is performed on these two difference images. Finally, the Ostu algorithm and morphological filtering are combined to binarize the images and obtain the target foreground.

[0105] The target exhibits scale invariance in adjacent or near-adjacent frames, and employs the frame skipping mechanism from the previous section, ensuring that the target displacement in near-adjacent frames is within the detection area of ​​the target in the current frame. Based on these premises, the target association algorithm based on kernel correlation filtering effectively guarantees the robustness of target detection. In signal processing, convolution is often used to perform Fourier transforms on discrete signals. The kernel correlation filtering algorithm, combined with ridge regression, uses cyclic shift matrix diagonalization to transform the image from the spatial domain to the frequency domain for computation, reducing computational complexity while maintaining detection and tracking accuracy.

[0106] Please refer to the accompanying drawing, which is a flowchart of an infrared weak target detection method based on feature point matching and kernel correlation filtering disclosed in this invention. The infrared weak target detection method based on feature point matching and kernel correlation filtering disclosed in this invention is as follows: Figure 2 and Figure 3 As shown, the specific implementation steps are as follows:

[0107] Step S1: Extract three adjacent image data frames from the input video sequence.

[0108] Step S2: Perform feature point detection on the three frames of images (e.g., Figure 3 (as shown in a).

[0109] Step S3: Based on the KLT background correction algorithm, calculate the feature point offset and correct the image.

[0110] Step S4: Differentiate the three frames to obtain the foreground mask image (e.g., Figure 3 (As shown in b).

[0111] Step S5: Binarize the foreground mask and perform target segmentation and blind flash pixel removal (e.g.) Figure 3 (as shown in c).

[0112] Step S6: Associate the current frame data with historical frame data to obtain new target information (such as...). Figure 3 (as shown in d).

[0113] Furthermore, after the above steps and in combination with the KCF tracking algorithm, the following can be obtained: Figure 3 The test results are shown in Figure e.

[0114] This invention discloses an infrared weak target detection method based on feature point matching and kernel correlation filtering. By combining feature point matching based on KLT with target association algorithm based on KCF, it solves the problem of detecting and tracking airborne downward-looking moving weak targets. Under complex ground conditions, it can detect moving targets and achieve good detection and tracking results. The main feature is that it proposes a target detection process to improve the algorithm performance. From subjective visual effects and objective numerical evaluation, it can be seen that our method improves the performance of infrared weak target detection, especially in suppressing false alarms caused by complex background clutter.

[0115] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0116] Through the description of specific embodiments, a more in-depth and specific understanding should be gained of the technical means and effects adopted by the present invention to achieve the intended purpose. However, the accompanying drawings are only provided for reference and illustration and are not intended to limit the present invention.

Claims

1. A method of infrared target detection, characterized in that, The method comprises: inputting continuous video frame data; correcting camera shake and background change of the continuous video frame data by an image registration method based on a foreground mask algorithm; correlating adjacent frame targets in the continuous video frame data in time and space; further correlating and confirming the moving targets in combination with a KCF algorithm to obtain tracking results; finally determining whether the target is a moving target according to the correlation times and the tracking results of the target; the correction of the camera shake and the background change by the image registration method based on the foreground mask algorithm specifically comprises: for three continuously input images in the continuous video frame data, taking the last image as a reference to give a candidate region; calculating the offset of the candidate region of the other two images by a feature point matching algorithm based on a KLT corner detection method, and correcting the image offset of the three images; obtaining foreground mask data by a three-frame difference method for the three images after the image offset correction; finally obtaining a foreground image by post-processing the foreground mask image; the correlation of the adjacent frame targets in the video frame data in time and space specifically comprises: segmenting the target according to the foreground image, and extracting position information and image information; calculating the similarity according to the position information to correlate the targets of adjacent frames.

2. The infrared target detection method according to claim 1, characterized in that, The post-processing comprises normalization, corrosion, expansion and binarization.

3. The infrared target detection method according to claim 1, characterized in that, The calculation of the similarity according to the position information to correlate the targets of adjacent frames specifically comprises: the Euclidean distance of the targets of adjacent frames is less than a specific threshold value, which is a potential target, and the potential target of the adjacent frames is correlated.

4. The infrared target detection method according to claim 1, characterized in that, the further correlation and confirmation of the moving targets in combination with the KCF algorithm specifically comprises: setting the target with a correlation time greater than a preset value as a highly suspicious target; training a regressor according to the KCF algorithm, and tracking the highly suspicious target by using the regressor.

5. The infrared target detection method according to claim 4, characterized in that, the tracking of the highly suspicious target by using the regressor specifically comprises: obtaining a response value by the regressor according to the target data of the latest frame and the last frame of the latest frame of the highly suspicious target, and when the distance between the latest frame and the last frame and the response value meet preset conditions respectively, the target of the latest frame of the highly suspicious target and the last frame of the latest frame are correlated as the same target.

6. The infrared target detection method according to claim 4, characterized in that, The infrared target detection method further comprises: collecting positive and negative samples in the target surrounding area by using a circulant matrix, and training a regressor by using ridge regression.

7. A computer readable storage medium characterized in that, The computer readable storage medium stores one or more programs, which can be executed by one or more processors to implement the method steps of any one of claims 1 to 6.

Images