A method for detecting and tracking small infrared targets based on mask and adaptive filtering

Abstract

A mask- and adaptive-filter-based infrared small target detection and tracking method first leverages the fixed field of view of satellite remote sensing reconnaissance satellites to generate a target foreground and background mask based on an initial multi-frame (5-10 frames) infrared image sequence, used to record the characteristic information of the interfering target and the real target. The method then detects the newly input infrared image based on this and constructs an adaptive filter based on the target characteristic information contained in the mask. This method can dynamically extract features of different targets at different motion moments, thereby improving the target matching and tracking accuracy. The method utilizes a target mask and an adaptive filter to address the noise interference and multi-target interference issues of infrared small targets in high-speed motion. It possesses enhanced robustness and can achieve highly reliable matching and tracking of multiple infrared small targets under various complex backgrounds, meeting the real-time, highly reliable reconnaissance requirements on sensitive target satellites.

Description

Technical Field

[0001] The present invention provides a method for detecting and tracking infrared small targets, in particular an on-orbit rapid and highly reliable detection and tracking method for high-speed infrared small targets, belonging to the field of aerospace remote sensing. Background Art

[0002] Infrared cameras primarily observe targets by receiving their own infrared radiation. They are particularly sensitive to high-speed, high-heat-radiating targets, such as missiles and aircraft. This makes infrared target detection and tracking technology crucial for military reconnaissance and early warning. In wide-band satellite infrared remote sensing images, these sensitive flying targets are small, often appearing as spots or dots. The targets have a low signal-to-noise ratio and are susceptible to interference from noise, clutter, or cloud cover, often being lost in the background. Therefore, for small, weak infrared targets, detection is often based on target motion or change characteristics. Methods such as inter-frame differencing and background differencing are often used. These methods are computationally simple, direct, and fast, making them ideal for real-time on-orbit reconnaissance and early warning missions involving resource-limited satellites.

[0003] However, the inter-frame difference method and the background difference method are sensitive to interference factors such as noise and background changes caused by target motion. Moreover, when the target is occluded, the algorithm can easily mistake the reappearing target for another moving target and cannot cope with occlusion changes. In addition, because the inter-frame difference method mainly uses the difference in grayscale values ​​to detect moving targets, when the grayscale values ​​of most pixels within the target are the same, the difference image obtained by the inter-frame difference method only contains the images of the two sides of the target object, and a "hole" phenomenon is generated within the target, making it difficult to obtain the complete outline of the target. Moreover, due to the variable motion state of the target, the background constructed by the background difference method is difficult to completely eliminate various real targets. Summary of the Invention

[0004] The technical problem solved by the present invention is to overcome the shortcomings of the existing technology, solve the problems of high false alarm and poor trajectory association in satellite on-orbit infrared small target detection and tracking (tracking is lost when target trajectory points are incomplete or blocked, and the two trajectories cannot be associated when they appear again), and provide a real-time high-precision infrared small target detection and tracking method.

[0005] The technical solution of the present invention is:

[0006] A method for detecting and tracking infrared small targets based on mask and adaptive filtering includes the following steps:

[0007] 1) Using the previous frame image I t-1 Corresponding background image From the image of the current frame I t Extract multiple suspected targets and obtain a suspected target coordinate set consisting of multiple suspected target coordinates in the current frame and each image slice of a suspected target If the suspected target set of the current frame is empty, go to step 6); otherwise go to step 2);

[0008] 2) For the suspected target coordinate set obtained in step 1) The coordinates of the kth suspected target in In the current frame image I t China-Israel An image block is extracted as the center of the pixel point at the position, and edge detection is performed on the image block. The suspected targets that do not meet the target size range are removed from the suspected target coordinate set. Eliminate and traverse the suspected target coordinate set All elements in , then go to step 3);

[0009] 3) According to the suspected target coordinate set The coordinates of the kth suspected target in From the previous frame image I t-1 Extract multiple candidate matching targets corresponding to the suspected target k to form a candidate matching target set, and obtain the image slice of each candidate matching target

[0010] 4) If the candidate matching target set corresponding to a suspected target k If it is empty, the suspected target k is determined to be a newly detected suspected target, and the information of the suspected target k is assigned to the mask set M{m×n}, and the process returns to step 3) to process the next suspected target until all suspected targets are traversed and the process goes to step 6); otherwise, the process goes to step 5); the mask set M{m×n} consists of elements with m rows and n columns, and m is equal to the image I t The number of pixels in the length direction, n is equal to the image I t The number of pixels in the width direction; each element contains information used to represent the corresponding pixel point in the image;

[0011] 5) Using the suspected target image slice obtained in step 1) and the candidate matching target image slice obtained in step 3) Determine the suspected target k and the corresponding J k The matching coefficients of candidate matching targets are used to extract candidate matching targets that meet the threshold requirements as candidate targets of suspected target k. For the mask set M{m×n} and the candidate target The information of the corresponding element of the position is updated; if a suspected target k does not have a corresponding candidate target The suspected target k is determined to be a newly detected suspected target, and the information of the suspected target k is assigned to the mask set M{m×n}, and then return to step 3) to process the next suspected target until all suspected targets are traversed and enter step 6);

[0012] 6) Using the mask set M{m×n}, the previous frame image I t-1 Corresponding background image Update and obtain the current frame image I t Corresponding background image When the suspected target set of the current frame is empty, the previous frame image I t Corresponding background image Equal to the previous frame image I t-1 Corresponding background image

[0013] 7) Using the mask set M{m×n}, determine whether the suspected target is a real target or an interference target, and update the information of the element corresponding to the suspected target in the mask set M{m×n};

[0014] 8) Using the information of the corresponding elements of the real target in the mask set M{m×n}, trajectory association is performed to complete the complete tracking of the same target.

[0015] Preferably, the information used to characterize the corresponding pixel point on the image in each element of the mask set M{m×n} is represented by 6 feature vectors, which are:

[0016] M{i}{1} is the target type of the position corresponding to the element M{i}; target types include: background point, corresponding M{i}{1} value is 0; interference target, corresponding M{i}{1} value is 1; suspected target, corresponding M{i}{1} value is 2; real target, corresponding M{i}{1} value is 3; 1≤i≤m×n;

[0017] M{i}{2} is the number of the target corresponding to the position of element M{i};

[0018] M{i}{3} is the number of pixels in the length and width directions of the image at the latest moment of the target corresponding to the element M{i};

[0019] M{i}{4} is the complete trajectory set of the target corresponding to the position of element M{i}; the complete trajectory set consists of the position information of the center element of the target corresponding to the trajectory in each frame image;

[0020] M{i}{5} is the speed of the target at the latest moment corresponding to the position of element M{i};

[0021] M{i}{6} is the latest moving direction of the target at the position corresponding to element M{i}.

[0022] Preferably, in step 1), a suspected target coordinate set consisting of multiple suspected target coordinates in the current frame is obtained. and each image slice of a suspected target The method is as follows:

[0023] 11) Solve the pixel difference between the current frame image and the previous frame image to obtain the difference image D between the current frame image and the previous frame image f =I t -I t-1 ; At the same time, solve the pixel difference between the current frame and the background frame image to obtain the difference image between the current frame image and the background frame image

[0024] 12) The difference image D f The absolute value is greater than Thr f The difference point or difference image D b The absolute value is greater than Thr b The difference points are retained as candidate points, and the pixel points corresponding to the candidate point positions are found from the current frame image, and the connected areas are generated. Then, the center coordinate points of these connected areas are recorded respectively. According to the target type of the element corresponding to the center coordinate point position in the mask set, the center coordinate points of the target type as the interference target are eliminated from the center coordinate points, and the remaining center coordinate points are taken as the center coordinates of the suspected target and added to the suspected target coordinate set to obtain the suspected target coordinate set. k=1,2,3,…,K;Thr f With Thr b The value range is 8 to 12;

[0025] 13) For the obtained suspected target coordinate set The coordinates of the kth suspected target in In the current frame image I t China-Israel An image block is extracted with the pixel point at the position as the center, edge detection is performed on the image block, the edge shape of the suspected target k is extracted, and an image slice consisting of pixels surrounded by the edge shape of the suspected target k is calculated;

[0026] 14) If the image slice size of the pixels surrounded by the edge shape of the suspected target k obtained in step 13) is not within the target prior size range, the suspected target k is removed from the suspected target coordinate set. Otherwise, the image slice obtained in step 13) is used as the image slice of the suspected target k

[0027] Preferably, the size of the image block in step 13) is larger than 1.25 to 1.7 times the maximum size of the target and smaller than 2 times the maximum size of the target.

[0028] Preferably, the step 3) is to start from the previous frame image I t-1 Extract multiple candidate matching targets corresponding to the suspected target k to form a candidate matching target set, and obtain the image slice of each candidate matching target The method is as follows:

[0029] 31) In the previous frame image I t-1 The same as in The pixel point at position is taken as the center, and an image block of size q×q is selected as the target candidate area;

[0030] 32) Combined with the mask set M{m×n}, from the target candidate area, select the elements whose target type of the elements in the corresponding mask set is not the background target as candidate matching targets, forming a candidate matching target set ,j=1,2,3,…,J k , where J k is the number of candidate matching targets in the target candidate area corresponding to the suspected target k;

[0031] 33) According to the information of the feature vector M{i}{3} in the mask set M{m×n}, obtain the image slice of each candidate matching target

[0032] Preferably, the value range of step 31) is as follows:

[0033] 3v·Δt / r≤q≤5v·Δt / r

[0034] Where r is the image resolution, v is the maximum motion speed of the target, and Δt is the time difference between adjacent frame images.

[0035] Preferably, the method for determining the matching coefficient in step 5) is specifically:

[0036]

[0037] Among them, α is the filter difference coefficient, and the value range of α is 0.01~0.2; is the adaptive filter DF k Slice the suspected target image The calculation results are: is the adaptive filter DF k Slice the candidate matching target image The calculation results of and Candidate matching target In the current frame image I t The predicted position in the row and column directions; is the coordinate of the kth suspected target.

[0038] Preferably, the and The method for determining is as follows:

[0039]

[0040] in, Candidate matching target In the previous frame image I t-1 The position coordinates in , Δt is the time interval between two adjacent frames, is the candidate matching target obtained by the mask set M{m×n} The speed of movement, is the candidate matching target obtained by the mask set M{m×n} The angle between the image and the row direction.

[0041] Preferably, the step 6) obtains the current frame image I t Corresponding background image The method is as follows:

[0042]

[0043] Among them, λ is the background update coefficient, and the value range of λ is 0.7~0.9; Indicates that according to the mask set, the current frame image I t The pixel values ​​of the pixels corresponding to the target type of background or interference target remain unchanged, and the pixel values ​​of the other pixels are processed to zero to obtain the pixel matrix; Indicates that in the current frame image I t In the above example, the pixel blocks corresponding to the suspected target and the real target are taken as the replaced area, and the replaced area is replaced as a whole with the pixel blocks of the neighborhood background, and the current frame image I is converted according to the mask set. t The pixel matrix obtained after the pixel points corresponding to the target type of background and interference target are zeroed; the value of each pixel in the pixel block of the neighborhood background is equal to the average value of the corresponding pixel in the four equal-sized pixel blocks in the up, down, left, and right directions outside the replaced area.

[0044] Preferably, the method of judging whether the suspected target is a real target or an interference target in step 7) is as follows: for the element whose feature vector target type is a suspected target in the mask set, when the number of times the target appears in the entire image sequence exceeds Num dOr when the target trajectory stops updating for more than Numo, the length of the complete trajectory set of the target corresponding to the element is calculated. If the trajectory length exceeds Thr track ; If the target type of the element is determined to be a real target, otherwise it is determined to be an interference target; Num d The value range of is equal to the number of image frames corresponding to 5~10s; the value range of Numo is equal to the number of image frames corresponding to 10~15s; Thr track The value range is 3 to 5.

[0045] Preferably, the method for performing trajectory association in step 8) is specifically as follows:

[0046] When the trajectory corresponding to the element whose target type is a real target in the mask set stops updating for more than Numo frames, it is determined that the real target movement is completed, and the target number and trajectory are added to the trajectory set Track;

[0047] When a new trajectory is added to the trajectory set, the trajectory is predicted based on the speed and direction of the end point of the known trajectory in the trajectory set. The predicted position of the known trajectory in the corresponding new trajectory is obtained. The target is fully tracked by comparing the starting position of the new trajectory with the speed and direction of the target corresponding to the new and old trajectories. The value range of Numo is equal to the number of image frames corresponding to 10 to 15 seconds.

[0048] Preferably: initial background image The pixel value of each pixel in is equal to the average value of the pixels at the corresponding position in the N frames of image initially obtained, and the value range of N is 5 to 10.

[0049] The advantages of the present invention compared with the prior art are:

[0050] (1) The present invention generates a target mask to record various types of target information, thereby significantly reducing the impact of interfering targets during detection and improving detection accuracy.

[0051] (2) In view of the fact that sensitive targets show dynamic changes during flight, the present invention constructs an adaptive filter that can extract the characteristic information of the target at all times, thereby increasing the matching accuracy.

[0052] (3) The present invention associates the trajectories of all real targets, which can solve the target occlusion problem to a certain extent. Combined with the target features contained in the mask, it can generate complete situation information of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Flow chart of the method of the present invention;

[0054] Figure 2This is a diagram of the adaptive filter generation method of the present invention. DETAILED DESCRIPTION

[0055] For satellite-based infrared reconnaissance and early warning systems, the reliability and timeliness of intelligence information are key to battlefield support, especially for high-speed flying targets. Only real-time or quasi-real-time detection and tracking can truly realize the role of space-based reconnaissance and early warning systems. Therefore, the frame difference method and background difference method are used for joint detection to achieve rapid processing of massive infrared image sequences. However, such methods are susceptible to interference from noise, bad pixels, bright spots, and moving clouds. These interferences are significantly different from the surrounding environment and, similar to real infrared weak targets, often appear in the field of view as spots or dots, resulting in high false alarms when detecting real targets. Moreover, tracking can also occur when the target is obscured. However, unlike real targets, the shape features and motion characteristics of these interference targets are somewhat different from those of real targets. Therefore, based on this prior knowledge, the present invention uses image sequences to generate a mask information, extracts the complete outline features of various targets through edge detection operators to support detection, and simultaneously stores and analyzes various important feature information during the detection and tracking process in real time. This information is used to construct an adaptive filter to further confirm the real target, significantly reduce false alarms, and improve the reliability of intelligence information while ensuring real-time processing.

[0056] like Figure 1 As shown, the specific implementation steps of the method of the present invention are as follows:

[0057] (1) Based on the size of the acquired remote sensing infrared image (m×n), an initial mask set M{m×n} is pre-generated, where m is the number of pixels in each row of the remote sensing infrared image and n is the number of pixels in each column of the remote sensing infrared image. That is, M{m×n} contains information about all observation points in the camera's field of view. For any element M{i}, 1≤i≤m×n, the position corresponding to the element in the remote sensing image's field of view can be found based on its index i. Each element M{i} in the mask set M{m×n} corresponds to six eigenvectors M{i}{j}, 1≤i≤m×n, and 1≤j≤6.

[0058] in,

[0059] Mi{i}{1} represents the target type at the corresponding position of the element. Target types include: background points, corresponding to Mi{i}{1} values ​​of 0; interference targets, corresponding to Mi{i}{1} values ​​of 1; suspected targets, corresponding to Mi{i}{1} values ​​of 2; and true targets, corresponding to Mi{i}{1} values ​​of 3. Only the center point element of the target slice has a value; the rest are empty. Suspected targets include interference targets and true targets. For background points, Mi{i} contains one eigenvector; the rest are empty. For suspected targets, interference targets, and true targets, Mi{i} contains six eigenvectors.

[0060] M{i}{2} is the number of the target to which the corresponding position of the element belongs;

[0061] M{i}{3} is the size feature of the target at the corresponding position of the element at the latest moment (i.e., the number of pixels in the length and width directions of the remote sensing infrared image);

[0062] M{i}{4} is the complete trajectory set of the target corresponding to the element; the complete trajectory set consists of the position information of the central element of the target corresponding to the trajectory in each frame image;

[0063] M{i}{5} is the speed of the target at the corresponding position of the element at the latest moment;

[0064] M{i}{6} is the latest moving direction of the target at the corresponding position of this element.

[0065] The initial value of the feature vector of each element in the initial mask set is set to 0, that is, it is assumed that the initial target type of all pixels is background points.

[0066] (2) Based on the N frames of remote sensing infrared images initially obtained (N is generally 5 to 10 images) {I 1 ,I 2 ,...,I N}, sum and average the pixels at each corresponding position to obtain the initial background image The pixel value of each pixel in the initial background image is equal to the average value of the pixels at the corresponding position in the N frames of remote sensing infrared images.

[0067] (3) Using the initial background image, the remote sensing infrared image of the current frame is used to extract suspected targets, and a set of suspected target coordinates consisting of the suspected target coordinates in the current frame and image slices of each suspected target are obtained. If the suspected target set of the current frame is empty, repeat step (3) to extract the suspected target from the remote sensing infrared image of the next frame until the suspected target coordinate set is not empty, then enter step (4); if the suspected target set of the current frame is empty, then the background image of the current frame Same background image as the previous frame;

[0068] (31) Using the difference between the current frame image and the previous frame image, as well as the difference between the current frame image and the previous frame image, The target is detected by the difference of the background frame image, and the pixel difference between the current frame image and the previous frame image is solved to obtain the difference image D between the current frame image and the previous frame image. f =I t -I t-1; At the same time, solve the pixel difference between the current frame and the background frame image to obtain the difference image between the current frame image and the background frame image Each point in the difference image is considered a difference point. The pixel value of each difference point in the difference image between the current frame image and the previous frame image is equal to the difference between the value of the corresponding pixel point in the current frame image and the value of the corresponding pixel point in the previous frame image. The pixel value of each difference point in the difference image between the current frame image and the background frame image is equal to the difference between the value of the corresponding pixel point in the current frame image and the value of the corresponding pixel point in the background frame image.

[0069] (32) The difference image D f The absolute value is greater than Thr f The difference point or difference image D b The absolute value is greater than Thr b The difference points are retained (statistically, Thr f With Thr b The value range is generally 8 to 12) as the candidate point, find the pixel point corresponding to the candidate point position from the current frame image, and use the connected domain marking method (a mature method) to connect these pixel points to generate K connected areas, and then record the center coordinate points of these connected areas respectively. According to the target type of the element corresponding to the center coordinate point position in the mask set, the center coordinate point of the target type is eliminated from the center coordinate point. The remaining center coordinate points are used as the center coordinates of the suspected target and added to the suspected target coordinate set to obtain the suspected target coordinate set. k=1,2,3,…,K, where Indicates that the center of the kth suspected target is in the observation field of view Column, No. The pixel point (position) of the row. (When the suspected target coordinate set is initially obtained, the target type of each element in the suspected target coordinate set is a suspected target, that is,

[0070] (33) For the coordinates of the kth suspected target in the suspected target coordinate set obtained in step (32), In the current frame image I t China-Israel An image block is extracted with the pixel point at the position as the center. The size of the image block is larger than 1.25 to 1.7 times the maximum size of the target and smaller than 2 times the maximum size of the target. (The reason for this selection is that the orbit of infrared remote sensing observation satellites is fixed and their resolution is also determinable. Therefore, this prior knowledge can be used to statistically obtain the size range of the target in the observed image). The "sobel" operator is used to perform edge detection on the image block, extract the edge shape of the suspected target k, and calculate the size of the suspected target and the pixels surrounded by the edge shape to form an image slice of the suspected target. If the size of the image slice of the suspected target k is not within the size range of the statistically obtained target corresponding observation image, then the target k is removed from the suspected target coordinate set. Eliminate them, traverse all elements in the suspected target coordinate set, and then go to step (4);

[0071] (4) Based on the suspected target coordinate set obtained in step (3), extract the candidate matching target set and candidate matching target image slice corresponding to each suspected target from the previous frame image.

[0072] (41) For the suspected target k, in the previous frame image I t-1 The same as in The pixel point at the position is taken as the center, and an image block of size q×q is selected (the image block size q is mainly determined by combining the image resolution r and the maximum motion speed v of the target, 3v·Δt / r≤q≤5v·Δt / r, where Δt is the time difference between adjacent frame images) as the target candidate area;

[0073] (42) Combined with the mask set M{m×n}, from the target candidate area, select the points whose target type of the corresponding elements in the mask set is not a background target as candidate matching targets to form a candidate matching target set ,j=1,2,3,…,J k , where J k is the number of candidate matching targets in the target candidate area corresponding to the suspected target k. According to the information of the feature vector Mi{i}{3} in the mask set M{m×n}, the image slice of each candidate matching target is obtained.

[0074] (43) Repeat steps (41) to (42) K times to obtain a set of candidate matching targets for each suspected target.

[0075] (5) If the candidate matching target set corresponding to a suspected target k If it is empty, it means that the suspected target k in the current frame is a newly detected suspected target (corresponding to The value is 2), assign the new number to the mask set , and assign the size feature and target position to and Complete the target detection of this frame and go to step (7);

[0076] If the candidate matching target set {T P _k(j)} is not empty, go to step (6);

[0077] (6) Using the suspected target image slice obtained in step (3) and the candidate matching target image slice obtained in step (4) Determine each suspected target and the corresponding J k The matching coefficients of candidate matching targets are used to extract candidate matching targets that meet the threshold requirements as candidate targets of suspected target k. According to the candidate target Get the movement direction and speed of the suspected target k, and The corresponding element's feature vector is updated; if a suspected target k does not have a corresponding candidate target Then define the suspected target k as the newly detected suspected target (i.e. ), update the feature vector of the element at the corresponding position in the mask;

[0078] (61) Using the suspected target image slice obtained in step (3) and the candidate matching target image slice obtained in step (4) Using an adaptive filter DF of the same size as the target k The candidate matching target set corresponding to the suspected target k and the suspected target k All candidate matching targets are matched, and the matching coefficient is calculated as follows:

[0079]

[0080] In the above formula, α is the filter difference coefficient (α is generally 0.01 to 0.2, which is negatively correlated with the dynamic range of the image pixel value). is the adaptive filter DF k Slice the suspected target image The calculation results are: is the adaptive filter DF k Slice the candidate matching target image The calculation results of is the suspected target k in image I t The matrix composed of all pixels occupied by , that is, the suspected target image slice (the slice is mainly extracted using the edge detection result in step 33), For the jth candidate matching target In image I t-1 The candidate matching target image slice in (represents the matrix composed of all pixels occupied by the target and scaled to the same size as image I t same size), and Candidate matching target In the current frame image I t The predicted position in the row and column directions is calculated as follows:

[0081]

[0082] In the above formula, Candidate matching target In image I t-1 The position in the frame, Δt is the time interval between two adjacent frames, and Candidate matching targets The movement speed and movement direction (the direction is the angle between the movement direction and the row direction x).

[0083] When constructing a dynamic filter, since the imaging size of the target in the infrared remote sensing image generally changes continuously within the range of 1×1 to 10×10, and the tail flame temperature of the high-speed moving target is high, the infrared camera will also record part of the tail flame during imaging, resulting in the formation of an elliptical bright spot with a direction in the image when the target moves at high speed. The axis direction of the bright spot is close to the target movement direction. For dynamically changing infrared weak targets, if a fixed filter is used to extract their features, it is difficult to achieve stable capture of the target at all times. Therefore, the present invention designs a rectangular adaptive filter with an angle based on the target characteristics. The filter is a 5×5 filter DF. base As a benchmark, according to the length l of target k k and width k Generate adaptive filter DF k , DF k The length is l k +2, width is w k +2, whose filter coefficient is determined by DF base The coefficients are interpolated or sampled, and their direction is the same as the axis direction of the target k, such as Figure 2 shown.

[0084] (62) It can be seen that the suspected target k and the candidate matching target The greater the difference, the better the matching coefficient. The bigger; The smaller the value, the more likely the suspected target k is to be matched with the candidate target. The more similar, the more satisfied (Thr R It is a statistic. The design varies according to the task. In this example, among the candidate matching targets of 2 to 5), the candidate matching target with the smallest matching coefficient is selected as the candidate matching target k. Candidate target The target is the same as the suspected target k. The movement direction and speed of target k can be calculated by the change of the position between the current frame and the previous frame. On this basis, the type and number of the target remain unchanged, and the target size, trajectory set, speed, and movement direction are characterized in the mask. Update and remove the mask If there is no historical feature information with a matching coefficient less than Thr R The target is also considered as a new target, so a new number is created for it and Update the target number, size, and trajectory.

[0085] (7) After completing the current frame image I t After detection, the mask M is used to complete the background image corresponding to the current frame The pixel values ​​corresponding to the suspected target and the real target are updated. The updating principle is:

[0086]

[0087] In the above formula, λ is the background update coefficient, which is generally between (0.7 and 0.9). Indicates that according to the mask set, from the current frame image I t The pixel values ​​of the pixels whose target type is background or interference target are extracted remain unchanged, and the pixel values ​​of the remaining pixels (suspected target and real target pixel points) are processed to zero to obtain the pixel matrix (the pixel points corresponding to the background, interference target, suspected target and real target can be obtained according to the values ​​of Mi{i}{1} and Mi{i}{3}). Indicates that in the current frame image I t On this basis, the pixel blocks corresponding to the suspected target and the real target are replaced as a whole with the neighborhood background of the target (the neighborhood background indicates the target in the image I t The size of the neighborhood background pixel block depends on the size of the corresponding target. It is obtained by averaging the weighted sum of four background pixel blocks of the same size around the target slice, and the pixels corresponding to the background and the interference target are set to zero.

[0088] (8) For the elements whose feature vector target type in the mask set is a suspected target, when the number of times the target appears in the entire image sequence exceeds Num d (generally the number of image frames corresponding to 5 to 10 seconds of imaging) or the target trajectory stops updating for more than Numo (generally the number of image frames corresponding to 10 to 15 seconds of imaging, that is, if the target has no new trajectory in the image sequence of 10 to 15 seconds, it is determined that the target has disappeared) frames, the trajectory length is calculated, and if the trajectory length exceeds Thr track (Thr track The value range is 3 to 5, and in the embodiment of the present invention, it is 3, that is, the target moves more than 3 pixels), then the target type of the element is determined to be a real target, otherwise it is determined to be an interference target, and the mask M is updated. The updating principle is: if it is a real target, The target type, number, shape characteristics, target speed, and movement direction are updated at this point; if it is an interference target, only the type information is retained, and the values ​​of the feature vectors Mi{i}{2} to Mi{i}{6} are all empty, that is:

[0089]

[0090] (9) For the element O in the mask set whose feature vector target type is a real target T When the target trajectory stops updating for more than Numo frames, the target movement is determined to be complete, and the target number and trajectory are added to the track set Track. is the coordinate position of the lth track in the tth frame image, t b (l) is the starting frame of the lth trajectory, t e (l) is the frame where the end of the lth track appears. When the new track Track(l+1) is added to the track set, the trajectory prediction is performed based on the movement speed and direction of the end track point in the track, and its predicted position in the corresponding new track is obtained. By comparing it with the starting position of the new track and the movement speed and direction of the target corresponding to the new and old tracks, the target is fully tracked.

[0091] Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention are within the scope of protection of the technical solutions of the present invention. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0092] The contents not described in detail in the specification of the present invention belong to the common knowledge of those skilled in the art.

Claims

1. A method for detecting and tracking infrared small targets based on mask and adaptive filtering, characterized in that: The steps include: 1) Using the previous frame image I t-1 Corresponding background image From the image of the current frame I t Extract multiple suspected targets and obtain a suspected target coordinate set consisting of multiple suspected target coordinates in the current frame and each image slice of a suspected target If the suspected target set of the current frame is empty, go to step 6); otherwise go to step 2); 2) For the suspected target coordinate set obtained in step 1) The coordinates of the kth suspected target in In the current frame image I t China-Israel An image block is extracted as the center of the pixel point at the position, and edge detection is performed on the image block. The suspected targets that do not meet the target size range are removed from the suspected target coordinate set. Eliminate and traverse the suspected target coordinate set All elements in , then go to step 3); 3) According to the suspected target coordinate set The coordinates of the kth suspected target in From the previous frame image I t-1 Extract multiple candidate matching targets corresponding to the suspected target k to form a candidate matching target set, and obtain the image slice of each candidate matching target 4) If the candidate matching target set corresponding to a suspected target k If it is empty, the suspected target k is determined to be a newly detected suspected target, and the information of the suspected target k is assigned to the mask set M{m×n}, and the process returns to step 3) to process the next suspected target until all suspected targets are traversed and the process goes to step 6); Otherwise, go to step 5); the mask set M{m×n} consists of elements in m rows and n columns, where m is equal to the image I t The number of pixels in the length direction, n is equal to the image I t The number of pixels in the width direction; each element contains information used to represent the corresponding pixel point in the image; 5) Using the suspected target image slice obtained in step 1) and the candidate matching target image slice obtained in step 3) Determine the suspected target k and the corresponding J k The matching coefficients of candidate matching targets are used to extract candidate matching targets that meet the threshold requirements as candidate targets of suspected target k. For the mask set M{m×n} and the candidate target The information of the corresponding element of the position is updated; if a suspected target k does not have a corresponding candidate target The suspected target k is determined to be a newly detected suspected target, and the information of the suspected target k is assigned to the mask set M{m×n}, and then return to step 3) to process the next suspected target until all suspected targets are traversed and enter step 6); 6) Using the mask set M{m×n}, the previous frame image I t-1 Corresponding background image Update and obtain the current frame image I t Corresponding background image When the suspected target set of the current frame is empty, the previous frame image I t Corresponding background image Equal to the previous frame image I t-1 Corresponding background image 7) Using the mask set M{m×n}, determine whether the suspected target is a real target or an interference target, and update the information of the element corresponding to the suspected target in the mask set M{m×n}; 8) Using the information of the corresponding elements of the real target in the mask set M{m×n}, trajectory association is performed to complete the complete tracking of the same target.

2. The infrared small target detection and tracking method based on mask and adaptive filtering according to claim 1, characterized in that: The information of each element in the mask set M{m×n} used to characterize the corresponding pixel on the image is represented by 6 feature vectors, which are: M{i}{1} is the target type of the corresponding position of element M{i}; Target types include: background points, corresponding to Mi{i}{1} value 0; interference targets, corresponding to Mi{i}{1} value 1; suspected targets, corresponding to Mi{i}{1} value 2; real targets, corresponding to Mi{i}{1} value 3; 1≤i≤m×n; M{i}{2} is the number of the target corresponding to the position of element M{i}; M{i}{3} is the number of pixels in the length and width directions of the image at the latest moment of the target corresponding to the element M{i}; M{i}{4} is the complete trajectory set of the target corresponding to the position of element M{i}; the complete trajectory set consists of the position information of the center element of the target corresponding to the trajectory in each frame image; M{i}{5} is the speed of the target at the latest moment corresponding to the position of element M{i}; M{i}{6} is the latest moving direction of the target at the position corresponding to element M{i}.

3. The infrared small target detection and tracking method based on mask and adaptive filtering according to claim 2, characterized in that: In step 1), a suspected target coordinate set consisting of multiple suspected target coordinates in the current frame is obtained. and each image slice of a suspected target The method is as follows: 11) Solve the pixel difference between the current frame image and the previous frame image to obtain the difference image D between the current frame image and the previous frame image f =I t -I t-1 ; At the same time, solve the pixel difference between the current frame and the background frame image to obtain the difference image between the current frame image and the background frame image 12) The difference image D f The absolute value is greater than Thr f The difference point or difference image D b The absolute value is greater than Thr b The difference points are retained as candidate points, and the pixel points corresponding to the candidate point positions are found from the current frame image, and the connected areas are generated. Then, the center coordinate points of these connected areas are recorded respectively. According to the target type of the element corresponding to the center coordinate point position in the mask set, the center coordinate points of the target type as the interference target are eliminated from the center coordinate points, and the remaining center coordinate points are taken as the center coordinates of the suspected target and added to the suspected target coordinate set to obtain the suspected target coordinate set. Thr f With Thr b The value range is 8 to 12; 13) For the obtained suspected target coordinate set The coordinates of the kth suspected target in In the current frame image I t China-Israel An image block is extracted with the pixel point at the position as the center, edge detection is performed on the image block, the edge shape of the suspected target k is extracted, and an image slice consisting of pixels surrounded by the edge shape of the suspected target k is calculated; 14) If the image slice size of the pixels surrounded by the edge shape of the suspected target k obtained in step 13) is not within the target prior size range, the suspected target k is removed from the suspected target coordinate set. Otherwise, the image slice obtained in step 13) is used as the image slice of the suspected target k 4. The infrared small target detection and tracking method based on mask and adaptive filtering according to claim 3, characterized in that: In step 13), the size of the image block is larger than 1.25 to 1.7 times the maximum size of the target and smaller than 2 times the maximum size of the target.

5. The infrared small target detection and tracking method based on mask and adaptive filtering according to claim 2, characterized in that: The step 3) is to obtain the image from the previous frame I t-1 Extract multiple candidate matching targets corresponding to the suspected target k to form a candidate matching target set, and obtain the image slice of each candidate matching target The method is as follows: 31) In the previous frame image I t-1 The same as in The pixel point at position is taken as the center, and an image block of size q×q is selected as the target candidate area; 32) Combined with the mask set M{m×n}, from the target candidate area, select the elements whose target type of the elements in the corresponding mask set is not a background target as candidate matching targets to form a candidate matching target set Among them, J k is the number of candidate matching targets in the target candidate area corresponding to the suspected target k; 33) According to the information of the feature vector M{i}{3} in the mask set M{m×n}, obtain the image slice of each candidate matching target 6. The infrared small target detection and tracking method based on mask and adaptive filtering according to claim 5, characterized in that: The value range of q in step 31) is as follows: 3v·Δt / r≤q≤5v·Δt / r Where r is the image resolution, v is the maximum target motion speed, and Δt is the time difference between adjacent frame images.

7. The infrared small target detection and tracking method based on mask and adaptive filtering according to claim 2, characterized in that: Step 5) The method for determining the matching coefficient is as follows: Among them, α is the filter difference coefficient, and the value range of α is 0.01~0.2; is the adaptive filter DF k Slice the suspected target image The calculation results are: is the adaptive filter DF k Slice the candidate matching target image The calculation results of and Candidate matching target In the current frame image I t The predicted position in the row and column directions; is the coordinate of the kth suspected target.

8. The infrared small target detection and tracking method based on mask and adaptive filtering according to claim 7, characterized in that: described and The method for determining is as follows: in, Candidate matching target In the previous frame image I t-1 The position coordinates in , Δt is the time interval between two adjacent frames, is the candidate matching target obtained by the mask set M{m×n} The speed of movement, is the candidate matching target obtained by the mask set M{m×n} The angle between the image and the row direction.

9. The infrared small target detection and tracking method based on mask and adaptive filtering according to claim 2, characterized in that: Step 6) obtaining the current frame image I t Corresponding background image The method is as follows: Among them, λ is the background update coefficient, and the value range of λ is 0.7~0.9; Indicates that according to the mask set, the current frame image I t The pixel values ​​of the pixels corresponding to the target type of background or interference target remain unchanged, and the pixel values ​​of the other pixels are processed to zero to obtain the pixel matrix; Indicates that in the current frame image I t In the above example, the pixel blocks corresponding to the suspected target and the real target are taken as the replaced area, and the replaced area is replaced as a whole with the pixel blocks of the neighborhood background, and the current frame image I is converted according to the mask set. t The pixel matrix obtained after the pixel points corresponding to the target type of background and interference target are zeroed; the value of each pixel in the pixel block of the neighborhood background is equal to the average value of the corresponding pixel in the four equal-sized pixel blocks in the up, down, left, and right directions outside the replaced area.

10. The infrared small target detection and tracking method based on mask and adaptive filtering according to any one of claims 1 to 9, characterized in that: The method of step 7) judging whether the suspected target is a real target or an interference target is as follows: for the element whose feature vector target type in the mask set is a suspected target, when the number of times the target appears in the entire image sequence exceeds Num d Or the target trajectory stops updating for more than Num o When , the length of the complete trajectory set of the target corresponding to the element is calculated. If the trajectory length exceeds Thr track ; If the target type of the element is determined to be a real target, otherwise it is determined to be an interference target; Num d The value range is equal to the number of image frames corresponding to 5 to 10 seconds; Num o The value range of Thr is equal to the number of image frames corresponding to 10 to 15 seconds; track The value range is 3 to 5.

11. The infrared small target detection and tracking method based on mask and adaptive filtering according to any one of claims 1 to 9, characterized in that: The method for performing trajectory association in step 8) is specifically as follows: When the target type of the element in the mask set is a real target, the corresponding trajectory stops updating and exceeds Num o When the frame is reached, the real target is judged to have completed its motion, and the target number and trajectory are added to the trajectory set Track; When a new trajectory is added to the trajectory set, the trajectory is predicted based on the movement speed and direction of the end trajectory point of the known trajectory in the trajectory set, and the predicted position of the known trajectory in the corresponding new trajectory is obtained. By comparing the starting position of the new trajectory with the movement speed and direction of the target corresponding to the new and old trajectories, the target is fully tracked; Num o The value range is equal to the number of image frames corresponding to 10 to 15 seconds.

12. The infrared small target detection and tracking method based on mask and adaptive filtering according to any one of claims 1 to 9, characterized in that: Initial background image The pixel value of each pixel in is equal to the average value of the pixels at the corresponding position in the N frames of image initially obtained, and the value range of N is 5 to 10.

Images