A method for detecting and tracking high-speed moving targets in the air based on continuous frames

Abstract

This invention provides a method for detecting and tracking high-speed moving targets in the air based on consecutive frames. It highlights target motion information by using frame differences between adjacent frames, extracts target candidate regions after target saliency enhancement, and further refines target discrimination based on target motion characteristics after false alarm rejection, thus accurately detecting the target and solving the problem of low target detection accuracy. Finally, it utilizes least-squares trajectory prediction technology to predict and detect simultaneously, enabling stable long-term tracking of the detected target. This invention significantly improves the performance of detecting high-speed moving targets in the air in remote sensing images and the accuracy during stable tracking, solving the problem of difficulty in extracting detailed feature information of high-speed moving targets in the air against complex backgrounds in visible light / infrared remote sensing image sequences.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a method for detecting and tracking high-speed moving targets in the air based on continuous frames. Background Technology

[0002] Against the complex background of remote sensing images, high-speed moving targets in the air have few pixels, and their features such as edges and textures are difficult to describe. They are highly maneuverable, and the contrast between the target and the surrounding background is low, making it difficult for traditional region extraction techniques to achieve high accuracy in detecting and tracking high-speed moving targets in the air for long periods of time. Summary of the Invention

[0003] To address the issues of poor detection performance and low tracking accuracy in existing technologies, this invention provides a method for detecting and tracking high-speed moving targets in the air based on continuous frames, which can accurately capture and continuously monitor and track high-speed moving targets in the air in remote sensing images.

[0004] A method for detecting and tracking high-speed moving targets in the air based on continuous frames includes the following steps:

[0005] S1: Perform multi-directional gradient target saliency enhancement on the frame difference map of two adjacent frames to obtain the enhanced image;

[0006] S2: The local extremum method is used to extract target candidate regions on the enhanced image;

[0007] S3: False alarm removal is performed on the extracted target candidate regions based on differences in motion direction;

[0008] S4: Utilize the target's motion characteristics to determine the target's coordinate information in the remaining target candidate region;

[0009] S5: Based on the target's coordinate information, the least squares prediction technique is used to track the target.

[0010] Furthermore, the method for obtaining the enhanced image is as follows:

[0011] Frame difference is calculated between two adjacent frames to obtain a frame difference map that highlights the differences in target motion;

[0012] The gradient detection function Δ(m,n) for each pixel on the frame difference map is obtained as follows:

[0013] Δ(m,n)=Δ(x+)*Δ(x-)*Δ(y+)*Δ(y-)

[0014] Where Δ(x+) represents the gradient of the pixel (m,n) in the m-th row and n-th column in the positive horizontal direction x+, Δ(x-) represents the gradient of the pixel (m,n) in the m-th row and n-th column in the negative horizontal direction x-, Δ(y+) represents the gradient of the pixel (m,n) in the m-th row and n-th column in the positive vertical direction y+, and Δ(y-) represents the gradient of the pixel (m,n) in the m-th row and n-th column in the negative vertical direction y-.

[0015] Pixels with gradient detection function Δ(m,n) = 0 are designated as background, and pixels with gradient detection function Δ(m,n) = 1 are designated as target, thus obtaining an enhanced image. The gradient detection function Δ(m,n) = 1 is only defined when Δ(x+), Δ(x-), Δ(y+), and Δ(y-) are all greater than a set value.

[0016] Furthermore, the method of extracting target candidate regions from the enhanced image using local extrema specifically involves:

[0017] The image is enhanced by traversing a sliding window of size S*S with a step size of S / 2. For each region extracted by each sliding window, the maximum value of the region is obtained, and then the seed points are increased according to the maximum value until the increase value is greater than the set value. After the increase stops, the pixels accumulated during the seed point increase process are used as the target candidate regions in the current sliding window.

[0018] Furthermore, determining the target's coordinate information in the remaining target candidate region using the target's motion characteristics specifically involves:

[0019] Perform the following operations on each candidate target as the current candidate target:

[0020] S41: In the current frame image, with the current candidate target as the center and the maximum inter-frame motion distance of the target as the radius, search for other candidate targets that fall within the radius, and connect them in pairs to form target pairs. Determine the identity of the target features of the matching target pairs and remove target pairs that do not meet the identity condition. The maximum inter-frame motion distance of the target is the maximum offset distance of the target between two adjacent frames calculated by the target motion speed, image resolution and imaging frame rate.

[0021] S42: Based on the extracted target pairs, the least squares method is used to predict the possible position of the current candidate target in the next frame image. Then, the possible position is used as the predicted position on the frame difference map between the next frame image and the next frame image after that, and the target is searched around the predicted position.

[0022] S43: Using target characteristics and motion characteristics, the targets extracted in step S42 and the targets that meet the same identity conditions in step S41 are reconfirmed for identity, and false alarms are eliminated again.

[0023] S44: Based on the target trajectory coordinates obtained from 3 consecutive frames, calculate the target heading and speed, and perform target identification. If a real target is detected, determine the coordinate information of the real target. If no target is detected, take the next frame image as the new current frame image and repeat steps S41 to S43 until a real target is detected.

[0024] Furthermore, based on the target's coordinate information, the least squares prediction technique is used to track the target, specifically as follows:

[0025] S51: Set the initial tracking confidence value to N, and predict the target position of the next frame based on the latest M-frame target position information that has been tracked. Search for the target within a range centered on the prediction point and with a radius of r. If the search finds a target, the tracking is considered normal and the tracking confidence is incremented by 1. If the search does not find a target, the tracking is considered abnormal and the tracking confidence is decremented by 1.

[0026] S52: Determine if the current tracking confidence is greater than the initial value N. If it is greater than N, reset the tracking confidence to N, consider the tracking to be normal, continue target tracking in the next frame, and output the tracking chain and target slice information. If the current tracking confidence is less than N but greater than 0, keep the tracking confidence unchanged, consider the tracking to be normal, continue target tracking in the next frame, and output the tracking chain and target slice information. If the current tracking confidence is 0 or less than 0, consider the tracking to be abnormal, the target is lost, and repeat steps S1 to S4 for target detection.

[0027] Beneficial effects:

[0028] 1. This invention provides a method for detecting and tracking high-speed moving targets in the air based on continuous frames. It highlights target motion information by using frame differences between adjacent frames, extracts target candidate regions after target saliency enhancement, and then uses target motion characteristics for fine-grained target discrimination after false alarm elimination, accurately detecting the target and solving the problem of low target detection accuracy. Finally, it utilizes least-squares trajectory prediction technology to predict and detect simultaneously, enabling stable long-term tracking of the detected target. This invention can significantly improve the detection performance of high-speed moving targets in the air in remote sensing images and the accuracy during stable tracking, solving the problem of difficulty in extracting detailed feature information of high-speed moving targets in the air against complex backgrounds in visible light / infrared remote sensing image sequences.

[0029] 2. This invention provides a method for detecting and tracking high-speed moving targets in the air based on continuous frames. The frame difference between two adjacent frames can eliminate most of the background interference. Based on the frame difference map, the target is enhanced with multi-directional gradient saliency, which can improve the target detection probability.

[0030] 3. This invention provides a method for detecting and tracking high-speed moving targets in the air based on continuous frames. It uses gradients in four directions—positive vertical y+, negative vertical y-, positive horizontal x+, and negative horizontal x-—to construct the gradient detection function for each pixel on the frame difference map. This simplifies the analysis process of the enhanced image and reduces the amount of computation required to obtain the enhanced image.

[0031] 4. This invention provides a method for detecting and tracking high-speed moving targets in the air based on continuous frames. Based on the characteristic that targets in the local background of enhanced images have strong salience, the method of local extrema is used to extract target candidate regions in the enhanced images, which can solve the problem of difficulty in target extraction due to the low contrast between the overall gray level of the target and the surrounding background in the image. Attached Figure Description

[0032] Figure 1 The flowchart illustrates a method for detecting and tracking high-speed moving targets in the air based on continuous frames, as provided by this invention. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0034] like Figure 1 As shown, a method for detecting and tracking high-speed moving targets in the air based on continuous frames includes the following steps:

[0035] S1: Perform multi-directional gradient target saliency enhancement on the frame difference map of two adjacent frames to obtain the enhanced image, specifically:

[0036] The method for obtaining the enhanced image is as follows:

[0037] Frame difference is calculated between two adjacent frames to obtain a frame difference map that highlights the differences in target motion;

[0038] The gradient detection function Δ(m,n) for each pixel on the frame difference map is obtained as follows:

[0039] Δ(m,n)=Δ(x+)*Δ(x-)*Δ(y+)*Δ(y-)

[0040] Where Δ(x+) represents the gradient of the pixel (m,n) in the m-th row and n-th column in the positive horizontal direction x+, Δ(x-) represents the gradient of the pixel (m,n) in the m-th row and n-th column in the negative horizontal direction x-, Δ(y+) represents the gradient of the pixel (m,n) in the m-th row and n-th column in the positive vertical direction y+, and Δ(y-) represents the gradient of the pixel (m,n) in the m-th row and n-th column in the negative vertical direction y-.

[0041] Pixels with gradient detection function Δ(m,n) = 0 are designated as background, and pixels with gradient detection function Δ(m,n) = 1 are designated as target, thus obtaining an enhanced image. The gradient detection function Δ(m,n) = 1 is only defined when Δ(x+), Δ(x-), Δ(y+), and Δ(y-) are all greater than a set value.

[0042] It should be noted that, theoretically, "multi-directional gradient" actually refers to the 360-degree direction on a planar image; however, for the sake of simplicity and reduced computation, this invention refers to multi-directional gradient only in four directions: positive vertical y+, negative vertical y-, positive horizontal x+, and negative horizontal x-. The gradient detection function Δ(m,n) is only 1 when the gradients at a certain position (m,n) are all high along all four directions, exceeding a certain threshold; otherwise, it is 0. The multi-directional gradient function values ​​for each candidate target point are calculated according to the above formula. Points with a value of 1 are retained, while points with a value of 0 are considered background and should be discarded.

[0043] S2: The local extremum method is used to extract target candidate regions on the enhanced image;

[0044] It should be noted that in images, the overall grayscale of the target often has low contrast with the surrounding background, making target extraction difficult. Observation revealed that in enhanced images, targets in local backgrounds exhibit strong salience. Based on this characteristic, this invention employs a local extremum method to extract target candidate regions, specifically:

[0045] The image is enhanced by traversing a sliding window of size S*S with a step size of S / 2. For each region extracted by each sliding window, the maximum value of the region is obtained, and then the seed points are increased according to the maximum value until the increase value is greater than the set value. After the increase stops, the pixels accumulated during the seed point increase process are used as the target candidate regions in the current sliding window.

[0046] S3: False alarm removal is performed on the extracted target candidate regions based on differences in motion direction;

[0047] It should be noted that for image frame sequences where complex fragmented clouds and targets coexist, since there is a significant difference in the direction of motion between the target and the cloud background, a technique based on the difference in motion direction is used in target detection. Based on the direction information of the target and the false alarm, most of the false alarms in the same direction are eliminated.

[0048] S4: Utilize the target's motion characteristics to determine the target's coordinate information within the remaining target candidate region, specifically:

[0049] Perform the following operations on each candidate target as the current candidate target:

[0050] S41: In the current frame image, with the current candidate target as the center and the maximum inter-frame motion distance of the target as the radius, search for other candidate targets that fall within the radius, and connect them in pairs to form target pairs. Determine the identity of the target features of the matching target pairs and remove target pairs that do not meet the identity condition. The maximum inter-frame motion distance of the target is the maximum offset distance of the target between two adjacent frames calculated by the target motion speed, image resolution and imaging frame rate.

[0051] S42: Based on the extracted target pairs, the least squares method is used to predict the possible position of the current candidate target in the next frame image. Then, the possible position is used as the predicted position on the frame difference map between the next frame image and the next frame image after that, and the target is searched around the predicted position.

[0052] S43: Using target characteristics and motion characteristics, the targets extracted in step S42 and the targets that meet the same identity conditions in step S41 are reconfirmed for identity, and false alarms are eliminated again.

[0053] S44: Based on the target trajectory coordinates obtained from 3 consecutive frames, calculate the target heading and speed, and perform target identification. If a real target is detected, determine the coordinate information of the real target. If no target is detected, take the next frame image as the new current frame image and repeat steps S41 to S43 until a real target is detected.

[0054] It should be noted that if a real target is detected, the target image feature information and coordinate information are saved. At the same time, the initial confidence value is set to N during the target tracking process, and the information is used as one of the inputs for the tracking stage.

[0055] S5: Based on the target's coordinate information, the least squares prediction technique is used to track the target, specifically:

[0056] S51: Set the initial tracking confidence value to N, and predict the target position of the next frame based on the latest M-frame target position information that has been tracked. Search for the target within a range centered on the prediction point and with a radius of r. If the search finds a target, the tracking is considered normal and the tracking confidence is incremented by 1. If the search does not find a target, the tracking is considered abnormal and the tracking confidence is decremented by 1.

[0057] S52: Determine if the current tracking confidence is greater than the initial value N. If it is greater than N, reset the tracking confidence to N, consider the tracking normal, continue target tracking in the next frame, and output the tracking chain and target slice information. If the current tracking confidence is less than N but greater than 0, keep the tracking confidence unchanged, consider the tracking normal, continue target tracking in the next frame, and output the tracking chain and target slice information. If the current tracking confidence is 0 or less than 0, consider the tracking abnormal and the target lost. The possible location can be estimated based on the target's previous movement direction and speed, that is, re-execute steps S1 to S4 for target detection.

[0058] Of course, the present invention may have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications according to the present invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A method for detecting and tracking high-speed moving targets in the air based on continuous frames, characterized in that, Includes the following steps: S1: Perform multi-directional gradient target saliency enhancement on the frame difference map of two adjacent frames to obtain the enhanced image; S2: The local extremum method is used to extract target candidate regions on the enhanced image; S3: False alarm removal is performed on the extracted target candidate regions based on differences in motion direction; S4: Utilize the target's motion characteristics to determine the target's coordinate information in the remaining target candidate region; S5: Based on the target's coordinate information, the least squares prediction technique is used to track the target; The method for obtaining the enhanced image is as follows: Frame difference is calculated between two adjacent frames to obtain a frame difference map that highlights the differences in target motion; Obtain the gradient detection function for each pixel on the frame difference map. as follows: in, Represents the pixel in the m-th row and n-th column. The gradient in the positive horizontal x+ direction Represents the pixel in the m-th row and n-th column. The gradient in the negative horizontal x-axis, Represents the pixel in the m-th row and n-th column. The gradient in the positive vertical direction y+ Represents the pixel in the m-th row and n-th column. The gradient in the negative vertical y-direction; Gradient detection function The pixels are denoted as background, and the gradient detection function is used. The pixels are denoted as the target, thus obtaining the enhanced image. Only when... , , , When all values ​​are greater than the set value, the gradient detection function .

2. The method for detecting and tracking high-speed moving targets in the air based on continuous frames as described in claim 1, characterized in that, The method of extracting target candidate regions from enhanced images using local extrema is as follows: The image is enhanced by traversing a sliding window of size S*S with a step size of S / 2. For each region extracted by each sliding window, the maximum value of the region is obtained, and then the seed points are increased according to the maximum value until the increase value is greater than the set value. After the increase stops, the pixels accumulated during the seed point increase process are used as the target candidate regions in the current sliding window.

3. The method for detecting and tracking high-speed moving targets in the air based on continuous frames as described in claim 1, characterized in that, The specific steps for determining the target's coordinate information within the remaining candidate target region using the target's motion characteristics are as follows: Perform the following operations on each candidate target as the current candidate target: S41: In the current frame image, with the current candidate target as the center and the maximum inter-frame motion distance of the target as the radius, search for other candidate targets that fall within the radius, and connect them in pairs to form target pairs. Determine the identity of the target features of the matching target pairs and remove target pairs that do not meet the identity condition. The maximum inter-frame motion distance of the target is the maximum offset distance of the target between two adjacent frames calculated by the target motion speed, image resolution and imaging frame rate. S42: Based on the extracted target pairs, the least squares method is used to predict the possible position of the current candidate target in the next frame image. Then, the possible position is used as the predicted position on the frame difference map between the next frame image and the next frame image after that, and the target is searched around the predicted position. S43: Using target characteristics and motion characteristics, the targets extracted in step S42 and the targets that meet the same identity conditions in step S41 are reconfirmed for identity, and false alarms are eliminated again. S44: Based on the target trajectory coordinates obtained from 3 consecutive frames, calculate the target heading and speed, and perform target identification. If a real target is detected, determine the coordinate information of the real target. If no target is detected, take the next frame image as the new current frame image and repeat steps S41~S43 until a real target is detected.

4. A method for detecting and tracking high-speed moving targets in the air based on continuous frames as described in any one of claims 1 to 3, characterized in that, Based on the target's coordinate information, the least squares prediction technique is used to track the target, specifically as follows: S51: Set the initial tracking confidence value to N, and predict the target position of the next frame based on the latest M-frame target position information that has been tracked. Search for the target within a range centered on the prediction point and with a radius of r. If the search finds a target, the tracking is considered normal and the tracking confidence is incremented by 1. If the search does not find a target, the tracking is considered abnormal and the tracking confidence is decremented by 1. S52: Determine whether the current tracking confidence is greater than the initial value N. If it is greater than N, reset the tracking confidence to N and consider the tracking to be normal. Continue the target tracking in the next frame and output the tracking chain and target slice information. If the current tracking confidence is less than N and greater than 0, the tracking confidence will remain unchanged, and the tracking will be considered normal. The target tracking will continue in the next frame, and the tracking chain and target slice information will be output. If the current tracking confidence is 0 or less than 0, the tracking is considered abnormal and the target is lost. Steps S1 to S4 are then re-executed to perform target detection.

Images