Method for screening moving targets based on multi-scale track management

Abstract

The application discloses a kind of motion target screening methods based on multiscale track management, including the following steps: multiscale curvature feature point extraction: according to the curvature-based near-ground infrared small target detection algorithm, the extraction of point target curvature feature point is completed, and it is expanded from point target to spot target, and the curvature feature point extraction of multiscale target is completed;Multiscale track adaptive association: according to the historical point track information of a certain object, determine its motion state, predict the center position and the size of the wave door, and accurately find the object point track at this time in the wave door range, realize association;Multiscale track decision and merging: motion target track is screened out in numerous tracks, and multiscale tracks of the same target are merged.The application can consider the calculation speed and algorithm stability while stably screening different size targets in complex scenes.

Description

Technical Field

[0001] This invention belongs to the field of target detection, specifically a method for screening moving targets based on multi-scale track management. Background Technology

[0002] For the problem of multi-scale target detection, many scholars at home and abroad have proposed different detection methods based on the multi-scale characteristics of the target itself and other significant features.

[0003] (1) Traditional detection algorithms: Multi-scale targets can be described using features such as SIFT, SURF, and HOG, which have scale invariance. The DPM algorithm proposed by Felzenszwalb is an extension of HOG and has good robustness to multi-scale targets. Yu Li et al. proposed a detection method based on multi-scale deformable templates. Based on the existing Snake algorithm, shape constraints are added, and the multi-scale characteristics of wavelet transform are used to achieve multi-scale target matching. Lu Jun et al. proposed a time-based multi-scale moving target detection method. Based on the motion saliency criterion, a new method for testing motion saliency is proposed to detect regions with significant motion in the image, thereby achieving the detection of targets of different sizes.

[0004] (2) Deep Learning Methods: The PVANet algorithm proposed by KimKH et al. is an improved R-CNN network. This network fuses multi-layer features before making predictions, and has good performance in multi-scale object detection. Lin et al., considering that deep features have strong semantic information and shallow features have strong spatial information, proposed the FPN architecture, which combines deep feature maps through multiple upsampling and shallow feature maps one by one. The output is based on the feature map after multi-layer fusion, which can better detect targets of different scales and is a milestone in multi-scale object detection. Singh B et al. proposed the SNIPER algorithm, which can train the model at multiple scales to achieve multi-scale object detection. The YOLOv3 network model proposed by Redmon J et al. uses three branches to detect multi-scale objects. Zhu Peipei et al. proposed a real-time multi-scale object detection and recognition model based on an improved feature pyramid network structure. This model adds a feature pyramid to cover targets of a wider scale.

[0005] Despite significant progress made by traditional detection algorithms and deep learning-based methods in multi-scale object detection, some problems still exist:

[0006] Feature point extraction methods such as Surf, Harris, and Sift are primarily designed for detecting larger targets. However, they cannot reliably extract these feature points from small targets, and existing feature point extraction frameworks cannot solve the problem of detecting small targets at multiple scales. Furthermore, deep learning-based methods typically require a large amount of labeled data for training, especially for multi-scale target detection, where training data involving targets of different scales is even more crucial. Obtaining large-scale, multi-scale labeled data is a challenging task. Some application scenarios, such as video surveillance and autonomous driving, have high real-time requirements for multi-scale target detection, where traditional detection algorithms and some deep learning-based methods may suffer from high computational complexity. Summary of the Invention

[0007] The purpose of this invention is to provide a moving target screening method based on multi-scale track management, which can stably detect targets of different sizes in an image from three levels: multi-scale feature point extraction, multi-scale track adaptive stable association, and multi-scale track decision.

[0008] The technical solution to achieve the purpose of this invention is: a moving target screening method based on multi-scale track management, comprising the following steps:

[0009] (10) Multi-scale curvature feature point extraction: Curvature feature points are extracted from point targets and extended to spot targets to complete the extraction of curvature feature points of multi-scale targets and construct a multi-scale point trace set;

[0010] (20) Multi-scale adaptive association of tracks: Match and associate multi-scale point set with tracks of corresponding scale to realize track update and creation; and perform track deletion and track quality evaluation to output stable associated multi-scale tracks.

[0011] (30) Multi-scale track decision and merging: Select moving target tracks and merge multi-scale tracks of the same target.

[0012] Preferably, the specific steps for extracting multi-scale curvature feature points in step (10) are as follows:

[0013] (11) Use a two-dimensional Gaussian filter template to downsample the original target image to generate a multi-scale image set;

[0014] (12) Use curvature filtering algorithm to initially extract the curvature feature points of small targets, and complete the extraction of curvature feature points at multiple scales.

[0015] (13) Use the SOT values ​​at each position of the whole image to sparsify the region with dense curvature feature points.

[0016] Preferably, the specific steps of step (20) multi-scale track adaptive association include:

[0017] (21) Multi-scale adaptive observation filtering:

[0018] For the curvature feature points detected in a new frame of an image at any scale, traverse all the associated tracks at that scale, calculate the track observation velocity, and apply a first-order low-pass filter to the track observation velocity to obtain the filtered velocity.

[0019] (22) Multi-scale adaptive gate selection based on track quality:

[0020] The gate center position is predicted using the filtering velocity, and the gate size is selected based on the historical residuals.

[0021] (23) Matching of in-gate track and point track:

[0022] Determine if there are any traces within the gate. If so, calculate the cosine similarity and select the trace with the highest similarity that is greater than the similarity threshold for association.

[0023] (24) Update a series of track information for tracks that are associated with new points, create new tracks directly for unassociated points in the frame, and delete tracks that have not been associated with new points for a long time in the scale.

[0024] Preferably, the specific method for filtering out moving target tracks and merging multi-scale tracks of the same target is as follows:

[0025] (31) Traverse all tracks at all scales, select tracks with more than a certain threshold of associated points and also with a certain threshold of quality, and calculate the average speed of such tracks.

[0026] (32) Using the velocity mapping relationship, the velocity of the track in the downsampled image is multiplied by the average factor, and the average velocity of the track at different scales is mapped to the original scale;

[0027] (33) Based on the mean and standard deviation of the average speed of the multi-scale track after velocity mapping, estimate the range of the average speed of the background track, and preliminarily identify the track whose speed is outside the estimated range as a suspected target.

[0028] (34) Calculate the actual displacement of the tracks that are initially identified as suspected targets, and determine the tracks with actual displacement greater than a certain threshold as moving target tracks.

[0029] Compared with the prior art, the significant advantages of this invention are:

[0030] 1. Feature points of multi-scale targets can be extracted: This invention can handle targets of different sizes at multiple scales. Compared with traditional weak target detection algorithms that can only target one target scale, this invention can handle targets of multiple different scales at the same time.

[0031] 2. For multi-scale adaptive trajectory association: This invention can achieve independent and stable association of multi-scale points and tracks based on the adaptive trajectory association algorithm.

[0032] 3. Stable decision-making for moving targets: The concepts of velocity mapping, unified clustering, and true displacement are proposed. Multi-scale tracks need to be mapped first and then clustered. Tracks with large velocity differences are first identified, and then the true displacement of these tracks is calculated. Only tracks with large true displacements are identified as moving target tracks. Attached Figure Description

[0033] Figure 1 This is a flowchart of a moving target selection method based on multi-scale track management.

[0034] Figure 2 The target drawing is 20*15 pixels.

[0035] Figure 3 This is a diagram showing the results of target detection.

[0036] Figure 4 To simulate approximately uniform motion.

[0037] Figure 5 To simulate approximately uniformly accelerated motion.

[0038] Figure 6 The results show the trajectory association in the downscaled image.

[0039] Figure 7 This is a diagram illustrating the effect of trajectory decision-making. Detailed Implementation

[0040] like Figure 1 As shown, a moving target screening method based on multi-scale track management includes the following steps:

[0041] (10) Multi-scale curvature feature point extraction: Curvature feature points are extracted from point targets and extended to spot targets to complete the extraction of curvature feature points of multi-scale targets;

[0042] (11) Downsampling of the original image:

[0043] Figure 2 For a 20*15 sized target original image against an architectural background, a two-dimensional Gaussian filter template is used to downsample the original image, generating a multi-scale image set;

[0044] Difference-of-Gaussian (DoG) image pyramids are commonly used for multi-scale image decomposition. This invention utilizes a two-dimensional Gaussian filtering function to calculate the convolution template for spatial filtering, performing convolution and downsampling on the image to obtain a multi-scale, multi-resolution image set that preserves features. The following is the functional expression for two-dimensional Gaussian filtering:

[0045]

[0046] Among them, G σ (x,y) is the filter response at point (x,y); σ is the standard deviation, which determines the “width” of the Gaussian distribution; x and y are the horizontal and vertical distances from the center of the filter.

[0047] The three-dimensional response of small infrared targets exhibits a Gaussian-like distribution, with the curvature values ​​in all directions of the target region significantly greater than those of the background. To address this characteristic, a curvature filtering algorithm is used to initially extract curvature feature points from the small target, ultimately completing the extraction of curvature feature points at multiple scales.

[0048] Figure 3 (a) is from Figure 2 The downscaled image after one pyramid downsampling step, the high-resolution original image I 0 The image pyramid downsampling operation is denoted as DownSampling{·}, and the image obtained after one downsampling is denoted as I. 1 Therefore, image downsampling can be expressed by the formula:

[0049] I i+1 =DownSampling{I i}, (i = 0, 1, 2...)

[0050] (12) Curvature-based multi-scale target feature point extraction:

[0051] For any original image or downsampled image I i Its curvature diagram in the β direction It can be expressed as follows:

[0052]

[0053] in,

[0054] The downsampled image I in the β direction is obtained. i curvature diagram Then, the final curvature map of the image can be obtained.

[0055]

[0056] The fused curvature map of the downsampled image is obtained using the following formula.

[0057]

[0058] In the formula Indicates the downsampled image Ii The result diagram of the top cap.

[0059] Similarly, based on the fusion curvature diagram To calculate the adaptive binarization threshold Thre i :

[0060] Thre i =μ i +k i σ i2 (i = 0, 1, 2...)

[0061] Where, μ i To merge curvature maps The mean, k i σ is a coefficient. i2 To merge curvature maps The variance.

[0062] After binarization, the curvature map will be merged. The value is greater than the binarization threshold Thre i Setting the value to 1 in the image will result in values ​​less than the binarization threshold Thre. i Setting the value to 0 in the image yields a binary image of the fused curvature. This process can be represented as:

[0063]

[0064] Finally, after obtaining the original image or the downsampled image I i Binary plot of fused curvature Then, a sliding window approach is used to traverse all positions in this series of multi-resolution binary images, excluding edge rows and columns. First, it is determined whether a given position is within the binary image of the fused curvature. If the value is 1, then check if the curvature value at that position is the maximum curvature value within the sliding window. If it is, retain the curvature value at that position and record its pixel coordinates as the curvature feature point output of the image. Otherwise, discard it, and finally output the curvature feature point map.

[0065] like Figure 3 (b) is the three-dimensional response map of the target area in the downsampled image. Although it is not peak-shaped due to the influence of the building background, it is still a protruding curved surface, indicating that its grayscale curves at 0°, 45°, 90° and 135° conform to one-dimensional Gaussian curves. The overall three-dimensional response map of the target conforms to the characteristics of a Gaussian-like distribution. Figure 3 (c) is the downsampled image Figure 3 (a) Feature point map extracted by curvature filtering, compared with... Figure 3 (a) and Figure 3(c) In the red box area, it can be seen that the target feature points can be effectively extracted after the size is reduced.

[0066] (13) Sparsification of curvature feature points based on target index:

[0067] Using the SOT values ​​at various locations across the entire image, sparsification is applied to regions with dense curvature feature points. Specifically, within a fixed window size, only the curvature feature points with the highest SOT values ​​are retained. The formula for calculating the SOT value is as follows:

[0068]

[0069] In the formula, I g (x,y) represents the gradient intensity; I gavg (x,y) represents the average gradient intensity; I k (x,y) represents the curvature value; I kavg (x,y) represents the average curvature value; h(g) (x,y) ) and h(k (x,y) ) is a correction or adjustment term for the gradient strength and curvature value; a and b are weight parameters that determine the relative importance of the gradient strength and curvature value in the calculation results.

[0070] (20) Multi-scale adaptive association of tracks: Match and associate multi-scale point set with tracks of corresponding scale to realize track update and creation; then perform track deletion and track quality evaluation to output stable associated multi-scale tracks;

[0071] (21) Multi-scale adaptive observation filtering:

[0072] This invention employs a strategy of independent correlation between points at different scales, eliminating cross-scale point correlation and thus forming multi-scale tracks. For different tracks within the same scale, an independent prediction gate method is also used to achieve independent adaptive correlation between tracks.

[0073] (211) Predict the current position of the point:

[0074] Px k =x k-1 +Vx predicted

[0075] Py k =y k-1 +Vy predicted

[0076] In the formula, Px k Py k To predict the position of the point at time k, x k-1 y k-1Vx represents the target's position information on the image at the previous moment. predicted Vy predicted This represents the current predicted speed.

[0077] (212) Calculate the velocity at the current moment:

[0078] Vx k =x k -x k-1

[0079] Vy k =y k -y k-1

[0080] In the formula, Vx k Vy k The velocity at the current moment can be expressed as:

[0081] Px k =x k-1 +Vx k-1

[0082] Py k =y k-1 +Vy k-1

[0083] (213) Perform a first-order low-pass filter on the velocity:

[0084]

[0085]

[0086] and Let represent the filtered velocity of the j-th track at time t on a scale of s. and Let represent the filtered velocity of the j-th track at time t-1 on scale s. and Let represent the observed velocity of the j-th track at time t on scale s, and 'a' represent the filtering coefficient, ranging from 0 to 1. A smaller value for 'a' results in a smaller weight for new observations, a smoother filtering result, and a less sensitive algorithm. The filtered velocity at the current time is calculated as the predicted angular velocity at the current time.

[0087] (214) Define the deviation between the predicted position and the actual position as the residual, and calculate the residual:

[0088] If the observed velocity is used as the predicted velocity, the residual of the j-th track at time t on the s-scale is called the observation residual. It can be represented as:

[0089]

[0090] If the filtered velocity is used as the predicted velocity, the residual of the j-th track at time t on the s-scale is called the filtered residual. It can be represented as:

[0091]

[0092] Observation residuals and filter residual The magnitude of these values ​​represents the deviation between the observation speed and the filtering speed and the actual speed. The filtering coefficient 'a' represents the proportion of the observation speed and the filtering speed, and the mean absolute value of the residuals over a period of time is used to replace the residuals over a short period of time.

[0093] (215) Observation residuals and filter residual With filter coefficients The relationship can be represented as:

[0094]

[0095] It is calculated from the residual of the previous time step, i.e., time step t-1, a (s,j,t) It is calculated from the cumulative residuals before time t.

[0096] In the formula, This represents the mean absolute value of the residuals of a certain trajectory before time t-1. Less than When the position deviation predicted by the filtered velocity is less than that predicted by the observed velocity, the proportion of the observed velocity should be reduced and the proportion of the filtered velocity should be increased when predicting the position of the track at the next moment.

[0097] Figure 4 When simulating a target moving at approximately a constant speed, and a (s,j,t) The changes.

[0098] (216) For cases involving variable-speed target motion, when it is determined that the target's acceleration is not approximately zero, acceleration information needs to be incorporated into the formula for calculating the filtered velocity. For obvious variable-speed motion, the target's observed velocity will change significantly between two consecutive observations, at which point the target's acceleration information can be calculated.

[0099]

[0100] (217) Calculate the filter coefficient a over a period of time. (s,j,t) and acceleration information The value of the filter coefficient a over a period of time (s,j,t) All are much greater than 0.5, and the acceleration information If the values ​​are not close to 0, it can be determined that the target is undergoing variable speed motion, thus forcibly correcting the filter coefficients. At the same time, acceleration information is also introduced in the process of calculating the filter speed.

[0101] Therefore, the original formula for calculating the filter coefficients can be modified as follows:

[0102]

[0103] Therefore, the original formula for calculating the filtering speed can be modified as follows:

[0104]

[0105] Adding the above correction strategy to the uniform acceleration simulation, we obtain the following results. Figure 5 When the target trajectory is determined to be undergoing variable speed motion, forcibly increasing the filter coefficient and adding acceleration information results in a rapid decrease in the filter residual. Furthermore, the residual of the filtered velocity is significantly smaller than the residual of the observed velocity, with the residual value being almost zero. The improved first-order filter can better adapt to situations where the target is undergoing variable speed motion.

[0106] (22) Multi-scale adaptive gate selection based on track quality:

[0107] The selection of the adaptive gate size ensures that each track contains the track's points at the next time step within the prediction gate, while not containing too many other track points at the next time step.

[0108] Therefore, this invention introduces a track quality model to adaptively adjust the track gate size. The principle for adjusting the gate size is as follows: for high-quality tracks, i.e., tracks with stable and regular motion states, a small gate is used for correlation to reduce computational load; for low-quality tracks, i.e., tracks with changing motion states or not continuously correlated, a large gate is used for correlation to ensure that the points fall within the gate. and Let represent the mass of the j-th track at time t in the x and y directions, with a value ranging from [0 to 1].

[0109] use and Let represent the gate size of the j-th track at scale s in the x and y directions at time t'. and It can be calculated by the following formula and continuously updated adaptively. Since the speed of the detected moving target within the field of view has a range, the gate size of the track also has upper and lower limits. Setting a lower limit for the gate ensures that the gate will not keep getting smaller and tending towards 0, which would lead to failure to associate; setting an upper limit for the gate prevents the gate from getting larger and increasing the probability of association errors.

[0110]

[0111]

[0112] Taking the gate size in the x-direction as an example, the gate size in the x-direction of the j-th track at scale s at time t' is related to the gate size in the x-direction of the track at time t and the track quality at times t-1 and t'-1.

[0113] This formula explains the situation where the track quality at time t'-1 is 0, indicating that the track motion is highly unstable, and the maximum gate size at that scale is forcibly used for correlation. If the track quality at time t'-1 is not 0 and the track quality at t'-1 is higher than that at t-1, it indicates that the track quality is increasing, and the track motion is becoming more stable. This will be a number less than 1, which is then multiplied by the gate size in the x-direction at time t to achieve gate reduction; conversely, when the track quality at t'-1 is lower than the track quality at t-1, i.e., the track quality decreases and the track motion state becomes unstable, due to... It will be a number greater than 1, which is then multiplied by the gate size in the x direction at time t to achieve gate amplification.

[0114] use and Let represent the upper and lower limits of the motion velocity of the detected object in the x and y directions within a 20ms time interval, respectively. Then, the upper and lower limits of the gate for tracks of different scales can be expressed as:

[0115]

[0116] α s The multiplier coefficient representing the s-th scale is used because the upper and lower speed limits of the detected object are relative to the original scale (s=0). For other downscaled tracks, the movement speed of tracks at adjacent scales is approximately twice that of each other. For targets with different refresh rates, automatic adjustment is performed based on a refresh period of 20ms, thus adapting the target to different refresh rates and movement speeds. In this embodiment, α is set to 2.5.

[0117] (23) Matching of in-gate track and point track:

[0118] Determine if there are any traces within the gate. If so, calculate the cosine similarity and select the trace with the highest similarity that is greater than the similarity threshold for association.

[0119] A dot can be represented by the pixel coordinates in the x-direction, pixel coordinates in the y-direction, azimuth angle (az), pitch angle (el), gray value (gray), curvature value (cur), and sot value.

[0120] use Let represent the description vector of the j-th track at time t-1 on scale s, which can be expressed as:

[0121]

[0122] This indicates the predicted gate center position at time t based on the flight path information. This represents the curvature value of the track associated with the point at time t-1.

[0123] use Let be the description vector representing the i-th point trace detected in the image at time t on scale s, which can be expressed as:

[0124]

[0125] Cur represents the angular position of the i-th point detected in the image at time t on scale s. i (s,t) This represents the curvature value of the point.

[0126] This embodiment evaluates the similarity between the two description vectors by calculating the cosine of the angle between them, that is, by calculating the cosine similarity between the two vectors. The calculation formula is as follows:

[0127]

[0128] · represents the dot product operation, and ||·|| represents the modulus operation of a vector. This represents the similarity between the j-th track at scale s at time t and the i-th point detected in the image at time t at scale s. Since a track can be associated with at most one point at any given time, associating it with the point with the highest similarity is the optimal choice. Furthermore, for all points to be matched, the similarity must be greater than a certain threshold; only points with similarity greater than the threshold can be considered true target points. In summary, matching tracks and points within a gate requires two conditions: first, the similarity must be greater than a similarity threshold; second, the point with the highest similarity must be selected. This can be expressed mathematically as:

[0129]

[0130] In the formula, csim_thre represents the similarity threshold, and the value selected in this embodiment is 0.95.

[0131] This means that similarity calculation is performed only on the dots within the gate range. This represents the maximum similarity between the j-th track at scale s at time t and the in-gate point traces detected in the image at scale s at time t. If the value is not 0, it means that the trajectory has achieved a match with the current point within the gate.

[0132] (24) Update a series of track information for tracks that are associated with new points, create new tracks directly for the batch of unassociated points, and delete tracks that have not been associated with new points for a long time at this scale.

[0133] (30) Multi-scale track decision and merging: Select moving target tracks from numerous tracks and merge multi-scale tracks of the same target.

[0134] (31) Traverse all tracks at all scales, select tracks with more than a certain threshold of associated points and also with a certain threshold of quality, and calculate the average speed of such tracks.

[0135] Let t1 represent the current time. Taking the azimuth angle as an example, the average velocity of the j-th track on the s-scale during the time interval t0 to t1 can be expressed as:

[0136]

[0137] Let represent the azimuth angle of the j-th track at time t on scale s. To ensure more accurate calculation of the average track velocity, this invention stipulates that the track must have more than 10 associated points between t0 and t1. The total number of tracks satisfying this condition at time t1 is denoted as .

[0138] (32) Using the velocity mapping relationship, the velocity of the track in the downsampled image is multiplied by the average factor, and the average velocity of the track at different scales is mapped to the original scale;

[0139] (33) Estimate the range of average speed of background tracks based on the mean and standard deviation of the average speed of multi-scale tracks after speed mapping, and preliminarily identify tracks whose speed is outside the estimated range as suspected targets.

[0140] A unified velocity clustering method using tracks at different scales is employed for track decision-making, and the mean of the average velocity of the multi-scale tracks is calculated. with standard deviation The formula is:

[0141]

[0142]

[0143] In the formula, This represents the number of all tracks at time t1 on scale s; This represents the total number of tracks that satisfy the condition that the number of associated points is greater than 10 during the time period t0 to t1; M s The multiple by which velocity is mapped to tracks at different scales.

[0144] After obtaining the average velocity of all eligible tracks within the same time period, the range of the average velocity of background tracks is estimated based on the average velocity of a large number of tracks. This invention uses a fixed multiple of the standard deviation of the mean of all eligible tracks' average velocities as the velocity range of background track points. Tracks with average velocities within this range can be identified as background tracks; those outside the range may be moving target tracks or background tracks. Based on this formula, tracks from any set of images at different resolutions can be uniformly clustered to identify background tracks and suspected target tracks.

[0145] Figure 6 It is a drone whose moving target is in the air. Figure 6 (a) is the original image I 0 The resolution is 640*512. Figure 6 (b) is from the original image I 0 Figure I obtained by downsampling 1 The resolution is 320*256. Figure 6 (c) is from Figure I 1 Figure I obtained by downsampling 2 The resolution is 160*128. Because the large-scale drone targets in the original image were downsampled twice, in... Figure 6 (b) already shows a point target that can characterize the spot target, so there is no need for further downsampling. After a period of time, tracks were formed in three images at different resolutions.

[0146] (34) Calculate the actual displacement of the tracks that are initially identified as suspected targets, and determine the tracks with actual displacement greater than a certain threshold as moving target tracks.

[0147] The displacement of the background track over a period of time is closely related to the camera's motion. For the background track, the difference between the displacement over a period of time and the camera's displacement over that period should approach 0.

[0148] The camera's motion at time t causes the background track to have a velocity at time t. Conversely, statistically analyzing the velocity of the background track at time t can characterize the camera's motion at time t. Clearly, to obtain a more accurate picture of the camera's motion, all tracks must be filtered, excluding those with significantly different motion states, and only selecting tracks with similar motion states as samples. This approach minimizes the chance of moving target tracks or abnormally related tracks being excluded from the statistics.

[0149] The main purpose here is to obtain the camera motion state at a specific moment, so instantaneous velocity is used for clustering. Let represent the velocity of the j-th track at time t on a scale of s. Then, the mean velocity of all tracks at time t on a scale of s is:

[0150]

[0151] Similarly, using a fixed multiple of the mean and standard deviation of the velocities of all tracks at time t as the selection range, the standard deviation of the velocities of all tracks at time t on the s-scale is:

[0152]

[0153] After obtaining the mean and standard deviation of the track velocity at time t, the estimated range of the background track velocity at time t can be expressed as:

[0154]

[0155] For tracks whose velocities at time t fall within this range, the average velocity at time t is recalculated to characterize the general impact of camera motion on the background track. This average velocity is denoted as...

[0156] Therefore, the actual displacement value of the j-th track at scale s during the time interval t0 to t2 can be expressed as:

[0157]

[0158] A track can be identified as a moving target track when its average speed is outside the estimated range and its actual displacement value is greater than the displacement threshold thre_dis over a period of time.

[0159] like Figure 7 The image shows the motion detection results, which are achieved by first clustering the average velocity of the flight path and then calculating the actual displacement for decision-making. The detection results demonstrate that even when the background changes from sky to a building background, the moving drone can still be reliably detected.

[0160] This invention replaces traditional feature point detection algorithms such as SURF, Harris, and Sift with a curvature feature point detection algorithm. Because curvature points can detect small targets, curvature point extraction is an enhanced version of the "suspected point extraction" step in the traditional moving small target detection process. Based on the self-similarity of the image, a two-dimensional Gaussian filter is used to downsample the image, converting speckled targets into point targets, making them suitable for the aforementioned curvature filtering algorithm. This achieves multi-scale curvature feature point extraction, applying the detection method to multi-scale targets.

Claims

1. A moving target screening method based on multi-scale track management, characterized in that, Includes the following steps: (10) Multi-scale curvature feature point extraction: Curvature feature points are extracted from point targets, and then extended to spot targets to complete the extraction of curvature feature points of multi-scale targets, and a multi-scale point trace set is constructed. The specific steps are as follows: (11) Use a two-dimensional Gaussian filter template to downsample the original target image to generate a multi-scale image set; (12) Use curvature filtering algorithm to perform preliminary extraction of curvature feature points of small targets, and complete the extraction of curvature feature points at multiple scales; (13) Using the SOT values ​​at various locations in the entire image, sparsification is performed on regions with dense curvature feature points. The specific method is as follows: Within a fixed window size, only the curvature feature point with the largest SOT value is retained. The formula for calculating the SOT value is: In the formula, For gradient strength; The average gradient intensity; This is the curvature value; This is the average curvature value; and It is a correction or adjustment term for the gradient strength and curvature value; a and b are weight parameters; (20) Multi-scale adaptive correlation of tracks: Match and correlate multi-scale point set with tracks of corresponding scale to realize track update and creation; and perform track deletion and track quality evaluation to output stable correlated multi-scale tracks; (30) Multi-scale track decision and merging: Select moving target tracks and merge multi-scale tracks of the same target.

2. The moving target screening method based on multi-scale track management according to claim 1, characterized in that, The curvature filtering algorithm is used to initially extract the curvature feature points of small targets. The specific method for extracting multi-scale curvature feature points is as follows: Any downsampled image in a multi-scale image set exist Curvature diagram in the direction is ; Determine the final curvature plot of the image. : Determine the fused curvature map of the downsampled image : In the formula, Indicates the downsampled image The top cap result diagram; According to the fusion curvature diagram Calculate the adaptive binarization threshold : in, To merge curvature maps The mean, For coefficients, To merge curvature maps The variance; Merge curvature map Medium greater than the binarization threshold Setting the value to 1 in the image will result in values ​​less than the binarization threshold. Setting the value to 0 in the image yields a binary image of the fused curvature. ; A sliding window approach is used to traverse all positions in the multi-resolution binary image except for edge rows and columns; it is then determined whether any position falls within the range of the fused curvature binary image. Is the value 1? If it is 1, continue to check if the curvature value at that position is the maximum curvature value within the sliding window. If it is the maximum curvature value, retain the curvature value at that position and record its pixel coordinates as the curvature feature point output of the image. If it is not the maximum curvature value, discard it, and finally output the curvature feature point map. If it is not 1, continue iterating through other positions.

3. The moving target screening method based on multi-scale track management according to claim 1, characterized in that, Step (20) Multi-scale track adaptive association includes the following specific steps: (21) Multi-scale adaptive observation filtering: For the curvature feature points detected in a new frame of an image at any scale, traverse all the associated tracks at that scale, calculate the track observation velocity, and apply a first-order low-pass filter to the track observation velocity to obtain the filtered velocity. (22) Multi-scale adaptive gate selection based on track quality: The gate center position is predicted using the filtering velocity, and the gate size is selected based on the historical residuals. (23) Matching of in-gate track and point track: Determine if there are any traces within the gate. If so, calculate the cosine similarity and select the trace with the highest similarity that is greater than the similarity threshold for association. (24) Update a series of track information for tracks that are associated with new points, create new tracks directly for unassociated points in the frame, and delete tracks that have not been associated with new points for a long time in the scale.

4. The moving target screening method based on multi-scale track management according to claim 3, characterized in that, The filter coefficients of a first-order low-pass filter are: In the case of non-approximate uniform velocity: express The first scale The flight path is The filter coefficients at time points; Under approximately uniform velocity conditions: express The first scale The flight path is The filtered residual at time step, express The first scale The flight path is The observation residual at any given moment; The filtering speed is: express The first scale The flight path is Filtering speed at time t, express The first scale The flight path is Filtering speed at time t, express The first scale The flight path is The speed of observation at any given moment For the goal The first scale The flight path is Acceleration information at any given moment.

5. The moving target screening method based on multi-scale track management according to claim 3, characterized in that, The upper and lower limits of the gate for tracks at different scales are represented as follows: In the formula, Indicates the scale number. Indicates the first Scale factor, The target refresh cycle is defined as follows: for targets with different refresh rates, Based on refresh cycle Based on the existing conditions, automatic adjustments are made to adapt to different target refresh rates and different movement speeds. and These respectively represent the detection objects in The upper and lower limits of the velocity in the x and y directions within a given time period.

6. The moving target screening method based on multi-scale track management according to claim 3, characterized in that, The formula for calculating the maximum cosine similarity is: In the formula, Indicates the similarity threshold. This means that similarity is calculated only for points within the gate area. It means The first scale The flight path is Time and Scale Maximum similarity of gate-intra-point traces detected in the time-lapse image: In the formula, Indicates in Scale The first time detected in the time image A description vector of a point trace, express The first scale The flight path is The time-description vector.

7. The moving target screening method based on multi-scale track management according to claim 1, characterized in that, The specific method for filtering out moving target tracks and merging multi-scale tracks of the same target is as follows: (31) Traverse all tracks at all scales, select tracks with more than a certain threshold of associated points and also with a certain threshold of quality, and calculate the average speed of such tracks. (32) Using the velocity mapping relationship, the velocity of the track in the downsampled image is multiplied by the average factor, and the average velocity of the track at different scales is mapped to the original scale; (33) Based on the mean and standard deviation of the average speed of the multi-scale track after velocity mapping, estimate the range of the average speed of the background track, and preliminarily identify the track whose speed is outside the estimated range as a suspected target. (34) Calculate the actual displacement of the track that is initially identified as a suspected target, and determine the track with the actual displacement greater than a certain threshold as the track of the moving target.

8. The moving target screening method based on multi-scale track management according to claim 7, characterized in that, Preliminary identification of flight path as a suspected target The first scale The flight path is The formula for calculating the actual displacement over a given time period is: In the formula, express The first scale The flight path is The position and angle at any moment. This represents the average speed.

Images