A non-cooperative method for airport surface movement target perception and tracking

Abstract

The present invention discloses a non - cooperative airport surface operation target perception and tracking method, which includes the following steps: S1, obtaining a video to be processed; S2, calculating the instantaneous gray - level Laplacian approximation of the video to be processed, and calculating the inter - frame instantaneous gray - level of the video to be processed according to the instantaneous gray - level Laplacian approximation, so as to obtain a sample set of moving targets; S3, using optical flow density clustering to perform pixel point aggregation on the sample set of moving targets to obtain an aggregation result; the aggregation result includes L clusters; each cluster represents a moving target; S4, using L tracking frames to perform one - to - one marking on the L moving targets; S5, using a tracking algorithm to sequentially perform real - time tracking on the L marked moving targets. The present invention can automatically identify and mark surface moving targets; perform real - time surface target visual tracking to avoid the curse of dimensionality; reduce the influence of target movement, pose change, occlusion, and illumination change on the tracking performance.

Description

Technical Field

[0001] The present invention relates to the technical field of airport monitoring, and in particular to a non-cooperative airport surface operation target perception and tracking method. Background Art

[0002] In the past decade, airport ground monitoring has gradually transformed from manual operation to informatization. The Airport Panorama Enhanced Surveillance System (APES) independently developed in China receives ADS-B collaborative information for aircraft positioning and has been widely used at airports. APES extremely relies on ADS-B to provide location information, but ADS-B does not equip encrypted machines to resist malicious attacks, and it is easy for hackers to intervene, eavesdrop, modify, inject, and delete tracking data. With the development of visual tracking technology, non-cooperative target tracking on the surface has been greatly developed, and the security and credibility of information have been improved. However, there are two intractable problems in non-cooperative target tracking on the surface:

[0003] 1) It is necessary to automatically mark multiple moving objects on the surface. Traditional automatic marking methods use background or time difference, and frame difference of consecutive grayscale images, which are only effective for fast-moving objects and ineffective for slow targets. In addition, traditional methods have deficiencies in anti-noise, automatic object marking, adaptive scaling, etc.;

[0004] 2) It is necessary to achieve real-time visual tracking. Some well-known visual tracking algorithms have been proposed in recently published and peer-reviewed works, such as generative algorithms or discriminative algorithms, but the high complexity of the scene and high-dimensional features are the key reasons for the failure of real-time visual tracking. Summary of the Invention

[0005] Aiming at the defects in the prior art, the present invention provides a non-cooperative airport surface operation target perception and tracking method to solve at least one of the above problems.

[0006] The present invention provides a non-cooperative airport surface operation target perception and tracking method, including the following steps:

[0007] S1, obtaining a video to be processed;

[0008] S2, calculating the instantaneous grayscale Laplacian approximation of the video to be processed, and calculating the inter-frame instantaneous grayscale of the video to be processed according to the instantaneous grayscale Laplacian approximation, so as to obtain a sample set of moving targets;

[0009] S3, using optical flow density clustering to perform pixel point aggregation on the sample set of the moving targets to obtain an aggregation result; the aggregation result includes L clusters; each cluster represents a moving target;

[0010] S4. Use L tracking boxes to mark L moving targets one by one;

[0011] S5. Use the tracking algorithm to perform real-time tracking on the marked L moving targets in sequence.

[0012] Preferably, the step S2 specifically includes:

[0013] Let cov2 represent the two-dimensional convolution kernel. For any time pixel value, the first-order differential of each micro variable in the n-th iteration is expressed as:

[0014]

[0015]

[0016]

[0017] Define:

[0018]

[0019] The instantaneous gray Laplacian approximation of the video to be processed is calculated as:

[0020]

[0021]

[0022] According to the first-order differential of each micro variable in the n-th iteration and the instantaneous gray Laplacian approximation, the instantaneous gray level can be obtained as:

[0023]

[0024] Among them, The instantaneous gray level between frames can be expressed as:

[0025]

[0026] According to the instantaneous gray level between frames, the sample set of the moving target can be obtained as: length represents the dimension operator;

[0027] Among them, Q(ξ, η, t) represents the coordinate (ξ, η) at any time pixel value; α and β are the instantaneous gray levels of the x-axis and y-axis respectively, represents the Laplacian operator, λ is the smoothing factor, radius represents the condensation radius, Γ is the minimum condensation pixel, and Alpha represents the abscissa scale factor.

[0028] Preferably, step S3 specifically includes:

[0029] Use the density-based clustering algorithm to complete the condensation of pixel points:

[0030]

[0031] where Density-Forming is the condensation operator, L is the number of clusters, and each cluster represents a moving target; represents the coordinates of each point in each cluster.

[0032] Preferably, step S4 specifically includes:

[0033] Define the tracking coordinates as: Frame l =[IndexYB, IndexXB, IndexXA, IndexYA];

[0034] where,

[0035] represents comparing all the vertical coordinates of the point traces in, and taking the maximum value;

[0036] represents comparing all the vertical coordinates of the point traces in, and taking the minimum value;

[0037] represents comparing all the horizontal coordinates of the point traces in, and taking the maximum value;

[0038] represents comparing and all the horizontal coordinates of the point traces, and taking the minimum value;

[0039] Record all the vertex coordinates of the tracking boxes: Rect={Frame1, Frame2…Frame L};

[0040] Generate the tracking box Initstate matrix: Initstate=[Rect(:,l)), Rect(:,2), Rect(:,4), Rect(:,l), Rect(:,3)-Rect(:,2)];

[0041] Thus, the automatic marking of each of the L moving targets is realized.

[0042] Preferably, step S5 specifically includes:

[0043] Perform a rough search on the L tracking boxes through a 0-1 classifier to obtain the rough positions of the L moving targets;

[0044] Further, a fine search is performed on the rough positions through a 0-1 classifier to obtain the final positions of L moving targets.

[0045] Preferably, the rough search for the L tracking frames through the 0-1 classifier to obtain the rough positions of L moving targets specifically includes:

[0046] Rough search:

[0047]

[0048] Among them, is the Initstate matrix of the tracking frame in the (t - 1)-th frame;

[0049] Using the 0-1 classifier:

[0050]

[0051] Let p(z i |y = 1) and p(z i |y = 0) respectively satisfy the Gaussian distribution Utilize the maximum response of the 0-1 classifier, and extract low-dimensional features using the sparse measurement matrix to correspondingly find the tracking sample label information of max(H(Z));

[0052] Among them, r c is the coarse step size, r f is the fine step size, μ is the positive sample search radius, v is the negative sample search radius, and it is set that y ∈ {0, 1} represents the positive and negative labels of the sample.

[0053] Preferably, the further fine search for the rough positions through the 0-1 classifier to obtain the final positions of L moving targets specifically includes:

[0054] Fine search:

[0055]

[0056] Further classify using the 0-1 classifier to obtain the positive and negative sample label information of the fine search tracking, and construct the following sets:

[0057] Positive sample set:

[0058] Negative sample set:

[0059] And satisfy (μ < ρ < v).

[0060] Preferably, the optical flow density clustering is the optical flow density clustering based on Horn-Schunck.

[0061] Preferably, the tracking algorithm is a multi-object fast compressive tracking algorithm.

[0062] Preferably, the tracking box is automatically updated at a preset time interval.

[0063] The beneficial effects of the present invention are as follows:

[0064] (1) It can accurately identify multiple moving targets and is more robust than traditional methods;

[0065] (2) It can automatically mark and frame the moving targets;

[0066] (3) The tracking box can be automatically updated on a certain time scale;

[0067] (4) It can track multiple scene targets simultaneously;

[0068] (5) It has excellent tracking performance for small targets and crossing targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0070] Figure 1 It is a schematic flowchart of an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0071] The following will describe in detail the embodiments of the technical solutions of the present invention with reference to the drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention, so they are only examples and cannot be used to limit the protection scope of the present invention.

[0072] It should be noted that unless otherwise specified, the technical terms or scientific terms used in this application should have the ordinary meaning understood by those skilled in the art to which the present invention belongs.

[0073] As Figure 1 shown, an embodiment of the present invention provides a non-cooperative airport surface operation target perception and tracking method, including the following steps:

[0074] S1. Obtain the video to be processed;

[0075] S2. Calculate the instantaneous grayscale Laplacian approximation of the video to be processed, and calculate the instantaneous grayscale between frames of the video to be processed based on the instantaneous grayscale Laplacian approximation, so as to obtain a sample set of moving targets;

[0076] S3. Use optical flow density clustering to perform pixel condensation on the sample set of moving objects to obtain a condensation result; the condensation result includes L clusters; each cluster represents a moving object.

[0077] S4. Use L tracking frames to mark the L moving objects one by one.

[0078] S5. Use a tracking algorithm to perform real-time tracking on the marked L moving objects in sequence.

[0079] Among them, step S2 specifically includes:

[0080] Generate optical flow according to the movement of the moving object. The optical flow equation is:

[0081]

[0082] Define:

[0083]

[0084] Calculate the instantaneous gray Laplacian approximation of the video to be processed as:

[0085]

[0086]

[0087] Also because:

[0088]

[0089]

[0090]

[0091] According to the first-order differentials of each micro variable and the instantaneous gray Laplacian approximation in the n-th iteration, the instantaneous gray level can be obtained as:

[0092]

[0093] Among them, The instantaneous gray level between frames can be expressed as:

[0094]

[0095] According to the instantaneous gray level between frames, the sample set of moving objects can be obtained as: length represents the dimension operator;

[0096] Among them, Q(ξ, η, t) represents the pixel value at the coordinate (ξ, η) at any time; α and β are the instantaneous gray levels of the x-axis and y-axis respectively. is expressed as the Laplacian operator, λ is the smoothing factor, radius represents the condensation radius, Γ is the minimum condensation pixel, and Alpha represents the abscissa scale factor.

[0097] Step S3 specifically includes:

[0098] Use the density-based clustering algorithm to complete pixel condensation:

[0099]

[0100] Density-Forming is the condensation operator, and common condensation algorithms such as DBSCAN and KNN. L is the number of clusters, representing the number of moving targets; represents the coordinates of each point in each cluster.

[0101] Step S4 specifically includes:

[0102] Automatically mark the moving target, using the following logical operations:

[0103] Input: Alpha

[0104] Execute the loop For l = 1, 2,... L do

[0105] represents comparison for all the vertical coordinates of the points in, take the maximum value;

[0106] represents comparison for all the vertical coordinates of the points in, take the minimum value;

[0107] represents comparison for all the horizontal coordinates of the points in, take the maximum value;

[0108] represents comparison for all the horizontal coordinates of the points in, take the minimum value;

[0109] Tracking box vertex coordinates: Frame l = [IndexYB, IndexXB, IndexXA, IndexYA];

[0110] end for

[0111] Record all the tracking box vertex coordinates: Rect = {Frame1, Frame2... Frame L};

[0112] Generate the tracking box Initstate matrix: Initstate = [Rect(:, 1), Rect(:, 2), Rect(:, 4), Rect(:, 1), Rect(:, 3) - Rect(:, 2)];

[0113] Thus, automatic labeling of each of the L moving targets is achieved.

[0114] In the embodiment of the present invention, the Horn-Schunck optical flow density formation has strong robustness. The detection result can be updated every once in a while and can be implemented simultaneously with the tracking algorithm. In addition, local adaptive scale adjustment can also be performed using the Horn-Schunck optical flow density formation.

[0115] Step S5 specifically includes:

[0116] Extend the Fast Compressive Sensing Tracking (FCT) algorithm to multi-object tracking, and propose the multi-object FCT (M-FCT) algorithm. It is divided into the following steps:

[0117] 1) Input: the t-th frame image, the tracking box Initstate matrix of the (t - 1)-th frame, where r c is the coarse step size, r f is the fine step size, μ is the positive sample search radius, v is the negative sample search radius, and y ∈ {0, 1} is set to represent the positive and negative labels of the sample.

[0118] Execute the loop: For l = 1, 2,... L

[0119] 2) Coarse search:

[0120]

[0121] where is the tracking box Initstate matrix of the (t - 1)-th frame

[0122] 3) Use a 0-1 classifier:

[0123]

[0124] Let p(z i |y = 1) and p(z i |y = 0) respectively satisfy the Gaussian distribution Use the 0-1 classifier to have the maximum response, and use the sparse measurement matrix to extract low-dimensional features, that is, find the tracking sample label information corresponding to max(H(Z)).

[0125] 4) Fine search:

[0126]

[0127] 5) Further use the 0-1 classifier in step (3) for classification, and the fine-search tracking positive and negative sample label information can be obtained, and the following sets are constructed:

[0128] Positive sample set:

[0129] Negative sample set:

[0130] And satisfy (μ < ρ < v),

[0131] 6) Scale parameter update ( The update is the same as the update rule in FCT)

[0132] ENDFOR

[0133] Output: Tracking frame of the t-frame image Matrix.

[0134] The embodiment of the present invention provides a non-cooperative airport surface operation target perception and tracking method. First, a global target estimation method for video dense optical flow field based on the Horn-Schunck theory is adopted, and multiple targets are automatically labeled through density formation; then the fast compressive target tracking algorithm is extended to the multi-target domain, and the multi-target fast compressive target tracking algorithm is proposed to perform real-time tracking on the labeled moving targets. The embodiment of the present invention can automatically identify and label multiple moving targets and perform real-time tracking on multiple moving targets. Compared with the time difference method, when the object posture changes violently and the scale changes rapidly, the embodiment of the present invention can reliably label the moving targets; the embodiment of the present invention can well simulate the appearance of the object through the dense optical flow field of the video and has robustness to illumination, scale, and posture changes; the embodiment of the present invention can reduce the influence of the background by differentiating and learning the object and the background, can handle occlusion and posture changes, and has good tracking performance.

[0135] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered by the scope of the claims and the description of the present invention.

Claims

1. A non-cooperative airport surface movement target perception and tracking method, characterized in that, It includes the following steps: S1. Obtain the video to be processed; S2. Calculate the instantaneous grayscale Laplacian approximation of the video to be processed, and calculate the inter-frame instantaneous grayscale of the video to be processed based on the instantaneous grayscale Laplacian approximation, so as to obtain a sample set of moving targets; S3. Use optical flow density clustering to perform pixel point aggregation on the sample set of moving targets to obtain an aggregation result; the aggregation result includes L clusters; each cluster represents a moving target; S4. Use L tracking frames to mark L moving targets one by one; S5. Use a tracking algorithm to perform real-time tracking on the marked L moving targets in sequence; The specific steps of step S2 include: Let cov2 represent a two-dimensional convolution kernel. For any time pixel value, the first-order differential of each micro variable in the n-th iteration is expressed as: Define: The calculated instantaneous grayscale Laplacian approximation of the video to be processed is: The instantaneous grayscale can be obtained according to the first-order differential of each micro variable in the n-th iteration and the instantaneous grayscale Laplacian approximation: The sample set of the moving target can be obtained according to the instantaneous gray level between frames as follows: length represents the dimensional operator; Among them, Q(ξ, η, t) represents the pixel value at coordinates (ξ, η) at any time; α and β are the instantaneous gray levels of the x-axis and y-axis respectively, is represented as the Laplacian operator, λ is the smoothing factor, radius represents the condensation radius, Γ is the minimum condensation pixel, and Alpha represents the abscissa scale factor.

2. The non-cooperative airport surface operation target perception and tracking method according to claim 1, wherein The specific steps of step S3 include: Use a density-based clustering algorithm to complete pixel point aggregation: Among them, Density-Forming is the condensation operator, L is the number of clusters, and each cluster represents a moving target; It represents the coordinates of each trace within each cluster.

3. The non-cooperative airport surface operation target perception and tracking method according to claim 2, characterized in that The specific steps of step S4 include: Define the tracking coordinates as: Framel = [IndexYB, IndexXB, IndexXA, IndexYA] Among them, represents comparison of all the ordinate values of the dots in and takes the maximum value; Indicate comparison Take the minimum value of the vertical coordinates of all the plot points in; Indicate comparison Take the maximum value of the abscissas of all the dots in Indicate comparison Take the minimum value of the abscissas of all the dots in Record the vertex coordinates of all tracking boxes: Rect = {Frame1, Frame2…Frame L} Generate a tracking frame Initstate matrix: Initstate = [Rect(:, 1)), Rect((:, 2), Rect(:, 4), Rect (:, 1), Rect(:, 3)-Rect(:, 2)] Thus, automatic marking of L moving targets one by one is realized.

4. The non-cooperative airport surface movement target perception and tracking method according to claim 3, characterized in that The specific steps of step S5 include: Perform a rough search on the L tracking frames through a 0-1 classifier to obtain the rough positions of L moving targets; Further perform a fine search on the rough positions through a 0-1 classifier to obtain the final positions of L moving targets.

5. The non-cooperative airport surface operation target perception and tracking method according to claim 4, characterized in that, The specific process of performing a rough search on the L tracking frames through a 0-1 classifier to obtain the rough positions of L moving targets includes: Rough search: Among them, is the Initstate matrix of the tracking box in the (t - 1)-th frame; Use a 0-1 classifier: Let p(z i |y = 1) and p(z i |y = 0) satisfy Gaussian distributions respectively Utilize the 0-1 classifier with the maximum response, and extract low-dimensional features using a sparse measurement matrix, that is, correspondingly find the tracking sample label information of max(H(Z)); where r c is the coarse step size, r f is the fine step size, μ is the search radius of positive samples, v is the search radius of negative samples, and it is set that y ∈ {0, 1} represents the positive and negative labels of the samples.

6. The non-cooperative airport surface operation target perception and tracking method according to claim 5, characterized in that The specific process of further performing a fine search on the rough positions through a 0-1 classifier to obtain the final positions of L moving targets includes: Fine search: Further use a 0-1 classifier for classification to obtain the positive and negative sample label information of the fine search tracking, and construct the following set: Positive sample set: Negative sample set: And satisfy (μ < ρ < v).

7. A non-cooperative airport surface movement target perception and tracking method according to claim 1, characterized in that, The optical flow density clustering is optical flow density clustering based on Horn-Schunck.

8. A non-cooperative airport surface movement target perception and tracking method according to claim 1, characterized in that The tracking algorithm is a multi-target fast compressive tracking algorithm.

9. A non-cooperative airport surface operation target perception and tracking method according to claim 1, characterized in that, The tracking frame is automatically updated on a preset time scale.

Images