Correlation filtering method for enhancing tracking performance of unmanned aerial vehicle

Abstract

The invention relates to a correlation filtering method for enhancing the tracking performance of an unmanned aerial vehicle, and the method comprises the steps: S1, carrying out the collection of an original image through an RGB camera, and training a tracking model of a discriminant unmanned aerial vehicle; s2, performing feature extraction on the new image by fusing CN and fHOG features to obtain new sample information; s3, performing dimensionality reduction processing on the obtained features by using a principal component analysis (PCA) mode to obtain sample information with smaller dimensionality and lower complexity so as to improve the tracking speed; s4, performing similarity comparison on the obtained sample information and the trained model information by using an elastic regularization network, judging whether learning is performed or not, discarding image information with relatively low similarity, and determining that the learned image information is trained to become a new tracking model; s5, optimizing the trained tracker model by using an alternating direction multiplier method (ADMM); and S6, target position information is determined, and unmanned aerial vehicle tracking is realized through continuous iteration. The tracking rate and the accuracy rate in the method are both at a high level.

Description

Technical Field

[0001] The invention belongs to the technical field of pattern recognition and data processing, and in particular relates to a correlation filtering method for enhancing the tracking performance of an unmanned aerial vehicle. Background Art

[0002] With the introduction of the country's "low-altitude economy" policy, unmanned aerial vehicles have become a research hotspot that major manufacturers have invested time, energy and resources in. Unmanned aerial vehicles have the advantages of being small, fast, and not occupying ground channels. They have a good development prospect and can be mainly used for: (1) reducing manpower expenditure, identifying crops at fixed points and spraying the required fertilizers; (2) avoiding land traffic jams, and realizing high-altitude express delivery and take-out delivery to homes; (3) enhancing urban security, using small drones as a platform to carry out urban inspections and other activities, and once illegal activities or criminals are discovered, they will be tracked immediately. In summary, unmanned aerial vehicles are needed to track the target to be operated, and then use their flight advantages to achieve different purposes, thereby benefiting the people.

[0003] Chinese patent document CN118247305A discloses a correlation filtering tracking method that is resistant to occlusion and interference from similar targets. The tracking method includes the following steps: S1: obtaining a WPSR response value corresponding to a current frame image, and determining whether the WPSR response value is greater than a set threshold value t1. If yes, the target is not occluded on the current frame image, and a KCF algorithm is used to detect the position of the target on the current frame image. If no, the target is occluded on the current frame image; S2: using a Kalman filter to obtain a predicted position of the target on the current frame image, and obtaining an NCC response map corresponding to a sub-image formed by a set neighborhood centered on the predicted position; S3: obtaining a motion information weighted map based on motion information corresponding to an image that is closest to the current frame image and is not occluded, and multiplying the motion information weighted map by the NCC response map corresponding to the current frame image to obtain an improved NCC response map. Figure; S4: Determine whether the maximum response value in the improved NCC response map exceeds the set threshold value t2. If yes, proceed to step S5. If not, expand the set neighborhood and reselect the next frame image to execute steps S2 to S4; S5: Obtain the KCF template corresponding to the image closest to the current frame image that is not obstructed, and use the KCF template to perform KCF detection on the first M positions selected from the improved NCC response map that exceed the set threshold value t2 in turn, and obtain M KCF detection response values, where M is at most 3; S6: Determine whether there is at least one KCF detection response value greater than the set threshold value t3. If yes, it is considered that the position of the target on the current frame image is detected, and the position of the maximum value among the KCF detection response values ​​greater than the set threshold value t3 is used as the position of the target on the current frame image; if not, reselect the next frame image to execute steps S2 to S4.

[0004] Chinese patent document CN117789083A discloses a video tracking method based on deep network multi-layer feature fusion, the tracking method comprising: obtaining the t-th frame image of the target video; extracting candidate target region features for the t-th frame image; calculating the kernel matrix of the current region features and the target region features of the previous frame based on the candidate target region features; calculating an output response map based on the kernel matrix of the current region features and the target region features of the previous frame and a filter template; determining the target position based on the output response map; wherein the method for obtaining the filter template comprises: extracting target region features for continuous frame training sample images respectively to form a feature matrix; calculating the kernel matrix of the target region features themselves using a kernel function based on the feature matrix; and training a filter template by outputting responses through the kernel matrix and an ideal Gaussian kernel function.

[0005] Chinese patent document CN117348021A discloses a navigation method for a medium- and high-altitude UAV based on vision and laser ranging. The tracking method comprises: determining a key frame based on the laser ranging acquisition time in a monocular image collected by the UAV, and performing feature tracking on the current frame based on the key frame based on motion compensation; calculating the horizontal displacement and the horizontal displacement covariance between the current frame and the key frame based on the UAV POS data, the position of the laser ranging point in the key frame, and the feature tracking position of the laser ranging point in the current frame; calculating the heading offset and the heading offset covariance between the current frame and the key frame based on the UAV POS data, the feature block of the key frame, and the relative rotation angle of the feature block of the current frame after the corresponding feature tracking; using the obtained horizontal displacement and horizontal displacement covariance information, the heading offset and the heading offset covariance information as the observation information of the flight control Kalman filter framework, and fusing the inertial navigation and visual observation information through the flight control Kalman filter algorithm to obtain the navigation information of the UAV.

[0006] At present, in order to pursue high accuracy, deep learning is usually used, but it requires the use of expensive GPUs and cannot achieve the minimum speed of unmanned aerial vehicles (30 frames / s) in natural scenes. Among traditional methods, the discriminant correlation filter algorithm (DCF) has achieved higher accuracy than the kernel correlation filter (KCF), but it usually uses a predefined regularization term to learn image features frame by frame and then update the appearance model to achieve the purpose of improving tracking accuracy. When the unmanned aerial vehicle faces an unknown situation, it will fail to track because it cannot match the model and lose the tracking target.

[0007] Therefore, for the existing unmanned aerial vehicle tracking methods, it is necessary to rely on expensive GPUs to achieve high precision, or to obtain high-speed kernel correlation filtering aircraft at the expense of accuracy. It is necessary to provide a correlation filtering method that enhances the unmanned aerial vehicle tracking performance to solve the shortcomings of traditional methods that cannot ensure both accuracy and speed at the same time. Summary of the invention

[0008] In view of the problems existing in the prior art, the present invention proposes the following technical solutions: a correlation filtering method for enhancing the tracking performance of unmanned aerial vehicles, and the specific technical solutions include the following:

[0009] In order to solve the above problems, the technical solution provided by the present invention comprises the following steps:

[0010] S1. Image acquisition: Use RGB camera to collect raw images and train the discriminative UAV tracking model;

[0011] S2. Feature extraction: Extract features of new images by fusing color histogram features (Colour Name, CN) with gradient histogram features with lower dimensions (fast Histogram of Oriented Gradient, fHOG) to obtain new sample information;

[0012] S3. Feature dimensionality reduction: Use principal component analysis (PCA) to reduce the dimensionality of the obtained features to obtain sample information with smaller dimensions and lower complexity, thereby improving the tracking speed;

[0013] S4. Update the tracking model: Use the elastic regularization network to compare the obtained sample information with the trained model information for similarity, determine whether to learn, and discard the image information with low similarity. Determine that the learned image information is trained to become a new tracking model;

[0014] S5. Optimize model: Optimize the trained tracker model using alternating direction multiplier method (ADMM);

[0015] S6. Tracking: Determine the target location information and track the UAV through continuous iteration.

[0016] Step S3: The principal component analysis process includes the following steps:

[0017] S31: Center all samples: In order to conveniently calculate the variance of the projection point, it can be calculated by the sum of the distances between each point and the origin.

[0018] S32: Calculate the covariance matrix of the sample: E = XX T , the distance from the point to the origin on the coordinate is expressed by the variance between the mapped points, that is, the sum of the absolute values ​​of the dot product of the original vector and the projected vector.

[0019] S33: Eigenvalues ​​λ1, λ2, ...λ of the covariance matrix E mDecompose and take the eigenvectors ω1, ω2, …ω corresponding to the largest d′ eigenvalues d′ .

[0020] S34: Output projection matrix W = (ω1, ω2, ...ω d′ ).

[0021] Transform the problem into a Find a vector ω1 that maximizes the above formula, and the maximum value is XX T The maximum eigenvalue of , vector ω1 is the eigenvector corresponding to the eigenvalue, and the proportion of components retained by the data after principal component analysis is Where d′ is the number of principal components.

[0022] In the above step S4, the elastic regularization network includes the following steps:

[0023] S41: Set σ to the current frame sample information, σ first is the initial frame sample information, The smaller the difference, the higher the similarity between the current frame and the initial frame, and the more the sample should be learned.

[0024]

[0025] S42: Introduce the EL equation to get formula (2) combined with the gradient descent method We get equation (3), and the approximate solution of equation (3) is the value of θ.

[0026]

[0027] S43: Through θ t and The composition time regularization achieves the smoothness of the elastic regularization network,

[0028]

[0029] S44: According to Parseval's theorem, the power of the signal in the time domain and frequency domain is equal. An auxiliary variable is introduced. The symbol ^ represents the discrete Fourier transform (DFT) of the signal, which transforms equation (4) into the frequency domain, where: is the Fourier transform, where F∈R T×T is an orthogonal matrix:

[0030]

[0031] In the above step S5, optimizing the tracker model using the alternating direction multiplier method (ADMM) includes the following steps:

[0032] S51: Through the iterative solution of the alternating direction multiplier method (ADMM), the objective function of the original problem is equivalently decomposed into several solvable sub-problems, and each sub-problem is solved in parallel, which not only reduces the complexity of the algorithm but also improves the tracking speed. The augmented Lagrangian form of the equation is as follows:

[0033]

[0034] In the formula, is the Lagrange multiplier. γ is the step size parameter. The superscript H indicates the conjugate transpose of the matrix.

[0035] S52: Introduction V=[v1,v2,…v K ], formula (6) can be transformed into:

[0036]

[0037] S53: By using the ADMM algorithm, the equation can be decomposed into θ t ,H three sub-problems, formula (5) can be solved to get the optimal solution by iterating these three sub-problems.

[0038] 1) Solution Assume that given The objective function is:

[0039]

[0040] Where conj(.) represents the complex conjugate operation, which is calculated separately The elements in To simplify the calculation. Represented at pixel j The vector of all K channel values ​​of , solution:

[0041]

[0042] The time complexity of formula (9) is O(TK 3 ), after deducing using the Sherman-Morrison formula, its solution can be transformed into:

[0043]

[0044] in, The time complexity at this time is O(TK), which significantly shortens the program calculation time.

[0045] 2) Solve H: Assume that H∈R T×1 Its optimization kIt can be expressed as:

[0046]

[0047] h k The closed form solution is:

[0048]

[0049] Among them, U=diag(u)∈R T×T , The time complexity of solving H is O(KTlgT).

[0050] 3) Solve the optimized time regularization parameter θ t :

[0051] Given the other variables H in equation (7), For θ t Use ADMM algorithm to find θ t The optimal solution can be determined as:

[0052]

[0053] 4) Lagrange multiplier update

[0054]

[0055] In formula (14), i and i+1 represent the number of iterations, ι is the update parameter, and It means that in the (i+1)th iteration, the subproblem of formula (5) is And the sub-problem of equation (11) The solution.

[0056] τ i+1 =min(τ max ,βτ i ) (15)

[0057] In formula (15), β = 10, τ max =10000.

[0058] The specific positioning algorithm in the above step S6 is shown in (16).

[0059]

[0060] In formula (16), R t is the t-frame response graph, is the Fourier form of the feature map extracted from frame t, It represents the Fourier form of the correlation tracker trained in the t-1th frame. The maximum response value is the center position of the tracked target, thereby locating the target. After repeated iterations, the tracking process is realized.

[0061] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects:

[0062] (1) The present invention is applicable to unmanned aerial vehicles and uses a correlation filter tracking algorithm that can be implemented by a CPU with a lower cost instead of a deep learning algorithm that requires a GPU;

[0063] (2) Propose a flexible regularized network to change the current situation where the predefined regularization in traditional algorithms cannot recognize new situations;

[0064] (3) In terms of feature extraction, the CN feature is fused with the fHOG feature with lower dimension to ensure more accurate and complete target information;

[0065] (4) Use PCA for dimensionality reduction to reduce time complexity and improve recognition rate;

[0066] (5) The iterative solution using the alternating direction multiplier method (ADMM) reduces the complexity of the algorithm and improves the tracking speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 The overall flow chart of the correlation filtering method for enhancing the tracking performance of unmanned aerial vehicles of the present invention;

[0068] Figure 2 A comparison diagram before and after PCA dimensionality reduction in the correlation filtering method for enhancing the tracking performance of unmanned aerial vehicles of the present invention;

[0069] Figure 3 A sample learned image in the correlation filtering method for enhancing the tracking performance of an unmanned aerial vehicle according to the present invention;

[0070] Figure 4 The sample is not learned image in the correlation filtering method for enhancing the tracking performance of unmanned aerial vehicle of the present invention;

[0071] Figure 5 The experimental results of comparing the correlation filtering method of the present invention with 7 algorithms on different data sets to enhance the tracking performance of unmanned aerial vehicles;

[0072] Figure 6 The results are compared between the average tracking speed (frames / second) of the correlation filtering method for enhancing the tracking performance of unmanned aerial vehicles of the present invention and nine algorithms on the video sequence of the public data set. DETAILED DESCRIPTION

[0073] The technical solution is further described below in conjunction with the accompanying drawings.

[0074] Example: Figure 1 As shown, the relevant filtering method for enhancing the tracking performance of the unmanned aerial vehicle comprises the following specific steps:

[0075] S1. Image acquisition: The camera is placed in a well-lit environment to collect original images, obtain original sample information, and train a discriminative UAV tracking model;

[0076] S2. Feature extraction: extract features of the new image by fusing color histogram features to obtain new sample information;

[0077] S3. Feature dimensionality reduction: Use principal component analysis to reduce the dimensionality of the obtained features to obtain sample information with smaller dimensions and lower complexity, thereby improving tracking speed;

[0078] S4. Update the tracking model: Use the elastic regularization network to compare the obtained sample information for similarity, determine whether to learn, and discard the image information with low similarity. The image information determined to be learned will be trained to become a new tracking model;

[0079] S5. Optimize model: Optimize the trained tracker model using alternating direction multiplier method (ADMM);

[0080] S6. Tracking: Determine the target location information and track the UAV through continuous iteration.

[0081] The principal component analysis process in step S3 includes the following steps:

[0082] S31: Center all samples: In order to conveniently calculate the variance of the projection point, it can be calculated by the sum of the distances between each point and the origin.

[0083] S32: Calculate the covariance matrix of the sample: E = XX T , the distance from the point to the origin on the coordinate is expressed by the variance between the mapped points, that is, the sum of the absolute values ​​of the dot product of the original vector and the projected vector.

[0084] S33: Eigenvalues ​​λ1, λ2, ...λ of the covariance matrix E m Decompose and take the eigenvectors ω1, ω2, …ω corresponding to the largest d′ eigenvalues d′ .

[0085] S34: Output projection matrix W = (ω1, ω2, ...ω d′ ).

[0086] Transform the problem into a Find a vector ω1 that maximizes the above formula, and the maximum value is XX T The maximum eigenvalue of , vector ω1 is the eigenvector corresponding to the eigenvalue, and the proportion of components retained by the data after principal component analysis is Where d′ is the number of principal components, and the result is as follows Figure 2 shown.

[0087] In the above step S4, the elastic regularization network includes the following steps:

[0088] S41: Set σ to the current frame sample information, σ first is the initial frame sample information, The smaller the difference, the higher the similarity between the current frame and the initial frame, and the more the sample should be learned. Figure 3 As shown, on the contrary Figure 4 shown.

[0089]

[0090] S42: Introduce the EL equation to get formula (2) combined with the gradient descent method We get formula (3), and the approximate solution of formula (3) is The value of .

[0091]

[0092] S43: Through θ t and The composition time regularization achieves the smoothness of the elastic regularization network,

[0093]

[0094] S44: According to Parseval's theorem, the power of the signal in the time domain and frequency domain is equal. An auxiliary variable is introduced. The symbol ^ represents the discrete Fourier transform (DFT) of the signal, which transforms equation (4) into the frequency domain, where: is the Fourier transform, where F∈R T×T is an orthogonal matrix:

[0095]

[0096] In the above step S5, using ADMM to optimize the tracker model includes the following steps:

[0097] S51: Through ADMM iterative solution, the objective function of the original problem is equivalently decomposed into several solvable sub-problems, and each sub-problem is solved in parallel, which not only reduces the complexity of the algorithm but also improves the tracking speed. The augmented Lagrangian form of the equation is as follows:

[0098]

[0099] In the formula, is the Lagrange multiplier. γ is the step size parameter. The superscript H indicates the conjugate transpose of the matrix.

[0100] S52: Introduction V=[v1,v2,…v K ], formula (6) can be transformed into:

[0101]

[0102] S53: By using the ADMM algorithm, the equation can be decomposed into θ t ,H three sub-problems, formula (5) can be solved to get the optimal solution by iterating these three sub-problems.

[0103] (1) Solution Assume that H k ,θ t , The objective function is:

[0104]

[0105] Where conj(.) represents the complex conjugate operation, which is calculated separately The elements in To simplify the calculation. Represented at pixel j The vector of all K channel values ​​of , solution:

[0106]

[0107] The time complexity of formula (9) is O(TK 3 ), after deducing using the Sherman-Morrison formula, its solution can be transformed into:

[0108]

[0109] in, The time complexity at this time is O(TK), which significantly shortens the program calculation time.

[0110] (2) Solving H: Assuming that θ is given t , H∈RT×1 Its optimization k It can be expressed as:

[0111]

[0112] h k The closed form solution is:

[0113]

[0114] Among them, U=diag(u)∈R T×T , The time complexity of solving H is O(KTlgT).

[0115] (3) Solve the optimized time regularization parameter θ t :

[0116] Given the other variables H in equation (7), For θ t Use ADMM algorithm to find θ t The optimal solution can be determined as:

[0117]

[0118] (4) Lagrange multiplier update

[0119]

[0120] Where i and i+1 represent the number of iterations, ι is the update parameter, and It means that in the (i+1)th iteration, the subproblem of formula (5) is And the sub-problem of equation (11) The solution.

[0121] τ i+1 =min(τ max ,βτ i ) (15)

[0122] In formula (15), β = 10, τ max =10000.

[0123] The specific positioning algorithm in the above step S6 is shown in (16).

[0124]

[0125] In formula (16), R t is the t-frame response graph, is the Fourier form of the feature map extracted from frame t, It represents the Fourier form of the correlation tracker trained in the t-1th frame. The maximum response value is the center position of the tracked target, thereby realizing the positioning of the target. After repeated iterations, the tracking process is realized. Figure 5 and Figure 6 As shown, after testing on a variety of public data sets, it is found that the algorithm of the present invention has higher robustness than other tracking algorithms.

[0126] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention, such as changes in certain materials or reaction conditions, should be included in the scope of protection of the present invention.

Claims

1. A correlation filtering method for enhancing the tracking performance of an unmanned aerial vehicle, characterized in that: The specific steps are: S1. Image acquisition: Use RGB camera to collect raw images and train the discriminative UAV tracking model; S2. Feature extraction: Extract features of new images by fusing color histogram features (Colour Name, CN) with gradient histogram features with lower dimensions (fast Histogram of Oriented Gradient, fHOG) to obtain new sample information; S3. Feature dimensionality reduction: Use principal component analysis (PCA) to reduce the dimensionality of the obtained features to obtain sample information with smaller dimensions and lower complexity, thereby improving the tracking speed; S4. Update the tracking model: Use the elastic regularization network to compare the obtained sample information with the trained model information for similarity, determine whether to learn, and discard the image information with low similarity. Determine that the learned image information is trained to become a new tracking model; S5. Optimize model: Optimize the trained tracker model using alternating direction multiplier method (ADMM); S6. Tracking: Determine the target location information and track the UAV through continuous iteration.

2. The correlation filtering method for enhancing the tracking performance of unmanned aerial vehicles according to claim 1, characterized in that: The acquisition device in step S1 is an RGB device.

3. The correlation filtering method for enhancing the tracking performance of unmanned aerial vehicles according to claim 1, characterized in that: The specific steps of step S3 are: S31: Center all samples: In order to conveniently calculate the variance of the projection point, it can be calculated by the sum of the distances between each point and the origin. S32: Calculate the covariance matrix of the sample: E = XX T , the distance from the point to the origin on the coordinate is expressed by the variance between the mapped points, that is, the sum of the absolute values ​​of the dot product of the original vector and the projected vector. S33: Eigenvalues ​​λ1, λ2, ...λ of the covariance matrix E m Decompose and take the eigenvectors ω1, ω2, …ω corresponding to the largest d′ eigenvalues d′ . S34: Output projection matrix W = (ω1, ω2, ...ω d′ ). Transform the problem into a Find a vector ω1 that maximizes the above formula, and the maximum value is XX T The maximum eigenvalue of , vector ω1 is the eigenvector corresponding to the eigenvalue, and the proportion of components retained by the data after principal component analysis is Where d′ is the number of principal components.

4. The correlation filtering method for enhancing the tracking performance of an unmanned aerial vehicle according to claim 3, characterized in that: The specific steps of step S4 are: S41: Set σ to the current frame sample information, σ first is the initial frame sample information, The smaller the difference, the higher the similarity between the current frame and the initial frame, and the more the sample should be learned. S42: Introduce the EL equation to obtain formula (2) combined with the gradient descent method We get formula (3), and the approximate solution of formula (3) is The value of . S43: Through θ t and The composition time regularization achieves the smoothness of the elastic regularization network, S44: According to Parseval's theorem, the power of the signal in the time domain and frequency domain is equal. An auxiliary variable is introduced. The symbol ^ represents the discrete Fourier transform (DFT) of the signal, which transforms equation (4) into the frequency domain, where: is the Fourier transform, where F∈R T×T is an orthogonal matrix:

5. The correlation filtering method for enhancing the tracking performance of unmanned aerial vehicles according to claim 4, characterized in that: The specific steps of step S5 are: S51: Through the iterative solution of the alternating direction multiplier method (ADMM), the objective function of the original problem is equivalently decomposed into several solvable sub-problems, and each sub-problem is solved in parallel, which not only reduces the complexity of the algorithm but also improves the tracking speed. The augmented Lagrangian form of the equation is as follows: In the formula, is the Lagrange multiplier. γ is the step size parameter. The superscript H indicates the conjugate transpose of the matrix. S52: Introduction V=[v1,v2,…v K ], formula (6) can be transformed into: S53: By using the ADMM algorithm, the equation can be decomposed into θ t ,H three sub-problems, formula (5) can be solved to get the optimal solution by iterating these three sub-problems. (1) Solution Assume that H k ,θ t , The objective function is: Where conj(.) represents the complex conjugate operation, which is calculated separately The elements in To simplify the calculation. Represented at pixel j The vector of all K channel values ​​of , solution: The time complexity of formula (9) is O(TK 3 ), after using the Sherman-Morrison formula to derive, its solution can be transformed into: in, The time complexity at this time is O(TK), which significantly shortens the program calculation time. (2) Solving H: Assuming that θ is given t , H∈R T×1 Its optimization k It can be expressed as: h k The closed form solution is: Among them, U=diag(u)∈R T×T , The time complexity of solving H is O(KTlgT). (3) Solve the optimized time regularization parameter θ t : Given the other variables H in equation (7), For θ t Use ADMM algorithm to find θ t The optimal solution can be determined as: (4) Lagrange multiplier update In formula (14), i and i+1 represent the number of iterations, ι is the update parameter, and It means that in the (i+1)th iteration, the subproblem of formula (5) is And the sub-problem of equation (11) The solution. t i+1 =min(τ max ,bt i ) (15) In formula (15), β = 10, τ max =10000.

6. The correlation filtering method for enhancing the tracking performance of an unmanned aerial vehicle according to claim 5, characterized in that: The specific algorithm of step S6 is: In formula (16), R t is the t-frame response graph, is the Fourier form of the feature map extracted from frame t, It represents the Fourier form of the correlation tracker trained in the t-1th frame. The maximum response value is the center position of the tracked target, thereby locating the target. After repeated iterations, the tracking process is realized.

Images