Remote sensing small target tracking method and device combining feature enhancement and short-term memory
By combining feature enhancement and short-term memory in remote sensing video target tracking, KCF algorithm and feature fusion strategy are used to solve the problem of small-target tracking under remote sensing imaging, and the occlusion processing capability and tracking accuracy are improved.
Patent Information
- Application Number
- CN202311828281.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-27
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2043-12-27
AI Technical Summary
The existing remote sensing video target tracking algorithm is difficult to effectively track small targets with few pixels and few features under remote sensing imaging, especially vehicles and small ships, and the tracking effect is not good under occlusion.
Using a method that combines feature enhancement and short-term memory, a target tracking framework is built through the KCF algorithm, adaptive guide filtering and LOG edge features are fused with HOG gradient features to enhance the target feature, and short-term tracking state memory strategies and variable parameters are used. Kalman filtering method performs occlusion judgment and position prediction.
The feature representation ability and occlusion processing ability of small targets are improved, and the tracking success rate and accuracy of remote sensing small targets are significantly improved, solving the problems of few pixels, few features and easy occlusion.
Smart Images

Figure CN117830659B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of remote sensing target tracking, and in particular to a remote sensing small target tracking method and device combining feature enhancement and short-term memory. Background Art
[0002] As one of the basic problems in the field of remote sensing, remote sensing video target tracking plays an important role in remote sensing target monitoring, situational awareness and other occasions. With the rapid development of space-based remote sensing technology, satellites equipped with visible light or infrared video imaging payloads have been launched in recent years. Among them, the Skysat series of Skybox Company in the United States, the "Jilin-1" series of Changguang Satellite Company in China, and the "Zhuhai-1" series of Zhuhai Orbital Company in China have all released their video data, expanding the new direction of remote sensing satellite video processing.
[0003] Although space-based remote sensing video imaging has the advantage of high frame rate compared with traditional push-broom imaging, the research work on space-based remote sensing video target tracking is less than that on conventional target tracking due to the current lack of open source data. Wuhan University started research on satellite video target tracking earlier at home and abroad, and successively proposed satellite video target tracking methods including the combination of kernel correlation filtering and three-frame difference method, the combination of optical flow method and multi-frame difference method, the combination of correlation filtering and Kalman filtering, the correlation filtering method based on velocity features, the background compensation and the convolution regularization network. At the same time, researchers from the Space Application Engineering and Technology Center of the Chinese Academy of Sciences combined the correlation filtering method with target motion estimation to achieve the tracking of moving targets in satellite videos under certain occlusion conditions. Although the above methods have a certain tracking effect on moving targets such as aircraft, ships, and vehicles in typical satellite videos, due to the common characteristics of vehicles, small ships, and other targets in space-based remote sensing imaging, such as few pixels, few features, and easy occlusion, coupled with the shortcomings of existing algorithms in target feature representation and occlusion processing, the tracking success rate and accuracy of small remote sensing targets are still difficult to meet the requirements. Summary of the invention
[0004] In order to solve the above problems, an embodiment of the present invention provides a remote sensing small target tracking method combining feature enhancement and short-term memory, which solves the problem that small rigid targets such as vehicles and small ships are difficult to effectively track under remote sensing imaging due to the small number of pixels, few features and easy occlusion.
[0005] In a first aspect, an embodiment of the present invention includes a remote sensing small target tracking method combining feature enhancement and short-term memory, including S1, obtaining target video data, and building a target tracking framework using a KCF algorithm; S2, reading a video frame sequence of the target video data, and determining a first frame target candidate area; S3, based on the first frame target candidate area, performing adaptive guided filtering to obtain a target enhanced image; S4, based on the target enhanced image, extracting LOG edge features and HOG gradient features, and inputting the two into the target tracking framework to obtain a KCF feature fusion tracking result; S5, based on the KCF feature fusion tracking result, using a short-term tracking state memory strategy to perform occlusion judgment on the target, obtain a judgment result and update the filter template; S6, when the judgment result is that occlusion exists, using a short-term memory strategy to update the filter template, and using a variable parameter Kalman filtering method to predict the target position to determine the final position of the occluded target.
[0006] Optionally, in S1, the use of the KCF algorithm to build a target tracking framework includes: S11, constructing training samples and response label functions; S12, using the KCF algorithm to calculate the training samples and the response label function to obtain a filter expression; S13, introducing a kernel function method to express the filter expression in the form of feature space samples; S14, calculating the filter based on the feature space sample form to confirm the response map of the current frame; S15, calculating the maximum response value of the response map, obtaining the target position obtained by the KCF algorithm, and completing the construction of the target tracking framework.
[0007] Optionally, S4 includes: S41, using a Gaussian function filter to filter the target enhanced image to obtain a filtered image; S42, performing a Laplacian operator operation on the filtered image to obtain the LOG edge feature; S43, extracting the HOG gradient feature as a complementary feature of the LOG edge feature; S44, inputting the HOG gradient feature and the LOG edge feature into the target tracking framework to obtain the KCF feature fusion tracking result.
[0008] Optionally, S5 includes: S51, an occlusion judgment method based on a short-time tracking state memory strategy, selecting the maximum response value, mean and standard deviation within a preset time length as the occlusion judgment threshold; S52, when the current frame response maximum value of the KCF feature fusion tracking result is less than the occlusion judgment threshold, a judgment result is obtained that occlusion exists and the filter template is updated using the short-time tracking state memory strategy; S53, when the current frame response maximum value of the KCF feature fusion tracking result is greater than the occlusion judgment threshold, a judgment result is obtained that occlusion does not exist and the filter template is updated using a linear combination.
[0009] Optionally, S6 includes: S61. When the judgment result is that occlusion exists, the short-term tracking state memory strategy is used to update the filter template, and based on the dynamic change of the noise level during the target tracking process, the variable parameter Kalman filtering method is used to perform position prediction to obtain the final position of the occluded target.
[0010] In a second aspect, an embodiment of the present invention also includes a remote sensing small target tracking device that combines feature enhancement and short-term memory, including a framework building module, used to obtain target video data and build a target tracking framework using a KCF algorithm; a video frame confirmation module, used to read the video frame sequence of the target video data and determine the first frame target candidate area; an image enhancement module, used to perform adaptive guided filtering based on the first frame target candidate area to obtain a target enhanced image; a feature extraction module, used to extract LOG edge features and HOG gradient features based on the target enhanced image, and input the two into the target tracking framework to obtain a KCF feature fusion tracking result; an occlusion judgment module, used to perform occlusion judgment on the target based on the KCF feature fusion tracking result using a short-term tracking state memory strategy to obtain a judgment result; a prediction module, used to update the filter template using a short-term memory strategy when the judgment result is that occlusion exists, and use a variable parameter Kalman filtering method to predict the target position to determine the final position of the occluded target.
[0011] Optionally, the framework building module includes: a sample construction module, which is used to construct training samples and response label functions; a function calculation module, which is used to use the KCF algorithm to calculate the training samples and the response label function to obtain a filter expression; a feature representation module, which is used to introduce a kernel function method to express the filter expression in the form of a feature space sample; a response graph calculation module, which is used to calculate the filter based on the feature space sample form and confirm the response graph of the current frame; and a construction completion module, which is used to calculate the maximum response of the response graph, obtain the target position obtained by the KCF algorithm, and complete the construction of the target tracking framework.
[0012] Optionally, the feature extraction module includes: an image filtering module, used to perform image filtering on the target enhanced image using a Gaussian function filter to obtain a filtered image; an operator operation module, used to perform Laplacian operator operation on the filtered image to obtain the LOG edge feature; a feature complementary module, used to extract the HOG gradient feature as a complementary feature of the LOG edge feature; and a feature input module, used to input the HOG gradient feature and the LOG edge feature into the target tracking framework to obtain the KCF feature fusion tracking result.
[0013] Optionally, the occlusion judgment module includes: a threshold setting module, which is used to select the maximum response value, mean and standard deviation within a preset time length as the occlusion judgment threshold based on the occlusion judgment method of the short-time tracking state memory strategy; an occlusion confirmation module, which is used to draw a judgment result that occlusion exists and update the filter template using the short-time tracking state memory strategy when the current frame response maximum value of the KCF feature fusion tracking result is less than the occlusion judgment threshold; and a non-occlusion confirmation module, which is used to draw a judgment result that occlusion does not exist and update the filter template using a linear combination when the current frame response maximum value of the KCF feature fusion tracking result is greater than the occlusion judgment threshold.
[0014] Optionally, the prediction module includes: a position prediction module, which is used to update the filter template using the short-term tracking state memory strategy when the judgment result is that occlusion exists, and based on the dynamic changes of the noise level during the target tracking process, use the variable parameter Kalman filtering method to perform position prediction to obtain the final position of the occluded target.
[0015] The embodiments of the present invention include the following advantages:
[0016] The present invention (1) aims at the problem that small targets such as vehicles and small ships have few pixels and few features under remote sensing imaging conditions, and proposes a feature enhancement method based on adaptive guided filtering, thereby improving the feature representation capability of small targets. (2) On the basis of adaptive guided filtering target enhancement, combined with its edge preservation advantage, the feature enhancement is further achieved through the LOG edge feature and HOG gradient feature fusion strategy. (3) In view of the problem that the target is difficult to effectively continue tracking when it encounters occlusion, an occlusion judgment and template update method based on a short-term tracking state memory strategy is proposed. This method can effectively update the response maximum value template within a certain period of time before occlusion, thereby ensuring subsequent correct occlusion judgment and continued tracking. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0018] Figure 1 It is a flow chart of a remote sensing small target tracking method combining feature enhancement and short-term memory provided by an embodiment of the present invention;
[0019] Figure 2 It is a schematic diagram of the structure of a remote sensing small target tracking device combining feature enhancement and short-term memory provided by an embodiment of the present invention;
[0020] FIG3( a ) is a graph comparing the tracking accuracy of the target tracking algorithm of the present invention and other algorithms;
[0021] FIG3( b ) is a graph comparing the tracking success rates of the target tracking algorithm of the present invention and other algorithms;
[0022] Figure 4 It is a schematic diagram of the visualization effect of the target tracking algorithm of the present invention and other algorithms.
[0023] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. DETAILED DESCRIPTION
[0024] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.
[0025] In conjunction with the accompanying drawings, the remote sensing small target tracking method and device combining feature enhancement and short-term memory provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.
[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0027] Embodiment 1
[0028] Embodiment 1 of the present invention relates to a remote sensing small target tracking method combining feature enhancement and short-term memory, the process of which is as follows: Figure 1As shown, the method includes acquiring target video data, building a target tracking framework using a KCF algorithm, reading a video frame sequence of the target video data, confirming a manually annotated first frame target candidate area, and performing adaptive guided filtering based on the first frame target candidate area to obtain a target enhanced image. Based on the target enhanced image, LOG edge features and HOG gradient features are extracted, and the two are used as feature inputs of the target tracking framework to obtain a KCF feature fusion tracking result. According to the KCF feature fusion tracking result, a short-term tracking state memory strategy is used to perform occlusion judgment on the target, and a judgment result is obtained and a filter template is updated. When the judgment result is that occlusion exists, the filter template is updated using the short-term memory strategy, and a variable parameter Kalman filtering method is used to predict the target position to determine the final position of the occluded target.
[0029] S1. Obtain target video data and build a target tracking framework using the KCF algorithm;
[0030] It should be noted that the target video data is video data including the existence of the target.
[0031] In S1, the use of the KCF algorithm to build a target tracking framework includes:
[0032] S11, construct training samples and response label functions;
[0033] S12, using the KCF algorithm to calculate the training sample and the response label function to obtain a filter expression;
[0034] S13, introducing a kernel function method to express the filter expression in the form of feature space samples;
[0035] S14, performing calculation based on the filter in the feature space sample form to confirm the response graph of the current frame;
[0036] S15, calculating the maximum response value of the response graph, obtaining the target position obtained by the KCF algorithm, and completing the construction of the target tracking framework.
[0037] This paper adopts the KCF algorithm as the basic framework for target tracking. Assuming that the training sample is x i , the response label function y i is a Gaussian function, and the KCF algorithm attempts to find a suitable filter w such that f(z) = w T z and Gaussian function response y i The sum of squared errors between them is the smallest, that is:
[0038]
[0039] Where λ is a regularization parameter used to prevent overfitting. The above formula can be expressed in matrix form:
[0040]
[0041] Where X represents the training sample constructed by the circulant matrix, y is the Gaussian function response column vector, and the expression of w can be obtained according to the above formula:
[0042] w=(X T X+λI) -1 X T y
[0043] In order to improve the performance of the KCF algorithm, the kernel function method is further introduced to express w as a feature space sample:
[0044]
[0045] And the kernel function κ can be expressed as:
[0046]
[0047] Then f(z) can be expressed as:
[0048]
[0049] In the above formula, α can be obtained by minimizing the objective function:
[0050] α=(K+λI) -1 y
[0051] Where K is the kernel function matrix. Using discrete Fourier transform, α can be further transformed into frequency domain form:
[0052]
[0053] in and Represents y and k respectively xz The discrete Fourier transform of . When the kernel function is a Gaussian kernel, then k xz It can be expressed as:
[0054]
[0055] Thus, the response graph of the current frame can be obtained as:
[0056]
[0057] The maximum value position in the response graph f(z) is the target position obtained by the KCF algorithm.
[0058] Complete the target tracking framework.
[0059] S2. Read a video frame sequence of the target video data and determine a first frame target candidate region.
[0060] The target video data includes a first frame image in which a manually annotated target image exists, and the first frame can be confirmed through the annotation.
[0061] S3. Based on the target candidate area of the first frame, perform adaptive guided filtering to obtain a target enhanced image.
[0062] According to the known target candidate area in the first frame, before it is input into the KCF target tracking framework, it is used as the input image p and the guide image I. Let the guided filtered image be q, and their relationship can be expressed as:
[0063]
[0064] where ω k is a square window area with a radius of r, (a k ,b k ) is the linear coefficient defined in this region. Based on the above linear model, (a) can be solved by minimizing the sum of the square differences between the input image p and the filtered image q. k ,b k ). In order to achieve adaptive feature enhancement of small targets with different signal-to-noise ratios in remote sensing videos, this patent defines the following objective function:
[0065]
[0066] Where ε is to prevent a k Too large regularization parameter, |ω| is the window ω k The number of pixels in the i ) is a function describing the signal-to-noise ratio of a small target, which is defined as:
[0067]
[0068] where σ 2 (p b ) is the variance of the background area in the target candidate area, and its size reflects the interference size of the road or sea background where the vehicle or ship is located; and are the mean values of the target and background areas respectively, and the difference between the two reflects the contrast between the target and the background. By solving the above objective function, the coefficient (a k ,b k )for:
[0069]
[0070] where μk and are window ω k The mean and variance of the guide image I in , For window ω k The mean of the input image p in . Since the filtered image q i Each pixel i is simultaneously captured by multiple windows ω k Therefore, the final filtered image q i Take all windows ω involved in covering pixel i k Corresponding to (a k ,b k ) to obtain the mean value:
[0071]
[0072] in:
[0073]
[0074] By adopting an adaptive guided filtering method that takes the target signal-to-noise ratio into consideration, effective target enhancement can be achieved while maintaining the target edge features in the candidate area.
[0075] S4. Based on the target enhanced image, LOG edge features and HOG gradient features are extracted, and the two are input into the target tracking framework to obtain KCF feature fusion tracking results.
[0076] On the basis of target enhancement based on the proposed adaptive guided filtering, in order to further improve the feature enhancement effect of the tracked target, combined with the edge preservation advantage of adaptive guided filtering and the objective fact that small rigid targets such as vehicles and small ships are almost invisible under space-based remote sensing imaging, it is proposed to further enhance the features by fusing edge and gradient features.
[0077] The S4 includes:
[0078] S41, performing image filtering on the target enhanced image using a Gaussian function filter to obtain a filtered image;
[0079] S42, performing Laplacian operator operation on the filtered image to obtain LOG edge features.
[0080] In the framework of KCF target tracking algorithm, based on the target enhanced image q(x,y), the Laplacian Gaussian (LOG) edge detection operator is used to represent the edge features of small targets. First, the image q(x,y) is filtered using a Gaussian function filter, and then the Laplacian operator is operated on the filtered image to obtain the target edge features:
[0081]
[0082] in:
[0083]
[0084] Gaussian function variance σ in target edge feature extraction process 2 It will directly affect the edge feature extraction effect. Therefore, combined with the target area marked in the first frame, let the Gaussian function variance σ 2 The values are as follows:
[0085] σ 2 =σ 2 (p t )
[0086] where σ 2 (p t ) represents the pixel variance of the target area in the first frame.
[0087] S43, extracting the HOG gradient feature as a complementary feature of the LOG edge feature.
[0088] At the same time, combined with the experimental results showing that small targets such as vehicles and small ships present different gradient distribution characteristics relative to other interference targets, the HOG gradient feature is extracted as the complementary feature of the edge feature.
[0089] S44, inputting the HOG feature and the LOG edge feature into the target tracking framework to obtain the KCF feature fusion tracking result.
[0090] Based on the extracted LOG edge and HOG gradient features, they are used as the feature input of the KCF target tracking framework, and the feature responses of the two are obtained respectively. log With f hog On this basis, the feature fusion strategy is used to further enhance the features. The feature fusion weight adopts the APCE form, which is defined as:
[0091]
[0092] Where i = log or hog represents the LOG edge and HOG gradient features respectively, and the final response after feature enhancement is:
[0093]
[0094] Response f fusion The maximum value position is the KCF feature fusion tracking result.
[0095] S5. According to the KCF feature fusion tracking result, a short-term tracking state memory strategy is used to perform occlusion judgment, obtain a judgment result and update the filter template.
[0096] It should be noted that the target may be occluded during the target tracking process. The conventional response strategy is to use methods such as Kalman filtering to predict the position and then continue to track the target after determining that the target is occluded. However, this type of method uses the filter template of the last frame before the occlusion is determined when continuing to track the target. In fact, the target is partially occluded in the first few frames or even more than ten frames before the current frame is determined to be occluded. At the same time, when similar background or severe noise interference appears during the target tracking process, it is easy to be misjudged as a target occlusion, which leads to filter template update errors.
[0097] In order to solve this problem, this patent proposes an occlusion judgment and filter template update method combined with short-term tracking state memory.
[0098] The S5 includes:
[0099] S51. An occlusion judgment method based on a short-term tracking state memory strategy selects the maximum value mean and standard deviation of the response within a preset time length as an occlusion judgment threshold.
[0100] First, in the target occlusion state judgment, since the maximum response value may change in a trend during the target movement, the mean and standard deviation of the maximum response value within a certain period of time are selected as the occlusion judgment threshold:
[0101] T occ =mean(max(f period ))-m·std(max(f period ))
[0102] Where max(f period ) represents the maximum value of the response within the time period, mean(·) represents the mean operation, std(·) represents the standard deviation operation, and m is a positive integer whose value is obtained by experiment.
[0103] S52. When the maximum value of the current frame response of the KCF feature fusion tracking result is less than the occlusion judgment threshold, it is determined that occlusion exists and the filter template is updated using the short-term tracking state memory strategy.
[0104] When the current frame responds to the maximum value max(f fusion ) is less than the threshold, it is judged that the target is blocked. At this time, the variable parameter Kalman filter algorithm is used to predict the target position, and at the same time, the KCF filter template is updated based on the short-term tracking state memory strategy using the following formula:
[0105]
[0106] Where l represents the current frame, and The short-term tracking state memory strategy ensures that the occlusion judgment method can be applied to different imaging conditions and scenes with gradual changes in target response trends. On the other hand, it ensures that when the target is judged to be occluded, even if partial occlusion has occurred in the previous frames but has not yet been judged as occlusion, the template corresponding to the maximum response value within a certain period of time can be updated using the short-term memory function, thereby better ensuring the subsequent correct occlusion judgment and continuous tracking.
[0107] S53. When the maximum value of the current frame response of the KCF feature fusion tracking result is greater than the occlusion judgment threshold, a judgment result is obtained that there is no occlusion and the filter template is updated using a linear combination.
[0108] When the current frame responds to the maximum value max(f fusion ) is greater than the threshold, it is determined that there is no target occlusion. At this time, the KCF algorithm is continued to be used to determine the target position, and the filter template is updated in the following way:
[0109]
[0110] Where l represents the current frame, η represents the learning rate, and Respectively represent the filter template parameters of the previous frame, and They respectively represent the filter parameters calculated by the KCF feature fusion tracking results of the current frame.
[0111] S6. When the judgment result is that there is occlusion, a variable parameter Kalman filter method is used to predict the target position to determine the final position of the occluded target.
[0112] When the short-term tracking state memory strategy is used to determine that the target is blocked, the variable parameter Kalman filter method is used for position prediction.
[0113] The S6 includes:
[0114] S61. When the judgment result is that there is occlusion, the short-term tracking state memory strategy is used to update the filter template, and based on the dynamic change of the noise level during the target tracking process, the variable parameter Kalman filtering method is used to perform position prediction to obtain the final position of the occluded target.
[0115] The state and measurement equations of the variable parameter Kalman filter method are:
[0116] X n =AX n-1 +Bu n-1+w n-1
[0117] Z n =Hx n +v n
[0118] Where X n With X n-1 Represent the state vector at time n and time n-1 respectively, Z n represents the measurement vector at time n, x n With y n They represent the X and Y positions of the target at time n, respectively; A represents the state transfer matrix, B represents the control matrix, and H represents the measurement matrix; u n-1 represents the control variable at time n-1. Since the target motion acceleration is unknown, the unknown acceleration is regarded as noise and merged into w n-1 Among n-1 With v n They represent the state noise and measurement noise at time n-1 and time n respectively. Both are Gaussian white noise with zero mean and covariance matrices Q and R. t represents the time interval between two frames of images. The relevant expressions are:
[0119]
[0120] On this basis, the following Kalman filter formula is used to estimate the target motion state:
[0121]
[0122] Where P n represents the noise covariance matrix at time n, K n Represents the filter gain at time n. The symbols - and + in the upper right corner represent the prior estimate and the posterior estimate, respectively.
[0123] It should be noted that when using Kalman filtering to predict the target position, the values of the state noise and measurement noise covariance matrix have a direct impact on the accuracy of the target motion state estimation. Existing methods usually use constant form Q and R matrices for Kalman filtering operations, but when there are large errors in the state equation or measurement equation, the constant form Q and R matrices cannot reflect the changes.
[0124] This patent uses a variable parameter Kalman filter method to correct the target motion trajectory in response to the dynamic change of noise level during target tracking. In the initial stage of target tracking, the initial values of the Q and R matrices are set to the following forms:
[0125]
[0126] When the target is not occluded, the Q and R matrices remain unchanged. When the target is occluded, the parameters Q and R matrices become as follows:
[0127]
[0128] Among them, q1 and r1, q2 and r2 correspond to the Kalman filter parameters under unobstructed and obstructed conditions respectively, and the target position under target occlusion is provided by the Kalman filter.
[0129] It should be noted that the above prediction process includes motion trajectory correction, which is not estimated first and then corrected but performed in parallel. That is, in the actual calculation process, the final position of the occluded target can be obtained by prediction after determining the occlusion.
[0130] The target tracking algorithm of the present invention and other algorithms are compared in tracking accuracy as shown in FIG3( a );
[0131] A comparison chart of the tracking effect success rate of the target tracking algorithm of the present invention and other algorithms is shown in FIG3( b );
[0132] The target tracking algorithm of the present invention and other algorithms are shown in the following figure: Figure 4 shown.
[0133] It can be seen from the above figures that the accuracy, success rate and effect of the target tracking algorithm of the present application are better than other algorithms, which obviously enhances the tracking efficiency of small targets and solves the problem that small rigid targets such as vehicles and small ships are difficult to effectively track under remote sensing imaging due to few pixels, few features and easy occlusion.
[0134] The present invention includes obtaining target video data, building a target tracking framework using a KCF algorithm, reading a video frame sequence of the target video data, confirming a manually marked first frame target candidate area, performing adaptive guided filtering based on the first frame target candidate area, obtaining a target enhanced image, extracting LOG edge features and HOG gradient features based on the target enhanced image, using the two as the target tracking framework feature input, obtaining a KCF feature fusion tracking result, and using a short-term tracking state memory strategy and a filter template update method to perform occlusion judgment on the target according to the KCF feature fusion tracking result, obtaining a judgment result and updating the filter template, and when the judgment result is that there is occlusion, using the variable parameter Kalman filtering method to predict the target position and determine the final position of the occluded target. The present invention solves the problem that rigid small targets such as vehicles and small ships are generally difficult to effectively track under remote sensing imaging due to few pixels, few features, and easy occlusion.
[0135] Embodiment 2
[0136] The second embodiment of the present invention relates to a remote sensing small target tracking device 200 combining feature enhancement and short-term memory, such as Figure 2 As shown, including:
[0137] The framework building module 201 is used to obtain target video data and build a target tracking framework using the KCF algorithm;
[0138] A video frame confirmation module 202 is used to read a video frame sequence of the target video data and determine a first frame target candidate area;
[0139] An image enhancement module 203 is used to perform adaptive guided filtering based on the target candidate region of the first frame to obtain a target enhanced image;
[0140] A feature extraction module 204 is used to extract LOG edge features and HOG gradient features based on the target enhanced image, and input the two into the target tracking framework to obtain a KCF feature fusion tracking result;
[0141] The occlusion judgment module 205 is used to perform occlusion judgment on the target according to the KCF feature fusion tracking result, using the short-term tracking state memory strategy, obtain the judgment result and update the filter template;
[0142] The prediction module 206 is used to adopt a short-term memory strategy to update the filter template when the judgment result is that there is occlusion, and adopt a variable parameter Kalman filter method to predict the target position to determine the final position of the occluded target.
[0143] Furthermore, the framework building module 201 includes:
[0144] A sample construction module 2011 is used to construct training samples and response label functions;
[0145] Function calculation module 2012, used to calculate the training sample and the response label function using KCF algorithm to obtain a filter expression;
[0146] A feature representation module 2013, used to introduce a kernel function method to represent the filter expression in the form of feature space samples;
[0147] A response map calculation module 214 is used to perform calculation based on the filter in the feature space sample form to confirm the response map of the current frame;
[0148] The module 2015 is constructed to calculate the maximum value of the response of the response graph, obtain the target position obtained by the KCF algorithm, and complete the construction of the target tracking framework.
[0149] Furthermore, the feature extraction module 204 includes:
[0150] An image filtering module 2041 is used to perform image filtering on the target enhanced image using a Gaussian function filter to obtain a filtered image;
[0151] An operator operation module 2042 is used to perform a Laplacian operator operation on the filtered image to obtain the LOG edge feature;
[0152] A feature complementation module 2043 is used to extract the HOG gradient feature as a complementary feature of the LOG edge feature;
[0153] The feature input module 2044 is used to input the HOG gradient feature and the LOG edge feature into the target tracking framework to obtain the KCF feature fusion tracking result.
[0154] Furthermore, the occlusion determination module 205 includes:
[0155] The threshold setting module 2051 is used for selecting the maximum value mean and standard deviation of the response within a preset time as the occlusion judgment threshold based on the occlusion judgment method of the short-term tracking state memory strategy;
[0156] The occlusion confirmation module 2052 is used to determine that there is occlusion when the maximum value of the current frame response of the KCF feature fusion tracking result is less than the occlusion judgment threshold and update the filter template using the short-term tracking state memory strategy;
[0157] The module 2053 for confirming that there is no occlusion is used to determine that there is no occlusion and update the filter template using a linear combination when the maximum value of the current frame response of the KCF feature fusion tracking result is greater than the occlusion judgment threshold.
[0158] Furthermore, the prediction module 206 includes:
[0159] The position prediction module 2061 is used to update the filter template using the short-term tracking state memory strategy when the judgment result is that there is occlusion, and to perform position prediction using the variable parameter Kalman filtering method based on the dynamic change of the noise level during the target tracking process to obtain the final position of the occluded target.
[0160] The invention constructs a remote sensing small target tracking method combining feature enhancement and short-term memory, comprising a framework building module, which is used to obtain target video data and build a target tracking framework by using a KCF algorithm; a video frame confirmation module, which is used to read a video frame sequence of the target video data and confirm a first frame target candidate area manually marked; an image enhancement module, which is used to perform adaptive guided filtering based on the first frame target candidate area to obtain a target enhanced image; a feature extraction module, which is used to extract LOG edge features and HOG gradient features based on the target enhanced image, and use the two as the target tracking framework feature input to obtain a KCF feature fusion tracking result; an occlusion judgment module, which is used to perform occlusion judgment on the target by using a short-term tracking state memory strategy according to the KCF feature fusion tracking result, obtain a judgment result and update a filter template; and a prediction module, which is used to update the filter template by using the short-term memory strategy when the judgment result is that occlusion exists, and use a variable parameter Kalman filtering method to predict the target position to determine the final position of the occluded target. It solves the problem that small rigid targets such as vehicles and small ships are difficult to effectively track under remote sensing imaging due to the small number of pixels, few features and easy occlusion.
[0161] The technical solution provided by the present application is described in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the present application, and the content of this specification should not be understood as limiting the present application. At the same time, for those skilled in the art, according to the present application, there will be changes in different forms in the specific implementation methods and application scopes. It is not necessary and impossible to exhaust all implementation methods here, and the obvious changes or changes derived from this are still within the scope of protection of the present application. It should be noted that the sequence of the above embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above specific embodiments of the present specification are described. Other embodiments are within the scope of the attached claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and the desired results can still be achieved. In addition, the process depicted in the drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0162] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
[0163] This specification and drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.
Claims
1. A remote sensing small target tracking method combining feature enhancement and short-term memory, characterized in that: include: S1. Obtain target video data and build a target tracking framework using the KCF algorithm; S2, reading a video frame sequence of the target video data, and determining a first frame target candidate region; S3, performing adaptive guided filtering based on the target candidate area of the first frame to obtain a target enhanced image; S4, based on the target enhanced image, extracting LOG edge features and HOG gradient features, inputting the two features into the target tracking framework, and obtaining KCF feature fusion tracking results; S5. According to the KCF feature fusion tracking result, a short-term tracking state memory strategy is used to perform occlusion judgment on the target to obtain a judgment result; S6. When the judgment result is that there is occlusion, a short-term memory strategy is used to update the filter template, and a variable parameter Kalman filter method is used to predict the target position to determine the final position of the occluded target.
2. The remote sensing small target tracking method combining feature enhancement and short-term memory according to claim 1 is characterized in that: In S1, the use of the KCF algorithm to build a target tracking framework includes: S11, construct training samples and response label functions; S12, using the KCF algorithm to calculate the training sample and the response label function to obtain a filter expression; S13, introducing a kernel function method to express the filter expression in the form of feature space samples; S14, performing calculation based on the filter in the feature space sample form to confirm the response graph of the current frame; S15, calculating the maximum response value of the response graph, obtaining the target position obtained by the KCF algorithm, and completing the construction of the target tracking framework.
3. The remote sensing small target tracking method combining feature enhancement and short-term memory according to claim 1 is characterized in that: The S4 includes: S41, performing image filtering on the target enhanced image using a Gaussian function filter to obtain a filtered image; S42, performing Laplacian operator operation on the filtered image to obtain the LOG edge feature; S43, extracting the HOG gradient feature as a complementary feature of the LOG edge feature; S44, inputting the HOG gradient feature and the LOG edge feature into the target tracking framework to obtain the KCF feature fusion tracking result.
4. The remote sensing small target tracking method combining feature enhancement and short-term memory according to claim 1 is characterized in that: The S5 includes: S51, based on the occlusion judgment and filter template updating method of the short-term tracking state memory strategy, the maximum value mean and standard deviation of the response within a preset time length are selected as the occlusion judgment threshold; S52, when the maximum value of the current frame response of the KCF feature fusion tracking result is less than the occlusion judgment threshold, a judgment result is obtained that occlusion exists and the filter template is updated using the short-term tracking state memory strategy; S53. When the maximum value of the current frame response of the KCF feature fusion tracking result is greater than the occlusion judgment threshold, a judgment result is obtained that there is no occlusion and the filter template is updated using a linear combination.
5. The remote sensing small target tracking method combining feature enhancement and short-term memory according to claim 1 is characterized in that: The S6 includes: S61. When the judgment result is that there is occlusion, the short-term tracking state memory strategy is used to update the filter template, and based on the dynamic change of the noise level during the target tracking process, the variable parameter Kalman filtering method is used to perform position prediction to obtain the final position of the occluded target.
6. A remote sensing small target tracking device combining feature enhancement and short-term memory, characterized in that: include: The framework building module is used to obtain target video data and build a target tracking framework using the KCF algorithm; A video frame confirmation module, used for reading a video frame sequence of the target video data and determining a first frame target candidate area; An image enhancement module, configured to perform adaptive guided filtering based on the target candidate region of the first frame to obtain a target enhanced image; A feature extraction module is used to extract LOG edge features and HOG gradient features based on the target enhanced image, and input the two into the target tracking framework to obtain a KCF feature fusion tracking result; An occlusion judgment module is used to perform occlusion judgment on the target according to the KCF feature fusion tracking result, use the short-term tracking state memory strategy, obtain the judgment result and update the filter template; The prediction module is used to update the filter template using a short-term memory strategy when the judgment result is that there is occlusion, and to use a variable parameter Kalman filtering method to predict the target position to determine the final position of the occluded target.
7. The remote sensing small target tracking device combining feature enhancement and short-term memory according to claim 6, characterized in that: The framework building module includes: Sample construction module, used to construct training samples and response label functions; A function calculation module, used to calculate the training sample and the response label function using a KCF algorithm to obtain a filter expression; A feature representation module, used for introducing a kernel function method to represent the filter expression in the form of feature space samples; A response map calculation module, used to perform calculation based on the filter in the form of feature space samples to confirm the response map of the current frame; The module is built to calculate the maximum response value of the response graph, obtain the target position obtained by the KCF algorithm, and complete the target tracking framework.
8. The remote sensing small target tracking device combining feature enhancement and short-term memory according to claim 6, characterized in that: The feature extraction module comprises: An image filtering module, used for performing image filtering on the target enhanced image using a Gaussian function filter to obtain a filtered image; An operator operation module, used for performing Laplacian operator operation on the filtered image to obtain the LOG edge feature; A feature complementation module, used for extracting the HOG gradient feature as a complementary feature of the LOG edge feature; The feature input module is used to input the HOG gradient feature and the LOG edge feature into the target tracking framework to obtain the KCF feature fusion tracking result.
9. The remote sensing small target tracking device combining feature enhancement and short-term memory according to claim 6, characterized in that: The occlusion judgment module comprises: A threshold setting module is used for selecting the maximum value mean and standard deviation of the response within a preset time as the occlusion judgment threshold based on the occlusion judgment method based on the short-term tracking state memory strategy; An occlusion confirmation module is used to determine that occlusion exists and update the filter template using the short-term tracking state memory strategy when the maximum value of the current frame response of the KCF feature fusion tracking result is less than the occlusion judgment threshold; The module for confirming that there is no occlusion is used to determine that there is no occlusion and update the filter template using a linear combination when the maximum value of the current frame response of the KCF feature fusion tracking result is greater than the occlusion judgment threshold.
10. The remote sensing small target tracking device combining feature enhancement and short-term memory according to claim 6, characterized in that: The prediction module comprises: The position prediction module is used to update the filter template using the short-term tracking state memory strategy when the judgment result is that there is occlusion, and to perform position prediction using the variable parameter Kalman filtering method based on the dynamic change of the noise level during the target tracking process to obtain the final position of the occluded target.
Citation Information
Patent Citations
Relevant filtering tracking method based on feature fusion and SVD (Singular Value Decomposition) adaptive model update
CN108734139A
Anti-occlusion pedestrian tracking method based on multi-peak detection
CN110414439A
Photoelectric information processing method based on multilayer rotary memory model
CN113538512A
Long-time target tracking method based on correlation filtering
CN114066935A
Video satellite target tracking method based on multi-feature fusion and motion estimation
CN115439777A
