Target tracking method for optical satellite video

Abstract

The application provides a target tracking method for optical satellite video, comprising the following steps: step 1, feature extraction is performed on a slice in an optical satellite video frame; step 2, the feature extraction result is distinguished from a target and a background by using an observation model; and step 3, the distinguishing result of the observation model is corrected by using an anti-interference model updater, and a target position is output. The application effectively solves the problems of a blurred vehicle target and few features, and greatly improves the vehicle target tracking precision and reliability.

Description

Technical Field

[0001] This invention belongs to the technical field of intelligent remote sensing, specifically relating to a target tracking method for optical satellite video. Background Technology

[0002] Target tracking is an important research direction in the field of computer vision. Its main research content is tracking moving targets in videos and obtaining the potential position of the targets in each frame of the video using computers. Optical satellites are a new type of space-based payload for video data in recent years, with advantages such as a large shooting range and high resolution, enabling continuous monitoring of land vehicle targets. However, vehicles, as weak targets in optical satellite videos, are characterized by their small size, indistinct features, and susceptibility to environmental factors, making them relatively difficult to track stably.

[0003] In recent years, correlation filtering algorithms have made great strides in the field of target tracking. Because of the strong ability of correlation filtering to distinguish between background information and target information, correlation filtering-based algorithms have a great advantage in the field of target tracking. In ordinary videos, target tracking algorithms have achieved success, but when tracking vehicle targets in optical satellite videos, the target tracking task still faces many challenges: (1) Image blurring, the target in the video is not easy to identify, and it is difficult to track; (2) Target change of direction, the target does not move in a straight line during the movement; (3) Environmental interference, the target is occluded or interfered with by similar objects around it during the movement. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing a target tracking method for optical satellite video. This method solves the problems of image blurring, target deviation, and environmental interference that affect the accuracy of vehicle target tracking results, thus greatly improving the accuracy and reliability of vehicle target tracking.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A target tracking method for optical satellite video includes the following steps:

[0007] Step 1: Extract features from slices in optical satellite video frames;

[0008] Step 2: Use the observation model to distinguish between the target and the background based on the feature extraction results;

[0009] Step 3: Use an anti-interference model updater to correct the discrimination results of the observation model and output the target location.

[0010] Furthermore, a slice is a region in the current frame where the target may appear; it is a custom-sized image centered on the location of the target in the previous frame of video footage.

[0011] Furthermore, the feature extraction method in step 1 involves vector linking the color features and grayscale features in the slice to form a vector combination.

[0012] Furthermore, the observation model is a correlation filtering model.

[0013] Furthermore, step 2 specifically includes the following steps:

[0014] Step 2.1: Input the feature vector of the first frame image slice of the video into the correlation filter to obtain the original filter;

[0015] Step 2.2: Input the feature vector of the current frame image slice into the correlation filter to obtain a new filter, and update the filter based on the new filter and the exponential weighted average method;

[0016] Step 2.3: Convolve the feature vector of the next frame slice image with the updated filter to obtain the slice score map.

[0017] Furthermore, the correlation filtering model is expressed as:

[0018]

[0019] In the formula, <·, ·> denote the inner product, and w is the filter. Let x be the feature of slice x, and b be a constant parameter of the linear classifier;

[0020] The solution for w is as follows:

[0021]

[0022] Where, x i It is a training sample, y i It is a marker; L(y) i f(x) i )) is the loss function.

[0023] Furthermore, the specific implementation steps of step 3 are as follows:

[0024] Step 3.1: Use APCE to quantize the slice score map shape to detect the target state, where the formula for APCE is:

[0025]

[0026] In the formula, F max F min， F w，hThese correspond to the highest, lowest, and (w, h) values ​​on the slice rating map, respectively.

[0027] Step 3.2: Predict the target location using a Kalman filter;

[0028] Step 3.3: Update the target position based on the APCE quantization result or the Kalman filter prediction result;

[0029] Among them, when APCE and F max When all values ​​are greater than the set threshold, the target position output is F. max The location is determined by the location itself; otherwise, the target location is predicted using a Kalman filter.

[0030] Furthermore, the specific implementation steps in step 3.2 are as follows:

[0031] (1) Input the target's position information from the previous frame into the Kalman filter to obtain the state transition matrix;

[0032] (2) Multiply the state transition matrix by the target's current frame position information to predict the target's position in the next frame. Further, the Kalman filter is defined as:

[0033] p t =Ap t-1 +u t ;

[0034] d t =Bp t +v t ;

[0035] Where A represents the state transition matrix, B represents the state observation matrix, and p t d represents the system state at time t. t u represents the measurement value at time t. t and v t These represent process noise and measurement noise, respectively.

[0036] Furthermore, the Kalman filter state update process is as follows:

[0037]

[0038] The state prediction process is as follows:

[0039]

[0040] Among them, G t C represents the Kalman gain. t Representing the true state pt and the estimated state The error covariance between them Represents the true state p t and predicted state The error covariance between them, V t and U t They represent v respectively t and u t The covariance, where I represents the identity matrix; in this example, A, B, V t and U t It can be set according to the actual situation. and C t It can be randomly initialized as a non-zero matrix; the target position predicted by the Kalman filter is...

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0042] (1) The features extracted from optical satellite video by this invention are color and grayscale fusion features constructed using the feature vector linking method, which effectively solves the problems of blurred vehicle targets and few features. Compared with other feature extraction methods, this method has a better representation effect on vehicles in optical satellite video.

[0043] (2) The observation model of the present invention adopts the kernel correlation filter algorithm. Compared with other classifiers, this algorithm processes data in the frequency domain through Fourier transform, which greatly improves the computational efficiency. At the same time, because of the reasonable use of background information, the algorithm's ability to identify targets is significantly improved, making it a more suitable observation model for optical satellite video vehicle target tracking.

[0044] (3) In response to the environmental interference problem faced by vehicle targets, this invention proposes an anti-interference model updater to correct the discrimination results of the observation model. This model updater can play a role when the vehicle target is occluded or interfered with by similar targets, avoiding the observation model from losing track or mistracking. Compared with other target tracking algorithms, this method of adding a model updater is more robust and more suitable for tracking vehicle targets in optical satellite video. Attached Figure Description

[0045] Figure 1 This is a flowchart of a target tracking method for optical satellite video according to an embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram of the feature extraction method according to an embodiment of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0048] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0049] The present invention will be further described below with reference to specific embodiments, but these are not intended to limit the scope of the invention.

[0050] This invention provides a target tracking method for optical satellite video, comprising the following steps:

[0051] Step 1: Extract features from slices in optical satellite video frames;

[0052] See Figure 1 The open-source library OpenCV is used to decompose optical satellite video into frames and obtain slices. In this embodiment, the slices are the regions in the current frame where the target may appear. They are obtained by cropping video frames and are custom-sized images centered on the location of the target in the previous frame.

[0053] Feature extraction is performed on the slices. In this embodiment, the feature extraction method is to vector-link the color features and grayscale features in the slices. See Figure 2 Specifically, in this example, the color feature vector is the same size as the slice and refers to the probability that the slice pixels are black, blue, brown, green, gray, orange, pink, purple, red, white, and yellow (11 colors in total). It is obtained through an open-source function library and is an 11-dimensional vector, which is then reduced to 10 dimensions using PCA. The grayscale feature vector in this example is also the same size as the slice and refers to the grayscale value of the slice pixels, ranging from 0 to 255. It is a 1-dimensional vector. The vector link between the color feature and grayscale feature in this example is a combination of the color feature vector and the grayscale feature vector, resulting in an 11-dimensional vector. Dimensions 1-10 represent the color features, and the 11th dimension represents the grayscale features.

[0054] Step 2: Use the observation model to distinguish between the target and the background based on the feature extraction results;

[0055] In this embodiment, the observation model is a correlation filtering model. Video frame slices are input into the observation model, and the model outputs a score for each pixel in the slice. The position with the highest score is the target position. The specific implementation of this step is as follows:

[0056] Step 2.1: Input the feature vector of the first frame slice of video into the correlation filtering model. In this embodiment, the filtering model is:

[0057] Define a linear classifier as follows:

[0058]

[0059] Where <·, ·> represent the inner product, and w is the filter. Let w be the feature of slice x, and b be a constant parameter of the linear classifier; w is calculated as follows:

[0060]

[0061] Where, x i It is a training sample, y i It is a marker; L(y) i f(x) i The loss function is L(y), which is the Regularized Least Squares (RLS) loss function used here. i f(x) i ))=(y i -f(x i )) 2 ;λ||w|| 2 It is a regular expression term;

[0062] The feature vector of the first frame slice of the video Inputting this into the filtering model above yields the original filter w. O ;

[0063] Step 2.2: Input the feature vector of the current frame image slice into the above filtering model, and update the filter using the exponential weighted average method;

[0064] Specifically, the feature vector of the current frame slice image of the video. By inputting the data into the filtering model described above, a new filter w can be obtained. N The filter w used for tracking is updated using an exponentially weighted average method. i :

[0065] w i =ηw N +(1-η)w i-1 ;

[0066] Where η is the learning rate, i is the frame number, and w i and w i-1 These represent the filters used for tracking in the current frame and the previous frame, respectively;

[0067] Step 2.4: Convolve the feature vectors of the next frame slice image with the updated filter to obtain the slice score map;

[0068] w i Convolve the slice image with the feature vector of the next frame slice image to obtain the slice score map.

[0069] Step 3: Use an anti-interference model updater to correct the discrimination results of the observation model in Step 2, and output the target location;

[0070] In this embodiment, the step specifically includes:

[0071] Step 3.1: Use APCE to detect the target status;

[0072] First, the shape of the slice score map is quantified using APCE (average peak-to-correlation energy):

[0073]

[0074] Among them, F max F min F w，h These represent the highest, lowest, and (w, h) values ​​on the slice rating map, respectively.

[0075] Specifically, when the target state is normal, the slice score map shows a single peak with smooth edges, and APCE and F max The peak corresponding to the target in the slice scoring image is relatively high when the target is obscured or when there are similar interfering objects in the image. Alternatively, multiple peaks may appear simultaneously in the slice scoring image. In this case, APCE and F... max It will decrease;

[0076] Step 3.2: Predict the target location using a Kalman filter. Specifically, the Kalman filter is defined as follows:

[0077] p t =Ap t-1 +u t ;

[0078] d t =Bp t +v t ;

[0079] Where A represents the state transition matrix, B represents the state observation matrix, and p t d represents the system state at time t. t u represents the measurement value at time t. t and v t These represent process noise and measurement noise, respectively.

[0080] In this example, The true state of the target, including its position and velocity. The target position is output by the tracker; the Kalman filter first predicts the state using the previous frame. Obtain the target estimation state in the current frame. Then use Multiply by the state transition matrix A to obtain the predicted state for the next frame. ^ represents the estimated value, and - represents the predicted value; specifically, the state update process is as follows:

[0081]

[0082] The state prediction process is as follows:

[0083]

[0084] Among them, G t C represents the Kalman gain. t Representing the true state pt and the estimated state The error covariance between them Representing the actual state pt and the predicted state The error covariance between them, V t and U t They represent v respectively t and u t The covariance, where I represents the identity matrix; in this example, A, B, V t and U t It can be set according to the actual situation. and C t It can be randomly initialized as a non-zero matrix; the target position predicted by the Kalman filter is...

[0085] Step 3.3: Update the target location;

[0086] When APCE and F max When both are greater than the preset threshold, the target position output is F. max If the location is unknown, use a Kalman filter to predict the target location.

[0087] The anti-interference model updater in this example consists of a slice scoring evaluation algorithm and a target location prediction algorithm. The slice scoring algorithm evaluates the shape of the slice scoring map; a single-peaked, smooth-around scoring map is considered "normal," with the target location corresponding to the maximum value of the scoring map. A multi-peaked, oscillating scoring map is considered "interferenced," and the target location is given by the target location prediction algorithm. APCE and F... maxThe preset threshold is adjusted according to the specific tracking effect. The target position prediction algorithm in this example consists of a Kalman filter algorithm and a uniform motion velocity estimation algorithm. Before the Kalman filter converges, the uniform motion estimation algorithm calculates the possible velocity of the target and predicts the possible position of the target in the next frame. After the Kalman filter converges, the prediction function of the Kalman filter directly predicts the target position in the next frame.

[0088] This invention addresses the problems of blurred, directionally changeable, and easily interfered-with vehicle targets in optical satellite video by designing a vehicle target tracking method based on correlation filtering. This method utilizes a feature extraction approach more suitable for vehicle target tracking. Through the correlation filtering algorithm, only the position of a single vehicle target in the first frame of the optical satellite video needs to be input to automatically locate the target in subsequent frames. Furthermore, considering the inherent characteristics of optical satellite video, to prevent tracking loss during the tracking process, this invention also designs an interference-resistant model updater to adapt to potential environmental interference issues encountered during vehicle target tracking. This invention represents an extension of target tracking algorithms to optical satellite video, providing a more suitable method for vehicle target tracking in optical satellite video, and has high application value in fields such as transportation, defense, and intelligence.

[0089] In the feature extraction stage, a suitable feature extraction algorithm was designed to extract features based on the characteristics of vehicles in optical satellite video. Grayscale features were obtained directly from image pixel values, while color features were extracted using eleven-dimensional color features, and principal component analysis was used to select the most significant color. These extracted features were then fused using vector linking to obtain the desired feature set.

[0090] During the observation model training phase, features extracted by the feature extraction algorithm are input into the observation model. By adjusting the parameters of the observation model, the model is optimized to achieve the best results. The adjusted observation model parameters can be transferred to vehicle tracking applications using optical satellite video.

[0091] During the model updater phase, when the target is occluded or similar interference appears in the image, the peak corresponding to the target in the slice score map will not be particularly high, or multiple peaks will appear simultaneously in the slice score map. In this case, APCE will decrease. To prevent tracking failure, updating the observation model stops when APCE decreases, and the Kalman filter prediction is used to predict the target position. Only when APCE and F... max The observation model is only updated when all values ​​exceed a certain threshold, and the results of the observation model are trusted. In this way, erroneous information will not be updated into the observation model, and the observation model will be more robust.

[0092] The above are merely preferred embodiments of the present invention and are not intended to limit the implementation methods and scope of protection of the present invention. Those skilled in the art should recognize that any equivalent substitutions and obvious changes made based on the content of this specification should be included within the scope of protection of the present invention.

Claims

1. A target tracking method for optical satellite video, characterized by, The method comprises the following steps: Step 1: feature extraction is performed on a slice in an optical satellite video frame; Step 2: the feature extraction result is distinguished from a target and a background by using an observation model; step 2 specifically comprises the following steps: Step 2.1: a feature vector of a slice of a first frame of video image is input into correlation filtering to obtain an original filter; Step 2.2: a feature vector of a slice of a current frame of video image is input into correlation filtering to obtain a new filter, and the filter is updated according to the new filter and an exponential weighted average method; Step 2.3: the feature vector of the next frame of slice image is convolved with the updated filter to obtain a slice score map; Step 3: the distinguishing result of the observation model is corrected by using an anti-interference model updater, and a target position is output; Step 3 specifically comprises: Step 3.1: APCE is used to quantize the shape of the slice score map to detect the target state, wherein the formula of the APCE is: wherein corresponding to the highest, lowest and value on the position, respectively. Step 3.2: the target position is predicted by using Kalman filtering; Step 3.3: According to The quantization result or the Kalman filter prediction result updates the target position; wherein, when and are all greater than a set threshold, the target position output is the position where the object is located, otherwise the target position predicted using Kalman filtering.

2. The object tracking method against optical satellite video according to claim 1, characterized in that, The slice is a region where the target may appear in the current frame, and is a self-defined image with the target position in the previous frame of video image as the center and a certain size. 3.The target tracking method against optical satellite video of claim 1, wherein, In step 1, the feature extraction method is to link the color features and the gray features in the slice into a vector combination.

4. The object tracking method against optical satellite video of claim 1, wherein, The observation model is a correlation filtering model.

5. The object tracking method against optical satellite video according to claim 4, characterized in that, The correlation filtering model is represented as: ； wherein denotes an inner product, is a filter, is a slice of a feature, is a constant parameter of the linear classifier; wherein The solution of the equation is as follows: wherein, is a training sample, is a label; is a loss function.

6. The object tracking method against optical satellite video of claim 1, wherein, In step 3.2, the specific implementation steps are: (1) the position information of the target in the previous frame is input into Kalman filtering to obtain a state transition matrix; (2) the state transition matrix is multiplied by the position information of the target in the current frame to predict the position of the target in the next frame.

7. The object tracking method against optical satellite video according to claim 1 or 6, characterized in that, The Kalman filtering is defined as: ； ； wherein, denotes a state transition matrix, denotes a state observation matrix, denotes the system state at time k, denotes the measurement at time k, and denote process noise and measurement noise, respectively.

8. The object tracking method against optical satellite video according to claim 7, characterized in that, The state updating process of the Kalman filtering is as follows: ； The state prediction process is as follows: ； wherein, denotes the Kalman gain, denotes the real state and the estimated state error covariance between the real state and the predicted state error covariance between the real state and the predicted state and denote the covariance of and respectively, denotes the unit matrix; in the present example, , , and may be set according to actual conditions, and may be randomly initialized as a matrix other than zero; the target position predicted by the Kalman filter is .

Images