Satellite video target tracking method, device and equipment and storage medium

Abstract

The invention provides a satellite video target tracking method, device and equipment and a storage medium, which are applied to the technical field of target tracking, and the method comprises the following steps: obtaining a current frame image in a satellite video; inputting the current frame image into a preset target tracking model for target tracking processing, and determining a tracking result corresponding to the target in the current frame image; the target tracking model is obtained by training according to a plurality of satellite training images, a real bounding box of a sample object in each satellite training image and a regression loss between corresponding predicted bounding boxes; the regression loss is determined according to a first Gaussian Wasserstein distance between respective two-dimensional Gaussian distributions of a real bounding box and a corresponding predicted bounding box of the sample object, and the density of central region pixels of the real bounding box in the two-dimensional Gaussian distributions corresponding to the real bounding box is greater than the density of edge region pixels. By adopting the technical scheme of the invention, the satellite target tracking effect can be improved.

Description

Technical Field

[0001] The present invention relates to the field of target tracking technology, and in particular to a satellite video target tracking method, device, equipment and storage medium. Background Art

[0002] With the continuous development of remote sensing technology, satellite video target tracking has become an important means of monitoring the dynamic changes of the earth's surface. At present, satellite target tracking algorithms mainly include tracking algorithms based on correlation filters and tracking algorithms based on deep learning models. Among them, tracking algorithms based on deep learning models include SiamFC++ tracking algorithm (twin tracking algorithm based on full convolution) and SiamMDM tracking algorithm (tracking algorithm based on twin network and adaptive fusion network).

[0003] In the related technology, taking the SiamFC++ tracking algorithm as an example, its tracking principle includes: first, in the first frame of the video, select the target and crop the template image (i.e., template frame), extract the template features through a branch of the feature extraction network (a twin network with shared weights); for the current frame (i.e., search frame), extract the features of the search area through another branch of the network; calculate the similarity between the template features and the search area features, and predict the position and bounding box of the target through the classification branch, quality assessment branch, and regression branch to achieve tracking of the target in the current frame. However, the above technology has the problem of poor tracking effect on satellite video targets. Summary of the invention

[0004] The present invention provides a satellite video target tracking method, device, equipment and storage medium, which are used to solve the defect of poor satellite video target tracking effect caused by measuring the similarity between the predicted and real target tracking frames by the intersection-over-union ratio in the prior art, and realize the calculation of the Gaussian Wasserstein distance between two bounding boxes through a two-dimensional Gaussian distribution in which the density of pixels in the central area of ​​the bounding box is greater than the density of pixels in the edge area, and determine the regression loss based on the Gaussian Wasserstein distance to train the target tracking model, thereby achieving the purpose of accurately tracking the satellite target.

[0005] The present invention provides a satellite video target tracking method, comprising: Acquire a current frame image in the satellite video; the current frame image includes the target; Input the current frame image into a preset target tracking model for target tracking processing, and determine the tracking result corresponding to the target in the current frame image; the above tracking result includes the target bounding box corresponding to the target in the current frame image; The target tracking model is trained based on multiple satellite training images, a true bounding box of a sample object in each satellite training image, and a regression loss between the corresponding predicted bounding box. The regression loss is determined based on the first Gauss Wasserstein distance between the two-dimensional Gaussian distributions of the true bounding box of the sample object and the corresponding predicted bounding box. In the two-dimensional Gaussian distribution corresponding to the true bounding box, the density of pixels in the central area of ​​the true bounding box is greater than the density of pixels in the edge area.

[0006] According to a satellite video target tracking method provided by the present invention, the real bounding box includes position information of the real bounding box, and the training method of the target tracking model includes: The satellite training image is input into the initial target tracking model for target tracking processing, and the prediction bounding box corresponding to the sample object in the satellite training image and the position information of the prediction bounding box are determined; Determine a two-dimensional Gaussian distribution corresponding to the real bounding box according to the position information of the real bounding box, and determine a two-dimensional Gaussian distribution corresponding to the predicted bounding box according to the position information of the predicted bounding box; Calculate the first Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the true bounding box and the two-dimensional Gaussian distribution of the predicted bounding box, and determine the regression loss between the true bounding box and the predicted bounding box according to the first Gaussian Wasserstein distance; The initial target tracking model is trained according to the regression loss to determine the target tracking model.

[0007] According to a satellite video target tracking method provided by the present invention, the first Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the real bounding box and the two-dimensional Gaussian distribution of the predicted bounding box is calculated, and the regression loss between the real bounding box and the predicted bounding box is determined according to the first Gaussian Wasserstein distance, including: Calculate the Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the true bounding box and the two-dimensional Gaussian distribution of the predicted bounding box; The calculated Gauss-Wasserstein distance is normalized using the first normalization formula to determine the first Gauss-Wasserstein distance; The regression loss between the true bounding box and the predicted bounding box is determined by taking the difference between 1 and the first Gaussian Wasserstein distance.

[0008] According to a satellite video target tracking method provided by the present invention, the position information of the real bounding box includes the coordinates of the center point of the real bounding box and the width and height of the real bounding box, and the two-dimensional Gaussian distribution corresponding to the real bounding box is determined according to the position information of the real bounding box, including: The mean is determined according to the coordinates of the center point of the true bounding box, and the variance is determined according to the width and height of the true bounding box; Determine the two-dimensional Gaussian distribution corresponding to the true bounding box based on the mean and variance.

[0009] According to a satellite video target tracking method provided by the present invention, the current frame image is input into a preset target tracking model for target tracking processing, and the tracking result corresponding to the target in the current frame image is determined, including: Input the current frame image into a preset target tracking model for target tracking processing, and determine multiple candidate bounding boxes corresponding to the target in the current frame image; In the target tracking model, a first bounding box corresponding to a previous frame image of the current frame image is obtained, and a second Gaussian Wasserstein distance between each candidate bounding box and the first bounding box is calculated; In the target tracking model, the confidence corresponding to each candidate bounding box is determined according to the second Gauss-Wasserstein distance corresponding to each candidate bounding box, and the target bounding box corresponding to the target in the current frame image is determined in each candidate bounding box according to the confidence corresponding to each candidate bounding box; The tracking result corresponding to the target is determined according to the target bounding box corresponding to the target in the current frame image.

[0010] According to a satellite video target tracking method provided by the present invention, the confidence corresponding to each candidate bounding box is determined according to the second Gaussian Wasserstein distance corresponding to each candidate bounding box, including: Calculating a first ratio between the area of ​​each candidate bounding box and the area of ​​the first bounding box, and calculating a second ratio between the aspect ratio of each candidate bounding box and the aspect ratio of the first bounding box; The confidence level corresponding to each candidate bounding box is determined according to the first proportion, the second proportion, and the second Gauss-Wasserstein distance corresponding to each candidate bounding box.

[0011] According to a satellite video target tracking method provided by the present invention, the above-mentioned calculation of the second Gaussian Wasserstein distance between each candidate bounding box and the first bounding box includes: Calculate the initial second Gaussian Wasserstein distance between each candidate bounding box and the first bounding box; The initial second Gauss-Wasserstein distance of each candidate bounding box is normalized by using the second normalization formula to determine the second Gauss-Wasserstein distance corresponding to each candidate bounding box.

[0012] The present invention also provides a satellite video target tracking device, comprising the following modules: An acquisition module is used to acquire a current frame image in the satellite video; the current frame image includes the target; A tracking module is used to input the current frame image into a preset target tracking model for target tracking processing, and determine the tracking result corresponding to the target in the current frame image; the above tracking result includes a target bounding box corresponding to the target in the current frame image; The target tracking model is trained based on multiple satellite training images, a true bounding box of a sample object in each satellite training image, and a regression loss between the corresponding predicted bounding box. The regression loss is determined based on the first Gaussian Wasserstein distance between the two-dimensional Gaussian distributions of the true bounding box of the sample object and the corresponding predicted bounding box. In the two-dimensional Gaussian distribution corresponding to the true bounding box, the density of pixels in the central area of ​​the true bounding box is greater than the density of pixels in the edge area.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a satellite video target tracking method as described in any one of the above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the satellite video target tracking method as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the satellite video target tracking method as described in any one of the above is implemented.

[0016] The present invention provides a satellite video target tracking method, device, equipment and storage medium. The method obtains a current frame image including a target in a satellite video, inputs the current frame image into a preset target tracking model for target tracking processing, and determines a tracking result corresponding to the target in the current frame image; wherein the tracking result includes a target bounding box corresponding to the target in the current frame image; the target tracking model is trained based on a plurality of satellite training images, a real bounding box of a sample object in each satellite training image, and a regression loss between the corresponding predicted bounding box, the regression loss is determined based on the Gaussian Wasserstein distance between the two-dimensional Gaussian distributions of the real bounding box of the sample object and the corresponding predicted bounding box, and the density of pixels in the central area of ​​the real bounding box in the two-dimensional Gaussian distribution corresponding to the real bounding box is greater than the density of pixels in the edge area. In this method, since the pixel density in the central area of ​​the two-dimensional Gaussian distribution of the two bounding boxes is greater than the density of the pixels in the edge area, the weight distribution of the pixels inside the bounding box can be accurately characterized. Therefore, in the target tracking model training process, more emphasis can be placed on target learning in the central area, thereby improving the performance / effect of satellite video target tracking. At the same time, the regression loss is determined by the Gaussian Wasserstein distance calculated by the two-dimensional Gaussian distribution of the two bounding boxes. The gradient descent of the regression loss determined in this way is relatively gentle in various tracking scenarios, so it has scale invariance, that is, it is insensitive to changes in the size and position of the target, and can effectively distinguish between non-overlapping or fully contained bounding boxes. Therefore, it is more suitable for tracking small targets in satellite videos, thereby improving the accuracy of satellite video target tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 This is one of the flow charts of a satellite video target tracking method provided by the present invention.

[0019] Figure 2 This is the second flow chart of a satellite video target tracking method provided by the present invention.

[0020] Figure 3 It is a schematic diagram of two-dimensional Gaussian distribution modeling of a bounding box provided by the present invention.

[0021] Figure 4 It is a schematic diagram comparing the deviation curves of the Gauss-Wasserstein distance and the intersection-over-union ratio provided by the present invention in three different scenarios.

[0022] Figure 5 This is the third flow chart of a satellite video target tracking method provided by the present invention.

[0023] Figure 6 The present invention is a schematic structural diagram of a satellite video target tracking device provided by the present invention.

[0024] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] As an important means of monitoring the dynamic changes of the earth's surface, satellite video target tracking technology can provide unprecedented data support for environmental monitoring, urban planning, disaster response, military reconnaissance and other fields through high-resolution and high-frequency observation capabilities. At present, most satellite video target tracking algorithms still draw on conventional video tracking algorithms, which can be mainly divided into two categories: one is the tracking algorithm based on correlation filters, such as MOSSE (Minimum Output Sum of Squared Error), CSK (Circulant Structure of Tracking-by-Detection with Kernels) and KCF (Kernelized Correlation Filters). This method has high computational efficiency and simple implementation, and is suitable for real-time tracking scenarios, but its performance is often affected by boundary effects, which may reduce tracking accuracy; the other type of algorithm is based on deep learning models, such as SiamRPN++, SiamFC++, MixFormer and SeqTrack. This type of method shows higher accuracy and stronger robustness in complex environments, but its computational complexity is high and may not be suitable for real-time processing application scenarios.

[0027] Compared with conventional video target tracking, satellite video target tracking usually involves smaller tracking targets. These small-sized targets have limited visual information and are particularly susceptible to interference from factors such as complex backgrounds and unstable lighting conditions. In addition, tracking such small targets usually requires extremely high position accuracy. Even a slight positioning deviation may cause tracking failure. Therefore, conventional target tracking algorithms are not effective for tracking such small targets. Based on this, the industry has proposed many tracking algorithms for small targets in satellite videos, such as using motion estimation methods to accurately track objects, using frame difference or optical flow methods to improve tracking performance, and using target information enhancement strategies to optimize the feature representation of small targets to improve tracking performance. Most of these methods are still based on traditional bounding box similarity measurement strategies, such as intersection over union, to determine the bounding box of the target in each frame to achieve target tracking in each frame.

[0028] However, traditional bounding box similarity measurement strategies, such as intersection over union, are too sensitive to slight position deviations. Slight deviations may cause the performance of the target tracking algorithm to drop significantly, and they cannot effectively measure the situation where the predicted bounding box does not overlap or is completely contained between the true bounding box. In addition, when the shape, size, and orientation of the bounding box change, the traditional bounding box similarity measurement strategy may cause the bounding box similarity measurement results to differ too much, that is, the measurement method is not stable enough, which will weaken the robustness of the target tracking algorithm in complex scenes, especially when the target object moves in an irregular manner, changes significantly in shape, or changes dramatically in perspective, it is more difficult to capture the true position of the target; at the same time, the target tracking algorithm using the traditional bounding box similarity measurement strategy cannot accurately measure the difference in shape, size, and position between the predicted bounding box and the true bounding box; in short, the above methods all ignore the importance of effective measurement of bounding box similarity, resulting in unstable target tracking algorithms and poor tracking performance or tracking effects.

[0029] Based on this, an embodiment of the present invention provides a satellite video target tracking method, device, equipment and storage medium, which can solve the above technical problems.

[0030] Combine the following Figure 1-Figure 5 A satellite video target tracking method according to an embodiment of the present invention is described.

[0031] It should be noted that the execution subject of the embodiment of the present invention may be a satellite video target tracking device, or an electronic device including the satellite video target tracking device, or other devices or apparatuses, which are not specifically limited here. The following embodiments are described by taking the electronic device as the execution subject as an example.

[0032] Figure 1This is one of the flow charts of a satellite video target tracking method provided by the present invention, such as Figure 1 As shown, the method comprises the following steps: Step 102, obtaining a current frame image in the satellite video; the current frame image includes the target.

[0033] In this step, during the satellite video shooting process, each frame of the satellite video can be continuously acquired, and a frame of the image at the current moment is used as the current frame of the image, which generally includes the target to be tracked, and of course can also include other objects. Among them, the target to be tracked can be specifically set according to the actual situation, such as a car, an airplane, a ship, a train, etc.

[0034] Step 104: input the current frame image into a preset target tracking model for target tracking processing, and determine the tracking result corresponding to the target in the current frame image; the tracking result includes a target bounding box corresponding to the target in the current frame image; wherein the target tracking model is trained based on a plurality of satellite training images, a real bounding box of a sample object in each satellite training image, and a regression loss between the corresponding predicted bounding box, the regression loss being determined based on a first Gaussian Wasserstein distance between the two-dimensional Gaussian distributions of the real bounding box of the sample object and the corresponding predicted bounding box, and the density of pixels in the central area of ​​the real bounding box in the two-dimensional Gaussian distribution corresponding to the real bounding box is greater than the density of pixels in the edge area.

[0035] Among them, the target tracking model in this embodiment can be a target tracking model that is improved from the SiamFC++ model of the twin tracking algorithm based on full convolution. The target tracking model can include two network branches, namely a template branch and a search branch, wherein the network architecture of the template branch and the search branch are the same and share the same convolutional network parameters. Both the template branch and the search branch can input a frame of image, and the network structure of the target tracking model is a Siamese (similarity metric) structure, which can ensure that the features of the two input images can be compared in the same space.

[0036] The target tracking model is pre-trained before use, and its training process may include: collecting satellite training videos, which may include multiple / frame satellite training images of sample objects, wherein each satellite training image may be marked with a real bounding box of the sample object in the satellite training image, wherein the marked information may include position information, size information, etc. of the real bounding box, and the sample object may be set according to actual conditions, such as a car, an airplane, a ship, a train, etc. Then, the area where the real bounding box corresponding to the first frame of the satellite training image of the satellite training video is located may be used as the input of the template branch of the initial target tracking model, and each satellite training image of the subsequent frame of the satellite training video may be used as the input of the search branch of the initial target tracking model to perform target tracking processing and output the predicted bounding box of each satellite training image; then, by constructing a two-dimensional Gaussian distribution of the predicted bounding box and the real bounding box of each satellite training image of the subsequent frame of the satellite training video, and calculating the Gaussian Wasserstein distance between the two two-dimensional Gaussian distributions, and then determining the regression loss based on the calculated Gaussian Wasserstein distance, the regression loss is used to train the initial target tracking model to obtain a trained target tracking model.

[0037] When calculating / solving the Gaussian Wasserstein distance between the predicted bounding box and the true bounding box of each satellite training image, the weight set for the pixels in the central area of ​​the predicted bounding box is higher than the weight of the pixels in the edge area. Similarly, the weight set for the pixels in the central area of ​​the true bounding box is higher than the weight of the pixels in the edge area. This allows the target tracking model to focus more on learning the features of the central area of ​​the bounding box during training, making it easier to identify smaller satellite targets. At the same time, it can also reduce the sensitivity of the target tracking model to the size or slight position changes of the target, so that the target tracking model can effectively distinguish between non-overlapping or completely contained bounding boxes, further improving the performance of tracking small satellite targets.

[0038] After the above target tracking model is trained, target tracking can be performed based on the trained target tracking model. The target tracking process is as follows: the template image and the search image (i.e., the current frame image) are input into the target tracking model, and the template image and the search image are respectively subjected to feature extraction using the backbone network (backbone) of the template branch and the search branch with two shared parameters to obtain the feature maps of the template image and the search image. After obtaining the feature maps of the template image and the search image, the two feature maps can be subjected to correlation operations to measure the similarity of the target in the search image, that is, to find the area in the search image that best matches the feature map of the template image. After the search, target classification and positioning processing can be performed, and after the correlation operations, the bounding box corresponding to the target in the current frame image is obtained, which is recorded as the target bounding box.

[0039] It should be noted that the template image is generally the frame image where the target appears for the first time in the satellite video, or it can also be the area / boundary box where the target is located in the first frame. The template image can be saved in the target tracking model during the continuous tracking process. In this way, when tracking the target in subsequent frame images, it is only necessary to continuously input the current frame image into the target tracking model without repeatedly inputting the template image.

[0040] In this embodiment, a current frame image including a target in a satellite video is obtained, and the current frame image is input into a preset target tracking model for target tracking processing, so as to determine a tracking result corresponding to the target in the current frame image; wherein the tracking result includes a target bounding box corresponding to the target in the current frame image; the target tracking model is trained based on a plurality of satellite training images, a real bounding box of a sample object in each satellite training image, and a regression loss between the corresponding predicted bounding box, wherein the regression loss is determined based on a Gaussian Wasserstein distance between the two-dimensional Gaussian distributions of the real bounding box of the sample object and the corresponding predicted bounding box, and the density of pixels in the central area of ​​the real bounding box in the two-dimensional Gaussian distribution corresponding to the real bounding box is greater than the density of pixels in the edge area. In this method, since the pixel density in the central area of ​​the two-dimensional Gaussian distribution of the two bounding boxes is greater than the density of the pixels in the edge area, the weight distribution of the pixels inside the bounding box can be accurately characterized. Therefore, in the target tracking model training process, more emphasis can be placed on target learning in the central area, thereby improving the performance / effect of satellite video target tracking. At the same time, the regression loss is determined by the Gaussian Wasserstein distance calculated by the two-dimensional Gaussian distribution of the two bounding boxes. The gradient descent of the regression loss determined in this way is relatively gentle in various tracking scenarios, so it has scale invariance, that is, it is insensitive to changes in the size and position of the target, and can effectively distinguish between non-overlapping or fully contained bounding boxes. Therefore, it is more suitable for tracking small targets in satellite videos, thereby improving the accuracy of satellite video target tracking.

[0041] The following embodiments illustrate the specific training process of the target tracking model.

[0042] Figure 2 FIG. 2 is a flow chart of a satellite video target tracking method provided by the present invention. Figure 2 As shown, the training method of the above target tracking model includes the following steps: Step 202: input the satellite training image into the initial target tracking model for target tracking processing, and determine the predicted bounding box corresponding to the sample object in the satellite training image and the position information of the predicted bounding box.

[0043] The area where the real bounding box corresponding to the first frame of the satellite training video is located can be used as the input of the template branch of the initial target tracking model, and each satellite training image of the subsequent frames of the satellite training video can be used as the input of the search branch of the initial target tracking model to perform target tracking processing and predict the predicted bounding box corresponding to the sample object in each satellite training image of the subsequent frames. At the same time, the position information of the predicted bounding box is obtained, and the position information of the predicted bounding box can include the coordinates of the center point of the predicted bounding box and the width and height of the predicted bounding box.

[0044] Step 204 , determining a two-dimensional Gaussian distribution corresponding to the real bounding box according to the position information of the real bounding box, and determining a two-dimensional Gaussian distribution corresponding to the predicted bounding box according to the position information of the predicted bounding box.

[0045] In this step, as an optional embodiment, the position information of the real bounding box may also include the center point coordinates of the real bounding box and the width and height of the real bounding box, similar to the position information of the predicted bounding box. Then, the mean may be determined according to the center point coordinates of the real bounding box, and the variance may be determined according to the width and height of the real bounding box; and the two-dimensional Gaussian distribution corresponding to the real bounding box may be determined according to the mean and the variance.

[0046] For example, for a real bounding box, its position information R can be expressed as ,in represents the coordinates of the center point of the real bounding box, w and h represent the width and height of the real bounding box, that is, the size information of the real bounding box. The two-dimensional Gaussian distribution constructed for the real bounding box can be recorded as ,in represents the mean of a two-dimensional Gaussian distribution, The variance of the two-dimensional Gaussian distribution can be expressed by the following formulas: , .

[0047] Further, see Figure 3The schematic diagram of modeling the two-dimensional Gaussian distribution of the bounding box is shown. The two-dimensional Gaussian distribution of the bounding box may include a Gaussian distribution curve (X Distribution) on the X axis and a Gaussian distribution curve (Y Distribution) on the Y axis. When constructing the two-dimensional Gaussian distribution of the real bounding box, it can be assumed that the target corresponding pixels are concentrated in the central area of ​​the bounding box, and the background corresponding pixels are mainly distributed in the edge area of ​​the bounding box. In this way, a set of axis-aligned ellipses can be constructed in the bounding box to represent the equal probability contour lines (2D Gaussian Distribution Contour) of the two-dimensional Gaussian distribution in the bounding box. The two-dimensional Gaussian distribution has the highest density in the central area of ​​the bounding box, and the corresponding density gradually decreases from the central area to the edge area. In other words, the weight corresponding to the pixels in the central area of ​​the bounding box is the largest, and the weight corresponding to the pixels in the central area to the edge area gradually decreases, which can enhance the focus on the target in the central area and minimize the impact of background noise.

[0048] Similarly, the position information of the predicted bounding box includes the coordinates of the center point of the predicted bounding box and the width and height of the predicted bounding box. By modeling the two-dimensional Gaussian distribution of the real bounding box, the two-dimensional Gaussian distribution corresponding to the predicted bounding box can be modeled.

[0049] Step 206 , calculate the first Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the true bounding box and the two-dimensional Gaussian distribution of the predicted bounding box, and determine the regression loss between the true bounding box and the predicted bounding box according to the first Gaussian Wasserstein distance.

[0050] Among them, the Gaussian Wasserstein distance in this embodiment is mainly used to measure the difference between the predicted bounding box and the corresponding true bounding box. The larger the Gaussian Wasserstein distance, the greater the difference between the two bounding boxes. After obtaining the two-dimensional Gaussian distribution of the predicted bounding box and the two-dimensional Gaussian distribution of the corresponding true bounding box, as an optional embodiment, the Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the true bounding box and the two-dimensional Gaussian distribution of the predicted bounding box can be calculated. Specifically, the Gaussian Wasserstein distance between the two-dimensional Gaussian distributions can be calculated using the following formula, which is as follows: .

[0051] in, N a A 2D Gaussian distribution representing the ground-truth bounding box; N b represents a two-dimensional Gaussian distribution of predicted bounding boxes; express N aand N b The Gaussian Wasserstein distance between these two 2D Gaussian distributions; Represents the center point coordinates, width and height of the real bounding box; Represents the center point coordinates, width and height of the predicted bounding box; T represents transposition; ||2 represents the 2-norm.

[0052] Then, as an optional embodiment, the calculated Gauss-Wasserstein distance may be normalized using a first normalization formula to determine a first Gauss-Wasserstein distance. The first normalization formula is as follows: .

[0053] Among them, C is a hyperparameter that can be determined based on experiments, for example, it can be set to 26; express N a and N b The first Gaussian Wasserstein distance between these two two-dimensional Gaussian distributions, where TrGWD stands for Trainingnormalized Gaussian Wasserstein Distance.

[0054] By performing the normalization process on the Gaussian Wasserstein distance between two two-dimensional Gaussian distributions during training, the Gaussian Wasserstein distance between the two two-dimensional Gaussian distributions can be scaled to a range of 0 to 1, so that the first calculated Gaussian Wasserstein distance can directly calculate the regression loss in the training stage of the target tracking model, thereby improving the training efficiency.

[0055] After obtaining the first Gaussian Wasserstein distance, as an optional embodiment, 1 and the first Gaussian Wasserstein distance may be subtracted to determine the regression loss between the true bounding box and the predicted bounding box. The regression loss is calculated as follows: .

[0056] in, L reg represents the regression loss, L gwd Represents the loss calculated by Gaussian Wasserstein distance.

[0057] It should be noted that in this embodiment, Gaussian Wasserstein distance is used to measure the similarity between two bounding boxes. Compared with the intersection-over-union ratio and other methods in the traditional tracking algorithm to measure the similarity between two bounding boxes, the difference between the two bounding boxes calculated by the method of this embodiment is more effective and more suitable for tracking small satellite targets.

[0058] In order to more clearly illustrate that the effect of using Gaussian Wasserstein distance (GWD) to measure the similarity between two bounding boxes in this embodiment is better than the effect of using the traditional method of using intersection-over-union ratio to measure the similarity between two bounding boxes, three different scenarios are described as follows: the predicted bounding box and the true bounding box are completely non-overlapping, partially overlapping, and completely contained. Figure 4 The following is a comparison diagram of the deviation curves of Gauss-Wasserstein distance and intersection-over-union ratio in three different scenarios, where A represents the true bounding box, B, C, and D represent the predicted bounding boxes in three different scenarios, respectively. In scenario B, the predicted bounding box B will not overlap with the true bounding box A after moving to the lower right corner, and the predicted bounding box C in scenario C will still partially overlap with the true bounding box A after moving to the lower right corner, and the predicted bounding box D in scenario D will still completely contain the middle part of the true bounding box A after moving to the lower right corner. The middle image is the deviation curve (GWD Deviation Curve) of Gauss-Wasserstein distance, with the horizontal axis being the deviation (Deviation-pixel) in pixels and the vertical axis being the calculated Gauss-Wasserstein distance (GWD). The right image is the deviation curve (IOU Deviation Curve) of intersection-over-union ratio (IOU), with the horizontal axis being the deviation (Deviation-pixel) in pixels and the vertical axis being the calculated IOU.

[0059] It can be seen from the deviation curve in the figure that in this embodiment, the deviation of GWD in various bounding box scenarios is measured by GWD. The gradient descent of the GWD deviation curve in various bounding box scenarios is relatively gentle (i.e., gradually responding), that is, by measuring the difference between two bounding boxes by GWD, it is possible to accurately distinguish the two bounding boxes in these scenarios that are completely non-overlapping and completely contained, while the value of IOU to measure the difference between two bounding boxes is unchanged in the completely contained scenario (i.e., the green curve), and it is impossible to distinguish the two bounding boxes in the completely contained scenario. At the same time, the GWD deviation curves in various bounding box scenarios are relatively similar, that is, the GWD deviation curves are relatively close at different scales. In other words, the way GWD measures the difference between bounding boxes has scale invariance, which is a continuous and consistent measurement method, while the way IOU measures the difference between bounding boxes is too sensitive to small displacements. Therefore, it can be seen that in this embodiment, the Gaussian Wasserstein distance is used to measure the similarity between two bounding boxes, and its effect is better than the effect of using the intersection-over-union ratio and other methods to measure the similarity between two bounding boxes, and it is more suitable for tracking small satellite targets.

[0060] Step 208: Train the initial target tracking model according to the regression loss to determine the target tracking model.

[0061] In this step, similar to the use process of the target tracking model, during the training process, each satellite training image will be input into the search branch of the initial target tracking model for feature extraction to obtain the feature map of the satellite training image. At the same time, the area where the real bounding box corresponding to the first frame of the satellite training video is located will also be input into the template branch for feature extraction to obtain the feature map of the real bounding box; then, feature matching and similarity calculation are performed on these two feature maps to search for multiple bounding boxes that best match the real bounding box in the satellite training image; then, target classification and positioning processing can be performed on multiple bounding boxes to finally obtain the predicted bounding box of the sample object in the satellite training image and the location information of the predicted bounding box.

[0062] Here, after obtaining the predicted bounding box and its position information, the initial target tracking model can be trained by calculating the regression loss; or the initial target tracking model can be trained in combination with other losses. For the scenario of training the initial target tracking model with multiple losses, the initial target tracking model can be trained by the weighted sum of the classification loss, quality assessment loss and regression loss. The classification loss is mainly used to measure the difference between the model's predicted probability of the existence of the target and the actual label (whether the target exists); the quality assessment loss is mainly used to measure the difference between the center point coordinates of the predicted bounding box and the true bounding box; the regression loss is mainly used to measure the difference between the position and size of the predicted target bounding box and the true target bounding box.

[0063] In this embodiment, the regression loss is calculated by calculating the Gaussian Wasserstein distance between the two-dimensional Gaussian distributions of the two bounding boxes, and the target tracking model is trained accordingly. In this way, the scale and position changes between the two bounding boxes can be better captured by the Gaussian Wasserstein distance, thereby improving the accuracy of the target bounding box finally predicted by the trained target tracking model. In addition, by normalizing the Gaussian Wasserstein distance between the Gaussian distributions of the two bounding boxes, the calculated Gaussian Wasserstein distance is nonlinearly transformed, which can facilitate the direct calculation of the regression loss through the normalized Gaussian Wasserstein distance to train the model, thereby improving the efficiency of model training. Further, the mean and variance of the Gaussian distribution are determined by the center point coordinates, width and height of the two bounding boxes, which can improve the efficiency and accuracy of determining the two-dimensional Gaussian distribution of the two bounding boxes, and at the same time, it can make the subsequent model training pay more attention to the central area of ​​the bounding box, weaken the sensitivity of size and position, and thus improve the tracking accuracy of small satellite targets.

[0064] In order to further improve the accuracy of the predicted target bounding box, an embodiment of the present invention proposes to adjust the confidence of the predicted bounding box by using the Gaussian Wasserstein distance to improve the accuracy of the predicted target bounding box. The following embodiment illustrates this process.

[0065] Figure 5 FIG. 3 is a flow chart of a satellite video target tracking method provided by the present invention. Figure 5 As shown, the above step 104 may include the following steps: Step 302: input the current frame image into a preset target tracking model to perform target tracking processing, and determine multiple candidate bounding boxes corresponding to the target in the current frame image.

[0066] In this step, after the target tracking model is trained, the current frame image can be input into the trained target tracking model for target tracking processing, where the target tracking processing includes feature extraction, feature matching and similarity calculation, target classification and positioning, etc. After processing the target area in the current frame image in the target tracking model, multiple predicted bounding boxes can be obtained, all of which are recorded as candidate bounding boxes, and the position information of each candidate bounding box can also be obtained.

[0067] Step 304: obtain a first bounding box corresponding to a frame image before the current frame image in the target tracking model, and calculate a second Gaussian Wasserstein distance between each candidate bounding box and the first bounding box.

[0068] In this step, before predicting the target bounding box corresponding to the target in the current frame image, the previous frame image of the current frame image is used as the current frame image at the previous moment, and target tracking processing is also performed to obtain multiple bounding boxes and determine a bounding box from them as the target bounding box corresponding to the target in the previous frame image, recorded as the first bounding box, and the position information of the first bounding box can also be obtained.

[0069] Then, after obtaining multiple candidate bounding boxes corresponding to the target in the current frame image at the current moment, as an optional embodiment, the initial second Gaussian Wasserstein distance between each candidate bounding box and the first bounding box can be calculated. The specific calculation can be to determine the two-dimensional Gaussian distribution of each candidate bounding box through the position information of each candidate bounding box, and determine the two-dimensional Gaussian distribution corresponding to the first bounding box through the position information of the first bounding box, and then the above-mentioned formula for calculating the Gaussian Wasserstein distance can be used to calculate the Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the candidate bounding box and the two-dimensional Gaussian distribution of the first bounding box, respectively, and the Gaussian Wasserstein distance calculated for each candidate bounding box is recorded as the initial second Gaussian Wasserstein distance.

[0070] Since the values ​​of Gaussian Wasserstein distance, area ratio and aspect ratio are significantly different, in order to facilitate the subsequent rapid adjustment of the confidence of the bounding box, the initial second Gaussian Wasserstein distance calculated for each candidate bounding box can be scaled to the range of 0 to 1 so that its ratio is consistent with the area ratio and aspect ratio. As an optional embodiment, the initial second Gaussian Wasserstein distance of each candidate bounding box can be normalized using a second normalization formula to determine the second Gaussian Wasserstein distance corresponding to each candidate bounding box. The second normalization formula is as follows: .

[0071] in, N c A two-dimensional Gaussian distribution representing candidate bounding boxes; N d a two-dimensional Gaussian distribution representing the first bounding box; express N c and N d The initial second Gaussian Wasserstein distance between these two 2D Gaussian distributions; express N c and N dThe second Gaussian Wasserstein distance between these two two-dimensional Gaussian distributions, where TeGWD stands for Testing normalized Gaussian Wasserstein Distance.

[0072] Through the above normalization process, the initial second Gaussian Wasserstein distance calculated for each candidate bounding box can be scaled to a range of 0 to 1, thereby obtaining the second Gaussian Wasserstein distance of each candidate bounding box.

[0073] When a candidate bounding box When it is close to 1, that is, the initial second Gaussian Wasserstein distance between the candidate bounding box and the first bounding box of the target in the previous frame image is close to 0, and the second Gaussian Wasserstein distance is close to 1. At this time, it can be indicated that the displacement of the target between the current frame image and the previous frame image is small, and the predicted bounding box is more likely to match the actual movement of the target. This method enables the target tracking model to take into account the dynamic changes of the target, thereby improving the overall target tracking accuracy and robustness.

[0074] Step 306, in the target tracking model, according to the second Gauss-Wasserstein distance corresponding to each candidate bounding box, the confidence corresponding to each candidate bounding box is determined, and according to the confidence corresponding to each candidate bounding box, the target bounding box corresponding to the target in the current frame image is determined in each candidate bounding box.

[0075] Typically, when a target tracking model predicts the bounding box corresponding to the target in each frame, it will output a set of predicted bounding boxes, namely the above-mentioned multiple candidate bounding boxes, and will also output the confidence of each candidate bounding box, which indicates the trustworthiness of the target tracking model for the predicted candidate bounding box. In order to further improve the stability of target tracking, traditional solutions generally adjust the confidence of each candidate bounding box in the current frame image based on the area ratio and aspect ratio between the predicted bounding box in the current frame image and the bounding box of the target in the previous frame image. However, the confidence determined in this way may still not be accurate enough.

[0076] Based on this, in this step, after normalizing to obtain the second Gauss-Wasserstein distance of each candidate bounding box, as an optional embodiment, a first ratio between the area of ​​each candidate bounding box and the area of ​​the first bounding box can be calculated, and a second ratio between the aspect ratio of each candidate bounding box and the aspect ratio of the first bounding box can be calculated; according to the first ratio, the second ratio and the second Gauss-Wasserstein distance corresponding to each candidate bounding box, the confidence corresponding to each candidate bounding box is determined.

[0077] Specifically, the confidence of each candidate bounding box can be calculated using the following formula: .

[0078] in, Represents the confidence of each candidate bounding box; Represents the initial confidence of each candidate bounding box, which can be obtained based on the classification results and quality assessment results of the candidate bounding box; and They respectively represent a first ratio between the area of ​​the candidate bounding box and the area of ​​the first bounding box and a second ratio between the aspect ratio of the candidate bounding box and the aspect ratio of the first bounding box, and both range from 0 to 1.

[0079] After obtaining the confidence of each candidate bounding box predicted in the current frame image through the Gaussian Wasserstein distance, the confidence of each candidate bounding box can be sorted, and the candidate bounding box corresponding to the maximum confidence is selected for output and used as the target bounding box corresponding to the target in the current frame image. In other words, the target tracking model finally outputs the adjusted confidence and predicted target bounding box, and uses the result as the initial state of target tracking in the next frame image.

[0080] In summary, in this embodiment, when adjusting the confidence of each candidate bounding box in the current frame image, the Gaussian Wasserstein distance between each candidate bounding box and the target bounding box of the previous frame is added, and the Gaussian Wasserstein distance can better measure the difference between the two bounding boxes, thereby improving the credibility of the final candidate bounding box, that is, improving the accuracy of the confidence of the candidate bounding box.

[0081] Step 308: determining a tracking result corresponding to the target according to a target boundary box corresponding to the target in the current frame image.

[0082] In this step, after obtaining the target bounding box corresponding to the target in the current frame image, the position information and confidence of the target bounding box can also be obtained, and then all this information can be used as the tracking result of the target in the current frame image.

[0083] In this embodiment, by introducing the Gaussian Wasserstein distance between the bounding boxes of the two frames of images when adjusting the confidence of the bounding box predicted by the target tracking model, the credibility of the candidate bounding box finally determined can be improved, that is, the accuracy of the confidence of the candidate bounding box can be improved, thereby improving the accuracy and robustness of target tracking. In addition, by normalizing the Gaussian Wasserstein distance between the bounding boxes of the two frames of images, the target tracking model can take into account the dynamic changes of the target, thereby improving the overall accuracy and robustness of target tracking. Further, the confidence of the predicted bounding box can be determined by combining the Gaussian Wasserstein distance between the bounding boxes of the two frames of images with the area ratio and the aspect ratio, so that the confidence of the predicted bounding box can be calculated from multiple dimensions, so that the confidence of the bounding box finally determined has higher accuracy and stronger robustness, thereby further improving the accuracy and robustness of target tracking.

[0084] It can be seen from the description of the above embodiments that in the embodiments of the present invention, the Gaussian Wasserstein distance is introduced to optimize the regression loss in the training process of the target tracking model, and the Gaussian Wasserstein distance is introduced to optimize the confidence of the predicted bounding box in the prediction process of the target tracking model. This can effectively reduce the sensitivity of the regression loss under the traditional IOU method to the size of small targets, and significantly improve the accuracy of the confidence of the determined bounding box in the test phase, so that small targets can be tracked more efficiently in satellite video tracking tasks, and higher tracking accuracy and robustness can be achieved.

[0085] Furthermore, in the embodiment of the present invention, the target tracking model of the embodiment of the present invention (denoted as GWD-SiamFC++) is tested and compared with the three trackers of the traditional SiamFC++ tracking algorithm and the SiamMDM tracking algorithm in terms of tracking success rate, accuracy rate and tracking speed through the satellite video dataset SatSOT. The comparison results are shown in the following table:

[0086] Among them, SiamMDM is a satellite video target tracking algorithm used for comparison. Accuracy evaluation indicators include precision and success rate. The precision rate can be measured by calculating the ratio of the number of frames in which the deviation between the center point coordinates of the predicted bounding box and the center point coordinates of the true bounding box is within a certain threshold to the total number of frames. The success rate can be determined by calculating the proportion of frames with successful tracking under different intersection-over-union (IOU) thresholds, and further calculating the area under the curve (AUC, Area Under the ROC Curve, area under the ROC curve) of these proportions. Speed ​​evaluation indicators include tracking speed, which can be evaluated by calculating the average frame rate (FPS, Frames Per Second, that is, the number of frames tracked per second).

[0087] As can be seen from the above table, the target tracking model GWD-SiamFC++ of the embodiment of the present invention has significant advantages in the satellite video small target tracking task. Compared with the baseline model SiamFC++, the tracking success rate and accuracy rate are increased by 3.1% and 6.3% respectively. In addition, the tracking speed of the target tracking model GWD-SiamFC++ of the embodiment of the present invention reaches 208.8FPS, which is comparable to the baseline model SiamFC++. In other words, the module integrated into the target tracking model GWD-SiamFC++ to calculate the Gaussian Wasserstein distance to optimize the regression loss and confidence does not affect the reasoning speed, thereby achieving efficient tracking of small satellite targets.

[0088] It can be further concluded from the above table and the above embodiments that the beneficial effects of this embodiment include: 1. Through a new bounding box similarity measurement strategy based on Gaussian Wasserstein distance, the geometric and statistical characteristics of the target can be captured more accurately. Compared with traditional measurement methods (such as intersection over union (IOU)), the target tracking method of the embodiment of the present invention can not only accurately characterize the weight distribution of pixels inside the bounding box, but also has scale invariance, is insensitive to changes in the size and position of the target, and can effectively distinguish between non-overlapping or fully contained bounding boxes. It is particularly suitable for scenes with large changes in target scale, complex shapes, and more background interference, such as satellite video small target tracking mission scenes.

[0089] 2. Through the nonlinear transformation technology of normalizing the calculated Gaussian Wasserstein distance (GWD), the GWD is converted into normalized distances (TrGWD and TeGWD) suitable for training and testing. This can enhance the adaptability of GWD, enable it to be quickly applied to a variety of target tracking algorithms, and greatly improve the versatility and adaptability of the target tracking model.

[0090] A satellite video target tracking device provided by the present invention is described below. The satellite video target tracking device described below and the satellite video target tracking method described above can be referred to each other.

[0091] Figure 6 is a schematic diagram of the structure of a satellite video target tracking device provided by the present invention, see Figure 6 As shown, the device may include: The acquisition module 410 is used to acquire a current frame image in the satellite video; the current frame image includes the target; The tracking module 420 is used to input the current frame image into a preset target tracking model to perform target tracking processing, and determine the tracking result corresponding to the target in the current frame image; the above tracking result includes a target bounding box corresponding to the target in the current frame image; The target tracking model is trained based on multiple satellite training images, a true bounding box of a sample object in each satellite training image, and a regression loss between the corresponding predicted bounding box. The regression loss is determined based on the first Gaussian Wasserstein distance between the two-dimensional Gaussian distributions of the true bounding box of the sample object and the corresponding predicted bounding box. In the two-dimensional Gaussian distribution corresponding to the true bounding box, the density of pixels in the central area of ​​the true bounding box is greater than the density of pixels in the edge area.

[0092] In some embodiments, the above-mentioned device may also include a training module, which is used to input the satellite training image into the initial target tracking model for target tracking processing, determine the predicted bounding box corresponding to the sample object in the satellite training image and the position information of the predicted bounding box; determine the two-dimensional Gaussian distribution corresponding to the real bounding box according to the position information of the real bounding box, and determine the two-dimensional Gaussian distribution corresponding to the predicted bounding box according to the position information of the predicted bounding box; calculate the first Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the real bounding box and the two-dimensional Gaussian distribution of the predicted bounding box, and determine the regression loss between the real bounding box and the predicted bounding box according to the first Gaussian Wasserstein distance; train the initial target tracking model according to the regression loss to determine the target tracking model.

[0093] Optionally, the above-mentioned training module is specifically used to calculate the Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the real bounding box and the two-dimensional Gaussian distribution of the predicted bounding box; use the first normalization formula to normalize the calculated Gaussian Wasserstein distance to determine the first Gaussian Wasserstein distance; and perform a difference processing between 1 and the first Gaussian Wasserstein distance to determine the regression loss between the real bounding box and the predicted bounding box.

[0094] Optionally, the position information of the real bounding box includes the coordinates of the center point of the real bounding box and the width and height of the real bounding box. The training module is specifically used to determine the mean according to the coordinates of the center point of the real bounding box, and to determine the variance according to the width and height of the real bounding box; and to determine the two-dimensional Gaussian distribution corresponding to the real bounding box according to the mean and the variance.

[0095] In some embodiments, the tracking module 420 is specifically used to input the current frame image into a preset target tracking model for target tracking processing, and determine multiple candidate bounding boxes corresponding to the target in the current frame image; obtain the first bounding box corresponding to the previous frame image of the current frame image in the target tracking model, and calculate the second Gaussian Wasserstein distance between each candidate bounding box and the first bounding box; determine the confidence corresponding to each candidate bounding box in the target tracking model according to the second Gaussian Wasserstein distance corresponding to each candidate bounding box, and determine the target bounding box corresponding to the target in the current frame image in each candidate bounding box according to the confidence corresponding to each candidate bounding box; determine the tracking result corresponding to the target according to the target bounding box corresponding to the target in the current frame image.

[0096] Optionally, the tracking module 420 is specifically used to calculate a first ratio between the area of ​​each candidate bounding box and the area of ​​the first bounding box, and to calculate a second ratio between the aspect ratio of each candidate bounding box and the aspect ratio of the first bounding box; and to determine the confidence level corresponding to each candidate bounding box according to the first ratio, the second ratio and the second Gauss-Wasserstein distance corresponding to each candidate bounding box.

[0097] Optionally, the tracking module 420 is specifically used to calculate an initial second Gaussian Wasserstein distance between each candidate bounding box and the first bounding box; and to normalize the initial second Gaussian Wasserstein distance of each candidate bounding box using a second normalization formula to determine the second Gaussian Wasserstein distance corresponding to each candidate bounding box.

[0098] It should be noted here that the above-mentioned device provided in the embodiment of the present invention can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.

[0099] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7As shown, the electronic device may include: a processor (processor) 510, a communication interface (Communications Interface) 520, a memory (memory) 530 and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call the logic instructions in the memory 530 to execute a satellite video target tracking method, which includes: obtaining a current frame image in the satellite video; the current frame image includes the target; inputting the current frame image into a preset target tracking model for target tracking processing, and determining a tracking result corresponding to the target in the current frame image; the tracking result includes a target bounding box corresponding to the target in the current frame image; wherein the target tracking model is trained based on a plurality of satellite training images, a real bounding box of a sample object in each satellite training image, and a regression loss between the corresponding predicted bounding box, the regression loss is determined based on a first Gauss Wasserstein distance between the two-dimensional Gaussian distributions of the real bounding box of the sample object and the corresponding predicted bounding box, and the density of pixels in the central area of ​​the real bounding box in the two-dimensional Gaussian distribution corresponding to the real bounding box is greater than the density of pixels in the edge area.

[0100] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0101] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute a satellite video target tracking method provided by the above methods, the method including: obtaining a current frame image in a satellite video; the current frame image includes a target; inputting the current frame image into a preset target tracking model for target tracking processing, and determining a tracking result corresponding to the target in the current frame image; the tracking result includes a target bounding box corresponding to the target in the current frame image; wherein the target tracking model is trained based on multiple satellite training images, a real bounding box of a sample object in each satellite training image, and a regression loss between the corresponding predicted bounding box, the regression loss is determined based on the first Gauss Wasserstein distance between the two-dimensional Gaussian distributions of the real bounding box of the sample object and the corresponding predicted bounding box, and the density of pixels in the central area of ​​the real bounding box in the two-dimensional Gaussian distribution corresponding to the real bounding box is greater than the density of pixels in the edge area.

[0102] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented by a processor to execute a satellite video target tracking method provided by the above methods when the computer program is executed, the method comprising: obtaining a current frame image in a satellite video; the current frame image includes a target; inputting the current frame image into a preset target tracking model for target tracking processing, and determining a tracking result corresponding to the target in the current frame image; the tracking result includes a target bounding box corresponding to the target in the current frame image; wherein the target tracking model is trained based on a plurality of satellite training images, a true bounding box of a sample object in each satellite training image, and a regression loss between the corresponding predicted bounding box, the regression loss being determined based on a first Gauss Wasserstein distance between the two-dimensional Gaussian distributions of the true bounding box of the sample object and the corresponding predicted bounding box, and the density of pixels in the central area of ​​the true bounding box in the two-dimensional Gaussian distribution corresponding to the true bounding box is greater than the density of pixels in the edge area.

[0103] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0104] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A satellite video target tracking method, characterized in that: include: Get the current frame image in the satellite video; The current frame image includes a target; Inputting the current frame image into a preset target tracking model for target tracking processing, and determining a tracking result corresponding to the target in the current frame image; The tracking result includes a target bounding box corresponding to the target in the current frame image; The target tracking model is trained based on multiple satellite training images, a true bounding box of a sample object in each of the satellite training images, and a regression loss between the corresponding predicted bounding box, wherein the regression loss is determined based on a first Gauss Wasserstein distance between the two-dimensional Gaussian distributions of the true bounding box of the sample object and the corresponding predicted bounding box, and the density of pixels in the central area of ​​the true bounding box in the two-dimensional Gaussian distribution corresponding to the true bounding box is greater than the density of pixels in the edge area.

2. A satellite video target tracking method according to claim 1, characterized in that: The real bounding box includes position information of the real bounding box, and the training method of the target tracking model includes: Inputting the satellite training image into an initial target tracking model for target tracking processing, determining a predicted bounding box corresponding to a sample object in the satellite training image and position information of the predicted bounding box; Determine a two-dimensional Gaussian distribution corresponding to the real bounding box according to the position information of the real bounding box, and determine a two-dimensional Gaussian distribution corresponding to the predicted bounding box according to the position information of the predicted bounding box; Calculating a first Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the true bounding box and the two-dimensional Gaussian distribution of the predicted bounding box, and determining a regression loss between the true bounding box and the predicted bounding box according to the first Gaussian Wasserstein distance; The initial target tracking model is trained according to the regression loss to determine the target tracking model.

3. A satellite video target tracking method according to claim 2, characterized in that: The calculating a first Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the true bounding box and the two-dimensional Gaussian distribution of the predicted bounding box, and determining a regression loss between the true bounding box and the predicted bounding box according to the first Gaussian Wasserstein distance, comprises: Calculating the Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the true bounding box and the two-dimensional Gaussian distribution of the predicted bounding box; The calculated Gauss-Wasserstein distance is normalized using a first normalization formula to determine the first Gauss-Wasserstein distance; A difference process is performed between 1 and the first Gaussian Wasserstein distance to determine a regression loss between the true bounding box and the predicted bounding box.

4. A satellite video target tracking method according to claim 2, characterized in that: The position information of the real bounding box includes the coordinates of the center point of the real bounding box and the width and height of the real bounding box, and determining the two-dimensional Gaussian distribution corresponding to the real bounding box according to the position information of the real bounding box includes: Determine a mean value according to the coordinates of the center point of the true bounding box, and determine a variance according to the width and height of the true bounding box; A two-dimensional Gaussian distribution corresponding to the true bounding box is determined according to the mean and the variance.

5. A satellite video target tracking method according to any one of claims 1 to 4, characterized in that: The step of inputting the current frame image into a preset target tracking model for target tracking processing and determining a tracking result corresponding to the target in the current frame image includes: Inputting the current frame image into a preset target tracking model to perform target tracking processing, and determining a plurality of candidate bounding boxes corresponding to the target in the current frame image; Acquire a first bounding box corresponding to a frame image before the current frame image in the target tracking model, and calculate a second Gaussian Wasserstein distance between each of the candidate bounding boxes and the first bounding box; Determining, in the target tracking model, a confidence level corresponding to each candidate bounding box according to a second Gauss-Wasserstein distance corresponding to each candidate bounding box, and determining, in each candidate bounding box, a target bounding box corresponding to the target in the current frame image; A tracking result corresponding to the target is determined according to a target boundary box corresponding to the target in the current frame image.

6. A satellite video target tracking method according to claim 5, characterized in that: The step of determining the confidence level corresponding to each candidate bounding box according to the second Gauss-Wasserstein distance corresponding to each candidate bounding box includes: Calculating a first ratio between an area of ​​each of the candidate bounding boxes and an area of ​​the first bounding box, and calculating a second ratio between an aspect ratio of each of the candidate bounding boxes and an aspect ratio of the first bounding box; The confidence level corresponding to each candidate bounding box is determined according to the first proportion, the second proportion, and the second Gauss-Wasserstein distance corresponding to each candidate bounding box.

7. A satellite video target tracking method according to claim 5, characterized in that: The calculating the second Gauss-Wasserstein distance between each of the candidate bounding boxes and the first bounding box comprises: Calculating an initial second Gaussian Wasserstein distance between each of the candidate bounding boxes and the first bounding box; The initial second Gauss-Wasserstein distance of each candidate bounding box is normalized by using a second normalization formula to determine the second Gauss-Wasserstein distance corresponding to each candidate bounding box.

8. A satellite video target tracking device, characterized in that: include: An acquisition module is used to acquire the current frame image in the satellite video; The current frame image includes a target; A tracking module, used for inputting the current frame image into a preset target tracking model to perform target tracking processing, and determining a tracking result corresponding to the target in the current frame image; The tracking result includes a target bounding box corresponding to the target in the current frame image; The target tracking model is trained based on multiple satellite training images, a true bounding box of a sample object in each of the satellite training images, and a regression loss between the corresponding predicted bounding box, wherein the regression loss is determined based on a first Gauss Wasserstein distance between the two-dimensional Gaussian distributions of the true bounding box of the sample object and the corresponding predicted bounding box, and the density of pixels in the central area of ​​the true bounding box in the two-dimensional Gaussian distribution corresponding to the true bounding box is greater than the density of pixels in the edge area.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the satellite video target tracking method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a satellite video target tracking method as described in any one of claims 1 to 7 is implemented.

Images