A satellite video target tracking method, device, equipment and storage medium

Abstract

The application provides a satellite video target tracking method, device and equipment and a storage medium, and is applied to the technical field of target tracking, and the method comprises the following steps: acquiring a current frame image in a satellite video; inputting the current frame image into a preset target tracking model for target tracking processing, and determining a tracking result corresponding to a target in the current frame image; the target tracking model is obtained by training a plurality of satellite training images, a regression loss between a real boundary box of a sample object in each satellite training image and a corresponding predicted boundary box, the regression loss is determined according to a first Gaussian Wasserstein distance between two two-dimensional Gaussian distributions of the real boundary box of the sample object and the corresponding predicted boundary box, and the density of the center region pixels of the real boundary box in the two-dimensional Gaussian distribution corresponding to the real boundary box is greater than the density of the edge region pixels. The technical scheme of the application can improve the effect of satellite target tracking.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target tracking, and in particular relates to a satellite video target tracking method, device, equipment and storage medium. BACKGROUND

[0002] With the continuous development of remote sensing technology, satellite video target tracking has become an important means for monitoring dynamic changes on the earth's surface. At present, satellite target tracking related algorithms mainly include tracking algorithms based on correlation filters and tracking algorithms based on deep learning models, etc., wherein the tracking algorithms based on deep learning models include SiamFC++ tracking algorithm (twin tracking algorithm based on full convolution), SiamMDM tracking algorithm (tracking algorithm based on twin network and adaptive fusion network), etc.

[0003] In related technologies, taking the SiamFC++ tracking algorithm as an example, its tracking principle includes: first, in the first frame of the video, the target is selected and the template image (i.e. the template frame) is cropped out, and the template features are extracted through one branch of the feature extraction network (twin network with shared weights); the features of the search region are extracted through another branch of the network for the current frame (i.e. the search frame); the similarity of the template features and the search region features is calculated, and the position and the bounding box of the target are predicted through the classification branch, the quality evaluation branch and the regression branch, so as to realize the tracking of the target in the current frame. However, the above-mentioned technology has the problem of poor satellite video target tracking effect. SUMMARY

[0004] The present application provides a satellite video target tracking method, device, equipment and storage medium, which solves the defect of poor satellite video target tracking effect caused by measuring the similarity of the predicted and real target tracking boxes by the intersection over union in the prior art, realizes the calculation of the Gaussian Wasserstein distance between two bounding boxes by the two-dimensional Gaussian distribution of the pixel density of the center region of the bounding box being greater than the pixel density of the edge region, and determines the regression loss according to the Gaussian Wasserstein distance to train the target tracking model, thereby realizing the purpose of accurately tracking the satellite target.

[0005] The present application provides a satellite video target tracking method, comprising:

[0006] obtaining a current frame image in a satellite video; the current frame image includes a target;

[0007] inputting the current frame image into a preset target tracking model for target tracking processing to determine a tracking result corresponding to the target in the current frame image; the tracking result includes a target bounding box corresponding to the target in the current frame image;

[0008] The target tracking model is trained according to a plurality of satellite training images, a regression loss between a real bounding box of a sample object in each satellite training image and a corresponding predicted bounding box, and the regression loss is determined according to a first Gaussian Wasserstein distance between two-dimensional Gaussian distributions of the real bounding box and the corresponding predicted bounding box, and a center region pixel of the real bounding box in the two-dimensional Gaussian distribution corresponding to the real bounding box has a higher density than an edge region pixel.

[0009] According to the satellite video target tracking method provided by the application, the real bounding box includes position information of the real bounding box, and the training method of the target tracking model includes:

[0010] The satellite training image is input into an initial target tracking model for target tracking processing, and a predicted bounding box corresponding to a sample object in the satellite training image and position information of the predicted bounding box are determined.

[0011] The two-dimensional Gaussian distribution corresponding to the real bounding box is determined according to the position information of the real bounding box, and the two-dimensional Gaussian distribution corresponding to the predicted bounding box is determined according to the position information of the predicted bounding box.

[0012] The first Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the real bounding box and the two-dimensional Gaussian distribution of the predicted bounding box is calculated, and the regression loss between the real bounding box and the predicted bounding box is determined according to the first Gaussian Wasserstein distance.

[0013] The initial target tracking model is trained according to the regression loss to determine the target tracking model.

[0014] According to the satellite video target tracking method provided by the application, the first Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the real bounding box and the two-dimensional Gaussian distribution of the predicted bounding box is calculated, and the regression loss between the real bounding box and the predicted bounding box is determined according to the first Gaussian Wasserstein distance, including:

[0015] The Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the real bounding box and the two-dimensional Gaussian distribution of the predicted bounding box is calculated.

[0016] The calculated Gaussian Wasserstein distance is normalized by using a first normalization formula to determine the first Gaussian Wasserstein distance.

[0017] The first Gaussian Wasserstein distance is subtracted from 1 to determine the regression loss between the real bounding box and the predicted bounding box.

[0018] According to the satellite video target tracking method provided by the application, the position information of the real bounding box includes the center point coordinates of the real bounding box and the width and height of the real bounding box, the two-dimensional Gaussian distribution corresponding to the real bounding box is determined according to the position information of the real bounding box, and the two-dimensional Gaussian distribution corresponding to the real bounding box is determined according to the position information of the real bounding box.

[0019] The mean value is determined according to the center point coordinates of the real bounding box, and the variance is determined according to the width and height of the real bounding box.

[0020] The two-dimensional Gaussian distribution corresponding to the real bounding box is determined according to the mean value and the variance.

[0021] According to the satellite video target tracking method provided by the application, the position information of the real bounding box includes the center point coordinates of the real bounding box and the width and height of the real bounding box, the two-dimensional Gaussian distribution corresponding to the real bounding box is determined according to the position information of the real bounding box, and the two-dimensional Gaussian distribution corresponding to the real bounding box is determined according to the position information of the real bounding box.

[0022] The current frame image is input into the preset target tracking model for target tracking processing to determine the tracking result corresponding to the target in the current frame image.

[0023] The first bounding box corresponding to the previous frame image of the current frame image is obtained in the target tracking model, and the second Gaussian Wasserstein distance between each candidate bounding box and the first bounding box is calculated.

[0024] The confidence corresponding to each candidate bounding box is determined according to the second Gaussian Wasserstein distance corresponding to each candidate bounding box in the target tracking model, and the target bounding box corresponding to the target in the current frame image is determined from the candidate bounding boxes according to the confidence corresponding to each candidate bounding box.

[0025] The tracking result corresponding to the target is determined according to the target bounding box corresponding to the target in the current frame image.

[0026] According to the satellite video target tracking method provided by the application, the position information of the real bounding box includes the center point coordinates of the real bounding box and the width and height of the real bounding box, the two-dimensional Gaussian distribution corresponding to the real bounding box is determined according to the position information of the real bounding box, and the two-dimensional Gaussian distribution corresponding to the real bounding box is determined according to the position information of the real bounding box.

[0027] The first proportion between the area of each candidate bounding box and the area of the first bounding box is calculated, and the second proportion between the width-height ratio of each candidate bounding box and the width-height ratio of the first bounding box is calculated.

[0028] The confidence corresponding to each candidate bounding box is determined according to the first proportion, the second proportion and the second Gaussian Wasserstein distance corresponding to each candidate bounding box.

[0029] According to the satellite video target tracking method provided by the application, the second Gaussian Wasserstein distance between each candidate bounding box and the first bounding box is calculated, and the second Gaussian Wasserstein distance between each candidate bounding box and the first bounding box is calculated.

[0030] calculate an initial second Gaussian Wasserstein distance between each candidate bounding box and the first bounding box;

[0031] normalize the initial second Gaussian Wasserstein distance of each candidate bounding box by using a second normalization formula to determine the second Gaussian Wasserstein distance corresponding to each candidate bounding box.

[0032] The application further provides a satellite video target tracking device, comprising the following modules:

[0033] an acquisition module configured to acquire a current frame image in a satellite video, wherein the current frame image comprises a target;

[0034] a tracking module configured to input the current frame image into a preset target tracking model to perform target tracking processing and determine a tracking result corresponding to the target in the current frame image, wherein the tracking result comprises a target bounding box corresponding to the target in the current frame image.

[0035] The target tracking model is trained according to a plurality of satellite training images, a regression loss between a real bounding box of a sample object in each satellite training image and a corresponding predicted bounding box, and a first Gaussian Wasserstein distance between two-dimensional Gaussian distributions of the real bounding box and the corresponding predicted bounding box, respectively, wherein the density of the center region pixels of the real bounding box in the two-dimensional Gaussian distribution corresponding to the real bounding box is greater than the density of the edge region pixels.

[0036] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the satellite video target tracking method according to any one of the above when executing the computer program.

[0037] The application further provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the satellite video target tracking method according to any one of the above.

[0038] The application further provides a computer program product comprising a computer program, wherein the computer program is executable on a processor to implement the satellite video target tracking method according to any one of the above.

[0039] The application provides a satellite video target tracking method, device, equipment and storage medium. The current frame image including a target in a satellite video is acquired, and the current frame image is input into a preset target tracking model for target tracking processing to determine a tracking result corresponding to the target in the current frame image. The tracking result includes a target bounding box corresponding to the target in the current frame image. The target tracking model is obtained by training according to a plurality of satellite training images, a regression loss between a real bounding box of a sample object in each satellite training image and a corresponding predicted bounding box, and the regression loss is determined according to a Gaussian Wasserstein distance between two two-dimensional Gaussian distributions of the real bounding box of the sample object and the corresponding predicted bounding box, and the density of the center region pixels in the two-dimensional Gaussian distribution corresponding to the real bounding box is greater than the density of the edge region pixels. In the method, the density of the center region pixels in the two-dimensional Gaussian distribution of the two bounding boxes is greater than the density of the edge region pixels, so that the weight distribution of the pixels inside the bounding box can be accurately represented, and therefore the target learning of the center region can be more focused on in the target tracking model training process, and the performance / effect of the satellite video target tracking is improved. Meanwhile, the regression loss is determined by the Gaussian Wasserstein distance calculated by the two-dimensional Gaussian distribution of the two bounding boxes, so that the gradient descent of the regression loss determined in various tracking scenes is relatively smooth, and therefore the regression loss is scale invariant, that is, not sensitive to the size and position changes of the target, and can effectively distinguish the bounding boxes that do not overlap or completely contain, and therefore is more suitable for tracking small targets in satellite videos, so that the accuracy of the satellite video target tracking can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort.

[0041] Figure 1 is one of the flowcharts of the satellite video target tracking method provided by the application.

[0042] Figure 2 is the second flowchart of the satellite video target tracking method provided by the application.

[0043] Figure 3 is the two-dimensional Gaussian distribution modeling diagram of the bounding box provided by the application.

[0044] Figure 4 is the comparison diagram of the deviation curves of the Gaussian Wasserstein distance and the intersection-over-union in three different scenes provided by the application.

[0045] Figure 5 Figure 3 is a flow diagram of a satellite video target tracking method according to an embodiment of the present application.

[0046] Figure 6 Figure 4 is a structural diagram of a satellite video target tracking device according to an embodiment of the present application.

[0047] Figure 7 Figure 5 is a structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0049] Satellite video target tracking technology, as an important means of monitoring dynamic changes on the earth's surface, can provide unprecedented data support for multiple fields such as environmental monitoring, urban planning, disaster response and military reconnaissance through high-resolution and high-frequency observation capabilities. At present, most satellite video target tracking algorithms still draw lessons from conventional video tracking algorithms, which can be mainly divided into two categories: one category is tracking algorithms based on correlation filters, such as MOSSE (Minimum Output Sum of Squared Error, visual tracking algorithm based on minimum output square error), CSK (Circulant Structure of Tracking-by-Detection with Kernels, circulant structure of tracking-by-detection with kernels) and KCF (Kernelized Correlation Filters, kernelized correlation filters), which are high in computational efficiency, simple to implement, and suitable for real-time tracking scenarios, but their performance is often affected by boundary effects, which may reduce tracking accuracy; the other category of algorithms is based on deep learning models, such as SiamRPN++, SiamFC++, MixFormer and SeqTrack, which show higher accuracy and stronger robustness in complex environments, but have higher computational complexity and may not be suitable for real-time processing application scenarios.

[0050] Compared with a conventional video target tracking, satellite video target tracking usually involves smaller tracking targets, and the visual information of the small targets is limited and is particularly susceptible to interference from complex backgrounds and unstable lighting conditions. In addition, tracking such small targets usually requires extremely high positional accuracy, and even slight positioning deviation can cause tracking failure, so the conventional target tracking algorithm is not good for tracking such small targets. Based on this, the industry currently proposes many tracking algorithms for satellite video small targets, such as using a motion estimation method to accurately track objects, using a frame difference or optical flow method to improve tracking performance, using a target information enhancement strategy to optimize the feature representation of small targets, and the like to improve tracking performance. These methods are mostly based on a traditional bounding box similarity measurement strategy, such as intersection over union, to determine the bounding box of the target in each frame of image to achieve tracking of the target in each frame of image.

[0051] However, the traditional bounding box similarity measurement strategy such as intersection over union is too sensitive to slight positional deviation, and slight deviation can cause a significant decline in the performance of the target tracking algorithm, and cannot effectively measure the non-overlapping or complete inclusion between the predicted bounding box and the real bounding box. In addition, the traditional bounding box similarity measurement strategy can cause a too large difference in the bounding box similarity measurement result when the shape, size and direction of the bounding box change, that is, the measurement method is not stable enough, which can weaken the robustness of the target tracking algorithm in a complex scene, especially when the target object moves in an irregular manner, the shape changes significantly, or the viewing angle changes dramatically, it is more difficult to capture the real position of the target. At the same time, the target tracking algorithm using the traditional bounding box similarity measurement strategy cannot accurately measure the shape, size and position difference between the predicted bounding box and the real bounding box. In summary, the above methods all ignore the importance of effectively measuring the similarity of the bounding box, thereby causing the target tracking algorithm to be not stable enough and the tracking performance or tracking effect to be not good enough.

[0052] Based on this, the embodiments of the present application provide a satellite video target tracking method, device, equipment and storage medium, which can solve the above technical problems.

[0053] The embodiments of the present application will be described below in conjunction with Figures 1-5 A satellite video target tracking method is described.

[0054] It should be noted that the execution subject of the embodiments of the present application can be a satellite video target tracking device, or an electronic device including the satellite video target tracking device, or other devices or apparatuses, which are not specifically limited here. The following embodiments will be described by taking an electronic device as an execution subject.

[0055] Figure 1is one of flowcharts of a satellite video target tracking method provided by the present application, as shown in the figure, the method comprises the following steps: Figure 1

[0056] Step 102, acquiring a current frame image in a satellite video; the current frame image comprises a target.

[0057] In this step, during the satellite video shooting process, each frame image of the satellite video can be continuously acquired, and a frame image at the current time is taken as the current frame image, which generally comprises the target to be tracked, and of course can also comprise other objects. The target to be tracked can be set according to actual conditions, such as a car, an airplane, a ship, a train, etc.

[0058] Step 104, inputting the current frame image into a preset target tracking model for target tracking processing to determine a tracking result corresponding to the target in the current frame image; the tracking result comprises a target bounding box corresponding to the target in the current frame image; wherein the target tracking model is obtained by training according to a plurality of satellite training images, a real bounding box of a sample object in each satellite training image and a regression loss between the corresponding predicted bounding box, the regression loss is determined according to a first Gaussian Wasserstein distance between two-dimensional Gaussian distributions of the real bounding box of the sample object and the corresponding predicted bounding box, and the density of the central region pixels of the real bounding box in the two-dimensional Gaussian distribution corresponding to the real bounding box is greater than that of the edge region pixels.

[0059] Wherein, the target tracking model in the embodiment can be a target tracking model improved based on a full convolution-based twin tracking algorithm SiamFC++ model. The target tracking model can comprise two network branches, namely a template branch and a search branch, wherein the network architecture of the template branch and the search branch is the same, and the same convolution network parameters are shared. The template branch and the search branch can input a frame image, the network structure of the target tracking model is Siamese (similarity measurement) structure, so that the features of the two input images can be compared in the same space.

[0060] ​The target tracking model is pre-trained before use, and the training process can include: collecting a satellite training video, which can include multiple frames of satellite training images of a sample object, wherein each satellite training image can correspond to a labeled real bounding box of the sample object in the satellite training image, wherein the labeled information can include position information, size information, etc. of the real bounding box, and the sample object can be set according to actual conditions, such as a car, an airplane, a ship, a train, etc. Then the area where the real bounding box corresponding to the first frame of satellite training image of the satellite training video is located is taken as the input of the template branch of the initial target tracking model, and each satellite training image of the subsequent frames of the satellite training video is taken as the input of the search branch of the initial target tracking model, target tracking processing is performed, and the predicted bounding box of each satellite training image is output. Then, by constructing a two-dimensional Gaussian distribution of the predicted bounding box and the real bounding box of each satellite training image of the subsequent frames of the satellite training video, and calculating the Gaussian Wasserstein distance between the two two-dimensional Gaussian distributions, a regression loss is determined based on the calculated Gaussian Wasserstein distance, and the initial target tracking model is trained using the regression loss to obtain a trained target tracking model.

[0061] When calculating / solving the Gaussian Wasserstein distance between the predicted bounding box and the real bounding box of each satellite training image, the weight of the center region pixels of the predicted bounding box is set to be higher than that of the edge region pixels, and the weight of the center region pixels of the real bounding box is also set to be higher than that of the edge region pixels. This can make the target tracking model focus more on learning the features of the center region of the bounding box during training, making it easier to identify smaller satellite targets, while also reducing the sensitivity of the target tracking model to size or slight position changes of the target, allowing the target tracking model to effectively distinguish between non-overlapping or completely contained bounding boxes, further improving the performance of satellite small target tracking.

[0062] After the target tracking model is trained, target tracking can be performed based on the trained target tracking model, and the target tracking process is as follows: inputting a template image and a search image (i.e. a current frame image) into the target tracking model, performing feature extraction on the template image and the search image using two shared-parameter backbone networks of the template branch and the search branch, respectively, to obtain feature maps of the template image and the search image. After obtaining the feature maps of the template image and the search image, relevant operations can be performed on the two feature maps to measure the similarity of the target in the search image, i.e. to find the region in the search image that best matches the feature map of the template image. After searching, target classification and positioning processing can be performed, and a bounding box corresponding to the target in the current frame image is obtained after the relevant operation, which is denoted as the target bounding box.

[0063] It should be noted that the template image is generally the first frame image in the satellite video in which the target appears, or it can be the region / boundary box where the target is located in the first frame. This template image can be saved in the target tracking model during the continuous tracking process. In this way, when tracking the target in subsequent frame images, you only need to continuously input the current frame image into the target tracking model, without having to repeatedly input the template image.

[0064] In this embodiment, the current frame image containing the target is acquired from satellite video, and the current frame image is input into a preset target tracking model for target tracking processing to determine the tracking result corresponding to the target in the current frame image. The tracking result includes the target bounding box corresponding to the target in the current frame image. The target tracking model is trained based on the regression loss between multiple satellite training images, the true bounding boxes of sample objects in each satellite training image, and the corresponding predicted bounding boxes. The regression loss is determined based on the Gaussian Wasserstein distance between the two-dimensional Gaussian distributions of the true bounding boxes and the corresponding predicted bounding boxes of the sample objects, and the pixel density in the center region of the true bounding box in the two-dimensional Gaussian distribution corresponding to the true bounding box is greater than the pixel density in the edge region. In this method, since the pixel density in the central region of the two-dimensional Gaussian distribution of the two bounding boxes is greater than that in the edge region, the weight distribution of pixels inside the bounding boxes can be accurately represented. Therefore, during the training of the target tracking model, more emphasis can be placed on target learning in the central region, improving the performance / effect of satellite video target tracking. At the same time, the regression loss is determined by the Gaussian Wasserstein distance calculated by the two-dimensional Gaussian distribution of the two bounding boxes. The gradient descent of the regression loss determined in this way is relatively smooth in various tracking scenarios, so it has scale invariance, that is, it is not sensitive to changes in the size and position of the target. Moreover, it can effectively distinguish between non-overlapping or completely contained bounding boxes, so it is more suitable for tracking small targets in satellite video, thereby improving the accuracy of satellite video target tracking.

[0065] The following examples illustrate the specific training process of the target tracking model.

[0066] Figure 2 This is a second flowchart illustrating a satellite video target tracking method provided by the present invention, as shown below. Figure 2 As shown, the training method for the above target tracking model includes the following steps:

[0067] Step 202: Input the satellite training image into the initial target tracking model for target tracking processing, and determine the predicted bounding box corresponding to the sample object in the satellite training image and the position information of the predicted bounding box.

[0068] This process involves using the region containing the ground truth bounding box in the first frame of the satellite training video as input to the template branch of the initial target tracking model. Simultaneously, each subsequent frame of the satellite training video can be used as input to the search branch of the initial target tracking model for target tracking processing, predicting the bounding box corresponding to the sample object in each subsequent frame. The location information of the predicted bounding box is also obtained, including the coordinates of its center point, width, and height.

[0069] Step 204: Determine the two-dimensional Gaussian distribution corresponding to the true bounding box based on the location information of the true bounding box, and determine the two-dimensional Gaussian distribution corresponding to the predicted bounding box based on the location information of the predicted bounding box.

[0070] In this step, as an optional embodiment, the location information of the actual bounding box, similar to the predicted bounding box location information, may also include the center point coordinates of the actual bounding box and the width and height of the actual bounding box. Then, the mean can be determined based on the center point coordinates of the actual bounding box, and the variance can be determined based on the width and height of the actual bounding box; the two-dimensional Gaussian distribution corresponding to the actual bounding box can be determined based on the mean and variance.

[0071] For example, for a ground truth bounding box, its location information R can be represented as ,in Let w and h represent the coordinates of the center point of the true bounding box, respectively, and w and h represent the width and height of the true bounding box, i.e., the dimensions of the true bounding box. The two-dimensional Gaussian distribution constructed for this true bounding box can be denoted as: ,in This represents the mean of a two-dimensional Gaussian distribution. The variance of a two-dimensional Gaussian distribution can be expressed by the following formulas:

[0072] , .

[0073] Further, see Figure 3The diagram shown illustrates the 2D Gaussian distribution modeling of the bounding box. The 2D Gaussian distribution of the bounding box can include a Gaussian distribution curve along the X-axis (X Distribution) and a Gaussian distribution curve along the Y-axis (Y Distribution). When constructing the 2D Gaussian distribution of the real bounding box, it can be assumed that the target pixels are concentrated in the central region of the bounding box, while the background pixels are mainly distributed in the edge region. This allows for the construction of a set of axis-aligned ellipses within the bounding box to represent the equiprobability contour of the 2D Gaussian distribution. This 2D Gaussian distribution has the highest density in the central region of the bounding box, with the density gradually decreasing from the central region to the edge region. In other words, the weight of pixels in the central region is the highest, and the weight of pixels in the central region gradually decreases from the edge region. This enhances the focus on the target in the central region and minimizes the impact of background noise.

[0074] Similarly, the location information of the predicted bounding box includes the coordinates of the center point of the predicted bounding box, as well as the width and height of the predicted bounding box. By modeling the two-dimensional Gaussian distribution of the real bounding box, the two-dimensional Gaussian distribution corresponding to the predicted bounding box can be modeled and obtained.

[0075] Step 206: Calculate the first Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the true bounding box and the two-dimensional Gaussian distribution of the predicted bounding box, and determine the regression loss between the true bounding box and the predicted bounding box based on the first Gaussian Wasserstein distance.

[0076] In this embodiment, the Gaussian Wasserstein distance is primarily used to measure the difference between the predicted bounding box and the corresponding ground truth bounding box. A larger Gaussian Wasserstein distance indicates a greater difference between the two bounding boxes. After obtaining the two-dimensional Gaussian distributions of the predicted bounding box and its corresponding ground truth bounding box, as an optional embodiment, the Gaussian Wasserstein distance between the two-dimensional Gaussian distributions of the ground truth bounding box and the predicted bounding box can be calculated. Specifically, the Gaussian Wasserstein distance between these two two-dimensional Gaussian distributions can be calculated using the following formula:

[0077] .

[0078] in, N a A two-dimensional Gaussian distribution representing the true bounding box; N b This represents a two-dimensional Gaussian distribution representing the predicted bounding box; express Na and N b The Gaussian Wasserstein distance between these two two-dimensional Gaussian distributions; Represents the center point coordinates, width, and height of the actual bounding box; This represents the center point coordinates, width, and height of the predicted bounding box; T represents the transpose; ||2 represents the 2-norm.

[0079] Subsequently, as an optional embodiment, the calculated Gaussian Wasserstein distance can be normalized using a first normalization formula to determine the first Gaussian Wasserstein distance. The first normalization formula is as follows:

[0080] .

[0081] Where C is a hyperparameter, which can be determined experimentally, for example, it can be set to 26; express N a and N b The first Gaussian Wasserstein distance between these two two-dimensional Gaussian distributions, where TrGWD represents the Training normalized Gaussian Wasserstein Distance.

[0082] By normalizing the Gaussian Wasserstein distance between the two two-dimensional Gaussian distributions during training as described above, the Gaussian Wasserstein distance between the two two-dimensional Gaussian distributions can be scaled to the range of 0 to 1. This allows the calculated first Gaussian Wasserstein distance to be directly used to calculate the regression loss during the target tracking model training phase, thereby improving training efficiency.

[0083] After obtaining the first Gaussian Wasserstein distance as described above, as an optional embodiment, the difference between 1 and the first Gaussian Wasserstein distance can be calculated to determine the regression loss between the true bounding box and the predicted bounding box. The formula for calculating this regression loss is as follows:

[0084] .

[0085] in, L reg Indicates regression loss, L gwd This represents the loss calculated using the Gaussian Wasserstein distance.

[0086] It should be noted that in this embodiment, the Gaussian Wasserstein distance is used to measure the similarity between two bounding boxes. Compared with the intersection-union ratio and other methods used in traditional tracking algorithms to measure the similarity between two bounding boxes, the method in this embodiment is more effective in calculating the difference between two bounding boxes and is more suitable for tracking small satellite targets.

[0087] To more clearly illustrate that this embodiment uses Gaussian Wasserstein distance (GWD) to measure the similarity between two bounding boxes, and that this method outperforms traditional methods such as intersection-over-union (IoU), we will use three different scenarios—complete non-overlap, partial overlap, and complete containment—to illustrate this. See [link to relevant documentation]. Figure 4 The diagram shows a comparison of the deviation curves of Gaussian Wasserstein distance and Intersection over Union (IOU) in three different scenarios. A represents the ground truth bounding box, and B, C, and D represent the predicted bounding boxes in the three scenarios. In scenario B, after moving the predicted bounding box B to the lower right corner, it no longer overlaps with the ground truth bounding box A. In scenario C, after moving the predicted bounding box C to the lower right corner, it still partially overlaps with the ground truth bounding box A. In scenario D, after moving the predicted bounding box D to the lower right corner, it still completely contains the middle part of the ground truth bounding box A. The middle image shows the Gaussian Wasserstein distance deviation curve (GWD Deviation Curve), with the horizontal axis representing the deviation in pixels and the vertical axis representing the calculated Gaussian Wasserstein distance (GWD). The right image shows the Intersection over Union (IOU) deviation curve (IOU Deviation Curve), with the horizontal axis representing the deviation in pixels and the vertical axis representing the calculated IOU.

[0088] As can be seen from the deviation curves in the figure, this embodiment uses GWD to measure the GWD deviation under various bounding box scenarios. The gradient descent of the GWD deviation curve is relatively smooth (i.e., gradually responding) under various bounding box scenarios. This means that using GWD to measure the difference between two bounding boxes can accurately distinguish between two bounding boxes in scenarios where they do not overlap completely and those where they completely contain each other. In contrast, the IOU method, which measures the difference between two bounding boxes, has an invariant value in scenarios where they completely contain each other (i.e., the green curve), and cannot distinguish between two bounding boxes in completely contained scenarios. Furthermore, the GWD deviation curves are quite similar under various bounding box scenarios, meaning that the GWD deviation curves are close at different scales. In other words, the GWD method of measuring bounding box differences is scale-invariant, providing a continuous and consistent measurement. The IOU method, on the other hand, is too sensitive to small displacements. Therefore, it can be seen that using Gaussian Wasserstein distance to measure the similarity between two bounding boxes in this embodiment is more effective than using methods such as intersection-union ratio (IU), and is more suitable for tracking small satellite targets.

[0089] Step 208: Train the initial target tracking model based on the regression loss to determine the target tracking model.

[0090] In this step, similar to the process of using the target tracking model, each satellite training image is input into the search branch of the initial target tracking model for feature extraction during training to obtain the feature map of the satellite training image. At the same time, the region where the ground truth bounding box corresponding to the first frame of the satellite training video is located is also input into the template branch for feature extraction to obtain the feature map of the ground truth bounding box. Then, feature matching and similarity calculation are performed on these two feature maps to search for multiple bounding boxes in the satellite training image that best match the ground truth bounding box. After that, target classification and localization processing can be performed on multiple bounding boxes to finally obtain the predicted bounding box of the sample object in the satellite training image and the location information of the predicted bounding box.

[0091] After obtaining the predicted bounding box and its location information, the initial target tracking model can be trained by calculating a regression loss; alternatively, it can be trained by combining other losses. In the scenario of training the initial target tracking model with multiple losses, the model can be trained by a weighted sum of classification loss, quality assessment loss, and regression loss. The classification loss primarily measures the difference between the model's predicted probability of target existence and the actual label (whether the target exists); the quality assessment loss primarily measures the difference between the center point coordinates of the predicted bounding box and the ground truth bounding box; and the regression loss primarily measures the difference between the position and size of the predicted target bounding box and the ground truth target bounding box.

[0092] In this embodiment, the regression loss is calculated by determining the Gaussian Wasserstein distance between the two bounding boxes' respective two-dimensional Gaussian distributions, and the target tracking model is trained accordingly. This Gaussian Wasserstein distance better captures scale and positional changes between the two bounding boxes, thereby improving the accuracy of the target tracking model's final predicted bounding boxes. Furthermore, normalizing the Gaussian Wasserstein distance between the two bounding boxes involves a non-linear transformation of the calculated distance, facilitating direct calculation of the regression loss for model training and improving training efficiency. Further, the mean and variance of the Gaussian distribution are determined using the center point coordinates, width, and height of the two bounding boxes. This improves the efficiency and accuracy of determining the two-dimensional Gaussian distribution of the bounding boxes and allows subsequent model training to focus more on the central region of the bounding boxes, reducing sensitivity to size and position, thus improving tracking accuracy for small satellite targets.

[0093] To further improve the accuracy of the predicted target bounding box, this embodiment of the invention proposes to adjust the confidence of the predicted bounding box by using the Gaussian Wasserstein distance to improve the accuracy of the predicted target bounding box. The following embodiments illustrate this process.

[0094] Figure 5 This is the third flowchart of a satellite video target tracking method provided by the present invention, as shown below. Figure 5 As shown, step 104 above may include the following steps:

[0095] Step 302: Input the current frame image into the preset target tracking model for target tracking processing to determine multiple candidate bounding boxes corresponding to the target in the current frame image.

[0096] In this step, after the target tracking model is trained, the current frame image can be input into the trained target tracking model for target tracking processing. This target tracking processing includes feature extraction, feature matching and similarity calculation, target classification and localization, etc. After processing the target region in the current frame image within the target tracking model, multiple predicted bounding boxes can be obtained, all denoted as candidate bounding boxes. Simultaneously, the positional information of each candidate bounding box can also be obtained.

[0097] Step 304: Obtain the first bounding box corresponding to the previous frame image of the current frame image in the target tracking model, and calculate the second Gaussian Wasserstein distance between each candidate bounding box and the first bounding box.

[0098] In this step, before predicting the target bounding box corresponding to the target in the current frame image, the previous frame image of the current frame image is used as the current frame image at the previous moment. Target tracking processing is also performed to obtain multiple bounding boxes and determine one of them as the target bounding box corresponding to the target in the previous frame image, denoted as the first bounding box. At the same time, the position information of the first bounding box can also be obtained.

[0099] After obtaining multiple candidate bounding boxes corresponding to the target in the current frame image at the current moment, as an optional embodiment, an initial second Gaussian Wasserstein distance between each candidate bounding box and the first bounding box can be calculated. Specifically, the calculation can be performed by determining the two-dimensional Gaussian distribution of each candidate bounding box based on its position information, and by determining the two-dimensional Gaussian distribution corresponding to the first bounding box based on its position information. Then, the Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the candidate bounding box and the two-dimensional Gaussian distribution of the first bounding box can be calculated using the formula described above. The Gaussian Wasserstein distance calculated for each candidate bounding box is recorded as the initial second Gaussian Wasserstein distance.

[0100] Since the Gaussian Wasserstein distance differs significantly from the area ratio and aspect ratio, to facilitate rapid adjustment of the bounding box confidence, the initial second Gaussian Wasserstein distance calculated for each candidate bounding box can be scaled to a range of 0 to 1, making its ratio consistent with the area ratio and aspect ratio. As an optional embodiment, a second normalization formula can be used to normalize the initial second Gaussian Wasserstein distance of each candidate bounding box, determining the corresponding second Gaussian Wasserstein distance for each candidate bounding box. The second normalization formula is as follows:

[0101] .

[0102] in, N c A two-dimensional Gaussian distribution representing the candidate bounding box; N d Represents the two-dimensional Gaussian distribution of the first bounding box; express N c and N d The initial second Gaussian Wasserstein distance between these two two-dimensional Gaussian distributions; express N c and N dThe second Gaussian Wasserstein distance between these two two-dimensional Gaussian distributions, where TeGWD represents the Testing normalized Gaussian Wasserstein Distance.

[0103] By performing the normalization process described above, the initial second Gaussian Wasserstein distance calculated for each candidate bounding box can be scaled to the range of 0 to 1, thus obtaining the second Gaussian Wasserstein distance for each candidate bounding box.

[0104] When a candidate bounding box When the distance is close to 1, that is, the initial second Gaussian Wasserstein distance between the candidate bounding box and the first bounding box of the target in the previous frame image is close to 0 and the second Gaussian Wasserstein distance is close to 1, it indicates that the displacement of the target between the current frame image and the previous frame image is small. Therefore, the predicted bounding box is more likely to match the actual movement of the target. This method allows the target tracking model to take into account the dynamic changes of the target, thereby improving the overall accuracy and robustness of target tracking.

[0105] Step 306: In the target tracking model, determine the confidence level of each candidate bounding box based on the second Gaussian Wasserstein distance, and determine the target bounding box corresponding to the target in the current frame image based on the confidence level of each candidate bounding box.

[0106] Typically, when a target tracking model predicts the bounding box corresponding to the target in each frame, it outputs a set of predicted bounding boxes, namely the multiple candidate bounding boxes mentioned above. It also outputs the confidence score of each candidate bounding box, which represents the target tracking model's confidence in the predicted candidate bounding box. To further improve the stability of target tracking, traditional methods generally adjust the confidence scores of each candidate bounding box in the current frame based on the area ratio and aspect ratio between the predicted bounding box in the current frame and the target's bounding box in the previous frame. However, the confidence scores determined in this way may still not be accurate enough.

[0107] Based on this, in this step, after obtaining the second Gaussian Wasserstein distance for each candidate bounding box through normalization, as an optional embodiment, a first ratio between the area of ​​each candidate bounding box and the area of ​​the first bounding box can be calculated, and a second ratio between the aspect ratio of each candidate bounding box and the aspect ratio of the first bounding box can be calculated; based on the first ratio, the second ratio, and the second Gaussian Wasserstein distance corresponding to each candidate bounding box, the confidence level corresponding to each candidate bounding box is determined.

[0108] Specifically, the confidence score of each candidate bounding box can be calculated using the following formula:

[0109] .

[0110] in, This represents the confidence level of each candidate bounding box; This represents the initial confidence level for each candidate bounding box, which can be obtained based on the classification results and quality assessment results of the candidate bounding boxes. and These represent the first ratio between the area of ​​the candidate bounding box and the area of ​​the first bounding box, and the second ratio between the aspect ratio of the candidate bounding box and the aspect ratio of the first bounding box, respectively, with values ​​ranging from 0 to 1.

[0111] After obtaining the confidence scores of each predicted candidate bounding box in the current frame image using the Gaussian Wasserstein distance, the confidence scores of each candidate bounding box can be sorted, and the candidate bounding box with the highest confidence score can be output as the target bounding box in the current frame image. In other words, the target tracking model finally outputs the adjusted confidence score and the predicted target bounding box, and uses this result as the initial state for target tracking in the next frame image.

[0112] In summary, this embodiment incorporates the Gaussian Wasserstein distance between each candidate bounding box and the target bounding box in the previous frame when adjusting the confidence of each candidate bounding box in the current frame image. The Gaussian Wasserstein distance can better measure the difference between two bounding boxes, thereby improving the confidence of the finally determined candidate bounding boxes, that is, improving the accuracy of the confidence of the candidate bounding boxes.

[0113] Step 308: Determine the tracking result corresponding to the target based on the target bounding box in the current frame image.

[0114] In this step, after obtaining the target bounding box corresponding to the target in the current frame image, the position information of the target bounding box and its confidence level can also be obtained. Then, all of this information can be used as the tracking result of the target in the current frame image.

[0115] In this embodiment, by introducing the Gaussian Wasserstein distance between the bounding boxes of two frames when adjusting the confidence of the bounding boxes predicted by the target tracking model, the confidence of the finally determined candidate bounding boxes can be improved, i.e., the accuracy of the candidate bounding box confidence can be improved, thereby improving the accuracy and robustness of target tracking. Furthermore, by normalizing the Gaussian Wasserstein distance between the bounding boxes of two frames, the target tracking model can take into account the dynamic changes of the target, thereby improving the overall accuracy and robustness of target tracking. Further, the confidence of the predicted bounding box can be determined by combining the Gaussian Wasserstein distance between the bounding boxes of two frames with the area ratio and aspect ratio. This allows for multi-dimensional calculation of the confidence of the predicted bounding box, resulting in higher accuracy and stronger robustness of the finally determined bounding box confidence, thus further improving the accuracy and robustness of target tracking.

[0116] As described in the above embodiments, the present invention introduces Gaussian Wasserstein distance-optimized regression loss during the training process of the target tracking model, and simultaneously introduces the confidence of the bounding box predicted by Gaussian Wasserstein distance optimization during the prediction process of the target tracking model. This effectively reduces the sensitivity of the regression loss under the traditional IOU method to small target size, and significantly improves the accuracy of the confidence of the determined bounding box during the testing phase. This enables more efficient tracking of small targets in satellite video tracking tasks, achieving higher tracking accuracy and robustness.

[0117] Furthermore, in this embodiment of the invention, the target tracking model of this embodiment (denoted as GWD-SiamFC++) was compared with three trackers—the traditional SiamFC++ tracking algorithm and the SiamMDM tracking algorithm—using the SatSOT satellite video dataset in terms of tracking success rate, accuracy, and tracking speed. The comparison results are shown in the table below:

[0118]

[0119] SiamMDM is used as a comparison satellite video target tracking algorithm. Accuracy metrics include precision and success rate. Precision is measured by the proportion of frames whose deviation from the center point coordinates of the predicted bounding box is within a certain threshold. Success rate is determined by calculating the proportion of successfully tracked frames at different Intersection over Union (IOU) thresholds, and further by calculating the area under the ROC curve (AUC). Speed ​​metrics include tracking speed, which can be evaluated by calculating the average frame rate (FPS, Frames Per Second).

[0120] As can be seen from the table above, the target tracking model GWD-SiamFC++ of this invention has significant advantages in satellite video small target tracking tasks. Compared with the baseline model SiamFC++, the tracking success rate and accuracy are improved by 3.1% and 6.3%, respectively. Furthermore, the target tracking model GWD-SiamFC++ of this invention achieves a tracking speed of 208.8 FPS, comparable to the baseline model SiamFC++. In other words, the module integrated into the target tracking model GWD-SiamFC++ to calculate the Gaussian Wasserstein distance to optimize the regression loss and confidence does not affect the inference speed, thus enabling efficient tracking of small satellite targets.

[0121] Based on the table above and the embodiments described above, it can be further concluded that the beneficial effects of this embodiment include:

[0122] 1. By employing a novel bounding box similarity measurement strategy based on Gaussian Wasserstein distance, the geometric and statistical characteristics of targets can be captured more accurately. Compared to traditional measurement methods (such as Intersection over Union (IOU), the target tracking method in this invention can not only accurately characterize the weight distribution of pixels inside the bounding box, but also possess scale invariance, is insensitive to changes in target size and position, and can effectively distinguish between non-overlapping or completely contained bounding boxes. It is particularly suitable for scenarios with large target scale variations, complex shapes, and significant background interference, such as satellite video small target tracking tasks.

[0123] 2. By using a nonlinear transformation technique to normalize the calculated Gaussian Wasserstein distance (GWD), GWD is converted into a normalized distance (TrGWD and TeGWD) suitable for training and testing. This enhances the adaptability of GWD, enabling it to be quickly applied to various target tracking algorithms and greatly improving the versatility and adaptability of the target tracking model.

[0124] The present invention provides a satellite video target tracking device, which can be referred to in correspondence with the satellite video target tracking method described above.

[0125] Figure 6 This is a schematic diagram of the structure of a satellite video target tracking device provided by the present invention. See also: Figure 6 As shown, the device may include:

[0126] The acquisition module 410 is used to acquire the current frame image in the satellite video; the current frame image includes the target.

[0127] Tracking module 420 is used to input the current frame image into a preset target tracking model for target tracking processing, and determine the tracking result corresponding to the target in the current frame image; the tracking result includes the target bounding box corresponding to the target in the current frame image;

[0128] The target tracking model is trained using multiple satellite training images, the true bounding boxes of sample objects in each satellite training image, and the regression loss between the corresponding predicted bounding boxes. The regression loss is determined based on the first Gaussian Wasserstein distance between the two-dimensional Gaussian distributions of the true bounding boxes and the corresponding predicted bounding boxes. In the two-dimensional Gaussian distribution corresponding to the true bounding boxes, the pixel density in the center region of the true bounding box is greater than the pixel density in the edge region.

[0129] In some embodiments, the above-described apparatus may further include a training module, which is configured to input satellite training images into an initial target tracking model for target tracking processing; determine the predicted bounding boxes corresponding to sample objects in the satellite training images and the position information of the predicted bounding boxes; determine the two-dimensional Gaussian distribution corresponding to the real bounding boxes based on the position information of the real bounding boxes, and determine the two-dimensional Gaussian distribution corresponding to the predicted bounding boxes based on the position information of the predicted bounding boxes; calculate the first Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the real bounding boxes and the two-dimensional Gaussian distribution of the predicted bounding boxes, and determine the regression loss between the real bounding boxes and the predicted bounding boxes based on the first Gaussian Wasserstein distance; and train the initial target tracking model based on the regression loss to determine the target tracking model.

[0130] Optionally, the training module described above is specifically used to calculate the Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the true bounding box and the two-dimensional Gaussian distribution of the predicted bounding box; normalize the calculated Gaussian Wasserstein distance using a first normalization formula to determine the first Gaussian Wasserstein distance; and calculate the difference between 1 and the first Gaussian Wasserstein distance to determine the regression loss between the true bounding box and the predicted bounding box.

[0131] Optionally, the location information of the real bounding box includes the coordinates of the center point of the real bounding box and the width and height of the real bounding box. The training module is specifically used to determine the mean based on the coordinates of the center point of the real bounding box and the variance based on the width and height of the real bounding box; and to determine the two-dimensional Gaussian distribution corresponding to the real bounding box based on the mean and variance.

[0132] In some embodiments, the tracking module 420 is specifically configured to input the current frame image into a preset target tracking model for target tracking processing, determine multiple candidate bounding boxes corresponding to the target in the current frame image; obtain the first bounding box corresponding to the previous frame image of the current frame image in the target tracking model, and calculate the second Gaussian Wasserstein distance between each candidate bounding box and the first bounding box; determine the confidence level corresponding to each candidate bounding box in the target tracking model based on the second Gaussian Wasserstein distance corresponding to each candidate bounding box, and determine the target bounding box corresponding to the target in the current frame image based on the confidence level corresponding to each candidate bounding box; and determine the tracking result corresponding to the target based on the target bounding box corresponding to the target in the current frame image.

[0133] Optionally, the tracking module 420 is specifically used to calculate the first ratio between the area of ​​each candidate bounding box and the area of ​​the first bounding box, and to calculate the second ratio between the aspect ratio of each candidate bounding box and the aspect ratio of the first bounding box; and to determine the confidence level of each candidate bounding box based on the first ratio, the second ratio and the second Gaussian Wasserstein distance.

[0134] Optionally, the tracking module 420 is specifically used to calculate the initial second Gaussian Wasserstein distance between each candidate bounding box and the first bounding box; and to normalize the initial second Gaussian Wasserstein distance of each candidate bounding box using a second normalization formula to determine the second Gaussian Wasserstein distance corresponding to each candidate bounding box.

[0135] It should be noted that the apparatus provided in this embodiment of the invention can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0136] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7 As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communications bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other through the communications bus 540. The processor 510 can call logic instructions in the memory 530 to execute a satellite video target tracking method. The method includes: acquiring a current frame image in a satellite video; the current frame image includes a target; inputting the current frame image into a preset target tracking model for target tracking processing, and determining the tracking result corresponding to the target in the current frame image; the tracking result includes the target bounding box corresponding to the target in the current frame image; wherein the target tracking model is trained based on a regression loss between multiple satellite training images, the true bounding boxes of sample objects in each satellite training image, and the corresponding predicted bounding boxes, and the regression loss is determined based on the first Gaussian-Wasserstein distance between the two-dimensional Gaussian distributions of the true bounding boxes and the corresponding predicted bounding boxes of the sample objects, wherein the density of pixels in the central region of the true bounding box in the two-dimensional Gaussian distribution corresponding to the true bounding box is greater than the density of pixels in the edge region.

[0137] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0138] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute a satellite video target tracking method provided by the above methods. The method includes: acquiring a current frame image in a satellite video; the current frame image includes a target; inputting the current frame image into a preset target tracking model for target tracking processing, and determining the tracking result corresponding to the target in the current frame image; the tracking result includes the target bounding box corresponding to the target in the current frame image; wherein the target tracking model is trained based on a regression loss between multiple satellite training images, the true bounding boxes of sample objects in each satellite training image, and the corresponding predicted bounding boxes. The regression loss is determined based on the first Gaussian-Wasserstein distance between the two-dimensional Gaussian distributions of the true bounding boxes and the corresponding predicted bounding boxes of the sample objects, wherein the density of pixels in the central region of the true bounding box in the two-dimensional Gaussian distribution corresponding to the true bounding box is greater than the density of pixels in the edge region.

[0139] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a satellite video target tracking method provided by the methods described above. The method includes: acquiring a current frame image from a satellite video; the current frame image including a target; inputting the current frame image into a preset target tracking model for target tracking processing, and determining a tracking result corresponding to the target in the current frame image; the tracking result including a target bounding box corresponding to the target in the current frame image; wherein the target tracking model is trained based on a regression loss between multiple satellite training images, the true bounding boxes of sample objects in each satellite training image, and the corresponding predicted bounding boxes, and the regression loss is determined based on the first Gaussian-Wasserstein distance between the two-dimensional Gaussian distributions of the true bounding boxes and the corresponding predicted bounding boxes of the sample objects, wherein the pixel density in the central region of the true bounding box in the two-dimensional Gaussian distribution corresponding to the true bounding box is greater than the pixel density in the edge region.

[0140] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0141] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A satellite video target tracking method, characterized in that, include: Acquire the current frame image from satellite video; The current frame image includes the target; The current frame image is input into a preset target tracking model for target tracking processing to determine the tracking result corresponding to the target in the current frame image; The tracking result includes the target bounding box corresponding to the target in the current frame image; The target tracking model is trained based on multiple satellite training images, the true bounding boxes of sample objects in each satellite training image, and the regression loss between the corresponding predicted bounding boxes. The regression loss is determined based on the first Gaussian-Wasserstein distance between the two-dimensional Gaussian distributions of the true bounding boxes and the corresponding predicted bounding boxes of the sample objects. The density of pixels in the center region of the true bounding box in the two-dimensional Gaussian distribution corresponding to the true bounding box is greater than the density of pixels in the edge region. The step of inputting the current frame image into a preset target tracking model for target tracking processing, and determining the tracking result corresponding to the target in the current frame image, includes: The current frame image is input into a preset target tracking model for target tracking processing to determine multiple candidate bounding boxes corresponding to the target in the current frame image and the initial confidence of each candidate bounding box; In the target tracking model, the first bounding box corresponding to the previous frame image of the current frame image is obtained, and the second Gaussian Wasserstein distance between each candidate bounding box and the first bounding box is calculated. In the target tracking model, the confidence level of each candidate bounding box is determined based on the second Gaussian Wasserstein distance and the initial confidence level of each candidate bounding box, and the target bounding box corresponding to the target in the current frame image is determined based on the confidence level of each candidate bounding box. The tracking result corresponding to the target is determined based on the target bounding box corresponding to the target in the current frame image.

2. The satellite video target tracking method according to claim 1, characterized in that, The true bounding box includes the position information of the true bounding box, and the training method of the target tracking model includes: The satellite training image is input into the initial target tracking model for target tracking processing to determine the predicted bounding box corresponding to the sample object in the satellite training image and the position information of the predicted bounding box. The two-dimensional Gaussian distribution corresponding to the true bounding box is determined based on the location information of the true bounding box, and the two-dimensional Gaussian distribution corresponding to the predicted bounding box is determined based on the location information of the predicted bounding box. Calculate the first Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the true bounding box and the two-dimensional Gaussian distribution of the predicted bounding box, and determine the regression loss between the true bounding box and the predicted bounding box based on the first Gaussian Wasserstein distance; The initial target tracking model is trained based on the regression loss to determine the target tracking model.

3. The satellite video target tracking method according to claim 2, characterized in that, The step of calculating the first Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the true bounding box and the two-dimensional Gaussian distribution of the predicted bounding box, and determining the regression loss between the true bounding box and the predicted bounding box based on the first Gaussian Wasserstein distance, includes: Calculate the Gaussian Wasserstein distance between the two-dimensional Gaussian distribution of the true bounding box and the two-dimensional Gaussian distribution of the predicted bounding box; The calculated Gaussian Wasserstein distance is normalized using the first normalization formula to determine the first Gaussian Wasserstein distance; The regression loss between the true bounding box and the predicted bounding box is determined by subtracting 1 from the first Gaussian Wasserstein distance.

4. The satellite video target tracking method according to claim 2, characterized in that, The location information of the true bounding box includes the coordinates of the center point of the true bounding box and the width and height of the true bounding box. Determining the two-dimensional Gaussian distribution corresponding to the true bounding box based on the location information includes: The mean is determined based on the center point coordinates of the true bounding box, and the variance is determined based on the width and height of the true bounding box. The two-dimensional Gaussian distribution corresponding to the true bounding box is determined based on the mean and the variance.

5. The satellite video target tracking method according to claim 1, characterized in that, The step of determining the confidence level of each candidate bounding box based on the second Gaussian Wasserstein distance corresponding to each candidate bounding box includes: Calculate a first ratio between the area of ​​each candidate bounding box and the area of ​​the first bounding box, and calculate a second ratio between the aspect ratio of each candidate bounding box and the aspect ratio of the first bounding box. The confidence level of each candidate bounding box is determined based on the first proportion, the second proportion, and the second Gaussian Wasserstein distance.

6. The satellite video target tracking method according to claim 1, characterized in that, The calculation of the second Gaussian Wasserstein distance between each of the candidate bounding boxes and the first bounding box includes: Calculate the initial second Gaussian Wasserstein distance between each of the candidate bounding boxes and the first bounding box; The initial second Gaussian Wasserstein distance of each candidate bounding box is normalized using a second normalization formula to determine the second Gaussian Wasserstein distance corresponding to each candidate bounding box.

7. A satellite video target tracking device, characterized in that, include: The acquisition module is used to acquire the current frame image from satellite video; The current frame image includes the target; The tracking module is used to input the current frame image into a preset target tracking model for target tracking processing, and to determine the tracking result corresponding to the target in the current frame image; The tracking result includes the target bounding box corresponding to the target in the current frame image; The target tracking model is trained based on multiple satellite training images, the true bounding boxes of sample objects in each satellite training image, and the regression loss between the corresponding predicted bounding boxes. The regression loss is determined based on the first Gaussian-Wasserstein distance between the two-dimensional Gaussian distributions of the true bounding boxes and the corresponding predicted bounding boxes of the sample objects. The density of pixels in the center region of the true bounding box in the two-dimensional Gaussian distribution corresponding to the true bounding box is greater than the density of pixels in the edge region. The tracking module is specifically configured to input the current frame image into a preset target tracking model for target tracking processing, determine multiple candidate bounding boxes corresponding to the target in the current frame image and the initial confidence of each candidate bounding box; obtain the first bounding box corresponding to the previous frame image of the current frame image in the target tracking model, and calculate the second Gaussian Wasserstein distance between each candidate bounding box and the first bounding box; determine the confidence of each candidate bounding box in the target tracking model based on the second Gaussian Wasserstein distance and the initial confidence of each candidate bounding box, and determine the target bounding box corresponding to the target in the current frame image based on the confidence of each candidate bounding box; and determine the tracking result corresponding to the target based on the target bounding box corresponding to the target in the current frame image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements a satellite video target tracking method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a satellite video target tracking method as described in any one of claims 1 to 6.

Images