Satellite video target tracking method, device and equipment based on refined positioning

Abstract

The present disclosure provides a satellite video target tracking method, device, equipment and medium based on refined positioning, including: acquiring a satellite video image, and extracting feature images of a template area and a search area in the satellite video image by using a pre-trained target tracking model; performing background suppression on the feature images of the template area and the search area respectively by using the target tracking model to obtain a template feature map and a search feature map after background suppression; performing a matching operation between the template feature map and the search feature map to obtain a response map; based on the response map, using an enhanced head network in the target tracking model to locate the target in the satellite video image.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing target tracking or visible light remote sensing image processing, and in particular to a satellite video target tracking method, device, equipment and storage medium based on refined positioning. Background Art

[0002] Satellite video target tracking has wide application value in military and civilian technical fields such as key target detection and port management. However, due to the orbital altitude of the satellite, the resolution of the imaging equipment and the actual size of the target, the targets in the satellite video are mostly small targets with weak features and lack of obvious appearance texture and geometric structure information. In addition, the targets in the satellite video images are usually in a complex background, and similar targets and noise clutter in the background will also have a direct impact on target tracking. The current field of satellite video single target tracking consists of two types of mainstream methods, namely the traditional target tracking algorithm based on correlation filtering and the deep learning target tracking method based on convolutional neural network. However, the deep learning target tracking method is designed for general optical targets and directly uses conventional classification heads and regression heads to locate the targets in the image. When locating small targets in satellite video images, it is easy to have problems such as inaccurate center positioning and loose prediction bounding box, which directly affects the final positioning effect for small targets. At the same time, the existing target tracking network based on deep learning does not fully consider the impact of complex background on the extraction of the main target. Summary of the invention

[0003] 1. Technical issues to be resolved

[0004] In view of the above problems, the present disclosure provides a satellite video target tracking method, device, equipment and storage medium based on refined positioning, so as to at least partially solve the technical problems existing in traditional satellite video target tracking methods, such as weak small target tracking and positioning capability, inaccurate center positioning, loose predicted bounding box and background interference.

[0005] (II) Technical solution

[0006] The present disclosure provides a satellite video target tracking method based on refined positioning, comprising: acquiring a satellite video image, and extracting feature images of a template area and a search area in the satellite video image by using a pre-trained target tracking model; performing background suppression on the feature images of the template area and the search area respectively by using the target tracking model to obtain a template feature map and a search feature map after background suppression; performing a matching operation between the template feature map and the search feature map to obtain a response map; and based on the response map, using an enhanced head network in the target tracking model to locate a target in the satellite video image.

[0007] According to an embodiment of the present disclosure, the target tracking model is used to perform background suppression on the feature images of the template area and the search area respectively to obtain the template feature map and the search feature map after background suppression, including: using the target tracking model to identify target feature information in the feature images of the template area and the search area; generating a filter based on the target feature information; filtering the feature images of the template area and the search area by the filter to suppress the background information of the feature images, and obtain the template feature map and the search feature map.

[0008] According to an embodiment of the present disclosure, the use of the target tracking model to identify target feature information in the feature images of the template area and the search area includes: inputting the feature image of the template area or the search area into the twin network of the target tracking model to obtain the original feature map of the template area or the search area; using a pre-trained linear learnable parameter matrix to transform and project the original feature map, and calculating a probability distribution vector based on the transformed original feature map, wherein the probability distribution vector is used to characterize the importance of each pixel feature vector in the original feature map; based on the probability distribution vector, calculating the target feature information of the original feature map.

[0009] According to an embodiment of the present disclosure, the classification branch of the enhanced head network includes multiple attention heads, each of which includes at least a graph generator and a graph updater; based on the response graph, using the enhanced head network in the target tracking model to locate the target in the satellite video image includes: based on the response graph, generating a first classification feature graph; using the graph generator to generate vertex information and edge information of the first classification feature graph; using the graph updater to update the vertex information and edge information to obtain updated vertex information and edge information; integrating the updated vertex information and edge information, and connecting them with the first classification feature graph to obtain a refined feature graph; aggregating the refined feature graphs obtained by each of the attention heads to obtain a second classification feature graph.

[0010] According to an embodiment of the present disclosure, the positioning of the target in the satellite video image based on the response map and using the enhanced head network in the target tracking model also includes: generating a first regression feature map based on the response map, and predicting an initial bounding box based on the first regression feature map; using the progressive regression mechanism introduced by the regression branch of the enhanced head network to capture the geometry and contextual information of the initial bounding box, and learning the offset of the first regression feature map in different directions; applying the offset to the first regression feature map to generate a second regression feature map.

[0011] According to an embodiment of the present disclosure, applying the offset to the first regression feature map to generate the second regression feature map includes: integrating the offset to form the offset matrix, and multiplying the offset matrix by the first regression feature map pixel by pixel to obtain the second regression feature map.

[0012] According to an embodiment of the present disclosure, the positioning of the target in the satellite video image based on the response map and using the enhanced head network in the target tracking model also includes: fusing the second classification feature map with the second regression feature map to obtain a fused feature map; adjusting the initial bounding box based on the second regression feature map, and positioning the target in the fused feature map using the adjusted initial bounding box; performing category recognition on the positioned target based on the second classification feature map; and outputting the positioned and recognized target and the corresponding adjusted bounding box.

[0013] The second aspect of the present disclosure provides a satellite video target tracking device based on refined positioning, including: an extraction module, used to obtain a satellite video image, and use a pre-trained target tracking model to extract feature images of a template area and a search area in the satellite video image; a suppression module, used to perform background suppression on the feature images of the template area and the search area respectively through the target tracking model, and obtain a template feature map and a search feature map after background suppression; a matching module, used to perform a matching operation between the template feature map and the search feature map to obtain a response map; and a positioning module, used to locate the target in the satellite video image based on the response map and using an enhanced head network in the target tracking model.

[0014] The third aspect of the present disclosure provides an electronic device, comprising: one or more processors; a storage device for storing one or more computer programs, which, when executed by the one or more processors, enables the one or more processors to execute the satellite video target tracking method based on refined positioning.

[0015] The fourth aspect of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the satellite video target tracking method based on refined positioning.

[0016] (III) Beneficial effects

[0017] The satellite video target tracking method, device, equipment and storage medium based on refined positioning provided by the present disclosure suppress the background of the feature images of the template area and the search area, that is, suppress irrelevant information in the background and enhance the target-related information, thereby enhancing the distinction between the target features and background interference. At the same time, multiple graph network attention heads based on the classification branch of the enhancement head network achieve more accurate tracking of weak targets in satellite videos. In addition, the initial bounding box of the original prediction is fine-tuned using a progressive regression mechanism, so that the final target bounding box has a tighter frame for the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] For a more complete understanding of the present disclosure and its advantages, reference will now be made to the following description taken in conjunction with the accompanying drawings, in which:

[0019] Figure 1 The overall process diagram of the SiamCAR target tracking network provided by the embodiment of the present disclosure is schematically shown;

[0020] Figure 2 The flowchart of the satellite video target tracking method based on refined positioning provided by the embodiment of the present disclosure is schematically shown;

[0021] Figure 3 The overall algorithm structure diagram of the target tracking model provided by the embodiment of the present disclosure is schematically shown;

[0022] Figure 4 The schematic diagram of the structure of the background suppression module provided by the embodiment of the present disclosure is shown;

[0023] Figure 5 The schematic diagram of the structure of the multi-head graph attention refinement module provided by the embodiment of the present disclosure is shown;

[0024] Figure 6 The structure block diagram of the satellite video target tracking device based on refined positioning according to the embodiment of the present disclosure is schematically shown;

[0025] Figure 7 The hardware structure diagram of an electronic device is schematically shown. DETAILED DESCRIPTION

[0026] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0027] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0028] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0029] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some blocks or combinations thereof in the block diagrams and / or flow charts may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that these instructions, when executed by the processor, may create a device for implementing the functions / operations described in these block diagrams and / or flow charts.

[0030] Therefore, the technology of the present disclosure can be implemented in the form of hardware and / or software (including firmware, microcode, etc.). In addition, the technology of the present disclosure can take the form of a computer program product on a computer-readable medium storing instructions, which can be used by an instruction execution system or in combination with an instruction execution system. In the context of the present disclosure, a computer-readable medium can be any medium that can contain, store, transmit, propagate, or transmit instructions. For example, a computer-readable medium can include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, device, or propagation medium. Specific examples of computer-readable media include: magnetic storage devices, such as magnetic tape or hard disk (HDD); optical storage devices, such as compact disk (CD-ROM); memory, such as random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0031] The inventor has found through research that in the relevant technology, accurate positioning and tracking of weak targets in satellite videos in complex scenes is a key issue in the field of satellite video target tracking, which is challenging. The traditional target algorithm based on correlation filtering in the current field of single target tracking in satellite videos is mainly based on the Kalman filter. It uses the target's historical trajectory information or motion information for modeling to enhance the target tracking framework's ability to continuously track the target and the stability of the framework when dealing with challenges such as interference or occlusion. The target tracking algorithm based on deep learning mainly models the features of the template area and the search area respectively through the twin network architecture to obtain the feature maps of the template area and the search area. The features between the two are then matched, and the matched feature response map is sent to the head network for classification and regression to obtain the final target position prediction result.

[0032] Figure 1 The overall process diagram of the SiamCAR target tracking network provided by the embodiment of the present disclosure is schematically shown.

[0033] like Figure 1As shown in the figure, representative target tracking methods based on deep learning (such as SiamCAR) adopt an end-to-end training method, which enables the network to optimize classification and regression tasks simultaneously. It adopts a twin network structure to extract image features from the input template area image and search area image respectively, where the template area image is a 127×127 image block containing the target in the initial frame, and the search area image is a 255×255 image block centered at the center point of the prediction box of the previous frame in the current frame. The target image and the search image are respectively extracted by two convolutional neural networks (CNNs) with shared parameters. Subsequently, the two extracted feature maps are subjected to deep cross-correlation operations to generate a response map. Based on the generated response map, the head network of SiamCAR adopts a classification branch, a regression branch, and an additional center branch. The classification branch distinguishes foreground information from background information; the regression branch predicts the precise position and scale of the target in the search image, that is, the bounding box of the target; and the center branch outputs. The network integrates the results of each branch to obtain the final positioning of the target in the current frame. The SiamCAR target tracking algorithm has achieved excellent results in general video target tracking. However, there are still some problems when this method is applied to satellite video tracking of small targets. On the one hand, unlike the target size features in general target tracking, the target size in satellite video images is small, and the identifiable features contained are weak, which directly affects the final positioning effect for small targets, resulting in SiamCAR's positioning ability for small target tracking still has a lot of room for improvement; on the other hand, in the process of extracting target features in the image by the SiamCAR method, there is a large amount of interference noise in the background of the satellite video target, as well as background interference objects similar to the target, that is, the background of the satellite video target is usually very noisy, and excessive background noise affects the network's accurate judgment of the target position. After observing the above problems, the embodiment of the present disclosure proposes a satellite video target tracking method based on refined positioning on the basis of the SiamCAR target tracking method, aiming to solve the above problems.

[0034] like Figure 2 As shown, the flowchart of the satellite video target tracking method based on refined positioning includes S1~S4.

[0035] In operation S1, a satellite video image is acquired, and a pre-trained target tracking model is used to extract feature images of a template area and a search area in the satellite video image.

[0036] In operation S2, background suppression is performed on the feature images of the template area and the search area respectively through the target tracking model to obtain a template feature map and a search feature map after background suppression.

[0037] In operation S3, a matching operation is performed between the template feature map and the search feature map to obtain a response map.

[0038] In operation S4, based on the response map, the enhanced head network in the target tracking model is used to locate the target in the satellite video image.

[0039] Specifically, the overall algorithm structure of the target tracking model provided by the embodiment of the present disclosure is as follows: Figure 3 As shown in the figure, based on the SiamCAR target tracking algorithm, the feature modeling and target positioning are strengthened accordingly. The strengthened algorithm mainly includes a background suppression module, a multi-head image attention refinement module of the classification branch, and a progressive regression mechanism of the regression branch. In some exemplary embodiments, the target tracking model first extracts the feature images of the template area and the search area in the satellite video image respectively. Subsequently, the background suppression module further processes the extracted feature images to obtain a feature map pair after background suppression. Then, a matching operation is performed between the template area feature map (template feature map) and the search area feature map (search feature map) to obtain a response map. Finally, the response map is sent to the enhanced tracking head network (i.e., the enhanced head network) for target positioning.

[0040] It can be understood that the satellite video target tracking method based on refined positioning provided by the embodiment of the present disclosure, based on the SiamCAR target tracking network, combines the characteristics of the satellite video target, and proposes a new target tracking algorithm based on the enhanced head network and background suppression, which enhances the ability to distinguish weak targets in complex backgrounds and achieves better satellite video target tracking performance. On the one hand, different from the conventional head network used in the SiamCAR target tracking algorithm, the embodiment of the present disclosure improves the head network of the general target tracking algorithm and proposes an enhanced head network, which respectively enhances the classification branch and the regression branch in the head network through the multi-head graph attention refinement module and the progressive regression mechanism in the overall algorithm structure. On the other hand, since the SiamCAR target tracking algorithm does not specifically design the network for the problem of weak satellite video target features, the embodiment of the present disclosure uses the background suppression module to suppress the background features in the extracted features, thereby enhancing the key feature information of the target itself.

[0041] Furthermore, the method of performing background suppression on the feature images of the template area and the search area respectively through the target tracking model to obtain the template feature map and the search feature map after background suppression includes: using the target tracking model to identify target feature information in the feature images of the template area and the search area; generating a filter based on the target feature information; filtering the feature images of the template area and the search area through the filter to suppress the background information of the feature images, and obtaining the template feature map and the search feature map.

[0042] Specifically, the structure diagram of the background suppression module is as follows: Figure 4 As shown in the figure, the background suppression module uses a specific network structure to identify the most important part of the feature image in the template area and the search area, namely the target feature information, and uses it as a filter to filter the original feature map. When the information in the original feature map is consistent with the information in the filter, this part of information (i.e., target information) will be enhanced or retained; when the information in the original feature map is inconsistent with the information in the filter, this part of information (i.e., background information) will be removed or weakened.

[0043] It can be understood that by using filters for background suppression, it is possible to suppress cluttered background interference and highlight the key parts of the original feature map, thereby facilitating the classification and regression of the subsequent head network.

[0044] Furthermore, the use of the target tracking model to identify target feature information in the feature images of the template area and the search area includes: inputting the feature image of the template area or the search area into the twin network of the target tracking model to obtain an original feature map of the template area or the search area; using a pre-trained linear learnable parameter matrix to transform and project the original feature map, and calculating a probability distribution vector based on the transformed original feature map, wherein the probability distribution vector is used to characterize the importance of each pixel feature vector in the original feature map; based on the probability distribution vector, calculating the target feature information of the original feature map.

[0045] Continue to refer to Figure 4 In some exemplary embodiments, first, a feature image of a given template area (template image) or a feature image of a search area (search image) is input into a twin network to obtain an extracted original feature map. . Then, using the linear learnable parameter matrix W e For the original feature map X input Transform and project, and then input the result into the Softmax function to get the probability distribution vector The expression of this process is as follows:

[0046]

[0047] Where σ represents the Softmax activation function. input Any pixel feature vector in The formal expression of the above process is

[0048]

[0049] Used to measure the input original feature map The importance of each pixel feature vector in . Then use Process the input original feature map, the processing steps are as follows.

[0050]

[0051] In the above formula, Represents the most important feature in the original feature map, that is, the target feature information.

[0052] Furthermore, X key It is used as a filter to filter the original input feature map along the channel dimension. The specific formula is as follows:

[0053]

[0054] Where a is a constant hyperparameter used to tune X input and X key The ratio in the result feature map (template feature map and search feature map) Y.

[0055] It should be noted that the LeakyReLU function can be used to adjust X key The value of .

[0056] It can be understood that by suppressing the background information in the original feature map and retaining the feature information of the key part of the target, the distinction between the target feature and the background interference is enhanced, which helps the precise positioning of the subsequent head network. In addition, the only parameter that needs to be learned in the background suppression module is the linear learnable parameter matrix, which ensures that the module can achieve effective background suppression without the need for complex structures and learning strategies.

[0057] In an embodiment of the present disclosure, the classification branch of the enhanced head network includes multiple attention heads, each of which includes at least a graph generator and a graph updater; based on the response graph, using the enhanced head network in the target tracking model to locate the target in the satellite video image includes: based on the response graph, generating a first classification feature graph; using the graph generator to generate vertex information and edge information of the first classification feature graph; using the graph updater to update the vertex information and edge information to obtain updated vertex information and edge information; integrating the updated vertex information and edge information, and connecting them with the first classification feature graph to obtain a refined feature graph; aggregating the refined feature graphs obtained by each of the attention heads to obtain a second classification feature graph.

[0058] Specifically, in order to further improve the responsiveness of the tracking head network to small targets, the disclosed embodiment proposes a multi-head graph attention refinement module, the specific structure of which is as follows: Figure 5 As shown in Figure 2. The multi-head graph attention refinement module consists of multiple attention heads to create multiple sub-semantic spaces, allowing the object tracking model to focus on information from semantic spaces of different dimensions. For a single attention head, it mainly consists of two parts: a graph generator and a graph updater.

[0059] For example, we first construct a pixel graph P using an adjacency matrix as a key guide for analyzing and optimizing the relationship between pixels. In the pixel graph P, nodes consist of points in the classification feature graph, and edges connect these nodes, highlighting the close relationship between points.

[0060] Given the first classification feature map , the expressions for generating the vertex set V and edge set E in the pixel graph are as follows:

[0061]

[0062] in this case, Represents a graph generator. Vertex set , which contains the information of each vertex in the first classification feature graph. The edge set E is obtained by converting the first classification feature graph R cls The nodes are stacked after circular shift along eight directions (up, down, left, right, upper left, lower left, upper right and lower right). This means that each position initializes the relationship information between the current node and its eight neighboring nodes. cls After constructing the vertex set V and edge set E, we then continue to update them in the second phase. The update formula is as follows:

[0063]

[0064] For a single-head graph attention layer, vertex features and edge features are first concatenated and then updated. The expression of the above operation is as follows:

[0065]

[0066] Linear represents the linear layer, Cat represents the feature connection operation, and BN and ReLU represent the batch normalization layer and nonlinear activation layer respectively. Recorded as , represents a complete linear transformation process.

[0067] After updating the vertex information in the vertex set V, continue to update the edge information in the edge set E in the pixel map P. First, the stacked V features and edges E are connected. The specific formula is as follows:

[0068]

[0069] in This means that the vertices are stacked 8 times to connect with edge E. Represents two consecutive linear transformation processes. The first linear transformation updates the edge information, while the second linear transformation reduces the dimension of the edge feature to obtain the updated edge After completing the update of the vertices and edges in the pixel map P, integrate their features and apply them to the first classification feature map R cls On the top, we get the refined feature map The specific formula is as follows:

[0070]

[0071] Finally, the refined feature maps obtained by each head are aggregated and the aggregated features are averaged to obtain the final second classification feature map .

[0072] It can be understood that in the embodiment of the present disclosure, by adopting a multi-head graph attention mechanism, the sensitivity of a single attention head to noise is reduced, and the target tracking model is allowed to learn and focus on different relationships and patterns in different subspaces, thereby learning diversified feature representations.

[0073] Furthermore, based on the response map, using the enhanced head network in the target tracking model to locate the target in the satellite video image also includes: generating a first regression feature map based on the response map, and predicting an initial bounding box based on the first regression feature map; using the progressive regression mechanism introduced by the regression branch of the enhanced head network to capture the geometry and context information of the initial bounding box, and learning the offset of the first regression feature map in different directions; applying the offset to the first regression feature map to generate a second regression feature map.

[0074] Specifically, in order to enable the tracker to obtain a more compact target bounding box, thereby improving the performance of the tracker, the embodiment of the present disclosure introduces a progressive regression mechanism in the regression branch. The core of the progressive regression mechanism is to further correct the initial bounding box offset based on the first regression feature map initially obtained, thereby generating a more refined second regression feature map.

[0075] Exemplarily, a star-shaped bounding box representation with 9 fixed sampling points is used to capture the geometry and context information of the initial bounding box. The sampling position of the convolution kernel is adjusted by mapping the sampling points to the relative offset (x, y) of the first regression feature map to better capture the deformation characteristics of the target.

[0076] It can be understood that by using the progressive regression mechanism to fine-tune the original predicted initial bounding box, the final target bounding box has a tighter frame for the target.

[0077] Furthermore, applying the offset to the first regression feature map to generate the second regression feature map includes: integrating the offset to form the offset matrix, and multiplying the offset matrix by the first regression feature map pixel by pixel to obtain the second regression feature map.

[0078] Specifically, first, for an initially predicted first regression feature map , sparse dynamic convolution (SDConv) can be used to learn the offsets of the first regression feature map in different directions. Subsequently, the offsets are integrated to form an offset matrix, and the learned offset matrix is ​​multiplied pixel by pixel with the first regression feature map to obtain the final second regression feature map The formal expression of this process is as follows:

[0079]

[0080] Furthermore, based on the response map, using the enhanced head network in the target tracking model to locate the target in the satellite video image also includes: fusing the second classification feature map with the second regression feature map to obtain a fused feature map; based on the second regression feature map, adjusting the initial bounding box, and using the adjusted initial bounding box to locate the target in the fused feature map; based on the second classification feature map, performing category recognition on the located target; and outputting the located and identified target and the corresponding adjusted bounding box.

[0081] Specifically, the second classification feature map is first fused with the second regression feature map, so that the fused fusion feature map contains both the classification information of the target and the precise location information. Then, based on the offset information predicted by the second regression feature map, the initial bounding box is adjusted to obtain a more accurate target location. The adjusted bounding box is then mapped to the fusion feature map for target positioning, and the second classification feature map portion corresponding to the target area is extracted. Based on the extracted second classification feature map portion corresponding to the target area, the probability that the area belongs to each category is calculated, and the category of the final target is determined. Finally, the located and identified target and its corresponding adjusted bounding box are output.

[0082] Illustratively, the satellite video target tracking method based on refined positioning provided by the embodiment of the present disclosure is applied, and the tracking effect obtained by experiment is shown in Table 1.

[0083] Table 1 Tracking effect ablation experiment table

[0084]

[0085] It can be understood that compared to the traditional target tracking algorithm, the disclosed embodiment adds a background suppression module in the feature modeling process to obtain the key target feature information obtained by the feature extraction module, and uses the target feature information as a filter to suppress irrelevant information in the background and enhance the target related information, thereby solving the problem of sensitivity to surrounding background noise in the traditional algorithm. In addition, it can also avoid excessive complexity inside the module, which is more conducive to the training of the algorithm model. As shown in Table 1, by adding a background suppression module, in the overall tracking results of the satellite video target tracking public data set SatSOT, the disclosed embodiment is compared with the SiamCAR target tracking algorithm in terms of the Precision index. Improve 2.3 percentage points, and the Success index is improved by 1.2 percentage points.

[0086] The disclosed embodiment adds a multi-head graph attention refinement module of the classification branch and a progressive regression mechanism of the regression branch to the tracking head network part. In the classification branch, multiple attention heads are used to refine the original first classification feature map. Through this design, the disclosed embodiment can more accurately distinguish the foreground and background of the first classification feature map. In the regression branch, in view of the problem that the satellite video target size is small and a slight target bounding box prediction offset will lead to a significant decrease in tracking performance, the disclosed embodiment introduces a progressive regression mechanism to fine-tune the original predicted target bounding box, so that the final target bounding box has a tighter frame for the target. As shown in Table 1, after adding the multi-head graph attention refinement module and the progressive regression mechanism, the target tracking network's accuracy tracking capability (Precision score) and tracking success rate (Success score) for weak targets in satellite videos are improved by more than 2 percentage points.

[0087] Furthermore, in terms of the overall tracking effect for small targets in satellite videos, the target tracking model proposed in the embodiment of the present disclosure improves the accuracy tracking capability (Precision score) by 5.2 percentage points and the tracking success rate (Success score) by 3.6 percentage points compared to the baseline model SiamCAR, which fully proves that the embodiment of the present disclosure can achieve more accurate tracking and provide a tighter prediction bounding box than SiamCAR.

[0088] Based on the satellite video target tracking method based on refined positioning, the present disclosure also provides a satellite video target tracking device based on refined positioning. Figure 6 The device is described in detail.

[0089] like Figure 6 As shown, the satellite video target tracking device 600 based on refined positioning of this embodiment includes an extraction module 601 , a suppression module 602 , a matching module 603 and a positioning module 604 .

[0090] The extraction module 601 is used to obtain a satellite video image and extract feature images of a template area and a search area in the satellite video image using a pre-trained target tracking model.

[0091] The suppression module 602 is used to perform background suppression on the feature images of the template area and the search area respectively through the target tracking model to obtain a template feature map and a search feature map after background suppression.

[0092] The matching module 603 is used to perform a matching operation between the template feature graph and the search feature graph to obtain a response graph.

[0093] The positioning module 604 is used to locate the target in the satellite video image based on the response graph and using the enhanced head network in the target tracking model.

[0094] It is understandable that the extraction module 601, the suppression module 602, the matching module 603 and the positioning module 604 can be combined in one module, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the extraction module 601, the suppression module 602, the matching module 603 and the positioning module 604 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented in hardware or firmware in any other reasonable way of integrating or packaging the circuit, or in a proper combination of software, hardware and firmware. Alternatively, at least one of the extraction module 601, the suppression module 602, the matching module 603 and the positioning module 604 can be at least partially implemented as a computer program module, and when the program is run by a computer, the function of the corresponding module can be executed.

[0095] Figure 7 The structural block diagram of an electronic device provided by an embodiment of the present disclosure is schematically shown.

[0096] like Figure 7 As shown, the electronic device described in this embodiment includes: the electronic device 700 includes a processor 710 and a computer readable storage medium 720. The electronic device 700 can execute the above reference Figure 2 Describes the method to achieve detection of specific operations.

[0097] Specifically, the processor 710 may include, for example, a general-purpose microprocessor, an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 710 may also include an onboard memory for cache purposes. The processor 710 may be a processor for executing reference Figure 2 The described method flow according to the embodiment of the present disclosure may be a single processing unit or multiple processing units for different actions.

[0098] Computer readable storage medium 720, for example, can be any medium capable of containing, storing, conveying, propagating or transmitting instructions. For example, readable storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices, devices or propagation media. Specific examples of readable storage media include: magnetic storage devices, such as magnetic tapes or hard disks (HDDs); optical storage devices, such as compact disks (CD-ROMs); memory, such as random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0099] The computer readable storage medium 720 may include a computer program 721, which may include code / computer executable instructions, which when executed by the processor 710 causes the processor 710 to perform, for example, the above-mentioned combination Figure 2 The described method sequences and any variations thereof.

[0100] The computer program 721 may be configured to have, for example, computer program codes including computer program modules. For example, in an exemplary embodiment, the codes in the computer program 721 may include one or more program modules, for example, including 721A, module 721B, ... It should be noted that the division method and number of modules are not fixed, and those skilled in the art may use appropriate program modules or program module combinations according to actual conditions. When these program module combinations are executed by the processor 710, the processor 710 may execute, for example, the above combination of Figure 1 to Figure 7 The described method sequences and any variations thereof.

[0101] According to an embodiment of the present invention, at least one of the extraction module 601, the suppression module 602, the matching module 603 and the positioning module 604 can be implemented as a reference Figure 7 The computer program modules described herein, when executed by the processor 710, can implement the corresponding operations described above.

[0102] The present disclosure also provides a computer-readable medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0103] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0104] Although the present disclosure has been shown and described with reference to specific exemplary embodiments of the present disclosure, it should be understood by those skilled in the art that various changes in form and details may be made to the present disclosure without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents. Therefore, the scope of the present disclosure should not be limited to the above-mentioned embodiments, but should be determined not only by the appended claims, but also by the equivalents of the appended claims.

Claims

1. A satellite video target tracking method based on refined positioning, characterized in that: include: Acquire a satellite video image, and extract feature images of a template area and a search area in the satellite video image using a pre-trained target tracking model; Using the target tracking model, background suppression is performed on the feature images of the template area and the search area respectively, so as to obtain a template feature map and a search feature map after background suppression; Performing a matching operation between the template feature graph and the search feature graph to obtain a response graph; Based on the response graph, the target in the satellite video image is located using an enhanced head network in the target tracking model, wherein the classification branch of the enhanced head network includes a plurality of attention heads, and each of the attention heads includes at least a graph generator and a graph updater; The method of locating the target in the satellite video image based on the response graph and using the enhanced head network in the target tracking model includes: generating a first classification feature graph based on the response graph; generating vertex information and edge information of the first classification feature graph by using the graph generator; updating the vertex information and edge information by using the graph updater to obtain updated vertex information and edge information; integrating the updated vertex information and edge information and connecting them with the first classification feature graph to obtain a refined feature graph; and aggregating the refined feature graphs obtained by each of the attention heads to obtain a second classification feature graph.

2. The satellite video target tracking method based on refined positioning according to claim 1 is characterized in that: The method of performing background suppression on the feature images of the template area and the search area respectively by using the target tracking model to obtain the template feature map and the search feature map after background suppression includes: Using the target tracking model, identifying target feature information in the feature images of the template area and the search area; Based on the target feature information, generating a filter; The feature images of the template area and the search area are filtered by the filter to suppress background information of the feature images, thereby obtaining the template feature map and the search feature map.

3. The satellite video target tracking method based on refined positioning according to claim 2 is characterized in that: The using the target tracking model to identify target feature information in the feature images of the template area and the search area includes: Inputting the feature image of the template area or the search area into the twin network of the target tracking model to obtain an original feature map of the template area or the search area; Using a pre-trained linear learnable parameter matrix, transforming and projecting the original feature map, and calculating a probability distribution vector based on the transformed original feature map, wherein the probability distribution vector is used to characterize the importance of each pixel feature vector in the original feature map; Based on the probability distribution vector, target feature information of the original feature map is calculated.

4. The satellite video target tracking method based on refined positioning according to claim 1 is characterized in that: The positioning of the target in the satellite video image based on the response graph and using the enhanced head network in the target tracking model further includes: Based on the response map, generating a first regression feature map, and predicting an initial bounding box according to the first regression feature map; Utilizing a progressive regression mechanism introduced by the regression branch of the enhancement head network to capture the geometry and context information of the initial bounding box and learn offsets of the first regression feature map in different directions; The offset is applied to the first regression feature map to generate a second regression feature map.

5. The satellite video target tracking method based on refined positioning according to claim 4 is characterized in that: Applying the offset to the first regression feature map to generate a second regression feature map includes: The offsets are integrated to form an offset matrix, and the offset matrix is ​​multiplied pixel by pixel by the first regression feature map to obtain the second regression feature map.

6. The satellite video target tracking method based on refined positioning according to claim 4 is characterized in that: The positioning of the target in the satellite video image based on the response graph and using the enhanced head network in the target tracking model further includes: Fusing the second classification feature map with the second regression feature map to obtain a fused feature map; Based on the second regression feature map, adjusting the initial bounding box, and using the adjusted initial bounding box to locate the target in the fusion feature map; Based on the second classification feature map, performing category recognition on the located target; The located and identified target and the corresponding adjusted bounding box are output.

7. A satellite video target tracking device based on refined positioning, characterized in that: The device comprises: An extraction module, used to obtain a satellite video image, and extract a feature image of a template area and a search area in the satellite video image using a pre-trained target tracking model; A suppression module, used to perform background suppression on the feature images of the template area and the search area respectively through the target tracking model to obtain a template feature map and a search feature map after background suppression; A matching module, used for performing a matching operation between the template feature graph and the search feature graph to obtain a response graph; A positioning module is used to locate the target in the satellite video image based on the response graph and using the enhanced head network in the target tracking model, wherein the classification branch of the enhanced head network includes multiple attention heads, and each of the attention heads includes at least a graph generator and a graph updater; the positioning of the target in the satellite video image based on the response graph and using the enhanced head network in the target tracking model includes: generating a first classification feature graph based on the response graph; generating vertex information and edge information of the first classification feature graph using the graph generator; updating the vertex information and edge information using the graph updater to obtain updated vertex information and edge information; integrating the updated vertex information and edge information and connecting them with the first classification feature graph to obtain a refined feature graph; aggregating the refined feature graphs obtained by each of the attention heads to obtain a second classification feature graph.

8. An electronic device, comprising: one or more processors; a storage device for storing one or more computer programs, It is characterized in that when the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having executable instructions stored thereon, characterized in that: When the instruction is executed by a processor, the processor executes the method according to any one of claims 1 to 6.

Images