A method and system for target tracking and prediction of unmanned aerial vehicles (UAVs)

Abstract

This invention provides a method and system for UAV target tracking and prediction, comprising: extracting template features from the target position in the initial frame using an AT-ResNet18 network; extracting features from the target position in any current frame to obtain current frame features and a current frame similarity map; convolving the current frame similarity map and inputting it together with the temporal prior knowledge of the previous frame into AT-Trans for encoding to obtain the current frame temporal prior knowledge; convolving it with the current frame similarity map and inputting it together into AT-Trans for decoding to obtain the current frame enhanced similarity map; classifying and regressing the current frame enhanced similarity map to obtain the final target prediction result for the current frame. This invention incorporates temporal context into the network to dynamically adjust the feature extraction process, effectively solving the target prediction accuracy problem under environments with interference such as target blur, target occlusion, and background interference.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method and system for target tracking and prediction in unmanned aerial vehicles (UAVs). Background Technology

[0002] Unmanned aerial vehicle (UAV) target tracking is an extremely important computer vision task. It aims to track and predict the target's position and state in subsequent frames after providing tracking target information in the initial frame. It is a key technology in the UAV-related industries.

[0003] In recent years, researchers have devoted considerable effort to improving the accuracy and robustness of UAV target tracking. While breakthroughs have been achieved in both constrained and unconstrained environments, the limited power of aerial platforms restricts computational resources, inevitably introducing unique challenges such as motion blur, occlusion, and the inability to guarantee tracking efficiency and performance. These challenges make UAV target tracking tasks still very challenging. The aerial platform prevents the deployment of state-of-the-art methods for UAV target tracking, and the unique challenges of aerial operation further hinder efficient implementation. In recent years, anchor-box-less twin network tracking algorithms have become the mainstream approach for solving a range of UAV target tracking problems. Although these methods have achieved varying degrees of success, they fail to incorporate the inherent temporal context between consecutive frames into the target tracking process, making it difficult to perceive the motion information of the target object. Furthermore, due to limited computational resources, it is difficult to effectively balance target tracking accuracy with limited resources, leading to a decrease in the accuracy and robustness of UAV target tracking.

[0004] Therefore, a new method for UAV target tracking and prediction is needed. Summary of the Invention

[0005] This invention provides a method and system for predicting UAV target tracking, which addresses the shortcomings of existing UAV target tracking technologies that fail to incorporate the inherent temporal context between consecutive frames into target tracking, resulting in insufficient accuracy and robustness in UAV target tracking.

[0006] In a first aspect, the present invention provides a method for unmanned aerial vehicle (UAV) target tracking and prediction, comprising:

[0007] The target position of the initial frame collected by the UAV is obtained, and the initial features are extracted from the target position of the initial frame using the AT-ResNet18 network. The initial features are then used as template features.

[0008] The target location of any current frame collected by the UAV is obtained, and the AT-ResNet18 network is used to extract features from the target location of the current frame to obtain the current frame features. The current frame features are cross-correlated with the template features to obtain the current frame similarity map.

[0009] After convolving the current frame similarity map, it is input into AT-Trans along with the previous frame temporal prior knowledge of any current frame target position for encoding to obtain the current frame temporal prior knowledge.

[0010] The current frame temporal prior knowledge and the current frame similarity map are convolved and then input together into AT-Trans for decoding to obtain the current frame enhanced similarity map;

[0011] The enhanced similarity map of the current frame is classified and regressed to obtain the final target prediction result of the current frame.

[0012] According to the present invention, a UAV target tracking prediction method is provided, wherein the AT-ResNet18 network includes a backbone network composed of a ResNet18 network, the backbone network is used for feature extraction, and the last two convolutional layers of the ResNet18 network are replaced with temporally adaptive convolutional TAdaConv layers.

[0013] The Adam algorithm, which is based on the minimum sample size (mini-batch), is adopted as the optimization strategy for the AT-ResNet18 network.

[0014] According to the present invention, a method for predicting unmanned aerial vehicle (UAV) target tracking replaces the last two convolutional layers of the ResNet18 network with TAdaConv layers, comprising:

[0015] Obtain the input feature X at any current frame target position t. t ;

[0016] For the input feature X t The current frame descriptor is obtained after global average pooling.

[0017]

[0018] GAP represents the global pooling operation;

[0019] Sure Frame descriptor of the first L-1 frames L is the total number of frames;

[0020] The current frame descriptor Frame descriptor of the previous L-1 frame Establish a connection to obtain the time context queue X. p ∈ L×C :

[0021]

[0022] Where Cat represents the concatenation operation, R represents the vector space, and C represents the number of feature channels;

[0023] For the current frame descriptor Perform two one-dimensional convolutions to obtain the temporal convolution kernel. and bias value convolution kernel

[0024]

[0025]

[0026] Where φ w and φ b They represent the generation and One-dimensional convolution operation;

[0027] φ w and φ b The weights are initialized to 0, and the time weight learnable parameter W and the bias value learnable parameter b are obtained.

[0028] By temporal convolution kernel The temporal weights W can be obtained from the learnable parameters W during the convolution process. t :

[0029]

[0030] Convolution kernel with bias value The bias value and the learnable parameter b are used to obtain the bias value b during the convolution process. t :

[0031]

[0032] The time weights W in the convolution process t With the input feature X t Perform convolution, adding the bias value b during the convolution process. t The input feature X is obtained. t Output characteristics

[0033]

[0034] According to the present invention, a method for predicting target tracking of unmanned aerial vehicles (UAVs) is provided, wherein the current frame descriptor is... Frame descriptor of the previous L-1 frame Establish a connection to obtain the time context queue X. p ∈ L×C It also includes:

[0035] If it is determined that the target position t ≤ L-1 in any of the current frames, then the frame descriptor of the first frame is used. For time context queue X p Fill in the blanks.

[0036] According to the UAV target tracking and prediction method provided by the present invention, after convolving the current frame similarity map, it is jointly input into AT-Trans with the previous frame temporal prior knowledge of any current frame target position for encoding to obtain the current frame temporal prior knowledge, including:

[0037] The current frame similarity map R t Convolution is performed to obtain F t , will F t As the key and value of the multi-head attention layer, the prior knowledge of the previous frame's time is used. As a query for the multi-head attention layer, the output of the multi-head attention layer superimposed on frame t is obtained.

[0038]

[0039]

[0040] Where Norm represents layer normalization operation, and MultiHead represents multi-head attention layer operation;

[0041] The output of the first multi-head attention layer Convolution and global average pooling are performed sequentially, and then the calibration factor α is obtained through a feedforward neural network:

[0042]

[0043] Where φ represents a convolutional layer;

[0044] The output of the first multi-head attention layer The output of the multi-head attention layer superimposed on frame t The filtered time information F is obtained by combining the calibration factor α. t f :

[0045]

[0046] According to time information F t fObtain the time prior knowledge of the current t-th frame.

[0047]

[0048] According to the UAV target tracking and prediction method provided by the present invention, the current frame similarity map R is... t Convolution is performed on the initial frame similarity map R0 to obtain the initial frame temporal prior knowledge.

[0049]

[0050] Where φ init This indicates that the initial frame similarity map is convolved.

[0051] According to the UAV target tracking prediction method provided by the present invention, the current frame temporal prior knowledge and the current frame similarity map are convolved and then jointly input into AT-Trans for decoding to obtain the current frame enhanced similarity map, including:

[0052] The time prior knowledge of the current t-th frame Similarity map R with the current frame t The data is sequentially fed into two multi-head attention layers, and then passed through a feedforward neural network to obtain the enhanced similarity map of the current frame.

[0053]

[0054]

[0055]

[0056] in This represents the output of the first multi-head attention layer in the decoder. This represents the output of the second multi-head attention layer in the decoder.

[0057] Secondly, the present invention also provides a UAV target tracking and prediction system, comprising:

[0058] The first extraction module is used to obtain the target position of the initial frame collected by the UAV, extract the initial features from the target position of the initial frame using the AT-ResNet18 network, and use the initial features as template features.

[0059] The second extraction module is used to obtain the target position of any current frame collected by the UAV, extract the features of the target position of the current frame using the AT-ResNet18 network to obtain the current frame features, and perform cross-correlation operation between the current frame features and the template features to obtain the current frame similarity map.

[0060] The encoding module is used to convolve the current frame similarity map and then input it together with the previous frame temporal prior knowledge of any current frame target position into AT-Trans for encoding to obtain the current frame temporal prior knowledge.

[0061] The decoding module is used to convolve the current frame temporal prior knowledge and the current frame similarity map, and then input them together into AT-Trans for decoding to obtain the current frame enhanced similarity map;

[0062] The classification and regression module is used to classify and regress the enhanced similarity map of the current frame to obtain the final target prediction result of the current frame.

[0063] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the UAV target tracking and prediction method as described above.

[0064] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the UAV target tracking and prediction method as described above.

[0065] The UAV target tracking and prediction method and system provided by this invention introduces the modules AT-ResNet18 and AT-Trans, which incorporate temporal context during UAV target tracking. By adding temporal context to the network, the feature extraction process can be dynamically adjusted. At the same time, an encoder-decoder structure is used to incorporate temporal prior knowledge into the similarity map, which enhances the network's expressive power and effectively solves the problem of target prediction accuracy in environments with certain interference, such as target blur, target occlusion, and background interference. Attached Figure Description

[0066] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0067] Figure 1 This is a flowchart illustrating the UAV target tracking and prediction method provided by the present invention;

[0068] Figure 2 This is an overall framework diagram of the UAV target tracking and prediction method provided by the present invention;

[0069] Figure 3This is the TAdaConv operation flowchart provided by the present invention;

[0070] Figure 4 This is the AT-Trans operation flowchart provided by the present invention;

[0071] Figure 5 This is a schematic diagram of the structure of the UAV target tracking and prediction system provided by the present invention;

[0072] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0074] Figure 1 This is a flowchart illustrating the UAV target tracking and prediction method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, it includes:

[0075] Step 100: Obtain the target position of the initial frame collected by the UAV, extract the initial features from the target position of the initial frame using the AT-ResNet18 network, and use the initial features as template features;

[0076] Step 200: Obtain the target location of any current frame collected by the UAV, extract features from the target location of the current frame using the AT-ResNet18 network to obtain the current frame features, and perform cross-correlation operation between the current frame features and the template features to obtain the current frame similarity map;

[0077] Step 300: After convolving the current frame similarity map, input it together with the previous frame temporal prior knowledge of any current frame target position into AT-Trans for encoding to obtain the current frame temporal prior knowledge;

[0078] Step 400: Convolve the current frame temporal prior knowledge with the current frame similarity map and then input them together into AT-Trans for decoding to obtain the current frame enhanced similarity map;

[0079] Step 500: Classify and regress the enhanced similarity map of the current frame to obtain the final target prediction result of the current frame.

[0080] Specifically, for a given target location in the initial frame, the Adaptive Time ResNet18 (AT-ResNet18) network is used, where the last two convolutional layers are replaced with Temporally Adaptive Convolutions (TAdaConv) to extract features from the target in the initial frame and save them as template features.

[0081] For a given current frame, the AT-ResNet18 network is used, with the last two convolutional layers replaced by TAdaConv. After feature extraction, the current frame features are cross-correlated with the template features to obtain the current frame similarity map.

[0082] The obtained similarity map is convolved and saved, and together with the temporal prior knowledge of the previous frame, it is used as the input of the Adaptive Time-Transformer (AT-Trans) encoder. The output of the encoder is the temporal prior knowledge of the current frame.

[0083] The temporal prior knowledge of the current frame and the result saved in the previous step are used as input to the AT-Trans decoder to obtain the final improved similarity map;

[0084] Finally, the finalized similarity map is classified and regressed to obtain the final prediction result of the target in the current frame.

[0085] like Figure 2 As shown, the overall framework diagram of this invention includes two parts: TAdaCNN and AT-Trans. For the time series formed by multiple current frames in the UAV target recognition process, such as... Figure 2 The first frame, second frame, ..., frame t shown are first processed into a baseline template frame T. The subsequent multiple current frames are then input into an improved ResNet-18 network, where the last two convolutional layers are replaced with TAdaConv. Then, similarity maps R1, R2, ..., R... of the multiple current frames are generated. t Convolution is performed to obtain F1, F2, ..., F t The input is fed into multiple time-adaptive encoders in AT-Trans for encoding, resulting in multiple temporal prior knowledge of the current frame. The data is then input into multiple time-adaptive encoders in AT-Trans for decoding, resulting in multiple enhanced similarity maps of the current frame. Finally, classification and regression are performed to obtain the prediction result for the current frame.

[0086] This invention ensures a good balance between algorithm accuracy and training speed by employing a deeper network as the backbone and a faster optimization strategy. AT-ResNet18 and AT-Trans incorporate temporal information into the tracking task, taking into account the inherent temporal correlation between each frame, thus enhancing the algorithm's robustness and the network's expressive power.

[0087] Based on the above embodiments, the AT-ResNet18 network includes a backbone network composed of a ResNet18 network. The backbone network is used for feature extraction, and the last two convolutional layers of the ResNet18 network are replaced with temporally adaptive convolutional TAdaConv layers.

[0088] The Adam algorithm, which is based on the minimum sample size (mini-batch), is adopted as the optimization strategy for the AT-ResNet18 network.

[0089] In this embodiment of the invention, compared to networks with fewer or deeper layers, ResNet-18 can not only learn the feature information of the tracking target more deeply, but also avoids the burden on the target tracking task caused by resource constraints due to the aerial platform, achieving a good balance between accuracy and algorithm time. AT-ResNet18 is an improvement on the traditional ResNet-18 network. Since the correlation and role of information between temporal contexts need to be considered during feature extraction, the last two convolutional layers of ResNet-18 are replaced with TAdaConv to better extract features.

[0090] like Figure 3 As shown, the TAdaConv in this embodiment of the invention is specifically described as follows:

[0091] In general, given the input features X of a certain stage in the network at frame t... t The output of TAdaConv From this, we can conclude that:

[0092]

[0093] Where * represents the convolution operation, W t and b t These are the time weights and biases during the convolution process.

[0094] In TAdaConv, the parameter (W) t and b t The value is calculated from learnable parameters (W and b) and a calibration factor, and it is different for each frame.

[0095] Specifically, TAdaConv maintains a time context queue X. p ∈ L×C This includes frame descriptors for the previous L-1 frames and the current frame. Here, Cat represents the concatenation operation, and the frame descriptor is obtained by performing global average pooling (GAP) on the input features of the current frame, that is: GAP is a global pooling operation. And for each frame's calibration factor... and The generation of the time context queue X p Perform two one-dimensional convolutions, with the kernel size being: Where φ w and φ b They represent the generation and One-dimensional convolution operation.

[0096] In addition, the weight of φ is initialized to 0, so during initialization, W t =, b t =. For the case where t≤L-1, since there are not enough frames to include in the computation, TAdaConv uses the descriptor of the first frame. To fill in the gaps.

[0097] Based on the above embodiments, after convolving the current frame similarity map, it is input into AT-Trans along with the previous frame temporal prior knowledge of any current frame target position for encoding to obtain the current frame temporal prior knowledge, including:

[0098] like Figure 4 As shown, the similarity graph R of the current frame is... t After performing the convolution operation, we obtain F. t , will F t As the Key and Value of the multi-head attention layer, it incorporates the temporal prior knowledge from the previous frame. Using the Query of the multi-head attention layer as the initial input to the encoder, the output of the multi-head attention layer superimposed on frame t can be obtained. Here, Norm represents the layer normalization operation, and MultiHead represents the multi-head attention layer operation. Then, it passes through a temporal information filter, which is mainly operated by a feedforward neural network (FFN). This filter first processes the output of the first multi-head attention layer... After one convolution, the result is then subjected to global average pooling, and finally a calibration factor α is obtained through a feedforward neural network. The final filtered time information F t f It can be obtained from the following: Here, φ represents a convolutional layer. Then, the temporal prior knowledge of the current t-th frame can be obtained as follows:

[0099] For the first frame of the tracking sequence, no unified temporal prior knowledge is used. Instead of performing initialization, it obtains and sets initial temporal prior knowledge by performing a convolution on the initial similarity map F0 of the tracked target in the template frame, i.e.: φ init This indicates that the initial frame similarity map is convolved.

[0100] Based on the above embodiments, the current frame temporal prior knowledge and the current frame similarity map are convolved and then jointly input into AT-Trans for decoding to obtain the current frame enhanced similarity map, including:

[0101] like Figure 4 As shown, based on the temporal prior knowledge of the current t-th frame obtained from the encoder... The aim is to refine and detail the similarity map. It employs two multi-head attention layers, and finally uses a feedforward neural network to obtain the output of the final decoder.

[0102] By generating the attention matrix for the current t-th frame, It can be used to help with similarity graphs R t To refine and detail the final output, we need to obtain the final output. Right now: in This represents the output of the first multi-head attention layer in the decoder. This represents the output of the second multi-head attention layer in the decoder.

[0103] This invention maintains high accuracy even in extreme environments such as target blurring, complete target occlusion, and background interference, and exhibits robustness for UAV target tracking on unique aerial platforms.

[0104] The UAV target tracking and prediction system provided by the present invention is described below. The UAV target tracking and prediction system described below can be referred to in correspondence with the UAV target tracking and prediction method described above.

[0105] Figure 5This is a schematic diagram of the structure of the UAV target tracking and prediction system provided in an embodiment of the present invention, as shown below. Figure 5 As shown, it includes: a first extraction module 51, a second extraction module 52, an encoding module 53, a decoding module 54, and a classification and regression module 55, wherein:

[0106] The first extraction module 51 is used to obtain the target position of the initial frame collected by the UAV, and to extract the initial features of the target position of the initial frame using the AT-ResNet18 network, and to use the initial features as template features; the second extraction module 52 is used to obtain the target position of any current frame collected by the UAV, and to extract the current frame features of any current frame target position using the AT-ResNet18 network, and to perform a cross-correlation operation between the current frame features and the template features to obtain a current frame similarity map; the encoding module 53 is used to convolve the current frame similarity map and input it together with the previous frame time prior knowledge of any current frame target position into AT-Trans for encoding to obtain current frame time prior knowledge; the decoding module 54 is used to convolve the current frame time prior knowledge and the current frame similarity map and input them together into AT-Trans for decoding to obtain a current frame enhanced similarity map; the classification and regression module 55 is used to classify and regress the current frame enhanced similarity map to obtain the final target prediction result of the current frame.

[0107] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a UAV target tracking and prediction method. This method includes: acquiring the target position of an initial frame collected by the UAV; extracting initial features from the initial frame target position using an AT-ResNet18 network; using the initial features as template features; acquiring any current frame target position collected by the UAV; extracting current frame features from any current frame target position using an AT-ResNet18 network; performing a cross-correlation operation between the current frame features and the template features to obtain a current frame similarity map; convolving the current frame similarity map and inputting it along with the previous frame temporal prior knowledge of any current frame target position into AT-Trans for encoding to obtain current frame temporal prior knowledge; convolving the current frame temporal prior knowledge with the current frame similarity map and inputting it along with AT-Trans for decoding to obtain a current frame enhanced similarity map; and classifying and regressing the current frame enhanced similarity map to obtain the final target prediction result for the current frame.

[0108] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0109] On the other hand, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the UAV target tracking and prediction method provided by the above methods. The method includes: acquiring the target position of an initial frame collected by the UAV; extracting features from the target position of the initial frame using an AT-ResNet18 network to obtain initial features; using the initial features as template features; acquiring any target position of a current frame collected by the UAV; extracting features from the target position of the current frame using an AT-ResNet18 network to obtain current frame features; performing a cross-correlation operation between the current frame features and the template features to obtain a current frame similarity map; convolving the current frame similarity map and inputting it together with the previous frame temporal prior knowledge of any current frame target position into AT-Trans for encoding to obtain current frame temporal prior knowledge; convolving the current frame temporal prior knowledge with the current frame similarity map and inputting it together into AT-Trans for decoding to obtain a current frame enhanced similarity map; classifying and regressing the current frame enhanced similarity map to obtain the final target prediction result of the current frame.

[0110] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0111] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for target tracking and prediction in unmanned aerial vehicles (UAVs), characterized in that, include: The target position of the initial frame collected by the UAV is obtained, and the initial features are extracted from the target position of the initial frame using the adaptive temporal residual AT-ResNet18 network. The initial features are then used as template features. The target location of any current frame collected by the UAV is obtained, and the AT-ResNet18 network is used to extract features from the target location of the current frame to obtain the current frame features. The current frame features are cross-correlated with the template features to obtain the current frame similarity map. After convolving the current frame similarity map, it is input together with the previous frame time prior knowledge of any current frame target position into the adaptive time transformer AT-Trans for encoding to obtain the current frame time prior knowledge; The current frame temporal prior knowledge and the current frame similarity map are convolved and then input together into AT-Trans for decoding to obtain the current frame enhanced similarity map; The enhanced similarity map of the current frame is classified and regressed to obtain the final target prediction result of the current frame; The AT-ResNet18 network includes a backbone network composed of ResNet18 networks, which is used for feature extraction. The last two convolutional layers of the ResNet18 network are replaced with temporally adaptive convolutional TAdaConv layers. The Adam algorithm for adaptive moment estimation based on the minimum sample size (mini-batch) is adopted as the optimization strategy for the AT-ResNet18 network. The last two convolutional layers of the ResNet18 network are replaced with TAdaConv layers, including: Obtain the target position of any current frame Input features ; For the input features The current frame descriptor is obtained after global average pooling. : in This is a global pooling operation; Sure The former Frame descriptor , Total number of frames; The current frame descriptor Compared to the past Frame descriptor Establish a connection to obtain the time context queue. : in For splicing operations, For a vector space, The number of feature channels; For the current frame descriptor Perform two one-dimensional convolutions to obtain the temporal convolution kernel. and bias value convolution kernel : in and They represent the generation and One-dimensional convolution operation; Will and The weights are initialized to 0, thus obtaining the time weights learnable parameters. and bias values ​​can be learned parameters ; By time convolution kernel and time weights learnable parameters Obtain the time weights during the convolution process : Convolution kernel with bias value and bias values ​​can be learned parameters Obtain the bias value during the convolution process : Time weights in the convolution process With the input features Perform convolution, and add the bias value during the convolution process. The input features are obtained. Output characteristics : 。 2. The UAV target tracking and prediction method according to claim 1, characterized in that, The current frame descriptor Compared to the past Frame descriptor Establish a connection to obtain the time context queue. It also includes: If the target position of any of the current frames is determined Then the frame descriptor of the first frame is used. Time Context Queue Fill in the blanks.

3. The UAV target tracking and prediction method according to claim 1, characterized in that, After convolving the current frame similarity map, it is input into AT-Trans along with the previous frame temporal prior knowledge of any target position in the current frame for encoding, to obtain the current frame temporal prior knowledge, including: The current frame similarity map Convolution is performed to obtain ,Will As the key and value of the multi-head attention layer, the prior knowledge of the previous frame's time is used. As a query for the multi-head attention layer, the output of the multi-head attention layer superimposed on frame t is obtained. : in Presentation layer normalization operation, This indicates a bullish attention layer operation; The output of the first multi-head attention layer Convolution and global average pooling are performed sequentially, and then the calibration factor is obtained through a feedforward neural network. : in This represents a convolutional layer; The output of the first multi-head attention layer The output of the multi-head attention layer superimposed on frame t. and calibration factor Obtain filtered time information : According to time information Obtain the time prior knowledge of the current t-th frame. : 。 4. The UAV target tracking and prediction method according to claim 3, characterized in that, For the current frame similarity map Initial frame similarity map Perform convolution to obtain the initial frame temporal prior knowledge. : in This indicates that the initial frame similarity map is convolved.

5. The UAV target tracking and prediction method according to claim 1, characterized in that, The current frame temporal prior knowledge and the current frame similarity map are convolved and then input together into AT-Trans for decoding to obtain the current frame enhanced similarity map, including: The time prior knowledge of the current t-th frame Similarity map with the current frame The data is sequentially fed into two multi-head attention layers, and then passed through a feedforward neural network to obtain the enhanced similarity map of the current frame. : in This represents the output of the first multi-head attention layer in the decoder. This represents the output of the second multi-head attention layer in the decoder.

6. A UAV target tracking and prediction system, based on the UAV target tracking and prediction method according to any one of claims 1 to 5, characterized in that, include: The first extraction module is used to obtain the target position of the initial frame collected by the UAV, extract the initial features from the target position of the initial frame using the AT-ResNet18 network, and use the initial features as template features. The second extraction module is used to obtain the target position of any current frame collected by the UAV, extract the features of the target position of the current frame using the AT-ResNet18 network to obtain the current frame features, and perform cross-correlation operation between the current frame features and the template features to obtain the current frame similarity map. The encoding module is used to convolve the current frame similarity map and then input it together with the previous frame temporal prior knowledge of any current frame target position into AT-Trans for encoding to obtain the current frame temporal prior knowledge. The decoding module is used to convolve the current frame temporal prior knowledge and the current frame similarity map, and then input them together into AT-Trans for decoding to obtain the current frame enhanced similarity map; The classification and regression module is used to classify and regress the enhanced similarity map of the current frame to obtain the final target prediction result of the current frame.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the UAV target tracking and prediction method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the UAV target tracking and prediction method as described in any one of claims 1 to 5.

Images