A multi-unmanned aerial vehicle target tracking method, device, equipment and medium under a visual range

Abstract

The application relates to a target tracking method, device and equipment under the view of multiple unmanned aerial vehicles and a medium. In the target tracking process, the following steps are repeatedly performed: acquiring images shot by multiple unmanned aerial vehicles and performing real-time updating; inputting a time sequence image sequence in sequence into a Transform network for feature extraction, wherein a first frame image is used as an initialized target template, and subsequent frame images are used as target search area images; inputting feature maps of the target template into two different convolution layers respectively to obtain a classification branch and a regression branch; meanwhile, inputting feature maps of the target search area into a convolution layer to obtain a convolution result; performing convolution on the convolution result by using the classification branch and the regression branch as convolution kernels respectively to obtain a classification result and a regression result; taking the regression result as a detection result of a target frame to realize target tracking; and updating the target template based on the similarity between the detection result of the target frame and the current target template. Compared with the prior art, the application can continuously and accurately track a moving target in a larger range.

Description

Technical Field

[0001] This invention relates to the field of target tracking technology, and in particular to a target tracking method, apparatus, device, and medium under multi-UAV field of view. Background Technology

[0002] Most recent research has focused on target tracking algorithms for single UAVs. However, single UAV target tracking systems have limitations such as small detection range and narrow viewing angle, resulting in insufficient field of view for tracking algorithms in complex and large-scale scenes. This makes it impossible to continuously track targets in complex environments. Multi-UAV collaborative target tracking can make up for the shortcomings of single UAV tracking, thereby improving the accuracy of tracking algorithms.

[0003] Target tracking is a popular research area in computer vision, applied in fields such as intelligent robot vision, military, medical, and unmanned systems. Recent years have seen unprecedented progress in target tracking algorithms. Some research has utilized the Siamese network framework; however, due to the lack of sufficient global context, Siamese networks are prone to getting trapped in local optima and may lose some semantic information, leading to inaccurate bounding box predictions. In recent years, the Transformer has been widely used in vision because its self-attention mechanism can model the global context to obtain rich contextual information, resulting in highly efficient algorithms. However, it still faces many challenges. For example, single drones have limited detection ranges, hindering large-scale target tracking, and have narrow viewing angles. These limitations result in insufficient detection fields in complex, large-scale scenes, making it impossible to continuously and accurately track moving targets in complex dynamic environments. Summary of the Invention

[0004] The purpose of this invention is to provide a target tracking method, apparatus, device, and medium under the multi-UAV field of view. The method transforms the large-scale image acquisition task into small-scale image acquisition by multiple UAVs. After location numbering, depth features are extracted using Transformer. Similarity is calculated using a Siamese network as a framework to finally complete the tracking. This method can overcome the limitations of single-UAV target tracking systems in terms of small detection range and small observation angle, and achieve effective and stable tracking of the target.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A target tracking method under multi-UAV field of view, wherein the following steps are repeatedly executed during the target tracking process:

[0007] S1. Acquire images of current and historical moments captured by multiple drones and update them in real time. The images are encoded in chronological order.

[0008] S2. Input the time-series image sequence captured by the drone swarm into the Transformer network in sequence for feature extraction. The first frame image is used as the initial target template, and subsequent frames are used as the search area images.

[0009] S3. Input the feature map of the target template into two different convolutional layers to obtain the classification branch and the regression branch. At the same time, input the feature map of the region to be searched into a convolutional layer to obtain the convolution result.

[0010] S4. Convolve the convolution result of the search region using the classification branch as the convolution kernel to obtain the classification result that distinguishes the foreground and background of the image. Convolve the convolution result of the search region using the regression branch as the convolution kernel to obtain the regression result, which is used as the detection result of the target box. Track the target based on the classification result and the regression result.

[0011] S5. Update the target template based on the similarity between the detection result of the target bounding box and the current target template.

[0012] The multi-drone formation forms a matrix above the tracking area, which is divided into rectangular areas of the same size. The number of rectangular areas is the same as the number of drones. Each drone in the formation is responsible for taking an image of a rectangular area and encoding the image in time sequence. There is no overlap between the rectangular areas taken by different drones.

[0013] In step S2, a small-scale image captured by a single drone is encoded in one dimension according to the drone's location information and then used as input to the Transformer network.

[0014] S2 includes the following steps:

[0015] S21. Input the small-range image after one-dimensional position encoding into the Transformer encoder in time encoding order as the image of the region to be searched.

[0016] S22. Input the first frame of the small-range image after one-dimensional position encoding into the Transformer encoder as the target template;

[0017] S23. Use a Transformer encoder to extract feature maps of the target template and the region to be searched, respectively. The Transformer encoder includes a Dropout layer, a first batch normalization module, a multi-head attention module, a second batch normalization module, and an MLP module connected in sequence.

[0018] S3 includes the following steps:

[0019] S31. The results of the Transformer encoder output from the image of the region to be searched are stitched together to form a complete large-scale image feature map.

[0020] S32. Input the feature map of the target template into different convolutional layers and output 2k*256 channel and 4k*256 channel data respectively. k means that each grid generates k anchors. The 2k*256 channel data is used as the classification branch and the 4k*256 channel data is used as the regression branch.

[0021] S33. Input a large-scale image feature map of the region to be searched into a convolutional layer and output a 256-channel data as the convolution result.

[0022] S4 includes the following steps:

[0023] S41. Convolve the 2k*256 channel data output from step S32 with the convolution result output from step S33 to output the classification result of the classification branch, which distinguishes between the foreground and background of the image. Specifically, it is defined as follows:

[0024]

[0025] Where, t(z) cls t(x) represents the 2k*256 channel data obtained after convolution of the target template in step S32, and t(x) represents the 256 channel data obtained after convolution of the image of the region to be searched in step S33. It contains 2k channel vectors, each point representing a positive or negative excitation, and is classified using softmax loss.

[0026] S42. Convolve the 4k*256 channel data output from step S32 with the convolution result output from step S33 to output the regression result of the regression branch, which is used as the prediction result of the target box, specifically defined as:

[0027]

[0028] Where, t(z) reg The data consists of 4k*256 channels obtained after convolution of the target template in step S32. Contains 4k channel vectors, each point representing the dx, dy, dw, dh values ​​between the anchor box and the standard box, through smoothing. L1 The loss is determined, specifically expressed as follows:

[0029]

[0030]

[0031]

[0032] Among them, A x A y A w A h These are the coordinates of the anchor frame's center point and its length and width; T x ,T y ,T w ,T h σ represents the center point and width of the standard bounding box. σ can change the sensitivity of the loss function to outliers, and σ is usually set to 1.

[0033] S5 includes the following steps:

[0034] S51. Based on the target bounding box detected in S4, calculate the similarity between the image within the target bounding box and the current target template;

[0035] S52. If the similarity between the image within the target bounding box and the current target template is lower than a preset threshold, the image is cropped according to the position of the target bounding box, and the target template is updated to the cropped image; if the similarity between the image within the target bounding box and the current target template is greater than or equal to the preset threshold, the target template is not updated.

[0036] A target tracking device with multiple UAV field of view includes:

[0037] The image acquisition module is used to acquire and update images taken by multiple drones at current and historical moments in real time. The images are encoded in chronological order.

[0038] The feature extraction module is used to input the time-series image sequence captured by the drone swarm into the Transformer network in sequence for feature extraction. The first frame image captured is used as the initial target template, and the remaining images are used as the search area.

[0039] The classification and regression modules are used to input the feature map of the target template into two different convolutional layers to obtain the classification branch and the regression branch. At the same time, the feature map of the region to be searched is input into a convolutional layer to obtain the convolution result.

[0040] The target tracking module performs the following steps: convolving the convolution result of the search region with the classification branch as the convolution kernel to obtain a classification result that distinguishes the foreground and background of the image; convolving the convolution result of the search region with the regression branch as the convolution kernel to obtain a regression result, which is used as the detection result of the target box; and tracking the target based on the classification result and the regression result.

[0041] The target template update module is used to update the target template based on the similarity between the detection result of the target bounding box and the current target template during the target tracking process.

[0042] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.

[0043] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1. This invention utilizes the characteristic that the Transformer module requires multiple images as input. By using multiple drones to capture multiple images, there is no need to divide the images into individual patches and then input them into the Transformer. Instead, the images are directly input into the Transformer encoder, which not only reduces computational redundancy but also improves the tracking speed.

[0046] 2. This invention introduces an online template update module, which determines whether to update the template image based on the correlation between the search image and the target template. This can cope with changes in the appearance of objects in complex scenes, enabling it to better handle difficulties such as severe occlusion, scale changes and complex backgrounds during the tracking process. It effectively captures time information and processes changes in the appearance of objects, thereby improving the performance of target tracking.

[0047] 3. This invention utilizes the advantage of multiple drones having a large field of view, making up for the limitations of a single drone target tracking system in terms of small detection range and small observation angle, and achieving effective and stable tracking of the target. Attached Figure Description

[0048] Figure 1 This is a flowchart of the method of the present invention;

[0049] Figure 2 This is a schematic diagram of the image processing process in one embodiment;

[0050] Figure 3 This is a structural block diagram of the Transformer module group of the feature extraction network in one embodiment;

[0051] Figure 4 This is a structural block diagram of an MLP module in one embodiment. Detailed Implementation

[0052] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0053] This embodiment provides a target tracking method under the field of view of multiple unmanned aerial vehicles (UAVs), such as Figure 1 As shown, the following steps are repeatedly performed during target tracking:

[0054] S1. Acquire images of current and historical moments captured by multiple drones and update them in real time. The images are encoded in chronological order.

[0055] In this embodiment, multiple drones form a matrix formation above the tracking area. The tracking area is divided into rectangular regions of the same size, with the number of rectangular regions matching the number of drones. Each drone in the formation is responsible for capturing an image of one rectangular region and encoding the image in temporal order. There is no overlap in the rectangular regions captured by different drones. The images captured by each drone in each frame are stitched together to form a complete image of the tracking area.

[0056] S2. Input the time-series image sequence captured by the drone cluster into the Transformer network in sequence for feature extraction. The first frame image is used as the initial target template z, and subsequent frame images are used as the search region image x.

[0057] All small-area images are stitched together to form a large-area search image. However, since the Transformer requires small-area images as input, the images need to be one-dimensionally positionally encoded according to the drone's location information. That is, the small-area images captured by a single drone are one-dimensionally positionally encoded according to the drone's location information and then used as input to the Transformer network. In this embodiment, four drones are used to capture images, and time-series images at four different times are analyzed. Therefore, the total search area image is four times the size of the target template image. The size of each small-area image is 16*16*3, so the size of the target template is 56*56*3.

[0058] Specifically, S2 includes the following steps:

[0059] S21. Input the small-range image after one-dimensional position encoding into the Transformer encoder in time encoding order as the image of the region to be searched.

[0060] S22. Input the first frame of the small-range image after one-dimensional position encoding into the Transformer encoder as the target template;

[0061] S23. Use the Transformer encoder to extract feature maps of the target template and the region to be searched, respectively, where, for example... Figure 3 As shown, the Transformer encoder comprises a Dropout layer, a first batch normalization module, a multi-head attention module, a second batch normalization module, and an MLP module connected in sequence. The structure of the MLP module is as follows: Figure 4 As shown, it includes, in sequence, a linear layer, a GELU layer, a Dropout layer, another linear layer, and a Dropout layer.

[0062] The multi-head attention module is defined as follows:

[0063]

[0064] Among them, Q i This represents the query, and we will subsequently work with each key. i Perform matching, K i This represents the key, which will be used by each Q in the future. i Match, V i This represents the information extracted from the image, followed by Q. i and K i The matching process can be understood as calculating the correlation between two things; the greater the correlation, the higher the V. i The greater the weight, the higher the d. k Representative vector K i The length of the feature fusion vector is calculated and the above operation is repeated 12 times.

[0065] like Figure 2 As shown, each image is converted into a one-dimensional vector of length 768 using Conv2d. The convolutional layer contains 768 convolutional kernels of size 16*16*3, with a stride equal to the kernel size. Then, the embedding vectors of all the smaller images are sequentially arranged to obtain the embedding sequence of the entire larger image. A special vector CLS is added at the beginning of the sequence, representing the one-dimensional positional encoding of the output. Afterwards, it passes through a Dropout layer, a Batch Normalization module to normalize the image data, and finally an MLP module.

[0066] S3. Input the feature map of the target template into two different convolutional layers to obtain the classification branch and the regression branch. At the same time, input the feature map of the region to be searched into a convolutional layer to obtain the convolution result.

[0067] Specifically, S3 includes the following steps:

[0068] S31. The results of the Transformer encoder output from the image of the region to be searched are stitched together to form a complete large-scale image feature map.

[0069] The complete image size after the target template is stitched together by the transformer network is 7*7*256, and the complete image size after the image of the search area is stitched together by the transformer network is 26*26*256.

[0070] S32. Input the feature map t(z) of the target template into different convolutional layers respectively, and output 2k*256 channel and 4k*256 channel data respectively. k means that each grid generates k anchors. The 2k*256 channel data is used as the classification branch and the 4k*256 channel data is used as the regression branch.

[0071] Specifically, in this embodiment, the classification branch has 5*5*(2k*256) channels, the regression branch has 5*5*(4k*256) channels, and the aspect ratio of the anchors is [0.33,0.5,1,2,3].

[0072] S33. Input the large-scale image feature map t(x) of the region to be searched into a convolutional layer and output a 256-channel data as the convolution result.

[0073] S4. Use the classification branch as the convolution kernel to convolve the convolution result of the search region to obtain the classification result that distinguishes the foreground and background of the image. Use the regression branch as the convolution kernel to convolve the convolution result of the search region to obtain the regression result, which is used as the detection result of the target box. Track the target based on the classification result and the regression result.

[0074] Specifically, S4 includes the following steps:

[0075] S41. Convolve the 2k*256 channel data output from step S32 with the convolution result output from step S33 to output the classification result of the classification branch, which distinguishes between the foreground and background of the image. Specifically, it is defined as follows:

[0076]

[0077] Where, t(z) cls t(x) represents the 2k*256 channel data obtained after convolution of the target template in step S32, and t(x) represents the 256 channel data obtained after convolution of the image of the region to be searched in step S33. It contains 2k channel vectors, each point representing a positive or negative excitation, and is classified using softmax loss.

[0078] S42. Convolve the 4k*256 channel data output from step S32 with the convolution result output from step S33 to output the regression result of the regression branch, which is used as the prediction result of the target box, specifically defined as:

[0079]

[0080] Where, t(z) reg The data consists of 4k*256 channels obtained after convolution of the target template in step S32. Contains 4k channel vectors, each point representing the dx, dy, dw, dh values ​​between the anchor box and the standard box, through smoothing. L1 The loss is determined, specifically expressed as follows:

[0081]

[0082]

[0083]

[0084] Among them, A x A y A w A h These are the coordinates of the anchor frame's center point and its length and width; T x ,T y ,T w ,T h σ represents the center point and width of the standard bounding box. σ can change the sensitivity of the loss function to outliers, and σ is usually set to 1.

[0085] S5. Update the target template based on the similarity between the detection result of the target bounding box and the current target template.

[0086] Specifically, S5 includes the following steps:

[0087] S51. Based on the target bounding box detected in S4, calculate the similarity between the image within the target bounding box and the current target template;

[0088] S52. If the similarity between the image within the target bounding box and the current target template is lower than a preset threshold, the image is cropped according to the target bounding box position to fit the input size of the target template, and the target template is updated to the cropped image; if the similarity between the image within the target bounding box and the current target template is greater than or equal to the preset threshold, the target template is not updated. In this embodiment, the preset threshold is 0.6.

[0089] Since the target's proportion does not change much between two adjacent frames, only the different proportions of the anchors are considered when fixing the anchor proportions. Three ratios were tried: [0.5,1,2], [0.33,0.5,1,2,3], and [0.25,0.33,0.5,1,2,3,4].

[0090] S1 above describes the data acquisition process, S2-S3 the target depth feature extraction process, S4 the target tracking state calculation process, and S5 the handling method for changes in the target template during target tracking. These steps together constitute the complete target tracking process. In actual tracking, steps S1-S5 are repeated to complete the entire target tracking process.

[0091] The above is an introduction to the method embodiments. The following describes the solution of the present invention further through device embodiments.

[0092] This embodiment provides a target tracking device under the field of view of multiple unmanned aerial vehicles (UAVs), including:

[0093] The image acquisition module is used to acquire and update images taken by multiple drones at current and historical moments in real time. The images are encoded in chronological order.

[0094] The feature extraction module is used to input the time-series image sequence captured by the drone swarm into the Transformer network in sequence for feature extraction. The first frame image captured is used as the initial target template, and the remaining images are used as the search area.

[0095] The classification and regression modules are used to input the feature map of the target template into two different convolutional layers to obtain the classification branch and the regression branch. At the same time, the feature map of the region to be searched is input into a convolutional layer to obtain the convolution result.

[0096] The target tracking module performs the following steps: convolving the convolution result of the search region with the classification branch as the convolution kernel to obtain a classification result that distinguishes the foreground and background of the image; convolving the convolution result of the search region with the regression branch as the convolution kernel to obtain a regression result, which is used as the detection result of the target box; and tracking the target based on the classification result and the regression result.

[0097] The target template update module is used to update the target template based on the similarity between the detection result of the target bounding box and the current target template during the target tracking process.

[0098] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0099] The electronic device of this invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) or loaded from a storage unit into random access memory (RAM). The RAM may also store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0100] Multiple components in the device are connected to the I / O interface, including: input units such as keyboards and mice; output units such as various types of displays and speakers; storage units such as disks and optical discs; and communication units such as network interface cards (NICs), modems, and wireless transceivers. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0101] The processing unit executes the various methods and processes described above, such as methods S1 to S5. For example, in some embodiments, methods S1 to S5 may be implemented as computer software programs tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps of methods S1 to S5 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S5 by any other suitable means (e.g., by means of firmware).

[0102] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0103] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0104] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0105] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A target tracking method under multi-UAV field of view, characterized in that, During target tracking, the following steps are repeated: S1. Acquire images of current and historical moments captured by multiple drones and update them in real time. The images are encoded in chronological order. S2. The time-series image sequence captured by the drone swarm is input sequentially into the Transformer network for feature extraction. The first frame image is used as the initial target template, and subsequent frames are used as the search area images. In S2, a small-area image captured by a single drone is encoded with one-dimensional position information according to the drone's location and then used as input to the Transformer network. S2 includes the following steps: S21. Input the small-range image after one-dimensional position encoding into the Transformer encoder in time encoding order as the image of the region to be searched. S22. Input the first frame of the small-range image after one-dimensional position encoding into the Transformer encoder as the target template; S23. Use a Transformer encoder to extract feature maps of the target template and the region to be searched, respectively. The Transformer encoder includes a Dropout layer, a first batch normalization module, a multi-head attention module, a second batch normalization module, and an MLP module connected in sequence. S3. Input the feature map of the target template into two different convolutional layers to obtain the classification branch and the regression branch. At the same time, input the feature map of the region to be searched into a convolutional layer to obtain the convolution result. S3 includes the following steps: S31. The results of the Transformer encoder output from the image of the region to be searched are stitched together to form a complete large-scale image feature map. S32. Input the feature map of the target template into different convolutional layers, and output 2k respectively. 256 channels and 4K The data consists of 256 channels, where k represents the number of anchors generated per grid, and 2k... 256 channels of data are used as classification branches, 4k Data from 256 channels was used as the regression branch; S33. Input a large-scale image feature map of the region to be searched into a convolutional layer and output a 256-channel data as the convolution result. S4. Convolve the convolution result of the search region using the classification branch as the convolution kernel to obtain the classification result that distinguishes the foreground and background of the image. Convolve the convolution result of the search region using the regression branch as the convolution kernel to obtain the regression result, which is used as the detection result of the target box. Track the target based on the classification result and the regression result. S5. Update the target template based on the similarity between the detection result of the target bounding box and the current target template; In this system, multiple drones form a matrix formation above the tracking area, which is divided into rectangular regions of the same size. The number of rectangular regions is the same as the number of drones. Each drone in the formation is responsible for capturing an image of a rectangular region and encoding the image in time sequence. There is no overlap between the rectangular regions captured by different drones.

2. The target tracking method under multi-UAV field of view according to claim 1, characterized in that, S4 includes the following steps: S41, convert the 2k output from step S32... The 256-channel data is convolved with the convolution result output from step S33 to output the classification result of the classification branch, which distinguishes between the foreground and background of the image. Specifically, it is defined as follows: in, The 2k value obtained after convolution of the target template through the S32 step. 256 channels of data, The data consists of 256 channels obtained after convolution of the image of the region to be searched in step S33. It contains 2k channel vectors, each point representing a positive or negative excitation, and is classified using softmax loss. S42, convert the 4k output from step S32 to... The 256-channel data and the convolution result output from step S33 are convolved together to output the regression result of the regression branch, which is used as the prediction result of the target box. Specifically, it is defined as follows: in, The 4k value obtained after the target template undergoes convolution in step S32. 256 channels of data, It contains 4k channel vectors, each point representing the distance between the anchor box and the standard box. dx , dy , dw , dh ,pass The loss is determined, specifically expressed as follows: in, These are the coordinates of the anchor frame's center point and its length and width; It is the center point and dimensions of the standard frame. Used to change the sensitivity of the loss function to outliers.

3. The target tracking method under multi-UAV field of view according to claim 1, characterized in that, S5 includes the following steps: S51. Based on the target bounding box detected in S4, calculate the similarity between the image within the target bounding box and the current target template; S52. If the similarity between the image within the target bounding box and the current target template is lower than a preset threshold, the image is cropped according to the position of the target bounding box, and the target template is updated to the cropped image; if the similarity between the image within the target bounding box and the current target template is greater than or equal to the preset threshold, the target template is not updated.

4. A target tracking device for a multi-UAV vision-based target tracking method according to any one of claims 1-3, characterized in that, include: The image acquisition module is used to acquire and update images taken by multiple drones at current and historical moments in real time. The images are encoded in chronological order. The feature extraction module is used to input the time-series image sequence captured by the drone swarm into the Transformer network in sequence for feature extraction. The first frame image captured is used as the initial target template, and the remaining images are used as the search area. The classification and regression modules are used to input the feature map of the target template into two different convolutional layers to obtain the classification branch and the regression branch. At the same time, the feature map of the region to be searched is input into a convolutional layer to obtain the convolution result. The target tracking module performs the following steps: convolving the convolution result of the search region with the classification branch as the convolution kernel to obtain a classification result that distinguishes the foreground and background of the image; convolving the convolution result of the search region with the regression branch as the convolution kernel to obtain a regression result, which is used as the detection result of the target box; and tracking the target based on the classification result and the regression result. The target template update module is used to update the target template based on the similarity between the detection result of the target bounding box and the current target template during the target tracking process.

5. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 3.

Images