A method for unmanned aerial vehicle target tracking and identification based on tir images

Abstract

The application provides a UAV target tracking and identifying method based on a TIR image, wherein a feature cross fusion unit ARCM for fusing multi-scale features extracted by a convolution module in a backbone network of Siam R-CNN is nested in the backbone network to obtain a UAV target tracking model; target detection and tracking are performed on an input UAV TIR video by using the UAV target tracking model, and a local tracking strategy combined with a space-time constraint and a global motion detection cooperation strategy are used in the tracking process to strengthen an online tracking method and promote stability and accuracy of an online tracking mechanism.

Description

Technical Field

[0001] This invention relates to the field of target tracking technology, and more specifically to a method for tracking and recognizing unmanned aerial vehicles (UAVs) based on TIR images. Background Technology

[0002] To effectively detect, probe, and identify drones, it is necessary to acquire images of the target area and perform drone detection and identification. However, in practice, to ensure effective capture of drone images, thermal infrared (TIR) ​​image acquisition technology is typically used to acquire TIR images of the target area. Due to the low imaging quality of the sensor itself, the semantic information in the target object image is poor. Furthermore, drone targets are usually very small and lack prominent textures or fixed shapes, making them very difficult to distinguish. Summary of the Invention

[0003] Purpose of the invention: To overcome the above technical problems, this invention proposes a method, device, and readable medium for UAV target tracking and recognition based on TIR images.

[0004] Summary of the Invention: To achieve the above objectives, the present disclosure proposes the following technical solutions:

[0005] A method for UAV target tracking and recognition based on TIR images, comprising the following steps:

[0006] (1) An ARCM (Automatic Cross-Fusing Unit) is nested in the backbone network of Siam R-CNN to fuse the multi-scale features extracted by the convolutional modules in the backbone network, thus obtaining a UAV target tracking model.

[0007] (2) Target detection and tracking of the input UAV TIR video using the UAV target tracking model, including the following steps:

[0008] (21) Perform target detection and recognition on the first frame image in the UAV TIR video and extract the initial target detection box;

[0009] (22) Create the sub-tracking trajectory sequence a of the initial target. i Initialize a i =[x gt ], x gt For the initial target's location information;

[0010] (23) Target detection is performed on the current frame, and the similarity between the detection result and the detection result of the previous frame is calculated. Based on the similarity, it is determined whether the initial target in the current frame is lost. If it is not lost, the position information of the detected target in the current frame is recorded into the sub-tracking sequence a. iIf the initial target is lost, the tracking of this segment of the trajectory ends. At this time, the remaining video segments are detected by the UAV target tracking model to establish a new initial target detection box and then return to step (22).

[0011] (3) All detected sub-trajectory tracking sequences are merged according to the detection target to obtain a complete trajectory tracking sequence.

[0012] As an optional implementation of the UAV target tracking and recognition method, the UAV target tracking model includes a template branch network, a detection branch network, and a re-identification network;

[0013] The template branch network is configured to extract the initial target detection box from the UAV TIR video;

[0014] The detection branch network is configured to extract candidate detection boxes that may contain the target object from the current frame;

[0015] The re-identification network is configured to select the detection box with the highest matching degree with the initial target detection box from the candidate detection boxes as the detection result of the current frame; and to calculate the similarity between the detection result of the current frame and the detection result of the previous frame, and output the similarity calculation result.

[0016] As an optional implementation of the UAV target tracking and recognition method, the backbone networks of the template branch network and the detection branch network are ResetNet18 network structures. The feature cross-fusion unit ARCM is set in the convolutional module of the FPN structure in the ResetNet18 network structure, and fuses the image features of different scales extracted by the residual module units cascaded after the convolutional module. The feature cross-fusion unit ARCM includes a Top-Down feature fusion branch structure, a Bottom-Up feature fusion branch structure, and a feature fusion layer. The Top-Down feature fusion branch structure is used to perform top-down layer-by-layer modulation embedding extraction on the high-level feature Y. The Bottom-Up feature fusion branch structure is used to aggregate the channel feature context of each spatial location of the low-level feature X separately. The feature fusion layer performs feature fusion on the output features of the Top-Down feature fusion branch structure and the prime number Bottom-Up feature fusion branch structure, and the calculation formula is:

[0017]

[0018] Where G(Y) represents the features output by the Top-Down feature fusion branch structure, L(X) represents the feature matrix output by the Bottom-Up feature fusion branch structure, and Z represents the fused feature matrix. This symbol represents the pixel-by-pixel multiplication of matrices.

[0019] As an optional implementation of the UAV target tracking and recognition method, the Top-Down feature fusion branch structure includes a GAP layer, a first FC layer, a ReLU layer, a second FC layer, and a Sigmoid layer cascaded in sequence.

[0020] As an optional implementation of the UAV target tracking and recognition method, the Bottom-Up feature fusion branch structure includes a first convolutional layer, a ReLU layer, a second convolutional layer, and a Sigmoid layer cascaded in sequence.

[0021] As an optional implementation of the UAV target tracking and recognition method, the specific steps for obtaining the complete trajectory tracking sequence in step (3) are as follows:

[0022] For each tracked target, a tracking trajectory quality evaluation score is constructed:

[0023]

[0024]

[0025] spatial_eval(a i ,a i+1 )=-|end_box(a i )-start_box(a i+1 )|

[0026] Where A represents the trajectory tracking sequence, score(A) represents the quality evaluation score of A, and similar_eval(a) represents the quality evaluation score of A. i ) is an image similarity calculation function used to evaluate the sub-tracking sequence a. i Similarity of detection results across different image frames; spatial_eval(a i ,a i+1 ) is the positional consistency evaluation function between adjacent sub-tracking sequences; w l ,w r The penalty coefficient is N, which represents the number of trajectory tracking sequences, and a is the number of sequences. i,t Represents the sub-tracking sequence a i Trajectory information corresponding to the image frame at time t, a i,start Represents the sub-tracking sequence a i The trajectory information corresponding to the image frame at the start time, where i represents the sub-tracking sequence number, end_a and start_a represent the start and end times of sub-tracking sequence a, respectively, and box = [x norm ynorm w norm h norm ], used to constrain the continuity of trajectories for the same target, (x norm ,y norm ) represents the center coordinates of the box, w norm The width of the box is represented by h. norm Indicates the height of the box;

[0027] The set of sub-tracking sequences that maximizes score(A) is searched among all sub-tracking sequences using an online dynamic programming method. The obtained sub-tracking sequences are then merged to obtain the track tracking sequence A.

[0028] As an optional implementation of the aforementioned UAV target tracking and identification method, the method further includes the following steps:

[0029] After step (3) is completed, if there is an independent frame, a local tracking strategy combining spatiotemporal constraints is used to track the target for the independent frame. The local tracking strategy includes the following steps:

[0030] Calculate the maximum aspect ratio R of the detected target in the merged trajectory tracking sequence. max and minimum value R min Calculate the maximum value S of the detected target size in the trajectory tracking sequence. max and minimum value S min ;

[0031] The local neighborhood around the center of the detected target in the previous frame of the independent frame is used as the search region. If the detected target in the independent frame appears in the search region, and the size S of the detected target is... c The aspect ratio R satisfies:

[0032] S c ∈[0.8*S min 1.2*S max ],R c ∈[0.8*R min 1.2*R max ]

[0033] Then update S min S max R min R max :

[0034] S min =min(S) min ,S c ),S max =max(S) max ,S c )

[0035] R min =min(R) min ,R c ),R max =max(R) max ,R c )

[0036] And for the independent frame c new Calculate a confidence score:

[0037]

[0038] Among them, c i,start c represents the initial state quantity of the merged trajectory tracking sequence. N,end This represents the end position state of the merged trajectory tracking sequence; IOU is the intersection-union ratio of the target detection boxes, used to calculate the correlation of the target position;

[0039] If score(c new If the preset threshold requirement is met, then c will be... new The corresponding location information is added to the merged trajectory tracking sequence; otherwise, a sequence is established using c new The corresponding position information is a new trajectory tracking sequence based on the starting position information.

[0040] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0041] This invention improves the backbone feature fusion module based on the Siam R-CNN network. By nesting an asymmetric context modulation module (ARCM) for cross-layer feature fusion in the backbone network of Siam R-CNN, multi-layer feature scale fusion is achieved, making the entire UAV target tracking model suitable for feature extraction requirements of infrared small target features.

[0042] Furthermore, this invention enhances the online tracking method by combining a local tracking strategy with spatiotemporal constraints and a collaborative strategy with global motion detection, thereby improving the stability and accuracy of the online tracking mechanism. Attached Figure Description

[0043] Figure 1 This is a structural diagram of the UAV target tracking model involved in the embodiment;

[0044] Figure 2 The diagram shows the infrared feature extraction and fusion network structure with the ARCM module incorporated in the embodiment.

[0045] Figure 3 This is a structural diagram of the ARCM module involved in the embodiment. Detailed Implementation

[0046] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. However, it should be understood that the present invention can be implemented in various forms. The exemplary and non-limiting embodiments shown in the drawings and described below are not intended to limit the invention to the specific embodiments illustrated.

[0047] It should be understood that, where technically feasible, the technical features listed above for different embodiments can be combined with each other to form other embodiments within the scope of this invention. Furthermore, the specific examples and embodiments described in this invention are non-limiting, and corresponding modifications can be made to the structures, steps, and order described above without departing from the protection scope of this invention.

[0048] This embodiment proposes a UAV target tracking and recognition method based on TIR images. The method mainly includes the following steps:

[0049] S1. Constructing a target tracking model based on unmanned aerial vehicles (UAVs)

[0050] Please refer to Figure 1 , Figure 1 The specific structure of a UAV target tracking model is illustrated. In this embodiment, an Asymmetric Robust Contextual Modulation (ARCM) feature cross-fusion unit is nested within the backbone network of Siam R-CNN to form the UAV target tracking model described in this embodiment.

[0051] Siam R-CNN is a visual tracking algorithm based on re-detection, jointly proposed by RWTH Aachen University and Oxford University. Siam R-CNN consists of a template branch network, a detection branch network, and a re-detection network. The input to the template branch network is typically the first frame of the video containing the target object, also known as the template frame. The input to the detection branch network is the detection frames, which are the other image frames in the video besides the template frame.

[0052] The template branch network consists of a backbone layer, a head-DNN object detection network layer, and a ROI-align layer. The backbone layer extracts multi-scale features from the template frame. The extracted multi-scale feature images are then fused with semantic features by the object detection network layer. The fused features are then processed by the ROI-align layer to obtain the initial object detection bounding box, which serves as the reference region.

[0053] The detection branch includes a backbone layer, a head-DNN (Distributed Neural Network) layer, a candidate box recommendation network (PRN), and a ROI-align layer. The backbone layer extracts multi-scale features from the detection frames. The extracted multi-scale feature images are then fused with semantic features by the head-DNN layer. The RPN network outputs several predicted potential object region values ​​based on the extracted multi-scale feature images. Finally, ROI-align is performed to generate all candidate detection boxes in the current frame.

[0054] The candidate detection boxes (boxes) of the current frame and the initial target detection box (gt) are fed into the re-detection network. FirstStage Re-Detection selects candidate detection boxes from the current frame's candidate boxes that are similar to the initial target detection box (gt), and then regresses them using a cascaded RPN to obtain better boxes. The similarity between the obtained boxes and the initial target detection box (gt) is calculated, and the candidate detection box with the highest similarity is selected as the target detection box for the current frame.

[0055] The target detection box det of the current frame and the target detection box det-1 of the previous frame are fed into the Second Stage Re-Detection network in the re-detection network. The similarity between the two target detection boxes is calculated, and the similarity is used to determine whether the initial target is lost and whether the current frame is included in the tracking trajectory sequence of the initial target (since the target may be lost in the UAV TIR video, the trajectory of an initial UAV target may consist of multiple trajectories).

[0056] In this embodiment, the ResetNet18 network structure is used as the Backbone network layer. An ARCM module is integrated into the FPN structure of ResetNet18 to perform semantic feature fusion at different scales, forming an infrared feature extraction and fusion network. Please refer to... Figure 2 Following the convolutional module of the FPN structure, three standard residual module units, ResNetBlock1-3, are sequentially connected to extract image features at different scales. This embodiment adds an ARCM module after the convolutional module of the FPN structure to fuse the image features extracted from adjacent residual module units at different scales. Specifically, ARCM2 fuses the image features extracted from ResNetBlock2 and ResNetBlock3, and the fused features are then fed into ARCM1 for fusion with the image features extracted from ResNetBlock1. To mitigate the impact of pooling layers on small targets, downsampling operations in the network can be used only in the first module CONV of the FPN structure.

[0057] Please refer to Figure 3 , Figure 3 The specific structure of the ARCM module is shown. The ARCM module combines the advantages of Top-Down and Bottom-Up feature fusion networks, containing two branch structures. The first branch structure is the Top-Down feature fusion branch structure, including a cascaded GAP layer, a first FC layer, a ReLU layer, a second FC layer, and a Sigmoid layer. The second branch structure is the Bottom-Up feature fusion branch structure, including a cascaded first convolutional layer, a ReLU layer, a second convolutional layer, and a Sigmoid layer. The Top-Down feature fusion branch structure is used for top-down layer-by-layer modulation embedding extraction of high-level (semantic) features Y. The Bottom-Up feature fusion branch structure uses the channel and spatial separation convolution module Conv to separately aggregate the channel feature context at each spatial location of the low-level feature X. Finally, we use feature fusion to enrich the high-level features by utilizing the spatial details of the low-level features. The calculation formula for feature fusion is: Where G(Y) represents the features output by the Top-Down feature fusion branch structure, L(X) represents the feature matrix output by the Bottom-Up feature fusion branch structure, and Z represents the fused feature matrix. This symbol represents the pixel-by-pixel multiplication of matrices.

[0058] Traditional methods utilize high-level features generated by GAP (global average pooling) and FC (fully connected) to provide more accurate semantic information about targets. The contextual feature association between global channels is also an effective feature quantity (the highest global response value) that helps distinguish targets. However, as networks become deeper, small infrared targets such as UAVs are easily obscured by the background in high-level features. While top-down modulation embedding of semantic information can alleviate processing ambiguity, it still relies on preserving the features of small targets at the feature level. Therefore, to highlight the subtle details of deep infrared small targets, this embodiment combines the advantages of Top-Down and Bottom-Up feature fusion and redesigns... Figure 3 The ARCM module shown further enhances the interaction and fusion of top-down global features and bottom-up local features to exchange multi-scale contextual information, thereby obtaining richer semantic information and spatial detail encoding.

[0059] S2. Input the TIR video to be detected into the constructed UAV target tracking model to perform online tracking of the UAVs appearing in the TIR video. The specific steps are as follows:

[0060] S21. After obtaining the initial target detection box, create a sub-trajectory tracking sequence a. i Used to record the tracking trajectory of the initial target, ai Each element in the expression represents the location information of the detected target in the corresponding image frame, denoted by [bbox, score, RoIs]. Here, bbox represents the position of the target detection box, denoted by (x, y, w, h), where (x, y) represents the center coordinates of the target detection box, w represents the width of the target detection box, and h represents the height of the target detection box. The score represents the similarity between the target detection box and the initial target detection box, and ROIS represents the image region of the target detection box in the corresponding image frame. Initialize a i =[x gt ],x gt This indicates the initial target's location information.

[0061] S22. Based on the similarity between the re-detection results of the current frame and the previous frame, determine whether the initial target in the current detection frame is lost. If not lost, then include the position information of the detected target in the current frame into the sub-tracking sequence a. i If the initial target is lost, the tracking of this segment ends. At this time, the UAV target tracking model is used to detect the remaining video segments to determine the new initial target and sub-tracking sequence. The new initial target is tracked in the remaining video segments, and then the process returns to step S21.

[0062] S3. During the detection process, different sub-trajectory tracking sequences are merged. The specific steps are as follows:

[0063] S31. Construct a tracking trajectory quality evaluation score for each tracking target:

[0064]

[0065]

[0066] spatial_eval(a i ,a i+1 )=-|end_box(a i )-start_box(a i+1 )|

[0067] Where A represents the tracking trajectory sequence, score(A) represents the tracking trajectory quality evaluation score, and similar_eval(a) represents the tracking trajectory quality evaluation score. i ) is an image similarity calculation function used to evaluate the trajectory tracking sequence a. i Similarity of detection results across different image frames; spatial_eval(a i ,a i+1 ) is the positional consistency evaluation function between adjacent sub-trajectories; w l ,w rThe penalty coefficient is N, which represents the number of trajectory tracking sequences, and a is the number of sequences. i,t Represents the trajectory tracking sequence a i Trajectory information corresponding to the image frame at time t, a i,start Represents the trajectory tracking sequence a i The trajectory information corresponding to the image frame at the start time, where i represents the sequence number of the trajectory tracking sequence, end_a and start_a represent the start and end times of trajectory tracking sequence a, respectively, and box = [x norm y norm w norm h norm ], used to constrain the continuity of trajectories for the same target, (x norm ,y norm ) represents the center coordinates of the box, w norm The width of the box is represented by h. norm This indicates the height of the box.

[0068] S32. Using online dynamic programming, search for the set of sub-tracking sequences that maximize score(A) among all sub-tracking sequences. Merge the obtained sub-tracking sequences to obtain the tracking trajectory sequence A. The effective maximum and minimum tracking sequence lengths can be designed according to the business scenario.

[0069] In practical TIR image tracking, UAV targets are typically very small and lack prominent textures or fixed shapes, making them extremely difficult to distinguish. To overcome this problem, this embodiment also employs a local tracking strategy constrained by position and aspect ratio in a local neighborhood to better differentiate potential distractions from the real target.

[0070] The specific details of the local tracking strategy are as follows:

[0071] From a spatial perspective, considering that the target object does not undergo drastic positional movement, and that local consistency between adjacent frames can achieve good target object detection, this embodiment introduces an online memory to store the historical patterns of high-quality detection results from a temporal dimension. After merging the trajectory tracking sequences, if some independent frames (image frames that cannot be included in any merged trajectory tracking sequence) appear, a local tracking strategy combining spatiotemporal constraints can be used for target tracking. The specific steps are as follows:

[0072] The maximum and minimum aspect ratio R of the detected target in the merged trajectory tracking sequence are represented by R. max and R min The maximum and minimum sizes of the detected targets in the merged trajectory tracking sequence are represented by S. max and S minThe potential scale variation range of the target is calculated explicitly.

[0073] The local neighborhood around the center of the target in the previous frame of the independent frame is designated as the search area where the target is most likely to appear. If the detected target in the current independent frame appears in the search area, and the size S of the detected target is... c The aspect ratio R satisfies:

[0074] S c ∈[0.8*S min 1.2*S max ],R c ∈[0.8*R min 1.2*R max ]

[0075] If the current detection result of the target is considered reliable and the trajectory change is continuous, then the stored target state value is updated according to the current detected object state:

[0076]

[0077] To improve the discriminability of target tracking, for each independent frame c where the local search successfully tracks the detection box position... new Calculate a confidence score:

[0078]

[0079] Among them, c i,start c represents the initial state quantity of the merged trajectory tracking sequence. N,end This represents the end position state of the merged trajectory tracking sequence; IOU is the intersection-union ratio of the target detection boxes, used to calculate the correlation of the target position;

[0080] If score(c new If the preset threshold requirement is met, then c will be... new The corresponding location information is added to the merged trajectory tracking sequence; otherwise, a sequence is established using c new The corresponding position information is a new trajectory tracking sequence based on the starting position information.

[0081] As an alternative implementation, in order to alleviate the problem of short-term target disappearance when the target is occluded, a global tracking algorithm based on optical flow tracking ORB feature points can be used to track moving targets in addition to the local tracking algorithm.

[0082] As an alternative implementation, when the background is static or moving slowly, pixels that follow a normal distribution in the time domain and whose grayscale changes between adjacent frames are less than a certain threshold are identified as background pixels; otherwise, they are identified as moving targets. Therefore, when the background is static, a global motion estimation algorithm can be used to locate the target; conversely, a correlation tracker can be used to search for the target in the local area where it last appeared.

[0083] This embodiment also proposes a computer-readable storage medium, such as a memory including program code, which can be executed by a processor to complete the UAV target tracking and identification method in the above embodiment. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CDROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0084] This disclosure also provides a computer program product comprising one or more lines of program code stored in a computer-readable storage medium. A processor of an electronic device reads the program code from the computer-readable storage medium and executes the program code to complete the implementation steps of the UAV target tracking and identification method provided in the above embodiments.

[0085] This disclosure also proposes an apparatus including a processor and a memory. The memory stores a computer program, and the processor executes the computer program to implement the UAV target tracking and identification method described in this disclosure.

[0086] Specifically, memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the training process of the UAV target tracking model and the program instructions / modules corresponding to the UAV target tracking and recognition method in the embodiments of this application. The processor can implement the UAV target tracking and recognition method in the embodiments of this disclosure by running the non-transitory software programs, instructions, and modules stored in the memory. The memory may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0087] The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0088] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0089] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for tracking and recognizing unmanned aerial vehicle (UAV) targets based on TIR images, characterized in that, Including the following steps: (1) A feature cross-fusion unit (ARCM) for fusing multi-scale features extracted by the convolutional modules in the Siam R-CNN backbone network is nested to obtain a UAV target tracking model; the UAV target tracking model includes a template branch network, a detection branch network, and a re-identification network; the template branch network is configured to extract initial target detection boxes from the UAV TIR video; the detection branch network is configured to extract candidate detection boxes that may contain target objects from the current frame; The re-identification network is configured to select the detection box with the highest matching degree with the initial target detection box from the candidate detection boxes, and use it as the detection result of the current frame; And to calculate the similarity between the detection result of the current frame and the detection result of the previous frame, and output the similarity calculation result; (2) Target detection and tracking of the input UAV TIR video using the UAV target tracking model, including the following steps: (21) Perform target detection and recognition on the first frame image in the UAV TIR video and extract the initial target detection box; (22) Create the sub-tracking trajectory sequence a of the initial target. i Initialize a i =[x gt ], x gt For the initial target's location information; (23) Perform target detection on the current frame and calculate the similarity between the detection result and the detection result of the previous frame. Determine whether the initial target in the current frame is lost based on the similarity. If it is not lost, record the position information of the detected target in the current frame into the sub-tracking trajectory sequence a. i ; If the initial target is lost, the tracking of this segment of the trajectory ends. At this time, the remaining video segments are detected by the UAV target tracking model to establish a new initial target detection box and then return to step (22). (3) All detected sub-trajectory tracking sequences are merged according to the detection target to obtain a complete trajectory tracking sequence.

2. The UAV target tracking and identification method according to claim 1, characterized in that, The backbone networks of the template branch network and the detection branch network are ResetNet18 network structures. The feature cross-fusion unit ARCM is set in the convolution module of the FPN structure in the ResetNet18 network structure, and fuses the image features of different scales extracted by the residual module units cascaded after the convolution module. The feature cross-fusion unit ARCM includes a Top-Down feature fusion branch structure, a Bottom-Up feature fusion branch structure, and a feature fusion layer. The Top-Down feature fusion branch structure is used to perform top-down layer-by-layer modulation embedding extraction on the high-level feature Y. The Bottom-Up feature fusion branch structure is used to aggregate the channel feature context for each spatial location of the low-level feature X separately. The feature fusion layer performs feature fusion on the output features of the Top-Down feature fusion branch structure and the prime number Bottom-Up feature fusion branch structure, and the calculation formula is as follows: Where G(Y) represents the features output by the Top-Down feature fusion branch structure, L(X) represents the feature matrix output by the Bottom-Up feature fusion branch structure, and Z represents the fused feature matrix. This symbol represents the pixel-by-pixel multiplication of matrices.

3. The UAV target tracking and identification method according to claim 2, characterized in that, The Top-Down feature fusion branch structure includes a GAP layer, a first FC layer, a ReLU layer, a second FC layer, and a Sigmoid layer that are cascaded in sequence.

4. The UAV target tracking and identification method according to claim 3, characterized in that, The Bottom-Up feature fusion branch structure includes a first convolutional layer, a ReLU layer, a second convolutional layer, and a Sigmoid layer cascaded in sequence.

5. The UAV target tracking and identification method according to claim 1, characterized in that, The specific steps for obtaining the complete trajectory tracking sequence in step (3) are as follows: For each tracked target, a tracking trajectory quality evaluation score is constructed: spatial_eval(a i ,a i+1 )=-|end_box(a i )-start_box(a i+1 )| Where A represents the trajectory tracking sequence, score(A) represents the quality evaluation score of A, and similar_eval(a) represents the quality evaluation score of A. i ) is an image similarity calculation function used to evaluate the sub-tracking sequence a. i Similarity of detection results across different image frames; spatial_eval(a i ,a i+1 ) is the positional consistency evaluation function between adjacent sub-tracking sequences; w l ,w r The penalty coefficient is N, which represents the number of trajectory tracking sequences, and a is the number of sequences. i,t Represents the sub-tracking sequence a i Trajectory information corresponding to the image frame at time t, a i,start Represents the sub-tracking sequence a i The trajectory information corresponding to the image frame at the start time, where i represents the sub-tracking sequence number, end_a and start_a represent the start and end times of sub-tracking sequence a, respectively, and box = [x norm y norm w norm h norm ], used to constrain the continuity of trajectories for the same target, (x norm ,y norm ) represents the center coordinates of the box, w norm The width of the box is represented by h. norm Indicates the height of the box; The set of sub-tracking sequences that maximizes score(A) is searched among all sub-tracking sequences using an online dynamic programming method. The obtained sub-tracking sequences are then merged to obtain the track tracking sequence A.

6. The UAV target tracking and identification method according to claim 5, characterized in that, It also includes the following steps: After step (3) is completed, if there is an independent frame, a local tracking strategy combining spatiotemporal constraints is used to track the target for the independent frame. The local tracking strategy includes the following steps: Calculate the maximum aspect ratio R of the detected target in the merged trajectory tracking sequence. max and minimum value R min Calculate the maximum value S of the detected target size in the trajectory tracking sequence. max and minimum value S min ; The local neighborhood around the center of the detected target in the previous frame of the independent frame is used as the search region. If the detected target in the independent frame appears in the search region, and the size S of the detected target is... c The aspect ratio R satisfies: S c ∈[0.8*S min ,1.2*S max ],R c ∈[0.8*R min ,1.2*R max ] Then update S min S max R min R max : S min =min(S min ,S c ),S max =max(S max ,S c ) R min =min(R min ,R c ),R max =max(R max ,R c ) And for the independent frame c new Calculate a confidence score: Among them, c i,start c represents the initial state quantity of the merged trajectory tracking sequence. N,end This represents the end position state of the merged trajectory tracking sequence; IOU is the intersection-union ratio of the target detection boxes, used to calculate the correlation of the target position; If score(c new If the preset threshold requirement is met, then c will be... new The corresponding location information is added to the merged trajectory tracking sequence; otherwise, a sequence is established using c new The corresponding position information is a new trajectory tracking sequence based on the starting position information.

Images