A Transformer-based collaborative tracking method for occluded targets in air and ground

Abstract

The present invention relates to a method for collaborative air-ground tracking of occluded targets based on a Transformer network, and belongs to the technical field of collaborative target tracking for unmanned driving systems. It solves the problem in the prior art that occluded targets cannot be tracked, and improves the robustness of target tracking tasks. The method of the present invention utilizes the advantages of the land-air collaborative perspective to separately maintain the tracking status of the target in the field of view during the tracking process. At the same time, the land and air platforms each have the ability to accurately identify the target when it is partially occluded. When the tracking target in the field of view of one intelligent agent is partially occluded, the other party locates the tracking target, helping the intelligent agent that has lost the tracking target to relocate the target in the field of view. The robustness of target tracking under interference such as target deformation, occlusion, and background similarity is enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of collaborative target tracking of unmanned driving systems, and in particular relates to a method for collaborative air-ground tracking of occluded targets based on a Transformer network. Background Art

[0002] Target tracking is a fundamental subtask in mobile agent applications, with numerous applications in security, smart agriculture, search and rescue, and more. However, existing target tracking methods used by mobile agents often fail due to relative motion between the agent and the target, combined with the limited field of view of onboard cameras. Furthermore, existing air-ground collaborative systems only utilize the high-altitude field of view of drones to provide environmental or target awareness for unmanned vehicles. These systems are only suitable for ideal, open environments with no obstructions above the target, and are ineffective against obstructions in urban and forest environments. Summary of the Invention

[0003] In view of the above problems, the present invention provides a method for collaborative air-ground tracking of occluded targets based on a Transformer network, which solves the problem of being unable to track occluded targets in the prior art and improves the robustness of the target tracking task.

[0004] The present invention provides a Transformer network-based collaborative tracking method for occluded targets, which is characterized by the following specific steps:

[0005] Step 1: Build a target tracking network model; deploy the target tracking network model on the UAV and unmanned vehicle of the air-ground collaborative intelligent body; the target tracking network model includes the VGG feature extraction network, the Transformer feature enhancement network and the prediction network;

[0006] Step 2: Select the tracking target in the initial frame images of the UAV and the unmanned vehicle respectively; the target area where the selected tracking target is located is used as the target template;

[0007] Step 3: Determine the target search area of ​​the next frame of image based on the target area where the tracking target is located in the previous frame of image; splice the target template and the search area to obtain a spliced ​​feature vector;

[0008] Step 4: Use the VGG feature extraction network of the target tracking network model to extract the features of the spliced ​​feature vector to obtain the preliminary feature vector;

[0009] Step 5: Use the Transformer feature enhancement network of the target tracking network model to extract the preliminary feature vector to obtain the enhanced feature vector;

[0010] Step 6: Use the prediction network of the target tracking network model to process the enhanced feature vector to obtain the pixel position of the minimum rectangular box containing the tracked target in the current frame;

[0011] Step 7: Convert the pixel position of the tracking target from the pixel coordinate system to the world coordinate system;

[0012] Step 8: Using the coordinate positions of the tracking target and the air-ground intelligent body in the world coordinate system obtained in step 7, the tracking control of the UAV and the unmanned vehicle is performed based on visual servoing;

[0013] Step 9: Based on the tracking control of the UAV and the unmanned vehicle obtained in step 8, individual target tracking and air-ground collaborative tracking control are performed.

[0014] Optionally, the specific steps for building the target tracking model in step 1 are:

[0015] Get the training dataset;

[0016] Establish a training loss function, the expression is:

[0017] loss = w1GIoU + w2L1;

[0018] Where w1 and w2 are the weights of the GIoU loss function and the L1 loss function respectively;

[0019] Among them, the expression of GIoU loss function is:

[0020]

[0021] Where A and B are the accurate box and predicted box respectively; C is the smallest box containing the accurate box A and the predicted box B;

[0022] The expression of L1 loss function is:

[0023]

[0024] Among them, x i and y i are the values ​​of the i-th corner point coordinates of the predicted box and the accurate box respectively;

[0025] The target tracking network is trained based on the training data set and the training loss function to obtain the target tracking network model.

[0026] Optionally, in step 1, the target tracking network model is optimized using the ONNX model.

[0027] Optionally, the encoder in the Transformer feature enhancement network includes a normalization layer, a multi-head self-attention layer, a residual connection layer, a normalization layer, a fully connected layer and a residual connection layer.

[0028] Optionally, the prediction network is a multi-layer fully connected network, and each layer of the fully connected network includes a linear layer, a normalization layer and an activation layer.

[0029] Optionally, the pixel position of the minimum rectangular frame containing the tracking target in the current frame in step 6 includes the pixel positions of the upper left corner and the lower right corner of the minimum rectangular frame, and the pixel position of the center point of the tracking target.

[0030] Optionally, the pixel positions of the upper left corner and the lower right corner of the minimum rectangular frame are expressed in a pixel coordinate system as follows:

[0031] [u1,v1,u2,v2];

[0032] Wherein, u1 and v1 represent the horizontal coordinate value and vertical coordinate value of the pixel position of the upper left corner point of the minimum rectangular frame in the pixel coordinate system respectively; u2 and v2 represent the horizontal coordinate value and vertical coordinate value of the pixel position of the lower right corner point of the minimum rectangular frame in the pixel coordinate system respectively;

[0033] The expression of the pixel position of the center point of the tracking target in the pixel coordinate system is:

[0034]

[0035] Among them, [u,v] T They are the horizontal and vertical coordinate values ​​of the pixel position of the tracking target center point in the pixel coordinate system.

[0036] Optionally, the specific steps of step 7 are:

[0037] Step 71: Obtain the image device coordinate system positions of the tracking target in the UAV and the unmanned vehicle respectively, expressed as:

[0038]

[0039] Where K is the intrinsic parameter matrix of the image device; [x,y,1] T is the position of the tracking target in the image device coordinate system; z is the vertical distance between the tracking target and the image device;

[0040] Step 72: Based on the image device coordinate system position of the tracking target in the drone, obtain the drone view coordinate position of the tracking target in the world coordinate system. The expression is:

[0041]

[0042] Among them, [X a ,Y a ] T To track the target's coordinate position from the drone's perspective in the world coordinate system; [Xuav ,Y uav ] T is the coordinate position of the UAV in the world coordinate system; [x uav ,y uav ] T is the position of the tracking target in the image device coordinate system of the UAV obtained based on step 71; R wa is the transformation matrix from the image device coordinate system to the drone coordinate system;

[0043] Based on the image device coordinate system position of the tracking target in the unmanned vehicle, the coordinate position of the tracking target in the unmanned vehicle's viewpoint in the world coordinate system is obtained. The expression is:

[0044]

[0045] Among them, [X g ,Y g ] T To track the coordinate position of the target from the unmanned vehicle’s perspective in the world coordinate system; [X ugv ,Y ugv ] T is the coordinate position of the unmanned vehicle in the world coordinate system; [x ugv ,y ugv ] T is the position of the tracking target in the image device coordinate system of the unmanned vehicle obtained based on step 71; R wg is the transformation matrix from the image device coordinate system to the unmanned vehicle coordinate system.

[0046] Optionally, in step 8, for the tracking control of the UAV based on visual servoing, the expression is:

[0047]

[0048]

[0049] in, represents the control output of the UAV at time t, preferably, represents the speed control amount in the X direction and the speed control amount in the Y direction of the UAV at time t respectively; e uav (t) represents the tracking error of the UAV tracking the target at time t; K represents the tracking differential error of the UAV tracking the target at time t; p and K d They represent the control gains respectively;

[0050] For the tracking control of the unmanned vehicle based on visual servoing, the expression is:

[0051]

[0052]

[0053] in, and represents the control output of the unmanned vehicle at time t, preferably, represents the linear velocity control amount and the Z-axis angular velocity control amount of the unmanned vehicle at time t respectively; e ugv (t) represents the tracking error of the unmanned vehicle tracking the target; Tracking differential error of an unmanned vehicle tracking a target.

[0054] Optionally, in step 9, when the tracking targets in the field of view of the imaging devices of the drone and the unmanned vehicle are not obscured, the drone and the unmanned vehicle respectively perform separate target tracking tasks according to the tracking control of the drone and the unmanned vehicle obtained in step 8; when the tracking target in the field of view of the imaging devices of the drone or the unmanned vehicle is obscured, the intelligent agent that has not lost the tracking target continues to track the target, and sends the tracking target position obtained in the world coordinate system to the intelligent agent that has lost the tracking target; the intelligent agent that has lost the tracking target re-autonomously navigates to the tracking target position based on the tracking target position in the world coordinate system provided by the intelligent agent that has not lost the tracking target, and repositions to the tracking target.

[0055] Compared with the prior art, the present invention has at least the following beneficial effects:

[0056] 1. The air-ground collaborative tracking method of the present invention can take advantage of the land-air collaborative perspective and maintain the tracking status of the target in the field of view during the tracking process. The land and air platforms each have the ability to accurately identify the target when it is partially obscured, and find the target and continue tracking when the target is lost.

[0057] 2. The single target tracking method carried by the air-to-ground intelligent platform in the present invention has high accuracy, small number of parameters, and fast result acquisition speed, which can realize real-time tracking of the mobile intelligent body's airborne equipment; and the method of the present invention can locate the target when the target is partially blocked, and relocate the target when the target appears in the field of view again after being completely blocked.

[0058] 3. The method of the present invention enhances the robustness of target tracking under interference such as target deformation, occlusion, and background similarity.

[0059] 4. The method of the present invention selects the target to be tracked in the initial frame of the camera, uses the selected area as the target template, and continuously tracks the target in subsequent frames based on the obtained target template, and can output the specific position of the target in the image in real time and accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The drawings are only for purposes of illustrating particular embodiments and are not to be considered limiting of the invention.

[0061] Figure 1 This is an overall flow chart of the air-ground collaborative tracking method of the present invention;

[0062] Figure 2 This is a flow chart of the single target tracking method of the present invention;

[0063] Figure 3 This is the tracking control based on visual servoing in the present invention;

[0064] Figure 4 This is a schematic diagram of the result of tracking a target using the air-ground collaborative tracking method of the present invention. DETAILED DESCRIPTION

[0065] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. In addition, the present invention can also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited by the specific embodiments disclosed below.

[0066] The air-ground tracking method of this invention uses air-ground intelligent agents, including drones and unmanned vehicles, to track targets. During the tracking process, both drones and unmanned vehicles, using a Transformer-based single-target tracking method, possess autonomous tracking capabilities in complex environments and independently complete target tracking tasks. Furthermore, if a drone or unmanned vehicle loses its target, it can autonomously navigate to the target location and relocate the target, achieving collaborative tracking.

[0067] A specific embodiment of the present invention, as Figure 1-4 , discloses a Transformer network-based collaborative air-ground tracking method for occluded targets. The specific steps are as follows:

[0068] Step 1: Build a target tracking network model; deploy the target tracking network model on drones and unmanned vehicles;

[0069] Get the training dataset;

[0070] Establish a training loss function, the expression is:

[0071] loss = w1GIoU + w2L1;

[0072] Where w1 and w2 are the weights of the GIoU loss function and the L1 loss function, respectively.

[0073] Among them, the expression of GIoU loss function is:

[0074]

[0075] Where A and B are the accurate box and predicted box respectively; C is the smallest box containing the accurate box A and the predicted box B;

[0076] It can be understood that the accurate box is the smallest rectangular box containing the tracked target, and the predicted box is the rectangular box output by the target tracking network model.

[0077] The expression of L1 loss function is:

[0078]

[0079] Among them, x i and y i are the values ​​of the i-th corner point coordinates of the predicted box and the accurate box respectively;

[0080] Train the target tracking network based on the training dataset and training loss function to obtain the target tracking network model. The target tracking network model includes the VGG feature extraction network, the Transformer feature enhancement network, the prediction network and network parameters;

[0081] Use the ONNX model to optimize the target tracking network model; the optimized model can improve the real-time performance of target tracking and reduce the occupation of airborne equipment resources.

[0082] The optimized target tracking network model is deployed on drones and unmanned vehicles.

[0083] Step 2: Select the tracking target in the initial frame images of the UAV and the unmanned vehicle respectively; the target area where the selected tracking target is located is used as the target template;

[0084] Step 3: Determine the target search area of ​​the next frame of image based on the target area where the tracking target is located in the previous frame of image; splice the target template and the search area to obtain a spliced ​​feature vector;

[0085] Step 4: Use the VGG feature extraction network of the target tracking network model to extract the features of the spliced ​​feature vector to obtain the preliminary feature vector;

[0086] Step 5: Use the Transformer feature enhancement network of the target tracking network model to extract the preliminary feature vector to obtain the enhanced feature vector;

[0087] The encoder structure of the Transformer feature enhancement network is a normalization layer, a multi-head self-attention layer, a residual connection layer, a normalization layer, a fully connected layer and a residual connection layer; the present invention adopts a one-layer encoder, which can effectively improve real-time performance.

[0088] Step 6: Use the prediction network of the target tracking network model to process the enhanced feature vector to obtain the pixel position of the minimum rectangular box containing the tracked target in the current frame;

[0089] The prediction network is a multi-layer fully connected network, including a three-layer network structure, and each layer of the network structure includes a linear layer, a normalization layer and an activation layer.

[0090] Optionally, the pixel position of the minimum rectangular frame containing the tracking target in the current frame includes the pixel positions of the upper left corner and the lower right corner of the minimum rectangular frame, and the pixel position of the center point of the tracking target;

[0091] The pixel positions of the upper left corner and the lower right corner of the minimum rectangular frame are expressed in the pixel coordinate system as follows:

[0092] [u1,v1,u2,v2];

[0093] Wherein, u1 and v1 represent the horizontal coordinate value and vertical coordinate value of the pixel position of the upper left corner point of the minimum rectangular frame in the pixel coordinate system respectively; u2 and v2 represent the horizontal coordinate value and vertical coordinate value of the pixel position of the lower right corner point of the minimum rectangular frame in the pixel coordinate system respectively;

[0094] The expression of the pixel position of the center point of the tracking target in the pixel coordinate system is:

[0095]

[0096] Among them, [u,v] T They are the horizontal and vertical coordinate values ​​of the pixel position of the tracking target center point in the pixel coordinate system.

[0097] Step 7: Convert the pixel position of the tracking target from the pixel coordinate system to the world coordinate system;

[0098] Step 71: Obtain the image device coordinate system positions of the tracking target in the UAV and the unmanned vehicle respectively, expressed as:

[0099]

[0100] Where K is the intrinsic parameter matrix of the image device; [x,y,1] T is the position of the tracking target in the image device coordinate system, that is, the relative position between the image device and the tracking target; z is the vertical distance between the tracking target and the image device.

[0101] Preferably, the imaging device is a camera, and the imaging device coordinate system is a camera coordinate system.

[0102] Step 72: Based on the image device coordinate system position of the tracking target in the drone, obtain the drone view coordinate position of the tracking target in the world coordinate system. The expression is:

[0103]

[0104] Among them, [X a ,Y a ] T To track the target's coordinate position from the drone's perspective in the world coordinate system; [X uav ,Y uav ] T is the coordinate position of the UAV in the world coordinate system; [x uav ,y uav ] T is the position of the tracking target in the image device coordinate system of the UAV obtained based on step 71; R wa is the transformation matrix from the image device coordinate system to the drone coordinate system.

[0105] Based on the image device coordinate system position of the tracking target in the unmanned vehicle, the coordinate position of the tracking target in the unmanned vehicle's viewpoint in the world coordinate system is obtained. The expression is:

[0106]

[0107] Among them, [X g ,Y g ] T To track the coordinate position of the target from the unmanned vehicle’s perspective in the world coordinate system; [X ugv ,Y ugv ] T is the coordinate position of the unmanned vehicle in the world coordinate system; [x ugv ,y ugv ] T is the position of the tracking target in the image device coordinate system of the unmanned vehicle obtained based on step 71; R wg is the transformation matrix from the image device coordinate system to the unmanned vehicle coordinate system.

[0108] Step 8: Figure 3 As shown, the coordinate positions of the tracking target and the air-ground intelligent body in the world coordinate system obtained in step 7 are used to track and control the UAV and the unmanned vehicle based on visual servoing;

[0109] For the tracking control of the UAV based on visual servoing, the expression is:

[0110]

[0111]

[0112] in, represents the control output of the UAV at time t, preferably, represents the speed control amount in the X direction and the speed control amount in the Y direction of the UAV at time t respectively; e uav (t) represents the tracking error of the UAV tracking the target at time t; K represents the tracking differential error of the UAV tracking the target at time t; p and K d They represent the control gains respectively.

[0113] For the tracking control of the unmanned vehicle based on visual servoing, the expression is:

[0114]

[0115]

[0116] in, and represents the control output of the unmanned vehicle at time t, preferably, represents the linear velocity control amount and the Z-axis angular velocity control amount of the unmanned vehicle at time t respectively; e ugv (t) represents the tracking error of the unmanned vehicle tracking the target; Tracking differential error of the unmanned vehicle tracking the target; K p and K d They represent the control gains respectively.

[0117] Step 9: Based on the tracking control of the UAV and the unmanned vehicle obtained in step 8, individual target tracking and air-ground collaborative tracking control are performed;

[0118] During the target tracking process, when the tracking targets in the field of view of the imaging devices of the UAV and the unmanned vehicle are not obscured, the UAV and the unmanned vehicle respectively autonomously complete the individual target tracking tasks according to steps 1-8; when the tracking target in the field of view of the imaging devices of the UAV or the unmanned vehicle is obscured, the other intelligent agent continues to track the target and sends the tracking target position in the world coordinate system obtained by the method of step 7 to the intelligent agent that has lost the tracking target; the intelligent agent that has lost the tracking target autonomously navigates to the tracking target position again according to the tracking target position in the new world coordinate system and repositions to the tracking target.

[0119] It is understood that after the drone and unmanned vehicle are activated, they autonomously perform target tracking tasks according to steps 1-8 and transmit the target location to the local area network via ROS and WiFi. If the drone or unmanned vehicle loses the target, it can receive the target location transmitted by the unmanned vehicle / drone within the local area network, autonomously navigate to the vicinity of the target, and re-identify the target. Once the target is re-identified, it stops receiving target location information from the local area network and instead autonomously performs the target tracking task.

[0120] In order to illustrate the effectiveness of the method proposed in the present invention, the above technical solution of the present invention is described in detail below through a specific embodiment, including two platforms, a drone and an unmanned vehicle. The specific implementation steps are as follows:

[0121] (1) The drone and the unmanned vehicle each independently initiate a target tracking algorithm. This target tracking algorithm is a Transformer-based single-target tracking algorithm that can identify targets under partial occlusion. It also features high accuracy, a small number of parameters, and fast computation. After acquiring the camera's video frame, manually select the initial target. The algorithm calculates and outputs the pixel position of the target in subsequent frames in real time.

[0122] (2) If both the air-ground platform and the ground platform detect the target, the position is calculated and tracking control based on visual servoing is performed to continuously track the target.

[0123] (3) If the UAV loses the target due to visual occlusion, it receives the target location sent by the unmanned vehicle in the local area network. The UAV autonomously navigates to the vicinity of the target location and re-identifies the target based on the target tracking algorithm.

[0124] (4) If the unmanned vehicle loses the target due to visual occlusion, it receives the target location sent by the drone in the local area network. The unmanned vehicle autonomously navigates to the vicinity of the target location and re-identifies the target based on the target tracking algorithm.

[0125] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A Transformer network-based collaborative air-ground tracking method for occluded targets, characterized by: The specific steps are as follows: Step 1: Build a target tracking network model; deploy the target tracking network model on the UAV and unmanned vehicle of the air-ground collaborative intelligent body; the target tracking network model includes the VGG feature extraction network, the Transformer feature enhancement network and the prediction network; Step 2: Select the tracking target in the initial frame images of the UAV and the unmanned vehicle respectively; the target area where the selected tracking target is located is used as the target template; Step 3: Determine the target search area of ​​the next frame of image based on the target area where the tracking target is located in the previous frame of image; splice the target template and the search area to obtain a spliced ​​feature vector; Step 4: Use the VGG feature extraction network of the target tracking network model to extract the features of the spliced ​​feature vector to obtain the preliminary feature vector; Step 5: Use the Transformer feature enhancement network of the target tracking network model to extract the preliminary feature vector to obtain the enhanced feature vector; Step 6: Use the prediction network of the target tracking network model to process the enhanced feature vector to obtain the pixel position of the minimum rectangular box containing the tracked target in the current frame; Step 7: Convert the pixel position of the tracking target from the pixel coordinate system to the world coordinate system; Step 8: Using the coordinate positions of the tracking target and the air-ground intelligent body in the world coordinate system obtained in step 7, the tracking control of the UAV and the unmanned vehicle is performed based on visual servoing; Step 9: Based on the tracking control of the UAV and the unmanned vehicle obtained in step 8, individual target tracking and air-ground collaborative tracking control are performed.

2. The method for collaborative tracking of occluded targets in air and ground according to claim 1, characterized in that: The specific steps for building the target tracking model in step 1 are: Get the training dataset; Establish a training loss function, the expression is: loss = w1GIoU + w2L1; Where w1 and w2 are the weights of the GIoU loss function and the L1 loss function respectively; Among them, the expression of GIoU loss function is: Where A and B are the accurate box and predicted box respectively; C is the smallest box containing the accurate box A and the predicted box B; The expression of L1 loss function is: Among them, x i and y i are the values ​​of the i-th corner point coordinates of the predicted box and the accurate box respectively; The target tracking network model is obtained based on the training dataset and training loss function.

3. The method for collaborative tracking of occluded targets in air and ground according to claim 2, characterized in that: In step 1, the ONNX model is used to optimize the target tracking network model.

4. The method for collaborative tracking of occluded targets in air and ground according to claim 1, characterized in that: The encoder in the Transformer feature enhancement network includes a normalization layer, a multi-head self-attention layer, a residual connection layer, a normalization layer, a fully connected layer and a residual connection layer.

5. The method for collaborative tracking of occluded targets in air and ground according to claim 1, characterized in that: The prediction network is a multi-layer fully connected network, and each layer of the fully connected network includes a linear layer, a normalization layer and an activation layer.

6. The method for collaborative tracking of occluded targets in air and ground according to claim 1, characterized in that: The pixel positions of the minimum rectangular frame containing the tracking target in the current frame in step 6 include the pixel positions of the upper left corner and the lower right corner of the minimum rectangular frame, and the pixel position of the center point of the tracking target.

7. The method for collaborative tracking of occluded targets in air and ground according to claim 6, characterized in that: The pixel positions of the upper left corner and the lower right corner of the minimum rectangular frame are expressed in the pixel coordinate system as follows: [u1,v1,u2,v2]; Wherein, u1 and v1 represent the horizontal coordinate value and vertical coordinate value of the pixel position of the upper left corner point of the minimum rectangular frame in the pixel coordinate system respectively; u2 and v2 represent the horizontal coordinate value and vertical coordinate value of the pixel position of the lower right corner point of the minimum rectangular frame in the pixel coordinate system respectively; The expression of the pixel position of the center point of the tracking target in the pixel coordinate system is: Among them, [u,v] T They are the horizontal and vertical coordinate values ​​of the pixel position of the tracking target center point in the pixel coordinate system.

8. The method for collaborative air-ground tracking of an occluded target according to claim 7, characterized in that: The specific steps of step 7 are: Step 71: Obtain the image device coordinate system positions of the tracking target in the UAV and the unmanned vehicle respectively, expressed as: Where K is the intrinsic parameter matrix of the image device; [x,y,1] t is the position of the tracking target in the image device coordinate system; z is the vertical distance between the tracking target and the image device; Step 72: Based on the image device coordinate system position of the tracking target in the drone, obtain the drone view coordinate position of the tracking target in the world coordinate system. The expression is: Among them, [X a ,Y a ] T To track the target's coordinate position from the drone's perspective in the world coordinate system; [X uav ,Y uav ] T is the coordinate position of the UAV in the world coordinate system; [x uav ,y uav ] T is the position of the tracking target in the image device coordinate system of the UAV obtained based on step 71; R wa is the transformation matrix from the image device coordinate system to the drone coordinate system; Based on the image device coordinate system position of the tracking target in the unmanned vehicle, the coordinate position of the tracking target in the unmanned vehicle's viewpoint in the world coordinate system is obtained. The expression is: Among them, [X g ,Y g ] T To track the coordinate position of the target from the unmanned vehicle’s perspective in the world coordinate system; [X ugv ,Y ugv ] T is the coordinate position of the unmanned vehicle in the world coordinate system; [x ugv ,y ugv ] T is the position of the tracking target in the image device coordinate system of the unmanned vehicle obtained based on step 71; R wg is the transformation matrix from the image device coordinate system to the unmanned vehicle coordinate system.

9. The method for collaborative tracking of occluded targets in air and ground according to claim 8, characterized in that: In step 8, for the tracking control of the UAV based on visual servoing, the expression is: in, They represent the speed control amount in the X direction and the speed control amount in the Y direction of the UAV at time t respectively; e uav (t) represents the tracking error of the UAV tracking the target at time t; K represents the tracking differential error of the UAV tracking the target at time t; p and K d They represent the control gains respectively; For the tracking control of the unmanned vehicle based on visual servoing, the expression is: in, and They represent the linear velocity control value and Z-axis angular velocity control value of the unmanned vehicle at time t respectively; e ugv (t) represents the tracking error of the unmanned vehicle tracking the target; Tracking differential error of an unmanned vehicle tracking a target.

10. The method for collaborative air-ground tracking of an occluded target according to any one of claims 1 to 9, characterized in that: In step 9, when the tracking target in the field of view of the drone and the unmanned vehicle's imaging device is not obstructed, the drone and the unmanned vehicle respectively perform independent target tracking tasks according to the tracking control of the drone and the unmanned vehicle obtained in step 8; when the tracking target in the field of view of the drone or the unmanned vehicle's imaging device is obstructed, the intelligent agent that has not lost the tracking target continues to track the target and sends the obtained tracking target position in the world coordinate system to the intelligent agent that has lost the tracking target; The agent that has lost the tracking target will autonomously navigate to the tracking target position again based on the tracking target position in the world coordinate system provided by the agent that has not lost the tracking target, and reposition itself to the tracking target.

Images