A method, device and medium for tracking targets using unmanned aerial vehicles (UAVs).

Abstract

This application relates to the field of intelligent visual detection technology and discloses a method for UAV target tracking. The method involves acquiring a sequence of reference images within a specified time window, including multiple frames of reference images of the tracked object; performing target detection on each frame of the reference image to obtain a reference state vector of the tracked object; constructing a state vector sequence using the reference state vectors corresponding to each frame of the reference image; predicting the state vector of the tracked object at a predicted time based on the state vector sequence; and inputting the predicted state vector into a reinforcement learning policy network to generate control commands for the UAV at the current time, thereby controlling the UAV to track the tracked object. The reward function of the reinforcement learning policy network is determined based on the predicted state vectors obtained during the training process. The beneficial effect is that it achieves real-time fusion of target detection and UAV control, improving the accuracy and reliability of the UAV when tracking small targets.

Description

Technical Field

[0001] This application relates to the field of intelligent visual inspection technology, and in particular to a method, device and medium for tracking unmanned aerial vehicles (UAVs). Background Technology

[0002] With the development of intelligent vision inspection technology, Unmanned Aerial Vehicle (UAV) technology, with its flexibility, high maneuverability, and low cost, has been widely used in tasks such as UAV inspection, search and rescue, and monitoring to accurately detect and track ground or aerial targets in real time. It also has broad applications in environmental monitoring, infrastructure inspection, and disaster relief. In real-world scenarios, when performing target tracking tasks, UAVs need to simultaneously achieve clear imaging of small targets and maintain flight control. Furthermore, these small targets are often moving rapidly, posing a significant challenge to UAV tracking.

[0003] In related technologies, the reliability of drones tracking small targets still needs to be improved. Summary of the Invention

[0004] This application provides a method, device, and medium for tracking unmanned aerial vehicles (UAVs). By predicting the state vector of the tracked object at future moments and using a reinforcement learning network trained with the distribution state of the tracked object in the field of view as a reward function, control commands for the UAV are generated, thereby achieving real-time fusion of target detection and UAV control and improving the accuracy and reliability of the UAV when tracking small targets.

[0005] To achieve the above objectives, the main technical solutions adopted in this application include:

[0006] In a first aspect, embodiments of this application provide a method for tracking unmanned aerial vehicle (UAV) targets, the method comprising:

[0007] Obtain a reference image sequence within a specified time window; wherein, the reference image sequence includes multiple frames of reference images about the tracked object; the last moment of the specified time window is the current moment;

[0008] Target detection is performed on each frame of reference image to obtain the reference state vector of the tracked object, and a state vector sequence is constructed using the reference state vectors corresponding to each frame of reference image; wherein, the reference state vector is used to represent the actual distribution state of the tracked object in each frame of reference image;

[0009] The state vector of the tracked object at the prediction time is predicted based on the state vector sequence to obtain the predicted state vector of the tracked object; wherein, the predicted state vector is used to represent the predicted distribution state of the tracked object in the current field of view of the UAV at the prediction time; the prediction time is the time after the current time and the prediction time step.

[0010] The predicted state vector is input into the reinforcement learning policy network to generate control commands for the UAV at the current moment, so as to control the UAV to track the target object; wherein, the reward function of the reinforcement learning policy network is determined based on the predicted state vector during the training process.

[0011] The UAV target tracking method proposed in this application obtains the state vector of the tracked object in these reference images through target detection based on a sequence of reference images of the tracked object within a specified time window. This state vector is then used as a prediction basis to predict the state vector of the tracked object at the predicted time, resulting in a predicted state vector. Based on this, the predicted state vector is used as a guiding target, and a reinforcement learning network trained with the state vector as a reward function is used to generate control commands for the UAV. Compared with related technologies, this application determines the reward function based on the state vector and uses it to train the reinforcement learning policy network. This allows the real-time distribution state of the tracked object to directly influence the generation process of control commands, achieving the fusion and coordination of target tracking and flight control tasks, effectively improving the tracking accuracy and reliability of the UAV when performing target tracking tasks. Furthermore, this application predicts the state vector of the tracked object at future times and uses this as a guiding target to generate control commands corresponding to the state vector at future times, thereby achieving forward-looking decision-making for the UAV and improving the UAV's dynamic tracking capability for highly maneuverable targets.

[0012] Optionally, the step of inputting the predicted state vector into the reinforcement learning policy network to generate the control command for the UAV at the current moment includes:

[0013] The predicted state vector is calculated by forward propagation through the reinforcement learning policy network, and the control command of the UAV at the current moment is output.

[0014] The control commands include flight control commands for the UAV and angle change commands for the camera mounted on the UAV.

[0015] Optionally, the predicted state vector during the training process includes the first predicted position, first predicted area, and first predicted confidence of the tracked object at the predicted time; the reward function is obtained in the following manner:

[0016] Based on the first predicted position, the center of the field of view is aligned to obtain a reward item for the predicted observation alignment.

[0017] Pixel area comparison is performed based on the first predicted area to obtain the predicted observation scale reward item;

[0018] Based on the distance data between the first predicted location, the first predicted area, the first predicted confidence level, and the actual state vector of the tracked object at the predicted time, a prediction accuracy reward is obtained.

[0019] The reward function is obtained by fusing the predicted observation alignment reward, the predicted observation scale reward, and the predicted accuracy reward.

[0020] Optionally, the step of aligning the view center based on the first predicted position to obtain a predicted observation alignment reward includes:

[0021] Using the first predicted position, center comparisons are performed in the first and second dimensions of the UAV's field of view to obtain the predicted observation alignment reward item that incentivizes the UAV to align the tracked object with the center of the field of view.

[0022] Optionally, the step of performing pixel area comparison based on the first predicted area to obtain a predicted observation scale reward includes:

[0023] The first predicted area is compared with the expected pixel area to obtain the predicted observation scale reward, which incentivizes the UAV to adjust its flight altitude so that the pixel area of ​​the tracked object in the field of view is close to the expected pixel area.

[0024] Optionally, the step of predicting the state vector of the tracked object at the prediction time based on the state vector sequence to obtain the predicted state vector of the tracked object includes:

[0025] The state vector sequence is input into a temporal state prediction network, and the temporal state prediction network performs pattern learning on the state vector sequence to output the predicted state vector; wherein, the temporal state prediction network includes a long short-term memory network.

[0026] Optionally, the method further includes:

[0027] Obstacle detection is performed on the environment in which the drone is located to obtain the detection distance between the drone and the obstacle;

[0028] When the detection distance is less than the safe distance threshold, obstacle avoidance path planning is performed on the UAV to obtain the obstacle avoidance command of the UAV;

[0029] The control commands of the UAV are replaced with the obstacle avoidance commands to control the UAV to avoid obstacles until the detection distance exceeds the safe distance threshold.

[0030] Optionally, the reference state vector includes the detection confidence of the tracked object in each frame of the reference image; the method further includes:

[0031] If the detection confidence of the tracked object is lower than the confidence threshold in multiple consecutive reference images, multiple consecutive moments before the detection confidence falls below the confidence threshold are selected as inferred moments to obtain an effective image sequence of the tracked object at the inferred moments; wherein, the effective image sequence includes multiple frames of effective images of the tracked object;

[0032] Based on the state vector of the tracked object in the valid image, the inferred state sequence of the tracked object is obtained;

[0033] Based on the inferred state sequence, the trajectory of the tracked object is inferred to obtain the predicted trajectory of the tracked object, and the search command of the UAV is generated based on the predicted trajectory to control the UAV to detect the tracked object.

[0034] Secondly, embodiments of this application provide a drone target tracking device, the device comprising:

[0035] An image sequence acquisition module is used to acquire a reference image sequence within a specified time window; wherein, the reference image sequence includes multiple frames of reference images about the tracked object; the last moment of the specified time window is the current moment;

[0036] The target detection module is used to perform target detection on each frame of reference image to obtain the reference state vector of the tracked object, and to construct a state vector sequence using the reference state vectors corresponding to each frame of reference image; wherein, the reference state vector is used to represent the actual distribution state of the tracked object in each frame of reference image;

[0037] The state vector prediction module is used to predict the state vector of the tracked object at the prediction time based on the state vector sequence, so as to obtain the predicted state vector of the tracked object; wherein, the predicted state vector is used to represent the predicted distribution state of the tracked object in the current field of view of the UAV at the prediction time; the prediction time is the time after the current time and the prediction time step.

[0038] The reinforcement learning decision module is used to input the predicted state vector into the reinforcement learning policy network to generate control commands for the UAV at the current moment, so as to control the UAV to track the target object; wherein, the reward function of the reinforcement learning policy network is determined based on the predicted state vector during the training process.

[0039] Thirdly, embodiments of this application provide a computer device, including: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the method described in any of the above embodiments.

[0040] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions, which are used to cause a computer to perform the method described in any one of the above embodiments.

[0041] Fifthly, embodiments of this application provide a computer program product, including computer instructions, which are used to cause a computer to perform the method described in any of the above embodiments. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0043] Figure 1 A flowchart illustrating the steps of the UAV target tracking method provided in this application embodiment;

[0044] Figure 2 This is a flowchart illustrating the steps involved in obtaining the reward function in an embodiment of this application.

[0045] Figure 3 This is a diagram illustrating the steps of controlling a drone to avoid obstacles in an embodiment of this application;

[0046] Figure 4 This is a system flowchart of the UAV in the embodiments of this application;

[0047] Figure 5 This is a diagram illustrating the steps of speculative search in the event of object loss in an embodiment of this application;

[0048] Figure 6 A block diagram of the unmanned aerial vehicle (UAV) target tracking device provided in the embodiments of this application;

[0049] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] With the development of intelligent vision inspection technology, Unmanned Aerial Vehicle (UAV) technology, with its flexibility, high maneuverability, and low cost, has been widely used in tasks such as UAV inspection, search and rescue, and monitoring to accurately detect and track ground or aerial targets in real time. It also has broad applications in environmental monitoring, infrastructure inspection, and disaster relief. In real-world scenarios, when performing target tracking tasks, UAVs need to simultaneously achieve clear imaging of small targets and maintain flight control. Furthermore, these small targets are often moving rapidly, posing a significant challenge to UAV tracking.

[0052] In related technologies, reactive control strategies are typically used to control UAVs to perform target tracking tasks. This involves generating control commands based on the detected state of the target object to adjust the UAV's flight status or field of view. Because real-time calculations are required, this control strategy suffers from lag, often limiting tracking accuracy and leading to target loss when tracking highly maneuverable targets such as fast-moving vehicles. Furthermore, these technologies usually execute UAV flight control and target detection separately, resulting in independent tasks that cannot be effectively integrated. In reality, the UAV's flight status significantly impacts target detection accuracy.

[0053] To address the aforementioned issues, this application provides a method, device, and medium for UAV target tracking. The method involves acquiring a sequence of reference images within a specified time window, including multiple frames of reference images of the tracked object; performing target detection on each frame of reference images to obtain a reference state vector of the tracked object; constructing a state vector sequence using the reference state vectors corresponding to each frame of reference images; predicting the state vector of the tracked object at a predicted time based on the state vector sequence; and inputting the predicted state vector into a reinforcement learning policy network to generate control commands for the UAV at the current time, thereby controlling the UAV to track the tracked object. The reward function of the reinforcement learning policy network is determined based on the predicted state vectors obtained during the training process.

[0054] The UAV target tracking method provided in this application embodiment obtains the state vector of the tracked object in these reference images through target detection based on the reference image sequence of the tracked object within a specified time window, and uses this as a prediction basis to predict the state vector of the tracked object at the prediction time to obtain the predicted state vector; on this basis, the predicted state vector is used as the guiding target, and a reinforcement learning network trained with the state vector as the reward function is used to generate control commands for the UAV.

[0055] Compared with related technologies, this application determines the reward function based on the state vector and uses it to train the reinforcement learning policy network, so that the real-time distribution state of the tracked object directly affects the generation process of control commands, realizing the integration and coordination of target tracking tasks and flight control tasks, and effectively improving the tracking accuracy and reliability of UAVs when performing target tracking tasks.

[0056] Furthermore, this application predicts the state vector of the tracked object at a future time and uses it as a guiding target to generate control commands corresponding to the state vector at the future time, thereby achieving forward-looking decision-making for the UAV and improving the UAV's dynamic tracking capability for highly maneuverable targets.

[0057] The drone target tracking method provided in this specification can be applied to drones equipped with target acquisition and detection capabilities to control the drone to track objects. This drone may include multi-rotor drones, fixed-wing drones, vertical takeoff and landing fixed-wing drones, or ornithopter drones, etc. It is understood that, with adaptive modifications, the drone target tracking method provided in this specification can also be used to control other similar flight devices to perform target tracking.

[0058] According to an embodiment of this application, an embodiment of a UAV target tracking method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0059] This embodiment provides a drone target tracking method, which can be used in the aforementioned drones equipped with target acquisition and detection functions. (Refer to...) Figure 1 As shown, the method includes:

[0060] S100. Obtain a reference image sequence within a specified time window; wherein, the reference image sequence includes multiple frames of reference images about the tracked object; the last moment of the specified time window is the current moment.

[0061] S200. Perform target detection on each frame of reference image to obtain the reference state vector of the tracked object, and use the reference state vector corresponding to each frame of reference image to form a state vector sequence; wherein, the reference state vector is used to represent the actual distribution state of the tracked object in each frame of reference image.

[0062] S300. Based on the state vector sequence, predict the state vector of the tracked object at the prediction time to obtain the predicted state vector of the tracked object; wherein, the predicted state vector is used to represent the predicted distribution state of the tracked object in the current field of view of the UAV at the prediction time; the prediction time is the time after the prediction time step after the current time.

[0063] S400. The predicted state vector is input into the reinforcement learning policy network to generate control commands for the UAV at the current moment, so as to control the UAV to track the target object; wherein, the reward function of the reinforcement learning policy network is determined based on the predicted state vector during the training process.

[0064] Specifically, the drone comprises a drone platform, a gimbal camera, and a computing platform. The drone platform is the main structure of the drone, primarily responsible for its flight and movement. For example, the drone platform can be a multi-rotor drone with good maneuverability and payload capacity, such as a quadcopter or hexacopter, whose built-in flight control system can receive and execute control commands from the computing platform. The gimbal camera can be a camera mounted on a three-axis gimbal, which can rotate in three directions, providing stability for the camera during flight and flexible angle control during target tracking. The computing platform contains multiple computing units responsible for running various algorithms during target tracking. For example, the computing platform can be a high-performance, low-power embedded AI computing unit, such as the NVIDIA Jetson series platform.

[0065] During flight, the drone acquires images within its field of view in real time via its camera, obtaining multiple real-time images corresponding to various acquisition times. These real-time images include the drone's surrounding environment and any potential tracking objects. When a tracking object is present in the real-time images, these images are used as reference images, serving as the data basis for generating control commands for the drone. Based on this, a specified time window is defined, and multiple reference images within that window are acquired to obtain a reference image sequence, which is then used to track moving objects. It should be noted that the specified time window includes the current moment and several consecutive moments preceding it, with the current moment being the last moment of the specified time window. The tracking object is present in the reference image corresponding to each moment within the specified time window.

[0066] Furthermore, target detection is performed on each frame of the reference image sequence. Based on the distribution of the tracked object in the reference image, a reference state vector for the tracked object in each frame is determined. Target detection can be achieved by inputting the reference image sequence into a deep learning target detection network on the onboard computing platform of the UAV. The deep learning target detection network performs real-time analysis of the reference image sequence and outputs a state vector sequence. It should be noted that the reference state vector can include the reference detection position, reference detection area, and reference detection confidence of the tracked object in the reference image. The reference detection position can be the detected location of the tracked object in the reference image, including the center coordinates of the tracked object in any dimension of the reference image, with the origin of the coordinates potentially being the upper left corner of the reference image. The reference detection area can be the normalized pixel area occupied by the bounding box corresponding to the tracked object in the reference image, which can be used to determine the distance between the UAV and the tracked object and the sharpness of the tracked object in the reference image. The reference detection confidence is the similarity between the detected tracked object and the real object in the reference image; it is a scalar between 0 and 1, representing the probability that the detected tracked object in the reference image is a real object.

[0067] For example, the reference state vector can be represented by the following formula:

[0068]

[0069] in, Represents the reference state vector; The reference detection position for the tracked object; The reference detection area for the tracked object; This serves as the reference detection confidence level for the tracked object. The resulting state vector sequence can be represented by the following formula:

[0070]

[0071] in, This represents a sequence of state vectors.

[0072] Furthermore, the motion pattern of the tracked object within a specified time window is learned based on the state vector sequence, and the distribution state of the tracked object after the predicted time step is predicted based on the results of the pattern learning, resulting in the predicted state vector of the tracked object at the predicted time. It should be noted that the predicted state vector represents the predicted distribution state of the tracked object in the current field of view of the UAV at the predicted time, and is used as the basis for generating UAV control commands. It can be understood that by predicting the distribution state of the tracked object in the UAV's field of view in advance, a clear aiming target can be provided to the UAV, thereby generating UAV control commands with the aiming target aligned. This overcomes the inherent delay in real-time target detection and command generation, improves the UAV's adaptability to complex dynamic environments, and significantly improves the tracking accuracy of the tracked object.

[0073] In some embodiments, the predicted state vector includes the predicted tracking position, predicted tracking area, and predicted tracking confidence of the tracked object at the prediction time. The predicted tracking position can be the detection position of the tracked object in the UAV's field of view at the prediction time, including the center coordinates of the tracked object in any dimension of the UAV's field of view, with the origin of these coordinates potentially being the upper left corner of the UAV's field of view. The predicted tracking area can be the normalized pixel area occupied by the bounding box corresponding to the tracked object in the UAV's field of view at the prediction time, and can be used to determine the distance between the UAV and the tracked object and the clarity of the tracked object in the UAV's field of view. The predicted tracking confidence can be the similarity between the tracked object detected in the UAV's field of view and the real object at the prediction time; it is a scalar ranging from 0 to 1, representing the probability that the tracked object detected in the UAV's field of view is a real object.

[0074] Further, after obtaining the predicted state vector, the predicted state vector is input into the reinforcement learning policy network. Based on the pre-trained reinforcement learning algorithm, control commands for the UAV at the current moment are generated to control the UAV to track the target object. The reinforcement learning policy network is trained offline in a digital simulation environment before tracking the target object. The digital simulation environment can be a high-fidelity virtual environment to ensure the efficiency and repeatability of the training process. In some embodiments, the digital simulation environment may include diverse virtual scenes, dynamic virtual targets, and UAV behavior models. Diverse virtual scenes can be used to simulate various obstacles in real-world situations, such as buildings and trees. Dynamic virtual targets can be virtual tracking objects with various complex motion modes, capable of movement such as acceleration, deceleration, turning, or S-shaped maneuvers. The UAV behavior model includes a physical model and a sensor model for simulating UAV behavior. The physical model can be a model with the same aerodynamic characteristics as a real UAV, simulating the UAV's flight state. The sensor model can be a model consistent with the distribution of real sensor data, simulating the UAV's behavior in perceiving its surrounding environment.

[0075] During training, reinforcement learning algorithms based on the Actor-Critic framework can be employed, including but not limited to Deep Deterministic Policy Gradient (DDPG) algorithms. The Actor network, acting as the decision-making network, uses a drone as the training agent to simulate multi-round interactions between the agent and the digital simulation environment, generating control commands for each interaction round. The Critic network, acting as the critic network, calculates the reward for the control commands output by the Actor network according to a pre-defined reward function, determining the cumulative reward for that interaction round. Based on the cumulative reward, the network parameters of the reinforcement learning policy network are adjusted, enabling the network to gradually provide better control commands.

[0076] It should be noted that the reward function is determined based on the predicted state vector during training. In each interactive round, the UAV performs the aforementioned steps of image acquisition, target detection, state vector prediction, and control command generation. It perceives diverse virtual scenes through a sensor model and obtains the predicted state vector based on the perception results. The reward function is related to the data in the predicted state vector. The cumulative reward is calculated based on the predicted state vector and the reward function to directly evaluate and adjust the UAV's aiming capability at dynamic virtual targets in the digital simulation environment, thereby improving the UAV's ability to target the tracked object.

[0077] Understandably, after receiving the control command from the drone at the current moment and controlling the drone according to the control command, the process of generating the aforementioned control command is repeated at the next moment to obtain the control command for the next moment. During the drone's tracking of the target object, the above process is repeated to continuously aim at the target object until the target tracking task is completed.

[0078] The UAV target tracking method provided in this embodiment obtains the state vector of the tracked object in these reference images through target detection based on the reference image sequence of the tracked object within a specified time window, and uses this as a prediction basis to predict the state vector of the tracked object at the prediction time to obtain the predicted state vector; on this basis, the predicted state vector is used as the guiding target, and a reinforcement learning network trained with the state vector as the reward function is used to generate control commands for the UAV.

[0079] Compared with related technologies, this application determines the reward function based on the state vector and uses it to train the reinforcement learning policy network, so that the real-time distribution state of the tracked object directly affects the generation process of control commands, realizing the integration and coordination of target tracking tasks and flight control tasks, and effectively improving the tracking accuracy and reliability of UAVs when performing target tracking tasks.

[0080] Furthermore, this application predicts the state vector of the tracked object at a future time and uses it as a guiding target to generate control commands corresponding to the state vector at the future time, thereby achieving forward-looking decision-making for the UAV and improving the UAV's dynamic tracking capability for highly maneuverable targets.

[0081] As one embodiment of this application, the predicted state vector is input into a reinforcement learning policy network to generate control commands for the UAV at the current moment, including:

[0082] S410. The predicted state vector is calculated by forward propagation through a reinforcement learning policy network, and the control commands of the UAV at the current moment are output; wherein, the control commands include the flight control commands of the UAV and the angle change commands of the camera on the UAV.

[0083] Specifically, the predicted state vector is input into the reinforcement learning policy network. The Actor network within the policy network performs forward computation on the predicted state vector, outputting control commands to control the UAV to track the target object. These control commands include flight control commands for the UAV and angle change commands for the onboard camera. The flight control commands control the UAV's movement speed in multiple directions, while the angle change commands control the camera's orientation. Through the coordination between the UAV's position and the camera angle, the UAV can aim at the target object in real time and achieve clear imaging of the target object within its field of view, thus enabling real-time tracking.

[0084] For example, control commands can be represented by the following formula:

[0085]

[0086] in, express Control commands for the drone at any time; This is the command for the forward flight speed of the UAV in the horizontal plane in the body coordinate system, in meters per second (m / s). This is the lateral flight speed command for the UAV in the horizontal plane of the body coordinate system, expressed in meters per second (m / s). This is the command for the rate of change of altitude of the UAV, in meters per second (m / s). This is the command for the rate of change of the pitch angle of the gimbal camera mounted on the drone, in degrees per second (° / s). This is the command for the rate of change of the yaw angle of the gimbal camera mounted on the drone, in degrees per second (° / s).

[0087] Reference Figure 2 As shown in one embodiment of this application, the predicted state vector during training includes the first predicted position, first predicted area, and first predicted confidence of the tracked object at the prediction time; the reward function is obtained in the following manner:

[0088] S420. Align the field of view center based on the first predicted position to obtain the prediction observation alignment bonus.

[0089] S430. Pixel area comparison is performed based on the first predicted area to obtain the predicted observation scale reward item.

[0090] S440. Based on the distance data between the first predicted position, the first predicted area, the first predicted confidence level, and the actual state vector of the tracked object at the predicted time, a prediction accuracy reward is obtained.

[0091] S450. The reward function is obtained by integrating the reward terms for prediction observation alignment, prediction observation scale, and prediction accuracy.

[0092] The predicted state vector during training can be the result predicted by the state vector when the UAV is controlled by the UAV target tracking method provided in this specification to track a dynamic virtual target during the training of the reinforcement learning policy network. The predicted state vector during training includes the first predicted position, first predicted area, and first predicted confidence of the tracked object at the prediction time. The first predicted position can be the detection position of the dynamic virtual target in the UAV's field of view at the prediction time, including the center coordinates of the dynamic virtual target in any dimension of the UAV's field of view, with the origin of the coordinates possibly being the upper left corner of the UAV's field of view. The first predicted area can be the normalized pixel area occupied by the bounding box corresponding to the dynamic virtual target in the UAV's field of view at the prediction time, which can be used to determine the distance between the UAV and the dynamic virtual target and the clarity of the dynamic virtual target in the UAV's field of view. The first predicted confidence is the similarity between the detected dynamic virtual target and the real target in the UAV's field of view at the prediction time, which is a scalar between 0 and 1, representing the probability that the detected dynamic virtual target in the UAV's field of view is a real target.

[0093] Specifically, when controlling a drone to perform target tracking, in order to present the tracked object completely and accurately in the field of view, it is usually necessary to align the tracked object with the center of the drone's field of view and image the tracked object at an optimal scale to reduce interference and improve the accuracy of target tracking, ensuring that the details of the tracked object can be captured. Considering the above reasons, during the training process of the reinforcement learning policy network, a prediction observation alignment reward term is obtained based on the degree of alignment between the first predicted position and the center of the drone's field of view, enabling the reinforcement learning policy network to learn to place the tracked object at the center of the drone's field of view when controlling the drone. Simultaneously, a prediction observation scale reward term is obtained based on the scale difference between the first predicted area and the expected pixel area, enabling the reinforcement learning policy network to learn to maintain an appropriate distance between the drone and the tracked object when controlling the drone, improving the ability to capture the details of the tracked object.

[0094] Furthermore, based on the reinforcement learning policy network used to make decisions about UAV behavior, the accuracy of the predicted state vector also has a significant impact on the quality of control command generation. To address this, this embodiment first obtains the predicted state vector of the dynamic virtual target at the predicted time by predicting the state vector during training. This includes the first predicted position, first predicted area, and first predicted confidence of the tracked object at the predicted time. The training process continues until the current time reaches the predicted time. At this point, the actual reference image of the dynamic virtual target at that time is acquired through the sensor model, and the actual state vector of the dynamic virtual target in the actual reference image is obtained through target detection. Based on the first predicted position, first predicted area, and first predicted confidence in the predicted state vector, and the actual state vector, the distance between the predicted state vector and the actual state vector is calculated to determine the prediction error of the state vector, thus obtaining a prediction accuracy reward.

[0095] For example, the distance data between the predicted state vector and the actual state vector can be Euclidean distance, in which case the prediction accuracy reward term can be expressed as follows:

[0096]

[0097] in, Rewards for accurate predictions; This is the actual state vector; This is the predicted state vector during the training process; This represents the calculation of Euclidean distance. It can be understood that the smaller the distance between the predicted state vector and the actual state vector, the closer the predicted state vector is to the actual state vector during training, indicating higher accuracy in state vector prediction, and thus a higher reward represented by the prediction accuracy reward term.

[0098] It should be noted that the reinforcement learning policy network is also coupled to the temporal state prediction network used for state vector prediction. The reinforcement learning policy network's training results are fed back to the temporal state prediction network, so that the temporal state prediction network can be prompted to adjust its parameters through prediction accuracy rewards, thereby incentivizing the temporal state prediction network to make more accurate state vector predictions and improving the accuracy of state vector prediction.

[0099] In some embodiments, the reward function further includes an energy consumption and smoothing penalty term. This penalty term is used to evaluate the flight state of the UAV, suppressing excessive maneuvering and energy consumption during flight, and incentivizing the UAV to perform smooth and energy-efficient flight. The energy consumption and smoothing penalty term can be expressed as follows:

[0100]

[0101] in, This represents the energy consumption and smoothing penalty term; , , , and They are respectively The corresponding commands in the control instructions for the drone at any time; and This represents the corresponding penalty coefficient. It's understandable that the energy consumption and smoothing penalty terms are used to constrain the drone, preventing significant changes in its flight path or camera angle, and reducing energy consumption during tracking.

[0102] Furthermore, a reward function is obtained by fusing the reward terms for predicted observation alignment, predicted observation scale, predicted accuracy, and energy consumption and smoothing penalties. The reward function can be expressed as follows:

[0103]

[0104] in, express The reward function at each time step; The reward item is the alignment of the predicted observations. Rewards are given for predicting the scale of observations. , , and These are the corresponding weighting coefficients.

[0105] As one embodiment of this application, field-of-view center alignment is performed based on a first predicted position to obtain a predicted observation alignment reward, including:

[0106] S422. Using the first predicted position, perform center comparison in the first and second dimensions of the UAV's field of view to obtain a predicted observation alignment reward item that incentivizes the UAV to align the tracked object with the center of the field of view.

[0107] Specifically, the first predicted position includes the center coordinates of the dynamic virtual target in the first dimension and the center coordinates in the second dimension. The coordinates in the two dimensions are compared with the center of the UAV's field of view in the corresponding dimension to determine the distance error between the center of the first predicted position and the center of the UAV's field of view, and the prediction observation alignment bonus is obtained.

[0108] For example, the prediction observation alignment reward term can be represented by the following formula:

[0109]

[0110] in, To predict the center coordinates of the dynamic virtual target in the first dimension at the current time. To predict the center coordinates of the dynamic virtual target in the second dimension at the current time. The total length of the drone's field of view in the first dimension; The total length of the drone's field of view in the second dimension; For the predicted time; This is a hyperparameter for controlling the shape of the reward function. It can be understood that the prediction-observation alignment reward term, as shown in the above equation, is a Gaussian function. The prediction-observation alignment reward term reaches its maximum value when the center of the first predicted position coincides with the center of the UAV's field of view; as the deviation between the center of the first predicted position and the center of the UAV's field of view increases, the prediction-observation alignment reward term decays exponentially.

[0111] As one embodiment of this application, pixel area comparison is performed based on the first predicted area to obtain a predicted observation scale reward, including:

[0112] S432. Compare the first predicted area with the expected pixel area to obtain a prediction observation scale reward item that incentivizes the UAV to adjust its flight altitude so that the pixel area of ​​the tracked object in the field of view is close to the expected pixel area.

[0113] Specifically, the scale error between the first predicted area and the expected pixel area is calculated by performing a scale comparison, thus obtaining the prediction observation scale reward term. For example, the prediction observation scale reward term can be expressed by the following formula:

[0114]

[0115] in, For the predicted area; The desired pixel area; This is a hyperparameter for controlling the shape of the reward function. Understandably, the expected pixel area can be determined based on the needs of the target tracking task. When the first predicted area matches the expected pixel area, the prediction observation scale reward term reaches its maximum value; when the first predicted area is greater than or less than the expected pixel area, the prediction observation scale reward term gradually decreases.

[0116] As one embodiment of this application, the predicted state vector of the tracked object at the prediction time is predicted based on the state vector sequence, including:

[0117] S310. Input the state vector sequence into the temporal state prediction network, perform pattern learning on the state vector sequence through the temporal state prediction network, and output the predicted state vector; wherein, the temporal state prediction network includes a long short-term memory network.

[0118] Specifically, the temporal state prediction network is pre-trained offline using supervised learning methods before tracking the object. The training method for the temporal state prediction network can be to adaptively adjust its parameters based on the reward term for prediction accuracy feedback during the training of the reinforcement learning policy network. It is understood that the types of temporal state prediction networks that can be used in this embodiment include, but are not limited to, Long Short-Term Memory (LSTM) networks, Convolutional Neural Networks (CNNs), or Transformer architectures.

[0119] Furthermore, the state vector sequence is input into a temporal state prediction network. The network performs pattern learning on the state vector sequence and outputs a predicted state vector. The predicted state vector can be expressed as follows:

[0120]

[0121] in, Represents the predicted state vector; To predict the center coordinates of the dynamic virtual target in the first dimension at a given time, To predict the center coordinates of the dynamic virtual target in the second dimension at the current time. The prediction width for tracking the dynamic virtual target at the prediction time; To predict the height of a dynamic virtual target at a given time; To track prediction confidence; To predict the time step, .

[0122] Reference Figure 3 As shown in one embodiment of this application, the method further includes:

[0123] S510. Perform obstacle detection on the environment in which the drone is located, and obtain the detection distance between the drone and the obstacle.

[0124] S520. When the detection distance is less than the safe distance threshold, perform obstacle avoidance path planning for the UAV and obtain obstacle avoidance instructions for the UAV.

[0125] S530. Replace the drone's control commands with obstacle avoidance commands to control the drone to avoid obstacles until the detection distance exceeds the safe distance threshold.

[0126] Specifically, after receiving control commands from the UAV, these commands are first input to the safe flight module before being input to the UAV's controller. The safe flight module performs safety analysis to determine the specific commands to be output to the controller. In this embodiment, the UAV may also be equipped with obstacle avoidance sensors, including LiDAR, depth cameras, and ultrasonic sensors, to acquire real-time three-dimensional spatial information of the UAV's environment and detect whether there are obstacles around the UAV.

[0127] Reference Figure 4 As shown, this embodiment utilizes LiDAR and a depth camera to acquire point cloud data of the drone's environment for obstacle detection. After detecting an obstacle, an ultrasonic sensor is used to obtain the detection distance between the obstacle and the drone to determine if there is a risk of collision. The detection distance is compared to a safe distance threshold. If the detection distance is greater than or equal to the safe distance threshold, it indicates that the drone maintains a sufficient safe distance from the obstacle and there is no risk of collision. In this case, the safe flight module can output control commands to the controller to control the drone to track the target object.

[0128] If the detected distance is less than the safe distance threshold, it indicates that the distance between the drone and the obstacle is too close, and there is a risk of collision. In this case, the safe flight module uses an obstacle avoidance algorithm to plan an obstacle avoidance path for the drone, obtaining the drone's obstacle avoidance command. After receiving the obstacle avoidance command, the safe flight module replaces the control command with the obstacle avoidance command and sends the obstacle avoidance command to the controller. The controller then guides the drone to avoid obstacles according to the obstacle avoidance path indicated by the command, until the detected distance between the drone and the obstacle in the environment exceeds the safe distance threshold. At this point, the drone is deemed to have successfully avoided the obstacle and can return to continue target tracking.

[0129] Furthermore, after the UAV completes obstacle avoidance, it repeats the aforementioned steps of image acquisition, target detection, state vector prediction, and control command generation to acquire new control commands and perform safety analysis again in the safe flight module. This ensures that the UAV's flight safety is given the highest priority and improves the UAV's flight safety.

[0130] Reference Figure 5 As shown, in one embodiment of this application, the reference state vector includes the tracking detection confidence of the tracked object in each frame of the reference image; the method further includes:

[0131] S610. If the tracking detection confidence of the tracked object in multiple consecutive reference images is lower than the confidence threshold, select multiple consecutive time points before the tracking detection confidence is lower than the confidence threshold as the inferred time points to obtain the effective image sequence of the tracked object at the inferred time points; wherein, the effective image sequence includes multiple frames of effective images of the tracked object.

[0132] S620. Based on the state vector of the tracked object in the valid image, obtain the inferred state sequence of the tracked object.

[0133] S630. Based on the predicted state sequence, the trajectory of the tracked object is predicted to obtain the predicted trajectory of the tracked object, and the search command of the UAV is generated based on the predicted trajectory to control the UAV to detect the tracked object.

[0134] Specifically, when performing target detection on each frame of the reference image sequence, the tracking detection confidence of the tracked object in each frame of the reference image is checked to determine whether the UAV has captured the tracked object. If the tracking detection confidence of the tracked object is lower than the confidence threshold in multiple consecutive frames of reference images, and the number of reference image frames in this situation exceeds the frame number threshold, it is determined that the UAV has lost the tracked object, and a search for the tracked object is required.

[0135] Furthermore, the loss time when the tracking detection confidence level begins to fall below the confidence threshold is determined. Multiple consecutive time points preceding the loss time are selected as inference time points, and a valid image sequence is obtained based on the valid images of the tracked object corresponding to the inference time points. Target detection is performed on each frame of the valid image sequence to obtain the valid state vector of the tracked object in the valid images. The inferred state sequence is constructed using the valid state vectors corresponding to each frame of the valid images to represent the change in the actual distribution state of the tracked object in the valid images over time.

[0136] Furthermore, the inferred state sequence is input into a temporal state prediction network. This network performs pattern learning on the inferred state sequence to predict the trajectory of the tracked object, resulting in a predicted trajectory. This predicted trajectory is then used to guide the generation of search commands for the UAV, controlling it to move to the possible directions and locations of the tracked object. The UAV then maneuvers and probes the object at these locations until it recaptures it. This embodiment uses trajectory prediction to control the UAV's detection of the tracked object, improving the efficiency of recapture and effectively enhancing the tracking performance.

[0137] Accordingly, please refer to Figure 6 This application provides a drone target tracking device, which includes:

[0138] The image sequence acquisition module 610 is used to acquire a reference image sequence within a specified time window; wherein, the reference image sequence includes multiple frames of reference images about the tracked object; the last moment of the specified time window is the current moment.

[0139] The target detection module 620 is used to perform target detection on each frame of reference image to obtain the reference state vector of the tracked object, and to construct a state vector sequence using the reference state vectors corresponding to each frame of reference image; wherein, the reference state vector is used to represent the actual distribution state of the tracked object in each frame of reference image.

[0140] The state vector prediction module 630 is used to predict the state vector of the tracked object at the prediction time based on the state vector sequence, so as to obtain the predicted state vector of the tracked object; wherein, the predicted state vector is used to represent the predicted distribution state of the tracked object in the current field of view of the UAV at the prediction time; the prediction time is the time after the prediction time step after the current time.

[0141] The reinforcement learning decision module 640 is used to input the predicted state vector into the reinforcement learning policy network to generate control commands for the UAV at the current moment, so as to control the UAV to track the object being tracked; wherein, the reward function of the reinforcement learning policy network is determined based on the predicted state vector during the training process.

[0142] In some alternative implementations, the reinforcement learning decision module 640 includes:

[0143] The control command output unit is used to perform forward propagation calculation on the predicted state vector through a reinforcement learning policy network and output the control command of the UAV at the current moment; wherein, the control command includes the flight control command of the UAV and the angle change command of the camera on the UAV.

[0144] In some alternative implementations, the reinforcement learning decision module 640 further includes:

[0145] The field-of-view center alignment unit is used to perform field-of-view center alignment based on the first predicted position to obtain a reward item for predicted observation alignment.

[0146] The pixel area comparison unit is used to perform pixel area comparison based on the first predicted area to obtain the predicted observation scale reward.

[0147] The vector distance calculation unit is used to obtain a prediction accuracy reward based on the distance data between the first predicted position, the first predicted area, the first prediction confidence, and the actual state vector of the tracked object at the prediction time.

[0148] The reward fusion design unit is used to fuse the reward items for prediction observation alignment, prediction observation scale, and prediction accuracy to obtain the reward function.

[0149] In some alternative implementations, the field-of-view center alignment unit includes:

[0150] The dimension center comparison sub-unit is used to perform center comparison in the first and second dimensions of the UAV's field of view using the first predicted position, and obtain a predictive observation alignment reward item that incentivizes the UAV to align the tracked object with the center of the field of view.

[0151] In some optional implementations, the pixel area comparison unit includes:

[0152] The pixel scale comparison subunit is used to perform scale comparison between the first predicted area and the expected pixel area to obtain a predicted observation scale reward item that incentivizes the UAV to adjust its flight altitude so that the pixel area of ​​the tracked object in the field of view is close to the expected pixel area.

[0153] In some alternative implementations, the state vector prediction module 630 includes:

[0154] The pattern learning prediction unit is used to input the state vector sequence into the temporal state prediction network, and to perform pattern learning on the state vector sequence through the temporal state prediction network to output the predicted state vector; wherein, the temporal state prediction network includes a long short-term memory network.

[0155] In some alternative implementations, the device further includes a safety obstacle avoidance control module, comprising:

[0156] The obstacle distance detection unit is used to detect obstacles in the environment where the drone is located and obtain the detection distance between the drone and the obstacles.

[0157] The obstacle avoidance path planning unit is used to plan obstacle avoidance paths for the UAV when the detection distance is less than the safe distance threshold, and to obtain obstacle avoidance commands for the UAV.

[0158] The control command switching unit is used to replace the control commands of the UAV with obstacle avoidance commands to control the UAV to avoid obstacles until the detection distance exceeds the safe distance threshold.

[0159] In some alternative implementations, the device further includes a loss estimation search module, comprising:

[0160] The effective image acquisition unit is used to acquire effective images that can be used for trajectory inference when the UAV loses the tracked object. If the detection confidence of the tracked object in multiple consecutive reference images is lower than the confidence threshold, multiple consecutive time moments before the detection confidence is lower than the confidence threshold are selected as inference time moments to obtain an effective image sequence of the tracked object at the inference time moment. The effective image sequence includes multiple frames of effective images of the tracked object.

[0161] The state vector detection unit is used to obtain the inferred state sequence of the tracked object based on the state vector of the tracked object in the valid image.

[0162] The trajectory inference search unit is used to infer the trajectory of the tracked object based on the inferred state sequence, obtain the predicted trajectory of the tracked object, and generate search instructions for the UAV based on the predicted trajectory to control the UAV to detect the tracked object.

[0163] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0164] In this embodiment, the UAV target tracking device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0165] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application, such as... Figure 7 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 7 Take a processor 10 as an example.

[0166] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GPA), or any combination thereof.

[0167] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0168] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0169] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0170] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0171] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the methods shown in the above embodiments are implemented.

[0172] This application provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method of any embodiment of this application.

[0173] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

[0174] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0175] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0176] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0177] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0178] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0179] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0180] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0181] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0182] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

[0183] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for tracking a target by a UAV, the method comprising: The method comprises: obtaining a reference image sequence in a specified time window; wherein the reference image sequence comprises multiple frames of reference images about a tracking object; the last time of the specified time window is the current time; performing target detection on each frame of reference image to obtain a reference state vector of the tracking object, and using the reference state vector corresponding to each frame of reference image to form a state vector sequence; wherein the reference state vector is used to represent the actual distribution state of the tracking object in each frame of reference image, and comprises a detection confidence of the tracking object in each frame of reference image; predicting a state vector of the tracking object at a predicted time according to the state vector sequence to obtain a predicted state vector of the tracking object; wherein the predicted state vector is used to represent the predicted distribution state of the tracking object in the current field of view of the unmanned aerial vehicle at the predicted time; the predicted time is a time after the current time by a predicted time step; inputting the predicted state vector into a reinforcement learning policy network to generate a control instruction of the unmanned aerial vehicle at the current time, so as to control the unmanned aerial vehicle to track the tracking object; wherein a reward function of the reinforcement learning policy network is determined according to the predicted state vector in a training process; if the detection confidence of the tracking object in multiple continuous frames of reference images is all lower than a confidence threshold, selecting multiple continuous times before the detection confidence is lower than the confidence threshold as a presumed time, obtaining an effective image sequence of the tracking object at the presumed time, which comprises multiple frames of effective images about the tracking object, obtaining a presumed state sequence of the tracking object according to the state vector of the tracking object in the effective images, and performing trajectory presumption on the tracking object according to the presumed state sequence to obtain a predicted trajectory of the tracking object, and generating a search instruction of the unmanned aerial vehicle based on the predicted trajectory, so as to control the unmanned aerial vehicle to detect the tracking object.

2. The method of claim 1, wherein, The inputting the predicted state vector into the reinforcement learning policy network to generate the control instruction of the unmanned aerial vehicle at the current time comprises: performing forward propagation calculation on the predicted state vector through the reinforcement learning policy network to output the control instruction of the unmanned aerial vehicle at the current time; wherein the control instruction comprises a flight control instruction of the unmanned aerial vehicle and an angle change instruction of a camera carried on the unmanned aerial vehicle.

3. The method of claim 1, wherein, The predicted state vector in the training process comprises a first predicted position, a first predicted area and a first predicted confidence of the tracking object at the predicted time; The reward function is obtained by: aligning a field of view center based on the first predicted position to obtain a predicted observation alignment reward item; comparing pixel areas based on the first predicted area to obtain a predicted observation scale reward item; obtaining a prediction accuracy reward item based on distance data between the first predicted position, the first predicted area, the first predicted confidence and an actual state vector of the tracking object at the predicted time; The prediction observation alignment reward item, the prediction observation scale reward item and the prediction accuracy reward item are fused to obtain the reward function.

4. The method of claim 3, wherein, The first prediction position is used to perform field of view center alignment to obtain a prediction observation alignment reward item. The first prediction position is used to perform center comparison in the first dimension and the second dimension of the field of view of the unmanned aerial vehicle to obtain the prediction observation alignment reward item that encourages the unmanned aerial vehicle to align the tracking object with the field of view center.

5. The method of claim 3, wherein, The first prediction area is used to perform pixel area comparison to obtain a prediction observation scale reward item. The first prediction area is compared with an expected pixel area to obtain the prediction observation scale reward item that encourages the unmanned aerial vehicle to adjust the flight height so that the pixel area of the tracking object in the field of view is close to the expected pixel area.

6. The method of claim 1, wherein, The state vector sequence is used to predict the state vector of the tracking object at a prediction moment to obtain a prediction state vector of the tracking object. The state vector sequence is input into a time sequence state prediction network, and the time sequence state prediction network is used to learn the mode of the state vector sequence and output the prediction state vector; the time sequence state prediction network includes a long short-term memory network.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Obstacle detection is performed on the environment where the unmanned aerial vehicle is located to obtain a detection distance between the unmanned aerial vehicle and an obstacle. When the detection distance is less than a safety distance threshold, obstacle avoidance path planning is performed on the unmanned aerial vehicle to obtain an obstacle avoidance instruction of the unmanned aerial vehicle. The control instruction of the unmanned aerial vehicle is replaced by the obstacle avoidance instruction to control the unmanned aerial vehicle to perform obstacle avoidance until the detection distance exceeds the safety distance threshold.

8. A computer device, comprising: The method includes: A memory and a processor are communicatively connected, and computer instructions are stored in the memory; the processor executes the computer instructions to perform the method in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, A computer readable storage medium stores computer instructions for causing a computer to perform the method in any one of claims 1 to 7.

Images