Unmanned aerial vehicle target tracking method, equipment and medium

Abstract

The invention relates to the technical field of intelligent visual inspection, and discloses an unmanned aerial vehicle target tracking method, which comprises the following steps: acquiring a reference image sequence in a specified time window, including multiple frames of reference images about a tracked object; performing target detection on each frame of reference image to obtain a reference state vector of the tracking object, and forming a state vector sequence by using the reference state vector corresponding to each frame of reference image; predicting a state vector of the tracking object at the prediction moment according to the state vector sequence to obtain a predicted state vector of the tracking object; and inputting the prediction state vector into a reinforcement learning strategy network, and generating a control instruction of the unmanned aerial vehicle at the current moment so as to control the unmanned aerial vehicle to track the tracked object, wherein the reward function of the reinforcement learning strategy network is determined according to the prediction state vector in the training process. The method has the beneficial effects that real-time fusion of target detection and unmanned aerial vehicle control is realized, and the accuracy and the reliability of the unmanned aerial vehicle when the unmanned aerial vehicle tracks a tiny target are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent visual detection, and in particular to a UAV target tracking method, device and medium. BACKGROUND

[0002] With the development of intelligent visual detection technology, the UAV (Unmanned Aerial Vehicle) technology has been widely used in UAV inspection, search and rescue and monitoring tasks due to its characteristics of flexibility and high maneuverability and the advantage of low cost, to accurately and real-timely detect and track ground or air targets, and has wide application in the fields of environmental monitoring, infrastructure inspection and disaster search and rescue. In actual scenarios, when performing a target tracking task, the UAV needs to realize clear imaging of a micro target and flight control of itself at the same time, and the micro target is usually in a fast-moving state, which brings great challenges to the tracking of the micro target by the UAV.

[0003] In the related art, the reliability of the UAV in tracking a micro target still needs to be improved. SUMMARY

[0004] The present application provides a UAV target tracking method, device and medium, which predicts the state vector of a tracking object at a future time, and generates a control instruction of the UAV by using a reinforcement learning network trained with a distribution state of the tracking object in a field of view as a reward function, so as to realize real-time fusion of target detection and UAV control, and improve the accuracy and reliability of the UAV in tracking a micro target.

[0005] In order to achieve the above purpose, the main technical scheme adopted by the present application comprises: In a first aspect, the present application provides a UAV target tracking method, which comprises: obtaining a reference image sequence in a specified time window; wherein the reference image sequence comprises a plurality of reference images of a tracking object; the last time of the specified time window is the current time; performing target detection on each reference image to obtain a reference state vector of the tracking object, and constructing a state vector sequence by using the reference state vector corresponding to each reference image; wherein the reference state vector is used to represent the actual distribution state of the tracking object in the each reference image; predicting the state vector of the tracking object at a predicted time according to the state vector sequence to obtain a predicted state vector of the tracking object; wherein the predicted state vector is used to represent the predicted distribution state of the tracking object in the current field of view of the UAV at the predicted time; the predicted time is a time after the current time by a predicted time step; inputting the predicted state vector into a reinforcement learning policy network to generate a control instruction of the UAV at the current time, so as to control the UAV to track the tracked object; wherein a reward function of the reinforcement learning policy network is determined according to the predicted state vector in the training process.

[0006] The UAV target tracking method provided in the embodiments of the present application obtains the state vector of the tracked object in the reference image sequence in the specified time window through target detection, and takes the state vector as a prediction basis to predict the state vector of the tracked object at the predicted time to obtain a predicted state vector. On this basis, the predicted state vector is taken as a guide target, and a reinforcement learning network taking the state vector as a reward function is used to generate a control instruction of the UAV. Compared with related technologies, the present application determines the reward function according to the state vector and uses it to train the reinforcement learning policy network, so that the real-time distribution state of the tracked object directly affects the generation process of the control instruction, realizes the fusion and cooperation of the target tracking task and the flight control task, and effectively improves the tracking accuracy and reliability of the UAV when performing the target tracking task. In addition, the present application predicts the state vector of the tracked object at the future time, takes it as a guide target, and generates a control instruction corresponding to the state vector at the future time, thereby realizing the forward-looking decision of the UAV and improving the dynamic tracking capability of the UAV on the high-maneuverability target.

[0007] Optionally, the inputting the predicted state vector into the reinforcement learning policy network to generate the control instruction of the UAV at the current time comprises: performing forward propagation calculation on the predicted state vector by the reinforcement learning policy network to output the control instruction of the UAV at the current time; The control instruction comprises a flight control instruction of the UAV and an angle change instruction of a camera carried on the UAV.

[0008] Optionally, the predicted state vector in the training process comprises a first predicted position, a first predicted area and a first predicted confidence of the tracked object at the predicted time; and the reward function is obtained by: aligning a field of view center based on the first predicted position to obtain a predicted observation alignment reward item; performing pixel area comparison based on the first predicted area to obtain a predicted observation scale reward item; obtaining a prediction accuracy reward item based on distance data between the first predicted position, the first predicted area, the first predicted confidence and an actual state vector of the tracked object at the predicted time; The prediction observation alignment reward item, the prediction observation scale reward item and the prediction accuracy reward item are fused to obtain the reward function.

[0009] Optionally, the field of view center alignment based on the first predicted position comprises: The first predicted position is used for center comparison in the first dimension and the second dimension of the field of view of the unmanned aerial vehicle, so as to obtain the prediction observation alignment reward item for encouraging the unmanned aerial vehicle to align the tracking object with the field of view center.

[0010] Optionally, the pixel area comparison based on the first predicted area comprises: The first predicted area is compared with an expected pixel area in scale, so as to obtain the prediction observation scale reward item for encouraging the unmanned aerial vehicle to adjust the flight height so that the pixel area of the tracking object in the field of view is close to the expected pixel area.

[0011] Optionally, the prediction of the state vector of the tracking object at the predicted moment according to the state vector sequence comprises: The state vector sequence is input into a time sequence state prediction network, and the time sequence state prediction network is used for mode learning on the state vector sequence, and the prediction state vector is output; wherein the time sequence state prediction network comprises a long short-term memory network.

[0012] Optionally, the method further comprises: Obstacle detection is performed on the environment where the unmanned aerial vehicle is located, so as to obtain a detection distance between the unmanned aerial vehicle and an obstacle; When the detection distance is less than a safety distance threshold, obstacle avoidance path planning is performed on the unmanned aerial vehicle, so as to obtain an obstacle avoidance instruction of the unmanned aerial vehicle; The control instruction of the unmanned aerial vehicle is replaced by the obstacle avoidance instruction, so as to control the unmanned aerial vehicle to perform obstacle avoidance until the detection distance exceeds the safety distance threshold.

[0013] Optionally, the reference state vector comprises a detection confidence of the tracking object in each frame of reference image; the method further comprises: If the detection confidence of the tracking object in a plurality of continuous frames of reference image is lower than a confidence threshold, a plurality of continuous time moments before the detection confidence is lower than the confidence threshold are selected as a speculation time moment, so as to obtain an effective image sequence of the tracking object at the speculation time moment; wherein the effective image sequence comprises a plurality of effective images about the tracking object; According to the state vector of the tracking object in the effective image, a predicted state sequence of the tracking object is obtained; According to the predicted state sequence, trajectory prediction of the tracking object is performed to obtain a predicted trajectory of the tracking object, and a search instruction of the unmanned aerial vehicle is generated based on the predicted trajectory to control the unmanned aerial vehicle to detect the tracking object.

[0014] In a second aspect, an embodiment of the present application provides an unmanned aerial vehicle target tracking device, and the device comprises: An image sequence acquisition module is configured to acquire a reference image sequence in a specified time window; wherein the reference image sequence comprises multiple frames of reference images of a tracking object; and the last time of the specified time window is the current time; A tracking target detection module is configured to perform target detection on each frame of reference image to obtain a reference state vector of the tracking object, and to construct a state vector sequence by using the reference state vector corresponding to each frame of reference image; wherein the reference state vector is used to represent an actual distribution state of the tracking object in each frame of reference image; A state vector prediction module is configured to predict a state vector of the tracking object at a predicted time according to the state vector sequence to obtain a predicted state vector of the tracking object; wherein the predicted state vector is used to represent a predicted distribution state of the tracking object in a current field of view of the unmanned aerial vehicle at the predicted time; and the predicted time is a time after the current time by a predicted time step; A reinforcement learning decision module is configured to input the predicted state vector into a reinforcement learning policy network to generate a control instruction of the unmanned aerial vehicle at the current time to control the unmanned aerial vehicle to track the tracking object; wherein a reward function of the reinforcement learning policy network is determined according to the predicted state vector in a training process.

[0015] In a third aspect, an embodiment of the present application provides a computer device, which comprises a memory and a processor, the memory and the processor are communicatively connected with each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the method in any of the above embodiments.

[0016] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores computer instructions, and the computer instructions are used to make a computer execute the method in any of the above embodiments.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product, which comprises computer instructions, and the computer instructions are used to make a computer execute the method in any of the above embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the specific embodiments or prior art of the present application, the drawings needed in the description of the specific embodiments or prior art will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0019] Figure 1 The step diagram of the unmanned aerial vehicle target tracking method provided by the embodiment of the present application is shown in the figure. Figure 2 The step diagram of obtaining the reward function in the embodiment of the present application is shown in the figure. Figure 3 The step diagram of controlling the unmanned aerial vehicle to avoid obstacles in the embodiment of the present application is shown in the figure. Figure 4 The system flowchart of the unmanned aerial vehicle in the embodiment of the present application is shown in the figure. Figure 5 The step diagram of the speculative search in the case of object loss in the embodiment of the present application is shown in the figure. Figure 6 The module diagram of the unmanned aerial vehicle target tracking device provided by the embodiment of the present application is shown in the figure. Figure 7 The structural schematic diagram of a computer device provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0021] With the development of intelligent visual detection technology, unmanned aerial vehicle (UAV) technology has been widely used in unmanned aerial vehicle inspection, search and rescue and monitoring tasks due to its flexible and high mobility characteristics and low cost advantages, to detect and track ground or air targets in real time, and has wide application in environmental monitoring, infrastructure inspection and disaster search and rescue. In actual scenarios, when performing target tracking tasks, the unmanned aerial vehicle needs to realize clear imaging of small targets and flight control of itself at the same time, and the small targets are usually in a fast moving state, which brings great challenges to the tracking of small targets by the unmanned aerial vehicle.

[0022] In the related art, a reactive control strategy is usually adopted to control the UAV to perform a target tracking task, that is, after detecting the state of the tracking object, a control instruction is generated according to the current state to control the UAV to adjust the flight state or adjust the field of view angle. Due to the need for real-time calculation, such a control strategy has a lag, which often limits the tracking accuracy when tracking a tracking object with high mobility such as a fast-moving vehicle, and the problem of target loss is prone to occur. In addition, the UAV flight control and target detection are usually performed separately in the related art, resulting in mutual independence between the two tasks, and effective fusion cannot be achieved, while in actual situations, the flight state of the UAV has a great influence on the target detection accuracy.

[0023] Based on the above problems, the present application provides a UAV target tracking method, device and medium, which acquires a reference image sequence in a specified time window, including multiple frames of reference images about the tracking object; performs target detection on each reference image to obtain a reference state vector of the tracking object, and uses the reference state vector corresponding to each reference image to form a state vector sequence; predicts the state vector of the tracking object at a prediction time according to the state vector sequence to obtain a predicted state vector of the tracking object; inputs the predicted state vector into a reinforcement learning policy network to generate a control instruction of the UAV at the current time to control the UAV to track the tracking object; wherein the reward function of the reinforcement learning policy network is determined according to the predicted state vector in the training process.

[0024] The UAV target tracking method provided by the embodiments of the present application acquires the reference image sequence of the tracking object in the specified time window, obtains the state vector of the tracking object in these reference images through target detection, and uses the same as the prediction basis to predict the state vector of the tracking object at the prediction time to obtain the predicted state vector; on this basis, the predicted state vector is used as the guide target, and the reinforcement learning network with the state vector as the reward function is used to generate the control instruction of the UAV.

[0025] Compared with the related art, the present application determines the reward function according to the state vector and uses it to train the reinforcement learning policy network, so that the real-time distribution state of the tracking object directly affects the generation process of the control instruction, realizes the fusion and cooperation of the target tracking task and the flight control task, and effectively improves the tracking accuracy and reliability of the UAV when performing the target tracking task.

[0026] In addition, the present application predicts the state vector of the tracking object at the future time, uses it as the guide target, and generates the control instruction corresponding to the state vector at the future time, thereby realizing the forward-looking decision of the UAV and improving the dynamic tracking capability of the UAV on the high-mobility target.

[0027] The unmanned aerial vehicle target tracking method provided in the specification can be applied to an unmanned aerial vehicle with target capturing and detecting functions to control the unmanned aerial vehicle to track a target. The unmanned aerial vehicle can include a multi-rotor unmanned aerial vehicle, a fixed-wing unmanned aerial vehicle, a vertical take-off and landing fixed-wing unmanned aerial vehicle, an ornithopter, or the like. It can be understood that, after adaptive modification, the unmanned aerial vehicle target tracking method provided in the specification can also be used for other flight devices similar to the unmanned aerial vehicle to control the flight devices to perform target tracking.

[0028] According to an embodiment of the present application, an unmanned aerial vehicle target tracking method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0029] An unmanned aerial vehicle target tracking method is provided in the present embodiment, which can be used for the unmanned aerial vehicle with target capturing and detecting functions described above. Referring to FIG. 1, the method comprises the following steps. Figure 1 S100. Obtain a reference image sequence in a specified time window; wherein the reference image sequence comprises a plurality of reference images about a tracking object; and the last time of the specified time window is the current time.

[0030] S200. Perform target detection on each reference image to obtain a reference state vector of the tracking object, and use the reference state vector corresponding to each reference image to form a state vector sequence; wherein the reference state vector is used to represent the actual distribution state of the tracking object in each reference image.

[0031] S300. Predict a state vector of the tracking object at a predicted time according to the state vector sequence to obtain a predicted state vector of the tracking object; wherein the predicted state vector is used to represent the predicted distribution state of the tracking object in the current field of view of the unmanned aerial vehicle at the predicted time; and the predicted time is a time after the current time by a predicted time step.

[0032] S400. Input the predicted state vector into a reinforcement learning policy network to generate a control instruction of the unmanned aerial vehicle at the current time to control the unmanned aerial vehicle to track the tracking object; wherein a reward function of the reinforcement learning policy network is determined according to the predicted state vector in the training process.

[0033] ​In particular, the UAV includes a UAV platform, a gimbal camera, and a computing platform, where the UAV platform is the main structure of the UAV and is mainly responsible for the flight movement of the UAV. Illustratively, the UAV platform can be a multi-rotor UAV with good maneuverability and load capacity, such as a quadcopter or a hexacopter, and the built-in flight control system of the UAV platform can receive and execute control instructions from the computing platform. The gimbal camera can be a camera mounted on a three-axis gimbal, where the three-axis gimbal can rotate in three directions respectively, providing stability for the camera during flight and flexible angle control for the camera during target tracking. The computing platform includes multiple computing units responsible for running various algorithms in the target tracking process. Illustratively, the computing platform can be a high-performance, low-power embedded AI computing unit, such as the NVIDIA Jetson series platform.

[0034] During flight, the UAV obtains images within the field of view through the camera in real time, obtaining multiple real-time images corresponding to multiple acquisition time points respectively, and the real-time images contain the environment around the UAV and the tracking object that can exist. When the tracking object exists in the real-time images, these real-time images can be used as reference images to generate control instructions for the UAV. On this basis, a specified time window is set, and multiple reference images within the specified time window are obtained to obtain a reference image sequence for tracking the tracking object in a motion state. It should be noted that the specified time window includes the current time point and multiple continuous time points before the current time point, the last time point of the specified time window is the current time point, and the tracking object exists in the reference image corresponding to each time point in the specified time window.

[0035] Further, target detection is performed on each frame of the reference image sequence respectively, and a reference state vector of the tracking object in each frame of the reference image sequence is determined according to the distribution state of the tracking object in the reference image. The target detection can be that the reference image sequence is input into a deep learning target detection network in an airborne computing platform carried by the UAV, the reference image sequence is analyzed in real time by the deep learning target detection network, and a state vector sequence is output. It should be noted that the reference state vector can include a reference detection position, a reference detection area and a reference detection confidence of the tracking object in the reference image. The reference detection position can be a detection position of the tracking object in the reference image, including the center coordinates of the tracking object in any dimension of the reference image, and the origin of the coordinates can be at the top left corner of the reference image. The reference detection area can be a normalized pixel area occupied by a target bounding box corresponding to the tracking object in the reference image, which can be used to determine the distance between the UAV and the tracking object and the clarity of the tracking object in the reference image. The reference detection confidence can be the similarity between the tracking object detected in the reference image and the real object, which is a scalar ranging from 0 to 1, indicating the probability that the tracking object detected in the reference image is the real object.

[0036] Exemplarily, the reference state vector can be represented according to the following formula:

[0037] wherein, represents the reference state vector; is the reference detection position of the tracking object; is the reference detection area of the tracking object; is the reference detection confidence of the tracking object. The state vector sequence obtained in this way can be represented according to the following formula:

[0038] wherein, represents the state vector sequence.

[0039] Further, a motion pattern of the tracking object in a specified time window is learned according to the sequence of state vectors, and a distribution state of the tracking object after a prediction time step is predicted according to a result of the pattern learning, to obtain a prediction state vector of the tracking object at a prediction moment. It should be noted that the prediction state vector represents a predicted distribution state of the tracking object in the current field of view of the UAV at the prediction moment, and is used as a basis for generating a UAV control instruction. It can be understood that by predicting the distribution state of the tracking object in the field of view of the UAV in advance, an explicit aiming target can be provided for the UAV in advance, so that the UAV control instruction is generated with the purpose of aligning the aiming target, overcoming the inherent delay in the process of real-time target detection and instruction generation, improving the adaptive ability of the UAV to a complex dynamic environment, and significantly improving the tracking accuracy of the UAV to the tracking object.

[0040] In some embodiments, the prediction state vector includes a tracking prediction position, a tracking prediction area and a tracking prediction confidence of the tracking object at the prediction moment, where the tracking prediction position can be a detection position of the tracking object in the field of view of the UAV at the prediction moment, including the center coordinates of the tracking object in any dimension of the field of view of the UAV, and the origin of the coordinates can be at the top left corner of the field of view of the UAV. The tracking prediction area can be a normalized pixel area occupied by a target bounding box corresponding to the tracking object in the field of view of the UAV at the prediction moment, and can be used to determine the distance between the UAV and the tracking object and the clarity of the tracking object in the field of view of the UAV. The tracking prediction confidence can be a similarity between the tracking object detected in the field of view of the UAV at the prediction moment and the real object, which is a scalar ranging from 0 to 1, representing the probability that the tracking object detected in the field of view of the UAV is the real object.

[0041] Further, after obtaining the predicted state vector, the predicted state vector is input into the reinforcement learning policy network to generate a control instruction for the UAV at the current time according to a pre-trained reinforcement learning algorithm, so as to control the UAV to track the tracking object. The reinforcement learning policy network is obtained by offline training using a digital simulation environment before tracking the tracking object. The digital simulation environment can be a virtual environment with high fidelity to ensure the efficiency and repeatability of the training process. In some embodiments, the digital simulation environment can include a diversified virtual scene, a dynamic virtual target, and a UAV behavior model, etc. The diversified virtual scene can be used to simulate various obstacles in real situations, such as buildings and trees, etc. The dynamic virtual target can be a virtual tracking object with multiple complex motion modes, which can move in acceleration, acceleration and deceleration, turning or S-shaped maneuvering, etc. The UAV behavior model includes a physical model for simulating the behavior of the UAV and a sensor model. The physical model can be a model with the same aerodynamic characteristics as the real UAV, which simulates the flight state of the UAV. The sensor model can be a model with consistent real sensor data distribution, which simulates the behavior of the UAV perceiving the surrounding environment.

[0042] During the training process, a reinforcement learning algorithm based on an Actor-Critic framework can be used, including but not limited to a Deep Deterministic Policy Gradient (DDPG) algorithm, etc. The Actor network as a decision network simulates the multi-round interaction between the training agent and the digital simulation environment using the UAV as the training agent, and generates a control instruction corresponding to each interaction round. The Critic network as a critic network calculates the reward of the control instruction output by the Actor network according to a preset reward function, determines the cumulative reward of the interaction round, and adjusts the network parameters of the reinforcement learning policy network according to the cumulative reward, so that the reinforcement learning policy network can gradually give better control instructions.

[0043] It should be noted that the reward function is determined according to the predicted state vector in the training process. In each interaction round, the UAV performs the above-mentioned image acquisition, target detection, state vector prediction, and control instruction generation steps, perceives the diversified virtual scene through the sensor model, and obtains the predicted state vector according to the perception result. The reward function is related to the data in the predicted state vector. The cumulative reward is calculated according to the predicted state vector and the reward function to directly evaluate and adjust the aiming ability of the UAV to the dynamic virtual target in the digital simulation environment, thereby improving the aiming ability of the UAV to the tracking object.

[0044] It can be understood that after obtaining the control instruction of the UAV at the current time and controlling the UAV according to the control instruction, the foregoing process of generating the control instruction is repeated at the next time of the current time to obtain the control instruction at the next time. In the process of tracking the tracking object by the UAV, the foregoing process is repeated to aim at the tracking object in real time until the target tracking task is completed.

[0045] The UAV target tracking method provided in the embodiment obtains the state vector of the tracking object in the reference image sequence of the tracking object in the specified time window through target detection, and takes the state vector as a prediction basis to predict the state vector of the tracking object at the prediction time to obtain a predicted state vector. On this basis, the predicted state vector is taken as a guide target, and a reinforcement learning network trained by taking the state vector as a reward function is used to generate the control instruction of the UAV.

[0046] Compared with the related art, the application determines the reward function according to the state vector and uses the reward function to train the reinforcement learning strategy network, so that the real-time distribution state of the tracking object directly affects the generation process of the control instruction, realizes the fusion and cooperation of the target tracking task and the flight control task, and effectively improves the tracking accuracy and reliability of the UAV when performing the target tracking task.

[0047] In addition, the application predicts the state vector of the tracking object at the future time, takes the state vector as a guide target, generates the control instruction corresponding to the state vector at the future time, and thus realizes the forward-looking decision of the UAV, and improves the dynamic tracking capability of the UAV on the high-maneuverability target.

[0048] As an embodiment of the application, the predicted state vector is input into the reinforcement learning strategy network to generate the control instruction of the UAV at the current time, including: S410. The predicted state vector is calculated by forward propagation through the reinforcement learning strategy network, and the control instruction of the UAV at the current time is output; wherein the control instruction includes the flight control instruction of the UAV and the angle change instruction of the camera carried on the UAV.

[0049] Specifically, the predicted state vector is input into the reinforcement learning policy network, the predicted state vector is forward calculated by using an Actor network in the reinforcement learning policy network, and a control instruction for realizing the predicted state vector is output to control the unmanned aerial vehicle to track the tracking object. The control instruction includes a flight control instruction of the unmanned aerial vehicle and an angle change instruction of a camera carried on the unmanned aerial vehicle, wherein the flight control instruction is used to control the moving speed of the unmanned aerial vehicle in multiple directions, and the angle change instruction is used to control the orientation of the camera. Through the cooperation between the position of the unmanned aerial vehicle itself and the angle of the camera, the unmanned aerial vehicle can aim at the tracking object in real time and clearly image the tracking object in the field of view, thereby realizing real-time tracking of the tracking object.

[0050] Exemplarily, the control instruction can be expressed according to the following formula:

[0051] wherein, represents the control instruction of the unmanned aerial vehicle at the moment; is a forward flight speed instruction of the unmanned aerial vehicle in the horizontal plane under the body coordinate system, and the unit is meter / second (m / s); is a lateral flight speed instruction of the unmanned aerial vehicle in the horizontal plane under the body coordinate system, and the unit is meter / second (m / s); is a height change rate instruction of the unmanned aerial vehicle, and the unit is meter / second (m / s); is a change rate instruction of the pitch angle of the gimbal camera carried on the unmanned aerial vehicle, and the unit is degree / second (° / s); is a change rate instruction of the yaw angle of the gimbal camera carried on the unmanned aerial vehicle, and the unit is degree / second (° / s).

[0052] Referring to FIG. 1, Figure 2 As an embodiment of the present application, the predicted state vector in the training process includes a first predicted position, a first predicted area and a first predicted confidence of the tracking object at the predicted moment; and the reward function is obtained by the following way: S420. The field of view center alignment is performed based on the first predicted position, and a predicted observation alignment degree reward item is obtained.

[0053] S430. The pixel area ratio is compared based on the first predicted area, and a predicted observation scale reward item is obtained.

[0054] S440. The predicted accuracy reward item is obtained based on the distance data between the first predicted position, the first predicted area, the first predicted confidence and the actual state vector of the tracking object at the predicted moment.

[0055] S450. The prediction observation alignment reward item, the prediction observation scale reward item, and the prediction accuracy reward item are fused to obtain a reward function.

[0056] The prediction state vector in the training process can be a result obtained by predicting the state vector when the unmanned aerial vehicle tracks the dynamic virtual target under the control of the unmanned aerial vehicle target tracking method provided in the specification during the training of the reinforcement learning policy network. The prediction state vector in the training process includes a first predicted position, a first predicted area, and a first predicted confidence of the tracking object at the predicted time. The first predicted position can be a detection position of the dynamic virtual target in the field of view of the unmanned aerial vehicle at the predicted time, including the center coordinates of the dynamic virtual target in any dimension of the field of view of the unmanned aerial vehicle, and the origin of the coordinates can be at the upper left corner of the field of view of the unmanned aerial vehicle. The first predicted area can be a normalized pixel area occupied by the target bounding box corresponding to the dynamic virtual target in the field of view of the unmanned aerial vehicle at the predicted time, which can be used to determine the distance between the unmanned aerial vehicle and the dynamic virtual target and the clarity of the dynamic virtual target in the field of view of the unmanned aerial vehicle. The first predicted confidence can be the similarity between the dynamic virtual target detected in the field of view of the unmanned aerial vehicle and the real target at the predicted time, which is a scalar ranging from 0 to 1, indicating the probability that the dynamic virtual target detected in the field of view of the unmanned aerial vehicle is the real target.

[0057] Specifically, when controlling the unmanned aerial vehicle to perform target tracking, in order to completely and accurately present the tracking object in the field of view, it is usually necessary to align the tracking object with the center of the field of view of the unmanned aerial vehicle and image the tracking object at the optimal scale to reduce interference and improve the accuracy of target tracking and ensure that the details of the tracking object can be captured. Considering the above reasons, in the training process of the reinforcement learning policy network, a prediction observation alignment reward item is obtained according to the alignment degree of the first predicted position and the center of the field of view of the unmanned aerial vehicle, so that the reinforcement learning policy network can learn to place the tracking object in the center of the field of view of the unmanned aerial vehicle when controlling the unmanned aerial vehicle. At the same time, a prediction observation scale reward item is obtained according to the scale difference between the first predicted area and the expected pixel area, so that the reinforcement learning policy network can learn to maintain a suitable distance from the tracking object when controlling the unmanned aerial vehicle, thereby improving the ability to capture details of the tracking object.

[0058] Further, on the basis of training the reinforcement learning policy network to make decisions on the behavior of the UAV, the accuracy of the predicted state vector also has a great influence on the generation quality of the control instruction. In view of the above reasons, in the training process, the embodiment first obtains the predicted state vector of the dynamic virtual target at the predicted time through state vector prediction, which includes the first predicted position, the first predicted area and the first predicted confidence of the tracking object at the predicted time. Then the training process is continued until the current time reaches the predicted time, at which time the actual reference image of the dynamic virtual target at the time is obtained through the sensor model, and the actual state vector of the dynamic virtual target in the actual reference image is obtained through target detection. According to the first predicted position, the first predicted area and the first predicted confidence in the predicted state vector and the actual state vector, the distance data between the predicted state vector and the actual state vector is calculated to determine the prediction error of the state vector, and the prediction accuracy reward item is obtained.

[0059] Exemplarily, the distance data between the predicted state vector and the actual state vector can be the Euclidean distance, and at this time the prediction accuracy reward item can be represented according to the following formula:

[0060] Wherein, represents the prediction accuracy reward item; is the actual state vector; is the predicted state vector in the training process; represents the Euclidean distance calculation. It can be understood that the smaller the distance between the predicted state vector and the actual state vector, the closer the predicted state vector in the training process to the actual state vector, indicating that the accuracy of the state vector prediction is higher, and the reward represented by the prediction accuracy reward item is higher.

[0061] It should be noted that the reinforcement learning policy network is also coupled to the time sequence state prediction network for state vector prediction, and the training results of the reinforcement learning policy network are fed back to the time sequence state prediction network to promote the time sequence state prediction network to adjust the parameters through the prediction accuracy reward item, and to encourage the time sequence state prediction network to make higher precision state vector prediction, and to improve the accuracy of the predicted state vector.

[0062] In some embodiments, the reward function also includes an energy consumption and smoothing penalty term, which is used to evaluate the flight state of the UAV, suppress the occurrence of excessive maneuvering and excessive energy consumption of the UAV during flight, and encourage the UAV to fly smoothly and energy-efficiently. The energy consumption and smoothing penalty term can be represented according to the following formula:

[0063] Wherein, represents the energy consumption and smoothing penalty term; , , , and They are respectively The corresponding commands in the control instructions for the drone at any time; and This represents the corresponding penalty coefficient. It's understandable that the energy consumption and smoothing penalty terms are used to constrain the drone, preventing significant changes in its flight path or camera angle, and reducing energy consumption during tracking.

[0064] Furthermore, a reward function is obtained by fusing the reward terms for predicted observation alignment, predicted observation scale, predicted accuracy, and energy consumption and smoothing penalties. The reward function can be expressed as follows:

[0065] in, express The reward function at each time step; The reward item is the alignment of the predicted observations. Rewards are given for predicting the scale of observations. , , and These are the corresponding weighting coefficients.

[0066] As one embodiment of this application, field-of-view center alignment is performed based on a first predicted position to obtain a predicted observation alignment reward, including: S422. Using the first predicted position, perform center comparison in the first and second dimensions of the UAV's field of view to obtain a predicted observation alignment reward item that incentivizes the UAV to align the tracked object with the center of the field of view.

[0067] Specifically, the first predicted position includes the center coordinates of the dynamic virtual target in the first dimension and the center coordinates in the second dimension. The coordinates in the two dimensions are compared with the center of the UAV's field of view in the corresponding dimension to determine the distance error between the center of the first predicted position and the center of the UAV's field of view, and the prediction observation alignment bonus is obtained.

[0068] For example, the prediction observation alignment reward term can be represented by the following formula:

[0069] in, To predict the center coordinates of the dynamic virtual target in the first dimension at the current time. To predict the center coordinates of the dynamic virtual target in the second dimension at the current time. a total length of the field of view of the UAV in the first dimension; a total length of the field of view of the UAV in the second dimension; a prediction time; a hyper-parameter for controlling the shape of the reward function. It can be understood that the prediction observation alignment reward term shown in the above formula is a Gaussian function, which takes the maximum value when the center of the first prediction position coincides with the center of the field of view of the UAV, and exponentially decays as the deviation between the center of the first prediction position and the center of the field of view of the UAV increases.

[0070] As an embodiment of the present application, the pixel area comparison based on the first prediction area is performed to obtain a prediction observation scale reward term, which includes: S432. Scale comparison between the first prediction area and the expected pixel area is performed to obtain a prediction observation scale reward term that encourages the UAV to adjust the flight height so that the pixel area of the tracked object in the field of view approaches the expected pixel area.

[0071] Specifically, the scale comparison between the first prediction area and the expected pixel area calculates the scale error between the first prediction area and the expected pixel area to obtain the prediction observation scale reward term. Exemplarily, the prediction observation scale reward term can be represented according to the following formula:

[0072] wherein, a prediction area; an expected pixel area; a hyper-parameter for controlling the shape of the reward function. It can be understood that the expected pixel area can be determined according to the needs of the target tracking task, and the prediction observation scale reward term takes the maximum value when the first prediction area coincides with the expected pixel area; the prediction observation scale reward term gradually decreases when the first prediction area is greater than or less than the expected pixel area.

[0073] As an embodiment of the present application, the state vector of the tracked object at the prediction time is predicted according to the state vector sequence to obtain the prediction state vector of the tracked object, which includes: S310. The state vector sequence is input into a time sequence state prediction network, and the time sequence state prediction network is used to learn the mode of the state vector sequence to output the prediction state vector; wherein the time sequence state prediction network includes a long short-term memory network.

[0074] Specifically, the time sequence state prediction network is obtained by offline training in advance through a supervised learning method before tracking the tracked object. The training manner of the time sequence state prediction network can be adaptive parameter adjustment of the time sequence state prediction network according to the feedback prediction accuracy reward item in the training process of the reinforcement learning policy network. It can be understood that the types of the time sequence state prediction network in the embodiment include but are not limited to Long Short-Term Memory (LSTM), Convolutional Neural Network (CNN) or Transformer architecture, etc.

[0075] Further, the state vector sequence is input into the time sequence state prediction network, and the time sequence state prediction network learns the mode of the state vector sequence and outputs a predicted state vector. The predicted state vector can be represented according to the following formula:

[0076] wherein, represents the predicted state vector; is a center coordinate of the dynamic virtual target in the first dimension at the prediction time, is a center coordinate of the dynamic virtual target in the second dimension at the prediction time; is a tracking prediction width of the dynamic virtual target at the prediction time; is a height of the dynamic virtual target at the prediction time; is a tracking prediction confidence; is a prediction time step, .

[0077] Referring to FIG. 1, Figure 3 as an embodiment of the present application, the method further comprises: S510. Obstacle detection is performed on the environment in which the unmanned aerial vehicle is located to obtain a detection distance between the unmanned aerial vehicle and the obstacle.

[0078] S520. When the detection distance is less than a safety distance threshold, obstacle avoidance path planning is performed on the unmanned aerial vehicle to obtain an obstacle avoidance instruction of the unmanned aerial vehicle.

[0079] S530. The control instruction of the unmanned aerial vehicle is replaced by the obstacle avoidance instruction to control the unmanned aerial vehicle to avoid obstacles until the detection distance exceeds the safety distance threshold.

[0080] Specifically, after obtaining the control instruction of the UAV, the control instruction is input into the safe flight module before being input into the controller of the UAV, and the safe flight module performs safety analysis to determine the specific instruction output to the controller. In the embodiment, the UAV can also be equipped with an obstacle avoidance sensor, including a laser radar, a depth camera, and an ultrasonic sensor, to obtain the three-dimensional spatial information of the environment in which the UAV is located in real time and detect whether there is an obstacle around the UAV.

[0081] Referring to Figure 4 As shown in the figure, in the embodiment, the laser radar and the depth camera are used to obtain the point cloud data of the environment in which the UAV is located to detect obstacles in the environment in which the UAV is located. After detecting the obstacles in the environment in which the UAV is located, the detection distance between the obstacles and the UAV is obtained by the ultrasonic sensor to determine whether there is a collision risk between the UAV and the obstacles. The detection distance is compared with the safety distance threshold value. If the detection distance is greater than or equal to the safety distance threshold value, it indicates that the UAV and the obstacles maintain a sufficient safety distance and there is no collision risk. At this time, the control instruction can be output from the safe flight module to the controller to control the UAV to track the tracking object.

[0082] If the detection distance is less than the safety distance threshold value, it indicates that the distance between the UAV and the obstacles is too close and there is a collision risk between them. At this time, the safe flight module uses an obstacle avoidance algorithm to plan an obstacle avoidance path for the UAV to obtain an obstacle avoidance instruction of the UAV. After obtaining the obstacle avoidance instruction, the safe flight module replaces the instruction with the obstacle avoidance instruction and sends the obstacle avoidance instruction to the controller, so that the UAV is controlled to avoid obstacles along the obstacle avoidance path indicated by the obstacle avoidance instruction until the detection distance between the obstacles in the environment in which the UAV is located and the UAV exceeds the safety distance threshold value. At this time, it is determined that the UAV has completed obstacle avoidance and can return to continue target tracking.

[0083] Further, after the UAV completes obstacle avoidance, the UAV executes the aforementioned image acquisition, target detection, state vector prediction, and control instruction generation steps again to obtain new control instructions and perform safety analysis in the safe flight module again, so as to ensure that the flight safety of the UAV is in the highest priority and improve the flight safety of the UAV.

[0084] Referring to Figure 5 As an embodiment of the present application, the reference state vector includes a tracking detection confidence of the tracking object in each frame of reference image; the method further includes: S610. If the tracking detection confidence of the tracking object in the continuous multiple frames of reference images is lower than the confidence threshold value, select multiple continuous time instants before the tracking detection confidence is lower than the confidence threshold value as the predicted time instants to obtain an effective image sequence of the tracking object at the predicted time instants; wherein the effective image sequence includes multiple frames of effective images of the tracking object.

[0085] S620. According to the state vector of the tracking object in the effective image, a predicted state sequence of the tracking object is obtained.

[0086] S630. According to the predicted state sequence, a predicted trajectory of the tracking object is obtained by trajectory prediction of the tracking object, and a search instruction of the UAV is generated based on the predicted trajectory to control the UAV to detect the tracking object.

[0087] Specifically, when performing target detection on each frame of reference image in the reference image sequence, the tracking detection confidence of the tracking object in each frame of reference image is checked to determine whether the UAV captures the tracking object. If the tracking detection confidence of the tracking object in consecutive multiple frames of reference images is lower than the confidence threshold, and the number of reference image frames in which the above situation occurs exceeds the frame threshold, it is determined that the UAV loses the tracking object and needs to search for the tracking object.

[0088] Further, the loss time when the tracking detection confidence starts to be lower than the confidence threshold is determined, a plurality of continuous time points before the loss time are selected as prediction time points, and an effective image sequence is obtained according to the effective images of the tracking object corresponding to the prediction time points. Target detection is performed on each frame of effective image in the effective image sequence to obtain an effective state vector of the tracking object in the effective image, and the effective state vector corresponding to each frame of effective image is used to form a predicted state sequence to represent the change of the actual distribution state of the tracking object in the effective image over time.

[0089] Further, the predicted state sequence is input into a time sequence state prediction network, the time sequence state prediction network is used to learn the mode of the predicted state sequence to perform trajectory prediction of the tracking object and obtain a predicted trajectory of the tracking object. The predicted trajectory of the tracking object is used as a guide to generate a search instruction of the UAV, the UAV is controlled to go to a direction and a position where the tracking object is likely to appear, and the UAV is used to perform maneuvering detection on the tracking object at the direction and the position until the UAV re-captures the tracking object. In this embodiment, the UAV is controlled to detect the tracking object in the trajectory prediction mode, the efficiency of re-capturing the tracking object by the UAV is improved, and the tracking effect of the tracking object is effectively improved.

[0090] Correspondingly, please refer to Figure 6 The embodiment of the present application provides a UAV target tracking device, which comprises: An image sequence acquisition module 610 is configured to acquire a reference image sequence in a specified time window; wherein the reference image sequence comprises a plurality of reference images of the tracking object; and the last time point of the specified time window is the current time point.

[0091] The tracking target detection module 620 is configured to perform target detection on each reference image to obtain a reference state vector of the tracking object, and to construct a state vector sequence by using the reference state vector corresponding to each reference image; the reference state vector is used to represent an actual distribution state of the tracking object in each reference image.

[0092] The state vector prediction module 630 is configured to predict a state vector of the tracking object at a prediction time according to the state vector sequence to obtain a prediction state vector of the tracking object; the prediction state vector is used to represent a predicted distribution state of the tracking object in the current field of view of the unmanned aerial vehicle at the prediction time; the prediction time is a time after the current time by a prediction time step.

[0093] The reinforcement learning decision module 640 is configured to input the prediction state vector into a reinforcement learning policy network to generate a control instruction of the unmanned aerial vehicle at the current time, so as to control the unmanned aerial vehicle to track the tracking object; a reward function of the reinforcement learning policy network is determined according to the prediction state vector in a training process.

[0094] In some optional embodiments, the reinforcement learning decision module 640 includes: The control instruction output unit is configured to perform forward propagation calculation on the prediction state vector by using the reinforcement learning policy network to output the control instruction of the unmanned aerial vehicle at the current time; the control instruction includes a flight control instruction of the unmanned aerial vehicle and an angle change instruction of a camera carried on the unmanned aerial vehicle.

[0095] In some optional embodiments, the reinforcement learning decision module 640 further includes: The field of view center alignment unit is configured to perform field of view center alignment based on the first prediction position to obtain a prediction observation alignment reward item.

[0096] The pixel area ratio comparison unit is configured to perform pixel area ratio comparison based on the first prediction area to obtain a prediction observation scale reward item.

[0097] The vector distance calculation unit is configured to obtain a prediction accuracy reward item based on distance data between the first prediction position, the first prediction area, the first prediction confidence, and an actual state vector of the tracking object at the prediction time.

[0098] The reward fusion design unit is configured to perform fusion design on the prediction observation alignment reward item, the prediction observation scale reward item, and the prediction accuracy reward item to obtain the reward function.

[0099] In some optional embodiments, the field of view center alignment unit includes: The dimension center comparison subunit is configured to perform center comparison in a first dimension and a second dimension of the field of view of the UAV respectively by using the first predicted position, to obtain a predicted observation alignment reward item that encourages the UAV to align the tracking object with the center of the field of view.

[0100] In some optional embodiments, the pixel area comparison unit includes: The pixel scale comparison subunit is configured to perform scale comparison between the first predicted area and the expected pixel area, to obtain a predicted observation scale reward item that encourages the UAV to adjust the flight height so that the pixel area of the tracking object in the field of view is close to the expected pixel area.

[0101] In some optional embodiments, the state vector prediction module 630 includes: The mode learning prediction unit is configured to input the state vector sequence into a time sequence state prediction network, to perform mode learning on the state vector sequence by using the time sequence state prediction network, and to output a predicted state vector; the time sequence state prediction network includes a long short-term memory network.

[0102] In some optional embodiments, the device further includes a safe obstacle avoidance control module, which includes: The obstacle distance detection unit is configured to perform obstacle detection on an environment in which the UAV is located, to obtain a detection distance between the UAV and an obstacle.

[0103] The obstacle avoidance path planning unit is configured to perform obstacle avoidance path planning on the UAV when the detection distance is less than a safe distance threshold, to obtain an obstacle avoidance instruction of the UAV.

[0104] The control instruction switching unit is configured to replace the control instruction of the UAV with the obstacle avoidance instruction, to control the UAV to perform obstacle avoidance until the detection distance exceeds the safe distance threshold.

[0105] In some optional embodiments, the device further includes a loss conjecture search module, which includes: The effective image acquisition unit is configured to acquire effective images that can be used for trajectory conjecture in a case where the UAV loses the tracking object, to select a plurality of continuous time instants before a detection confidence of the tracking object is lower than a confidence threshold as conjecture time instants if the detection confidence of the tracking object in a plurality of continuous reference images is all lower than the confidence threshold, and to obtain an effective image sequence of the tracking object at the conjecture time instants; the effective image sequence includes a plurality of effective images of the tracking object.

[0106] The state vector detection unit is configured to obtain a conjecture state sequence of the tracking object according to a state vector of the tracking object in the effective images.

[0107] The trajectory speculation searching unit is configured to perform trajectory speculation on the tracking object according to the speculation state sequence, to obtain a predicted trajectory of the tracking object, and to generate a search instruction of the UAV based on the predicted trajectory, so as to control the UAV to detect the tracking object.

[0108] Further functions of the above-mentioned modules and units are the same as those of the corresponding embodiments, and will not be described herein.

[0109] The UAV target tracking device in the embodiment is in the form of a functional unit. The unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0110] Please refer to Figure 7 , Figure 7 is a structural schematic diagram of a computer device provided by the embodiment of the present application, as Figure 7 shown, the computer device includes one or more processors 10, a memory 20, and an interface for connecting various components, including a high-speed interface and a low-speed interface. Various components communicate and connect with each other by using different buses, and can be installed on a common mainboard or in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in the memory or graphics information of the memory to display a GUI on an external input / output device, such as a display device coupled to the interface. In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used with multiple memories and multiple memories. Also, multiple computer devices can be connected, each providing part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 In the embodiment, the processor 10 is taken as an example.

[0111] The processor 10 can be a central processor, a network processor, or a combination thereof. The processor 10 can further include a hardware chip. The hardware chip can be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device can be a complex programmable logic device, a field programmable logic gate array, a generic array logic, or any combination thereof.

[0112] The memory 20 stores instructions executable by the at least one processor 10, so that the at least one processor 10 executes the method shown in the above-mentioned embodiments.

[0113] The memory 20 can include a program storage area and a data storage area. The program storage area can store an operating system, application programs required by at least one function, and the like. The data storage area can store data created according to the use of the computer device, and the like. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-transitory memory such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some alternative embodiments, the memory 20 can optionally include a memory disposed remotely from the processor 10, which can be connected to the computer device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0114] The memory 20 can include a volatile memory such as a random access memory, and can also include a non-volatile memory such as a flash memory, a hard disk, or a solid state disk. The memory 20 can also include a combination of the above-mentioned kinds of memories.

[0115] The computer device also includes a communication interface 30 for communication of the computer device with other devices or communication networks.

[0116] The embodiments of the present application also provide a computer readable storage medium. The above-mentioned method according to the embodiments of the present application can be implemented in hardware, firmware, or recorded in a storage medium, or implemented as computer code originally stored in a remote storage medium or non-transitory machine readable storage medium and downloaded to a local storage medium, so that the method described herein can be processed by such software on a storage medium using a general-purpose computer, a special-purpose processor, or programmable or special-purpose hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid state disk, etc. Further, the storage medium can also include a combination of the above-mentioned kinds of memories. It can be understood that the computer, processor, microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the method shown in the above embodiments.

[0117] The embodiments of the present application provide a computer program product, which includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to make the computer device execute the method of any of the embodiments of the present application.

[0118] Although the embodiments of the present application are described in conjunction with the accompanying drawings, various modifications and changes can be suggested to one skilled in the art, and it is intended that the present application encompass such modifications and changes as fall within the scope of the appended claims.

[0119] The systems, apparatuses, modules or units illustrated by the above embodiments can be specifically implemented by a computer chip or entity, or by a product with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0120] For the sake of brevity, the above description has been described in terms of functional modules. Of course, the modules are not required to be implemented in software and / or hardware. They can be implemented in software and / or hardware, for example.

[0121] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0122] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as a combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions, which are executed via the processor of the computer or other programmable data processing apparatus, generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions of one or more flows and / or blocks in the flowcharts and / or block diagrams can be implemented by an apparatus that specifically performs the specified functions, or by a product with certain functions. Figure 1 The functions of one or more flows and / or blocks in the flowcharts and / or block diagrams can be implemented by an apparatus that specifically performs the specified functions, or by a product with certain functions.

[0123] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions of one or more flows and / or blocks in the flowcharts and / or block diagrams can be implemented by an apparatus that specifically performs the specified functions, or by a product with certain functions. Figure 1the function(s) specified in the block or blocks.

[0124] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable devices to generate a computer implemented process, thus the instructions executed on the computer or other programmable devices provide a process for implementing the flowchart Figure 1 the flowchart or flowcharts and / or a block Figure 1 the function(s) specified in the block or blocks.

[0125] It should also be noted that the term "comprising" or "including" or any other variation thereof is intended to cover non-exclusive inclusion, such that processes, methods, articles, or apparatuses that comprise a list of elements are not limited to those elements, but can include other elements not expressly listed or inherent to such processes, methods, articles, or apparatuses. Without further limitation, an element preceded by "comprises... a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that includes the recited element.

[0126] Each of the embodiments in the present specification is described in a progressive manner, and the same or similar parts between the embodiments can be mutually referred to, and each embodiment focuses on the difference from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0127] The above only describes the embodiments of the present application and is not intended to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

[0128] Although the embodiments of the present application are described in conjunction with the drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes shall fall within the scope defined by the appended claims.

Claims

1. A method for tracking targets using an unmanned aerial vehicle (UAV), characterized in that, The method includes: Obtain a reference image sequence within a specified time window; wherein, the reference image sequence includes multiple frames of reference images about the tracked object; the last moment of the specified time window is the current moment; Target detection is performed on each frame of reference image to obtain the reference state vector of the tracked object, and a state vector sequence is constructed using the reference state vectors corresponding to each frame of reference image; wherein, the reference state vector is used to represent the actual distribution state of the tracked object in each frame of reference image; The state vector of the tracked object at the prediction time is predicted based on the state vector sequence to obtain the predicted state vector of the tracked object; wherein, the predicted state vector is used to represent the predicted distribution state of the tracked object in the current field of view of the UAV at the prediction time; the prediction time is the time after the current time and the prediction time step. The predicted state vector is input into the reinforcement learning policy network to generate control commands for the UAV at the current moment, so as to control the UAV to track the target object; wherein, the reward function of the reinforcement learning policy network is determined based on the predicted state vector during the training process.

2. The method according to claim 1, characterized in that, The step of inputting the predicted state vector into the reinforcement learning policy network to generate the control command for the UAV at the current moment includes: The predicted state vector is calculated by forward propagation through the reinforcement learning policy network, and the control command of the UAV at the current moment is output. The control commands include flight control commands for the UAV and angle change commands for the camera mounted on the UAV.

3. The method according to claim 1, characterized in that, The predicted state vector during the training process includes the first predicted position, first predicted area, and first predicted confidence of the tracked object at the predicted time. The reward function is obtained in the following way: Based on the first predicted position, the center of the field of view is aligned to obtain a reward item for the predicted observation alignment. Pixel area comparison is performed based on the first predicted area to obtain the predicted observation scale reward item; Based on the distance data between the first predicted location, the first predicted area, the first predicted confidence level, and the actual state vector of the tracked object at the predicted time, a prediction accuracy reward is obtained. The reward function is obtained by fusing the predicted observation alignment reward, the predicted observation scale reward, and the predicted accuracy reward.

4. The method according to claim 3, characterized in that, The step of aligning the field of view center based on the first predicted position to obtain a predicted observation alignment reward includes: Using the first predicted position, center comparisons are performed in the first and second dimensions of the UAV's field of view to obtain the predicted observation alignment reward item that incentivizes the UAV to align the tracked object with the center of the field of view.

5. The method according to claim 3, characterized in that, The step of comparing pixel areas based on the first predicted area to obtain a predicted observation scale reward includes: The first predicted area is compared with the expected pixel area to obtain the predicted observation scale reward, which incentivizes the UAV to adjust its flight altitude so that the pixel area of ​​the tracked object in the field of view is close to the expected pixel area.

6. The method according to claim 1, characterized in that, The step of predicting the state vector of the tracked object at the prediction time based on the state vector sequence to obtain the predicted state vector of the tracked object includes: The state vector sequence is input into a temporal state prediction network, and the temporal state prediction network performs pattern learning on the state vector sequence to output the predicted state vector; wherein, the temporal state prediction network includes a long short-term memory network.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Obstacle detection is performed on the environment in which the drone is located to obtain the detection distance between the drone and the obstacle; When the detection distance is less than the safe distance threshold, obstacle avoidance path planning is performed on the UAV to obtain the obstacle avoidance command of the UAV; The control commands of the UAV are replaced with the obstacle avoidance commands to control the UAV to avoid obstacles until the detection distance exceeds the safe distance threshold.

8. The method according to any one of claims 1 to 6, characterized in that, The reference state vector includes the detection confidence of the tracked object in each frame of the reference image; the method further includes: If the detection confidence of the tracked object is lower than the confidence threshold in multiple consecutive reference images, multiple consecutive moments before the detection confidence falls below the confidence threshold are selected as inferred moments to obtain an effective image sequence of the tracked object at the inferred moments; wherein, the effective image sequence includes multiple frames of effective images of the tracked object; Based on the state vector of the tracked object in the valid image, the inferred state sequence of the tracked object is obtained; Based on the inferred state sequence, the trajectory of the tracked object is inferred to obtain the predicted trajectory of the tracked object, and the search command of the UAV is generated based on the predicted trajectory to control the UAV to detect the tracked object.

9. A computer device, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method of any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method of any one of claims 1 to 8.

Images