Tracking and positioning method and tracking and positioning device for unmanned aerial vehicle

Abstract

The invention discloses a tracking and positioning method and a tracking and positioning device for an unmanned aerial vehicle. The method comprises the following steps: acquiring a reinforcement learning tracking network; a video stream of aerial photography of the unmanned aerial vehicle is obtained, an anchor frame area is arranged in the video stream, and the anchor frame area is used for representing the position and size of a tracking target; performing feature extraction on the video stream to obtain a feature vector of the anchor frame area in the video stream; inputting the feature vector into a reinforcement learning tracking network to obtain a current action vector of the unmanned aerial vehicle; and based on the current action vector, a flight strategy of the unmanned aerial vehicle is generated, and the flight strategy is used for controlling the unmanned aerial vehicle to track and position the tracking target. The technical problem that autonomous landing of the unmanned aerial vehicle is unstable and unreliable is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle tracking, and in particular to a method and a device for tracking and positioning an unmanned aerial vehicle. Background Art

[0002] With the rapid development and widespread application of drone technology, the demand for drone operations in various fields continues to increase. However, at present, rotary-wing drones have a high failure rate when performing landing tasks, which poses a serious threat to the smooth completion of the mission and flight safety. According to statistics, the number of failures during the landing of rotary-wing drones accounts for more than 80% of the total number of mission failures. This phenomenon highlights the urgency and importance of autonomous take-off and landing technology for rotary-wing drones. The landing mission of rotary-wing drones involves complex environmental factors such as wind speed, terrain, obstacles, etc. These factors increase the risk and failure rate of drones during landing.

[0003] At present, there are still some problems with autonomous landing technology, such as low accuracy, poor adaptability to environmental changes, and susceptibility to external interference, which leads to instability and unreliability of landing missions.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a tracking and positioning method and a tracking and positioning device for a drone, so as to at least solve the technical problem that the autonomous landing of the drone is unstable and unreliable.

[0006] According to one aspect of an embodiment of the present invention, a method for tracking and locating a drone is provided, comprising: obtaining a reinforcement learning tracking network; obtaining a video stream of aerial photography by the drone, and setting an anchor frame area in the video stream, wherein the anchor frame area is used to characterize the position and size of a tracking target; performing feature extraction on the video stream to obtain a feature vector of the anchor frame area in the video stream; inputting the feature vector into the reinforcement learning tracking network to obtain a current motion vector of the drone; and generating a flight strategy for the drone based on the current motion vector, wherein the flight strategy is used to control the drone to track and locate the tracking target.

[0007] Furthermore, obtaining a reinforcement learning tracking network includes constructing and training a reinforcement learning tracking network, and constructing and training the reinforcement learning tracking network includes: constructing an environment state, which is used to characterize the position relationship of the anchor frame area in the frame image, wherein the environment state includes a feature vector and the position information of the anchor frame area; constructing an action set, which is composed of historical action vectors; constructing a state transition function, which is used to generate a position result of the anchor frame area in the current frame image according to the environment state and the action set; constructing a reward function, which is used to generate a reward value according to the difference in the IoU value of the anchor frame area between the current frame image and the previous frame image, wherein the IoU value is the area coverage ratio of the predicted position and the actual position of the anchor frame area; based on the reward value, adjusting the network parameters of the reinforcement learning tracking network to obtain an updated reinforcement learning tracking network.

[0008] Furthermore, the reinforcement learning tracking network is a DQN model.

[0009] Furthermore, a video stream of drone aerial photography is obtained, and an anchor frame area is set in the video stream, including: setting the anchor frame area in the first frame image in the video stream; and processing the real-time frame image in the video stream according to the position information of the anchor frame area in the first frame image to identify the anchor frame area in the real-time frame image.

[0010] Furthermore, setting an anchor frame area in the first frame image in the video stream includes: selecting a target area in the first frame image, wherein the tracking target is located in the target area; and setting an anchor point in the target area, wherein the anchor point is used to indicate a direction of the tracking target.

[0011] Furthermore, feature extraction is performed on the video stream to generate a feature vector of the anchor frame area in the video stream, including: in a convolutional layer, the frame image in the video stream is divided into multiple image blocks by means of a sliding window, and feature extraction is performed on the image blocks to obtain a feature vector set; based on the position information of the anchor frame area in the frame image, the feature vector corresponding to the anchor frame area in the feature vector set is identified.

[0012] According to another aspect of an embodiment of the present invention, a tracking and positioning device for a drone is also provided, including: a first acquisition module, the first acquisition module is used to acquire a reinforcement learning tracking network; a second acquisition module, the second acquisition module is used to acquire a video stream of aerial photography of the drone, and set an anchor frame area in the video stream, wherein the anchor frame area is used to characterize the position and size of the tracking target; a feature extraction module, the feature extraction module is used to extract features from the video stream and obtain a feature vector of the anchor frame area in the video stream; a first generation module, the first generation module is used to input the feature vector into the reinforcement learning tracking network to generate a current action vector of the drone; and a second generation module, the generation module is used to generate an operation strategy of the drone based on the current action vector.

[0013] According to another aspect of an embodiment of the present invention, a computer storage medium is provided. The computer storage medium includes a stored program, wherein when the program is executed, the device where the computer storage medium is located is controlled to execute the above-mentioned tracking and positioning method.

[0014] According to another aspect of an embodiment of the present invention, a processor is further provided. The processor is used to run a program. The processor is configured to run a computer program to execute the above-mentioned tracking and positioning method.

[0015] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the above-mentioned tracking and positioning method is implemented.

[0016] In an embodiment of the present invention, the video stream of the drone aerial photography is processed in real time to obtain the feature vector of the anchor frame area, and the feature vector obtained in real time is input into the reinforcement learning tracking network to obtain the current action vector of the drone, and a flight strategy is generated based on the current action vector. The flight strategy controls the drone to track and locate the tracking target. In the above scheme, the drone uses the reinforcement learning tracking network to track and locate the tracking target. The reinforcement learning tracking network enables the drone to learn the optimal flight strategy in various environments. When facing a dynamic environment or target, it can quickly make decisions and adjustments to achieve smooth flight and landing, thereby solving the unstable and unreliable technical problems of autonomous landing of drones. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0018] Figure 1 It is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for tracking and locating a drone according to an embodiment of the present invention;

[0019] Figure 2 is a flow chart of the tracking and positioning method of the drone in the present invention;

[0020] Figure 3 It is a structural block diagram of the tracking and positioning device of the drone in the present invention. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] Example 1

[0024] According to an embodiment of the present invention, an embodiment of a method for tracking and positioning a drone is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0025] Figure 1 1 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for tracking and locating a drone according to an embodiment of the present invention. Figure 1 As shown, the computer terminal (or mobile device) may include one or more processors 102 (the processor may include but is not limited to a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing (DSP) chip, a microprocessor (MCU), a field programmable logic device (FPGA), a neural network processor (NPU), a tensor processing unit (TPU), an artificial intelligence (AI) type processor, etc.) and a memory 104 for storing data. In addition, a transmission device 106 for communication functions, an input and output device 108, and a display 110 may also be included. It can be understood by those skilled in the art that Figure 1The structure shown is for illustration only and does not limit the structure of the above-mentioned computer terminal (or mobile device). For example, the computer terminal may also include more or fewer components than the above-mentioned structure description, or have a configuration different from the above-mentioned structure description.

[0026] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the tracking and positioning method of the drone in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the tracking and positioning method of the drone described above is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0027] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0028] The display 110 may be a touch screen type liquid crystal display (LCD). The liquid crystal display may enable a user to interact with a user interface of the mobile terminal. In some embodiments, the mobile terminal has a graphical user interface (GUI), and a user may interact with the GUI by finger contact and / or gestures on a touch-sensitive surface, wherein the human-computer interaction functions here may optionally include the following interactions: creating web pages, drawing, word processing, making electronic documents, games, video conferencing, instant messaging, sending and receiving emails, call interfaces, playing digital videos, playing digital music and / or web browsing, etc. Executable instructions for performing the above human-computer interaction functions are configured / stored in a computer program product or readable storage medium executable by one or more processors.

[0029] Figure 2 FIG. 1 is a flow chart of a method for tracking and positioning a drone according to one embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0030] Step S1: Obtain reinforcement learning tracking network.

[0031] Specifically, the reinforcement learning tracking network is trained using a large amount of historical flight data as sample data, and is continuously iterated and updated during subsequent use.

[0032] Step S2: Obtain a video stream of drone aerial photography and set an anchor frame area in the video stream, where the anchor frame area is used to characterize the position and size of the tracking target.

[0033] Specifically, data processing is performed on the video stream acquired in real time to mark the real-time position of the tracking target in the video stream.

[0034] Step S3: extract features from the video stream to obtain a feature vector of the anchor frame area in the video stream.

[0035] Step S4: Input the feature vector into the reinforcement learning tracking network to obtain the current action vector of the drone.

[0036] Specifically, the current motion vector includes at least a flight direction and a flight altitude.

[0037] Step S5: Generate a flight strategy for the UAV based on the current motion vector, wherein the flight strategy is used to control the UAV to track and locate the target.

[0038] In an embodiment of the present application, the video stream of the drone aerial photography is processed in real time to obtain the feature vector of the anchor frame area, and the feature vector obtained in real time is input into the reinforcement learning tracking network to obtain the current action vector of the drone, and a flight strategy is generated based on the current action vector. The flight strategy controls the drone to track and locate the tracking target. In the above scheme, the drone uses the reinforcement learning tracking network to track and locate the tracking target. The reinforcement learning tracking network enables the drone to learn the optimal flight strategy in various environments. When facing a dynamic environment or target, it can quickly make decisions and adjustments to achieve smooth flight and landing, thereby solving the unstable and unreliable technical problems of autonomous landing of drones.

[0039] In an exemplary embodiment of the present application, obtaining the reinforcement learning tracking network in step S1 includes constructing and training the reinforcement learning tracking network, and constructing and training the reinforcement learning tracking network includes the following steps:

[0040] Step S11: constructing an environment state, where the environment state is used to characterize the positional relationship of the anchor frame area in the frame image, wherein the environment state includes a feature vector and positional information of the anchor frame area.

[0041] Specifically, the environment state is represented as a tuple (fv, L), where fv is the historical feature vector generated by the convolutional neural network and L is the position and size of the current anchor box area.

[0042] L = (x, y, w, h), where (x, y) represents the position coordinates of the upper left corner of the anchor box area, w represents the width of the anchor box area, and h represents the height of the anchor box area.

[0043] In the corresponding frame image F, the corresponding vector p can be generated t , p t =Ω(FL), where Ω represents the preprocessing function, which cuts out p from F t The anchor box area is obtained by , and the size of the corresponding anchor box area is adjusted to match the input requirements of the feature extractor.

[0044] Step S12: construct an action set, which consists of historical action vectors.

[0045] Specifically, the historical action vector is a preset action instruction generated by the reinforcement learning tracking network, that is, the environmental state of the previous frame can be generated after the corresponding action vector acts on the environmental state of the current frame.

[0046] Step S13: construct a state transition function, which is used to generate the position result of the anchor box area in the current frame image according to the environment state and action set.

[0047] Specifically, the previous environment state is si, and after taking action ai, it changes to the next environment state si+1. The environment state is represented by a tuple (fv, L), where fv = {fl, a1, f2, a2, f3, a3, f4}, fi is a feature vector generated by a convolutional neural network, and L is the position and size of the anchor box area corresponding to f4, L = (x, y, w, h).

[0048] Exemplarily, in each anchor box region, Δx=αw, Δy=αh, where α is 0.01.

[0049] Exemplarily, after taking an action, L = (x, y + Δy / 2, w, h - Δy), a new action vector a4 and a new feature vector f5 can be obtained, fvnew = {f2, a2, f3, a3, f4, a4, f5}.

[0050] Step S14: construct a reward function, which is used to generate a reward value according to the difference in the IoU value between the current frame image and the previous frame image of the anchor box area, where the IoU value is the area coverage ratio of the predicted position and the actual position of the anchor box area.

[0051] Specifically, the intersection-over-union (IoU) method is used to represent the visual tracking accuracy between the predicted anchor box area and the true anchor box area, that is, a reward function is given according to the difference in IoU between two adjacent environment states.

[0052] For example, L is regarded as the predicted anchor box area, G is the real anchor box area, and the IoU function between L and G is expressed as:

[0053]

[0054] Reward function R A (s, s0) indicates that the agent will receive a reward when the previous environment state s is transformed to the next environment state s0 by taking action A. Each state s has an associated anchor box region L, which contains the region of interest, i.e., the predicted location of the tracked target. A The function of (s,s0) is expressed as:

[0055]

[0056] Among them, the reward value is defined to be positive when the IoU increases, and the agent will receive a negative reward when the IoU decreases. Therefore, the model will be punished for moving the anchor box area out of the visible area of ​​the frame image, and will be rewarded for keeping the anchor box area in the visible area of ​​the frame image until there is no other transformation that can improve the positioning.

[0057] Step S15: Based on the reward value, adjust the network parameters of the reinforcement learning tracking network to obtain an updated reinforcement learning tracking network.

[0058] Specifically, the reinforcement learning tracking network is updated by minimizing the loss function, which is expressed as: L t (θ t )=E s,a [(y t -Q(s,a,θ t )) 2 ], where y t represents the target optimal Q value, λ is the discount factor, and α is the learning rate. By updating θ, the reinforcement learning tracking network is updated.

[0059] Preferably, the reinforcement learning tracking network is a DQN model.

[0060] In an exemplary embodiment of the present application, obtaining a video stream of drone aerial photography and setting an anchor frame area in the video stream in step S2 includes the following steps:

[0061] Step S21: setting an anchor frame area in the first frame image in the video stream.

[0062] Step S22: Process the real-time frame image in the video stream according to the position information of the anchor frame area in the first frame image to identify the anchor frame area in the real-time frame image.

[0063] Through the above steps S21 to S22, after the anchor frame area is manually determined in the first frame image, the anchor frame areas of subsequent other frame images are automatically identified using the system program.

[0064] Furthermore, in step S21, an anchor frame area is set in the first frame image in the video stream, and the specific steps are as follows:

[0065] Step S211: select a target area in the first frame image, and the tracking target is located in the target area.

[0066] Specifically, an anchor frame area is set on the ground control end, and after the target area is determined in the first frame image, the target area is manually framed and selected using a mouse to determine the boundary and range of the target.

[0067] Step S212: An anchor point is set in the target area, and the anchor point is used to indicate the direction of the tracking target.

[0068] Specifically, an anchor point is set at the edge or center of the target area, wherein the anchor point at the edge can be used to identify whether the target is performing a rotation action at a certain position.

[0069] In an exemplary embodiment, the specific process of obtaining the video stream of the drone aerial photography is as follows:

[0070] Connect the optical sensor equipped on the drone to the onboard computing device to ensure that the video stream can be transmitted in real time. This step requires selecting suitable sensors and computing devices and ensuring that the interface and communication protocol between them can stably transmit video data.

[0071] Using ffmpeg-based streaming technology, the ground control end can obtain the video stream of the drone aerial photography. Through Wi-Fi or 5G connection, the drone can be remotely monitored and controlled, and the video stream can be transmitted to the ground control end in real time so that the user can view the pictures taken by the drone in real time. On the ground control end, the ffmpeg-based streaming service is used to receive, decode and display the video stream.

[0072] Furthermore, in step S3, feature extraction is performed on the video stream to generate a feature vector of the anchor frame area in the video stream, including the following steps:

[0073] Step S31: In the convolution layer, the frame image in the video stream is divided into multiple image blocks by means of a sliding window, and features are extracted from the image blocks to obtain a feature vector set.

[0074] Step S32: According to the position information of the anchor frame area in the frame image, identify the feature vector corresponding to the anchor frame area in the feature vector set.

[0075] Specifically, in the processing of the convolution layer, the convolution operation is used to extract the features of the input data. In each convolution layer, convolution kernels of different sizes are used to capture feature information of different scales, thereby improving the network's ability to understand the input data. At the same time, the relu activation function is introduced to enhance the nonlinear expression ability of the network and better learn complex feature representations.

[0076] In the processing of the fully connected layer, the feature vector output by the convolutional layer is flattened and mapped to a 512-dimensional output vector through a fully connected operation. The fully connected layer combines and integrates features at different positions to obtain a more global and comprehensive feature representation. The dropout technology is also introduced in the fully connected layer to randomly discard the output of some neurons to prevent network overfitting and improve generalization ability.

[0077] Furthermore, in step S4, the feature vector is input into the reinforcement learning tracking network to obtain the current action vector of the drone.

[0078] For example, the probability of an action is predicted by combining a fully connected layer and a softmax layer. The whole process aims to convert the input into an effective feature representation and predict the action probability based on these representations, specifically:

[0079] (1) The output of the fully connected layer, i.e. the result of the linear transformation, is converted into a probability distribution through the Softmax function. Specifically, it is expressed as a vector, in which each element represents the predicted probability of the corresponding category.

[0080] (2) UAV action classification and prediction. In the context of UAV control, action categories include forward, backward, left, right, up, down, etc. Each category corresponds to a specific probability value, which is output by the softmax layer. According to the probability distribution output by the softmax layer, the action with the highest probability is selected as the prediction output, thereby guiding the actual action of the UAV.

[0081] (3) After the predicted action is determined, the Kalman filter smoothing process is used to optimize the UAV's action execution to ensure the safety and feasibility of the action.

[0082] According to another specific embodiment of the present application, a tracking and positioning device for a drone is also provided.

[0083] Specifically, Figure 3As shown, the tracking and positioning device includes a first acquisition module, a second acquisition module, a feature extraction module, a first generation module and a second generation module. The first acquisition module is used to acquire a reinforcement learning tracking network. The second acquisition module is used to acquire a video stream of aerial photography of a drone and set an anchor frame area in the video stream, wherein the anchor frame area is used to characterize the position and size of the tracking target. The feature extraction module is used to extract features from the video stream to obtain a feature vector of the anchor frame area in the video stream. The first generation module is used to input the feature vector into the reinforcement learning tracking network to generate the current action vector of the drone. The generation module is used to generate an operation strategy of the drone based on the current action vector.

[0084] In an embodiment of the present application, a tracking and positioning device is used to process the video stream of the drone aerial photography in real time to obtain the feature vector of the anchor frame area, and the feature vector obtained in real time is input into the reinforcement learning tracking network to obtain the current action vector of the drone, and a flight strategy is generated based on the current action vector. The flight strategy controls the drone to track and locate the tracking target. In the above scheme, the drone uses a reinforcement learning tracking network to track and locate the tracking target. The reinforcement learning tracking network enables the drone to learn the optimal flight strategy in various environments. When facing a dynamic environment or target, it can quickly make decisions and adjustments to achieve smooth flight and landing, thereby solving the unstable and unreliable technical problems of autonomous landing of drones.

[0085] An embodiment of the present application further provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the tracking and positioning method in the above embodiment when running.

[0086] In this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0087] Step S1: Obtain reinforcement learning tracking network.

[0088] Step S2: Obtain a video stream of drone aerial photography and set an anchor frame area in the video stream, where the anchor frame area is used to characterize the position and size of the tracking target.

[0089] Step S3: extract features from the video stream to obtain a feature vector of the anchor frame area in the video stream.

[0090] Step S4: Input the feature vector into the reinforcement learning tracking network to obtain the current action vector of the drone.

[0091] Step S5: Generate a flight strategy for the UAV based on the current motion vector, wherein the flight strategy is used to control the UAV to track and locate the target.

[0092] In an embodiment of the present application, the video stream of the drone aerial photography is processed in real time to obtain the feature vector of the anchor frame area, and the feature vector obtained in real time is input into the reinforcement learning tracking network to obtain the current action vector of the drone, and a flight strategy is generated based on the current action vector. The flight strategy controls the drone to track and locate the tracking target. In the above scheme, the drone uses the reinforcement learning tracking network to track and locate the tracking target. The reinforcement learning tracking network enables the drone to learn the optimal flight strategy in various environments. When facing a dynamic environment or target, it can quickly make decisions and adjustments to achieve smooth flight and landing, thereby solving the unstable and unreliable technical problems of autonomous landing of drones.

[0093] An embodiment of the present application further provides a processor, which is configured to run a computer program to execute the tracking and positioning method in the above embodiment.

[0094] In this embodiment, the processor may be configured to perform the following steps through a computer program:

[0095] Step S1: Obtain reinforcement learning tracking network.

[0096] Step S2: Obtain a video stream of drone aerial photography and set an anchor frame area in the video stream, where the anchor frame area is used to characterize the position and size of the tracking target.

[0097] Step S3: extract features from the video stream to obtain a feature vector of the anchor frame area in the video stream.

[0098] Step S4: Input the feature vector into the reinforcement learning tracking network to obtain the current action vector of the drone.

[0099] Step S5: Generate a flight strategy for the UAV based on the current motion vector, wherein the flight strategy is used to control the UAV to track and locate the target.

[0100] In an embodiment of the present application, the video stream of the drone aerial photography is processed in real time to obtain the feature vector of the anchor frame area, and the feature vector obtained in real time is input into the reinforcement learning tracking network to obtain the current action vector of the drone, and a flight strategy is generated based on the current action vector. The flight strategy controls the drone to track and locate the tracking target. In the above scheme, the drone uses the reinforcement learning tracking network to track and locate the tracking target. The reinforcement learning tracking network enables the drone to learn the optimal flight strategy in various environments. When facing a dynamic environment or target, it can quickly make decisions and adjustments to achieve smooth flight and landing, thereby solving the unstable and unreliable technical problems of autonomous landing of drones.

[0101] An embodiment of the present application further provides a computer program product, including a computer program, which is the tracking and positioning method in the above embodiment when the computer program is executed by a processor.

[0102] In this embodiment, the computer program product may be configured to perform the following steps through a computer program:

[0103] Step S1: Obtain reinforcement learning tracking network.

[0104] Step S2: Obtain a video stream of drone aerial photography and set an anchor frame area in the video stream, where the anchor frame area is used to characterize the position and size of the tracking target.

[0105] Step S3: extract features from the video stream to obtain a feature vector of the anchor frame area in the video stream.

[0106] Step S4: Input the feature vector into the reinforcement learning tracking network to obtain the current action vector of the drone.

[0107] Step S5: Generate a flight strategy for the UAV based on the current motion vector, wherein the flight strategy is used to control the UAV to track and locate the target.

[0108] In an embodiment of the present application, the video stream of the drone aerial photography is processed in real time to obtain the feature vector of the anchor frame area, and the feature vector obtained in real time is input into the reinforcement learning tracking network to obtain the current action vector of the drone, and a flight strategy is generated based on the current action vector. The flight strategy controls the drone to track and locate the tracking target. In the above scheme, the drone uses the reinforcement learning tracking network to track and locate the tracking target. The reinforcement learning tracking network enables the drone to learn the optimal flight strategy in various environments. When facing a dynamic environment or target, it can quickly make decisions and adjustments to achieve smooth flight and landing, thereby solving the unstable and unreliable technical problems of autonomous landing of drones.

[0109] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0110] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0111] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0112] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0113] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0114] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0115] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for tracking and positioning a drone, characterized in that: include: Get the reinforcement learning tracking network; Acquire a video stream of drone aerial photography, and set an anchor frame area in the video stream, wherein the anchor frame area is used to represent the position and size of the tracking target; Performing feature extraction on the video stream to obtain a feature vector of the anchor frame area in the video stream; Inputting the feature vector into the reinforcement learning tracking network to obtain the current action vector of the drone; A flight strategy of the UAV is generated based on the current motion vector, wherein the flight strategy is used to control the UAV to track and locate the tracking target.

2. The tracking and positioning method according to claim 1, characterized in that: Acquiring a reinforcement learning tracking network includes constructing and training the reinforcement learning tracking network, and constructing and training the reinforcement learning tracking network includes: Constructing an environment state, where the environment state is used to characterize the positional relationship of the anchor frame region in the frame image, wherein the environment state includes the feature vector and the positional information of the anchor frame region; Constructing an action set, wherein the action set is composed of historical action vectors; Constructing a state transition function, wherein the state transition function is used to generate a position result of the anchor frame area in the current frame image according to the environment state and the action set; Constructing a reward function, the reward function is used to generate a reward value according to the difference of the IoU value between the current frame image and the previous frame image of the anchor frame area, wherein the IoU value is the area coverage ratio of the predicted position and the actual position of the anchor frame area; Based on the reward value, the network parameters of the reinforcement learning tracking network are adjusted to obtain an updated reinforcement learning tracking network.

3. The tracking and positioning method according to claim 2, characterized in that: The reinforcement learning tracking network is a DQN model.

4. The tracking and positioning method according to claim 1, characterized in that: Obtain a video stream taken by a drone and set an anchor frame area in the video stream, including: Setting the anchor frame area in the first frame image in the video stream; The real-time frame image in the video stream is processed according to the position information of the anchor frame area in the first frame image to identify the anchor frame area in the real-time frame image.

5. The tracking and positioning method according to claim 4, characterized in that: Setting the anchor frame area in the first frame image in the video stream includes: Selecting a target area in the first frame image, wherein the tracking target is located in the target area; An anchor point is set in the target area, and the anchor point is used to indicate the direction of the tracking target.

6. The tracking and positioning method according to claim 1, characterized in that: Extracting features from the video stream to generate a feature vector of the anchor frame area in the video stream includes: In the convolution layer, the frame image in the video stream is divided into a plurality of image blocks by means of a sliding window, and features are extracted from the image blocks to obtain the feature vector set; According to the position information of the anchor frame area in the frame image, the feature vector corresponding to the anchor frame area in the feature vector set is identified.

7. A tracking and positioning device for an unmanned aerial vehicle, characterized in that: include: A first acquisition module, the first acquisition module is used to acquire a reinforcement learning tracking network; A second acquisition module, the second acquisition module is used to acquire a video stream of aerial photography by a drone, and set an anchor frame area in the video stream, wherein the anchor frame area is used to represent the position and size of the tracking target; A feature extraction module, wherein the feature extraction module is used to extract features from the video stream to obtain a feature vector of the anchor frame area in the video stream; A first generating module, the first generating module is used to input the feature vector into the reinforcement learning tracking network to generate a current action vector of the drone; A second generating module is used to generate an operation strategy of the UAV based on the current action vector.

8. A computer storage medium, characterized in that: The computer storage medium includes a stored program, wherein when the program is executed, the device where the computer storage medium is located is controlled to execute the tracking and positioning method according to any one of claims 1 to 6.

9. A processor, characterized in that: The processor is used to run a program, and the processor is configured to run a computer program to execute the tracking and positioning method according to any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the tracking and positioning method according to any one of claims 1 to 6 is implemented.

Images