Target tracking method, electronic device, and computer-readable storage medium

Abstract

The application discloses a target tracking method, an electronic device and a computer readable storage medium. The method comprises: acquiring a video captured by a UAV during movement, wherein a plurality of video frames in the video contain a target object; performing target tracking on the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory comprises initial positions of the target object in the plurality of video frames; determining a homographic transformation matrix between the plurality of video frames, wherein the homographic transformation matrix contains a mapping relationship between different video frames in the plurality of video frames; and transforming the initial trajectory based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein the position of the preset object remains unchanged. The application solves the technical problem of low accuracy in tracking a target object in the related art.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of target tracking, in particular, to a target tracking method, an electronic device and a computer readable storage medium. BACKGROUND

[0002] Target tracking is an important function application in the unmanned aerial vehicle scene. Due to the particularity of the unmanned aerial vehicle scene, the unmanned aerial vehicle itself is moving, so the target trajectory obtained by the current method is actually the result of superimposing the real trajectory of the target and the motion trajectory of the unmanned aerial vehicle. Therefore, the accuracy of tracking the target object using the unmanned aerial vehicle needs to be improved.

[0003] In view of the above problems, no effective solution has been proposed so far. SUMMARY

[0004] The embodiments of the present application provide a target tracking method, an electronic device and a computer readable storage medium to at least solve the technical problem of low accuracy in tracking a target object in the related art.

[0005] According to an aspect of an embodiment of the present application, a target tracking method is provided, comprising: obtaining a video captured by an unmanned aerial vehicle during movement, wherein a plurality of video frames in the video contain a target object; performing target tracking on the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory comprises initial positions of the target object in the plurality of video frames; determining a homographic transformation matrix between the plurality of video frames, wherein the homographic transformation matrix contains a mapping relationship between different video frames in the plurality of video frames; and transforming the initial trajectory based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein the position of the preset object remains unchanged.

[0006] According to another aspect of an embodiment of the present application, a target tracking method is also provided, comprising: in response to an input instruction acting on an operation interface, displaying a video captured by an unmanned aerial vehicle during movement on the operation interface, wherein a plurality of video frames in the video contain a target object; and in response to a target tracking instruction acting on the operation interface, displaying a target trajectory of the target object relative to a preset object on the operation interface, wherein the position of the preset object remains unchanged, the target trajectory is obtained by transforming an initial trajectory of the target object based on a homographic transformation matrix between the plurality of video frames, the initial trajectory is obtained by performing target tracking on the plurality of video frames, and the initial trajectory comprises initial positions of the target object in the plurality of video frames.

[0007] According to another aspect of the embodiments of the present application, a target tracking method is also provided, including: displaying a video taken by a UAV during movement on a presentation screen of a VR device or an AR device, wherein a plurality of video frames in the video contain a target object; performing target tracking on the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory includes initial positions of the target object in the plurality of video frames; determining a homographic transformation matrix between the plurality of video frames, wherein the homographic transformation matrix contains mapping relationships between different video frames in the plurality of video frames; transforming the initial trajectory based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein a position of the preset object remains unchanged; and driving the VR device or the AR device to render and display the target trajectory.

[0008] According to another aspect of the embodiments of the present application, a target tracking method is also provided, including: obtaining a video taken by a UAV during movement by calling a first interface, wherein the first interface includes a first parameter, and a parameter value of the first parameter is the video, and a plurality of video frames in the video contain a target object; performing target tracking on the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory includes initial positions of the target object in the plurality of video frames; determining a homographic transformation matrix between the plurality of video frames, wherein the homographic transformation matrix contains mapping relationships between different video frames in the plurality of video frames; transforming the initial trajectory based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein a position of the preset object remains unchanged; and outputting the target trajectory by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter is the target trajectory.

[0009] According to another aspect of the embodiments of the present application, an electronic device is also provided, including: a memory storing an executable program; and a processor configured to execute the program, wherein the program, when executed, performs the method of any one of the above embodiments.

[0010] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, including a stored executable program, wherein the executable program, when executed, controls a device where the computer readable storage medium is located to perform the method of any one of the above embodiments.

[0011] In the embodiments of the present application, a video captured by a UAV during movement is acquired, wherein a plurality of video frames in the video contain a target object; target tracking is performed on the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory includes initial positions of the target object in the plurality of video frames; a homographic transformation matrix between the plurality of video frames is determined, wherein the homographic transformation matrix contains a mapping relationship between different video frames in the plurality of video frames; the initial trajectory is transformed based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein the position of the preset object remains unchanged, thereby achieving the purpose of restoring the real motion trajectory of the target object. It is easy to note that the motion trajectory of the UAV during the video capturing process can be determined by determining the homographic transformation matrix between the plurality of video frames, and the influence of the motion of the UAV on the motion trajectory of the target object can be eliminated by transforming the initial trajectory based on the homographic transformation matrix, so that the real motion trajectory of the target object, i.e., the target trajectory described above, can be restored, thereby improving the accuracy of tracking the target object, and further solving the technical problem of low accuracy of tracking the target object in the related art.

[0012] It is easy to note that the general description and the following detailed description are only for the purpose of illustrating and explaining the present application, and do not constitute a limitation on the present application. BRIEF DESCRIPTION OF DRAWINGS

[0013] The drawings described herein are used to provide further understanding of the present application, constitute a part of the present application, and the illustrative embodiments of the present application and the description thereof are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0014] Figure 1 is a schematic diagram of a hardware environment of a virtual reality device according to a target tracking method of an embodiment of the present application;

[0015] Figure 2 is a structural block diagram of a computing environment according to a target tracking method of an embodiment of the present application;

[0016] Figure 3 is a flowchart of a target tracking method according to embodiment 1 of the present application;

[0017] Figure 4 is a schematic diagram of a homographic transformation matrix according to an embodiment of the present application;

[0018] Figure 5 is a flowchart of a target tracking method according to embodiment 2 of the present application;

[0019] Figure 6 is a flowchart of a target tracking method according to embodiment 3 of the present application;

[0020] Figure 7 is a flow chart of a target tracking method according to an embodiment of the present application;

[0021] Figure 8 is a schematic diagram of a target tracking device according to an embodiment of the present application;

[0022] Figure 9 is a schematic diagram of a target tracking device according to an embodiment of the present application;

[0023] Figure 10 is a schematic diagram of a target tracking device according to an embodiment of the present application;

[0024] Figure 11 is a schematic diagram of a target tracking device according to an embodiment of the present application;

[0025] Figure 12 is a structural block diagram of a computer terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the personnel in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0028] First, some nouns or terms appearing in the description of the embodiments of the present application are applicable to the following explanations:

[0029] Multi-target tracking: predicting the motion trajectories of all targets in a video through deep learning method;

[0030] Homography matrix: The homography matrix can represent the transformation relationship of shooting the same scene from different perspectives.

[0031] Example 1

[0032] According to an embodiment of this application, a target tracking method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0033] Figure 1 This is a schematic diagram of the hardware environment of a virtual reality device according to an embodiment of the target tracking method of this application. Figure 1 As shown, the virtual reality device 104 is connected to the terminal 106, and the terminal 106 is connected to the server 102 via a network. The virtual reality device 104 is not limited to: virtual reality helmets, virtual reality glasses, virtual reality all-in-one machines, etc. The terminal 104 is not limited to PCs, mobile phones, tablets, etc. The server 102 can be a server corresponding to a media file operator. The network includes, but is not limited to: wide area network, metropolitan area network, or local area network.

[0034] Optionally, the virtual reality device 104 in this embodiment includes a memory, a processor, and a transmission device. The memory stores an application program that can be used to perform: acquiring video captured by a drone during its movement, wherein multiple video frames in the video contain a target object; tracking the target object across the multiple video frames to obtain an initial trajectory of the target object, wherein the initial trajectory includes the initial position of the target object in the multiple video frames; determining a homography transformation matrix between the multiple video frames, wherein the homography transformation matrix contains the mapping relationship between different video frames in the multiple video frames; and transforming the initial trajectory based on the homography transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein the position of the preset object remains unchanged, thereby solving the technical problem of low accuracy in tracking target objects in related technologies.

[0035] The terminal in this embodiment can be used to display the target trajectory of the target object relative to a preset object on the display screen of a virtual reality (VR) device or an augmented reality (AR) device, and send the target trajectory of the target object relative to the preset object to the virtual reality device 104. After receiving the target trajectory of the target object relative to the preset object, the virtual reality device 104 displays it at the target projection position.

[0036] Optionally, the virtual reality device 104 of this embodiment has an eye tracking HMD (Head Mount Display) head set and an eye tracking module, which have the same functions as the above-mentioned embodiments, i.e., the screen in the HMD head set is used to display real-time images, and the eye tracking module in the HMD is used to obtain the real-time movement trajectory of the user's eyeballs. The terminal of this embodiment obtains the position information and movement information of the user in the real three-dimensional space through the tracking system, and calculates the three-dimensional coordinates of the user's head in the virtual three-dimensional space and the direction of the user's field of view in the virtual three-dimensional space.

[0037] Figure 1 The hardware structure diagram shown not only can be used as an exemplary block diagram of the above-mentioned AR / VR device (or mobile device), but also can be used as an exemplary block diagram of the above-mentioned server, and in an optional embodiment, Figure 2 The above-mentioned AR / VR device (or mobile device) is used as an embodiment of a computing node in a computing environment 201. Figure 1 The AR / VR device (or mobile device) shown is used as an embodiment of a computing node in a computing environment 201. Figure 2 is a structural block diagram of a computing environment of a target tracking method according to an embodiment of the present application, as shown in Figure 2 As shown, the computing environment 201 includes a plurality of (210-1, 210-2, …, are used in the figure to show) computing nodes (such as servers) running on a distributed network. Different computing nodes contain local processing and memory resources, and end users 202 can remotely run application programs or store data in the computing environment 201. Application programs can be provided as a plurality of services 220-1, 220-2, 220-3 and 220-4 in the computing environment 201, representing services "A", "D", "E" and "H", respectively.

[0038] End users 202 can provide and access services through web browsers or other software applications on the client, and in some embodiments, the provision and / or requests of end users 202 can be provided to an entry gateway 230. The entry gateway 230 can include a corresponding agent to process the provision and / or requests for services (one or more services provided in the computing environment 201).

[0039] Services are provided or deployed in accordance with various virtualization technologies supported by the computing environment 201. In some embodiments, services can be provided in accordance with virtual machine (VM)-based virtualization, container-based virtualization, and / or the like. VM-based virtualization can be emulating a real computer by initializing a virtual machine, executing programs and applications without directly accessing any actual hardware resources. While the virtual machine is virtualized, in accordance with container-based virtualization, a container can be launched to virtualize an entire operating system (OS) so that multiple workloads can run on a single OS instance.

[0040] In one embodiment of container-based virtualization, several containers of a service can be assembled into a Pod (e.g., a Kubernetes Pod). For example, as shown in Figure 2 Service 220-2 can be equipped with one or more Pods 240-1, 240-2, …, 240-N (collectively, Pods). A Pod can include a proxy 245 and one or more containers 242-1, 242-2, …, 242-M (collectively, containers). The one or more containers in a Pod handle requests related to one or more respective functions of the service, and the proxy 245 generally controls network functions related to the service, such as routing, load balancing, and the like. Other services can also be equipped with similar Pods.

[0041] In operation, executing a user request from the end user 202 can require invoking one or more services in the computing environment 201, and executing one or more functions of a service can require invoking one or more functions of another service. As shown in Figure 2 Service “A” 220-1 receives a user request from the end user 202 from the ingress gateway 230, and service “A” 220-1 can invoke service “D” 220-2, which can request service “E” 220-3 to execute one or more functions.

[0042] The computing environment described above can be a cloud computing environment, and the allocation of resources is managed by a cloud service provider, allowing the development of functions without considering the implementation, adjustment, or expansion of servers. The computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be split into a set of functions that can automatically scale independently, rather than expanding a single hardware device to handle potential loads.

[0043] In the above-described operating environment, the present disclosure provides a target tracking method as shown in Figure 3 It should be noted that the target tracking method of this embodiment can be executed by the mobile terminal of the embodiment shown in Figure 1 It should be noted that the target tracking method of this embodiment can be executed by the mobile terminal of the embodiment shown inFigure 3 is a flowchart of a target tracking method according to Embodiment 1 of the present application. As shown in Figure 3 , the method can include the following steps:

[0044] Step S302, acquiring a video photographed by a UAV in a movement process.

[0045] wherein the plurality of video frames in the video contain a target object.

[0046] The UAV described above can be a UAV with a photographing function.

[0047] The target object described above can be one or multiple, and the target object includes but is not limited to a person, an animal, and a machine, and the target object to be tracked can be determined according to actual conditions.

[0048] In an alternative embodiment, the UAV can photograph a video of the target object in a movement process, so as to track the target object through the plurality of video frames contained in the video.

[0049] Step S304, performing target tracking on the plurality of video frames to obtain an initial trajectory of the target object.

[0050] wherein the initial trajectory includes an initial position of the target object in the plurality of video frames.

[0051] In an alternative embodiment, the target tracking algorithm can be used to perform target tracking on the plurality of video frames to obtain the initial trajectory of the target object, for example, the initial position of the target object in the plurality of video frames can be marked by a detection box, and the initial positions in the plurality of video frames can be combined to obtain the initial trajectory of the target object. However, it should be noted that since the UAV is also moving, the obtained initial trajectory of the target object is not the actual trajectory of the target object, and subsequent processing of the initial trajectory of the target object is required to obtain the actual trajectory of the target object.

[0052] Step S306, determining a homographic transformation matrix between the plurality of video frames.

[0053] wherein the homographic transformation matrix contains a mapping relationship between different video frames in the plurality of video frames.

[0054] The homographic transformation matrix described above can be used to represent the conversion relationship between images photographed at different positions in the same scene.

[0055] In an alternative embodiment, the homographic transformation matrix between the plurality of video frames can be calculated by a feature point matching method and an algorithm for estimating a parameter model (RANSAC), the corresponding relationship between the plurality of video frames can be determined by using the feature point matching method, and then the homographic transformation matrix between the plurality of video frames determined according to the corresponding relationship between the plurality of video frames can be calculated by using the algorithm for estimating a parameter model.

[0056] In another alternative embodiment, the plurality of video frames can be sampled to obtain key video frames, the mapping relationship between other video frames in the plurality of video frames can be derived based on the mapping relationship between adjacent key video frames, so as to obtain the mapping relationship between the plurality of video frames, and the homographic transformation matrix can be obtained by summarizing the mapping relationship between the plurality of video frames.

[0057] In step S308, the initial trajectory is transformed based on the homographic transformation matrix to obtain a target trajectory of the target object relative to the preset object.

[0058] The position of the preset object remains unchanged.

[0059] The preset object described above can be an object in a stationary state in a scene where the target object is located, for example, trees, stones, buildings, etc. in the scene, which is not limited here and can be selected according to actual conditions.

[0060] In an alternative embodiment, the initial trajectory can be transformed according to the homographic transformation matrix to eliminate the influence of the motion trajectory of the unmanned aerial vehicle on the motion trajectory of the target object, so as to obtain the real trajectory of the target object relative to the preset object, that is, the target trajectory described above, and restore the real motion trajectory of the target object.

[0061] By the above steps, the video shot by the unmanned aerial vehicle in the movement process is obtained, wherein a plurality of video frames in the video contain the target object; target tracking is performed on the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory includes: initial positions of the target object in the plurality of video frames; a homographic transformation matrix between the plurality of video frames is determined, wherein the homographic transformation matrix contains a mapping relationship between different video frames in the plurality of video frames; the initial trajectory is transformed based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein the position of the preset object remains unchanged, thereby achieving the purpose of restoring the real motion trajectory of the target object. It is easy to note that the motion trajectory of the unmanned aerial vehicle in the process of shooting the video can be determined by determining the homographic transformation matrix between the plurality of video frames, and the influence of the motion of the unmanned aerial vehicle on the motion trajectory of the target object can be eliminated by transforming the initial trajectory by the homographic transformation matrix, so that the real motion trajectory of the target object, i.e. the above-mentioned target trajectory, can be restored, thereby improving the accuracy of tracking the target object, and further solving the technical problem of low accuracy of tracking the target object in the related art.

[0062] In the above embodiments of the present application, the homographic transformation matrix between the plurality of video frames is determined, including: sampling the plurality of video frames to obtain key video frames; performing feature point matching on adjacent key video frames to obtain a mapping relationship between the adjacent key video frames; determining a mapping relationship between a first video frame and a second video frame in the plurality of video frames based on the mapping relationship between the adjacent key video frames, wherein at least one of the first video frame and the second video frame is not a key video frame; and summarizing the mapping relationship between the adjacent key video frames and the mapping relationship between the first video frame and the second video frame to determine the homographic transformation matrix.

[0063] The key video frames described above can be obtained by sampling the plurality of video frames according to a pre-set sampling rule, or can be obtained by sampling the plurality of video frames at a fixed sampling interval, for example, one key video frame can be sampled every three frames, or one key video frame can be sampled every four frames. Here, three frames or four frames are only examples, and the interval of sampling can be set according to actual conditions when the plurality of video frames are sampled to obtain key video frames, which is not limited here.

[0064] The key video frames described above can also be obtained by randomly sampling the plurality of video frames, which is not limited here.

[0065] The adjacent key video frames described above can be any two adjacent key video frames in the plurality of video frames.

[0066] In an alternative embodiment, the feature point matching between the adjacent key video frames can refer to finding corresponding key points between the two adjacent key video frames, obtaining the correspondence between the adjacent key video frames according to the corresponding key points, and then determining the mapping relationship between the multiple video frames according to the correspondence between the adjacent key video frames by using an algorithm for estimating a parameter model.

[0067] The first video frame can be a key video frame in the multiple video frames, and the first video frame can also be a video frame other than the key video frame in the multiple video frames. The second video frame can be a key video frame in the multiple video frames, and the second video frame can also be a video frame other than the key video frame in the multiple video frames.

[0068] When the first video frame is a key video frame, the second video frame can be a video frame other than the key video frame; when the first video frame is a video frame other than the key video frame, the second video frame can be a key video frame, and the second video frame can also be a video frame other than the key video frame.

[0069] In an alternative embodiment, the mapping relationship between the key video frames and the non-key video frames can be calculated inversely according to the mapping relationship between the adjacent key video frames.

[0070] In another alternative embodiment, the mapping relationship between the adjacent key video frames and the mapping relationship between the first video frame and the second video frame can be represented in the form of a matrix, and the matrix corresponding to the adjacent key video frames and the matrix corresponding to the first video frame and the second video frame can be summarized to determine a homography matrix.

[0071] The homography matrix can be M i,j wherein j≤i, representing the mapping matrix from the ith frame to the jth frame.

[0072] In the above embodiments of the present application, the mapping relationship between the first video frame and the second video frame in the plurality of video frames is determined based on the mapping relationship between the adjacent key video frames, including: in the case that the first video frame is not a key video frame and the second video frame is a key video frame adjacent to the first video frame, inversely transforming the mapping relationship between the first key video frame and the second video frame to obtain the mapping relationship between the first video frame and the second video frame, wherein the first key video frame and the second video frame are adjacent key video frames, and the first video frame is located between the second video frame and the first key video frame; in the case that the first video frame is not a key video frame and the second video frame is a key video frame not adjacent to the first video frame, transforming and transferring the mapping relationship between the second key video frame and the second video frame to obtain the mapping relationship between the first video frame and the second video frame, wherein the second key video frame is a key video frame adjacent to the first video frame; in the case that the second video frame is not a key video frame, linearly interpolating the mapping relationship between the third key video frame and the fourth key video frame to obtain the mapping relationship between the first video frame and the second video frame, wherein the second video frame is located between the third key video frame and the fourth key video frame, and the third key video frame and the fourth key video frame are adjacent key video frames.

[0073] The serial numbers of the plurality of video frames can be 0, 1, 2, 3, 4, 5, 6, 7, 8, and 9. For 10 video frames, a key video frame is sampled every 3 frames, which can be described by taking three sampling segments as an example. The serial numbers of the key video frames can be 0, 3, 6, and 9, and the non-key video frames can be 1, 2, 4, 5, 7, and 8. The three sampling segments are 0-3, 3-6, and 6-9, which can be divided by the key video frames as a boundary.

[0074] In an optional embodiment, the first video frame is not a key video frame, and the second video frame is a key video frame adjacent to the first video frame, which belongs to the case of deriving the mapping relationship between the key video frame and the non-key video frame in the sampling segment according to the mapping relationship between the key video frames, that is, the mapping relationship between the first video frame and the second video frame in the same sampling segment can be calculated by inversely transforming the mapping relationship between the two adjacent key video frames.

[0075] In another optional embodiment, the first video frame is not a key video frame, and the second video frame is a key video frame not adjacent to the first video frame, which belongs to the case of deriving the mapping relationship between the key video frame and the non-key video frame across segments according to the mapping relationship between the key video frames, that is, the mapping relationship between the first video frame and the second video frame belonging to different sampling segments can be obtained by transforming and transferring the mapping relationship between the second key video frame and the second video frame.

[0076] In another alternative embodiment, the second video frame is not a key video frame, and the first video frame can also not be a key video frame, belonging to the case of deriving the mapping relationship between the non-key video frames across the segments according to the mapping relationship between the key video frames, i.e., the mapping relationship between the first video frame and the second video frame belonging to different sampling segments can be obtained by linear interpolation of the mapping relationship between the third key video frame and the fourth key video frame.

[0077] In yet another alternative embodiment, the calculation amount can be reduced by the method of sampling the key video frames in the plurality of video frames, and the mapping relationship between the plurality of video frames can be converted and derived through the mapping relationship between the key video frames.

[0078] Figure 4 is a schematic diagram of a homographic transformation matrix according to an embodiment of the present application, as shown in Figure 4 M i,j represents the mapping relationship between the first video frame and the second video frame, M 1, , M 2, , M 4, , M 5, , M 7, , M 8, can be obtained by sampling the mapping relationship between the key video frames and the non-key video frames within the segment; M 3, , M 6, , M 9, can be the mapping relationship between the adjacent two key video frames; M 4, , M 5, , M 6, , M 7, , M 8, , M 9, can be the mapping relationship between the key video frames and the non-key video frames across the segments; M 2, , M 3, , M 3, , M 4, , M 4, , M 5, , M 5, , M 5, , M 6, , M 6, , M 7, , M 7, , M 8, , M 8, , M 9, , M 9,5 , M 8, , M 9, , M 9, .

[0079] In the above embodiments of the present application, the mapping relationship between the first key video frame and the second video frame is inversely transformed to obtain the mapping relationship between the first video frame and the second video frame, including: inversely transforming the mapping relationship between the first key video frame and the first video frame to obtain an inverse mapping relationship; obtaining the product of the inverse mapping relationship and the mapping relationship between the first key video frame and the second video frame to obtain the mapping relationship between the first video frame and the second video frame.

[0080] In an optional embodiment, the first video frame described above is not a key video frame, the second video frame can be the previous key video frame adjacent to the first video frame, and the first key video frame can be the next key video frame adjacent to the second video frame.

[0081] The sequence number of the first video frame described above can be i, the sequence number of the second video frame described above can be j, and the first key video frame described above can be k; inversely transforming the mapping relationship between the first key video frame and the second video frame to obtain the mapping relationship between the first video frame and the second video frame can be represented by the following formula:

[0082]

[0083] wherein M i,j represents a matrix of the mapping relationship between the first video frame and the second video frame, represents a matrix of the mapping relationship between the first key video frame and the second video frame, M k,j represents a matrix of the mapping relationship between the first key video frame and the second video frame.

[0084] For example, the sequence number of the first video frame described above can be 8, the sequence number of the second video frame described above can be 6, and the first key video frame described above can be 9; inversely transforming the mapping relationship between the first key video frame and the second video frame to obtain the mapping relationship between the first video frame and the second video frame can be represented by the following formula:

[0085]

[0086] wherein M 8,6 represents a matrix of the mapping relationship between the first video frame and the second video frame, represents an inverse matrix of the mapping relationship between the first key video frame and the second video frame, M 9,6 represents a matrix of the mapping relationship between the first key video frame and the second video frame.

[0087] In the above embodiments of the present application, the mapping relationship between the second key video frame and the second video frame is transformed and transferred to obtain the mapping relationship between the first video frame and the second video frame, including: obtaining the product of the mapping relationship between the first video frame and the second key video frame and the mapping relationship between the second key video frame and the second video frame to obtain the mapping relationship between the first video frame and the second video frame.

[0088] In an alternative embodiment, the first video frame described above is not a key video frame, the second video frame can be a key video frame separated from the first video frame by one or more key video frames, and the second key video frame can be the last key video frame adjacent to the first video frame.

[0089] The sequence number of the first video frame described above can be i, the sequence number of the second video frame described above can be j, and the second key video frame described above can be q; the mapping relationship between the second key video frame and the second video frame is transformed and transferred to obtain the mapping relationship between the first video frame and the second video frame, which can be represented by the following formula:

[0090] M i,j = M i,q M q,j ;

[0091] Wherein, M i,j represents the matrix of the mapping relationship between the first video frame and the second video frame, M i,q represents the matrix of the mapping relationship between the first video frame and the second key video frame, and M q,j represents the matrix of the mapping relationship between the second key video frame and the second video frame.

[0092] For example, the sequence number of the first video frame described above can be 8, the sequence number of the second video frame described above can be 3, and the second key video frame described above can be 6; the mapping relationship between the second key video frame and the second video frame is transformed and transferred to obtain the mapping relationship between the first video frame and the second video frame, which can be represented by the following formula:

[0093] M 8,3 = M 8, M 6,3 ;

[0094] Wherein, M 8,3 represents the matrix of the mapping relationship between the first video frame and the second video frame, M 8, represents the matrix of the mapping relationship between the first video frame and the second key video frame, and M 6,3 represents the matrix of the mapping relationship between the second key video frame and the second video frame.

[0095] In the above embodiments of the present application, the mapping relationship between the third key video frame and the fourth key video frame is linearly interpolated to obtain the mapping relationship between the first video frame and the second video frame, including: the mapping relationship between the first video frame and the third key video frame and the mapping relationship between the first video frame and the fourth key video frame are weighted and summed to obtain the mapping relationship between the first video frame and the second video frame.

[0096] In an optional embodiment, the second video frame described above is not a key video frame, the third key video frame and the fourth key video frame can be two adjacent key video frames, the first video frame described above can not be a non-key video frame located between the third key video frame and the fourth key video frame, and the first video frame described above can be a non-key video frame spaced apart from the second video frame by one key video frame.

[0097] The sequence number of the first video frame described above can be i, the sequence number of the second video frame described above can be j, the third key video frame described above can be m, and the fourth key video frame described above can be n. The mapping relationship between the third key video frame and the fourth key video frame is linearly interpolated to obtain the mapping relationship between the first video frame and the second video frame, which can be represented by the following formula:

[0098] M i,j = αM i,m + (1)M i,n ;

[0099]

[0100] Wherein, M i,j represents the matrix of the mapping relationship between the first video frame and the second video frame, M i,m represents the matrix of the mapping relationship between the first video frame and the third key video frame, and M i,n represents the matrix of the mapping relationship between the first video frame and the fourth key video frame.

[0101] For example, the sequence number of the first video frame described above can be 8, the sequence number of the second video frame described above can be 4, the third key video frame described above can be 3, and the fourth key video frame described above can be 6. The mapping relationship between the third key video frame and the fourth key video frame is linearly interpolated to obtain the mapping relationship between the first video frame and the second video frame, which can be represented by the following formula:

[0102]

[0103] Wherein, M 8,4 represents the matrix of the mapping relationship between the first video frame and the second video frame, M 8, represents the matrix of the mapping relationship between the first video frame and the third key video frame, and M8,6 a matrix representing a mapping relationship between the first video frame and the fourth key video frame.

[0104] In the above embodiments of the present application, the method further comprises: determining a first weight based on the third key video frame, the second video frame and the fourth key video frame, wherein the first weight corresponds to a mapping relationship between the first video frame and the third key video frame; obtaining a difference between a preset value and the first weight to obtain a second weight, wherein the second weight corresponds to a mapping relationship between the first video frame and the fourth key video frame.

[0105] In an optional embodiment, the first weight can be determined according to positions of the third key video frame, the second video frame and the fourth key video frame among the plurality of video frames, the first weight can be determined according to a ratio between a first difference value and a second difference value, the first difference value can be a difference between the fourth key video frame and the second video frame, and the second difference value can be a difference between the fourth key video frame and the third key video frame.

[0106] The preset value can be 1, but is not limited thereto, which is only used as an example for illustration.

[0107] In an optional embodiment, the second weight can be obtained according to a difference between 1 and the first weight, and the second weight can represent a position mapping relationship between the first video frame and the fourth key video frame among the plurality of video frames.

[0108] In the above embodiments of the present application, a number of video frames between the second video frame and the first video frame is less than a preset number.

[0109] The preset number can be set according to actual conditions, and the preset number can also be a number of video frames, which is only used as an example for illustration and is not limited in any way.

[0110] In the above embodiments of the present application, the initial trajectory is transformed based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein a position of the preset object remains unchanged, and the method comprises: projecting a target position in the initial trajectory to a first video frame in the plurality of video frames based on the homographic transformation matrix to obtain a restored position, wherein the target position is used to represent an initial position of the target object in other video frames, and the other video frames are used to represent video frames other than the first video frame in the plurality of video frames; and generating the target trajectory based on a first initial position in the initial trajectory and the restored position.

[0111] The preset object can be a stationary object in a real scene, and the stationary object can be used as a reference to determine the target trajectory of the target object.

[0112] The target position can be a position where a detection box for labeling a target object is located, and the target position can also be a position of the target object. Here, no specific limitation is made, and the actual situation can be set.

[0113] In an optional embodiment, the target position in the initial trajectory can be projected into a first video frame in the plurality of video frames according to the homographic transformation matrix to obtain a restored position. The motion of the target object can be determined by comparing the restored position and a first initial position in the initial trajectory, that is, the target trajectory is obtained.

[0114] The first initial position in the initial trajectory can be an initial position of the target object in a first video frame in the plurality of video frames.

[0115] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to select authorization or refusal.

[0116] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0117] From the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and necessary general hardware platform, and of course, it can also be realized by hardware. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the method of each embodiment of the present application.

[0118] Embodiment 2

[0119] According to the embodiments of the present application, a target tracking method is also provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown.

[0120] Figure 5 is a flowchart of a target tracking method according to Embodiment 2 of the present application, as shown in Figure 5 the method comprises the following steps:

[0121] Step S502, in response to an input instruction acting on the operation interface, displaying a video taken by the unmanned aerial vehicle during movement on the operation interface.

[0122] wherein the plurality of video frames in the video contain the target object;

[0123] Step S504, in response to a target tracking instruction acting on the operation interface, displaying a target trajectory of the target object relative to a preset object on the operation interface.

[0124] wherein the position of the preset object remains unchanged, the target trajectory is obtained by transforming an initial trajectory of the target object based on a homographic transformation matrix between the plurality of video frames, the initial trajectory is obtained by target tracking on the plurality of video frames, and the initial trajectory includes initial positions of the target object in the plurality of video frames.

[0125] Through the above steps, in response to an input instruction acting on the operation interface, a video taken by the unmanned aerial vehicle during movement is displayed on the operation interface, wherein the plurality of video frames in the video contain the target object; in response to a target tracking instruction acting on the operation interface, a target trajectory of the target object relative to a preset object is displayed on the operation interface, wherein the position of the preset object remains unchanged, the target trajectory is obtained by transforming an initial trajectory of the target object based on a homographic transformation matrix between the plurality of video frames, the initial trajectory is obtained by target tracking on the plurality of video frames, and the initial trajectory includes initial positions of the target object in the plurality of video frames, achieving the purpose of restoring the real movement trajectory of the target object. It is easy to note that the movement trajectory of the unmanned aerial vehicle during video shooting can be determined by determining the homographic transformation matrix between the plurality of video frames, and the real movement trajectory of the target object, i.e. the target trajectory described above, can be restored by transforming the initial trajectory based on the homographic transformation matrix, which can eliminate the influence of the movement of the unmanned aerial vehicle on the movement trajectory of the target object, thereby improving the accuracy of tracking the target object, and further solving the technical problem of low accuracy of tracking the target object in the related art.

[0126] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same scheme, application scenario and implementation process as provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0127] Embodiment 3

[0128] According to the embodiments of the present application, a target tracking method applied to a virtual reality (VR) device, an augmented reality (AR) device or the like in a virtual reality scene is further provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0129] Figure 6 is a flowchart of a target tracking method according to Embodiment 3 of the present application. As shown in Figure 6 , the method can include the following steps:

[0130] Step S602: displaying a video taken by a drone during movement on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device.

[0131] Among them, a plurality of video frames in the video contain a target object.

[0132] Step S604: performing target tracking on the plurality of video frames to obtain an initial trajectory of the target object.

[0133] Among them, the initial trajectory includes an initial position of the target object in the plurality of video frames.

[0134] Step S606: determining a homographic transformation matrix between the plurality of video frames.

[0135] Among them, the homographic transformation matrix contains a mapping relationship between different video frames in the plurality of video frames.

[0136] Step S608: transforming the initial trajectory based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object.

[0137] Among them, the position of the preset object remains unchanged.

[0138] Step S610: driving the VR device or the AR device to render and display the target trajectory.

[0139] Optionally, in the embodiment, the target tracking method described above can be applied to a hardware environment composed of a server and a virtual reality device. The target trajectory is displayed on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device. The server can be a server corresponding to a media file operator. The network includes, but is not limited to, a wide area network, a metropolitan area network, or a local area network. The virtual reality device is not limited to a virtual reality headset, virtual reality glasses, a virtual reality all-in-one machine, and the like.

[0140] Optionally, the virtual reality device includes a memory, a processor, and a transmission device. The memory is used to store an application program, which can be used to execute the following steps: displaying a video captured by a UAV during movement on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein a plurality of video frames in the video contain a target object.

[0141] Tracking the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory includes initial positions of the target object in the plurality of video frames; determining a homographic transformation matrix between the plurality of video frames, wherein the homographic transformation matrix contains a mapping relationship between different video frames in the plurality of video frames; transforming the initial trajectory based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein the position of the preset object remains unchanged; and driving the VR device or the AR device to render and display the target trajectory.

[0142] It should be noted that the target tracking method described above in the VR device or the AR device in the embodiment can include Figure 3 The method of the embodiment is used to achieve the purpose of driving the VR device or the AR device to display the target trajectory.

[0143] Optionally, the processor of the embodiment can call the application program stored in the memory through the transmission device to execute the above steps. The transmission device can receive a media file sent by a server through a network, and can also be used for data transmission between the processor and the memory.

[0144] Optionally, in the virtual reality device, a head-mounted display with eye tracking is provided. A screen in the HMD head-mounted display is used to display the displayed video screen. An eye tracking module in the HMD is used to obtain the real-time movement trajectory of the user's eyeballs. A tracking system is used to track the position information and movement information of the user in a real three-dimensional space. A calculation processing unit is used to obtain the real-time position and movement information of the user from the tracking system, and calculate the three-dimensional coordinates of the user's head in a virtual three-dimensional space, and the direction of the user's field of view in the virtual three-dimensional space.

[0145] In the embodiments of the present application, the virtual reality device can be connected with a terminal, the terminal is connected with a server through a network, the virtual reality device is not limited to a virtual reality helmet, virtual reality glasses, a virtual reality all-in-one machine, etc., the terminal is not limited to a PC, a mobile phone, a tablet computer, etc., and the server can be a server corresponding to a media file operator. The network includes but is not limited to a wide area network, a metropolitan area network or a local area network.

[0146] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same application scenarios and implementation processes as the scheme provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0147] Embodiment 4

[0148] According to the embodiments of the present application, a target tracking method is also provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0149] Figure 7 is a flowchart of a target tracking method according to Embodiment 4 of the present application, as shown in Figure 7 , the method comprises the following steps:

[0150] Step S702, acquiring a video photographed by the unmanned aerial vehicle in a movement process by calling a first interface.

[0151] The first interface includes a first parameter, and the parameter value of the first parameter is the video. The plurality of video frames in the video contain the target object.

[0152] The first interface described above can be an interface for data interaction between the server and the client. The client can input the video photographed by the unmanned aerial vehicle in the movement process into the interface function as the first parameter of the interface function, so as to achieve the purpose of uploading the video photographed by the unmanned aerial vehicle in the movement process to the cloud server.

[0153] Step S704, performing target tracking on the plurality of video frames to obtain an initial trajectory of the target object.

[0154] The initial trajectory includes an initial position of the target object in the plurality of video frames.

[0155] Step S706, determining a homographic transformation matrix between the plurality of video frames.

[0156] The homographic transformation matrix contains a mapping relationship between different video frames in the plurality of video frames.

[0157] Step S708: transforming the initial trajectory based on the homographic transformation matrix to obtain a target trajectory of the target object relative to the preset object.

[0158] wherein the position of the preset object remains unchanged.

[0159] Step S710: outputting the target trajectory by calling the second interface.

[0160] wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target trajectory.

[0161] The second interface in the above steps can be an interface for data exchange between the cloud server and the client. The cloud server can pass the target trajectory into an interface function as a second parameter of the interface function to achieve the purpose of issuing the target trajectory to the client.

[0162] Through the above steps, the video captured by the unmanned aerial vehicle during the movement process is obtained by calling the first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is the video, and the plurality of video frames in the video contain the target object; target tracking is performed on the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory includes: initial positions of the target object in the plurality of video frames; a homographic transformation matrix between the plurality of video frames is determined, wherein the homographic transformation matrix contains a mapping relationship between different video frames in the plurality of video frames; the initial trajectory is transformed based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein the position of the preset object remains unchanged; and the target trajectory is output by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target trajectory, thereby achieving the purpose of restoring the real motion trajectory of the target object. It is easy to note that the motion trajectory of the unmanned aerial vehicle during the video shooting process can be determined by determining the homographic transformation matrix between the plurality of video frames, and the real motion trajectory of the target object, i.e., the target trajectory described above, can be restored by transforming the initial trajectory based on the homographic transformation matrix, thereby improving the accuracy of tracking the target object, and further solving the technical problem of low accuracy of tracking the target object in related technologies.

[0163] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same application scenarios and implementation processes as the schemes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0164] Embodiment 5

[0165] According to the embodiments of the present application, a target tracking device for implementing the target tracking method described above is also provided, Figure 8 is a schematic diagram of a target tracking device according to Embodiment 5 of the present application, asFigure 8 As shown in the figure, the apparatus 800 includes an acquisition module 802, a tracking module 804, a determination module 806, and a transformation module 808.

[0166] The acquisition module is configured to acquire a video captured by the UAV during movement, wherein a plurality of video frames in the video contain a target object. The tracking module is configured to perform target tracking on the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory includes initial positions of the target object in the plurality of video frames. The determination module is configured to determine a homographic transformation matrix between the plurality of video frames, wherein the homographic transformation matrix contains mapping relationships between different video frames in the plurality of video frames. The transformation module is configured to transform the initial trajectory based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein a position of the preset object remains unchanged.

[0167] It should be noted that the acquisition module 802, the tracking module 804, the determination module 806, and the transformation module 808 correspond to steps S302 to S308 in Embodiment 1, and the four modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules or units can be hardware components or software components stored in a memory (for example, the memory 104) and processed by one or more processors (for example, the processors 102a, 102b, …, 102n), or the above modules can be run in the computer terminal 10 provided in Embodiment 1 as part of the apparatus.

[0168] In the above embodiments, the determination module is further configured to sample the plurality of video frames to obtain key video frames, perform feature point matching on adjacent key video frames to obtain mapping relationships between the adjacent key video frames, determine a mapping relationship between a first video frame and a second video frame in the plurality of video frames based on the mapping relationships between the adjacent key video frames, wherein at least one of the first video frame and the second video frame is not a key video frame, and aggregate the mapping relationships between the adjacent key video frames and the mapping relationship between the first video frame and the second video frame to determine the homographic transformation matrix.

[0169] In the above embodiments of the present application, the determining module is further configured to, in a case where the first video frame is not a key video frame and the second video frame is a key video frame adjacent to the first video frame, inversely transform a mapping relationship between the first key video frame and the second video frame to obtain a mapping relationship between the first video frame and the second video frame, wherein the first key video frame and the second video frame are adjacent key video frames, and the first video frame is located between the second video frame and the first key video frame; in a case where the first video frame is not a key video frame and the second video frame is a key video frame not adjacent to the first video frame, transform a mapping relationship between the second key video frame and the second video frame to obtain the mapping relationship between the first video frame and the second video frame, wherein the second key video frame is a key video frame adjacent to the first video frame; and in a case where the second video frame is not a key video frame, linearly interpolate a mapping relationship between the third key video frame and the fourth key video frame to obtain the mapping relationship between the first video frame and the second video frame, wherein the second video frame is located between the third key video frame and the fourth key video frame, and the third key video frame and the fourth key video frame are adjacent key video frames.

[0170] In the above embodiments of the present application, the determining module is further configured to inversely transform a mapping relationship between the first key video frame and the first video frame to obtain an inverse mapping relationship, and obtain a product of the inverse mapping relationship and a mapping relationship between the first key video frame and the second video frame to obtain a mapping relationship between the first video frame and the second video frame.

[0171] In the above embodiments of the present application, the determining module is further configured to obtain a product of a mapping relationship between the first video frame and the second key video frame and a mapping relationship between the second key video frame and the second video frame to obtain a mapping relationship between the first video frame and the second video frame.

[0172] In the above embodiments of the present application, the determining module is further configured to perform weighted sum on a mapping relationship between the first video frame and the third key video frame and a mapping relationship between the first video frame and the fourth key video frame to obtain a mapping relationship between the first video frame and the second video frame.

[0173] In the above embodiments of the present application, the apparatus further includes that the determining module is further configured to determine a first weight based on the third key video frame, the second video frame and the fourth key video frame, wherein the first weight corresponds to a mapping relationship between the first video frame and the third key video frame; and the obtaining module is configured to obtain a difference between a preset value and the first weight to obtain a second weight, wherein the second weight corresponds to a mapping relationship between the first video frame and the fourth key video frame.

[0174] In the above embodiments of the present application, a number of video frames spaced between the second video frame and the first video frame is less than a preset number.

[0175] In the above embodiments of the present application, the transformation module is further configured to project a target position in the initial trajectory to a first video frame in the plurality of video frames based on the homographic transformation matrix to obtain a restored position, wherein the target position is used to represent an initial position of the target object in other video frames, and the other video frames are used to represent video frames other than the first video frame in the plurality of video frames; and generate the target trajectory based on a first initial position in the initial trajectory and the restored position.

[0176] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same scheme, application scenario and implementation process as provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0177] Embodiment 6

[0178] According to the embodiments of the present application, a target tracking device for implementing the above target tracking method is further provided, Figure 9 is a schematic diagram of a target tracking device according to Embodiment 6 of the present application, as Figure 9 shown, the device 900 includes a first display module 902, a second display module 904.

[0179] The first display module is configured to display a video captured by the unmanned aerial vehicle during the movement process on the operation interface in response to an input instruction acting on the operation interface, wherein a plurality of video frames in the video contain a target object; and the second display module is configured to display a target trajectory of the target object relative to a preset object on the operation interface in response to a target tracking instruction acting on the operation interface, wherein the position of the preset object remains unchanged, the target trajectory is obtained by transforming an initial trajectory of the target object based on a homographic transformation matrix between the plurality of video frames, the initial trajectory is obtained by target tracking on the plurality of video frames, and the initial trajectory includes initial positions of the target object in the plurality of video frames.

[0180] It should be noted that the first display module 902 and the second display module 904 correspond to steps S502 to S504 in Embodiment 2, and the two modules have the same instances and application scenarios as the corresponding steps, but are not limited to the contents disclosed in the above Embodiment 1. It should be noted that the above modules or units can be hardware components or software components stored in a memory (for example, the memory 104) and processed by one or more processors (for example, the processors 102a, 102b, …, 102n), and the above modules can also be run in the computer terminal 10 provided in Embodiment 1 as a part of the device.

[0181] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same scheme, application scenario and implementation process as provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0182] Example 7

[0183] According to an embodiment of this application, a target tracking apparatus for implementing the above-described target tracking method is also provided. Figure 10 This is a schematic diagram of a target tracking device according to Embodiment 7 of this application, as shown below. Figure 10 As shown, the device 1000 includes: a display module 1002, a tracking module 1004, a determination module 1006, a transformation module 1008, and a driving module 1010.

[0184] The system comprises the following modules: a display module for displaying video captured by a drone during its movement on the screen of a virtual reality (VR) or augmented reality (AR) device, where multiple video frames contain the target object; a tracking module for tracking the target across multiple video frames to obtain the initial trajectory of the target object, which includes the initial position of the target object within the multiple video frames; a determination module for determining the homography transformation matrix between multiple video frames, which contains the mapping relationship between different video frames; a transformation module for transforming the initial trajectory based on the homography transformation matrix to obtain the target trajectory of the target object relative to a preset object, where the position of the preset object remains unchanged; and a driving module for driving the VR or AR device to render and display the target trajectory.

[0185] It should be noted that the above-mentioned display module 1002, tracking module 1004, determination module 1006, transformation module 1008, and driving module 1010 correspond to steps S602 to S610 in Embodiment 3. The four modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above-mentioned modules or units can be hardware or software components stored in memory and processed by one or more processors. The above-mentioned modules can also be part of the device and run in the AR / VR device provided in Embodiment 1.

[0186] It should be noted that the preferred implementation schemes involved in the above embodiments of this application are the same as the schemes, application scenarios and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0187] Example 8

[0188] According to an embodiment of this application, a target tracking apparatus for implementing the above-described target tracking method is also provided. Figure 11 This is a schematic diagram of a target tracking device according to Embodiment 8 of this application, as shown below. Figure 11As shown, the apparatus 1100 includes an obtaining module 1102, a tracking module 1104, a determining module 1106, a transforming module 1108, and an outputting module 1110.

[0189] The obtaining module is configured to obtain a video captured by a UAV during movement by calling a first interface, where the first interface includes a first parameter, and a parameter value of the first parameter is the video, and a plurality of video frames in the video contain a target object.

[0190] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same application scenarios and implementation processes as the scheme provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0191] Embodiment 9

[0192] The embodiments of the present application can provide an electronic device, which can be an AR / VR device, and the AR / VR device can be any one of the AR / VR devices in a group of AR / VR devices. Alternatively, in the present embodiment, the AR / VR device can be replaced by a terminal device such as a mobile terminal.

[0193] Alternatively, in the present embodiment, the AR / VR device can be located in at least one of a plurality of network devices in a computer network.

[0194] In the present embodiment, the AR / VR device can execute program codes for the following steps in the target tracking method: obtaining a video captured by a UAV during movement, where a plurality of video frames in the video contain a target object; performing target tracking on the plurality of video frames to obtain an initial trajectory of the target object, where the initial trajectory includes initial positions of the target object in the plurality of video frames; determining a homographic transformation matrix between the plurality of video frames, where the homographic transformation matrix contains mapping relationships between different video frames in the plurality of video frames; and transforming the initial trajectory based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, where a position of the preset object remains unchanged.

[0195] Alternatively, Figure 12is a structural block diagram of a computer terminal according to an embodiment of the present application. As shown in Figure 12 The computer terminal A can include one or more (only one is shown in the figure) processors 102, a memory 104, a storage controller, and a peripheral interface, wherein the peripheral interface is connected with a radio frequency module, an audio module, and a display.

[0196] The memory can be used to store software programs and modules, such as program instructions / modules corresponding to the target tracking method and device in the embodiments of the present application. The processor executes various functions and data processing by running the software programs and modules stored in the memory, that is, implements the target tracking method described above. The memory can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged with respect to the processor, which can be connected to the terminal A through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0197] The processor can call information and application programs stored in the memory through a transmission device to perform the following steps: acquiring a video photographed by a UAV during movement, wherein a plurality of video frames in the video contain a target object; performing target tracking on the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory includes initial positions of the target object in the plurality of video frames; determining a homographic transformation matrix between the plurality of video frames, wherein the homographic transformation matrix contains a mapping relationship between different video frames in the plurality of video frames; and transforming the initial trajectory based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein the position of the preset object remains unchanged.

[0198] Optionally, the processor can further execute program codes of the following steps: sampling the plurality of video frames to obtain key video frames; performing feature point matching on adjacent key video frames to obtain a mapping relationship between the adjacent key video frames; determining a mapping relationship between a first video frame and a second video frame in the plurality of video frames based on the mapping relationship between the adjacent key video frames, wherein at least one of the first video frame and the second video frame is not a key video frame; and summarizing the mapping relationship between the adjacent key video frames and the mapping relationship between the first video frame and the second video frame to determine the homographic transformation matrix.

[0199] Optionally, the processor can further execute program codes of the following steps: in the case that the first video frame is not a key video frame and the second video frame is a key video frame adjacent to the first video frame, inversely transforming the mapping relationship between the first key video frame and the second video frame to obtain the mapping relationship between the first video frame and the second video frame, wherein the first key video frame and the second video frame are adjacent key video frames and the first video frame is located between the second video frame and the first key video frame; in the case that the first video frame is not a key video frame and the second video frame is a key video frame not adjacent to the first video frame, transforming the mapping relationship between the second key video frame and the second video frame to obtain the mapping relationship between the first video frame and the second video frame, wherein the second key video frame is a key video frame adjacent to the first video frame; in the case that the second video frame is not a key video frame, linearly interpolating the mapping relationship between the third key video frame and the fourth key video frame to obtain the mapping relationship between the first video frame and the second video frame, wherein the second video frame is located between the third key video frame and the fourth key video frame and the third key video frame and the fourth key video frame are adjacent key video frames.

[0200] Optionally, the processor can further execute program codes of the following steps: inversely transforming the mapping relationship between the first key video frame and the first video frame to obtain an inverse mapping relationship; obtaining the product of the inverse mapping relationship and the mapping relationship between the first key video frame and the second video frame to obtain the mapping relationship between the first video frame and the second video frame.

[0201] Optionally, the processor can further execute program codes of the following steps: obtaining the product of the mapping relationship between the first video frame and the second key video frame and the mapping relationship between the second key video frame and the second video frame to obtain the mapping relationship between the first video frame and the second video frame.

[0202] Optionally, the processor can further execute program codes of the following steps: weighting and summing the mapping relationship between the first video frame and the third key video frame and the mapping relationship between the first video frame and the fourth key video frame to obtain the mapping relationship between the first video frame and the second video frame.

[0203] Optionally, the processor can further execute program codes of the following steps: determining a first weight based on the third key video frame, the second video frame and the fourth key video frame, wherein the first weight corresponds to the mapping relationship between the first video frame and the third key video frame; obtaining the difference between a preset value and the first weight to obtain a second weight, wherein the second weight corresponds to the mapping relationship between the first video frame and the fourth key video frame.

[0204] Optionally, the processor can further execute program codes of the following steps: the number of video frames between the second video frame and the first video frame is less than the preset number.

[0205] Optionally, the processor can further execute program codes of the following steps: based on the homographic transformation matrix, projecting a target position in the initial trajectory to a first video frame in the plurality of video frames to obtain a restored position, wherein the target position is used to represent an initial position of the target object in other video frames, and the other video frames are used to represent video frames other than the first video frame in the plurality of video frames; and generating the target trajectory based on the first initial position in the initial trajectory and the restored position.

[0206] Optionally, the storage medium can further store program codes for executing the following steps: based on the homographic transformation matrix, projecting a target position in the initial trajectory to a first video frame in the plurality of video frames to obtain a restored position, wherein the target position is used to represent an initial position of the target object in other video frames, and the other video frames are used to represent video frames other than the first video frame in the plurality of video frames; and generating the target trajectory based on the first initial position in the initial trajectory and the restored position.

[0207] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: in response to an input instruction acting on the operation interface, displaying a video taken by the unmanned aerial vehicle in a motion process on the operation interface, wherein a plurality of video frames in the video contain a target object; and in response to a target tracking instruction acting on the operation interface, displaying a target trajectory of the target object relative to a preset object on the operation interface, wherein the position of the preset object remains unchanged, the target trajectory is obtained by transforming an initial trajectory of the target object based on a homographic transformation matrix between the plurality of video frames, the initial trajectory is obtained by tracking the target object in the plurality of video frames, and the initial trajectory includes initial positions of the target object in the plurality of video frames.

[0208] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: displaying a video taken by the unmanned aerial vehicle in a motion process on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein a plurality of video frames in the video contain a target object; tracking the target object in the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory includes initial positions of the target object in the plurality of video frames; determining a homographic transformation matrix between the plurality of video frames, wherein the homographic transformation matrix contains mapping relationships between different video frames in the plurality of video frames; transforming the initial trajectory based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein the position of the preset object remains unchanged; and driving the VR device or the AR device to render and display the target trajectory.

[0209] The processor can call information and application programs stored in the memory through the transmission device to perform the following steps: obtaining a video taken by the unmanned aerial vehicle during movement through calling a first interface, wherein the first interface comprises a first parameter, a parameter value of the first parameter is the video, and a plurality of video frames in the video contain a target object; performing target tracking on the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory comprises: initial positions of the target object in the plurality of video frames; determining a homographic transformation matrix between the plurality of video frames, wherein the homographic transformation matrix contains a mapping relationship between different video frames in the plurality of video frames; transforming the initial trajectory based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein a position of the preset object remains unchanged; and outputting the target trajectory through calling a second interface, wherein the second interface comprises a second parameter, and a parameter value of the second parameter is the target trajectory.

[0210] Through the above steps, the video taken by the unmanned aerial vehicle during movement is obtained, wherein a plurality of video frames in the video contain a target object; target tracking is performed on the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory comprises: initial positions of the target object in the plurality of video frames; a homographic transformation matrix between the plurality of video frames is determined, wherein the homographic transformation matrix contains a mapping relationship between different video frames in the plurality of video frames; the initial trajectory is transformed based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein a position of the preset object remains unchanged, thereby achieving the purpose of restoring the real movement trajectory of the target object. It is easy to note that the movement trajectory of the unmanned aerial vehicle during video shooting can be determined by determining the homographic transformation matrix between the plurality of video frames, and the real movement trajectory of the target object, i.e., the above-mentioned target trajectory, can be restored by transforming the initial trajectory based on the homographic transformation matrix, thereby improving the accuracy of tracking the target object, and further solving the technical problem of low accuracy of tracking the target object in the related art.

[0211] Those skilled in the art can understand that Figure 12 The structure shown is only schematic, and the computer terminal can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, or other terminal devices. Figure 12 The structure of the above-mentioned electronic device is not limited. For example, the computer terminal A can further include more or fewer components (such as a network interface, a display device, etc.) than Figure 12 shown, or have a different configuration than Figure 12 shown.

[0212] Those skilled in the art can understand that all or part of the steps in the above-mentioned various methods of the embodiments can be completed by instructing the terminal device related hardware through a program, and the program can be stored in a computer readable storage medium, which can include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0213] Embodiment 10

[0214] The embodiments of the present application also provide a computer readable storage medium. Optionally, in the embodiment, the computer readable storage medium can be used to save the program code executed by the target tracking method provided in the embodiment 1.

[0215] Optionally, in the embodiment, the computer readable storage medium can be located in any one of the computer terminals in the AR / VR device terminal group in the AR / VR device network, or in any one of the mobile terminals in the mobile terminal group.

[0216] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for performing the following steps: obtaining a video taken by the unmanned aerial vehicle during movement, wherein a plurality of video frames in the video contain a target object; performing target tracking on the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory includes initial positions of the target object in the plurality of video frames; determining a homographic transformation matrix between the plurality of video frames, wherein the homographic transformation matrix contains mapping relationships between different video frames in the plurality of video frames; and transforming the initial trajectory based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein the position of the preset object remains unchanged.

[0217] Optionally, the storage medium is further configured to store program code for performing the following steps: sampling the plurality of video frames to obtain key video frames; performing feature point matching on adjacent key video frames to obtain mapping relationships between the adjacent key video frames; determining a mapping relationship between a first video frame and a second video frame in the plurality of video frames based on the mapping relationships between the adjacent key video frames, wherein at least one of the first video frame and the second video frame is not a key video frame; and summarizing the mapping relationships between the adjacent key video frames and the mapping relationship between the first video frame and the second video frame to determine the homographic transformation matrix.

[0218] Optionally, the storage medium is further configured to store program code for performing the following steps: in a case that the first video frame is not a key video frame and the second video frame is a key video frame adjacent to the first video frame, performing inverse transformation on a mapping relationship between the first key video frame and the second video frame to obtain a mapping relationship between the first video frame and the second video frame, wherein the first key video frame and the second video frame are adjacent key video frames, and the first video frame is located between the second video frame and the first key video frame; in a case that the first video frame is not a key video frame and the second video frame is a key video frame not adjacent to the first video frame, performing transformation on a mapping relationship between the second key video frame and the second video frame to obtain a mapping relationship between the first video frame and the second video frame, wherein the second key video frame is a key video frame adjacent to the first video frame; and in a case that the second video frame is not a key video frame, performing linear interpolation on a mapping relationship between the third key video frame and the fourth key video frame to obtain a mapping relationship between the first video frame and the second video frame, wherein the second video frame is located between the third key video frame and the fourth key video frame, and the third key video frame and the fourth key video frame are adjacent key video frames.

[0219] Optionally, the storage medium is further configured to store program code for performing the following steps: performing inverse transformation on a mapping relationship between the first key video frame and the first video frame to obtain an inverse mapping relationship; and obtaining a product of the inverse mapping relationship and a mapping relationship between the first key video frame and the second video frame to obtain a mapping relationship between the first video frame and the second video frame.

[0220] Optionally, the storage medium is further configured to store program code for performing the following steps: obtaining a product of a mapping relationship between the first video frame and the second key video frame and a mapping relationship between the second key video frame and the second video frame to obtain a mapping relationship between the first video frame and the second video frame.

[0221] Optionally, the storage medium is further configured to store program code for performing the following steps: performing weighted sum on a mapping relationship between the first video frame and the third key video frame and a mapping relationship between the first video frame and the fourth key video frame to obtain a mapping relationship between the first video frame and the second video frame.

[0222] Optionally, the storage medium is further configured to store program code for performing the following steps: determining a first weight based on the third key video frame, the second video frame and the fourth key video frame, wherein the first weight corresponds to a mapping relationship between the first video frame and the third key video frame; and obtaining a difference between a preset value and the first weight to obtain a second weight, wherein the second weight corresponds to a mapping relationship between the first video frame and the fourth key video frame.

[0223] Optionally, the storage medium is further configured to store program code for performing the following step: the number of video frames spaced between the second video frame and the first video frame is less than a preset number.

[0224] Optionally, the storage medium is further configured to store program code for performing the following step: projecting a target position in the initial trajectory to a first video frame in the plurality of video frames based on the homographic transformation matrix to obtain a restored position, wherein the target position is used to represent an initial position of the target object in other video frames, and the other video frames are used to represent video frames other than the first video frame in the plurality of video frames; and generating the target trajectory based on the first initial position in the initial trajectory and the restored position.

[0225] Optionally, in this embodiment, the computer readable storage medium is configured to store program code for performing the following steps: in response to an input instruction acting on the operation interface, displaying a video taken by the unmanned aerial vehicle during movement on the operation interface, wherein a plurality of video frames in the video contain a target object; and in response to a target tracking instruction acting on the operation interface, displaying a target trajectory of the target object relative to a preset object on the operation interface, wherein the position of the preset object remains unchanged, the target trajectory is obtained by transforming an initial trajectory of the target object based on a homographic transformation matrix between the plurality of video frames, the initial trajectory is obtained by performing target tracking on the plurality of video frames, and the initial trajectory includes initial positions of the target object in the plurality of video frames.

[0226] Optionally, in this embodiment, the computer readable storage medium is configured to store program code for performing the following steps: displaying a video taken by the unmanned aerial vehicle during movement on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein a plurality of video frames in the video contain a target object; performing target tracking on the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory includes initial positions of the target object in the plurality of video frames; determining a homographic transformation matrix between the plurality of video frames, wherein the homographic transformation matrix contains mapping relationships between different video frames in the plurality of video frames; transforming the initial trajectory based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein the position of the preset object remains unchanged; and driving the VR device or the AR device to render and display the target trajectory.

[0227] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for performing the following steps: obtaining a video captured by the UAV during movement by calling a first interface, wherein the first interface comprises a first parameter, a parameter value of the first parameter is the video, and a plurality of video frames in the video contain the target object; performing target tracking on the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory comprises initial positions of the target object in the plurality of video frames; determining a homographic transformation matrix between the plurality of video frames, wherein the homographic transformation matrix contains mapping relationships between different video frames in the plurality of video frames; transforming the initial trajectory based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein a position of the preset object remains unchanged; and outputting the target trajectory by calling a second interface, wherein the second interface comprises a second parameter, and a parameter value of the second parameter is the target trajectory.

[0228] By the above steps, the video captured by the UAV during movement is obtained, wherein a plurality of video frames in the video contain the target object; target tracking is performed on the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory comprises initial positions of the target object in the plurality of video frames; a homographic transformation matrix between the plurality of video frames is determined, wherein the homographic transformation matrix contains mapping relationships between different video frames in the plurality of video frames; the initial trajectory is transformed based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein a position of the preset object remains unchanged, thereby achieving the purpose of restoring the real movement trajectory of the target object. It is easy to note that the movement trajectory of the UAV during video shooting can be determined by determining the homographic transformation matrix between the plurality of video frames, and the initial trajectory is transformed by the homographic transformation matrix, which can eliminate the influence of the movement of the UAV on the movement trajectory of the target object, and the real movement trajectory of the target object, i.e., the target trajectory described above, can be restored, thereby improving the accuracy of tracking the target object, and further solving the technical problem of low accuracy of tracking the target object in the related art.

[0229] The above sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0230] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0231] In several embodiments provided in the present application, it should be understood that the disclosed technology can be implemented by other ways. Among them, the above-described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, units or modules, and can be electrical or other forms.

[0232] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0233] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0234] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the technical solutions or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0235] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A target tracking method characterized by, The method comprises: acquiring a video captured by a UAV during movement, wherein a plurality of video frames in the video contain a target object; performing target tracking on the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory comprises initial positions of the target object in the plurality of video frames; determining a homographic transformation matrix between the plurality of video frames, wherein the homographic transformation matrix contains mapping relationships between different video frames in the plurality of video frames, the homographic transformation matrix is obtained by summarizing mapping relationships between adjacent key video frames and between a first video frame and a second video frame in the plurality of video frames, and the mapping relationship between the first video frame and the second video frame is obtained by performing corresponding operations on the mapping relationships between the adjacent key video frames; transforming the initial trajectory based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein a position of the preset object remains unchanged.

2. The method of claim 1, wherein, The method for determining the homographic transformation matrix between the plurality of video frames comprises: sampling the plurality of video frames to obtain key video frames; performing feature point matching on adjacent key video frames to obtain mapping relationships between the adjacent key video frames; performing corresponding operations based on the mapping relationships between the adjacent key video frames to determine a mapping relationship between a first video frame and a second video frame in the plurality of video frames, wherein at least one of the first video frame and the second video frame is not the key video frame, and the corresponding operations comprise one of inverse transformation, transformation transfer, and linear interpolation; summarizing the mapping relationships between the adjacent key video frames and between the first video frame and the second video frame to determine the homographic transformation matrix.

3. The method of claim 2, wherein, The method for determining the mapping relationship between the first video frame and the second video frame in the plurality of video frames based on the mapping relationships between the adjacent key video frames comprises: in a case where the first video frame is not the key video frame and the second video frame is the key video frame adjacent to the first video frame, performing inverse transformation on a mapping relationship between a first key video frame and the second video frame to obtain the mapping relationship between the first video frame and the second video frame, wherein the first key video frame and the second video frame are the adjacent key video frames, and the first video frame is located between the second video frame and the first key video frame; in a case where the first video frame is not the key video frame and the second video frame is the key video frame that is not adjacent to the first video frame, performing transformation transfer on a mapping relationship between a second key video frame and the second video frame to obtain the mapping relationship between the first video frame and the second video frame, wherein the second key video frame is the key video frame adjacent to the first video frame. In a case where the second video frame is not the key video frame, a mapping relationship between a third key video frame and a fourth key video frame is linearly interpolated to obtain a mapping relationship between the first video frame and the second video frame, wherein the second video frame is located between the third key video frame and the fourth key video frame, and the third key video frame and the fourth key video frame are the adjacent key video frames.

4. The method of claim 3, wherein, The inverse transformation of the mapping relationship between the first key video frame and the second video frame comprises: inverse transformation of a mapping relationship between the first key video frame and the first video frame to obtain an inverse mapping relationship; multiplication of the inverse mapping relationship and the mapping relationship between the first key video frame and the second video frame to obtain the mapping relationship between the first video frame and the second video frame.

5. The method of claim 3, wherein, The transformation of the mapping relationship between the second key video frame and the second video frame comprises: multiplication of the mapping relationship between the first video frame and the second key video frame and the mapping relationship between the second key video frame and the second video frame to obtain the mapping relationship between the first video frame and the second video frame.

6. The method of claim 3, wherein, The linear interpolation of the mapping relationship between the third key video frame and the fourth key video frame comprises: weighted sum of the mapping relationship between the first video frame and the third key video frame and the mapping relationship between the first video frame and the fourth key video frame to obtain the mapping relationship between the first video frame and the second video frame, wherein the weighted sum is used for weighted sum of a first weight and a second weight, the first weight corresponding to the mapping relationship between the first video frame and the third key video frame, and the second weight corresponding to the mapping relationship between the first video frame and the fourth key video frame.

7. The method of claim 6, wherein, The method further comprises: determination of a first weight based on the third key video frame, the second video frame and the fourth key video frame, wherein the first weight corresponds to the mapping relationship between the first video frame and the third key video frame, and the first weight is determined based on positions of the third key video frame, the second video frame and the fourth key video frame among the plurality of video frames; obtaining a difference between a preset value and the first weight to obtain a second weight, wherein the second weight corresponds to the mapping relationship between the first video frame and the fourth key video frame.

8. The method of claim 3, wherein, The number of video frames spaced between the second video frame and the first video frame is less than a preset number.

9. The method of claim 1, wherein, The transformation of the initial trajectory based on the homography matrix to obtain the target trajectory of the target object relative to a preset object, wherein the position of the preset object remains unchanged, comprises: projecting a target position in the initial trajectory to a first video frame in the plurality of video frames based on the homographic transformation matrix, wherein the target position is used to represent an initial position of the target object in other video frames, and the other video frames are used to represent video frames other than the first video frame in the plurality of video frames; generating the target trajectory based on a first initial position in the initial trajectory and the restored position.

10. A target tracking method characterized by, The method comprises the following steps: in response to an input instruction acting on the operation interface, displaying a video captured by a UAV during movement on the operation interface, wherein a plurality of video frames in the video contain a target object; in response to a target tracking instruction acting on the operation interface, displaying a target trajectory of the target object relative to a preset object on the operation interface, wherein the position of the preset object remains unchanged, the target trajectory is obtained by transforming an initial trajectory of the target object based on a homographic transformation matrix between the plurality of video frames, the initial trajectory is obtained by target tracking on the plurality of video frames, and the initial trajectory comprises initial positions of the target object in the plurality of video frames, the homographic transformation matrix is obtained by summarizing mapping relationships between adjacent key video frames and between a first video frame and a second video frame in the plurality of video frames, and the mapping relationship between the first video frame and the second video frame is obtained by performing corresponding operations on the mapping relationships between the adjacent key video frames.

11. A target tracking method characterized by, The method comprises the following steps: displaying a video captured by a UAV during movement on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein a plurality of video frames in the video contain a target object; performing target tracking on the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory comprises initial positions of the target object in the plurality of video frames; determining a homographic transformation matrix between the plurality of video frames, wherein the homographic transformation matrix contains mapping relationships between different video frames in the plurality of video frames, the homographic transformation matrix is obtained by summarizing mapping relationships between adjacent key video frames and between a first video frame and a second video frame in the plurality of video frames, and the mapping relationship between the first video frame and the second video frame is obtained by performing corresponding operations on the mapping relationships between the adjacent key video frames; transforming the initial trajectory based on the homographic transformation matrix to obtain a target trajectory of the target object relative to a preset object, wherein the position of the preset object remains unchanged; and driving the VR device or the AR device to render and display the target trajectory.

12. A target tracking method characterized by, The method comprises the following steps: obtaining a video captured by a UAV during movement by calling a first interface, wherein the first interface comprises a first parameter, a parameter value of the first parameter is the video, and a plurality of video frames in the video contain a target object; performing target tracking on the plurality of video frames to obtain an initial trajectory of the target object, wherein the initial trajectory comprises initial positions of the target object in the plurality of video frames; determine a homography matrix between the plurality of video frames, wherein the homography matrix comprises mapping relationships between different video frames in the plurality of video frames, and the homography matrix is obtained by summarizing mapping relationships between adjacent key video frames and between a first video frame and a second video frame in the plurality of video frames, and the mapping relationship between the first video frame and the second video frame is obtained by performing corresponding operations on the mapping relationships between the adjacent key video frames; transform the initial trajectory based on the homography matrix to obtain a target trajectory of the target object relative to a preset object, wherein a position of the preset object remains unchanged; output the target trajectory by invoking a second interface, wherein the second interface comprises a second parameter, and a parameter value of the second parameter is the target trajectory.

13. An electronic device, comprising: comprise: a memory storing an executable program; a processor configured to execute the program, wherein the program performs the method of any one of claims 1 to 12 when executed.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium comprises a stored executable program, wherein the computer-readable storage medium controls a device in which the computer-readable storage medium is located to perform the method of any one of claims 1 to 12 when the executable program is executed.

Images