A method and device for real-time labeling of aerial video
By using dynamic computational drone fence and view frustum filtering technology, the point library retrieval and rendering of the drone video surveillance system are optimized, solving the problems of low efficiency and resource waste during drone dynamic flight and achieving efficient real-time annotation effect.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NGH INSIGHTS CO LTD
- Filing Date
- 2026-02-09
- Publication Date
- 2026-06-05
AI Technical Summary
When drones are in dynamic flight, existing technologies suffer from inefficiency, resource waste, and a lack of adaptability, leading to problems such as delays, overlaps, or missing information when drones are deployed.
By dynamically calculating the fence around the drone's perimeter, and combining the view frustum equation and layered rendering technology, the retrieval and rendering process of the point library is optimized. Dynamic spatial grid indexing and view frustum filtering are used to reduce computational load and latency, and to achieve preloading of the dedicated point library.
It improved server performance by more than 80%, reduced server response latency to the millisecond level, avoided tag overlap, and improved screen smoothness and ease of operation.
Smart Images
Figure CN121661551B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of video surveillance technology, and more particularly to a method and apparatus for real-time annotation of aerial video. Background Technology
[0002] In recent years, with the rapid development of drone technology, it has been widely used in urban governance, emergency response, and geographic mapping. Drone video surveillance systems often need to dynamically load point information (such as geographic POI locations, building tags, or resource markers) onto real-time video footage to assist operators in decision-making. However, with the accumulation of massive point databases (such as millions of geographic POI data), drones face severe challenges during dynamic flight: drones need to update point information more than 20 times per second (due to changes in the drone's latitude and longitude, horizontal or vertical viewing angle, altitude, etc., the resource points on the screen need to change with the viewing angle, and the flight status changes rapidly). Directly searching in the massive database leads to low efficiency (response latency of up to hundreds of milliseconds), stuttering of point-spreading effects, and severe image overlap, affecting real-time performance. Existing solutions (such as global scanning or simple caching) cannot efficiently handle dynamically changing field of view and point density, resulting in performance bottlenecks and visual confusion; partitioning the database and tables by zoning ranges can also cause problems for drones at zoning boundaries, as the drone's field of view may span across regions. Drone scenarios are also more dynamic (frequent changes in location and altitude).
[0003] The existing technology currently has the following technical problems:
[0004] Inefficiency: When drones are flying dynamically, the server needs to retrieve data from a massive database of locations multiple times per second. Global scanning results in high latency, causing delays in data distribution and screen stuttering.
[0005] Waste of resources: Most points may be located outside the field of view, but traditional methods still perform coordinate transformation and rendering, consuming computing resources.
[0006] Lack of adaptability: The fixed indexing method cannot adapt to the rapid movement of drones and changes in resolution, resulting in overlapping or missing points.
[0007] There is currently no effective solution to the aforementioned problems in the relevant technologies. Summary of the Invention
[0008] The main objective of this application is to provide a method and apparatus for real-time annotation of aerial videos, so as to at least solve the problems of low efficiency, resource waste, and inaccurate annotation of drones in real-time loading of annotation information in related technologies.
[0009] To achieve the above objectives, according to one aspect of this application, a method for real-time annotation of aerial video is provided. The method includes: after the drone takes off, acquiring a drone fence based on the drone's position and storing the drone fence in a fence database, wherein the drone fence is a circular area centered on the drone's latitude and longitude points, and the radius of the circular area is a real number greater than 0; acquiring all points in the drone's camera coordinate system, retaining points that satisfy the frustum equation, and discarding points that do not satisfy the frustum equation; and aggregating the points based on the drone's altitude.
[0010] Optionally, the location of the drone is checked at preset intervals to see if it is within the latest fence, where the latest fence is the latest drone fence in the fence library.
[0011] Optionally, when the drone is within the newly created fence, the shortest distance from the drone to the edge of the newly created fence is calculated using the following formula: ,in, For the shortest distance, To check the drone's position coordinates, The coordinates of the point closest to the drone at the edge of the newly constructed fence. Let be the spherical distance between two points; the formula for calculating the spherical distance between two points is: , The distance is spherical. For the Earth's radius, , ,in, for and The difference in latitude between them for and The difference in longitude between them for latitude, for latitude, for longitude, for The longitude; when the shortest distance is less than the dynamic threshold, obtain the drone fence.
[0012] Optionally, a dynamic threshold is calculated using the following formula: , For dynamic thresholds, The current speed of the drone. This is the preset time.
[0013] Optionally, acquire the drone fence when the drone goes beyond the latest fence.
[0014] Optionally, drone fences are obtained based on the location of the drone and stored in a fence database, including: when the number of drone fences in the fence database exceeds a preset number, deleting the drone fence with the earliest storage time.
[0015] Optionally, determine whether all points in the camera coordinate system satisfy the view frustum equation, wherein the view frustum equation includes: ,( , ( ) represents the coordinates of the point. This is the front trimming surface of the view frustum. The back cut surface of the view frustum. For the field of view, This is the vertical field of view angle.
[0016] Optionally, the feature weights of the points are obtained; when the drone height is less than or equal to the low preset height, all points are rendered; when the drone height is greater than the low preset height, aggregation mode is entered.
[0017] Optionally, when the drone's altitude is greater than the lower preset altitude but less than or equal to the higher preset altitude, points with a feature weight less than 0.5 are aggregated; when the drone's altitude is greater than the higher preset altitude, points with a feature weight less than 0.8 are aggregated; points whose distance is less than or equal to the aggregated distance are aggregated, and the point distance is calculated using the following formula: ,in, Distance between points and The coordinates of two points are used to calculate the distance between them.
[0018] According to another aspect of this application, a device for real-time annotation of aerial video is provided. The device includes: a first acquisition unit, configured to acquire a drone fence based on the drone's position after takeoff and store the drone fence in a fence database, wherein the drone fence is a circular area centered on the drone's latitude and longitude points, and the radius of the circular area is a real number greater than 0; a second acquisition unit, configured to acquire all points in the drone's camera coordinate system, retaining points that satisfy the frustum equation and discarding points that do not satisfy the frustum equation; and an aggregation unit, configured to aggregate the points based on the drone's altitude.
[0019] This invention dynamically calculates and records the fence surrounding a drone. Based on the shortest distance between the drone and the latest fence, if a threshold is exceeded, the fence and its surrounding data are dynamically updated. A dedicated point database retains data from the latest n fences, enabling dynamic fence updates and pre-loading of the dedicated point database. This solves the bottleneck of massive database retrieval, improving server performance by over 80%. Dynamic spatial grid indexing reduces retrieval complexity from O(n) to O(log n), and combined with view frustum optimization, further reduces computational load, lowering server response latency to milliseconds. Dynamic grid fence updates and layered pre-calculation adapt to changes in drone speed and altitude, as well as multi-resolution devices, avoiding label overlap. View frustum equations are calculated based on camera parameters, efficiently filtering out points outside the field of view. The dedicated point database and view frustum filtering reduce memory and CPU load, significantly improving point distribution performance and resulting in smooth, lag-free visuals. Points within the camera are filtered based on clipping plane conditions, greatly reducing unnecessary computation. Dynamic point distribution achieves visual optimization with near-fine detail and far-aggregated aggregation. Layered rendering provides visual depth; interactively, it supports expanding aggregated points and individual operations, improving ease of use. The modular design (mesh, view frustum, layered) makes it easy to integrate into existing drone systems. Attached Figure Description
[0020] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0021] Figure 1 This is a flowchart of a method for real-time annotation of aerial video according to an embodiment of this application;
[0022] Figure 2 This is a structural block diagram of a device for real-time annotation of aerial video according to an embodiment of this application. Detailed Implementation
[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.
[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0026] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.
[0027] This embodiment provides a method for real-time annotation of aerial video that runs on a mobile terminal, computer terminal or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0028] Figure 1 This is a flowchart illustrating a method for real-time annotation of aerial video according to an embodiment of this application. Figure 1 As shown, the method includes the following steps:
[0029] Step S101: After the drone takes off, the drone fence is obtained based on the drone's location and stored in the fence database. The drone fence is a circular area centered on the drone's latitude and longitude points, and the radius of the circular area is a real number greater than 0.
[0030] Specifically, this solution addresses the technical problem of low efficiency in global scanning of massive databases, where drones need to search for points within their flight radius (latitude and longitude range) every second. When generating a drone fence, this solution pre-retrieves points within the drone fence from a massive point database and stores them in a dedicated drone point database. Points are divided into a dynamic grid based on latitude and longitude. The dedicated drone point database pre-loads data within the fence, avoiding real-time scanning of the massive database.
[0031] Step S102: Obtain all points in the camera coordinate system of the UAV, retain the points that satisfy the frustum equation, and remove the points that do not satisfy the frustum equation.
[0032] Specifically, after the dynamic spatial grid preloads surrounding data, it is still necessary to further remove most points outside the field of view. The camera's view frustum is acquired in real-time based on the UAV's attitude, and the coordinates of points within the field of view are transformed. The view frustum is defined as a square pyramid.
[0033] Step S103: Aggregate the points based on the drone's altitude;
[0034] Specifically, because rendering all points at high altitudes would cause a technical problem where the display clusters would be indistinguishable, the pre-loaded points need to adapt to changes in the drone's altitude.
[0035] The method of this invention consists of dynamic spatial grid indexing, frustum optimization (assisted optimization), and hierarchical pre-computation. The system includes a UAV front-end, a server, and a point database module. The server receives UAV parameters (position, focal length, attitude) and processes the point data in real time for point deployment.
[0036] In one alternative embodiment, the location of the drone is checked at preset intervals to see if it is within the latest fence, where the latest fence is the latest drone fence in the fence database.
[0037] Specifically, after the drone takes off, the server records the current fence of the drone into the fence database: a circular area with a radius of R meters (default R=1000 meters, which can be set) centered on the current latitude and longitude point of the drone, and marks this fence as the latest fence.
[0038] In one alternative embodiment, when the drone is within the newly created fence, the shortest distance from the drone to the edge of the newly created fence is calculated using the following formula: ,in, For the shortest distance, To check the drone's position coordinates, The coordinates of the point closest to the drone at the edge of the newly constructed fence. Let be the spherical distance between two points; the formula for calculating the spherical distance between two points is: , The distance is spherical. For the Earth's radius, , ,in, for and The difference in latitude between them for and The difference in longitude between them for latitude, for latitude, for longitude, for The longitude; when the shortest distance is less than the dynamic threshold, the drone fence is obtained. The dynamic threshold is calculated using the following formula: , For dynamic thresholds, The current speed of the drone. This is the preset time.
[0039] Specifically, by controlling the basic scale of the threshold through a preset time (t), the dynamic threshold ensures that data loading is not delayed due to late triggering during high-speed flight, and also avoids resource waste caused by premature triggering during low-speed flight, thus achieving the optimization goal of smooth operation. The calculation of the dynamic threshold does not require manual intervention and is entirely based on the automatic adjustment of the UAV's real-time flight parameters, making the fence update mechanism more in line with the actual flight behavior of the UAV and improving the intelligence level of the entire point processing system.
[0040] In one alternative embodiment, the drone fence is acquired when the drone goes beyond the latest fence.
[0041] In one alternative embodiment, when the number of drone fences in the fence library exceeds a preset number, the drone fence with the earliest storage time is deleted.
[0042] Specifically, without setting an upper limit on the number of fences and without clearing historical data, the fence database will grow indefinitely as flight time increases, potentially leading to server storage overflow, decreased data read / write performance, and even affecting the normal operation of core functions such as point preloading and fence updates. This cleanup mechanism dynamically controls the size of the fence database to prevent system lag and crashes due to data overload, ensuring the continuous and stable operation of the system in long-duration drone flight scenarios.
[0043] In one optional embodiment, it is determined whether all points in the camera coordinate system satisfy the view frustum equation, wherein the view frustum equation includes: ,( , ( ) represents the coordinates of the point. This is the front trimming surface of the view frustum. The back cut surface of the view frustum. For the field of view, This is the vertical field of view angle.
[0044] Specifically, the front clipping plane is defined as: located in front of the camera at a distance of... The location, all satisfying The points constitute this plane. The back clipping plane is defined as the point located in front of the camera at a distance of... The location, all satisfying The points constitute this plane. After frustum filtering, only a small portion of the points require coordinate transformation, significantly reducing the computational load. This solves the problem of redundant data outside the field of view that still exists after dynamic preloading: the dynamic spatial mesh preloads all points within the fence, but the UAV camera's field of view is much smaller than the fence area. By using the frustum equation (front / back clipping plane, field of view constraint), points located only within the camera's field of view can be accurately filtered out, eliminating a large amount of invalid data outside the field of view. This reduces the processing objects for subsequent aggregation and rendering from the "fence level" to the "field of view level," significantly reducing the server's computational load and data transmission costs.
[0045] In one optional embodiment, the feature weights of the points are obtained; when the drone height is less than or equal to a low preset height, all points are rendered; when the drone height is greater than the low preset height, aggregation mode is entered.
[0046] Specifically, the feature weights represent the importance of each POI (Point of Interest), with values ranging from 0 to 1. All points are rendered independently globally without aggregation. Closer points are displayed as detailed details, while distant points are aggregated into icons, achieving a technical effect that improves visual depth and performance.
[0047] In one optional embodiment, when the drone altitude is greater than a lower preset altitude and less than or equal to a higher preset altitude, points with a feature weight less than 0.5 are aggregated; when the drone altitude is greater than a higher preset altitude, points with a feature weight less than 0.8 are aggregated; points whose distance is less than or equal to the aggregated distance are aggregated, and the point distance is calculated using the following formula: ,in, Distance between points and The coordinates of two points are used to calculate the distance between them.
[0048] Specifically, aggregation is only applied to low-weight and close-range points, while high-weight points are rendered directly without aggregation, avoiding ineffective processing of core points. At the same time, the height layering allows the aggregation intensity to be dynamically adjusted according to the field of view, which greatly reduces the aggregation calculation pressure on the server and the rendering load of the drone, avoiding frame rate drops and stuttering caused by too many points, and ensuring smooth display during flight.
[0049] This invention dynamically calculates and records the fence surrounding a drone. Based on the shortest distance between the drone and the latest fence, if a threshold is exceeded, the fence and its surrounding data are dynamically updated. A dedicated point database retains data from the latest n fences, enabling dynamic fence updates and pre-loading of the dedicated point database. This solves the bottleneck of massive database retrieval, improving server performance by over 80%. Dynamic spatial grid indexing reduces retrieval complexity from O(n) to O(log n), and combined with view frustum optimization, further reduces computational load, lowering server response latency to milliseconds. Dynamic grid fence updates and layered pre-calculation adapt to changes in drone speed and altitude, as well as multi-resolution devices, avoiding label overlap. View frustum equations are calculated based on camera parameters, efficiently filtering out points outside the field of view. The dedicated point database and view frustum filtering reduce memory and CPU load, significantly improving point distribution performance and resulting in smooth, lag-free visuals. Points within the camera are filtered based on clipping plane conditions, greatly reducing unnecessary computation. Dynamic point distribution achieves visual optimization with near-fine detail and far-aggregated aggregation. Layered rendering provides visual depth; interactively, it supports expanding aggregated points and individual operations, improving ease of use. The modular design (mesh, view frustum, layered) makes it easy to integrate into existing drone systems.
[0050] This application also provides an apparatus for real-time annotation of aerial video. It should be noted that the apparatus for real-time annotation of aerial video provided in this application can be used to execute the method for real-time annotation of aerial video provided in this application. This apparatus is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.
[0051] The following describes a device for real-time annotation of aerial video provided in an embodiment of this application.
[0052] Figure 2 This is a structural block diagram of a device for real-time annotation of aerial video according to an embodiment of this application. Figure 2 As shown, the device includes: a first acquisition unit 201, used to acquire a drone fence based on the drone's position after the drone takes off, and store the drone fence in a fence database, wherein the range of the drone fence is a circular area centered on the drone's latitude and longitude points, and the radius of the circular area is a real number greater than 0; a second acquisition unit 202, used to acquire all points in the drone's camera coordinate system, retaining points that satisfy the frustum equation and discarding points that do not satisfy the frustum equation; and an aggregation unit 203, used to aggregate the points based on the drone's altitude.
[0053] In one optional embodiment, the first acquisition unit 201 includes: a checking subunit, configured to check at preset intervals whether the location of the drone is within the latest fence, wherein the latest fence is the latest drone fence in the fence library.
[0054] In one optional embodiment, the checking subunit includes: a first calculation module, configured to calculate the shortest distance from the drone to the edge of the latest fence when the drone is within the latest fence, using the following formula: ,in, For the shortest distance, To check the drone's position coordinates, The coordinates of the point closest to the drone at the edge of the newly constructed fence. The first module is the spherical distance between two points; the second module is used to calculate the spherical distance between two points, and the calculation formula is: , The distance is spherical. For the Earth's radius, , ,in, for and The difference in latitude between them for and The difference in longitude between them for latitude, for latitude, for longitude, for The longitude; the first acquisition module is used to acquire the drone fence when the shortest distance is less than the dynamic threshold.
[0055] In one optional embodiment, the first acquisition module includes: a calculation submodule, used to calculate a dynamic threshold, the calculation formula being as follows: , For dynamic thresholds, The current speed of the drone. This is the preset time.
[0056] In one alternative embodiment, the inspection subunit includes: a second acquisition module for acquiring a drone fence when the drone goes beyond the latest fence.
[0057] In one optional embodiment, the first acquisition unit 201 includes: a removal subunit, used to delete the drone fence with the earliest storage time when the number of drone fences in the fence library exceeds a preset number.
[0058] In an optional embodiment, the second acquisition unit 202 includes: a judgment subunit, configured to judge whether all points in the camera coordinate system satisfy the view frustum equation, wherein the view frustum equation includes: ,( , ( ) represents the coordinates of the point. This is the front trimming surface of the view frustum. The back cut surface of the view frustum. For the field of view, This is the vertical field of view angle.
[0059] In one optional embodiment, the aggregation unit 203 includes: an acquisition subunit for acquiring feature weights of points; a rendering subunit for rendering all points when the drone height is less than or equal to a low preset height; and an aggregation subunit for entering an aggregation mode when the drone height is greater than the low preset height.
[0060] In one optional embodiment, the aggregation subunit includes: a first aggregation module, used to aggregate points with a feature weight less than 0.5 when the drone altitude is greater than a lower preset altitude and less than or equal to a higher preset altitude; a second aggregation module, used to aggregate points with a feature weight less than 0.8 when the drone altitude is greater than a higher preset altitude; and a third calculation module, used to aggregate points whose distance is less than or equal to the aggregation distance, and calculate the point distance using the following formula: ,in, Distance between points and The coordinates of two points are used to calculate the distance between them.
[0061] The device for real-time annotation of aerial video includes a processor and a memory. The first acquisition unit 201 and others are all stored as program units in the memory, and the processor executes the program units stored in the memory to achieve the corresponding functions. All of the above modules are located in the same processor; or, the above modules are located in different processors in any combination.
[0062] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the technical problems of inefficient real-time loading of data distribution information by drones, resource waste, and inaccurate data distribution.
[0063] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.
[0064] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.
[0065] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0066] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0067] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0068] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0069] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0070] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0071] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0072] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0073] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A method for real-time annotation of aerial video, characterized in that, include: After the drone takes off, a drone fence is obtained based on the drone's location, and the drone fence is stored in the fence database. The range of the drone fence is a circular area centered on the latitude and longitude points of the drone, and the radius of the circular area is a real number greater than 0. Obtain all points in the camera coordinate system of the UAV, retain the points that satisfy the frustum equation, and discard the points that do not satisfy the frustum equation; Aggregate locations based on drone altitude; Obtain the feature weights of the points; when the drone height is less than or equal to a low preset height, render all points; when the drone height is greater than the low preset height, enter aggregation mode; When the drone's altitude is greater than a lower preset altitude and less than or equal to a higher preset altitude, points with a feature weight less than 0.5 are aggregated; when the drone's altitude is greater than a higher preset altitude, points with a feature weight less than 0.8 are aggregated; points whose distance is less than or equal to the aggregation distance are aggregated, and the point distance is calculated using the following formula: ,in, The distance to the point is... and The coordinates of the two points are used to calculate the distance between the two points.
2. The method according to claim 1, characterized in that, Obtaining a drone fence based on the drone's location and storing the drone fence in a fence database includes: The location of the drone is checked at preset intervals to see if it is within the latest fence, where the latest fence is the latest fence for the drone in the fence library.
3. The method according to claim 2, characterized in that, The location of the drone is checked at preset intervals to ensure it is within the latest fence, including: When the drone is within the newly constructed fence, calculate the shortest distance from the drone to the edge of the newly constructed fence, using the following formula: ,in, The shortest distance, The location coordinates of the drone were used to check the drone's position. The coordinates of the point closest to the drone at the edge of the latest fence. The spherical distance between the two points; The formula for calculating the spherical distance between two points is: , The spherical distance is... For the Earth's radius, , ,in, for and The difference in latitude between them for and The difference in longitude between them for latitude, for latitude, for longitude, for Longitude; When the shortest distance is less than the dynamic threshold, obtain the drone fence.
4. The method according to claim 3, characterized in that, When the shortest distance is less than a dynamic threshold, obtain the drone fence, including: The dynamic threshold is calculated using the following formula: , The dynamic threshold, The current speed of the drone. The preset time is [the time specified in the original text].
5. The method according to claim 2, characterized in that, The location of the drone is checked at preset intervals to ensure it is within the latest fence, including: When the drone goes beyond the latest fence, acquire the drone fence.
6. The method according to claim 1, characterized in that, Obtaining a drone fence based on the drone's location and storing the drone fence in a fence database includes: When the number of drone fences in the fence library exceeds a preset number, delete the drone fence with the earliest storage time.
7. The method according to claim 1, characterized in that, Obtain all points in the camera coordinate system of the UAV, retain the points that satisfy the frustum equation, and discard the points that do not satisfy the frustum equation, including: Determine whether all points in the camera coordinate system satisfy the view frustum equation, wherein the view frustum equation includes: ,( , () represents the coordinates of the point. This is the front trimming surface of the view frustum. The back cut surface of the view frustum. For the field of view, This is the vertical field of view angle.
8. A device for real-time annotation of aerial video, characterized in that, include: The first acquisition unit is used to acquire a drone fence based on the drone's location after the drone takes off, and store the drone fence in a fence database. The range of the drone fence is a circular area centered on the latitude and longitude points of the drone, and the radius of the circular area is a real number greater than 0. The second acquisition unit is used to acquire all points in the camera coordinate system of the UAV, retain the points that satisfy the frustum equation, and discard the points that do not satisfy the frustum equation. The aggregation unit is used to aggregate points based on the drone's altitude; The unit is configured to acquire the feature weights of the points; the unit is configured to render all points when the drone height is less than or equal to a low preset height; and the unit is configured to enter the aggregation mode when the drone height is greater than a low preset height. The first aggregation module is used to aggregate points with a feature weight less than 0.5 when the drone's altitude is greater than a lower preset altitude and less than or equal to a higher preset altitude; the second aggregation module is used to aggregate points with a feature weight less than 0.8 when the drone's altitude is greater than a higher preset altitude; the third calculation module is used to aggregate points whose distance is less than or equal to the aggregation distance and calculate the distance between the points, using the following formula: ,in, The distance to the point is... and The coordinates of the two points are used to calculate the distance between the two points.