An unmanned aerial vehicle formation aerial photogrammetry method and system based on flight image control points

Abstract

This invention proposes a method and system for aerial photogrammetry of UAV formations based on flight control points, comprising: Step S1: Determining the survey area and photogrammetry parameters, and planning the flight path and multiple shooting time points of the UAVs; Step S2: Based on the flight path, shooting time points, and preset formation geometric parameters of the UAVs; Step S3: Each UAV flies autonomously according to the predetermined flight path and time, and achieves time synchronization through GNSS timing; Step S4: The UAVs capture images at each shooting time; Step S5: Extracting the pixel coordinates of each marker point in the image; Step S6: Performing time interpolation on the continuous RTK trajectory recorded by each positioning UAV to obtain the spatial coordinates of the positioning UAVs that are strictly synchronized with each shooting time; Step S7: Using the spatial coordinates of the positioning UAVs at each shooting time and their pixel coordinates in the image, calculating the exterior orientation elements through spatial resection, and completing the encryption and image stitching.

Description

Technical Field

[0001] This invention proposes a method and system for aerial photogrammetry of UAV formations based on flight control points, which relates to the field of aerial photogrammetry technology. Background Technology

[0002] In existing UAV aerial photogrammetry, ground control points (GCPs) are crucial for ensuring aerial triangulation and image stitching accuracy. Traditional methods require manually setting up GCPs within the survey area and measuring their coordinates using RTK, which presents the following problems:

[0003] Ground control points cannot be deployed in areas with large bodies of water, swamps, dense forests, deserts, etc.

[0004] Manual deployment is time-consuming and labor-intensive, and operations in dangerous areas are difficult;

[0005] Water surface reflection and lack of texture lead to image matching failure, and aerial triangulation results in weakly connected or unconnected regions.

[0006] Single-machine operation relies on high-precision inertial navigation, which is expensive and has cumulative errors;

[0007] Existing alternatives each have their advantages and disadvantages: LiDAR is extremely expensive; the deployment speed of control points is slow and it cannot be used on water; traditional ground control points cannot be deployed on water / forest and have extremely slow response.

[0008] Therefore, a solution that is cost-effective, fast-responding, suitable for water use, and reliable in terms of accuracy is needed. Summary of the Invention

[0009] In view of this, in order to fill the gaps and deficiencies in the existing technology, this invention proposes a UAV formation aerial photogrammetry method and system based on flight control points. This invention transforms the traditional fixed ground control points into moving airborne control points, so that each photo taken by the UAV has at least three marker points with known spatial coordinates. Through multi-UAV formation, high-precision and high-fault-tolerant real-time control point calculation is achieved, completely solving the problem of missing control points in textureless areas. At the same time, it eliminates the need for high-precision inertial navigation, significantly reducing costs, and has engineering practicality with adaptability to various UAV models, flexible time synchronization, instant response, and water surface adaptability.

[0010] This invention proposes a method and system for aerial photogrammetry of UAV formations based on flight control points, including the following:

[0011] According to a first aspect of the present invention, the present invention proposes an aerial photogrammetry method for UAV formations based on flight control points, characterized in that it includes the following:

[0012] Step S1: Determine the survey area and photogrammetry parameters, and plan the flight path and multiple photo capture points of the photogrammetric drone;

[0013] Step S2: Based on the flight path of the photography drone, the shooting time point, and the preset formation geometry parameters, calculate the flight path of the positioning drone, wherein at each shooting time, all positioning drones are located at predetermined spatial coordinates.

[0014] Step S3: Each UAV flies autonomously according to the predetermined route and time, and achieves time synchronization through GNSS time synchronization; during the flight, the positioning UAV continuously records its own RTK coordinates and the corresponding GNSS time, and the photography UAV records the exposure time and GNSS time of each image.

[0015] Step S4: The camera drone takes pictures at each shooting moment, and each picture contains the tags of all the positioning drones;

[0016] Step S5: Extract the pixel coordinates of each marker point in the image;

[0017] Step S6: Using the GNSS time of each photo taken by the camera drone as a reference, perform time interpolation on the continuous RTK trajectory recorded by each positioning drone to obtain the spatial coordinates of the positioning drone that are strictly synchronized with each photo taken.

[0018] Step S7: Utilize the spatial coordinates of the drone at each shooting moment and its pixel coordinates in the image, calculate the exterior orientation elements of each image of the drone through spatial resection, and complete the encryption and image stitching.

[0019] Furthermore, the formation geometry parameters mentioned above include the following:

[0020] The flight altitude Hc of the photography drone, the flight altitude Hp of the positioning drone, the horizontal distance between the positioning drones, and the relative positional relationship between the photography drone and the group of positioning drones; wherein the formation geometry parameters are dynamically adjusted according to the mapping scale, camera parameters, marker size and environmental conditions of the survey area.

[0021] Furthermore, the ratio k = Hp / Hc of the flight altitude of the positioning drone to the flight altitude of the photography drone satisfies:

[0022] The smaller the k value, the greater the tolerance for the deviation between the actual position and the predetermined position of each positioning drone at the moment of taking a picture;

[0023] The allowable range of the aforementioned deviation is determined based on the k value, the marker size constraint, and the stability requirements of the flight path of the UAV being located.

[0024] Furthermore, the size S of the mark drawn on the top of the positioning drone satisfies the following relationship:

[0025] ;

[0026] Where f is the focal length of the drone camera and p is the pixel size, to ensure that the number of pixels in the image of the logo is not less than 10.

[0027] Furthermore, the polygon area enclosed by the projections of all the markers of the positioning drones in the image accounts for 30% to 80% of the total image area.

[0028] Furthermore, linear interpolation or higher-order interpolation methods are used for time interpolation; the frequency at which the positioning UAV records RTK coordinates is no less than 10Hz to ensure that the interpolation accuracy is within 0.1 meters.

[0029] Furthermore, the positioning drone is a multi-rotor drone or a fixed-wing drone.

[0030] Furthermore, the UAV formation aerial photogrammetry method based on flight control points uses the spatial coordinates and pixel coordinates of the marker points to solve for the exterior orientation elements of the image.

[0031] According to a second aspect of the present invention, the present invention provides a UAV formation aerial photogrammetry system based on flight control points, for performing a UAV formation aerial photogrammetry method based on flight control points as described in any one of the present invention, characterized in that the UAV formation aerial photogrammetry system based on flight control points includes the following:

[0032] The photography drone module includes at least one photography drone equipped with an aerial camera and an RTK module, used to fly along a planned route and capture images at a predetermined time, while recording the exposure time and GNSS time of each image.

[0033] The positioning drone module includes at least three positioning drones, each of which has a high-contrast mark painted on its top and is equipped with an RTK module for flying along a planned route and continuously recording its own RTK coordinates and corresponding GNSS time during flight.

[0034] The background data processing module is used to plan the flight path of the photography drones, calculate and locate the flight path of the drones based on the formation geometry parameters, generate the speed curve of each drone, and receive the data recorded by each drone after the flight mission is completed.

[0035] Furthermore, the background data processing module is also used to perform the following: Extract the pixel coordinates of marker points in the image; Based on the GNSS time of the photography drone, the RTK trajectory of the positioning drone is interpolated to obtain the spatial coordinates of the positioning drone at the time of the photo capture. By using the spatial and pixel coordinates of the marker points, the exterior orientation elements of the image are calculated to complete the encryption and image stitching.

[0036] The present invention has the following advantages:

[0037] Completely solves the problem of mapping areas without texture: For the first time, high-precision aerial triangulation calculations have been achieved in areas where ground control points cannot be set up, such as water surfaces, forests, and deserts.

[0038] Each photo has multiple known control points: the number is adjustable (≥3), the accuracy increases with the number of positioning drones, and can be checked through redundant observations.

[0039] Formation parameters are flexible and adjustable: the formation design can be optimized according to the mapping scale, camera parameters, and environmental conditions, reducing the difficulty of flight control while ensuring accuracy.

[0040] The height-to-weight ratio design enhances engineering tolerance: by lowering the altitude of the positioning drone, the requirements for flight position accuracy can be relaxed, allowing ordinary consumer-grade drones to meet mission requirements.

[0041] Coordinate interpolation ensures time synchronization accuracy: Even if the positioning drone does not record coordinates at the moment of taking the picture, the coordinates of the control point can still be obtained with sub-m accuracy through high-frequency RTK trajectory interpolation.

[0042] Eliminating the need for high-precision inertial navigation significantly reduces costs: By directly calculating attitude using marker points, photography drones do not require inertial navigation, saving hundreds of thousands of yuan in equipment costs, while also reducing load, extending battery life, and eliminating cumulative errors.

[0043] High fault tolerance: The more drones used for positioning, the stronger the fault tolerance (for example, when there are 4 drones, it can tolerate the failure of one drone).

[0044] Easy to implement: Allows for large positional deviations, reducing the difficulty of flight control; no communication scheme is needed, relying on predetermined routes and GNSS timing synchronization, eliminating the need for complex communication links.

[0045] Compatible with multiple aircraft types: It can be used with both multi-rotor and fixed-wing aircraft. Fixed-wing aircraft are faster and have a larger back, making them suitable for large-area aerial surveying and high-marking identification.

[0046] Flexible deployment and low barrier to entry: Teams can be temporarily formed through leasing or borrowing, without the need for long-term dedicated machine clusters.

[0047] Strong instant response capability: No need to set up control points in advance, it can start working immediately after takeoff, and is especially suitable for emergency mapping, dynamic water surface (waves do not affect) and other scenarios.

[0048] Highly scalable: Real-time communication can be added in the future to further improve accuracy, but this patent protects the core concept and does not limit the specific implementation. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the steps of the present invention.

[0050] Figure 2 This is a schematic diagram of the multi-aircraft formation pyramidal configuration of the present invention.

[0051] Figure 3 This is a schematic diagram of the fixed-wing dorsal marking of the present invention.

[0052] Figure 4 This is a schematic diagram of the back markings of the multi-rotor rotor of the present invention.

[0053] Figure 5 This is a flowchart of time synchronization and speed planning in the non-communication mode of the present invention.

[0054] Figure 6 This is a flowchart of the process of the present invention.

[0055] Figure 7 This is a schematic diagram comparing the present invention with other technical approaches. Detailed Implementation

[0056] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0057] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0058] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the present invention; as used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise; furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0059] like Figures 1 to 7 As shown, this invention proposes a method and system for aerial photogrammetry of UAV formations based on flight control points, including the following:

[0060] According to a first aspect of the present invention, the present invention proposes a method for aerial photogrammetry of UAV formations based on flight control points, wherein the method includes the following:

[0061] Step S1: Determine the survey area and photogrammetry parameters, and plan the flight path and multiple photo capture points of the photogrammetric drone;

[0062] Step S2: Based on the flight path of the photography drone, the shooting time point, and the preset formation geometry parameters, calculate the flight path of the positioning drone, wherein at each shooting time, all positioning drones are located at predetermined spatial coordinates.

[0063] Step S3: Each UAV flies autonomously according to the predetermined route and time, and achieves time synchronization through GNSS time synchronization; during the flight, the positioning UAV continuously records its own RTK coordinates and the corresponding GNSS time, and the photography UAV records the exposure time and GNSS time of each image.

[0064] Step S4: The camera drone takes pictures at each shooting moment, and each picture contains the tags of all the positioning drones;

[0065] Step S5: Extract the pixel coordinates of each marker point in the image;

[0066] Step S6: Using the GNSS time of each photo taken by the camera drone as a reference, perform time interpolation on the continuous RTK trajectory recorded by each positioning drone to obtain the spatial coordinates of the positioning drone that are strictly synchronized with each photo taken.

[0067] Step S7: Utilize the spatial coordinates of the drone at each shooting moment and its pixel coordinates in the image, calculate the exterior orientation elements of each image of the drone through spatial resection, and complete the encryption and image stitching.

[0068] In one embodiment of the present invention, the formation geometry parameters include the following:

[0069] The flight altitude Hc of the photography drone, the flight altitude Hp of the positioning drone, the horizontal distance between the positioning drones, and the relative positional relationship between the photography drone and the group of positioning drones; wherein the formation geometry parameters are dynamically adjusted according to the mapping scale, camera parameters, marker size and environmental conditions of the survey area.

[0070] In one embodiment of the present invention, the ratio k = Hp / Hc of the flight altitude Hp of the positioning drone to the flight altitude Hc of the photography drone satisfies:

[0071] The smaller the k value, the greater the tolerance for the deviation between the actual position and the predetermined position of each positioning drone at the moment of taking a picture;

[0072] The allowable range of the aforementioned deviation is determined based on the k value, the marker size constraint, and the stability requirements of the flight path of the UAV being located.

[0073] In one embodiment of the present invention, the size S of the mark drawn on the top of the positioning drone satisfies the following relationship:

[0074] ;

[0075] Where f is the focal length of the drone camera and p is the pixel size, to ensure that the number of pixels in the image of the logo is not less than 10.

[0076] In one embodiment of the invention, the polygon area enclosed by the projections of all the markers of the positioning drones in the image occupies 30% to 80% of the entire image area.

[0077] In one embodiment of the present invention, time interpolation employs linear interpolation or higher-order interpolation methods; the frequency at which the positioning UAV records RTK coordinates is not less than 10Hz to ensure interpolation accuracy within 0.1 meters.

[0078] In one embodiment of the present invention, the positioning drone is a multi-rotor drone or a fixed-wing drone.

[0079] In one embodiment of the present invention, the UAV formation aerial photogrammetry method based on flight control points uses the spatial coordinates and pixel coordinates of the marker points to solve for the exterior orientation elements of the image.

[0080] According to a second aspect of the present invention, an aerial photogrammetry system for unmanned aerial vehicle (UAV) formations based on flight control points is provided for performing an aerial photogrammetry method for UAV formations based on flight control points as described in any one of the present invention. In one embodiment of the present invention, the aerial photogrammetry system for UAV formations based on flight control points includes the following:

[0081] The photography drone module includes at least one photography drone equipped with an aerial camera and an RTK module, used to fly along a planned route and capture images at a predetermined time, while recording the exposure time and GNSS time of each image.

[0082] The positioning drone module includes at least three positioning drones, each of which has a high-contrast mark painted on its top and is equipped with an RTK module for flying along a planned route and continuously recording its own RTK coordinates and corresponding GNSS time during flight.

[0083] The background data processing module is used to plan the flight path of the photography drones, calculate and locate the flight path of the drones based on the formation geometry parameters, generate the speed curve of each drone, and receive the data recorded by each drone after the flight mission is completed.

[0084] In one embodiment of the present invention, the background data processing module is further configured to perform the following: Extract the pixel coordinates of marker points in the image; Based on the GNSS time of the photography drone, the RTK trajectory of the positioning drone is interpolated to obtain the spatial coordinates of the positioning drone at the time of the photo capture. By using the spatial and pixel coordinates of the marker points, the exterior orientation elements of the image are calculated to complete the encryption and image stitching.

[0085] In addition to the above, such as Figures 1 to 7 As shown, the present invention also has related embodiments, including the following:

[0086] In one embodiment of the present invention, the drone classification numbers are as follows:

[0087] For the photography drone, the quantity is set to 1 unit. Its designation is number 0.

[0088] The photography drone is designated as drone number 0. Drone number 0 is equipped with an aerial camera and an RTK module; it is used to perform aerial photography missions.

[0089] For positioning drones, the quantity should be at least 3 (4 recommended), and they should be numbered 1, 2, 3, and 4.

[0090] Four positioning drones are designated as 1, 2, 3, and 4. Each drone is equipped with a high-contrast ground control marker (such as a red and white cross) painted on its top surface and has an integrated RTK module. These markers are used to provide aerial ground control points; the more drones there are, the higher the accuracy and the stronger the redundancy.

[0091] In one embodiment of the present invention, the formation can be flexibly adjusted according to the mapping scale, camera parameters, positioning UAV size, and ground environment. The core design principles are as follows:

[0092] Geometric Relationship: The photography drone (drone 0) is positioned above, and the positioning drones (drones 1-4) are positioned below, forming an approximate pyramidal structure. The plane containing the positioning drones should be lower than that of the photography drones to ensure that all positioning drone markers are captured in the photographs and are evenly distributed.

[0093] Altitude ratio and error tolerance: The smaller the ratio hp / hc (the altitude of the positioning drone is lower than that of the photography drone), the smaller the projection deviation on the image caused by the same actual position deviation. Therefore, the requirements for the flight control accuracy of the positioning drone can be appropriately relaxed. For example, when hp=40 m and hc=100 m, the allowable position deviation can reach 5 m; if hp is reduced to 20 m, the allowable deviation can be greater (this needs to be checked in conjunction with the number of imaging pixels at the image control points).

[0094] Marker imaging size constraints: The lower the positioning drone flies, the larger the marker's image size on the image, making it easier to identify. However, this also requires a smaller physical size for the marker (small drones can be used). But be aware that flying too low may cause the marker to exceed the image frame or obscure too much ground imagery. Generally, a marker imaging size of ≥10×10 pixels is required. Based on this, the required physical size of the marker or the minimum flight altitude can be calculated.

[0095] Projection area constraint: The area of ​​the polygon formed by the projections of all positioning drone markers on the photograph should occupy 30% to 80% of the total photograph area to ensure the stability of aerial triangulation. This ratio can be achieved by adjusting the horizontal spacing and height difference between the positioning drones.

[0096] Environmental adaptability: This method is primarily designed for flat, unobstructed areas such as large bodies of water, deserts, and swamps. Therefore, the altitude of the positioning drone can be reduced to 20m or even lower (as long as it is above the safe height of the water or ground surface) to enhance tolerance to flight errors and reduce the size of the markers. If there are obstacles (such as trees or buildings) in the survey area, the altitude of the positioning drone needs to be increased accordingly.

[0097] In one embodiment of the present invention, example parameters (1:500 scale aerial survey) are as follows:

[0098] Photography drone altitude: 100m

[0099] Positioning drone altitude: 40m

[0100] The positioning drones are distributed horizontally in a rectangle with a front-to-back spacing of 40m and a left-to-right spacing of 70m.

[0101] The camera drone is located directly above the center of the rectangle.

[0102] Sign size: 1m×1m (multi-rotor) or 2m×2m (fixed-wing)

[0103] Permissible positional deviation: ≤5m (mean error)

[0104] The above parameters are for illustrative purposes only. In actual operations, they should be dynamically adjusted according to factors such as scale, camera parameters, and drone performance, all of which fall within the protection scope of this invention.

[0105] In one embodiment of the present invention, the operation process includes the following:

[0106] It adopts a non-real-time communication mode, relying on precise GNSS time synchronization and velocity planning to achieve:

[0107] Determine the flight path of the photography drone: First, plan the flight path and shooting time points t1, t2, ..., tn of the photography drone (drone 0).

[0108] Solving the positioning drone flight path: Based on the flight path of the photography drone, the formation geometry constraints (height, horizontal spacing) and the performance of each drone, the spatial coordinates PijPij that each positioning drone should reach at each shooting moment are calculated.

[0109] Speed ​​planning: Taking into account the maximum speed, endurance, and flight path length of each positioning drone, the Skybrush software automatically generates the speed curve for each drone to ensure that all positioning drones arrive at the predetermined location on time.

[0110] Independent Flight and Data Recording: Each drone flies autonomously according to a predetermined route and time, synchronized by a GNSS clock. During flight, the positioning drone continuously records its own RTK coordinates and corresponding GNSS time at a high frequency (e.g., 10Hz); the photography drone records the GNSS timestamp of the image exposure time at the moment of taking a picture.

[0111] Coordinate interpolation: Since the coordinates recorded by the positioning drone may not perfectly coincide with the time of the photographing drone, time interpolation is needed to obtain the precise coordinates of the positioning drone at the time of the photograph. The specific method is as follows: using the GNSS time of the photographing drone as a reference, select the nearest recording point before and after that time in the continuous RTK trajectory recorded by the positioning drone, and use linear interpolation (or higher-order interpolation) to calculate the coordinates of the photographing time, thereby obtaining the coordinates of the control point that is strictly synchronized with the image.

[0112] Error tolerance: The actual arrival position is allowed to deviate from the ideal point. The specific tolerance value is determined by the formation height ratio (see Section 4). The interpolation accuracy is affected by the RTK recording frequency (when recording at 10Hz, the interpolation error is usually less than 0.1m), which does not affect the accuracy of aerial triangulation.

[0113] In one embodiment of the present invention, the detailed process of the steps is as follows:

[0114] Step 1: Determine the survey area and design formation parameters (photography drone altitude, positioning drone altitude, horizontal spacing, marker size, etc.) based on the scale, camera parameters, and environmental conditions.

[0115] Step 2: Plan the flight path and photo spots for camera 0.

[0116] Step 3: The software (such as Skybrush) automatically calculates the flight path of the photography drones based on the flight path, formation parameters and performance of each drone, and generates the speed curve of each drone.

[0117] Step 4: Each drone flies autonomously according to the predetermined route and time, relying on GNSS time synchronization; during flight, the positioning drone continuously records RTK coordinates and GNSS time, and the photography drone records the exposure time and GNSS time of each photo.

[0118] Step 5: Take photos at the designated locations using drone #0. Each photo will contain the markers for all the drones being located.

[0119] Step 6: The aerial triangulation software automatically extracts the pixel coordinates of each marker point in the photo.

[0120] Step 7: Coordinate Interpolation: Based on the GNSS time of the photography drone, perform time interpolation on the RTK trajectory of each positioning drone to obtain the precise spatial coordinates of each positioning drone at the time of the photo capture.

[0121] Step 8: Using the known spatial coordinates and their image coordinates, calculate the six exterior orientation elements of the photograph through spatial resection (no inertial navigation required).

[0122] Step 9: Complete aerial triangulation encryption and image stitching.

[0123] In one embodiment of the invention, the information regarding the model and markings also includes the following:

[0124] The positioning drone can be a multi-rotor drone or a fixed-wing drone; the back of the fixed-wing drone is used to draw larger-sized markers to improve recognition distance and robustness.

[0125] Multi-rotor drones: Hexacopter, octacopter, etc. can all be used as positioning drones, and the top area is convenient for drawing marks;

[0126] Fixed-wing UAVs: They have a large back area, making them more suitable for drawing large-sized ground control markers, and they fly at high speeds, making them suitable for large-area aerial surveys; speed differences need to be considered when forming formations.

[0127] Logo design: High-contrast geometric patterns such as red and white crosses, circles, and QR codes;

[0128] Marker size: Designed based on flight altitude, camera resolution, and ground sampling distance (GSD) to ensure clear visibility in the image (recommended image size ≥ 10 × 10 pixels).

[0129] Larger markings can be painted on the back of the fixed wing, improving recognition distance and robustness.

[0130] In one embodiment of the present invention, the relevant technical features also include the following:

[0131] Multi-drone positioning: At least 3 positioning drones are required. The more drones there are, the higher the accuracy and the stronger the redundancy. It is recommended to have 4 drones to provide redundant observations.

[0132] Universal for various models: Suitable for multi-rotor, fixed-wing, and other types of drones. The fixed-wing drone has a larger back, making the markings clearer.

[0133] Adjustable formation parameters: The altitude of the photography drone, the altitude of the positioning drone, and the horizontal spacing can be flexibly adjusted according to the scale, camera parameters, and environmental conditions to adapt to different operational needs.

[0134] Altitude-to-tolerance design: The smaller the ratio of the altitude of the positioning drone to that of the photography drone, the greater the tolerance for flight control errors and the simpler the engineering implementation.

[0135] Sign size and altitude correlation: The physical size of the sign, the flight altitude, and the camera resolution must match to ensure clear imaging (≥10 pixels).

[0136] Projection area constraint: The projected area of ​​the marker points occupies 30% to 80% of the photo to ensure the stability of the solution.

[0137] Coordinate interpolation: Time interpolation is performed using the continuous RTK trajectory of the positioning drone and the time of the photography drone to ensure that the coordinates of the control point are strictly synchronized with the image.

[0138] Eliminating the need for inertial navigation: Attitude is directly calculated using marker points, eliminating the need for a high-precision IMU in photography drones.

[0139] GNSS time synchronization: Relies on a GNSS clock to achieve high-precision time synchronization with an error of <1ms, providing an accurate time reference for coordinate interpolation.

[0140] Skybrush planning: Skybrush software is used for formation route generation and speed planning, with a high degree of automation.

[0141] Compatible with existing software: Marker points can be automatically recognized by mainstream aerial triangulation software (Pix4D, ContextCapture, etc.).

[0142] Flexible deployment: Drones can be temporarily rented / borrowed to form a formation, lowering the barrier to entry.

[0143] In one embodiment of the present invention, a 1:500 scale topographic mapping of a coastal area is required, covering a large area of ​​sea. Five quadcopter drones are used: drone #0 is equipped with a single-lens camera (8.8mm focal length, 2.41μm pixel size), and drones #1- #4 are positioning drones, each with a 0.5m × 0.5m red and white cross mark painted on its top surface. The GSD is 2.74cm, and the camera drone is designed to fly at a height of 100m. To enhance tolerance to flight errors, the positioning drones are set at a height of 40m, horizontally distributed in a rectangle 40m forward and backward and 70m left and right, ensuring that the projected area of ​​the marker points occupies approximately 50% of the photograph. The photo-taking interval is 2 seconds. Skybrush software is used to calculate the flight path of the positioning drones and generate speed curves for each drone. During flight, the positioning drones record RTK coordinates and GNSS time at a frequency of 10Hz, while the camera drones record the exposure time of each photograph. In post-processing, the trajectories of each positioning drone are linearly interpolated based on the photo-taking times to obtain the precise coordinates of the photo-taking times. In actual flight, the position deviation is ≤5m, the interpolation accuracy is better than 0.1m, the aerial triangulation software successfully extracted four marker points, calculated the exterior orientation elements, the plane accuracy reached 5cm, the elevation accuracy was 8cm, which meets the requirements of 1:500 mapping.

[0144] In one embodiment of the invention, a 1:1000 scale orthophoto image was required for a desert area with flat, unobstructed terrain. One fixed-wing photography drone (equipped with a medium-format camera, 6μm pixel size, 80mm focal length) and four small fixed-wing drones (with 0.5m × 0.5m red and white crosses painted on their top surfaces) were used. To reduce flight control complexity, the positioning drone was set at an altitude of 20m, and the photography drone at 150m (meeting GSD ≤ 10cm). A rectangle with a horizontal distribution of 60m front-to-back and 100m left-to-right was used to ensure that the projected area of ​​the marker points occupied approximately 40% of the image. Due to the lower altitude of the positioning drone, a positional deviation of up to 10m was allowed (while still ensuring clear marker imaging). During flight, the positioning drone recorded the RTK trajectory (10Hz), and the photography drone recorded the time of image capture. The coordinates of the positioning drone at each image capture time were obtained through interpolation; the actual deviations were all within 8m, the aerial triangulation calculation was successful, and the accuracy met the 1:1000 specification requirements.

[0145] In one embodiment of the invention, an area affected by flooding urgently needed orthophotos due to damaged communication facilities. Five drones were used, operating in a pre-defined synchronized flight path mode. Based on the disaster area and camera parameters, the ground station designed the photography drone to fly at an altitude of 120m, the positioning drone at an altitude of 50m (considering water safety), with a horizontal spacing of 50m forward / backward and 80m left / right. Skybrush automatically generated the flight paths and speed curves for the four positioning drones. All drones flew independently, relying on GNSS time synchronization. During flight, the positioning drones recorded RTK trajectories, and the photography drones recorded the time of image capture. Afterwards, the coordinates of the image capture time were obtained through interpolation. Aerial triangulation was successful, and orthophotos of the disaster area were successfully generated.

[0146] In one embodiment of this invention, a research project required mapping a remote lake, but lacked a dedicated fleet of drones. The research team rented four ordinary quadcopter drones, temporarily attaching 0.5m x 0.5m red and white cross markers to their tops, and also rented one high-performance photography drone. Based on camera parameters (3.9μm pixel size, 24mm focal length) and the required GSD, the photography drone was designed to fly at an altitude of 80m, the positioning drone at 30m, with a horizontal spacing of 30m forward and backward and 50m left and right. The ground station software (Skybrush) imported the performance parameters of each drone and automatically calculated the flight path. During flight, all drones recorded RTK trajectories and timestamps, which were then interpolated to obtain the coordinates of the photo capture time. After the mission was completed, the drones were returned, retaining only the image data. This approach significantly reduced project costs.

[0147] In one embodiment of the present invention, the relevant parameters of the present invention further include the following:

[0148] Marker recognition: Mainstream aerial triangulation software (Pix4D, ContextCapture, etc.) supports automatic extraction of high-contrast marker points without the need for additional development.

[0149] Selection of the number of positioning drones: 3 drones can achieve basic functions, 4 drones provide redundancy, and 5 or more drones can further improve accuracy. Users can weigh the accuracy requirements and costs.

[0150] Formation parameter design tool: Dedicated software modules can be developed, which can input scale, camera parameters, drone performance, etc., and automatically recommend the optimal altitude of photography drones, altitude of positioning drones, horizontal spacing and marker size, and generate flight paths.

[0151] Time synchronization accuracy: GNSS time synchronization error <1ms, which has a negligible impact on coordinate interpolation and aerial triangulation.

[0152] Coordinate interpolation method: Generally, linear interpolation can meet the accuracy requirements; if the recording frequency of the positioning drone is high enough (such as 20Hz), the interpolation error can be controlled at the cm level.

[0153] Speed ​​planning algorithm: Skybrush automatically calculates speed curves based on each aircraft's maximum speed and flight path length to ensure on-time arrival.

[0154] Formation tolerance and altitude ratio relationship: The smaller the ratio of the positioning drone's altitude to the photography drone's altitude (k=hp / hc), the greater the allowable positional deviation. For example, when k=0.4, the allowable deviation can reach 5m; when k=0.2, the allowable deviation can be relaxed to 10m (needs to be verified in conjunction with the marker imaging size). In actual operations, the flight control accuracy requirements can be flexibly adjusted accordingly.

[0155] The relationship between marker size and the number of imaging pixels: The number of marker imaging pixels N = f⋅S / (H⋅p), where f is the focal length, S is the physical size of the marker, H is the relative height (the height difference between the photography drone and the positioning drone), and p is the pixel size. During the design phase, N should be ensured to be ≥ 10 to guarantee robust recognition.

[0156] Water surface adaptability: The markers are on drones and are not affected by waves or tides, making them particularly suitable for marine surveying and flood emergency response.

[0157] Formation geometry advantages: By adjusting the height difference and horizontal spacing, it can be ensured that the marker points are evenly distributed in each photo, the coverage area ratio meets 30% to 80%, and the solution stability is high.

[0158] The necessity of coordinate interpolation: Since it is difficult to strictly synchronize the recording and shooting time of the positioning drone, interpolation can eliminate the coordinate error caused by the time deviation, which is a key step to ensure the accuracy of the image control point.

[0159] In this invention, the core function of the positioning drone is to provide airborne control points with known spatial coordinates during flight. Therefore, its essential configuration includes: a high-contrast marker drawn on the top and an RTK positioning module (for recording coordinates). In addition, the positioning drone may, but is not limited to, carry only the aforementioned essential equipment, and may also carry other equipment, such as a camera, inertial measurement unit, and communication module, for performing other tasks or enhancing system redundancy. As long as the drone performs the role of a positioning drone in the method of this invention (i.e., providing aerial control points in formation and recording RTK trajectories for coordinate interpolation), it is considered an implementation of this invention. Whether the positioning drone possesses additional functions does not change its essential role in this invention, nor does it constitute a circumvention of this invention.

[0160] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for aerial photogrammetry of UAV formations based on flight control points, characterized in that, Includes the following: Step S1: Determine the survey area and photogrammetry parameters, and plan the flight path and multiple photo capture points of the photogrammetric drone; Step S2: Based on the flight path of the photography drone, the shooting time point, and the preset formation geometry parameters, calculate the flight path of the positioning drone, wherein at each shooting time, all positioning drones are located at predetermined spatial coordinates. Step S3: Each UAV flies autonomously according to the predetermined route and time, and achieves time synchronization through GNSS time synchronization; during the flight, the positioning UAV continuously records its own RTK coordinates and the corresponding GNSS time, and the photography UAV records the exposure time and GNSS time of each image. Step S4: The camera drone takes pictures at each shooting moment, and each picture contains the tags of all the positioning drones; Step S5: Extract the pixel coordinates of each marker point in the image; Step S6: Using the GNSS time of each photo taken by the camera drone as a reference, perform time interpolation on the continuous RTK trajectory recorded by each positioning drone to obtain the spatial coordinates of the positioning drone that are strictly synchronized with each photo taken. Step S7: Utilize the spatial coordinates of the drone at each shooting moment and its pixel coordinates in the image, calculate the exterior orientation elements of each image of the drone through spatial resection, and complete the encryption and image stitching.

2. The UAV formation aerial photogrammetry method based on flight control points according to claim 1, characterized in that, The formation geometry parameters mentioned above include the following: The flight altitude Hc of the photography drone, the flight altitude Hp of the positioning drone, the horizontal distance between the positioning drones, and the relative positional relationship between the photography drone and the group of positioning drones; wherein the formation geometry parameters are dynamically adjusted according to the mapping scale, camera parameters, marker size and environmental conditions of the survey area.

3. The UAV formation aerial photogrammetry method based on flight control points according to claim 2, characterized in that, The ratio k = Hp / Hc of the flight altitude of the positioning drone to the flight altitude of the photography drone satisfies: The smaller the k value, the greater the tolerance for the deviation between the actual position and the predetermined position of each positioning drone at the moment of taking a picture; The allowable range of the aforementioned deviation is determined based on the k value, the marker size constraint, and the stability requirements of the flight path of the UAV being located.

4. The UAV formation aerial photogrammetry method based on flight control points according to claim 3, characterized in that, The size S of the mark drawn on the top of the positioning drone satisfies the following relationship: ； Where f is the focal length of the drone camera and p is the pixel size, to ensure that the number of pixels in the image of the logo is not less than 10.

5. The UAV formation aerial photogrammetry method based on flight control points according to claim 1, characterized in that, The polygon area enclosed by the projections of all the markers of the positioning drones in the image accounts for 30% to 80% of the total image area.

6. The UAV formation aerial photogrammetry method based on flight control points according to claim 1, characterized in that, The time interpolation uses linear interpolation or higher-order interpolation methods; the frequency at which the positioning UAV records RTK coordinates is no less than 10Hz to ensure that the interpolation accuracy is within 0.1 meters.

7. The UAV formation aerial photogrammetry method based on flight control points according to claim 1, characterized in that, The positioning drone can be a multi-rotor drone or a fixed-wing drone.

8. The UAV formation aerial photogrammetry method based on flight control points according to claim 1, characterized in that, The aforementioned UAV formation aerial photogrammetry method based on flight control points uses the spatial coordinates and pixel coordinates of the marker points to solve for the exterior orientation elements of the image.

9. A UAV formation aerial photogrammetry system based on flight control points, used to perform a UAV formation aerial photogrammetry method based on flight control points as described in any one of claims 1 to 8, characterized in that, The UAV formation aerial photogrammetry system based on flight control points includes the following: The photography drone module includes at least one photography drone equipped with an aerial camera and an RTK module, used to fly along a planned route and capture images at a predetermined time, while recording the exposure time and GNSS time of each image. The positioning drone module includes at least three positioning drones, each of which has a high-contrast mark painted on its top and is equipped with an RTK module for flying along a planned route and continuously recording its own RTK coordinates and corresponding GNSS time during flight. The background data processing module is used to plan the flight path of the photography drones, calculate and locate the flight path of the drones based on the formation geometry parameters, generate the speed curve of each drone, and receive the data recorded by each drone after the flight mission is completed.

10. The UAV formation aerial photogrammetry system based on flight control points according to claim 9, characterized in that, The background data processing module is also used to perform the following: Extract the pixel coordinates of marker points in the image; Based on the GNSS time of the photography drone, the RTK trajectory of the positioning drone is interpolated to obtain the spatial coordinates of the positioning drone at the time of the photo capture. By using the spatial and pixel coordinates of the marker points, the exterior orientation elements of the image are calculated to complete the encryption and image stitching.

Images