A method for constructing a working area map of a UAV based on aerial images

Abstract

This invention belongs to the field of UAV autonomous positioning and navigation technology, and relates to a UAV map construction method for environments where satellite imagery is unavailable. Specifically, it provides a method for constructing a UAV work area map based on aerial imagery, addressing issues such as image distortion and edge misalignment in existing area map construction methods. This invention orthogonally divides the UAV's aerial photography work area into a grid according to the same geographic coordinate system. The UAV performs a coverage flight mission within the work area according to a planned flight path, maintaining an interval of 50% of the field of view width between adjacent flight paths. During aerial photography, the UAV simultaneously acquires image and geographic positioning information (GPS information and yaw angle information of the center point) as image location points. Based on UAV sensor data and the UAV's flight trajectory and pose, keyframes of geographic location information are extracted from the aerial video, and then used to fill the corresponding grid content to obtain a global map of the work area.

Description

Technical Field

[0001] This invention belongs to the field of UAV autonomous positioning and navigation technology, and relates to a method for constructing UAV maps in denial-of-access environments where satellite imagery is unavailable. Specifically, it provides a method for constructing a UAV work area map based on aerial imagery. Background Technology

[0002] Autonomous positioning and navigation of unmanned aerial vehicles (UAVs) generally rely on communication methods such as radar and satellites. However, radio signals and satellite signals are easily interfered with or intentionally attacked, leading to the loss of GPS signals and causing the UAV to lose its autonomous capabilities. With the development of computer vision, visual positioning methods have been extensively studied. Visual positioning uses a visible light camera, a common UAV-mounted device, without relying on other external equipment, and uses machine vision information to estimate highly accurate geographic positioning information. Therefore, this invention is based on serving visual positioning and constructing a visual visualization map for UAV visual positioning.

[0003] In denied-access environments, UAV mapping primarily utilizes aerial geographic information captured by onboard visible light cameras. This is combined with GPS coordinates of ground feature points obtained during flight using methods such as visual odometry and IMU, mapping these GPS coordinates to pixel positions in the aerial images. UAV-acquired aerial images typically exhibit high overlap and small image size. To gain a more intuitive and comprehensive understanding of the target area's actual situation, image stitching techniques are crucial. Image stitching involves combining two or more images (from different sensors, different viewpoints, or different times, containing partially identical areas) in a specific order to create a new, higher-resolution, wider-view image with more pixels and information elements than a single image. Traditional map stitching methods often employ manual annotation and selection, but due to the high overlap and small image size of UAV-acquired aerial images, these methods are unsuitable for constructing maps of large areas. Furthermore, existing research utilizes ground markers as identifiers to filter keyframes and complete map construction; however, these methods are unusable when the work area is inaccessible. In addition, existing research has also used the speed of drones and the calculation of ground distance to obtain video frames and realize map construction. However, the speed of drones is affected by factors such as wind speed and altitude, and it is impossible to achieve an absolutely uniform speed. Summary of the Invention

[0004] The purpose of this invention is to provide a method for constructing a drone work area map based on aerial imagery, thereby solving problems such as image distortion and edge misalignment in existing area map construction methods. This invention orthogonally divides the drone's aerial photography work area into a grid according to the same geographic coordinate system. The drone performs a coverage flight mission within the work area according to a planned flight path, maintaining the interval between adjacent flight paths at 50% of the field of view width. During aerial photography, the drone simultaneously acquires images and geographic positioning information (GPS information and yaw angle information of the center point) as image location points. Based on drone sensor data and the drone's flight trajectory and pose, keyframes of geographic location information are extracted from the aerial video, and then the corresponding grid content is filled to obtain a global map of the work area.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for constructing a drone work area map based on aerial imagery, characterized by the following steps:

[0007] Step 1. Calculate the distance calibration value and latitude / longitude change per unit pixel in the image captured by the UAV at a fixed altitude.

[0008] Fly the drone to a preset altitude H and capture an image containing markers A and B. Then, based on the GPS coordinates of markers A and B: (lon A ,lat A (lon) B ,lat B ) and the pixel coordinates of A and B in the image: (x A ,y A ), (x B ,y B Calculate the distance calibration value per unit pixel and the corresponding latitude and longitude change per unit pixel:

[0009]

[0010]

[0011] Dis lon =R×arccos(cos(lat) A cos(lat) A )cos(lon B -lon A )+sin(lat A sin(lat) A ))

[0012] Dis lat =R×arccos(sin(lat) A sin(lat)B ))

[0013]

[0014]

[0015] Among them, P lon The longitude calibration value per unit pixel, P lat Δ is the latitude calibration value per unit pixel. lon Δ represents the change in longitude per unit pixel. lat R represents the latitude change per unit pixel; R is the Earth's radius.

[0016] Step 2. Set the planned flight path for the drone;

[0017] The drone's flight altitude is set to H; the flight path adopts a standard surveyed flight path: it consists of multiple channels, each with the same distance. It first flies from north to south along channel 1 from the starting point. After reaching the designated mileage hall, it turns and flies to the next channel. Channels with odd-numbered indices run from north to south, while those with even-numbered indices run from south to north. The interval between adjacent channels is [interval]. lon for:

[0018]

[0019] Where width is the width of the image captured by the drone;

[0020] Step 3. Collect flight data;

[0021] Adjust the drone's camera lens to look downwards and lock the gimbal; align the timestamps for the drone's GPS data transmission, yaw angle data transmission, and image transmission to enable the drone to complete the flight along the planned path and collect flight videos of the work area; each video frame in the flight video contains a timestamp, GPS information of the center point, and yaw angle information;

[0022] Step 4. Create a work area map;

[0023] Step 4-1. Obtain all geographic information keyframes for the starting channel;

[0024] Step 4-1-1. Obtain the initial state geographic information keyframes of the starting channel;

[0025] The time when the UAV arrives at the starting point of the planned flight path is taken as the starting time to obtain the starting video frame. The geographical location information of the starting video frame is calculated. The starting video frame containing the geographical location information is taken as the starting state geographical information key frame and the flight path index is marked as 1.

[0026] Step 4-1-2. Obtain keyframes of flight status geographic information for the starting flight path;

[0027] For each video frame after the initial frame in the flight video, calculate the current flight distance (Distance) of the drone:

[0028] Distance = R × arccos[cos(lat) p cos(lat) n )cos(lon p -lon n )+sin(lat p sin(lat) n )]

[0029] Among them, (lon) p ,lat p (lon) represents the GPS information of the center point of the previous geographic keyframe. n ,lat n This refers to the GPS information of the center point of the current video frame.

[0030] Set the distance interval threshold. lat :

[0031]

[0032] Where length is the length of the image captured by the drone;

[0033] If the flight distance Distance equals the distance interval threshold interval lat If the current video frame is determined to be a geographic information keyframe, the geographic location information of the current video frame is calculated to obtain the flight status geographic information keyframe.

[0034] Based on the yaw angle information, determine whether the turning point has been reached. If not, mark the channel index as the same as the previous geographic information keyframe; if so, proceed to step 4-2.

[0035] Step 4-2. Obtain all geographic information keyframes for the remaining waterways;

[0036] Step 4-2-1. Obtain the initial state geographic information keyframes of the new waterway;

[0037] Because the starting position of the new channel is opposite to the course angle of the previous channel, to ensure seamless stitching, the latitude of the starting position of the new channel should be the same as the latitude of the ending position of the previous channel. That is, the latitudes are the same when the distance between the two points is equal to the channel interval. Therefore, based on the yaw angle information and GPS information, it is determined whether the coordinates of the center point of the current video frame simultaneously meet the following conditions:

[0038] |now yaw +pre yaw |=π

[0039] R×arccos[cos(lat p cos(lat) n )cos(lon p -lon n )+sin(lat p sin(lat) n )] = interval lon

[0040] Among them, now yaw This indicates the yaw angle information of the current video frame, pre yaw This indicates the yaw angle information of the previous geographic keyframe;

[0041] If so, the current video frame is used as the starting video frame of the new channel; the geographical location information of the starting video frame is calculated, the starting video frame containing the geographical location information is used as the starting state geographical information key frame, and the channel index is marked as the channel index value of the previous geographical information key frame plus 1.

[0042] Step 4-2-2. Obtain keyframes of the flight status geographic information of the new airway;

[0043] For each video frame following the initial video frame of the current flight path, calculate the current flight distance Distance of the UAV. If the flight distance Distance equals the distance interval threshold interval... lat If the current video frame is determined to be a geographic information keyframe, the geographic location information of the current video frame is calculated to obtain the flight status geographic information keyframe.

[0044] Determine whether the turning point has been reached based on the yaw angle information. If not, mark the channel index as the same as the previous geographic information keyframe; if yes, proceed to step 4-2-1.

[0045] Step 4-3. Concatenate all geographic information keyframes sequentially according to timestamp and channel index to obtain the working area map.

[0046] Furthermore, the calculation process for geographic location information is as follows:

[0047] Obtain GPS information of the center point of the video frame: (lon) centre ,lat centre The starting video frame is rotated to true north based on the yaw angle information, and the GPS information of the remaining pixels in the video frame is calculated.

[0048] lonother =lon centre +diff y ×Δ lon

[0049] lat other =lat centre +diff x ×Δ lat

[0050] Among them, (lon) other ,lat other () represents the GPS information of the current pixel in the video frame, diff x diff y These represent the pixel differences between the current pixel and the center point in the video frame in the width and length directions, respectively.

[0051] The GPS information of each pixel in the video frame is calculated to form the geographic location information of the video frame.

[0052] Based on the above technical solution, the beneficial effects of the present invention are as follows:

[0053] This invention provides a method for constructing a UAV working area map based on aerial imagery, which has the following advantages:

[0054] Firstly, image stitching based on grid filling is used;

[0055] The working area is divided into grids of the same size according to the field of view of the drone's camera. The assumption is that the drone's own position is the corner point of the grid. The key frames obtained by trajectory calculation are sequentially filled into the grid. This method effectively solves the problems of determining the order of the image sequence and edge misalignment.

[0056] Secondly, keyframe filtering based on trajectory calculation is adopted;

[0057] This invention proposes a trajectory calculation method based on relative flight distance to select key frames of geographic location information and form a sequence. At the same time, it proposes a turning point judgment method based on the minimum latitude distance combined with yaw angle, which can effectively and accurately determine the boundary key frames during mapping. Compared with the existing flight speed-based scheme, it has strong robustness to parameters such as flight altitude and speed.

[0058] In summary, this invention makes full use of the latitude and longitude information of the UAV itself, and proposes a grid stitching method and a trajectory calculation method based on relative distance, which has the advantages of edge alignment, low computational cost and strong robustness. Attached Figure Description

[0059] Figure 1This is a flowchart illustrating the method for constructing a drone work area map based on aerial images in an embodiment of the present invention.

[0060] Figure 2 This is a schematic diagram of UAV trajectory planning in an embodiment of the present invention.

[0061] Figure 3 This is a schematic diagram illustrating the principle of the grid-based image stitching method in an embodiment of the present invention. Detailed Implementation

[0062] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0063] This embodiment provides a method for constructing a drone work area map based on aerial imagery, including the following steps:

[0064] Step 1. Calculate the distance calibration value and latitude / longitude change per unit pixel in the image captured by the UAV at a fixed altitude.

[0065] Fly the drone to a designated altitude H and hover it. Following the instructions from the aerial imagery, place markers A and B at designated locations within the drone's field of view. Obtain the GPS coordinates of markers A and B respectively: (lon) A ,lat A (lon) B ,lat B ), and the pixel coordinates of A and B in the image: (x A ,y A ), (x B ,y B ), calculate the distance calibration value per unit pixel, and the latitude and longitude change corresponding to each unit pixel:

[0066]

[0067]

[0068] Dis lon =R×arccos(cos(lat) A cos(lat) A )cos(lon B -lon A )+sin(lat A sin(lat) A ))

[0069] Dis lat =R×arccos(sin(lat) A sin(lat) B ))

[0070]

[0071]

[0072] Among them, P lon The longitude calibration value per unit pixel, P lat The latitude calibration value per unit pixel, in m / pixel, Δ lon Δ represents the change in longitude per unit pixel. lat R is the latitude change per unit pixel; R is the Earth's radius; (x A ,y A ), (x B ,y B () represents the pixel coordinates of A and B in the image, where x represents the width direction and y represents the length direction. A ,lat A (lon) B ,lat B ) represents the GPS coordinates of A and B, where lon represents longitude and lat represents latitude;

[0073] Step 2. Set the planned flight path for the drone;

[0074] The drone's flight altitude is set to H; the flight path adopts a standard surveyed flight path: it consists of multiple channels, each with the same distance. It first flies from north to south along channel 1 from the starting point. After reaching the designated mileage hall, it turns and flies to the next channel. Channels with odd-numbered indices run from north to south, while those with even-numbered indices run from south to north. The interval between adjacent channels is [interval]. lon For, as attached Figure 2 As shown, S and E represent the starting point and ending point, respectively, and W... all For the width and height of the work area all The interval is the length of the work area (in meters). lon This represents the interval between adjacent flight paths; since the image resolution of the drone camera is 1080×1920 (length×width), an interval is set to ensure seamless stitching. lon for:

[0075]

[0076] Where width is the width of the image captured by the drone, i.e., 1920;

[0077] Step 3. Collect flight data;

[0078] Because the working area of ​​a drone is relatively large, its field of vision is limited at a certain altitude; therefore, it is necessary for the drone to fly over the working area according to its flight path, and then select key frames from the acquired video to construct a map of the working area.

[0079] Adjust the drone camera lens to face downwards and lock the gimbal to ensure that the optical center of the drone lens is at the drone's current position, that is, the drone is located at the center of each video frame; set the timestamp alignment of the drone's GPS data transmission, yaw angle data transmission, and image transmission to enable the drone to complete the flight according to the planned trajectory and acquire flight video of the work area; since the drone is located at the center of each video frame, the drone's GPS information is equivalent to the GPS information of the center point of each video frame in the flight video, that is, each video frame contains the center point GPS information and yaw angle information;

[0080] Step 4. Construct a work area map, such as... Figure 1 As shown, the specific process is as follows;

[0081] Step 4-1. Obtain all geographic information keyframes for the starting channel;

[0082] Step 4-1-1. Obtain the initial state geographic information keyframes of the starting channel;

[0083] The drone will reach the starting point of the planned flight path. Figure 2 The time of point S in the middle is used as the starting time to obtain the starting video frame, and the geographical location information of the starting video frame is calculated. The starting video frame containing the geographical location information is used as the starting state geographical information key frame, and the channel index is marked as 1.

[0084] The calculation process for geographic location information is as follows:

[0085] Obtain GPS information of the center point of the video frame: (lon) centre ,lat centre The initial video frame is rotated to true north based on the yaw angle information (to ensure the video frame meets satellite image coordinate requirements); after obtaining the GPS information of the center point, the remaining positions in the video frame are calculated using the center point as a reference point.

[0086] lon other =lon centre +diff y ×Δ lon

[0087] lat other =lat centre +diff x ×Δ lat

[0088] Among them, (lon) other ,lat other () represents the GPS information of the current pixel in the video frame, diff x diff y These represent the pixel differences between the current pixel and the center point in the video frame in the width (x) and length (y) directions, respectively.

[0089] Based on this, the GPS information of each pixel in the video frame is calculated, thus obtaining the keyframe containing the geographic location information; the geographic location information is saved in a fixed format in a .txt file with the same name, resulting in a small area map;

[0090] Step 4-1-2. Obtain keyframes of flight status geographic information for the starting flight path;

[0091] For each video frame after the initial frame in the flight video, calculate the current flight distance (Distance) of the drone:

[0092] Distance = R × arccos[cos(lat) p cos(lat) n )cos(lon p -lon n )+sin(lat p sin(lat) n )]

[0093] Among them, (lon) p ,lat p (lon) represents the GPS information of the center point of the previous geographic keyframe. n ,lat n ) represents the GPS information of the center point of the current video frame, and R is the Earth's radius;

[0094] To ensure seamless stitching, the two images maintain a 50% overlap along the longitude direction; therefore, the distance between the center points of the two keyframes should be interval. lat :

[0095]

[0096] Where length is the length of the image captured by the drone, i.e., 1080;

[0097] Therefore, set the distance interval threshold interval. lat If the flight distance Distance equals the distance interval threshold interval lat If the current video frame is determined to be a geographic information keyframe, the geographic location information of the current video frame is calculated to obtain the flight status geographic information keyframe.

[0098] Determine whether the turning point has been reached based on the yaw angle information. If not, mark the channel index as the same as the previous geographic information keyframe; if yes, proceed to step 4-2.

[0099] Step 4-2. Obtain all geographic information keyframes for the remaining waterways;

[0100] Step 4-2-1. Obtain the initial state geographic information keyframes of the new waterway;

[0101] Because the starting position of the new channel is opposite to the course angle of the previous channel, to ensure seamless stitching, the latitude of the starting position of the new channel should be the same as the latitude of the ending position of the previous channel. That is, the latitudes are the same when the distance between the two points is equal to the channel interval. Therefore, based on the yaw angle information and GPS information, it is determined whether the coordinates of the center point of the current video frame simultaneously meet the following conditions:

[0102] |now yaw +pre yaw |=π

[0103] R×arccos[cos(lat p cos(lat) n )cos(lon p -lon n )+sin(lat p sin(lat) n )] = interval lon

[0104] Among them, now yaw This indicates the yaw angle information of the current video frame, pre yaw This indicates the yaw angle information of the previous geographic keyframe;

[0105] If so, the current video frame is used as the starting video frame of the new channel; the geographical location information of the starting video frame is calculated, the starting video frame containing the geographical location information is used as the starting state geographical information key frame, and the channel index is marked as the channel index value of the previous geographical information key frame plus 1.

[0106] Step 4-2-2. Obtain keyframes of the flight status geographic information of the new airway;

[0107] For each video frame following the initial video frame of the current flight path, calculate the current flight distance Distance of the UAV. If the flight distance Distance equals the distance interval threshold interval... lat If the current video frame is determined to be a geographic information keyframe, the geographic location information of the current video frame is calculated to obtain the flight status geographic information keyframe.

[0108] Determine whether the turning point has been reached based on the yaw angle information. If not, mark the channel index as the same as the previous geographic information keyframe; if yes, proceed to step 4-2-1.

[0109] Step 4-3. Concatenate all geographic information keyframes sequentially according to their timestamps and channel indices to obtain the working area map; during the concatenation process, the concatenation direction of the geographic information keyframes in each channel should be consistent with the flight direction, such as... Figure 3 As shown.

[0110] The above description is merely a specific embodiment of the present invention. Any feature disclosed in this specification may be replaced by other equivalent or similar features unless otherwise specified. All disclosed features, or steps in all methods or processes, may be combined in any way except for mutually exclusive features and / or steps.

Claims

1. A method for constructing a UAV work area map based on aerial imagery, characterized in that, Includes the following steps: Step 1. Calculate the distance calibration value and latitude / longitude change per unit pixel in the image captured by the UAV at a fixed altitude. Fly the drone to a preset altitude H and capture an image containing markers A and B. Then, based on the GPS coordinates of markers A and B: (lon A ,lat A (lon) B ,lat B ) and the pixel coordinates of A and B in the image: (x A ,y A ), (x B ,y B Calculate the distance calibration value per unit pixel and the corresponding latitude and longitude change per unit pixel: early lon =R×arccos(cos(lat A )basket(lat A )co(lon B -lon A )+sin(lat A )sin(lat A )) Dis lat =R×arccos(sin(lat A )sin(lat B )) Among them, P lon The longitude calibration value per unit pixel, P lat Δ is the latitude calibration value per unit pixel. lon Δ represents the change in longitude per unit pixel. lat R represents the latitude change per unit pixel; R is the Earth's radius. Step 2. Set the planned flight path for the drone; Set the UAV's flight altitude to H, the flight path to a standard survey flight path, and the interval between adjacent airways. lon for: Where width is the width of the image captured by the drone; Step 3. Collect flight data; Adjust the drone's camera lens to look downwards and lock the gimbal; align the timestamps for the drone's GPS data transmission, yaw angle data transmission, and image transmission to enable the drone to complete the flight along the planned path and collect flight videos of the work area; each video frame in the flight video contains a timestamp, GPS information of the center point, and yaw angle information; Step 4. Create a work area map; Step 4-1. Obtain all geographic information keyframes for the starting channel; Step 4-1-1. Obtain the initial state geographic information keyframes of the starting channel; The time when the UAV arrives at the starting point of the planned flight path is taken as the starting time to obtain the starting video frame. The geographical location information of the starting video frame is calculated. The starting video frame containing the geographical location information is taken as the starting state geographical information key frame and the flight path index is marked as 1. Step 4-1-2. Obtain keyframes of flight status geographic information for the starting flight path; For each video frame after the initial frame in the flight video, calculate the current flight distance (Distance) of the drone: Distance=R×arccos[cos(lat p )basket(lat n )co(lon p -lon n )+sin(lat p )sin(lat n )] Among them, (lon) p ,lat p (lon) represents the GPS information of the center point of the previous geographic keyframe. n ,lat n This refers to the GPS information of the center point of the current video frame. Set the distance interval threshold. lat : Where length is the length of the image captured by the drone; If the flight distance Distance equals the distance interval threshold interval lat If the current video frame is determined to be a geographic information keyframe, the geographic location information of the current video frame is calculated to obtain the flight status geographic information keyframe. Based on the yaw angle information, determine whether the turning point has been reached. If not, mark the channel index as the same as the previous geographic information keyframe; if so, proceed to step 4-2. Step 4-2. Obtain all geographic information keyframes for the remaining waterways; Step 4-2-1. Obtain the keyframes of the initial state geographic information of the new waterway; Based on the yaw angle information and GPS information, determine whether the coordinates of the center point of the current video frame simultaneously meet the following conditions: |now yaw +pre yaw |=π R×arccos[cos(lat p )basket(lat n )co(lon p -lon n )+sin(lat p )sin(lat n )]=interval lon Among them, now yaw This indicates the yaw angle information of the current video frame, pre yaw This indicates the yaw angle information of the previous geographic keyframe; If so, the current video frame is used as the starting video frame of the new channel; the geographical location information of the starting video frame is calculated, the starting video frame containing the geographical location information is used as the starting state geographical information key frame, and the channel index is marked as the channel index value of the previous geographical information key frame plus 1. Step 4-2-2. Obtain keyframes of the flight status geographic information of the new airway; For each video frame following the initial video frame of the current flight path, calculate the current flight distance Distance of the UAV. If the flight distance Distance equals the distance interval threshold interval... lat If the current video frame is determined to be a geographic information keyframe, the geographic location information of the current video frame is calculated to obtain the flight status geographic information keyframe. Determine whether the turning point has been reached based on the yaw angle information. If not, mark the channel index as the same as the previous geographic information keyframe; if yes, proceed to step 4-2-1. Step 4-3. Concatenate all geographic information keyframes sequentially according to timestamp and channel index to obtain the working area map.

2. The method for constructing a UAV work area map based on aerial imagery as described in claim 1, characterized in that, The calculation process for geographic location information is as follows: Obtain GPS information of the center point of the video frame: (lon) centre ,lat centre The starting video frame is rotated to true north based on the yaw angle information, and the GPS information of the remaining pixels in the video frame is calculated. lon other =lon centre +diff y ×Δ lon years other =years centre +diff x ×Δ lat Among them, (lon) other ,lat other () represents the GPS information of the current pixel in the video frame, diff x diff y These represent the pixel differences between the current pixel and the center point in the video frame in the width and length directions, respectively. The GPS information of each pixel in the video frame is calculated to form the geographic location information of the video frame.

Images