Space measurement method based on real-time video of unmanned aerial vehicle

Abstract

The invention discloses a space measurement method based on an unmanned aerial vehicle real-time video, and the method comprises the steps: obtaining the parameters of an unmanned aerial vehicle in real time, including the pixel coordinates of a target point in a video image, the image size, the size of a camera sensor, the focal length, the zoom multiple, the attitude angle of an unmanned aerial vehicle camera, and the position and height of the unmanned aerial vehicle; defining a plurality of coordinate systems, including a pixel coordinate system, a camera coordinate system, an airframe north-east-earth coordinate system, an airframe north-east-sky coordinate system, an earth-centered earth-fixed coordinate system and a WGS84 geographic coordinate system; performing coordinate conversion calculation among a plurality of coordinate systems based on the unmanned aerial vehicle parameters; calculating geographic coordinates of all target points in the recognition area according to the step 3; and according to the geographic coordinates of each target point, constructing a geometric object representing the recognition region, and calculating the area of the geometric object as the area of the recognition region. The method can be suitable for real-time and rapid calculation of the target position and area in an emergency scene.

Description

Technical Field

[0001] This invention relates to the field of UAV image processing technology, and in particular to a spatial measurement method based on real-time video from a UAV. Background Technology

[0002] With the development of drone technology, drones are widely used in aerial photography, surveying, and AI-powered inspection. These applications often require converting target point locations into actual geographic coordinates and calculating the area of ​​the identified region. However, due to the attitude changes (pitch, roll, yaw) of drones during filming, as well as the influence of factors such as camera focal length, sensor size, and terrain undulations, and because this conversion process involves complex coordinate transformations, current technologies primarily focus on post-processing image calculations and lack a real-time, fast method or system for calculating the area of ​​a target in drone video. Summary of the Invention

[0003] In view of this, the purpose of this invention is to propose a spatial measurement method based on real-time video from a drone, which uses drone camera parameters and real-time pose information to calculate the geographic coordinates of a target point and the area of ​​the target graphic, and can be applied to the real-time and rapid calculation of target location and area in emergency scenarios.

[0004] To achieve the above-mentioned technical objectives, the technical solution adopted by this invention is as follows: This invention provides a spatial measurement method based on real-time video from an unmanned aerial vehicle (UAV), comprising the following steps: Step 1: Acquire drone parameters in real time, including the pixel coordinates of the target point in the video image, image size, camera sensor size, focal length, zoom factor, drone camera attitude angle, drone position and altitude; Step 2: Define multiple coordinate systems, including pixel coordinate system, camera coordinate system, body northeast-ground coordinate system, body northeast-sky coordinate system, geocentric-ground-fixed coordinate system and WGS84 geographic coordinate system; Step 3: Perform coordinate transformation calculations between multiple coordinate systems based on the UAV parameters, specifically including: Step 31: Based on pixel coordinates, image size, camera sensor size, focal length, and zoom factor, convert the pixel coordinates of the target point in the pixel coordinate system to camera coordinates in the camera coordinate system; Step 32: Based on the camera coordinates and the attitude angle of the UAV camera, the camera coordinates are transformed to northeast-northeast coordinates under the northeast-northeast coordinate system of the aircraft through a preset rotation matrix; Step 33: Based on the UAV's position and altitude, convert the northeast-ground coordinates to northeast-sky coordinates in the UAV's northeast-sky coordinate system; Step 34: Based on the northeast-sky coordinates, the UAV's position and altitude, convert the northeast-sky coordinates to geocentric-ground-fixed coordinates in the geocentric-ground-fixed coordinate system; Step 35: Convert the geocentric coordinates to geographic coordinates in the WGS84 geographic coordinate system; Step 4: Calculate the geographic coordinates of all target points in the identified area based on Step 3; Step 5: Based on the geographic coordinates of each target point, construct a geometric object representing the identification area, and calculate the area of ​​the geometric object as the area of ​​the identification area.

[0005] Furthermore, the camera sensor size refers to the width and height of the effective imaging area of ​​the photosensitive element used for camera imaging; The zoom ratio refers to the ratio of the current focal length of the camera lens to the reference focal length; The attitude angles include the yaw angle with due north as 0 degrees and due east as 90 degrees, the pitch angle with horizontal forward as 0 degrees and vertical downward as -90 degrees, and the roll angle with rightward roll as positive and leftward roll as negative relative to the direction of fuselage travel. The location of the drone is its longitude and latitude; The altitude includes both the absolute altitude and the drone's altitude relative to the ground.

[0006] Furthermore, step 2 specifically includes: Step 21: Establish a pixel coordinate system. Take the top left pixel of the image plane captured by the drone as the origin, define the horizontal direction to the right as the u-axis, and the vertical direction downward as the v-axis. The coordinate values ​​are integer pixels, which are used to mark the position of each pixel of the target point on the image plane. Step 22: Establish a camera coordinate system with the intersection of the UAV camera's optical axis and the image plane as the origin, which is located at the center of the image plane; define the horizontal direction to the right within the image plane as the x-axis, the vertical downward direction as the y-axis, and the direction perpendicular to the image plane and pointing towards the optical axis as the z-axis; the coordinate unit of the camera coordinate system is a physical length unit; Step 23: Establish a coordinate system for the aircraft, with the UAV's position as the origin. Define the direction pointing due north as the X-axis, the direction pointing due east as the Y-axis, and the direction perpendicular to the ground as the Z-axis. Step 24: Establish the aircraft's north-east-sky coordinate system, with the UAV's position as the origin, define the direction pointing due north as the X1 axis, the direction pointing due east as the Y1 axis, and the vertical upward direction as the Z1 axis; Step 25: Establish a geocentric coordinate system. With the Earth's center as the origin, define the direction parallel to the Earth's rotation axis and pointing towards the North Pole as the Z2 axis, the direction from the origin to the intersection of the Prime Meridian and the equator as the X2 axis, and the direction perpendicular to the Prime Meridian plane and pointing towards the intersection of 90 degrees east longitude and the equator as the Y2 axis. Step 26: Establish the WGS84 geographic coordinate system, with the Earth's center of mass as the origin, and with longitude, latitude, and altitude relative to the WGS84 reference ellipsoid. Furthermore, step 31 specifically includes: Step 311: Determine the pixel coordinates of the image center point in the pixel coordinate system. , The coordinates of the image center point on the u-axis Image width in image size Half of the coordinates on the v-axis Image height in image dimensions Half of it, the formula is as follows:

[0007]

[0008] Step 312: Calculate the physical size of the target point per pixel, including the width component of the physical size. Width in the camera sensor size Divide by image width The height component of the physical dimension Height in the camera sensor size Divide by image height The formula is as follows:

[0009]

[0010] Step 313: Subtract the coordinates of the image center point on the u-axis from the pixel coordinates (u, v) of the target point on the u-axis. Then multiply by the width component of the physical dimensions. This yields the x-axis coordinates in the camera coordinate system; the v-axis coordinates of the target point are then subtracted from the v-axis coordinates of the image center point. Then multiply by the height component of the physical dimensions. Obtain the coordinates on the y-axis in the camera coordinate system; then compare the focal length f with the zoom factor. Multiplying them gives the coordinate value on the z-axis in the camera coordinate system, as shown in the following formula:

[0011] Where x, y, and z are the camera coordinates in the camera coordinate system, respectively.

[0012] Furthermore, step 32 specifically includes: The rotation matrix R is an Euler angle rotation matrix. It is used to transform the camera coordinates in the camera coordinate system to the northeast-northeast coordinates in the aircraft's northeast-northeast coordinate system. The calculation formula is as follows:

[0013] In the formula, X, Y, and Z are the northeast coordinates of the aircraft in the northeast-east coordinate system, respectively; The construction of the rotation matrix R depends on the sine and cosine values ​​of the attitude angles after converting them from degrees to radians, and is defined as follows:

[0014]

[0015]

[0016]

[0017] In the formula, This is the radian value corresponding to the yaw angle. This is the pitch angle in radians. This is the radian value corresponding to the roll angle.

[0018] Furthermore, step 33 specifically includes: Step 331: Based on the aircraft's northeast-central coordinates, calculate the direction vector from the UAV's position to the ground point corresponding to the target point in the image, within the aircraft's northeast-central coordinate system.

[0019]

[0020]

[0021] in, This represents the direction vector on the X-axis in the northeast-northeast coordinate system of the machine. This represents the direction vector along the Y-axis in the northeast-northeast coordinate system of the machine. This represents the direction vector on the Y-axis in the northeast-northeast coordinate system of the aircraft. Step 332: Construct a ray originating from the UAV's position with a direction vector of (Xneu, Yneu, Zneu); Step 333: Check the direction vector ,like If Zneu < 0, then the camera lens is determined to be facing downwards, and step 334 is performed for calculation; if Zneu ≥ 0, then the camera lens is determined not to be facing downwards, and no calculation is performed. Step 334: Based on the drone's altitude agl relative to the ground, calculate the intersection parameter t of the ray and the ground plane. The calculation formula is as follows:

[0022] Step 335: Based on the intersection parameter t, calculate the northeast-east coordinates (X1, Y1, Z1) of the ground point in the aircraft's northeast-east coordinate system:

[0023]

[0024]

[0025] Where X1 is the north coordinate of the target point in the northeast-northeast coordinate system of the aircraft, Y1 is the east coordinate of the target point in the northeast-northeast coordinate system of the aircraft, and Z1 is the vertical coordinate of the target point in the northeast-northeast coordinate system of the aircraft.

[0026] Furthermore, in step 34, the northeast-sky coordinates are converted to geocentric-geocentric coordinates in the geocentric-geocentric coordinate system. The specific conversion method is as follows: Using the geographic coordinate transformation function, input the N-E-Celestial coordinates (X1, Y1, Z1), the latitude, longitude, and absolute altitude of the UAV's location, and output the geocentric coordinates (X2, Y2, Z2).

[0027] Furthermore, in step 35, the geocentric coordinates are converted to geographic coordinates in the WGS84 geographic coordinate system. The specific conversion method is as follows: Using the coordinate transformation library, the geocentric coordinates (X2, Y2, Z2) in the geocentric-fixed coordinate system are converted to longitude L, latitude B, and altitude H in the WGS84 geographic coordinate system. Longitude L, latitude B, and altitude H are used to represent the actual geographical location of the target point on Earth.

[0028] Furthermore, in step 4, the number of target points in the identified area is at least three, and when calculating the area, all target points are assumed to be at the same elevation level.

[0029] Furthermore, step 5 specifically includes: Step 51: According to the order of the target points in the identification area, concatenate the longitude and latitude values ​​of each target point in sequence to form a coordinate sequence text with longitude and latitude separated by spaces. Place this coordinate sequence text in a geometric description prefix and parentheses indicating the polygon type, and make the first and last coordinates of the coordinate sequence text the same, thus forming a WKT string. Step 52: Use the function of parsing the WKT string in the geometry calculation library to load the WKT string as a geometric object; Step 53: Call the area calculation method of the geometric object to obtain the area value of the recognition area it represents.

[0030] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art: 1. The spatial measurement method based on real-time video of UAV provided by the present invention can quickly calculate the geographical location and area of ​​the target in UAV inspection by using the real-time location information of the UAV, and has high real-time performance. 2. The ground point location calculation method in the spatial measurement method based on UAV real-time video provided by the present invention uses camera pose parameters and sensor parameters to obtain geographic coordinates through the transformation between multiple coordinate systems, which simplifies the input parameters and calculation process. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is an execution flowchart of a spatial measurement method based on real-time video from a UAV provided in an embodiment of the present invention.

[0033] Figure 2 This is a top view of the pixel coordinate system and camera coordinate system provided in an embodiment of the present invention.

[0034] Figure 3 This is a schematic diagram showing the relationship between the camera coordinate system, the aircraft's northeast-earth coordinate system, and the aircraft's northeast-sky coordinate system provided in an embodiment of the present invention. Detailed Implementation

[0035] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be particularly noted that the following embodiments are for illustrative purposes only and do not limit the scope of the invention. Similarly, the following embodiments are only some, not all, embodiments of the present invention, and all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] Please see Figure 1 This invention discloses a spatial measurement method based on real-time video from unmanned aerial vehicles (UAVs), applicable to UAV inspection, mapping, or emergency response scenarios, used to calculate the geographical location and outline area of ​​a specified target in the video image in real time; it includes the following steps: Step 1: Acquire drone parameters in real time. These parameters include the pixel coordinates (u, v) of the target point in the video image (in pixels), image size (image width x, image height y, in mm), camera sensor size (width sx, height sy, in mm), focal length (f, in mm), zoom factor (zoom factor, in integer multiples), drone camera attitude angles (including yaw angle heading, pitch angle, and roll angle), and drone position and altitude. These drone parameters are real-time information obtained from the drone. Taking the DJI M4TD aircraft mounted on the DJI Airport 3 as an example, real-time pose information can be obtained through cloud API. In this embodiment, the camera sensor size refers to the width and height of the effective imaging area of ​​the photosensitive element used for camera imaging, in millimeters; The zoom ratio refers to the ratio of the current focal length of the camera lens to the reference focal length; The attitude angles include the yaw angle with due north as 0 degrees and due east as 90 degrees, the pitch angle with horizontal forward as 0 degrees and vertical downward as -90 degrees, and the roll angle with rightward roll as positive and leftward roll as negative relative to the direction of fuselage travel. The location of the drone is its longitude (lon, in degrees) and latitude (lat, in degrees). The altitude includes absolute altitude (height in meters) and the drone's altitude relative to the ground (agl in meters).

[0037] Step 2: Define multiple coordinate systems, including the pixel coordinate system, camera coordinate system, aircraft N-E-T-Ground coordinate system, aircraft N-E-T-Sky coordinate system, Earth-Centered Earth-Fixed coordinate system, and WGS84 geographic coordinate system (WGS84 stands for World Geodetic System 1984). Define the various transformation coordinate systems required for calculation. The purpose is to convert the coordinates of each point (pixel coordinates) of the target on the image into the coordinates of points on the ground (i.e., WGS84 latitude and longitude coordinates). Steps 21, 22, and 23 are the position coordinates of the target on the image, while steps 24, 25, and 26 are the position coordinates of the target on the ground.

[0038] In this embodiment, step 2 specifically includes: Step 21: Establish a pixel coordinate system (u, v), with the top left pixel of the image plane captured by the drone as the origin, define the horizontal direction to the right as the u-axis, and the vertical direction downward as the v-axis, with coordinate values ​​being integer pixels (unit is pixels), used to mark the position of each pixel of the target point on the image plane; Step 22: Establish a camera coordinate system (x, y, z), with the intersection of the UAV camera's optical axis and the image plane as the origin, which is located at the center of the image plane; define the horizontal direction to the right within the image plane as the x-axis, the vertical downward direction as the y-axis, and the direction perpendicular to the image plane and pointing towards the optical axis as the z-axis; the coordinate unit of the camera coordinate system is the physical length unit (mm). Since the pixel coordinate system is based on pixels, we need to obtain the physical length unit. Therefore, we need to transform the pixel coordinate system to the camera coordinate system. At the same time, we know that once the focal length f and the zoom factor zoomFactor are determined, the position of the target point in the image plane on the z-axis in the camera coordinate system is constant as f·zoomFactor. The relationship between the pixel coordinate system and the camera coordinate system can be found in [reference]. Figure 2 . Figure 2 This is a top view of the pixel coordinate system and the camera coordinate system. imageX is the image width, imageY is the image height, Ouv is the origin of the pixel coordinate system, Oxyz is the origin of the camera coordinate system, and the position of Oxyz is the center of the image plane.

[0039] Step 23: Establish the Nodal-East-Ground (NED) coordinate system (X,Y,Z) with the UAV's position as the origin, define the direction pointing due north as the X-axis, the direction pointing due east as the Y-axis, and the direction perpendicular to the ground as the Z-axis; The purpose of establishing the northeast-northeast coordinate system of the machine is to obtain the position of the target point in the camera coordinate system. To obtain the geographical location of the target point, it is necessary to transform it to a certain geographical coordinate system. Therefore, it is necessary to construct the corresponding coordinate system.

[0040] Step 24: Establish the NEU coordinate system (X1, Y1, Z1) for the aircraft. With the UAV's position as the origin, define the direction pointing due north as the X1 axis, the direction pointing due east as the Y1 axis, and the vertical upward direction as the Z1 axis. To maintain consistency with the conventional geographic coordinate system, the aircraft's northeast-to-ground coordinate system needs to be converted to the aircraft's northeast-to-sky coordinate system. This step converts the target points in the image to ground points. The target points in the image and the points on the ground are in a perspective projection relationship. Due to the terrain undulations, there are calculation errors in the target point position coordinates obtained here, and the more obvious the terrain undulations, the greater the error. The relationship between the camera coordinate system, the aircraft's northeast-to-ground coordinate system, and the aircraft's northeast-to-sky coordinate system can be found in the following reference. Figure 3The relationship between the aircraft's northeast-ground coordinate system and the aircraft's northeast-sky coordinate system is as follows: the origin (OXYZ) of the aircraft's northeast-ground coordinate system is the same as the origin (OX1Y1Z1) of the aircraft's northeast-sky coordinate system, both being the aircraft's position. The X-axis and X1 axis both point due north, and the Y-axis and Y1 axis both point due east. However, the Z-axis of the aircraft's northeast-ground coordinate system points vertically downward, and the Z1 axis of the aircraft's northeast-sky coordinate system points vertically upward. A point p on the camera coordinate system and its corresponding ground point p2 have a perspective projection relationship. Step 25: Establish the geocentric-geofixed (ECEF) coordinate system (X2, Y2, Z2). With the Earth's center as the origin, define the direction parallel to the Earth's rotation axis and pointing towards the North Pole as the Z2 axis, the direction from the origin to the intersection of the Prime Meridian and the equator as the X2 axis, and the direction perpendicular to the Prime Meridian plane and pointing towards the intersection of 90 degrees east longitude and the equator as the Y2 axis. This step simplifies the calculation process by using Python's geospatial library to convert NEU coordinates to ECEF coordinates, and then converts the ECEF coordinate system to WGS84 coordinates.

[0041] Step 26: Establish the WGS84 geographic coordinate system (L, B, H), which is an internationally adopted geocentric coordinate system with the Earth's center of mass as the origin and has longitude L, latitude B, and height H relative to the WGS84 reference ellipsoid. Step 3: Based on the UAV parameters, perform coordinate transformation calculations between multiple coordinate systems (from the pixel coordinate system to the camera coordinate system, the aircraft's northeast-to-ground coordinate system, the aircraft's northeast-to-sky coordinate system, the geocentric-ground-fixed coordinate system, and the WGS84 geographic coordinate system). Specifically, this includes: Step 31: Based on pixel coordinates (u, v), image size (image width imageX, image height imageY), camera sensor size (width sx, height sy), focal length (f), and zoom factor (… ), converting the pixel coordinates (u, v) of the target point in the pixel coordinate system into camera coordinates (x, y, z) in the camera coordinate system; In this embodiment, step 31 specifically includes: Step 311: Determine the pixel coordinates of the image center point in the pixel coordinate system. , The coordinates of the image center point on the u-axis Image width in image size Half of the coordinates on the v-axis Image height in image dimensions Half of it, the formula is as follows:

[0042]

[0043] Step 312: Calculate the physical size of the target point per pixel, including the width component of the physical size. Width in the camera sensor size Divide by image width The height component of the physical dimension Height in the camera sensor size Divide by image height The formula is as follows:

[0044]

[0045] Step 313: Subtract the coordinates of the image center point on the u-axis from the pixel coordinates (u, v) of the target point on the u-axis. Then multiply by the width component of the physical dimensions. This yields the x-axis coordinates in the camera coordinate system; the v-axis coordinates of the target point are then subtracted from the v-axis coordinates of the image center point. Then multiply by the height component of the physical dimensions. Obtain the coordinates on the y-axis in the camera coordinate system; then compare the focal length f with the zoom factor. Multiplying them gives the coordinate value on the z-axis in the camera coordinate system, as shown in the following formula:

[0046] Where x, y, and z are the camera coordinates in the camera coordinate system, respectively.

[0047] Step 32: Based on the camera coordinates and the attitude angles (yaw angle, pitch angle, and roll angle) of the UAV camera, transform the camera coordinates (x, y, z) to the northeast-northeast coordinates (X, Y, Z) of the aircraft in the northeast-northeast coordinate system using a preset rotation matrix. In this embodiment, step 32 specifically includes: The rotation matrix R is an Euler angle rotation matrix (ZYX order: yaw → pitch → roll). The rotation matrix R is used to transform the camera coordinates in the camera coordinate system to the northeast-northeast coordinate system of the aircraft. The calculation formula is as follows:

[0048] In the formula, X, Y, and Z are the northeast coordinates of the aircraft in the northeast-east coordinate system, respectively; The construction of the rotation matrix R depends on the sine and cosine values ​​of the attitude angles after converting them from degrees to radians, and is defined as follows:

[0049]

[0050]

[0051]

[0052] In the formula, This is the radian value corresponding to the yaw angle. This is the pitch angle in radians. This is the radian value corresponding to the roll angle.

[0053] Step 33: Based on the UAV's position and its altitude relative to the ground, convert the northeast ground coordinates (X, Y, Z) to northeast sky coordinates (X1, Y1, Z1) in the UAV's northeast sky coordinate system. In this embodiment, step 33 specifically includes: Step 331: Based on the aircraft's northeast-central coordinates, calculate the direction vector from the UAV's position to the ground point corresponding to the target point in the image, within the aircraft's northeast-central coordinate system.

[0054]

[0055]

[0056] in, This represents the direction vector on the X-axis in the northeast-northeast coordinate system of the machine. This represents the direction vector along the Y-axis in the northeast-northeast coordinate system of the machine. This represents the direction vector on the Y-axis in the northeast-northeast coordinate system of the aircraft. Step 332: Construct a ray originating from the UAV's position (the origin of the UAV's northeast-central coordinate system) with a direction vector of (Xneu, Yneu, Zneu); Step 333: Check the direction vector ,like If Zneu < 0, then the camera lens is determined to be facing downwards, and step 334 is performed for calculation; if Zneu ≥ 0, then the camera lens is determined not to be facing downwards, and no calculation is performed. Step 334: Based on the drone's altitude agl relative to the ground, calculate the intersection parameter t of the ray and the ground plane. The calculation formula is as follows:

[0057] Step 335: Based on the intersection parameter t, calculate the northeast-east coordinates (X1, Y1, Z1) of the ground point in the aircraft's northeast-east coordinate system:

[0058]

[0059]

[0060] Where X1 is the north coordinate of the target point in the northeast-east coordinate system of the aircraft, Y1 is the east coordinate of the target point in the northeast-east coordinate system of the aircraft, and Z1 is the vertical coordinate of the target point in the northeast-east coordinate system of the aircraft. The calculation results are more accurate only when the terrain is flat or has minimal undulation. This method assumes the ground is flat (at the same height as the drone's location). If the terrain is undulating, the actual ground height will not match the assumed plane (Z=-agl), leading to errors. The greater the terrain undulation, the greater the error; in mountainous areas, this error may reach tens of meters or even more. In the drone's north-east-sky coordinate system, the drone's position is at the origin (0,0,0), with an absolute height of height. The height of the ground point is height - agl (because the drone's height relative to the ground is agl, the ground point's height is agl lower than the drone's). In the drone's north-east-sky coordinate system, the Z-coordinate (upward direction) of the ground point is the ground point's height minus the drone's height, i.e., (height - agl) - height = -agl. Therefore, we require points on the ray to satisfy Z-coordinate = -agl. The ray parameter equation is: P(t) = (Xneu * t, Yneu * t, Zneu * t). Let the Z-coordinate be equal to -agl.

[0061] Note: When the camera is facing downwards, Zneu is negative, so t is positive, and the ray intersects the ground; When the camera is facing upwards, Zneu is positive, and t is negative, indicating that the ray will not intersect the ground plane in the opposite direction (upwards) (because the ground is below). Therefore, the ray will only intersect the ground plane when Zneu < 0 (camera facing downwards) and t > 0. Camera orientation requirement: Ground points can only be calculated when the camera is facing down (Zneu<0). When the camera is horizontal, Zneu=0, and t is undefined (division by zero error). Therefore, it is necessary to check whether Zneu is 0 (or close to 0) and avoid calculation. This method is suitable for scenarios where the drone camera is facing down (such as orthophoto or oblique photography). Step 34: Based on the northeast-east coordinates (X1, Y1, Z1), the UAV's position (latitude lat, longitude lon), and absolute altitude (height), convert the northeast-east coordinates (X1, Y1, Z1) to geocentric coordinates (X2, Y2, Z2) in the geocentric-fixed coordinate system; the specific conversion method is as follows: Using a geographic coordinate transformation function (such as pymap3d.enu2ecef), input the N-E-Celestial coordinates (X1, Y1, Z1), the latitude (lat), longitude (lon), and absolute height of the UAV's location, and output the geocentric coordinates (X2, Y2, Z2).

[0062] Taking pymap3d as an example, its conversion function is: .

[0063] Step 35: Convert the geocentric coordinates (X2, Y2, Z2) to geographic coordinates (L, B, H) in the WGS84 geographic coordinate system; the specific conversion method is as follows: Use a coordinate transformation library (such as pyproj) to convert the geocentric coordinates (X2, Y2, Z2) in the geocentric-fixed coordinate system (EPSG:4978) to the longitude L, latitude B, and altitude H in the WGS84 geographic coordinate system (EPSG:4979). The longitude L, latitude B, and altitude H are used to represent the actual geographical location of the target point on Earth.

[0064] transformer = Transformer.from_crs("EPSG:4978", "EPSG:4979", always_xy=True) L, B, H =transformer.transform(X2, Y2, Z2) Step 4: Calculate the geographic coordinates of all target points in the recognition area (the boundary range of the target object in the real-time video image of the UAV) based on Step 3; In this embodiment, the number of target points in the identified area in step 4 is at least three, and when calculating the area, all target points are assumed to be at the same elevation level. The two-dimensional geometric area is calculated only based on their latitude and longitude coordinates, ignoring the impact of possible elevation differences between target points on the area.

[0065] Step 5: Based on the geographical coordinates (latitude and longitude coordinates) of each target point, construct a geometric object representing the identification area, and calculate the area of ​​the geometric object as the area of ​​the identification area.

[0066] In this embodiment, step 5 specifically includes: Step 51: According to the order of the target points in the identification area, concatenate the longitude and latitude values ​​of each target point in sequence to form a coordinate sequence text with longitude and latitude separated by spaces. Place this coordinate sequence text in a geometric description prefix and parentheses indicating the polygon type, and make the first and last coordinates of the coordinate sequence text the same, thus forming a WKT string. Assuming the identification area has N target points, and the latitude and longitude coordinates of each target point are (L1, B1), (L2, B2), (L3, B3)...(LN, BN), a WKT string for the identification area is constructed based on the latitude and longitude coordinates of each target point. WKT (Well-known text) is a text markup language developed by the Open GIS Consortium (OGC) for representing vector geometric objects, spatial reference systems, and conversions between spatial reference systems. WKT can be divided into point, line, polygon, and geometric set types. This method constructs a polygon WKT, the specific form of which is as follows: POLYGON((119.207744 26.048477, 119.207860 26.047983, 119.20844026.048150, 119.208426 26.048879, 119.207744 26.048477)) In this WKT string, POLYGON is used as a prefix, and the coordinates of each target point are in the double parentheses. The first and last points have the same coordinates, indicating that the string is closed. Step 52: Use the function of parsing the WKT string in the geometry calculation library to load the WKT string as a geometric object; Step 53: Call the area calculation method of the geometric object to obtain the area value of the recognition area it represents.

[0067] Assuming the constructed WKT string is wktpolygon, you can use the wkt.loads method of the Python shapely library (a geometry calculation library): geom = wkt.loads(wktpolygon) This method can convert a WKT string into a geometric object, and then obtain the perimeter, area, etc. of the geometric object. The final calculated area result is geom.area. Because area error is affected by point location error, and there may be cases where the point location error direction is consistent, resulting in a small area error, but with a large deviation from the overall and actual location, making it difficult to evaluate, this method only tests the average point location error of the target point under flat terrain and some terrain undulation (elevation difference within 30m). The specific performance is as follows. The specific error calculation method is the distance difference between the actual point location and the calculated point location. Since there may also be errors in obtaining the actual point location, the error here is for reference only: Table 1. Error performance on flat terrain with a camera pitch angle of -90°

[0068] Table 2 shows the error performance on flat terrain with camera pitch angles ranging from -90° to -45°.

[0069] Table 3. Error performance on flat terrain with camera pitch angles ranging from -45° to -20°.

[0070] Table 4 shows the error performance under certain terrain undulations and a camera pitch angle of -90°.

[0071] Table 5 shows the error performance for a given terrain undulation and a camera pitch angle range of (-90° to -45°).

[0072] In most orthographic (camera vertically downward, pitch angle -90°) and oblique shooting situations, the camera pitch angle range is generally between -90° and -45°. Since the error calculation method is the difference between the estimated point and the actual point in a straight line, the horizontal and vertical offsets are both less than this value. In orthographic situations, the total average error calculated by this method (only the pitch angle is -90° and other conditions vary) is about 4.52m. In oblique situations (pitch angle range between -90° and -45°, other conditions vary), the total average error calculated by this method is about 15.92m. When the terrain is undulating and the pitch angle is large, the error of this method is larger.

[0073] The above description is only a part of the embodiments of the present invention and does not limit the scope of protection of the present invention. Any equivalent device or equivalent process transformation made based on the content of the present invention specification and drawings, or direct or indirect application in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A spatial measurement method based on real-time video from an unmanned aerial vehicle (UAV), characterized in that, Includes the following steps: Step 1: Acquire drone parameters in real time, including the pixel coordinates of the target point in the video image, image size, camera sensor size, focal length, zoom factor, drone camera attitude angle, drone position and altitude; Step 2: Define multiple coordinate systems, including pixel coordinate system, camera coordinate system, body northeast-ground coordinate system, body northeast-sky coordinate system, geocentric-ground-fixed coordinate system and WGS84 geographic coordinate system; Step 3: Perform coordinate transformation calculations between multiple coordinate systems based on the UAV parameters, specifically including: Step 31: Based on pixel coordinates, image size, camera sensor size, focal length, and zoom factor, convert the pixel coordinates of the target point in the pixel coordinate system to camera coordinates in the camera coordinate system; Step 32: Based on the camera coordinates and the attitude angle of the UAV camera, the camera coordinates are transformed to northeast-northeast coordinates under the northeast-northeast coordinate system of the aircraft through a preset rotation matrix; Step 33: Based on the UAV's position and altitude, convert the northeast-ground coordinates to northeast-sky coordinates in the UAV's northeast-sky coordinate system; Step 34: Based on the northeast-sky coordinates, the UAV's position and altitude, convert the northeast-sky coordinates to geocentric-ground-fixed coordinates in the geocentric-ground-fixed coordinate system; Step 35: Convert the geocentric coordinates to geographic coordinates in the WGS84 geographic coordinate system; Step 4: Calculate the geographic coordinates of all target points in the identified area based on Step 3; Step 5: Based on the geographic coordinates of each target point, construct a geometric object representing the identification area, and calculate the area of ​​the geometric object as the area of ​​the identification area.

2. The spatial measurement method based on real-time video from an unmanned aerial vehicle as described in claim 1, characterized in that, The camera sensor size refers to the width and height of the effective imaging area of ​​the photosensitive element used for camera imaging; The zoom ratio refers to the ratio of the current focal length of the camera lens to the reference focal length; The attitude angles include the yaw angle with due north as 0 degrees and due east as 90 degrees, the pitch angle with horizontal forward as 0 degrees and vertical downward as -90 degrees, and the roll angle with rightward roll as positive and leftward roll as negative relative to the direction of fuselage travel. The location of the drone is its longitude and latitude; The altitude includes both the absolute altitude and the drone's altitude relative to the ground.

3. The spatial measurement method based on real-time video from an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, Step 2 specifically includes: Step 21: Establish a pixel coordinate system. Take the top left pixel of the image plane captured by the drone as the origin, define the horizontal direction to the right as the u-axis, and the vertical direction downward as the v-axis. The coordinate values ​​are integer pixels, which are used to mark the position of each pixel of the target point on the image plane. Step 22: Establish a camera coordinate system with the intersection of the UAV camera's optical axis and the image plane as the origin, which is located at the center of the image plane; define the horizontal direction to the right within the image plane as the x-axis, the vertical downward direction as the y-axis, and the direction perpendicular to the image plane and pointing towards the optical axis as the z-axis; the coordinate unit of the camera coordinate system is a physical length unit; Step 23: Establish a coordinate system for the aircraft, with the UAV's position as the origin. Define the direction pointing due north as the X-axis, the direction pointing due east as the Y-axis, and the direction perpendicular to the ground as the Z-axis. Step 24: Establish the aircraft's north-east-sky coordinate system, with the UAV's position as the origin, define the direction pointing due north as the X1 axis, the direction pointing due east as the Y1 axis, and the vertical upward direction as the Z1 axis; Step 25: Establish a geocentric coordinate system. With the Earth's center as the origin, define the direction parallel to the Earth's rotation axis and pointing towards the North Pole as the Z2 axis, the direction from the origin to the intersection of the Prime Meridian and the equator as the X2 axis, and the direction perpendicular to the Prime Meridian plane and pointing towards the intersection of 90 degrees east longitude and the equator as the Y2 axis. Step 26: Establish a WGS84 geographic coordinate system with the Earth's center of mass as the origin, and with longitude, latitude, and altitude relative to the WGS84 reference ellipsoid.

4. The spatial measurement method based on real-time video from an unmanned aerial vehicle as described in claim 1, characterized in that, Step 31 specifically includes: Step 311: Determine the pixel coordinates of the image center point in the pixel coordinate system. , The coordinates of the image center point on the u-axis Image width in image size Half of the coordinates on the v-axis Image height in image dimensions Half of it, the formula is as follows: Step 312: Calculate the physical size of the target point per pixel, including the width component of the physical size. Width in the camera sensor size Divide by image width The height component of the physical dimension Height in the camera sensor size Divide by image height The formula is as follows: Step 313: Subtract the coordinates of the image center point on the u-axis from the pixel coordinates (u, v) of the target point on the u-axis. Then multiply by the width component of the physical dimensions. This yields the x-axis coordinates in the camera coordinate system; the v-axis coordinates of the target point are then subtracted from the v-axis coordinates of the image center point. Then multiply by the height component of the physical dimensions. Obtain the coordinates on the y-axis in the camera coordinate system; then compare the focal length f with the zoom factor. Multiplying them gives the coordinate value on the z-axis in the camera coordinate system, as shown in the following formula: Where x, y, and z are the camera coordinates in the camera coordinate system, respectively.

5. A spatial measurement method based on real-time video from an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, Step 32 specifically includes: The rotation matrix R is an Euler angle rotation matrix. It is used to transform the camera coordinates in the camera coordinate system to the northeast-northeast coordinates in the aircraft's northeast-northeast coordinate system. The calculation formula is as follows: In the formula, X, Y, and Z are the northeast coordinates of the aircraft in the northeast-east coordinate system, respectively; The construction of the rotation matrix R depends on the sine and cosine values ​​of the attitude angles after converting them from degrees to radians, and is defined as follows: In the formula, This is the radian value corresponding to the yaw angle. This is the pitch angle in radians. This is the radian value corresponding to the roll angle.

6. The spatial measurement method based on real-time video from an unmanned aerial vehicle as described in claim 1, characterized in that, Step 33 specifically includes: Step 331: Based on the aircraft's northeast-central coordinates, calculate the direction vector from the UAV's position to the ground point corresponding to the target point in the image, within the aircraft's northeast-central coordinate system. in, This represents the direction vector on the X-axis in the northeast-northeast coordinate system of the machine. This represents the direction vector along the Y-axis in the northeast-northeast coordinate system of the machine. This represents the direction vector on the Y-axis in the northeast-northeast coordinate system of the aircraft. Step 332: Construct a ray originating from the UAV's position with a direction vector of (Xneu, Yneu, Zneu); Step 333: Check the direction vector ,like If Zneu < 0, then the camera lens is determined to be facing downwards, and step 334 is performed for calculation; if Zneu ≥ 0, then the camera lens is determined not to be facing downwards, and no calculation is performed. Step 334: Based on the drone's altitude agl relative to the ground, calculate the intersection parameter t of the ray and the ground plane. The calculation formula is as follows: Step 335: Based on the intersection parameter t, calculate the northeast-east coordinates (X1, Y1, Z1) of the ground point in the aircraft's northeast-east coordinate system: Where X1 is the north coordinate of the target point in the northeast-northeast coordinate system of the aircraft, Y1 is the east coordinate of the target point in the northeast-northeast coordinate system of the aircraft, and Z1 is the vertical coordinate of the target point in the northeast-northeast coordinate system of the aircraft.

7. The spatial measurement method based on real-time video from an unmanned aerial vehicle as described in claim 1, characterized in that, In step 34, the northeast-sky coordinates are converted to geocentric-geocentric coordinates in the geocentric-geocentric coordinate system. The specific conversion method is as follows: Using the geographic coordinate transformation function, input the N-E-Celestial coordinates (X1, Y1, Z1), the latitude, longitude, and absolute altitude of the UAV's location, and output the geocentric coordinates (X2, Y2, Z2).

8. A spatial measurement method based on real-time video from an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, In step 35, the geocentric coordinates are converted to geographic coordinates in the WGS84 geographic coordinate system. The specific conversion method is as follows: Using the coordinate transformation library, the geocentric coordinates (X2, Y2, Z2) in the geocentric-fixed coordinate system are converted to longitude L, latitude B, and altitude H in the WGS84 geographic coordinate system. Longitude L, latitude B, and altitude H are used to represent the actual geographical location of the target point on Earth.

9. A spatial measurement method based on real-time video from an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, In step 4, the number of target points in the identified area is at least three, and when calculating the area, all target points are assumed to be at the same elevation level.

10. A spatial measurement method based on real-time video from an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, Step 5 specifically includes: Step 51: According to the order of the target points in the identification area, concatenate the longitude and latitude values ​​of each target point in sequence to form a coordinate sequence text with longitude and latitude separated by spaces. Place this coordinate sequence text in a geometric description prefix and parentheses indicating the polygon type, and make the first and last coordinates of the coordinate sequence text the same, thus forming a WKT string. Step 52: Use the function of parsing the WKT string in the geometry calculation library to load the WKT string as a geometric object; Step 53: Call the area calculation method of the geometric object to obtain the area value of the recognition area it represents.

Images